ynput
Enhancement: Workfile template builder using publishes instead of direct filepath
Is there an existing issue for this?
- [X] I have searched the existing issues.
Please describe the feature you have in mind and explain what the current shortcomings are?
The workfile template builder currently uses just some scene files saved in a specific folder for a particular host for which the path is set in the Studio Settings. I would like to propose to refactor this functionality into using published workfile templates instead.
Pros
- Workfile templates are versioned and tracked
- Workfile templates could then sync automatically via site-sync
- Workfile templates could also easily be "loaded manually" to test a specific workfile template.
- Additional studio-specific validators could be deployed upon the generation of workfile templates.
- The 'loaded' workfile template could containerise into the workfile as a 'loaded container' and thus a publish from the workfile can then also report which template was used to build that workfile due to "input links"
Cons
- It'd require a publish instead of just a "drop a file there" which might be trickier to use a single template across multiple projects (unless maybe it'd be possible to load from another project like library loader too)
How would you imagine the implementation of the feature?
In settings instead of defining a path to the file to load you'd instead pass it the asset > subset to load for particular profile.
Note that the API for the workfile builder should still have an interface through which you could import a particular template builder.import_template and also build the current scene builder.populate_scene_placeholders. It's only the automatic "build workfile for current task" feature which would then find the relevant publish and load it from the filepath.
I'd even argue that the current logic is already failing there since you can't just build the current scene in maya using builder.populate_scene_placeholders but require the import_path to have been triggered so a PLACEHOLDER_SET exists. The only workaround for that currently is:
from openpype.hosts.maya.api.workfile_template_builder import MayaTemplateBuilder, PLACEHOLDER_SET
from openpype.pipeline import registered_host
host = registered_host()
builder = MayaTemplateBuilder(host)
if not cmds.objExists(PLACEHOLDER_SET):
cmds.sets(name=PLACEHOLDER_SET, empty=True)
builder.populate_scene_placeholders(keep_placeholders=False)
Are there any labels you wish to add?
- [X] I have added the relevant labels to the enhancement request.
Describe alternatives you've considered:
An alternative would be to, e.g. for maya, publish a maya scene and have the published path set in the studio settings. However it'd disable some of the cons of not loading an actual representation directly (e.g. site sync, etc).
Additional context:
No response
[cuID:OP-6162]
@fabiaserra @tokejepsen @ClementHector @mre7a @friquette - it seems each of you have been involved with the Workfile Builder templates and might have touched, tested or are even using it in production so I was hoping to get your valuable feedback on this issue from your perspectives.
I would like to propose to refactor this functionality into using published workfile templates instead.
It’s a very good idea, it was one of our goals. Today we have a library project that contains all our templates that we publish.
The 'loaded' workfile template could containerise into the workfile as a 'loaded container' and thus a publish from the workfile can then also report which template was used to build that workfile due to "input links"
Can you expand that?
It'd require a publish instead of just a "drop a file there" which might be trickier to use a single template across multiple projects (unless maybe it'd be possible to load from another project like library loader too)
Since we use the templates of a library project, we should be able to do so. Otherwise it means that we should publish them with each project. I see another constraint, the ability to stay on a specific version of publish rather than always the latest version.
I speak for @mre7a and @friquette, because we have the same objective, and we’re working together on that part.
Can you expand that?
Like any loaded publish it would then just show up as a loaded container in the manager. The publishing system tends to link the containers returned by host.ls() or host.get_containers() with the resulting workfile publish which could help track through links in the database what template version was used.
I see another constraint, the ability to stay on a specific version of publish rather than always the latest version.
Yes - an artist would not want to be warned (or maybe they would want to know?) about a template being outdated if the container would show it as not being the latest version. Good point! Maybe having it persist as data in the scene like that isn't the best solution then.
Also mentioned using published data here; https://community.ynput.io/t/maya-templates-improvements/174/9, so I'm completely on board.
Yes - an artist would not want to be warned (or maybe they would want to know?) about a template being outdated if the container would show it as not being the latest version. Good point! Maybe having it persist as data in the scene like that isn't the best solution then.
I would argue that we should at least flag it to the artist, so they can make the best decision for the asset/shot. They might have done nothing to the scene since building it so building with a new version would not be a bother. But I don't think the manager is the right place for flagging this since the artist will most likely follow pipeline workflow and just try to update the template, in which case what happens? Might be safer with a pre-flight validation similar to how the FPS in Maya warns the user when launching.
I would argue that we should at least flag it to the artist, so they can make the best decision for the asset/shot. They might have done nothing to the scene since building it so building with a new version would not be a bother. But I don't think the manager is the right place for flagging this since the artist will most likely follow pipeline workflow and just try to update the template, in which case what happens? Might be safer with a pre-flight validation similar to how the FPS in Maya warns the user when launching.
+1, that's how we had it implemented on my old studio, we didn't implement the logic to update the whole workfile template version as there was too many variables to take into account but if they had created the template using other "smaller" published assets we did allow for them to update those through the scene manager (i.e. the template used a Maya renderSetup that's a versioned asset, if someone updated that, they would be able to pull the update)
I suspect updating loaded content is the only reasonable safe feature to implement too - I just don't see how "rebuilding the workfile" including all local changes but with updated content will ever be a viable route in most DCCs. There's just to many cases where local changes will be lost and you're still basically starting new.
Having loaded content in the scene without being in the manager has already come up before with the look family, where you updating through the manager does not make sense but rather needs a completely new assignment, so similar concepts to rebuilding the workfile from a template.
Having loaded content in the scene without being in the manager has already come up before with the
lookfamily, where you updating through the manager does not make sense but rather needs a completely new assignment, so similar concepts to rebuilding the workfile from a template.
unrelated to this discussion but I would expect you should be able to update a look family through the manager, doesn't the update version logic provide ways to keep track of where that look has been assigned so you can re-import the look and re-do the same assignments before the update?
of where that look has been assigned so you can re-import the look and re-do the same assignments before the update?
It does not re-apply, no; currently in maya the design only updates the material. The user is free to however still re-apply via the Maya look manager itself - which is a separate tool.
Loading Workfile Templates via Loader
This is now implemented in ynput/OpenPype#5830
Just started playing with this idea of Workfile Templates just being publishes themselves again and created this very simple Loader for Maya:
from openpype.lib import (
BoolDef
)
from openpype.pipeline import (
load,
registered_host
)
from openpype.hosts.maya.api.workfile_template_builder import (
MayaTemplateBuilder
)
class LoadAsTemplate(load.LoaderPlugin):
"""Load workfile as a template """
families = ["workfile"]
label = "Load as template"
representations = ["ma", "mb"]
icon = "wrench"
color = "#775555"
order = 10
options = [
BoolDef("keep_placeholders",
label="Keep Placeholders",
default=False),
BoolDef("create_first_version",
label="Create First Version",
default=False),
]
def load(self, context, name, namespace, data):
keep_placeholders = data.get("keep_placeholders", False)
create_first_version = data.get("create_first_version", False)
path = self.filepath_from_context(context)
builder = MayaTemplateBuilder(registered_host())
builder.build_template(template_path=path,
keep_placeholders=keep_placeholders,
create_first_version=create_first_version)
I had to tweak the AbstractTemplateBuilder.build_template method so that if all three of those arguments are passed that it doesn't try to still check the setting' template preset for current context, adding:
if any(value is None for value in [template_path,
keep_placeholders,
create_first_version]):
template_preset = self.get_template_preset()
if template_path is None:
template_path = template_preset["path"]
if keep_placeholders is None:
keep_placeholders = template_preset["keep_placeholder"]
if create_first_version is None:
create_first_version = template_preset["create_first_version"]
What does that offer?
With that I can now:
- Create a Template in say a
templates/animationasset with e.g. taskmainto getworkfileMainpublishes there. - I can then browse the loader templates, right click the published template and load it.
Loading additional mini-templates for artists to add
Another benefit is that I can actually also make extra mini-templates an artist can load. Say I have a simple Smoke lighting kit that itself references a few separate things I can have an artist also just load that template and get it added into their current scene.
That is, if we remove the PLACEHOLDERS_SET check which is a bit odd to keep if keep_placeholders=False anyway?
To-do:
Allow the auto-load of current context template
The next step is for the Settings to allow the path for a profile to be set, e.g. via AYON URI:
ayon+entity://{project}/templates/animation?product=workfileMain&version=latest&representation=ma
So that the workfile builder would then just happen to resolve the URI and load the file via that. Unless someone knows a better settings format for that?
Persist with the scene what templates (and what versions) were loaded+built
Persist with the scene what templates it was built against so that e.g. the warning for artists that they are using an older template version could be displayed. Whether that's something for the Scene Inventory or not is up for debate as @tokejepsen pointed out rightly so, likely it's not the best place.
@fabiaserra @tokejepsen @mkolar Thoughts?
Hi @BigRoy In your latest comment, you stated this : This is now implemented in https://github.com/ynput/OpenPype/pull/5380
But I guess you really intended to write this : https://github.com/ynput/OpenPype/pull/5830 5380 vs 5830 😉
Really like the idea and implementation @BigRoy ! We were also thinking on requiring something along these lines and were thinking on using a Library project to host global scene templates just as a means to query it through OP API and then use them in some utility modules we have written (as opposed to hard-code the path on disk)... but adding support for it on the Loader and the rest of the workfile template workflow would be great
It would be a nice feature! I've just tried to figure out how to set up something like that for Nuke and got sad that Ayon doesn't have a such obvious and must-have option 😢
It would be a nice feature! I've just tried to figure out how to set up something like that for Nuke and got sad that Ayon doesn't have a such obvious and must-have option 😢
@leshavkleshah I wonder if https://github.com/ynput/ayon-core/pull/1292 may help you out.