AcademySoftwareFoundation
Standard Library Templating and Named Value implementation.
This PR proposes a concrete implementation inspired by proposals presented in #2148 and #2149. We will recap these briefly.
Problem Statement
The standard data library is repetitive due to the type strictness of MaterialX. A lot of node definitions need to be concretely instantiated for each type they support. For instance <add> ends up with 16 different node definitions, spanning about 80 lines of XML. Due to the repetitive nature of these lines of XML, it can be hard to identify typos, and encourages copy/pasting which can be error prone.
Proposed Solution
Instead of authoring each node definition explicitly, this PR introduces the concept of a <template> for specifying repetitive elements. We also introduce a new syntax to abstract the specification of common values that have a different expression for different types, but are semantically the same, ie. zero and one.
<template name="TP_ND_add" varName="typeName" options="float, integer, color3, color4, vector2, vector3, vector4">
<nodedef name="ND_add_@typeName@" node="add" nodegroup="math">
<input name="in1" type="@typeName@" value="Value:zero" />
<input name="in2" type="@typeName@" value="Value:zero" />
<output name="out" type="@typeName@" defaultinput="in1" />
</nodedef>
</template>
Here we wrap a variation of the node definition for <add> inside a container <template> element. The <template> element has the following three attributes:
-
name- unique identifier -
varName- the name of a variable to be identified and replace within the contents of the template -
options- a list of strings that will used to instantiate copies of the template.
The length of the options list determines how many copies of the template are instantiated. Each copy will substitute a value from the options list for each attribute that contains the value of the varName attribute wrapped in @ characters.
In the example above @typeName@ is replaced with each of the elements in the options list.
<nodedef name="ND_add_float" node="add" nodegroup="math">
<input name="in1" type="float" value="Value:zero" />
<input name="in2" type="float" value="Value:zero" />
<output name="out" type="float" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_integer" node="add" nodegroup="math">
<input name="in1" type="integer" value="Value:zero" />
<input name="in2" type="integer" value="Value:zero" />
<output name="out" type="integer" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_color3" node="add" nodegroup="math">
<input name="in1" type="color3" value="Value:zero" />
<input name="in2" type="color3" value="Value:zero" />
<output name="out" type="color3" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_color4" node="add" nodegroup="math">
<input name="in1" type="color4" value="Value:zero" />
<input name="in2" type="color4" value="Value:zero" />
<output name="out" type="color4" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_vector2" node="add" nodegroup="math">
<input name="in1" type="vector2" value="Value:zero" />
<input name="in2" type="vector2" value="Value:zero" />
<output name="out" type="vector2" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_vector3" node="add" nodegroup="math">
<input name="in1" type="vector3" value="Value:zero" />
<input name="in2" type="vector3" value="Value:zero" />
<output name="out" type="vector3" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_vector4" node="add" nodegroup="math">
<input name="in1" type="vector4" value="Value:zero" />
<input name="in2" type="vector4" value="Value:zero" />
<output name="out" type="vector4" defaultinput="in1" />
</nodedef>
The second expansion that happens is the resolution of the named values. In the expanded template above we still have inputs that have value attributes that are not currently legal values, we see Value:zero and Value:one.
The expansion of these named values relies upon additional attributes specified in the type definitions.
<typedef name="integer" zero="0" one="1"/>
<typedef name="float" zero="0.0" one="1.0"/>
<typedef name="color3" semantic="color" zero="0.0,0.0,0.0" one="1.0,1.0,1.0"/>
<typedef name="color4" semantic="color" zero="0.0,0.0,0.0,0.0" one="1.0,1.0,1.0,1.0"/>
<typedef name="vector2" zero="0.0,0.0" one="1.0,1.0"/>
<typedef name="vector3" zero="0.0,0.0,0.0" one="1.0,1.0,1.0"/>
<typedef name="vector4" zero="0.0,0.0,0.0,0.0" one="1.0,1.0,1.0,1.0"/>
Again the named value expansion is just another text replacement, we located any value attributes that starts with the string Value:, we then interrogate the type of that value, and look up the concrete value value in the corresponding type definition.
Build vs Runtime evaluation
In order to maintain compatibility with the existing specification, the build has been updated to perform these template expansions and string replacements during the build. This ensures that the generated data library remains compatible with the current specification. These build time expansions are performed by C++ based tools built during the project, and using the MaterialX library itself.
The MaterialX C++ library has also been updated to provide implementation to evaluate these new concepts at runtime. Concretely, the template expansion can be performed during file read if the read option XmlReadOptions.expandTemplateElems is enabled. If this is used during the runtime, the list of node definitions returned by Document::GetNodeDefs() would remain unchanged, and the complete list would be returned.
The runtime evaluation of the named value replacement takes place in ValueElement::GetValueString(), where we optionally apply the same string replacement logic, based on the type and the type definition.
CMake changes
CMake arguments have been added to control these build vs runtime choices.
-
MATERIALX_BUILD_EXPAND_TEMPLATE_ELEMS- if enabled, then the templates will be expanded during the build, and the standard data library generated by the build will contain the concrete instances of the template. -
MATERIALX_BUILD_BAKE_NAMED_VALUES- if enabled, the named values will be baked out to their concrete type based values. Note, if this option is enabled, thenMATERIALX_BUILD_EXPAND_TEMPLATE_ELEMSis forced to be true, as any named values inside a template may not be able to resolve to a specific type.
In order to support the cross compilation used to generate MaterialX library for uses on other platforms, such as iOS, we have to ensure that the tools used to generate the data library are built for the build machines architecture. The easiest way to support this cross compilation approach in CMake is to create a small subproject that allows the tools to be built with a separate toolchain. (source references here and here)
I like this node concept and the agree with the motivation. It does help reduce the duplication. I'm curious if we could just extend nodedef to support multiple types so that will also help with publishing/authoring workflows? In the current proposal, a template node is expanded based on types but authoring tools don't have any knowledge about this. I understand that is intentional to maintain compatibility but won't it still lead to authoring issues with repetitive edits?
Although I'm a huge fan of templated node definitions, and this feature has been a goal for the MaterialX project since before it was open-sourced, I think it's worthwhile to consider leading with a run-time implementation rather than a build-time implementation, as suggested in @ld-kerley's original GitHub Issue:
There's a lot to consider here, and just to round out the discussion, I'll note that the Runtime Polymorphic approach has the potential to dramatically reduce the size of our runtime data libraries, potentially improving runtime performance for all MaterialX and USD applications.
If we pursued the Runtime Polymorphic approach, then this would become a proposed feature for MaterialX v1.40, and we would use our version system to upgrade legacy documents to the latest syntax.
In our MaterialX TSC discussion today, we had the chance to refine this idea a bit further, as expressed in the following proposed steps:
- Write up a proposal for a
templateelement, supporting run-time templating of bothnodedefandnodegraphelements. - Work through the details of this proposal with the community.
- Implement the new
templateelement in MaterialXCore, with unit tests to validate all of its use cases. - Refine the new implementation with the community.
- When we're ready for MaterialX v1.40 or v2.0, switch our standard data libraries over to leverage the new
templateelement.
I think it's still worthwhile to consider the pros and cons of both the run-time and build-time solutions, and I'm very interested in additional thoughts from other teams.
I would like to highlight some of the other aspects of the discussion at today's TSC, as they relate to this PR.
- Numerous stakeholders and TSC participants expressed interest in pursuing this build time approach.
- Build time template expansion has a number of benefits.
- Low risk.
- Zero adoption cost downstream. No changes to the data library delivered downstream to MaterialX consumers.
- A practical working example will make it a lot easier to explore some of the real world complexities of this new language feature, without committing it to the spec.
- Incremental changes that are much easier to roll back if necessary, as the change is completely self contained inside MaterialX.
- Does not precludes future work towards a complete runtime polymorphic system.
- Makes building future runtime polymorphic system less work.
For those interested, a reminder of the robust discussions on the original proposals can be found here and here.
This is still in-progress thinking on my part, but I wanted to share some of the concrete cases that concern me about a build-time template system:
- Because the build-time template system would be outside of the MaterialX specification, with no guarantees of data versioning, developers could easily create custom nodes that would be stranded as the build-time template system evolves. The run-time template system, on the other hand, would be fully integrated with MaterialX data versioning, with all of the usual guarantees that legacy custom nodes remain valid indefinitely.
- The build-time template system could create a disconnect between the specified rules of MaterialX and the actual syntax that developers need to author when developing new custom nodes or implementing fixes to standard nodes. This seems like a cognitive burden that would potentially be significant, especially for new MaterialX developers that we're encouraging to work with MaterialX for the first time or make substantial contributions to our project. The run-time template system, on the other hand, would always maintain our traditional integration between the specified rules of MaterialX and the syntax that we expect developers to work with.
If we ended up stuck with a build-time template system for a long period of time, e.g. for months or years, I can imagine we might end up regretting our choice to follow this approach, where the costs of the build-time template system would greatly outweigh its advantages.
None of this is to say that a build-time template system is off the table, but I wanted to share some of my own thinking about the cases that concern me about this approach.
The scope of this PR is intended to be constrained solely to the standard data library files that live inside the MaterialX project.
I'm not sure I 100% understand what you mean by "custom nodes", but if that means nodes specified outside the MaterialX project codebase, i.e. nodes not in the current standard data library, then I would provide a point of clarification. The build time template expansion tools would not be installed as part of the MaterialX installation package, and thus not be available to custom node authors at all. Those authors would and should be constrained to the syntax as defined in the specification, as they always have. This allows us the safety of iterating on the data library template definition, without impacting anyone else.
I do concede that we are considering introducing syntax to the libraries folder that is not 'yet' defined in the specification, but I do not believe this introduces a cognitive burden at all. I believe its easier to read, understand and check for bugs in this
<template name="TP_ND_add" varName="typeName" options="float, integer, color3, color4, vector2, vector3, vector4">
<nodedef name="ND_add_@typeName@" node="add" nodegroup="math">
<input name="in1" type="@typeName@" value="Value:zero" />
<input name="in2" type="@typeName@" value="Value:zero" />
<output name="out" type="@typeName@" defaultinput="in1" />
</nodedef>
</template>
instead of this
<nodedef name="ND_add_float" node="add" nodegroup="math">
<input name="in1" type="float" value="0.0" />
<input name="in2" type="float" value="0.0" />
<output name="out" type="float" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_integer" node="add" nodegroup="math">
<input name="in1" type="integer" value="0" />
<input name="in2" type="integer" value="0" />
<output name="out" type="integer" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_color3" node="add" nodegroup="math">
<input name="in1" type="color3" value="0.0, 0.0, 0.0" />
<input name="in2" type="color3" value="0.0, 0.0, 0.0" />
<output name="out" type="color3" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_color4" node="add" nodegroup="math">
<input name="in1" type="color4" value="0.0, 0.0, 0.0, 0.0" />
<input name="in2" type="color4" value="0.0, 0.0, 0.0, 0.0" />
<output name="out" type="color4" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_vector2" node="add" nodegroup="math">
<input name="in1" type="vector2" value="0.0, 0.0" />
<input name="in2" type="vector2" value="0.0, 0.0" />
<output name="out" type="vector2" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_vector3" node="add" nodegroup="math">
<input name="in1" type="vector3" value="0.0, 0.0, 0.0" />
<input name="in2" type="vector3" value="0.0, 0.0, 0.0" />
<output name="out" type="vector3" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_vector4" node="add" nodegroup="math">
<input name="in1" type="vector4" value="0.0, 0.0, 0.0, 0.0" />
<input name="in2" type="vector4" value="0.0, 0.0, 0.0, 0.0" />
<output name="out" type="vector4" defaultinput="in2" />
</nodedef>
<nodedef name="ND_add_matrix33" node="add" nodegroup="math">
<input name="in1" type="matrix33" value="1.0,0.0,0.0, 0.0,1.0,0.0, 0.0,0.0,1.0" />
<input name="in2" type="matrix33" value="0.0,0.0,0.0, 0.0,0.0,0.0, 0.0,0.0,0.0" />
<output name="out" type="matrix33" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_matrix44" node="add" nodegroup="math">
<input name="in1" type="matrix44" value="1.0,0.0,0.0,0.0, 0.0,1.0,0.0,0.0, 0.0,0.0,1.0,0.0, 0.0,0.0,0.0,1.0" />
<input name="in2" type="matrix44" value="0.0,0.0,0.0,0.0, 0.0,0.0,0.0,0.0, 0.0,0.0,0.0,0.0, 0.0,0.0,0.0,0.0" />
<output name="out" type="matrix44" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_color3FA" node="add" nodegroup="math">
<input name="in1" type="color3" value="0.0, 0.0, 0.0" />
<input name="in2" type="float" value="0.0" />
<output name="out" type="color3" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_color4FA" node="add" nodegroup="math">
<input name="in1" type="color4" value="0.0, 0.0, 0.0, 0.0" />
<input name="in2" type="float" value="0.0" />
<output name="out" type="color4" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_vector2FA" node="add" nodegroup="math">
<input name="in1" type="vector2" value="0.0, 0.0" />
<input name="in2" type="float" value="0.0" />
<output name="out" type="vector2" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_vector3FA" node="add" nodegroup="math">
<input name="in1" type="vector3" value="0.0, 0.0, 0.0" />
<input name="in2" type="float" value="0.0" />
<output name="out" type="vector3" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_vector4FA" node="add" nodegroup="math">
<input name="in1" type="vector4" value="0.0, 0.0, 0.0, 0.0" />
<input name="in2" type="float" value="0.0" />
<output name="out" type="vector4" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_matrix33FA" node="add" nodegroup="math">
<input name="in1" type="matrix33" value="1.0,0.0,0.0, 0.0,1.0,0.0, 0.0,0.0,1.0" />
<input name="in2" type="float" value="0.0" />
<output name="out" type="matrix33" defaultinput="in1" />
</nodedef>
<nodedef name="ND_add_matrix44FA" node="add" nodegroup="math">
<input name="in1" type="matrix44" value="1.0,0.0,0.0,0.0, 0.0,1.0,0.0,0.0, 0.0,0.0,1.0,0.0, 0.0,0.0,0.0,1.0" />
<input name="in2" type="float" value="0.0" />
<output name="out" type="matrix44" defaultinput="in1" />
</nodedef>
Finally I'll also note, because this is completely self contained inside the MaterialX project - if we all agree it's been a terrible idea, and we want to revert back to a non-template world. It will be trivial to bake out the templates and commit them to the project. Leaving us exactly where we are today.
Thanks for those additional details, @ld-kerley, and although that provides a clearer picture, it doesn't address my concerns with releasing a build-time implementation before we've agreed upon a specification for its expected syntax and behavior.
In some ways, this sounds even more challenging for developers working with MaterialX, as standard nodes would be written in an entirely different syntax from custom nodes, with our specification only applying to the custom nodes.
Taking a step back, I wonder if we can find common ground between these two proposed paths forward, the run-time and build-time implementations of templating.
What if we were to start with our specification process for defining the expected syntax of template elements, writing up a new section in the Proposals document and vetting it with the community? Once this process was complete, we would be in a much better position to decide on the best step to follow it, where either a build-time or run-time implementation would have a community-vetted specification proposal to guide it.
I believe we would also learn a great deal about the best syntax for this feature during the specification vetting process, only changing the data libraries that our community relies upon when this important design process was complete.
It seems to me that the build-time solution is a stepping stone towards the run-time solution. Ie, the work on real-time will require code that bakes out the concrete nodedefs from the templates. It's just that the build-time solution exposes it sooner.
Personally I prefer incremental and evolutionary approach to development, so I would lean towards the build-time approach, because it allows assessment of the direction earlier in the process. As long as Jonathan's reservations are addressed.
So, for example, regarding custom nodes, I would imagine developers would still keep using the formal XML syntax. I see the build-time step as an unofficial prototype to do some ground work for run-time, for testing, and to get early benefits of the templates.
I understand Jonathan's concern about unsanctioned syntax, though. I'm not sure if there is a good way of isolating it (and template development) from the official .mtlx sources. Maybe one way is to use templates for some small subset of test (ie, non-standard) nodedefs only, until the proposed specification stabilizes?
It's also a valid concern about using the build-time solution for too long before run-time solution arrives. But than again, run-time solution will not be stable unless we test it, and therefore even the run-time-only solution may experience evolution and yield undesirable, stranded documents.
I also wonder how much is involved in going from build-time to run-time support.
I read the PR as "make it easier to author standard library nodes without changing how you use MaterialX"? If that assumption is correct; I think it makes a lot of sense.
I'm not sure if I fully understand the reasoning/goals behind the runtime proposal. I think we need to be very clear with our goals when we say that it will become more efficient. For example, is the goal to have an effective solution for desktop use, or do we want to simplify the use of MaterialX in a web environment? Are we targeting the complete MaterialX API or do we want to create minimal evaluation engine without authoring parts?
Maybe the solution for all of the above is (partly) a generic type system, but I think we need to clear on what we are trying to achieve?
Some first remarks: especially with the originally proposed syntax it feels more like function overloading than function templates.
For overloading it has a very restrictive set of allowed overloads (can be good and bad)
- all overloads have to have the same number of parameters
- inputs/outputs are either fixed type or unfixed (typed based on a defined list of types).
- all unfixed types are change synchronously based on that list or???? (whats the rule that allows ND_add_color3FA to be generated from ND_add_@typeName@ + types? what is altogether legal in this set of overloads?)
con: having both a list of overloads and a list of types, like in the original proposal, seems both superfluous and error prone.
pro: starting with a small set of allowed overloads and simple rules for resolving the overloads makes it easier to expand later
I don't think putting this into "build" is not at all more flexible than putting it into "runtime" when it comes to changing/reverting the feature. This needs to be reasonably sound in any case.
My primary concern would be to keep USD and MaterialX aligned and working together smoothly, which I think is doable with either option.
My understanding is that the build-time version wouldn't change the data library and a run-time option would be far enough in the future that we could start discussions early on how to keep USD in alignment.
Thanks to everyone for chiming in here. Having a wider set of community input for important projects within MaterialX is important, and really helps bring balance to the conversation. It's important that the project considers input from as many of the stakeholders as possible.
Firstly I'd like to highlight the great work added here by @anderslanglands. He's taken the time to template-ize a large section of the standard library here. This will not only hugely help with the ongoing documentation effort, but also helped highlight a few potential errors in the existing standard library. Anders nicely documents all the "diffs" in the PR he posted to my fork here.
Notably the atan2 input default values are inconsistent across different types. I believe this to be a bug in the standard data library, one that has likely been there for a while now. This example highlights the value of templates in making sure we don't have drift between implementations, and how adopting a template system will help make discrepancies a thing of the past. The templated stdlib_defs.mtx file is now less than half the size of the untemplated version, shrinking to under 2500 lines from just over 5000. I expect other files to reduce even further, especially those with many repeated nodegraph implementations.
This nicely feathers in to responding to @mapetikea. Yes, your understanding is correct, one the major benefits to this approach is exactly that we "make it easier to author standard library nodes without changing how you use MaterialX". I also think you raise very valid points regarding making sure we scope any possible runtime implementation of this system, particularly as we have different downstream implementors with very different use-cases.
As @jesschimein points out, this build-time approach would ensure the data library would remain compatible with USD today, and allow a much longer period of time for development and adoption of any possible run-time implementation.
@Jan-Jordan - I certainly agree with your observations, and somewhat intentionally opted to start with the simplest for of this templating system I could imagine, while still being useful. To my mind of on the benefits of this build-time approach is that it allows us to iterate internally to the project source, without commit to a system. I think its very possible the final system we end up with will be richer in terms of its expressiveness and capabilities, but as @rafalSFX points out that incremental journey is probably an easier one for the community to walk along together.
@rafalSFX - I'd like to try and provide an alternative view to your comment about unsanctioned syntax. This "work-in-progress" template syntax would only exist in the MaterialX source repository, and never in any build/deployment of the project. So in some senses I would consider that "private", until we are shipping mtlx files with the syntax in (in a potential runtime solution in the future). If there is a concern regarding people using the GitHub repo as a reference repository, which is certainly valid, then I'd point out first that no tooling will create files with this syntax, and if anyone does create it by hand, we could ensure that it fails MaterialX validation, and it wouldn't function in any of the existing MaterialX tooling correctly either. We could also bake out a version of the expanded data library to some location in the repository for reference, if that was deemed a helpful source of information for people wanting to understand the formal MaterialX syntax, without building it themselves.
Finally, @jstone-lucasfilm, I respectfully disagree that the syntax is "an entirely different syntax". I feel that anyone who is knowledgeable enough to be reading the MaterialX data library files directly, is likely to be able to infer the intention of the <template> element, and the significant reduction in the size of the data library, coupled with it being less error prone, seems like it would actually make the task easier. I'd also point out that the idea has been through a proposal process of sorts, in the form of the previously posted GitHub issues.
@ld-kerley I'd still really like to see this idea go through our specification proposal process before committing any code or changes to our data libraries, but I'm certainly open to further discussion on this!
One thing that caught my eye is how the uniform="true" attribute of an input element forces a separate template. It would be nice to have just one template, but I suspect it's gonna be hard to fold the uniform attribute into the template. Especially in the first version of the system. Anyway, I just wanted to point it out.
<template name="TP_ND_constant" varName="typeName" options="float, color3, color4, vector2, vector3, vector4, boolean, integer">
<nodedef name="ND_constant_@typeName@" node="constant" nodegroup="procedural">
<input name="value" type="@typeName@" value="Value:zero" />
<output name="out" type="@typeName@" default="Value:zero" />
</nodedef>
</template>
<template name="TP_ND_constant_uniform" varName="typeName" options="string, filename">
<nodedef name="ND_constant_@typeName@" node="constant" nodegroup="procedural">
<input name="value" type="@typeName@" value="Value:zero" uniform="true" />
<output name="out" type="@typeName@" default="Value:zero" />
</nodedef>
</template>
@ld-kerley This is more of a question for a future specification proposal, but would it be possible to restrict the names of templated elements to the allowable character set for MaterialX element names, so that they won't be marked as invalid when these elements appear in runtime documents?
Some design discussion is needed here, and I'd want @dbsmythe to provide his review of any syntax proposal for our specification, but one possibility would simply be to omit the bracketing @ characters in your templated names, applying the rule that any appearance of this name will be replaced by the appropriate type.
@rafalSFX - one of the really nice things about going through this process (well Anders doing it) is that we found a number of other small inconsistencies in the data library. Some that I believe are bugs (see the atan2 diff in Anders PR), and others that are more of a grey area (does <add> for a matrix really need its default values to be identity?), but would be good to re-examine. I think this is a really strong argument for having templates at least at the build step - as it makes these things so much easier to see.
@jstone-lucasfilm - I'm very open to changes here - my choice of @ was somewhat arbitrary. I just wanted to select a character that I didn't think would clash with anything else. When doing string substitution like this, it's important to ensure there is no accidental substitution, I've been bitten by that one too many times before. So if we wanted to select another character thats fine, but I think we should say that whatever character (or series of characters) we select is not valid in other attribute values, to try and avoid incorrect substitutions.
Closing this PR in favor of #2451 which includes some improvements based on feedback received, and also includes the document building tools - which have been really helpful to experiment with the template system.