[Feature] Make preprocessed input files relocatable

[klevzoff]

What is the requested feature? Would be nice if preprocessed files could be moved around on the filesystem without breaking.

Is your request related to a specific problem? When using preprocessor workflow with a model that has additional assets (e.g. mesh or table files) that are external to XML, the resulting XML must be placed in the same directory as the source in order for relative paths to work correctly. Would be great if that were not the case, i.e. when scripting a parameter study I'd like each preprocessed file to be placed in an output directory alongside job outputs, etc. and not clutter the model input directory.

Describe the solution you'd like I was thinking of introducing additional preprocessor syntax for paths that need to be resolved, e.g. using @ or some other symbol (exact details to be figured out):

<VTKMesh
  name="mesh"
  file="@data/mesh.vtu"/>

which the preprocessor turns into

<VTKMesh
  name="mesh"
  file="/home/klevtsov/models/AwesomeModel/data/mesh.vtu"/>

Describe alternatives you've considered Copying the extra assets to the output directory together with preprocessed XML is not feasible: some mesh files are gigabytes in size. For the moment I'm just keeping preprocessed inputs in the original directory and cleaning them up once in a while, this is not too much trouble. However it takes a bit of extra bookkeeping to match output files to the exact inputs they were obtained from.

Additional context I can implement this myself if we agree on direction. Seems like it would be easy enough to fit into existing regex-based infrastructure.

[cssherman]

@klevzoff - my strategy for this has been to use parameters holding absolute paths to handle swapping out meshes, tables, etc. For example:

<Problem>
  <Parameters>
    <Parameter name="mesh_root"
               value="/path/to/mesh/root"/>
    <Parameter name="mesh_name"
               value="mesh_a.vtk"/>           

    <Parameter name="table_root"
               value="/path/to/table/root"/>
    <Parameter name="table_name"
               value="table_a"/>
  </Parameters>
</Problem>

<!-- In some included file -->
<Problem>
  <Mesh>
  <VTKMesh
    name="external_mesh"
    file="$mesh_root$/$mesh_name$"/>
  </Mesh>
</Problem>

Also, if we are expecting to relocate xml files, I feel like it is safer to use absolute paths regardless.

[cssherman]

Although, do you mean you would want to do something like this (not manually move the files):

preprocess_xml some_file_with_relative_parameters.xml -c /some/other/path/file_with_fixed_relative_paths.xml

That would be pretty simple to implement

[klevzoff]

Although, do you mean you would want to do something like this (not manually move the files):

preprocess_xml some_file_with_relative_parameters.xml -c /some/other/path/file_with_fixed_relative_paths.xml

That would be pretty simple to implement

Yes, this is more or less what I want, although it would be nice if whatever solution we came up with also worked without the -c flag, i.e. letting preprocessor spit out an autogenerated name, then moving the file elsewhere.

But whether we chose to "fix" relative paths or replace them with absolute during preprocessing, we'd have to have some syntax to mark attributes that need path adjustment, since they are impossible to distinguish from regular strings otherwise (schema has that info via Path type for attributes, but I imagine it would be a pretty significant effort to use it for that).

Your solution with "root path" parameters seems like a pretty good workaround though. I can set them to . by default to keep paths relative, but substitute an absolute path to the model on preprocessor command line whenever I want to generate a relocatable XML. I'll give it a try, and maybe we can just close the issue.

[cssherman]

OK, then do we want allow users to mark paths to convert one-by-one, or should it be all-or-nothing? In the first case, I'd be happy with @ to mark the values. For the second case, we could indicate this preference via the command line. In this case, we would look for path attributes (likely requiring the schema) in the file and convert the ones that don't look like absolute paths.