Support for grafting trees

[psss]

Several times users mentioned that it would be useful to reference a remote tree and merge (or graft) the whole tree or part of it into specific node (branch) of the local tree. Here are two examples of possible usage:

• Referencing a single test (L1 metadata) from a different repository to be executed under a single plan (L2 metadata)
• Merging a remote plan (L2 metadata) into a local tree with plans in order to prevent duplication of the config

This is partially related to #51 and some parts of the remote referencing have been already discussed here.

[psss]

@ep69, let's discuss the concept of merging remote metadata here.

[ep69]

My current usecase:

I'd like to reference a test script (runtest.sh) from different git repository while specifying my own L1 metadata, in my specific example mostly interested in setting some environment variables that control the flow of the test.

[jscotka]

I've also implemented it in https://github.com/psss/fmf/pull/54 some way of referencing of nodes

[pcahyna]

Do you want to graft the whole filesystem trees, or only metadata trees? The problem is, fmf metadata are stored in directory trees that contain more than just fmf metadata trees, i.e. more than .fmf files. For example, test scripts (runtest.sh) and test program inputs. Grafting only metadata without the rest of the files is probably not useful, because metadata will most likely refer to other files.

I also believe it will be easier to start with grafting whole trees instead of part of trees, because then you don't need to worry about merging (multiple inheritance). Think of it like of filesystem mounts: you also mount whole filesystems (and not subdirectories) onto mount points.

[LecrisUT]

How about defining the grafting method in the .fmf folder? Most naturally it could be in .fmf/version, but the structure of that file would not allow for such extensions. So maybe in a .fmf/config or .fmf/include:

include:
- url: git+https://github.com/user/repo
  path: /path/to/remote/fmf_root
  include_files: false

Where this file woul be in /path/to/new/tree/root/.fmf/config. Then all the contents of the remote tree would be grafted in-place from the reading of the tree, from the copying of the fmf tree and from a cli command fmf sync that would copy all of the tree in-place.

[LecrisUT]

Recent discussion from #223. Another usecase that would be useful even on the python level is to support 2 operations:

• graft: have a pre-defined fmf.Tree from filesystem then add on top of that another fmf.Tree (from dict)
  • use-case: include/import mechanism to merge in the content of a remote tree, like a github action include
• inverse_graft: construct an fmf.Tree-like structure (from dict), then apply the fmf.Tree from filesystem
  • use-case: dynamically generated tests (from discover) and override some values for specific tests afterwards