compilers icon indicating copy to clipboard operation
compilers copied to clipboard

Add `artifacts_symlink` to `ProjectPathsConfig`

Open klkvr opened this issue 1 year ago • 8 comments

Per earlier discussion we've decided that it makes sense to separate cache and artifacts between profiles by default. However, start silently saving artifacts to out/{profile} might be a breaking change as /out would still keep old artifacts and might result in external tools silently using old versions of compiled contracts.

Thus, the solution we've came up with is to save artifacts to the cache directory and symlink latest build data to user specified out directory.

This PR enables support for this and will be accompanied by the foundry PR using this.

The solution is to add optional artifacts_symlink parameter to ProjectPathsConfig which will be used to link artifacts_symlink to artifacts on each create_artifacts call (currently invoked in ProjectPathsConfig::create_all and when writing artifacts.

IMO symlinking logic should be kept in compilers as we want to create/remove symlinks in the same place where artifact directories are created/removed to avoid situations in which we might end up having only one of two.

Still not sure how this will work out on Windows

cc @mds1

klkvr avatar Mar 13 '24 22:03 klkvr

if I understood this correctly, then this would be a non issue if we would clear the out folder?

what would the layout look like in that case?

Thus, the solution we've came up with is to save artifacts to the cache directory and symlink latest build data to user specified out directory.

I did not understand that part

The idea is that artifacts are saved to ./cache/{profile}/artifacts and latest build symlinked to ./out

klkvr avatar Mar 14 '24 01:03 klkvr

The idea is that artifacts are saved to ./cache/{profile}/artifacts and latest build symlinked to ./out

I really don't want to do any symlink stuff.

what would the layout look like in that case?

I thought:

out/{profile}

?

all of this is a bit complicated because we support setting the output folder for the profile, and I've seen setups where uses read artifacts from other profile outputs, for example in a test that is run with the default profile, you can read artifacts that were compiled with the optimized profile, this is a hack to compile some contracts with different settings

mattsse avatar Mar 14 '24 12:03 mattsse

all of this is a bit complicated because we support setting the output folder for the profile, and I've seen setups where uses read artifacts from other profile outputs, for example in a test that is run with the default profile, you can read artifacts that were compiled with the optimized profile, this is a hack to compile some contracts with different settings

yeah, we can start writing to {out}/{profile}, but that way all artifacts will just migrate one level deeper and such setups might read old artifacts from out/Counter.sol/Counter.json while fresh artifacts will actually be in out/default/Counter.sol/Counter.json

klkvr avatar Mar 14 '24 18:03 klkvr

gotcha, so this concerns setups that run a new version with already compiled artifacts?

haven't fully understood how symlinking helps here.

is the idea to write artifacts always into cache/profile/ and then symlink to the configured out folder?

mattsse avatar Mar 14 '24 18:03 mattsse

is the idea to write artifacts always into cache/profile/ and then symlink to the configured out folder?

yep, that way we separate cache and artifacts under cache/{profile}, but out still has the same layout with out/Counter.sol/Counter.json being the recent version

klkvr avatar Mar 14 '24 19:03 klkvr

If we want to avoid symlinks, we could just duplicate the output in both places, like literally copy/paste the files from one dir to the other

mds1 avatar Mar 14 '24 19:03 mds1

If we want to avoid symlinks, we could just duplicate the output in both places, like literally copy/paste the files from one dir to the other

That could work also, but would require copy-pasting even on cached runs to place most-recent profile data to out

klkvr avatar Mar 14 '24 20:03 klkvr

curios wdyt @DaniPopes ?

mattsse avatar Mar 14 '24 21:03 mattsse