aonez
So... I did some tests about how to preserve the resource forks only with what's included with the system
I was experimenting with this for a while... trying out different ways to preserve resource forks with different kinds of archives... and I discovered some interesting ways to do it with what's already included with the system 🙂 I also tried out goldin and Apple's old splitforks and I'll write about those too.
Anyway 🙂
Here's some way that just seems like a new approach to doing it... I haven't heard about someone doing something like this to save/restore resource forks before...
A convenient .tar or .cpio that just contains the parts needed to recreate the resource forks
To get a convenient .tar archive with just the files that are created for resource forks in archives (so it does not contain the "data" part of files), simply cd to the directory above the one you are trying to archive and let's say the folder you are trying to archive is called "your folder"... just enter this into the Terminal then...
tar cf - your_folder | tar cf your_folder_resource_forks.tar --include='*/._*' --include='._*' @-
for .cpio, use this
tar cf - your_folder | tar cf your_folder_resource_forks.cpio --format cpio --include='*/._*' --include='._*' @-
(BTW... It doesn't even have to be a folder actually... it can be a single file too... it should work also)
If you want to also include other hidden files like .DS_Store which aren't resource forks in it, so you can keep those separate from the rest of the archive (if you maybe also want to share that archive with Windows users, but want to have an option to put those back when you recreate the resource forks), then use this
tar cf - your_folder | tar cf your_folder_resource_forks.tar --include='*/.*' --include='.*' @-
if you want the .cpio format, then use this
tar cf - your_folder | tar cf your_folder_resource_forks.cpio --format cpio --include='*/.*' --include='.*' @-
I was trying to avoid unnecessarily writing to disk... so, with these commands above, what happens is that tar would start making an archive with all files from your folder and it would start creating separate hidden files in it for the resource forks, but instead of writing that whole archive to the disk, it gets piped to tar again, for tar to create a new archive just with the files that were created for resource forks (or if you used the 2nd command I mentioned, with other hidden files too) and write it to the disk... so if the resource forks in your folder are only a small part of everything else that's in it (if it's not mostly something like those old font files that stored the fonts in resource forks) and other hidden files don't take much space too (if you choose to include them), then such a .tar (or .cpio) archive should be tiny 🙂
After that, you can compress other files in your folder without having to use .tar with some utility that does not have any support for resource forks... you can use just the regular 7z or even non-archiving formats like .gz for single files without using tar.
BTW... Keep in mind that tar would need to read all the files in a folder that you want to create those archives with resource forks for... so if you maybe have some super-huge folder on an external HDD, especially if it uses USB2, it might take a while for it to read all those files.
A super-simple way to recreate the resource forks from that .tar
After you extract the files from 7z, you can recreate the resource forks for files that are missing them from that convenient .tar archive that just stores the resource forks 🙂
Just make sure that all files are still called the same (so the resource forks would get applied to the same files) and that the directory structure is the same...
Then go to the Terminal, make sure you are in the directory where the folder/file you uncompressed is (don't cd into the folder you uncompressed... you should be in its parent directory), make sure that your_folder_resource_forks.tar is there too (next to it, not inside of the folder you uncompressed or in some other directory) and extract that .tar archive like this
tar -xf your_folder_resource_forks.tar
In other words, make it extract to the same folder where the files that you want it to apply those resource forks to are located... as if you want to overwrite those files, but since the data forks aren't in that .tar archive, it just adds the resource forks back to those files/folders 🙂 If their tags, custom icons or backgrounds were missing after you extracted them from 7z or some other archive formats, tags, custom icons and backgrounds should be back... if some old fonts were broken or if the aliases were broken, they should work again 🙂
If you simply extracted that .tar into another folder, tar (the one that cones with macOS) just seem to try to turn the "Apple Double" files inside of that archive back into resource forks and it would create empty data files to apply resource forks to those if the data forks aren't in the archive also and those files are missing from the folder too... Archive Utility seems to do the same for .tar
If you created a .cpio archive, you can extract using ditto and ditto would also recreate the resource fork... as with that command for tar, just make sure you cd to the parent directory of the file/folder you want to apply those resource forks too first...
ditto -c your_folder_resource_forks.cpio .
Don't forget the dot in the end (for the current directory)
For .cpio, extracting to another folder with the Archive Utility or with just the "ditto -c your_folder_resource_forks.cpio ." does not seem to create empty files to apply resource forks to those if the files with data forks aren't there or in the archive... so the resource forks will not be recreated for missing files.
What if you just wanted to get just those hidden ._ files in some folder?
With ditto, there's an option you can use to extract those hidden ._ files from .cpio, without ditto turning them back into resource forks... so, if you have a folder called "your_folder" and you created the "your_folder_resource_forks.cpio" for it, use this:
ditto -x --norsrc your_folder_resource_forks.cpio your_folder_resource_forks
The reason to create another folder for those is because there should be a file called "._your_folder" for "your_folder" and if you just extract to the current directory, you maybe have lots of other files there... and Finder does not seem to show those even if you press CMD+Shift+. to show the hidden files (you can see it if you list what's in some folder with "ls -a")... so it just seems to me more convenient to have all those hidden files inside one folder.
You could later rename "your_folder_resource_forks" to something else... that's just an example to know to which folder those files belong, so you don't mix it up.
Another approach... using goldin
I tried splitforks (included with Apple's command line developer tools) and goldin also... they seem similar, but goldin seems better... if you don't have Xcode installed to build it, "Packages for Mac" (utilities for creating .pkg packages) includes goldin too.
Since goldin creates those hidden "._" files that are created for archives in the folder where those files are, if some folder is read-only or if you just don't want those hidden files to be added to some folder, here is what you can do...
If it's a HFS+ volume, you can use pax to create the same directory structure in some other place as a temporary folder with hard links to those files and create the hidden "." files for resource forks there... then, after you get those "." files, you can just delete that directory.
If your folder is called "your_folder" and you want to the directory structure and create hard links to files in it, cd to that folder's parent directory, create another folder next to it called "tmp_folder" and use this
pax -rwl your_folder tmp_folder
On APFS you can jus duplicate some folder, since APFS does not support hard links, but duplicating a folder on APFS seems kind of similar as creating hard links, since it's not writing all those files again.
If you want to recreate resource forks from those "._" files
If you have those "._" AppleDouble files and you want to recreate resource forks for files that have data forks only, you can use ditto for that...
If both the "._" files for resource forks and files with data forks are in a folder called "your_folder" and you want to recombine them and write them to a folder called "recombined_files", you could use this
ditto -c --norsrc your_folder - | ditto -x - recombined_files
I would jus recommend avoiding unnecessarily writing to the disk if you don't need to keep both versions as separate files.
You could cd into that folder and create a .cpio with those "._" from those files your current directory, that you could extract with ditto when you want to add the resource forks
find ._* */._* -print -depth | cpio -ov > resource_forks.cpio
You could also just pipe that to ditto, for ditto to add the resource forks in one go
find ._* */._* -print -depth | cpio -ov - | ditto -x - .
I did more tests to figure out how to place those "._" AppleDouble files in a "__MACOSX" folder and add it to some .zip archive, without having issues like the Archive Utility just extracting the "__MACOSX" folder as an ordinary folder or ignoring it (when it doesn't extract it, but also doesn't recreate the resource forks from it)... or issues like the unar binary complaining that it can't open some resource forks that you added to the "__MACOSX" folder yourself...
After various tests, I think I have figured out how it works...
To create a "__MACOSX" folder with those "._" AppleDouble files for some folder with one command, cd to the parent directory of the file/folder you want to zip and use this
tar cf - your_folder | tar cf - --format cpio --include='*/._*' --include='._*' @- | ditto -x --norsrc - __MACOSX
Don't open that folder in Finder after it's created (you can use ls -a to list the contents of it in the Terminal), so .DS_Store files would't get generated also... The reason why the Archive Utility sometimes just extracts the "__MACOSX" folder that you tried adding yourself seems to be that there are other files inside (files other than the "._" AppleDouble files), like the .DS_Store files.
It also seems important that files are added in a specific order.
When I just tried to create a .zip archive with the 7z binary and to add the "__MACOSX" folder with it (after checking that there weren't some extra files inside with ls -a), the Archive Utility was not recognizing that there was a "__MACOSX" folder with "._" AppleDouble files inside of the archive... it wasn't extracting it as an ordinary folder as it did when there were other files inside, but it also didn't seem to recreate the resource forks from it... it just seemed to ignore it completely.
When I tried creating the archive with the zip binary that's included with the system and to add the "__MACOSX" folder with it, after other files were added, then the Archive Utility recognized the "__MACOSX" folder and recreated the resource forks from the files in it.
After various tests to see what was the issue when the 7z binary added that folder, it seems the issue was about to which part of the archive were the files written... listing the archive with 7z showed that when the "__MACOSX" folder was added with the 7z binary after other files, that it still appeared first on the list... and when it was added with the zip binary after other files, that it appeared after other files.
The Archive Utility also didn't seem to recognize the "__MACOSX" folder if it was added before other files using the zip binary... so the order in which it was added did seem to matter.
When I tried creating a zip archive without the "__MACOSX" folder with the 7z binary and then just adding the "__MACOSX" folder with the zip binary, that worked for the Archive Utility too.
7z a your_folder.zip your_folder
and after 7z is done just add the "__MACOSX" directory like this
zip -r your_folder.zip __MACOSX
The Archive Utility seemed OK with that
Still, when I tried to extract that with the unar binary, it was complaining that it can't open some resource forks.
So I tried creating a zip archive from the same files with the Archive Utility and comparing it with archives created in other ways... after listing it with 7z, it seems that the Archive Utility adds files in a specific order...
When the Archive Utility writes the data fork for some file (or in other words a file as it is without the resource forks), it seems that it then writes the "." AppleDouble file for that file's resource forks to the "__MACOSX" folder... then moves on to write the data fork for some other file and writes the "." AppleDouble file for that other file's resource forks to the "__MACOSX" folder... and so on... which would have actually made a lot of sense back in the days when CDs and various floppy disk formats were common (I don't mean just the ordinary 1.44 MB floppy... there were the SuperDisk drives and ZIP drives too, which had much larger capacities than an ordinary floppy and had USB versions for Mac too... and many Mac users probably used those), so the resource fork would be near the data fork if someone was reading that .zip from a CD or some floppy disk format.
Although the Archive Utility didn't seem to have an issue uncompressing resource forks when the whole "__MACOSX" folder was just added after other files in one go... it just seemed to be important to add the files for data forks before adding the files for resource forks... but I should mention that I didn't test it with some huge .zip archives to see would it matter for the Archive Utility if there were a lot more files between some file's data fork and that files resource fork.
The unar binary does seem to have an issue with zip files that had the whole "__MACOSX" folder with all files in it just added like that after other files were already added, instead of alternating between data forks and resource forks as files are added, one by one. It was complaining that it "can't open" some resource forks for some tagged folders (even though the Archive Utility seemed to opened them just fine).
When you use the zip binary to add files in that specific order like the Archive Utility uses, then the unar seems fine with it.
With the zip binary you can add just a folder without adding anything from it (if you don't specify the "-r" option, then you can add files/folders one by one), then add the "." AppleDouble file with resource forks just for that folder... then start adding files from it, one by one, as you also add the "." AppleDouble for a specific file/folder after you add it... then the "." AppleDouble files are written right after the files they belong to and the unar binary doesn't seem to complain that it "can't read" those resource forks and seems to recreate the resource forks just fine... so there doesn't seem to be anything wrong with those "." AppleDouble files that were generated by using tar and ditto... the order in which those files are added to the zip seems to matter a lot for it to work properly with the current version of the unar binary.
The unar binary seems to show those complains for the 7z format too when the whole "__MACOSX" folder is added in one go... but I didn't find a way how to arrange files in a 7z archive in that specific order with the 7z binary, like it could be done for zip with the zip binary by just adding the files in a specific order.
The 7z binary seems to rearrange the files/folders in an archive automatically in an alphabetical order when it adds something to it, regardless of is it a .zip or a .7z archive... I also looked at the contents of archives with a hex editor, to check were the files actually written in an alphabetical order or it just listed them in an alphabetical order... and it seemed like the files/folders are arranged alphabetically, even if you were adding files one by one (with separate commands to add a file to an exiting archive) in some other order... and that seems to be the reason why the Archive Utility doesn't seem to recognize the "__MACOSX" folder inside .zip archives if it was added with the 7z binary, since it seems to expect it after other files, but 7z moves it before files that begin with an a, b, c, d... etc.
I'm not sure is there some way for the 7z binary alone to arrange files in that specific order instead of arranging them alphabetically... I didn't find a way to do that with it like it can be done for .zip with the zip binary.
Maybe some developers could make a fork of 7z that could make it easy to arranging files in such an order easily... or maybe someone could find a way to fix unar.
Some would probably argue that since CDs and various floppy disks are't widely used today and that it's unlikely that someone would make huge 7z archives to put them on LTO tapes (since LTO drives have their own compression and 7z doesn't seem to have been meant for tapes), that it wouldn't have much difference on performance if files that were going to be read one after another weren't written close to one another in an archive... but if arranging files alphabetically inside of archive was causing other issues, like archivers not recognizing resource forks, then they should probably have an obvious option to turn off such rearranging if someone doesn't want it.
Soooo... I'm not sure should the "__MACOSX" folder be added to 7z archives if there isn't some obvious and easy way to arrange the files in that order and if the unar binary shows complains about how it can't read some resource forks... I'm also not sure should the "__MACOSX" folder be added to .zip archives created with 7z by just adding the "__MACOSX" folder with the .zip binary after it in one go, since unar seems to complain about how it can't read some resource forks when that is done too.
But I think it would be good if there was an option in Keka to create like some additional .tar with just the resource forks and to maybe add that to .zip and .7z archives... the file name could be something like "archive.macos.rsrc.tar" and when Keka detects it inside an archive, it could just restore the resource forks from it... and other archivers that don't support it would just extract it as any other file, so the user could recreate resource forks from it with from the Terminal if they don't have Keka... and there could maybe be a tiny readme file inside that archive to tell the user how to do it if they try unpacking it and don't know what it does.
There could also maybe be an option to create two .tar archives in one go, one for data forks and one for resource forks 😁 It can be done with just one command 😁
tar cf - your_folder | tee >(tar cf your_folder_resource_forks.tar --include='*/._*' --include='._*' @- ) >(tar cf your_folder_data_forks.tar --exclude='*/._*' --exclude='._*' @- ) > /dev/null
So, as tar reads the files and creates those hidden files for resource forks, it gets piped to tee, so tee can make tar do two different things with that output... and so tee wouldn't also output what it gets from tar to the Terminal, what it would output to the Terminal gets redirected to /dev/null to be ignored.
BTW... Now that I figured out what needs to be done for other archivers to reliably recreate resource forks from the "._" AppleDouble files inside a .zip archive, I could do some more research to see if there is some multithreaded archiving binary that can add files in a specific order and with the help of some script it could be automated... and I'll look if there are some 7z forks that would maybe be more suited for Mac. I'll write more if I discover something interesting that's related to that 🙂