SPUD icon indicating copy to clipboard operation
SPUD copied to clipboard

Arrays of custom structs

Open krojew opened this issue 1 year ago • 7 comments

I've found that arrays of custom structs are unsupported, but the underlying reason is not clear. We could, in theory, serialize each item just as single nested structs already are. What is the reason for not allowing that in arrays, as it should be doable?

krojew avatar Sep 24 '24 17:09 krojew

The problem is dealing with those structures changing. For efficiency the SPUD data format only stores offsets so that if you need to load old data into a new object it knows how to skip to the right byte offset but avoids storing a large amount of metadata that would slow things down and bloat the save. Unfortunately in the case of arrays of custom structs you need more offset data that isn't currently stored. You need effectively struct "stride" BUT in the case of nested structs that becomes complicated. It's not impossible but needs some significant work that I've never had time for. Previously we had a fallback to using the (much slower, much more bloated) standard UE serialisation of custom structs, but they changed the APIs on archives in one of the 5.x updates and broke that so it's no longer available.

sinbad avatar Sep 24 '24 19:09 sinbad

That's quite problematic, since it would seem, now it's impossible to store arrays of anything but basic primitives. I don't see a workaround.

krojew avatar Sep 24 '24 19:09 krojew

Previously we had a fallback to using the (much slower, much more bloated) standard UE serialisation of custom structs

How much work would it be to get that working again? I don't care if it's slow; I do care that it's stable if the structs change.

maximumsomething avatar Mar 11 '25 21:03 maximumsomething

Unfortunately I couldn't get it working again because UE changed the FArchive related APIs and made the technique I was using impossible. I spent a few hours on it and couldn't figure out a workaround for the latest UEs - maybe someone else can figure it out

sinbad avatar Mar 12 '25 10:03 sinbad

This was the commit that removed support because it no longer worked with UE 5.2+ https://github.com/sinbad/SPUD/commit/17c75688815df9c6ea84458e0207e119a9f938eb

sinbad avatar Mar 12 '25 10:03 sinbad

What is the state of this issue? Not being able to store arrays of custom structs is quite problematic for a lot of games. Is there any good workarounds?

stormsc1 avatar Nov 10 '25 15:11 stormsc1

This was the commit that removed support because it no longer worked with UE 5.2+ 17c7568

It looks like this commit was essentially undone in https://github.com/sinbad/SPUD/commit/db196b31413e9d17e758df058d1c1784046fdb5a, and as far as I'm aware they're now supported as FSpudMemoryReader and FSpudMemoryWriter managed to get past the issue.

cyphusx avatar Nov 10 '25 15:11 cyphusx