Null padded representation?
cf #29 Rather than sorting the size at all we could store the string in null-padded form. This would let us use up to the full width of the backing primative for content.
Then to determine the ncodeunits we would count backwards from the end how meany consecutive bytes were not 0x00
and subtract that from the size of the backing primative
I don't think that really works, because then you can't store strings with \0
Yes, you can, so long as it isn't at the end. and if it is at the end, it is almost certainly a null-terminated string, and so you don't want to store it.
Its worth considering if handling strings that have nulls at the end but are not null terminated is worth giving up a whole byte.
ShortStrings don't store a terminating null anyway. I've frequently needed strings that were just "\0", or had a '\0' at the end. Having to deal with a single byte "lost" is a lot better IMO than the quirky issues that would come up with allowing '\0' some places, but not others.
Once this is all working correctly (there are still the access violations, and any fallbacks that use String(val) should be rewritten, maybe you could make a ShortNTS package, that works that way.