goblinhack
how to do some validation while reading (<< operator)?
how to do some validation while reading (>> operator)? can we read the first field and match with some kind of magical number? or other method
In one of my games I write the size and version I expect when reading e.g.
in >> bits(my.t.version); LOG("Read config: version = [%s]", my.t.version.c_str()); in >> bits(my.t.serialized_size); LOG("Read config: serialized_size = %d", my.t.serialized_size);
if (my.t.serialized_size != sizeof(Config)) { game_load_error = "bad save file header version"; return in; }
any checking is really meant to be done by the upper layer, so adding version, size information is up to you - as it's a pretty lightweight tool, it doesn't do some of the versioning stuff that say protobufs might do - you roll your own on top - hth
e.g.
00:00:00.009: Read config: version = [0.0.1] 00:00:00.009: Read config: serialized_size = 632 00:00:00.009: Read config: window_pix_height = 1080 00:00:00.009: Read config: window_pix_width = 1728 00:00:00.009: Read config: config_pix_height = 1080 00:00:00.009: Read config: config_pix_width = 1728 00:00:00.009: Read config: debug_mode = 0 00:00:00.009: Read config: fps_counter = 0 00:00:00.009: Read config: gfx_allow_highdpi = 0 00:00:00.009: Read config: gfx_borderless = 1 00:00:00.009: Read config: gfx_fullscreen = 0 00:00:00.009: Read config: gfx_fullscreen_desktop = 0 00:00:00.009: Read config: gfx_vsync_enable = 1 00:00:00.009: Read config: mouse_wheel_lr_negated = 0 00:00:00.009: Read config: mouse_wheel_ud_negated = 0 00:00:00.009: Read config: music_volume = 42 00:00:00.009: Read config: sdl_delay = 10 00:00:00.009: Read config: sound_volume = 64
Yep Do you know that if you de-serialize an invalid file, you can crash your entire OS! It happens to me in >> bits(map) will read the 4 first bytes as the size of the map and can exhaust your memory if the number is big, crashing your system!
I resolved the issue by putting a signature string has the first field and reading the file first to validate the signature The std::string is stored a 4 bytes length + the data So, if the signature is ok, I then read with in >> bits(map)
Example: const std::string SIGNATURE_RSAKEY = "RSAKEY_0001"; const long SIGNATURE_RSAKEY_POS = 4;
struct file_map_rsa
{
std::string signature = cryptoAL::SIGNATURE_RSAKEY;
std::map<std::string, cryptoAL::rsa::rsa_key> map;
friend std::ostream& operator<<(std::ostream &out, Bits<file_map_rsa & > my)
{
out << bits(my.t.signature)
<< bits(my.t.map) ;
return (out);
}
friend std::istream& operator>>(std::istream &in, Bits<file_map_rsa &> my)
{
in >> bits(my.t.signature)
>> bits(my.t.map) ;
return (in);
}
};
bool read_map_rsa(const std::string& filenameDB, std::map<std::string, cryptoAL::rsa::rsa_key>& map_rsa, std::string& serr, bool check)
{
if (file_util::fileexists(filenameDB) == false)
{
serr += "no file: " ; serr += filenameDB;
return false;
}
if (check)
{
if (file_util::check_signature(filenameDB, cryptoAL::SIGNATURE_RSAKEY, cryptoAL::SIGNATURE_RSAKEY_POS, serr) == false)
{
serr += "bad rsa file signature" ;
return false;
}
}
file_map_rsa m;
std::ifstream infile;
infile.open(filenameDB, std::ios_base::in);
infile >> bits(m);
infile.close();
map_rsa = m.map;
return true;
}
[[maybe_unused]] static bool check_signature(const std::string& filename, const std::string& SIGNATURE, long pos, std::string& err) { cryptoAL::cryptodata input_file; if (input_file.read_from_file(filename) == false) { err += "ERROR reading file: "; err += filename; return false; }
std::uint32_t sz_input = input_file.buffer.size();
if (sz_input < SIGNATURE.size() + pos)
{
err += "ERROR invalid file size ";
return false;
}
for(int i=0;i<SIGNATURE.size();i++)
{
if (input_file.buffer.getdata()[pos+i] != SIGNATURE[i])
{
err += "ERROR invalid signature";
return false;
}
}
return true;
}
Usage: std::string serr; std::map<std::string, rsa::rsa_key> map_RSA; bool ok = keymgr::read_map_rsa(filename, map_RSA, serr);
Sources: https://codeberg.org/alainalain/encryptions
yeah you have to be careful with what is fed to it - I assume the write and read order was out of sync? i.e what was it reading as the map size? perhaps an exception handler wrapping this would help ? not sure how the header file could handle this - it could check for large map sizes.. but not ideal. Or it could add magic numbers so we know to expect a map - but that does bloat things a bit...
On Fri, 11 Apr 2025 at 14:37, Alain Lanthier @.***> wrote:
Yep Do you know that if you de-serialize an invalid file, you can crash your entire OS! It happens to me in >> bits(map) will read the 4 first bytes as the size of the map and can exhaust your memory if the number is big, crashing your system!
I resolved the issue by putting a signature string has the first field and reading the file first to validate the signature The std::string is stored a 4 bytes length + the data So, if the signature is ok, I then read with in >> bits(map)
Example: const std::string SIGNATURE_RSAKEY = "RSAKEY_0001"; const long SIGNATURE_RSAKEY_POS = 4;
struct file_map_rsa { std::string signature = cryptoAL::SIGNATURE_RSAKEY; std::map<std::string, cryptoAL::rsa::rsa_key> map;
friend std::ostream& operator<<(std::ostream &out, Bits<file_map_rsa & > my) { out << bits(my.t.signature) << bits(my.t.map) ; return (out); } friend std::istream& operator>>(std::istream &in, Bits<file_map_rsa &> my) { in >> bits(my.t.signature) >> bits(my.t.map) ; return (in); }};
bool read_map_rsa(const std::string& filenameDB, std::map<std::string, cryptoAL::rsa::rsa_key>& map_rsa, std::string& serr, bool check) { if (file_util::fileexists(filenameDB) == false) { serr += "no file: " ; serr += filenameDB; return false; }
if (check) { if (file_util::check_signature(filenameDB, cryptoAL::SIGNATURE_RSAKEY, cryptoAL::SIGNATURE_RSAKEY_POS, serr) == false) { serr += "bad rsa file signature" ; return false; } }
file_map_rsa m;
std::ifstream infile; infile.open(filenameDB, std::ios_base::in); infile >> bits(m); infile.close();
map_rsa = m.map; return true; }
[[maybe_unused]] static bool check_signature(const std::string& filename, const std::string& SIGNATURE, long pos, std::string& err) { cryptoAL::cryptodata input_file; if (input_file.read_from_file(filename) == false) { err += "ERROR reading file: "; err += filename; return false; }
std::uint32_t sz_input = input_file.buffer.size(); if (sz_input < SIGNATURE.size() + pos) { err += "ERROR invalid file size "; return false; }for(int i=0;i<SIGNATURE.size();i++) { if (input_file.buffer.getdata()[pos+i] != SIGNATURE[i]) { err += "ERROR invalid signature"; return false; } } return true; }
Usage: std::string serr; std::map<std::string, rsa::rsa_key> map_RSA; bool ok = keymgr::read_map_rsa(filename, map_RSA, serr);
Sources: https://codeberg.org/alainalain/encryptions
— Reply to this email directly, view it on GitHub https://github.com/goblinhack/c-plus-plus-serializer/issues/4#issuecomment-2796941805, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADA3H7JWTPXMEIPGJ63LT32Y7ARJAVCNFSM6AAAAAB24IIQFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOJWHE2DCOBQGU . You are receiving this because you commented.Message ID: @.***> alanthie left a comment (goblinhack/c-plus-plus-serializer#4) https://github.com/goblinhack/c-plus-plus-serializer/issues/4#issuecomment-2796941805
Yep Do you know that if you de-serialize an invalid file, you can crash your entire OS! It happens to me in >> bits(map) will read the 4 first bytes as the size of the map and can exhaust your memory if the number is big, crashing your system!
I resolved the issue by putting a signature string has the first field and reading the file first to validate the signature The std::string is stored a 4 bytes length + the data So, if the signature is ok, I then read with in >> bits(map)
Example: const std::string SIGNATURE_RSAKEY = "RSAKEY_0001"; const long SIGNATURE_RSAKEY_POS = 4;
struct file_map_rsa { std::string signature = cryptoAL::SIGNATURE_RSAKEY; std::map<std::string, cryptoAL::rsa::rsa_key> map;
friend std::ostream& operator<<(std::ostream &out, Bits<file_map_rsa & > my) { out << bits(my.t.signature) << bits(my.t.map) ; return (out); } friend std::istream& operator>>(std::istream &in, Bits<file_map_rsa &> my) { in >> bits(my.t.signature) >> bits(my.t.map) ; return (in); }};
bool read_map_rsa(const std::string& filenameDB, std::map<std::string, cryptoAL::rsa::rsa_key>& map_rsa, std::string& serr, bool check) { if (file_util::fileexists(filenameDB) == false) { serr += "no file: " ; serr += filenameDB; return false; }
if (check) { if (file_util::check_signature(filenameDB, cryptoAL::SIGNATURE_RSAKEY, cryptoAL::SIGNATURE_RSAKEY_POS, serr) == false) { serr += "bad rsa file signature" ; return false; } }
file_map_rsa m;
std::ifstream infile; infile.open(filenameDB, std::ios_base::in); infile >> bits(m); infile.close();
map_rsa = m.map; return true; }
[[maybe_unused]] static bool check_signature(const std::string& filename, const std::string& SIGNATURE, long pos, std::string& err) { cryptoAL::cryptodata input_file; if (input_file.read_from_file(filename) == false) { err += "ERROR reading file: "; err += filename; return false; }
std::uint32_t sz_input = input_file.buffer.size(); if (sz_input < SIGNATURE.size() + pos) { err += "ERROR invalid file size "; return false; }for(int i=0;i<SIGNATURE.size();i++) { if (input_file.buffer.getdata()[pos+i] != SIGNATURE[i]) { err += "ERROR invalid signature"; return false; } } return true; }
Usage: std::string serr; std::map<std::string, rsa::rsa_key> map_RSA; bool ok = keymgr::read_map_rsa(filename, map_RSA, serr);
Sources: https://codeberg.org/alainalain/encryptions
— Reply to this email directly, view it on GitHub https://github.com/goblinhack/c-plus-plus-serializer/issues/4#issuecomment-2796941805, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADA3H7JWTPXMEIPGJ63LT32Y7ARJAVCNFSM6AAAAAB24IIQFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOJWHE2DCOBQGU . You are receiving this because you commented.Message ID: @.***>
yes you have to validate your file is some way before deserializing it blindly! unless you want your OS to crash!