babelfish Heap allocation size failed

Version

latest

Description

There is a heap allocation-size fail due to the unsafe "alloc" function used by "serde_transcode" in main() function.

Current Behavior

ASAN detect heap allocation size failed due to the allocation by "cbor2yaml" program exceed the maximum supported size

    #0 0x55afb3eff2e7 in malloc /home/nyw0102/s2fuzz/scripts/rust/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #1 0x55afb417ee74 in alloc::alloc::alloc::h55894437b2dde2b4 /home/nyw0102/s2fuzz/scripts/rust/library/alloc/src/alloc.rs:171:73
    #2 0x55afb417ee74 in alloc::alloc::Global::alloc_impl::h4817426d3ee57fb8 /home/nyw0102/s2fuzz/scripts/rust/library/alloc/src/alloc.rs:171:73
    #3 0x55afb41a1556 in _$LT$alloc..alloc..Global$u20$as$u20$core..alloc..Allocator$GT$::allocate::h91bb7805fc186578 /home/nyw0102/s2fuzz/scripts/rust/library/alloc/src/alloc.rs:231:9
    #4 0x55afb419131b in alloc::raw_vec::RawVec$LT$T$C$A$GT$::allocate_in::hee6887bf0ee6037d /home/nyw0102/s2fuzz/scripts/rust/library/alloc/src/raw_vec.rs:185:45
    #5 0x55afb40a789d in alloc::raw_vec::RawVec$LT$T$C$A$GT$::with_capacity_in::ha05b634d20c0874d /home/nyw0102/s2fuzz/scripts/rust/library/alloc/src/vec/mod.rs:483:9
    #6 0x55afb40a789d in alloc::vec::Vec$LT$T$C$A$GT$::with_capacity_in::hb0f6be8fc8c4e7b4 /home/nyw0102/s2fuzz/scripts/rust/library/alloc/src/vec/mod.rs:641:20
    #7 0x55afb40a789d in alloc::vec::Vec$LT$T$GT$::with_capacity::h4460949028964e4e /home/nyw0102/s2fuzz/scripts/rust/library/alloc/src/vec/mod.rs:483:9
    #8 0x55afb400a164 in _$LT$serde_transcode..Visitor$LT$S$GT$$u20$as$u20$serde..de..Visitor$GT$::visit_seq::h3c5d5e1ae42aee7b /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-transcode-1.1.0/src/lib.rs:222:21
    #9 0x55afb3f2e2c0 in serde_cbor::de::Deserializer$LT$R$GT$::parse_array::_$u7b$$u7b$closure$u7d$$u7d$::h0bcfac1263691eb3 /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:443:25
    #10 0x55afb3f5e1fe in serde_cbor::de::Deserializer$LT$R$GT$::recursion_checked::h043c35475cd958e8 /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:433:17
    #11 0x55afb3f2d508 in serde_cbor::de::Deserializer$LT$R$GT$::parse_array::h2b12f83534c83f9f /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:442:9
    #12 0x55afb3f53238 in serde_cbor::de::Deserializer$LT$R$GT$::parse_value::hced52b60c2113f6d /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:698:17
    #13 0x55afb4058c44 in _$LT$$RF$mut$u20$serde_cbor..de..Deserializer$LT$R$GT$$u20$as$u20$serde..de..Deserializer$GT$::deserialize_any::hf184ac496ddcc69d /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:788:9
    #14 0x55afb3ff6e86 in serde::ser::impls::_$LT$impl$u20$serde..ser..Serialize$u20$for$u20$$RF$T$GT$::serialize::hd7087fcb6bb24650 /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/ser/impls.rs:390:17
    #15 0x55afb40127ab in _$LT$serde_yaml..ser..SerializeMap$u20$as$u20$serde..ser..SerializeMap$GT$::serialize_key::ha109c73371b153c2 /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_yaml-0.8.17/src/ser.rs:760:30
    #16 0x55afb3fa5a66 in _$LT$serde_yaml..ser..ThenWrite$LT$W$C$serde_yaml..ser..SerializeMap$GT$$u20$as$u20$serde..ser..SerializeMap$GT$::serialize_key::h658a1fb283425809 /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_yaml-0.8.17/src/ser.rs:384:9
    #17 0x55afb4016aba in _$LT$serde_cbor..de..MapAccess$LT$R$GT$$u20$as$u20$serde..de..MapAccess$GT$::next_key_seed::h4d9f6f9fa5899def /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:1007:21
    #18 0x55afb40037bd in _$LT$serde_transcode..Visitor$LT$S$GT$$u20$as$u20$serde..de..Visitor$GT$::visit_map::h78626bd92ffa3e19 /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-transcode-1.1.0/src/lib.rs:231:30
    #19 0x55afb3f92f11 in serde_cbor::de::Deserializer$LT$R$GT$::parse_map::_$u7b$$u7b$closure$u7d$$u7d$::he3e17dbf46c301ae /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:474:25
    #20 0x55afb3f670e2 in serde_cbor::de::Deserializer$LT$R$GT$::recursion_checked::h772035660f4a854b /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:433:17
    #21 0x55afb3f9037f in serde_cbor::de::Deserializer$LT$R$GT$::parse_map::habeca42c0a074909 /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:473:9
    #22 0x55afb3f3bf49 in serde_cbor::de::Deserializer$LT$R$GT$::parse_value::h95fd9ee4a2b21945 /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:722:17
    #23 0x55afb4058bbd in _$LT$$RF$mut$u20$serde_cbor..de..Deserializer$LT$R$GT$$u20$as$u20$serde..de..Deserializer$GT$::deserialize_any::h9480ab181de8c9de /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs:788:9
    #24 0x55afb3fabd9f in serde_transcode::transcode::h98bb7df4ec333194 /home/nyw0102/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-transcode-1.1.0/src/lib.rs:52:5
    #25 0x55afb405b15e in cbor2yaml::main::hf47134d444d86b5b /home/nyw0102/Test-Sets/babelfish/src/bin/cbor2yaml.rs:11:5
    #26 0x55afb3fd144a in core::ops::function::FnOnce::call_once::hef85f256b1dc7949 /home/nyw0102/s2fuzz/scripts/rust/library/core/src/ops/function.rs:248:5
    #27 0x55afb3fc6516 in std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::h12f03545b4803481 /home/nyw0102/s2fuzz/scripts/rust/library/std/src/rt.rs:145:18
    #28 0x55afb42a9174 in std::rt::lang_start_internal::h4a61547abbd425a7 (/home/nyw0102/Test-Sets/babelfish/target/x86_64-unknown-linux-gnu/debug/cbor2yaml+0x481174) (BuildId: 827b9240f67b3e655dc439f39256d3881ff5a7f7)
    #29 0x55afb405c28f in main (/home/nyw0102/Test-Sets/babelfish/target/x86_64-unknown-linux-gnu/debug/cbor2yaml+0x23428f) (BuildId: 827b9240f67b3e655dc439f39256d3881ff5a7f7)

==2895554==HINT: if you don't care about these errors you may set allocator_may_return_null=1
SUMMARY: AddressSanitizer: allocation-size-too-big /home/nyw0102/s2fuzz/scripts/rust/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 in malloc
==2895554==ABORTING

Expected Behavior

Memory-safety code with no allocation-size failed. This might be handled by denying allocation when the size of object exceeds the maximum allocation size.

Oct 08 '24 06:10 nyw0102

Thanks for submitting this issue, can I know how to reproduce this issue?

Oct 10 '24 15:10 pickfire

It seems that the crash can be reproduced when the given input from stdin is converted to u64 which is bigger than 0x10000000000.

Oct 11 '24 12:10 nyw0102

I attach .docx file with brief explanation of this crash. Bug explanation.docx

Oct 11 '24 12:10 nyw0102

Do you have a sample code that I can reproduce instead of screenshots of crash?

Oct 14 '24 13:10 pickfire

Sure! Here is a sample command that I used and the given input file. You can use this with a command ./cbor2yaml < ./input Thanks for looking at this issue consistently. input.zip

Oct 14 '24 17:10 nyw0102

Thanks for providing a minimal reproducible file. Looks like it's not a valid cbor file but I guess what you mean is that it shouldn't panic. memory allocation of 20266425097440408 bytes failed.

Hmm, not quite sure how to go about this since it needs to dig into yaml part, cbor2json seemed fine.

Oct 21 '24 14:10 pickfire

Sorry for my late replay. I roughly analyze the bug, then I found the code that determine the size to be allocated in the heap looks like this: fn parse_u64(&mut self) -> Result<u64> { let mut buf = [0; 8]; self.read .read_into(&mut buf) .map(|()| u64::from_be_bytes(buf)) }

In the other function, the length returned by this function is filtered when the value is bigger than the maximum value of usize. But I assume that the maximum size to be allocated in the heap is much less than the maximum value of usize. So, I think adjust the value used in if-statement in the following function would be one of the solution.

let len = self.parse_u64()?; if len > usize::max_value() as u64 { return Err(self.error(ErrorCode::LengthOutOfRange)); } self.parse_map(len as usize, visitor)

The location of this function: $HOME/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.1/src/de.rs

This could be one of the solution, but It would be an heuristic way. If you want me to analyze the root cause more precisely, let me know! I'll gratefully do to contribute this program.

Oct 26 '24 17:10 nyw0102

@pickfire Hello, I hope you're doing well. If you don't mind, it will be grateful if you take a look at this issue.

Mar 26 '25 15:03 nyw0102

I am personally not using it now, and currently it fits my purpose, sorry not going to put in effort here but if you submit a PR I would take a look at it, I don't mind giving out ownership too.

May 05 '25 15:05 pickfire