hash
hash copied to clipboard
Avoid copying data when converting ArrayData from JS to Rust
🌟 What is the purpose of this PR?
When building ArrayData on the Rust side we copy all data from Javascript.
Rust and Javascript don't have the same expectation regarding what the array contains.
But sometimes they line up and we can skip the copy and just map the array.
🔗 Related links
- Asana task (internal)
🚫 Blocked by
- [ ] Investigate memory leak (internal)
🔍 What does this change?
- Update the arrow Rust library from v10 to v13
- Map array data when possible to avoid copying the data from Javascript to Rust
📜 Does this require a change to the docs?
No
🐾 Next steps
- Move to Arrow2 (internal)
🛡 What tests cover this?
- All tests involving Arrow and the Javascript runner
❓ How to test this?
- Checkout the branch / view the deployment
- Try
cargo test - Confirm that all tests pass