arrayvec icon indicating copy to clipboard operation
arrayvec copied to clipboard

Add "placement new"-like constructor API

Open aleksanderkrauze opened this issue 1 year ago • 6 comments

Feature description

Add new constructors to ArrayVec and ArrayString, that instead of returning initialized Self, write to user-provided out-pointer.

Rationale

Consider following case. I want to use a heap allocated Vec-like structure, but with const-known maximum capacity. I would like to use Box<ArrayVec<T, N>> as a backing storage. However creating such type is problematic. For sufficiently big N expression Box::new(ArrayVec::new()) may overflow stack. Box (and other types in standard library) have currently unstable (but stable in current beta, which will hit stable in 3 days) API new_uninit that helps to solve this exact case. However arrayvec does not have any API that would allow constructing its types in-place, which makes it impossible to safely use aforementioned std APIs. By adding this kind of constructors, ArrayVec becomes usable in described scenario (and others that require in-place initialization).

Drawbacks

  • Adding this will expose new API, which will increase stability burden on maintainers.
  • Implementation of such constructor will require unsafe code. While it wouldn't be very difficult one, it will require more attention when doing possible internal refactors.

Other possibilities

One possibility is to just do nothing. Users who wish to use placement-new-like constructors can just re-implement ArrayVec manually.

There is also a dark and unsafe way. Since ArrayVec has #[repr(C)], one can create their own mirror type, initialize it, and then std::mem::transmute it into arrayvec::ArrayVec. I think it requires no further explanation why this should not be preferred by anyone. :)

Third possibility would be to make ArrayVec's fields public, which would allow users to instantiate it how they wish. I do not want to endorse this, just mention it for the sake of completeness.

Possible implementation

Here is a possible implementation:

impl<T, const CAP: usize> ArrayVec<T, CAP> {
    pub fn new_in(dest: &mut MaybeUninit<Self>) {
        let dest = dest.as_mut_ptr();
        unsafe {
            let len_ptr = core::ptr::addr_of_mut!((*dest).len);
            len_ptr.write(0);
        }
    }
}

It could be used like this:

let mut stack: Box<ArrayVec<i32, N>> = {
    let mut uninit_stack = Box::new_uninit();

    ArrayVec::new_in(&mut uninit_stack);

    unsafe { Box::<MaybeUninit<_>>::assume_init(uninit_stack) }
};

Open questions

  • [ ] What should be the name of such constructor?
  • [ ] Should the out-pointer be &mut MaybeUninit<Self>, *mut Self or something different?
  • [ ] Should this method be unsafe? It should be sound anyway, but maybe it would be better that user thinks twice before calling it.
  • [ ] How to communicate to user that calling it on an already initialized value will not call Drop on contained value?

aleksanderkrauze avatar Oct 14 '24 07:10 aleksanderkrauze

If this feature request is accepted, I can provide PR implementing it.

aleksanderkrauze avatar Oct 14 '24 07:10 aleksanderkrauze

For sufficiently big N expression Box::new(ArrayVec::new()) may overflow stack.

Does this still happen with --opt-level 1? If not, maybe you could use that expression and enforce that optimization level even for development builds.

tbu- avatar Oct 14 '24 10:10 tbu-

As far as I know Rust may optimize out memcopy from stack to heap when initializing Box, but it does not guarantee it. So even if at some point in time compiler happens to perform this optimization (which indeed I can do on my machine when I compile my example in --release mode), that doesn't mean it will always do it.

Enabling optimizations for debug builds is also not an option, when I want to create a library.

aleksanderkrauze avatar Oct 14 '24 11:10 aleksanderkrauze

Consider following case. I want to use a heap allocated Vec-like structure, but with const-known maximum capacity. I would like to use Box<ArrayVec<T, N>> as a backing storage.

The use case you brought up in the original case is probably not the whole story, but for that specific case, I think a Vec<T> initialized with Vec::with_capacity(N) would work well. It has an extra usize that stores the capacity at run-time, but that doesn't sound like a big problem if N is so big that [T; N] does not fit onto the stack.

tbu- avatar Oct 14 '24 12:10 tbu-

I think the idea sounds good. It was long used that Box::default() (with T=ArrayVec<U>!) would use a box operator call and avoid putting the arrayvec on the stack. I don't know if that still works, but I assume it does.

bluss avatar Oct 14 '24 18:10 bluss

If we're placement new'ing, any input on best shape of api here: #278 ?

bluss avatar Oct 16 '24 14:10 bluss