[QUESTION]: Speedup for converting numpy array to std::vector<T>

[wuxian08]

Required prerequisites

• [X] Make sure you've read the documentation. Your issue may be addressed there.
• [X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• [ ] Consider asking first in the Gitter chat room or in a Discussion.

Problem description

I have a function foo which takes a std::vector as input, and I would like to expose the function to python, in which python programmers would call foo by passing in python lists or numpy arrays.

extern void foo(const std::vector<int64_t> &v);

PYBIND11_MODULE(test, m) {
    m.def("foo", &foo);
}

For numpy arrays, the performance of casting is a bit slower than I thought: in fact, it is even slower than converting a list to std::vector. After a bit of research, I found that to cast numpy arrays, one first converts numpy raw data to a PyObject (note for lists, this proc could be ignored) before converting it to T, and finally moves it to std::vector. For references, list_caster::load type_cast::load

So my question is, is this possible for a speed-up when casting a numpy array into std::vector? I have noticed that vector_buffer_impl does a pretty good job at converting a py::buffer to a std::vector, maybe this piece of code could be used when casting into std::vector. If this could be a way, I would volunteer to submit a pull request.

Any thoughts would be welcomed and appreciated. Thanks.

[etienne87]

i also struggle with this, i coded my c++ rountine with std::vector and i use py::array object.cast<std::vector> and this is abysmally slow (around 3s for a volume of 300x300x300)... what is the proper way of doing this?

[LeBron-Jian]

I had a similar problem, when I had a function foo on the C++ side, which accepted a std::vector as input, and I wanted to expose that function to python, where the python programmer would do it by passing in a python list or numpy array Call foo.

I found that if a large amount of data is stored in std::vector, the access is very slow. Is there any solution?