Read from stream in DataExchange

[Fairyon]

Hi everyone! I get a filestream of STEP and IGES files from a REST API and want to analyse it. I saw, that STEPControl_Reader supports reading from stream, but the DataExchange class does not.

I have wrote my own extension read_step_stream(datastream, ...), where I replaced just the line of read_step_file(filename, ...) status = step_reader.ReadFile(filename) with this one: status = step_reader.ReadStream(datastream) and the same for iges

Would it be possible for you to add function read_step_stream(datastream, ...) in a further releases?

Thanks in advance.

Alex aka Fairyon

[cafhach]

Hi Fairyon,

great work. The next step would be to create a pull request containing your extension for @tpaviot to oversee and potentially accept.

[JohannesVerherstraeten]

Hi, I have the same use case. And I believe this issue is about the same too.

It would be great if this feature can be added!

Problem statement

To be clear, instead of loading my STEP file like this (the default way):

def read_step_file(file_path: str) -> TopoDS_Shape: 
    step_reader = STEPControl_Reader()
    step_reader.ReadFile(file_path)
    step_reader.TransferRoot()
    return step_reader.Shape()
shape = read_step_file("my_file.step")

I'd like to be able to load it like this:

def read_step_file(file_stream: io.BytesIO) -> TopoDS_Shape: 
    step_reader = STEPControl_Reader()
    step_reader.ReadStream(file_stream)     # This method exists, but doesn't work with python streams. See below
    step_reader.TransferRoot()
    return step_reader.Shape()

# Here we read the stream from file, but in reality it would be a HTTP request body. 
with open("my_file.step", "rb") as f:
    stream = io.BytesIO(f.read())

shape = read_step_file(stream)

Possible hint towards a solution?

I noticed that the OCC.Core.STEPControl.STEPControl_Reader class inherits OCC.Core.XSControl.XSControl_Reader, which has a ReadStream method, but this method doesn't accept python streams.

class XSControl_Reader(object):
    ...
    def ReadStream(self, *args) -> "IFSelect_ReturnStatus":
        r"""
        Loads a file from stream and returns the read status.

        Parameters
        ----------
        theName: char *
        theIStream: std::istream
        """
        return _XSControl.XSControl_Reader_ReadStream(self, *args)

Or is there a way to wrap the python stream in some data structure that this ReadStream method accepts?

All help is very much appreciated!

[tpaviot]

Currently, the c++ std::istream type is not handled correctly by pythonocc, so the wrapper does not provide a way to pass file streams to methods. Let me have a look, the use case (step file from a rest API) is worth investigating this issue further.

[Omnipius]

I have this same use case: reading a STEP stream from a REST API. Having to save and load a file is a security vulnerability for my use case that needs to be closed.

I'm not sure if pythonocc uses boost at it's core, but this post on stack overflow shows how to do the conversion from python io.BytesIO to std::istream: https://stackoverflow.com/questions/24225442/converting-python-io-object-to-stdistream-when-using-boostpython

[deGravity]

I will add another use case for supporting the std::istream input; reading in step files from zip and tar archives. In machine learning workflows it is common to keep all of your data in a single archive file that you stream from. What I would like to be able to do is something like

for path in step_file_archive.nameslist():
    with step_file_archive.open(path, 'r') as f:
        file_stream = ...# convert file handle f to something that STEPControl_Reader can handle
        status = reader.ReadStream("rawtext", file_stream)
        ...

[anuraaga]

I was also interested in using step without temporary files, while it does not use python file objects themselves, I sent #1299 with support for strings. I suspect most uses cases here are already buffered (REST API response) or not a huge problem buffering (a single file out of a zip archive) so hoping this is helpful. Anyone that can use it it could be helpful to +1 on the PR. Cheers.