wasi-nn icon indicating copy to clipboard operation
wasi-nn copied to clipboard

Add enum for tensor layout

Open oovm opened this issue 1 year ago • 4 comments

Is it possible to add an interface to get the tensor layout (memory format) of tensor?

Such as NCHW or NWHC, I hope that incorrect arrangements can be determined and rejected in wasm ​

oovm avatar Mar 05 '24 16:03 oovm

We've discussed this a few times in the past; maybe a summary of the current state is that we need some way to read graph metadata and potentially attach it to tensors. Your feedback on what is motivating this would be pretty helpful — can you attend the ML working group on the 18th of this month (details)?

abrown avatar Mar 14 '24 17:03 abrown

Sorry, I can't attend this meeting, I am not an English speaker, I use Google and chatgpt to translate the conversation, but I will pay attention to the meeting minutes.

oovm avatar Mar 15 '24 15:03 oovm

No problem; I will relay this request in the meeting then.

abrown avatar Mar 15 '24 17:03 abrown

@oovm, we discussed this at the last ML meeting (here) and here is a summary of that discussion:

  • the difference between NHWC, NCHW, etc. is specific to tensors that represent images
  • wasi-nn is a system-level API and cannot possibly provide developer-friendly features for every specific ML use case
  • if it were to provide something like this metadata, it might need to be more unstructured than an enum (e.g., key-value strings) and be accessible from the graph resource, not at the tensor level
  • some of those present have discussed adding unstructured metadata to graphs, so that may be an avenue to explore
  • but, in the end, that additional information doesn't free the developer from knowing how to properly interface with the ML model they choose to do inference with; being able to extract some model metadata doesn't fully solve the problem of making this API easier to use

I believe there is plenty of room above the system API level for an ease-of-use wrapper library; there are the beginnings of this kind of thing over in this repository.

abrown avatar Mar 18 '24 18:03 abrown