oneapi-src
rfc: dnncompat compatibility layer
Description
This is a proposal for dnncompat - oneDNN compatibility layer.
Rendered document can be seen here.
@zhimingwang36, can you please provide your input on how you handle oneDNN/cuDNN scratchpad and workspace?
Below are related questions from @mgouicem:
Other thing is about memory management. In cuDNN scratchpad is handled by user and can be queried for specific primitives. However oneDNN could require scratchpad for some functionalities/platforms where cuDNN does not. Will we allow allocating scratchpad memory under dnncompat?
Other thing is that we might hit fundamental issues with training support. In particular, with respect to workspace management between forward and backward passes, as oneDNN requires workspace for some functionalities where cuDNN does not and vice-versa. If the compat API does not expose that to user, how would we handle it?
@zhimingwang36, can you please provide your input on how you handle oneDNN/cuDNN scratchpad and workspace?
Below are related questions from @mgouicem:
Other thing is about memory management. In cuDNN scratchpad is handled by user and can be queried for specific primitives. However oneDNN could require scratchpad for some functionalities/platforms where cuDNN does not. Will we allow allocating scratchpad memory under dnncompat?
Other thing is that we might hit fundamental issues with training support. In particular, with respect to workspace management between forward and backward passes, as oneDNN requires workspace for some functionalities where cuDNN does not and vice-versa. If the compat API does not expose that to user, how would we handle it?
I have added suggestions about handling workspace and scratchpad memory to the RFC. @zhimingwang36 / others please take a look.
@sgeor255 In SYCLomaitc, If some workspace and scratchpad memory needs in cuDNN, while not in oneDNN. Then SYCLomatic will replace the cuDNN query API call with 0. If some workspace and scratchpad memory needs in oneDNN, while not in cuDNN. These memory will be allocated in wrapper function.
@sgeor255 In SYCLomaitc, If some workspace and scratchpad memory needs in cuDNN, while not in oneDNN. Then SYCLomatic will replace the cuDNN query API call with 0. If some workspace and scratchpad memory needs in oneDNN, while not in cuDNN. These memory will be allocated in wrapper function.
@intwanghao as described in the RFC, scratchpad memory can be handled internally by dnncompat as an implementation detail. dnncompat is a replacement for the helper level functions and is at the same level as these helper function. The approach you mentioned can be seen as one way to handle this.