torchx icon indicating copy to clipboard operation
torchx copied to clipboard

[torchx][components] Enhance integ components test with automatic testing

Open aivanou opened this issue 4 years ago • 0 comments

Description

The feature changes the way we perform components integ tests. Instead of manually adding ComponentsProvider for each builtin, we find all components available in the code and test them.

Detailed Proposal

Today torchx implements components testing as integ tests. For each builtin component, one needs to create a class using ComponentsProvider interface. Without the corresponding class, the builtin component will not be tested.

The proposition is to change this behavior. The component can be one of two types: 1. function with all default parameters, 2. function with default and required parameters.

Components of type#1 can be automatically instantiated and tested on different schedulers. Components #2 are more tricky. It is possible for component to be complicated: e.g. CopyComponent needs existing file for testing, and when the job is done, it would leave a new file in the system. The components of type#2 can be wrapped with special decorator:

@torchx.test(provider=CopyUtilProvider)
def copy(..):
    return AppDef(..)

class CopyUtilProvider(ComponentProvider):
    def setUp(self) -> None:
        # invoked for before component is instantiated

    def tearDown(self) -> None:
      # invoked component after test finishes

    def get_app_def(self) -> specs.AppDef:
        return copy(
            src=self._src_path, dst=self._dst_path, image=self._image
        )

In the code above, we can associate component with component provider. ComponentProvider is interface with the following methods: tearDown, setUp, get_app_def.

aivanou avatar Nov 03 '21 01:11 aivanou