executorch icon indicating copy to clipboard operation
executorch copied to clipboard

Issue running XNN-pack unittests

Open freddan80 opened this issue 1 year ago • 0 comments

I get an runtime error when running XNN-pack unittests. Reproducer:

./install_requirements.sh --pybind or install_requirements.sh --pybind xnnpack

as per instructions in extension/pybindings/README.md.

pytest -c /dev/null -p no:warnings -s backends/xnnpack/test/models/mobilenet_v2.py -k test_qs8_mv2

Error message:

_________________________________________________________________________ TestMobileNetV2.test_qs8_mv2 __________________________________________________________________________

self = <mobilenet_v2.TestMobileNetV2 testMethod=test_qs8_mv2>

    def test_qs8_mv2(self):
        # Quantization fuses away batchnorm, so it is no longer in the graph
        ops_after_quantization = self.all_operators - {
            "executorch_exir_dialects_edge__ops_aten__native_batch_norm_legit_no_training_default",
        }
    
        dynamic_shapes = (
            {
                2: torch.export.Dim("height", min=224, max=455),
                3: torch.export.Dim("width", min=224, max=455),
            },
        )
    
        (
>           Tester(self.mv2, self.model_inputs, dynamic_shapes=dynamic_shapes)
            .quantize()
            .export()
            .to_edge()
            .check(list(ops_after_quantization))
            .partition()
            .check(["torch.ops.higher_order.executorch_call_delegate"])
            .check_not(list(ops_after_quantization))
            .to_executorch()
            .serialize()
            .run_method_and_compare_outputs(num_runs=10)
        )

backends/xnnpack/test/models/mobilenet_v2.py:66: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv/py310_et/lib/python3.10/site-packages/executorch/backends/xnnpack/test/tester/tester.py:566: in run_method_and_compare_outputs
    stage_output = self.stages[stage].run_artifact(inputs_to_run)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <executorch.backends.xnnpack.test.tester.tester.Serialize object at 0x16fddb7f0>
inputs = (tensor([[[[ 0.6119,  0.0162,  1.4876,  ...,  0.3651, -0.2745, -0.6686],
          [-0.8822, -1.8987,  0.2824,  ...,  ... 0.7997,  ..., -0.1194,  1.1729,  0.6180],
          [ 0.2851, -1.0034,  0.8266,  ..., -0.6135, -1.3600, -0.3300]]]]),)

    def run_artifact(self, inputs):
        inputs_flattened, _ = tree_flatten(inputs)
>       executorch_module = _load_for_executorch_from_buffer(self.buffer)
E       RuntimeError: loading method forward failed with error 0x20

.venv/py310_et/lib/python3.10/site-packages/executorch/backends/xnnpack/test/tester/tester.py:323: RuntimeError
------------------------------------------------------------------------------- Captured log call -------------------------------------------------------------------------------
WARNING  root:backend_api.py:373 Disabled validating the partitioner.
============================================================================ short test summary info ============================================================================
FAILED ../../../../dev/::TestMobileNetV2::test_qs8_mv2 - RuntimeError: loading method forward failed with error 0x20
======================================================================= 1 failed, 1 deselected in 17.67s ========================================================================

freddan80 avatar Apr 24 '24 16:04 freddan80