_recorder doesn't record when there's an exception

[olivierdalang]

Describe the bug

When decorating a function with recorder, nothing gets recorded if an exception is thrown in the function.

Additional context

No response

Version of responses

0.24.1

Steps to Reproduce

class MyTestCase(TestCase):

    @_recorder.record(file_path="a.yaml")
    def test_a(self):
        requests.get("https://example.com", verify=False)

    @_recorder.record(file_path="b.yaml")
    def test_b(self):
        requests.get("https://example.com", verify=False)
        self.assertEqual(1, 2) # throws an exception

Expected Result

Requests should be recorded in both tests (files a.yaml and b.yaml should be created)

Actual Result

Requests are only recorded in a (file b.yaml is NOT created)

[beliaev-maksim]

@olivierdalang the idea behind the recorder was that you create a YAML for the passing test. What would be the actual use case to record something that should fail as a test ?

[olivierdalang]

The failure may be unrelated to the request, in which case recording it can still make sense (and could for instance allow for faster iterations in solving the failure).

In my use case, I was recording an upstream production server's response to capture changes, then adapt the recorded response manually (to keep only items relevant to my test).

[beliaev-maksim]

@olivierdalang if you catch the error in your code, will the recorder still fail? I assume it will record then up to the moment when the code failed

[olivierdalang]

if you catch the error in your code, will the recorder still fail?

If the error is catched then there's no unhandled exception, so I think the recorder will work, but the issue is specifically about what happens when there is an exception.

I assume it will record then up to the moment when the code failed

I assumed that too, but it seems that if the function fails, nothing gets recorded at all (see the steps to reproduce in the original report).

Note that there may be some misunderstanding on my side as to how this tool is supposed to be used. In #696 I suggested to document a bit more how it is meant to be used (record, then activate to run the test offline, later on update the recorded results, etc.).

[beliaev-maksim]

@olivierdalang

here is the example of what I mean

as you can see, you will get a record of c.yaml since we allow decorator to gracefully exit. That represents the real production example that you mentioned

from unittest import TestCase

import requests

from responses import _recorder

def actual_function_that_fails():
    requests.get("http://example.com", verify=False)
    raise AssertionError("Error in assertion")

class MyTestCase(TestCase):



    @_recorder.record(file_path="a.yaml")
    def test_a(self):
        requests.get("http://example.com", verify=False)

    @_recorder.record(file_path="b.yaml")
    def test_b(self):
        actual_function_that_fails()

    @_recorder.record(file_path="c.yaml")
    def test_c(self):
        try:
            actual_function_that_fails()
        except:
            pass