Few troubles running tests.py

[bmusq]

Hi,

It's my very first time contributing to some else's project by submitting an issue so I hope I can make things right.

I have tried to run the tests.py file to make sure the installation unfold correctly on my environnement. It's a mere virtual env running in python 3.10 on Windows 10, though I mention it, it didn't cause any problems.

Hereunder the issues I have faced:

1. There seems to be a typo error in tests.py in func test_generate_data_with_wrong_name_format() line ~903. The name format is set to 3 which is not a valid name format. Based on the expected result folder I guess it is supposed to be 0

2. Trying to run the test test_personalfont_directory_unlocated() result in an FileNotFoundError because there is no guard in the run.py file to prevent from trying to open non existent directory. I suggest in run.py, in the bloc " Create font (path) list " replacing the first condition content with:

    if args.font_dir:
        if os.path.isdir(args.font_dir):
            fonts = [
                os.path.join(args.font_dir, p)
                for p in os.listdir(args.font_dir)
                if os.path.splitext(p)[1] == ".ttf"
            ]
        else:
            sys.exit(f"Cannot open font dir '{args.font_dir}'")

Note: in the sys.exit() I added an information about the actual parameter that causes an issue {args.font_dir}. I did the same for the other sys.exit() call in run.py

3. Though the main problem is that whenever a test from CommandLineInterface fails, triggering an assertion, the following line, namely empty_directory("tests/out/"), is not executed, at least on my computer. So the actual tests/out directory is not cleared and every following tests that consist in reading the number of files in this directory fail as well (that concerns all the CommandLineInterface tests). Moreover, there is no way to know whether a test itself has not terminated correctly.

Here's my suggestion to solve problem 3: Change these 3 lines for the concerned tests:

subprocess.Popen(args, cwd="trdg/").wait()
self.assertTrue(____)
empty_directory('tests/out/')

with:

try:
    p = subprocess.run(args, cwd="trdg/", stdout=subprocess.PIPE, encoding="utf-8")
    self.assertTrue(len(os.listdir("tests/out/")) == 0)
    empty_directory("tests/out/")
except AssertionError:
    empty_directory("tests/out/")
    raise
except Exception as e:
    empty_directory("tests/out/")
    print(f"{sys._getframe(0).f_code.co_name} execution failed. [Exeption] {e}")

• Using subprocess.run instead of subprocess.Popen is more reliable and the call to wait() is included.
• I have added a capture of the standard output in case you would need to verify it for a test.
• except AssertionError allows to catch whenever the test failed while the except Exception as e allows to catch any expcetions occuring with the call to subprocess.run which can be common with subprocessing. Though we do not catch SystemExit kind of exception (with return code=1) which is the expected behavior since a call to system exit means we have anticipated a problem and decided on purpose to terminate the process.
• Last but not least, the call to empty_directory() is included at every stage to ensure the tests/out directory is cleaned for every following tests regardless if the current one fails or not.

On the console the behavior and the display stay the same as the original one.

Also, I highly recommend cleaning tests/out and deleting tests/out_2 before running any tests to make sure they proceed correctly. You can add the following lines right under the declaration of class CommandLineInterface(unittest.TestCase):

empty_directory("tests/out/")
if os.path.exists("tests/out_2"):
    empty_directory("tests/out_2")
    os.rmdir("tests/out_2") 

Finnally, the expected_results folder cannot be reproduce since the TEST TEST.png kind of files are save but immediatly os.removed