restructure testing/checking directories

[mhmerrill]

We should a have a breif discussion of why the current state is confusing, and restructure the test directories.

We currently have several kinds of tests:

1. Chapel unit tests, so you don't have to compile the whole server to test a specific module... 1min compile vs. a 10min+ compile.
2. Python client/server tests to test a fully built arkouda server and a python client.

Questions to consider: Directory structure CI testing PYPI Mason Chapel Unit Testing make check vs. something like make test, or even make unittest

[reuster986]

I propose we restructre the test directories as

test/
    server/
    modules/
    diagnostic/

The server/ and modules/ dirs would be meant for CI-compatible tests that return a success/fail code for automated flows. The diagnostic/ dir would be for tests that a human runs and examines the output manually. Thus, most of the current tests/ dir would move to test/server, and most of the current test/ dir would move to test/modules, with a few from each going to test/diagnostic.

I further propose we setup make check to run everything in test/server. All the server tests complete relatively quickly on a single locale or multi-locale XC, although they can take minutes on a simulated multi-locale build on a laptop. I think the makefile already has logic for building individual Chapel unit tests (or all of them) using make test, so we could leave that convention in place and just update the target directory. My two cents.

[reuster986]

I forgot to explicitly state that the test/server dir would be for python tests that exercise the server, while the test/modules dir would be for Chapel unit tests that do not depend on having an arkouda_server built.

Also, should we consider moving benchmarks/ to test/benchmarks/?

[mhmerrill]

@Bears-R-Us/arkouda-core-dev @ronawho @bmcdonald3 what do you guys think, should we implement this? or should we close it?

[bmcdonald3]

This seems like a fairly simple and straightforward change to me that would make things a bit easier to understand, so I would be in favor of making this change, as I often struggle with the autocomplete of test vs tests. I could plan on implementing this in the near future if that sounds good with everyone. It seems to me like a good thing to get sorted out before the arkouda-contrib repo starts seeing a lot of use.