nunit
Unspecified behavior of the Parallelizable Attribute
In relation to #241 that is closed and was related to the Implementation bug, i would suggest to update the https://github.com/nunit/docs/wiki/Parallelizable-Attribute page accordingly to clarify the expected behavior of parallel executed tests.
FMPOV: Its important to know when the fixture will be instantiated and for how long it lives.
If it's no longer in the docs I agree it should be added. However it isn't related to parallel execution because the timing is the same whether tests are run in parallel or not.
@JPVenson Definitely... if all or part of nunit/nunit#2594 were implemented, the docs would need to change.
@CharliePoole i think it is related as i have upgraded today Nunit from 3.6.1 to 3.11 and my Parallel tests have stopped working. I have read the Release notes before so i knew that test methods can now run also in parallel (this is why i have linked nunit/nunit#2574 because of jnm2's first sentence). His thought was mine too, that for each test a new instance was created. Now after rethinking that, its obvious that this doesn't make sense at all but i would have a coffee less needed if there would be an extra Paragraf in the Parallel documentation that clearly describes how NUnit behaves in the specific cases of Parallel execution within tests. Its more of a hint that saves future users some time for researching this problem.
But to complete that, i recommend there should be a whole overview explanation of the flow of every case in the Parallel Scenario.
There was in the past a complete explanation of all this, which is why I wrote "if it's no longer in the docs." This is a very fundamental aspect of NUnit that every user needs to understand, even if they never look at the parallel attribute, so if it's no longer there I'd like to see it reinstated. No problem if it's referenced or restated with the parallel attribute.
If one or more of the changes discussed is implemented, then both pieces need to be revised, of course.
That said, I have to go on record that I was shocked at jnm2's first sentence! When NUnit was created, it was a big controversy that the same instance was reused for all tests. It was discussed endlessly then and repeatedly over the years because it differs from how JUnit works. I was late to the party, only starting on the project in 2002, but it was pretty much the first thing I learned about NUnit internals myself.
So while I know some people disagree with it, I can't imagine how anyone is not aware of it... but maybe that missing documentation explains the reason. :smile:
Here's the minimum documentation for parallelizable attribute wrt "unspecified" behavior...
"Parallel fixtures and test cases work exactly the same as non-parallel tests, except where specified differently on this page." :smile:
If we wanted to be snarky, we could write... "If you imagine it should work differently, please file a feature request." :smiling_imp:
^^ i got your point but i have to say that i started using NUnit in my live just ... 3 or 4 years ago and i guess that true for, not most but a essential part of all developers. That's where we have might missed each other.
And this might be the question of how detailed the documentation should be in general as you have been shocked by his statement but for me it was a eureka moment that someone with more experience has made the same fault just as me ;-) .
That was just the thought that its better to explain it for everyone who was not part of this discussion. Does nobody think of the Children ... of Nunit? :-D
Yeah... it might be that the page in question is lost (no longer working on this project, I'm not sure). There was always a page in the past that explained in great detail the life cycle of fixtures and the flow of tests. I think we are in furious agreement that such documentation should exist and be maintained. If we keep discussing something, that tells us it needs to be in the docs.
Where we seem to disagree is around where this documentation should go. I simply don't want to mislead people to imagine that it works differently when tests are run in parallel, since it doesn't.
I totally agree that there Must be some kind of deep detailed documentation that covers all environmental effects and "chain of cause" when using it. I would even expand it further to not just fixtures.
I also agree to this that you have concerns about a misleading documentation. From my perspective and with the public interest in this specific topic i guess its absolutely justifiable to include a "explicit" section in maybe this wiki (https://github.com/nunit/docs/wiki/Framework-Parallel-Test-Execution) that the execution behavior does not change in a Parallel Environment compared to the execution of a single one. In case there is such extended documentation how it behaves "normally" it would just require a footnote ( and if someone is not reading it ... welp not my problem ^^)
Followed by an new, or old (depends on the age of the reader ;-) ) wiki that clearly describes the life cycle of fixtures and tests.