Double-count issue when using two reporters

[AndrewGnagy]

I'm attempting to use a custom reporter to report results to both junit and the pretty-printer.

What I've found is that this will double-count the number of assertions and failures. For example:

    Ran 5 tests in 6.059 seconds
    10 assertions, 2 failures, 2 errors.

(Should be 1 assertion per test for 5 total. 1 failure and 1 error.)

A work-around suggested by @miikka is binding clojure.test/*report-counters* to nil for one of the reports:

    (defn report [m]
      (pretty/report m)
      (binding [clojure.test/*report-counters* nil]
        (xml-report m)))

Which works.

Is there a more proper way of handling this?

[weavejester]

Because of the way clojure.test works, that might be the best solution. We could add in something that combines multiple reporters automatically, or potentially we could try and create a new reporter abstraction we could layer on top of clojure.test/report, but adding new abstractions is something I tend to be cautious about.

[Deraen]

I'm using following function in boot-alt-test to combine collection of reporters, should we add this to eftest.report?

(defn combined-reporter
  "Combines the reporters by running first one directly,
  and others with clojure.test/*report-counters* bound to nil."
  [report & rst]
  (fn [m]
    (report m)
    (doseq [report rst]
      (binding [clojure.test/*report-counters* nil]
        (report m)))))

[aviflax]

~FWIW, I just wrote my own version of such a function, and I’m not encountering the double counting.~

Update: Ah, never mind, I didn’t read the TP closely enough. I am indeed seeing the same issue with my function.

My function

(defn- multi-report
  "Accepts n reporting functions, returns a reporting function that will call
  them all for their side effects and return nil. I tried to just use juxt but
  it didn’t work. Maybe because some of the reporting functions provided by
  eftest are multimethods, I don’t know."
  [& fs]
  (fn [event]
    (doseq [f fs]
      (f event))))

[aviflax]

Ah, never mind, I didn’t read the TP closely enough. I am indeed seeing the same issue with my function.

[buzzdan]

i'm upvoting the need for a duel reporter --> i wanna be able to run lein eftest in my CI that produces a junit report while outputing to console in development. (or just use both)