Merlin icon indicating copy to clipboard operation
Merlin copied to clipboard

[Task] Set up performance metrics from integration tests for Merlin example notebooks

Open karlhigley opened this issue 3 years ago • 2 comments

Problem:

We want customers to be confident that Merlin won't introduce major performance regressions on release of a new version.

Goal:

  • Capture performance metrics for our libraries in a way where we can review them easily as a part of our CI and release process.

Constraints:

  • ASVDB is our current place for capturing.

Starting Point:

Merlin Models

  • [x] #235
  • [x] #238

Merlin Systems

  • [x] #236
  • [x] #239

Merlin

  • [x] #237
  • [x] #240

karlhigley avatar May 27 '22 18:05 karlhigley

@EvenOldridge , who is the best person to provide the details for this ticket ?

viswa-nvidia avatar Jun 29 '22 22:06 viswa-nvidia

This is blocked on the following ticket: https://github.com/NVIDIA-Merlin/Merlin/issues/343 we need to refactor the way we leverage asvdb to accommodate for testbooks and non-notebook integration tests(<-- not seen before).

jperez999 avatar Jul 05 '22 15:07 jperez999