pynwb icon indicating copy to clipboard operation
pynwb copied to clipboard

Upgrade validation methods - make pynwb.validate behave similarly for io and paths

Open stephprince opened this issue 1 year ago • 1 comments

Motivation

Address #1807 (related to other validation upgrades described in #1808). This is a breaking change for the next major release.

The expected behavior is for validate to return the same output whether paths or io is provided. This was previously discussed here, where the 'correct' behavior for the example below is to return: "The namespace 'core' is included by the namespace 'ndx-testextension'. Please validate against that namespace instead".

How to test the behavior?

import h5py
from pynwb.validate import validate
from pynwb import NWBHDF5IO

path = 'tests/back_compat/2.1.0_nwbfile_with_extension.nwb'
io = NWBHDF5IO('tests/back_compat/2.1.0_nwbfile_with_extension.nwb')

print('Validating all with paths')
validate(paths=[path], namespace="core", verbose=True)

print('Validating core with io')
validate(io=io, namespace="core", verbose=True)

Checklist

  • [ ] Did you update CHANGELOG.md with your changes?
  • [ ] Have you checked our Contributing document?
  • [ ] Have you ensured the PR clearly describes the problem and the solution?
  • [ ] Is your contribution compliant with our coding style? This can be checked running flake8 from the source directory.
  • [ ] Have you checked to ensure that there aren't other open Pull Requests for the same change?
  • [ ] Have you included the relevant issue number using "Fix #XXX" notation where XXX is the issue number? By including "Fix #XXX" you allow GitHub to close issue #XXX when the PR is merged.

stephprince avatar May 29 '24 22:05 stephprince

Codecov Report

Attention: Patch coverage is 96.77419% with 4 lines in your changes missing coverage. Please review.

Project coverage is 92.69%. Comparing base (e47cd5a) to head (146d0e1).

Files with missing lines Patch % Lines
src/pynwb/__init__.py 78.57% 2 Missing and 1 partial :warning:
src/pynwb/validation.py 98.55% 0 Missing and 1 partial :warning:
Additional details and impacted files
@@                Coverage Diff                @@
##           release-3.0.0    #1911      +/-   ##
=================================================
- Coverage          92.69%   92.69%   -0.01%     
=================================================
  Files                 27       28       +1     
  Lines               2684     2710      +26     
  Branches             706      709       +3     
=================================================
+ Hits                2488     2512      +24     
- Misses               127      128       +1     
- Partials              69       70       +1     
Flag Coverage Δ
integration 73.14% <96.77%> (+0.26%) :arrow_up:
unit 83.63% <44.35%> (-0.45%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar May 29 '24 22:05 codecov[bot]

@rly this should be ready for review

There are a couple remaining TODO items related to validation support for NWBZarr files using the file path that could potentially be bumped to a separate PR:

  1. I started to try to implement support for NWBZarr file validation by allowing the ZarrIO.load_namespaces method to accept a ZarrIO.file object as input, but ran into this issue - https://github.com/hdmf-dev/hdmf-zarr/issues/243.
  2. We would also want to either rename the ZarrIO.file or HDF5IO._file property for consistent naming across backends: https://github.com/NeurodataWithoutBorders/pynwb/blob/11cc2328130f9420359282cee4657536415a449b/src/pynwb/validation.py#L58-L61

stephprince avatar Dec 17 '24 00:12 stephprince