espnet icon indicating copy to clipboard operation
espnet copied to clipboard

Add implementations of USES2 speech enhancement models

Open Emrys365 opened this issue 1 year ago • 4 comments

What?

This PR adds the implementations of the USES2-Swin and USES2-Comp speech enhancement models proposed in the ICASSP 2024 paper "Improving Design of Input Condition Invariant Speech Enhancement".

Why?

This is an upgraded version of the previously added USES model (https://github.com/espnet/espnet/pull/5482).

See also

https://arxiv.org/abs/2401.14271

Emrys365 avatar Apr 24 '24 11:04 Emrys365

I understand that you tried to separate this PR and the recipe PR https://github.com/espnet/espnet/pull/5810 If you fix the CI error, please let me know.

@LiChenda, can you also review this PR?

sw005320 avatar Jun 10 '24 11:06 sw005320

I understand that you tried to separate this PR and the recipe PR #5810 If you fix the CI error, please let me know.

@LiChenda, can you also review this PR?

Yes!

LiChenda avatar Jun 11 '24 02:06 LiChenda

I can't understand how the modified scripts can cause an error when installing espnet in the CI tests...

Emrys365 avatar Jun 17 '24 16:06 Emrys365

Codecov Report

Attention: Patch coverage is 17.79661% with 388 lines in your changes missing coverage. Please review.

Project coverage is 27.87%. Comparing base (1a0c358) to head (bfd3000). Report is 30 commits behind head on master.

Files Patch % Lines
espnet2/enh/layers/swin_transformer.py 16.02% 152 Missing :warning:
espnet2/enh/layers/uses2_comp.py 17.07% 102 Missing :warning:
espnet2/enh/layers/uses2_swin.py 21.27% 74 Missing :warning:
espnet2/enh/separator/uses2_separator.py 17.80% 60 Missing :warning:

:exclamation: There is a different number of reports uploaded between BASE (1a0c358) and HEAD (bfd3000). Click for more details.

HEAD has 8 uploads less than BASE
Flag BASE (1a0c358) HEAD (bfd3000)
test_utils 2 0
test_python_espnet1 2 0
test_integration_espnetez 5 1
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5761      +/-   ##
==========================================
- Coverage   34.39%   27.87%   -6.52%     
==========================================
  Files         780      551     -229     
  Lines       71754    47203   -24551     
==========================================
- Hits        24677    13160   -11517     
+ Misses      47077    34043   -13034     
Flag Coverage Δ
test_integration_espnetez 27.87% <17.79%> (-0.32%) :arrow_down:
test_python_espnet1 ?
test_utils ?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jul 14 '24 19:07 codecov[bot]