mycroft-precise icon indicating copy to clipboard operation
mycroft-precise copied to clipboard

Empty metrics from precise-test

Open ljj7975 opened this issue 5 years ago • 5 comments

I have correctly set up the dataset but I am seeing 0s for the counts

what might be the cause of this?

Command : precise-test -t 0.85 trained_model.net dataset

Data: <TrainData wake_words=502 not_wake_words=24641 test_wake_words=54 test_not_wake_words=2504>
=== False Positives ===


=== False Negatives ===


=== Counts ===
False Positives: 0
True Negatives: 0
False Negatives: 0
True Positives: 0


=== Summary ===
0 out of 0
0.00%

0.00% false positives
0.00% false negatives

ljj7975 avatar Oct 18 '20 17:10 ljj7975

Hmm, strange. I don't think this is the case, but can you check if the test files are the correct format? They should be 16 bit integer mono wav files.

MatthewScholefield avatar Oct 18 '20 18:10 MatthewScholefield

yea they are .wav with monochannel 16000 sample rates I generated them with librosa write file and for some datasets I generated, it works fine. It just randomly happens that I couldn't figure out why so I was asking here.

As you see in the log, it correctly locate the files Data: <TrainData wake_words=502 not_wake_words=24641 test_wake_words=54 test_not_wake_words=2504>

ljj7975 avatar Oct 18 '20 20:10 ljj7975

Things still fail with the correct format

import soundfile as sf

data = (item.audio_data.numpy() * 32767).astype('int16')
sf.write(output_path, data, 16000, subtype='PCM_16', endian='LITTLE')

ljj7975 avatar Oct 19 '20 13:10 ljj7975

I have debugged the issue. pyache was getting into the way.

I think others fall into the same trap. is it possible for u to add support for disabling the cache?

in my cases files might have the same name but contents can be different

ljj7975 avatar Oct 20 '20 01:10 ljj7975

Ahh, makes sense. It shouldn't be too hard to disable the cache. Out of curiosity, are you working off the dev branch or off of the fork with tf2 support?

MatthewScholefield avatar Oct 20 '20 01:10 MatthewScholefield