spleeter icon indicating copy to clipboard operation
spleeter copied to clipboard

How can I train spleeter based on pretrained_models

Open hsduren opened this issue 3 years ago • 3 comments

when I run the train command like 'spleeter train --verbose -p configs/musdb_config.json -d musdb18hq' ,and the musdb_config.json content like this : { "train_csv": "configs/musdb_train.csv", "validation_csv": "configs/musdb_validation.csv", "model_dir": "pretrained_models/2stems", "mix_name": "mix", "instrument_list": ["vocals", "other"], "sample_rate":44100, "frame_length":4096, "frame_step":1024, "T":512, "F":1024, "n_channels":2, "n_chunks_per_song":40, "separation_exponent":2, "mask_extension":"zeros", "learning_rate": 1e-4, "batch_size":4, "training_cache":"cache/training", "validation_cache":"cache/validation", "train_max_steps": 200000, "throttle_secs":1800, "random_seed":3, "save_checkpoints_steps":1000, "save_summary_steps":5, "model":{ "type":"unet.unet", "params":{ "conv_activation":"ELU", "deconv_activation":"ELU" } } } I got errors : 2 root error(s) found. (0) Not found: Key batch_normalization/beta/Adam not found in checkpoint [[node save/RestoreV2 (defined at /miniforge3/lib/python3.9/site-packages/tensorflow_estimator/python/estimator/estimator.py:1497) ]] [[save/RestoreV2/_301]] (1) Not found: Key batch_normalization/beta/Adam not found in checkpoint [[node save/RestoreV2 (defined at /miniforge3/lib/python3.9/site-packages/tensorflow_estimator/python/estimator/estimator.py:1497) ]] So,is there any way to train spleeter based on pretrained_models?

hsduren avatar May 05 '22 23:05 hsduren

I have got the same problem and can't find an answer. I can add that it works fine, when 'model_dir' is empty (training from scratch). Apparently this error comes up, when it tries to load the downloaded 2stems model.

Is there a certain tensorflow version that needs to be installed? Is it simply not possible?

Tomp0801 avatar Sep 20 '22 11:09 Tomp0801

I have no idea yet

hsduren avatar Sep 20 '22 12:09 hsduren

I found a solution! It's related to this issue: #408. The problem seems to be that it defaults to the AdamOptimizer, but apparently GradientDescentOptimizer was used for the pretrained model. Instead of changing the code like suggested there, I added "optimizer": "SGD" to the config file, which also worked.

Tomp0801 avatar Sep 21 '22 07:09 Tomp0801