OneTrainer icon indicating copy to clipboard operation
OneTrainer copied to clipboard

[Bug]: stop training if sample json exception

Open oO0 opened this issue 1 year ago • 7 comments

What happened?

  1. Train something
  2. Time to make sample but if sample json is absent or corrupted it stop training with exception. Not good.

What did you expect would happen?

Better to catch exception and continue process

Relevant log output

No response

Output of pip freeze

No response

oO0 avatar Jun 09 '24 11:06 oO0

Can you add your console log?

Nerogar avatar Jun 09 '24 12:06 Nerogar

File "...\OneTrainer\modules\ui\TrainUI.py", line 518, in __training_thread_function trainer.train() File "...\OneTrainer\modules\trainer\GenericTrainer.py", line 554, in train self.__execute_sample_during_training() File "....\OneTrainer\modules\trainer\GenericTrainer.py", line 190, in __execute_sample_during_training fun() File "....\OneTrainer\modules\trainer\GenericTrainer.py", line 537, in lambda: self.__sample_during_training(train_progress, train_device) File "....\OneTrainer\modules\trainer\GenericTrainer.py", line 274, in __sample_during_training with open(self.config.sample_definition_file_name, 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: 'training_samples/NewConfigBofereAddingSamples.json'

oO0 avatar Jun 11 '24 17:06 oO0

If no samples .json exists when you press "Start Training", a "samples.json" is created, even if you have not created any samples in the UI.

This is probably an obsolete issue that has been fixed in the meantime. If you disagree, please reply

dxqb avatar Feb 15 '25 12:02 dxqb

No reply

dxqb avatar Mar 30 '25 07:03 dxqb

No reply

Already self assigned, its in the dataset tools rework. (Made it fail gracefully if for whatever reason the samples file is deleted)

O-J1 avatar Mar 30 '25 07:03 O-J1

ok, sorry. I thought we don't understand the issue because no reply. reopened.

dxqb avatar Mar 30 '25 07:03 dxqb

ok, sorry. I thought we don't understand the issue because no reply. reopened.

Np, yeah he never clarified but at least made my best guess and tried to stop those scenarios.

O-J1 avatar Mar 30 '25 07:03 O-J1