config reset

[hovnatan]

Sometimes the config file at ~/.config/pudb/pudb.cfg gets reset to the default value. Using pudb 2019.1. This occurs in multiprocessing run where I use export PYTHONBREAKPOINT=pudb.remote.set_trace

[inducer]

I haven't had that happen. Do you know of a way to reproduce this behavior?

[hovnatan]

Please run the attached code with export PYTHONBREAKPOINT=pudb.remote.set_trace. Then telnet into one of the connections. This resets the pudb.cfg

from torch.utils.data import DataLoader, Dataset

class MyDataset(Dataset):
    def __getitem__(self, i):
        breakpoint()
        return i

    def __len__(self):
        return 1000000


mydata = MyDataset()
myloader = DataLoader(mydata, batch_size=64, num_workers=16)

it = iter(myloader)
item = next(it)
print(item)

[inducer]

I tried with a different file (I don't have PyTorch installed), and my configuration survived without a problem.

[inducer]

Maybe PyTorch is doing something that it shouldn't?

[asmeurer]

Right now save_config swallows all exceptions https://github.com/inducer/pudb/blob/3f041b88b7cf623a22d50e702663b1e3e8076f3c/pudb/settings.py#L150. Can you try removing the except Exception and seeing if an exception occurs. Ditto with load_config.

[asmeurer]

My guess is that load_config fails on loading the file, but doesn't do anything because of an except Exception: pass. So it sets everything as the default, then immediately saves it: https://github.com/inducer/pudb/blob/3f041b88b7cf623a22d50e702663b1e3e8076f3c/pudb/debugger.py#L44-L45

[hovnatan]

This is what I get if I comment out except Exception: ...

> /home/hovnatan/bug/temp.py(7)__getitem__()
-> return i
(Pdb) > /home/hovnatan/bug/temp.py(7)__getitem__()
-> return i
> /home/hovnatan/bug/temp.py(7)__getitem__()
-> return i

(Pdb) (Pdb)

> /home/hovnatan/bug/temp.py(7)__getitem__()
-> return i
(Pdb)
Traceback (most recent call last):
  File "temp.py", line 17, in <module>
    item = next(it)
  File "/home/hovnatan/miniconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 582, in __next__
> /home/hovnatan/bug/temp.py(7)__getitem__()
-> return i
(Pdb)
    return self._process_next_batch(batch)
  File "/home/hovnatan/miniconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch
> /home/hovnatan/bug/temp.py(7)__getitem__()
-> return i
(Pdb)     raise batch.exc_type(batch.exc_msg)

bdb.BdbQuit: Traceback (most recent call last):
  File "/home/hovnatan/miniconda3/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/home/hovnatan/miniconda3/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 99, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "temp.py", line 7, in __getitem__
    return i
  File "temp.py", line 7, in __getitem__
    return i
  File "/home/hovnatan/miniconda3/lib/python3.7/bdb.py", line 88, in trace_dispatch
    return self.dispatch_line(frame)
  File "/home/hovnatan/miniconda3/lib/python3.7/bdb.py", line 113, in dispatch_line
    if self.quitting: raise BdbQuit
bdb.BdbQuit

[hovnatan]

Oh no sorry, this is what I get with the regular pdb, I forgot to set the environment variable.

[hovnatan]

This is the exception:

Source contains parsing errors: '/home/hovnatan/.config/pudb/pudb.cfg'
        [line 21]: 'e\n'

[asmeurer]

Can you paste your pudb.cfg here?

[hovnatan]

I fixed the pudb.cfg this was not the original problem. Then I tried modifying breakpoint() to if i == 0: breakpoint() and it worked fine. So I guess multiple simultaneous breakpoint() are the issue. I will try to debug more later.

[hovnatan]

Also I get half, both vertically and horizontally, of the screen used without apparent reason:

[hovnatan]

Ignore my last comment, it was due the tee redirection. But the original issue stands.

[asmeurer]

If you can paste the pudb.cfg here that has the e\n on line 21 that would help to figure out what is causing this. In the debugger code, the config is only loaded once at the top of the module, and only saved after resizing a panel, closing the prefs screen, or closing the welcome screen. It isn't touched when a breakpoint happens, as far as I can see.

[hovnatan]

Here it is, but I don't think the problem is in the config. The best way to reproduce the problem is to install pytorch and run the code from my second comment in this feed.

breakpoints_weight = 1
current_stack_frame = top
custom_shell = 
custom_stringifier = 
custom_theme = 
default_variables_access_level = public
display = auto
jk_sidebar_scroll = False
line_numbers = False
prompt_on_quit = True
seen_welcome = e035
shell = internal
sidebar_width = 0.5
stack_weight = 1
stringifier = type
theme = classic
variables_weight = 1
wrap_variables = True

[mvanderkamp]

I encounter this issue from time to time as well. I installed my local fork of pudb using pip install --editable and added a traceback.print_exc() line after each except Exception (which might actually be a good idea to put in properly going forward, perhaps with some notation to warn users that this is an internal exception and to please post about it here along with some relevant info like the version, system, python version, and so on).

I finally hit it again today, and got several of these traceback messages:

Traceback (most recent call last):
  File "pudb/settings.py", line 123, in normalize_bool_inplace
    if conf_dict[name].lower() in ["0", "false", "off"]:
AttributeError: 'bool' object has no attribute 'lower'

[asmeurer]

I think a good first step here would be to actually log the exceptions when loading the config, rather than swallowing them all. If there are other except Exceptions in PuDB we should do the same there.

[asmeurer]

Also if configparser has a set of known exceptions it raises when the ini file has errors, we should raise those distinctly ("error loading config" vs. "potential bug in pudb").