How was Dask slow?

[mrocklin]

In the dask_generator.py file you have the following comment:

This file is named after dask for historical reasons. We first tried to use dask to coordinate the hdf5 buckets but it was slow and we wrote our own stuff.

I'm curious, how were you trying to use Dask and what performance limitations did you run into?

Feel free to close this issue immediately. This is just a question of curiosity.

[EderSantana]

you can concatenate several hdf5 buckets with dask and have a unified numpy like interface. That was pretty cool. But slicing was slower than what we did here.

[ahmedyahia3393]

in the view steering model.py file I found his error (ValueError: bad marshal data (unknown type code)) result when trying to execute the view steering model.py here is the result from the cmd prompt

Traceback (most recent call last): File "C:\Users\lenovo\Anaconda3\lib\site-packages\keras\utils\generic_utils.py ", line 229, in func_load raw_code = codecs.decode(code.encode('ascii'), 'base64') UnicodeEncodeError: 'ascii' codec can't encode character '\xe0' in position 46: ordinal not in range(128)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "view_steering_model.py", line 94, in model = model_from_json(json.load(jfile)) File "C:\Users\lenovo\Anaconda3\lib\site-packages\keras\models.py", line 349, in model_from_json return layer_module.deserialize(config, custom_objects=custom_objects) File "C:\Users\lenovo\Anaconda3\lib\site-packages\keras\layers_init_.py", l ine 55, in deserialize printable_module_name='layer') File "C:\Users\lenovo\Anaconda3\lib\site-packages\keras\utils\generic_utils.py ", line 144, in deserialize_keras_object list(custom_objects.items()))) File "C:\Users\lenovo\Anaconda3\lib\site-packages\keras\models.py", line 1349, in from_config layer = layer_module.deserialize(conf, custom_objects=custom_objects) File "C:\Users\lenovo\Anaconda3\lib\site-packages\keras\layers_init_.py", l ine 55, in deserialize printable_module_name='layer') File "C:\Users\lenovo\Anaconda3\lib\site-packages\keras\utils\generic_utils.py ", line 144, in deserialize_keras_object list(custom_objects.items()))) File "C:\Users\lenovo\Anaconda3\lib\site-packages\keras\layers\core.py", line 711, in from_config function = func_load(config['function'], globs=globs) File "C:\Users\lenovo\Anaconda3\lib\site-packages\keras\utils\generic_utils.py ", line 234, in func_load code = marshal.loads(raw_code) ValueError: bad marshal data (unknown type code)