easydata icon indicating copy to clipboard operation
easydata copied to clipboard

Error if no process function

Open acwooding opened this issue 5 years ago • 0 comments

make data and make sources both end in an error if there is no process function:


python3 -m src.data.make_dataset process
2020-03-21 12:35:54,219 - datasets - INFO - Running process on covid_nlp_20200319
2020-03-21 12:36:16,826 - fetch - WARNING - Can't infer `unpack_action` from filename all_sources_metadata_2020-03-13.csv. Defaulting to 'copy'.
2020-03-21 12:36:16,935 - fetch - WARNING - Can't infer `unpack_action` from filename all_sources_metadata_2020-03-13.readme. Defaulting to 'copy'.
Traceback (most recent call last):
  File "/Users/amywooding/miniconda3/envs/covid_nlp/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/amywooding/miniconda3/envs/covid_nlp/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/amywooding/easydata/covid_nlp/src/data/make_dataset.py", line 31, in <module>
    main()
  File "/Users/amywooding/miniconda3/envs/covid_nlp/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/amywooding/miniconda3/envs/covid_nlp/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/amywooding/miniconda3/envs/covid_nlp/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/amywooding/miniconda3/envs/covid_nlp/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/amywooding/easydata/covid_nlp/src/data/make_dataset.py", line 21, in main
    process_datasources(datasources=datasources, action=action)
  File "/Users/amywooding/easydata/covid_nlp/src/data/datasets.py", line 78, in process_datasources
    logger.info(f'{dataset_name}: processed data has shape:{ds.data.shape}')
AttributeError: 'NoneType' object has no attribute 'shape'
make: *** [.make.process_sources] Error 1

acwooding avatar Mar 21 '20 18:03 acwooding