appengine-mapreduce icon indicating copy to clipboard operation
appengine-mapreduce copied to clipboard

MapreducePipeline behavior is inconsistent

Open dalippa opened this issue 11 years ago • 1 comments

In the below snippet from mapreduce_pipeline.py (MapreducePipeline.run()), the mapper_params default to None, showing that they are not always required, but the first line of the constructor immediately de-references it.

  def run(self,
          job_name,
          mapper_spec,
          reducer_spec,
          input_reader_spec,
          output_writer_spec=None,
          mapper_params=None,
          reducer_params=None,
          shards=None,
          combiner_spec=None):
    # Check that you have a bucket_name set in the mapper_params and set it
    # to the default if not.
    if mapper_params.get("bucket_name") is None:
      try:
        mapper_params["bucket_name"] = (
            app_identity.get_default_gcs_bucket_name())

dalippa avatar Nov 18 '14 03:11 dalippa

Yeah, this should be fixed by simply making mapper_params an empty dict if it comes in as None.

AngryBrock avatar Nov 18 '14 20:11 AngryBrock