scalding icon indicating copy to clipboard operation
scalding copied to clipboard

Update template partition fields to format validation to a compile time macro

Open piyushnarang opened this issue 9 years ago • 3 comments

One of the suggestions that came up in PR: https://github.com/twitter/scalding/pull/1590 was to update the validation check in TemplatePartition to a compile time macro:

  assert(
    partitionFields.size == "%s".r.findAllIn(template).length,
    "Number of partition fields %s does not correspond to template (%s)".format(partitionFields, template))

One of the potential options might look something like: https://gist.github.com/xuwei-k/4991805 I think we'll also need to extend this to handle template strings that are passed as variables.

piyushnarang avatar Sep 16 '16 17:09 piyushnarang

Actually, I am not sure how to do this now... We can look at a literal string in the code, but getting the size of the partitionFields at compile time would be a pretty big pain.

This is not quite as easy as I thought.

johnynek avatar Sep 16 '16 17:09 johnynek

@johnynek yeah, even the string part will be easy if its a literal. Might not too do able if is a variable right.

piyushnarang avatar Sep 16 '16 17:09 piyushnarang

definitely won't work with a variable.

johnynek avatar Sep 16 '16 18:09 johnynek