The leftover arguments are empty in Click 6.0 Context

[ivankravets]

According to documentation args were accessed until Click < 6.

args = None the leftover arguments.

Is this API removed in 6.0?

[mitsuhiko]

In which situation?

[ivankravets]

Here ctx.args is equal to [] for any commands. For example, platformio serialports monitor --baud 115200.

[mitsuhiko]

So that you got args for children there was a bug. Do you need that? nargs > 1 still captures that.

[ivankravets]

So that you got args for children there was a bug

Exactly, I need args for children.

[ivankravets]

For example,

platformio -f -c eclipse run ...`

I just need run .... Do you have any ideas?

[ivankravets]

Do you need that? nargs > 1 still captures that.

@mitsuhiko don't understand what I should change. nargs option applies to arguments "type". However, the question was how to receive leftover arguments(including options, like valid portion of sys.args) from context using MultiCommand approach.

P.S: I have blocked issue for it https://github.com/platformio/platformio/issues/349 and reduced temporary Click to <6.

Thanks in advance!

[mitsuhiko]

Are you looking for this maybe? http://click.pocoo.org/5/advanced/#forwarding-unknown-options

[ivankravets]

Test code

@click.command(cls=PlatformioCLI,
               context_settings=dict(
                   help_option_names=["-h", "--help"],
                   ignore_unknown_options=True
               ))
@click.version_option(__version__, prog_name="PlatformIO")
@click.option("--force", "-f", is_flag=True,
              help="Force to accept any confirmation prompts.")
@click.option("--caller", "-c", help="Caller ID (service).")
@click.pass_context
def cli(ctx, force, caller):
    print ctx.args   

Click < 6

$ platformio -f -c test  list
['list']
....

$ platformio -f -c test run -h
['run', '-h']
...

Click 6

$ platformio -f -c test run -h
[]

$ platformio run --target upload
[]
...

What is wrong in this code? Thanks.

[mitsuhiko]

I will have a look at that.

[orome]

I'm getting downgrade to click 5.1 (by 2.7.1).

[ivankravets]

@orome It's ok. @PlatformIO requires some functionality which doesn't work in Click 6. See related issue https://github.com/platformio/platformio/issues/349

[mitsuhiko]

Just to leave some comments on this: I would like to somehow undo parts of Click 6 because it just does not work that way. Not entirely sure what the course of action is :(

[ivankravets]

@mitsuhiko do you have any news?

[mitsuhiko]

Not yet, for now please use Click 5 if you need this.

[paxnovem]

I have been trying to reproduce this issue, I have attempted to use Click 5.0, 6.0, 7.0 on both Python 2.7, Python 3.6. and I am getting consistent behavior between versions.

The following code is what I was using to reproduce the issue. If someone is experiencing this, please let me know how to modify this example to hit the broken behavior.

import click

@click.command(context_settings=dict(
                ignore_unknown_options=True,
                allow_extra_args=True
            ))
@click.option("--force", "-f", is_flag=True,
            help="Force to accept any confirmation prompts.")
@click.option("--caller", "-c")
@click.pass_context
def cmd(ctx, force, caller):
        print ctx.args

if __name__ == '__main__':
        cmd()

[jcrotts]

Thanks @paxnovem the above example seems to be working as expected in Click 7 with python 3.7 as well. Please reopen if you are still experiencing the issue.

[ivankravets]

The bug still persists. You don't have access to leftover arguments from any place/subcommands. See updated example to reproduce this issue:

test.py

import click


@click.group(context_settings=dict(ignore_unknown_options=True,
                                   allow_extra_args=True))
@click.option("--force",
              "-f",
              is_flag=True,
              help="Force to accept any confirmation prompts.")
@click.option("--caller", "-c")
@click.pass_context
def cli(ctx, force, caller):
    print(ctx.args)
    # do some preprocessing with leftover args... Do we have `--json-output`? etc.


@cli.command()
def run():
    pass


if __name__ == '__main__':
    cli()

Click 5.0

test.py -f -c foo run

['run']

Click 7.0

test.py -f -c foo run

[]

[jcrotts]

Quite right the two use cases above are slightly different.

I think this issue targets your use case. https://github.com/pallets/click/issues/347

[ivankravets]

Can you reopen it?

[jcrotts]

Sure, have you been able to find a work around for the behavior you want?

[ivankravets]

@jcrotts Thanks for reopening! The only solution is to parse manually sys.argv which loses sense in using click at all.

Is it difficult to fix it? The issue arose with Click 6.0 release 4 years ago.

[jcrotts]

This seems to be the commit that changed this behavior. https://github.com/pallets/click/commit/0a2919f34fcbc635d8530b4c5b60bf119b2bcedb

[ivankravets]

You are right, thanks! I removed these lines https://github.com/pallets/click/blob/master/click/core.py#L1122:L1123 and everything works now with Click 7.0.

@mitsuhiko can we remove these L1122:L1123 lines which reset context args?

[jcrotts]

So actually just removing the 2nd line seems to revert the behavior, and then changing the print to print(ctx.protected_args).

Tests pass after the change, but I think this could be overriding a design decision that I'm not fully aware of.

[ivankravets]

I find some workaround. We can actually overload command.invoke() and backups args:

class PlatformioCLI(click.MultiCommand):

    def invoke(self, ctx):
        ctx._args = ctx.args
        if hasattr(ctx, "protected_args"):
            ctx._args = ctx.protected_args + ctx.args
        return super(PlatformioCLI, self).invoke(ctx)

In this case, args will be in ctx._args. I may work in our case. The problem can occur when you need to get leftover arguments in subcommand. In this case, they will be empty.

So, what to do with this issue?

[ivankravets]

http://click.palletsprojects.com/en/7.x/api/#click.Context.args

args = None the leftover arguments.

It looks like a critical bug.

Another workaround could be ctx.leftover_args and set it to ctx.protected_args + ctx.args before reseting.

[jcrotts]

Seems related to https://github.com/pallets/click/issues/1323

[davidism]

In the first example https://github.com/pallets/click/issues/473#issuecomment-162561642, allow_extra_args is missing, and the top-level is a command, not a group, so extra arguments would be treated as errors.

In the second example https://github.com/pallets/click/issues/473#issuecomment-490260596, a Group is used and allow_extra_args is set (unneeded in this case, group requires this and sets it automatically). However, "run" is not an extra argument, it's the subcommand name.

Use click.argument("extra", nargs=-1) to explicitly capture the arguments. While it's not clearly documented this way, allow_extra_args is to allow the parser to handle nested subcommand parsing, not for user code to capture extra args. For the same reason that a group automatically sets allow_extra_args, do not use a nargs=-1 argument on a group, as it would never be able to dispatch to a command then since the command name and args would be considered "extra" to the parent parser.

• allow_extra_args is a parser flag for supporting subcommands on groups. It is already set correctly for these internal situations.
• Similarly, ctx.args and ctx.protected_args are also for internal tracking for dispatching subcommands. Command callbacks shouldn't be looking at them.
• To capture extra arguments for direct use, use argument(nargs=-1) on a command. The command callback will then receive an extra argument with the list of captured values.

I acknowledge that this isn't clear from the documentation. As part of rewriting the parser, I've been running across quite a bit of behavior that should be considered internal, or that needs much clearer documentation.