jgo
jgo copied to clipboard
scijava
Shortcuts expansion can fail with Python version
Maybe an issue with shortcut containing concatenated endpoints?
Create the following .jgorc:
[shortcuts]
groovy = org.codehaus.groovy:groovy-groovysh:org.codehaus.groovy.tools.shell.Main+commons-cli:commons-cli:1.3.1
Then try launching it:
$ jgo -v groovy
DEBUG 2019-06-28 18:56:35,467: Using settings: {'m2repo': '/Users/curtis/.m2/repository', 'cachedir': '/usr/local/miniconda3/envs/scijava/jgo', 'links': 'auto'}
DEBUG 2019-06-28 18:56:35,467: Using respositories: {}
DEBUG 2019-06-28 18:56:35,467: Using shortcuts: {'groovy': 'org.codehaus.groovy:groovy-groovysh:org.codehaus.groovy.tools.shell.Main+commons-cli:commons-cli:1.3.1'}
DEBUG 2019-06-28 18:56:35,467: Replacing groovy with org.codehaus.groovy:groovy-groovysh:org.codehaus.groovy.tools.shell.Main+commons-cli:commons-cli:1.3.1 in groovy.
DEBUG 2019-06-28 18:56:35,467: Returning expanded coordinate org.codehaus.groovy:groovy-groovysh:org.codehaus.groovy.tools.shell.Main+commons-cli:commons-cli:1.3.1.
INFO 2019-06-28 18:56:36,834: Failed to bootstrap the artifact.
INFO 2019-06-28 18:56:36,834:
INFO 2019-06-28 18:56:36,834: Possible solutions:
INFO 2019-06-28 18:56:36,834: * Double check the endpoint for correctness (https://search.maven.org/).
INFO 2019-06-28 18:56:36,834: * Add needed repositories to ~/.jgorc [repositories] block (see README).
INFO 2019-06-28 18:56:36,834: * Try with an explicit version number (release metadata might be wrong).
ERROR 2019-06-28 18:56:36,834: Error in `/usr/local/miniconda3/envs/scijava/bin/mvn -B -f /usr/local/miniconda3/envs/scijava/jgo/org/codehaus/groovy/groovy-groovysh/org.codehaus.groovy.tools.shell.Main+commons-cli/commons-cli/pom.xml dependency:resolve': 1
DEBUG 2019-06-28 18:56:36,834: Exception: Command '('/usr/local/miniconda3/envs/scijava/bin/mvn', '-B', '-f', '/usr/local/miniconda3/envs/scijava/jgo/org/codehaus/groovy/groovy-groovysh/org.codehaus.groovy.tools.shell.Main+commons-cli/commons-cli/pom.xml', 'dependency:resolve')' returned non-zero exit status 1.
DEBUG 2019-06-28 18:56:36,834: Debug Trace:
Traceback (most recent call last):
File "/usr/local/miniconda3/envs/scijava/lib/python3.7/site-packages/jgo/jgo.py", line 258, in jgo_main
completed_process = run(parser, argv=argv, stdout=stdout, stderr=stderr)
File "/usr/local/miniconda3/envs/scijava/lib/python3.7/site-packages/jgo/jgo.py", line 554, in run
link_type = link_type)
File "/usr/local/miniconda3/envs/scijava/lib/python3.7/site-packages/jgo/jgo.py", line 458, in resolve_dependencies
raise e
File "/usr/local/miniconda3/envs/scijava/lib/python3.7/site-packages/jgo/jgo.py", line 449, in resolve_dependencies
mvn_out = run_and_combine_outputs(mvn, *mvn_args)
File "/usr/local/miniconda3/envs/scijava/lib/python3.7/site-packages/jgo/jgo.py", line 200, in run_and_combine_outputs
return subprocess.check_output((command,) + args, stderr=subprocess.STDOUT)
File "/usr/local/miniconda3/envs/scijava/lib/python3.7/subprocess.py", line 395, in check_output
**kwargs).stdout
File "/usr/local/miniconda3/envs/scijava/lib/python3.7/subprocess.py", line 487, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '('/usr/local/miniconda3/envs/scijava/bin/mvn', '-B', '-f', '/usr/local/miniconda3/envs/scijava/jgo/org/codehaus/groovy/groovy-groovysh/org.codehaus.groovy.tools.shell.Main+commons-cli/commons-cli/pom.xml', 'dependency:resolve')' returned non-zero exit status 1.
DEBUG 2019-06-28 18:56:36,838: std out:
DEBUG 2019-06-28 18:56:36,839: b'[INFO] Scanning for projects...
DEBUG 2019-06-28 18:56:36,839: [INFO]
DEBUG 2019-06-28 18:56:36,839: [INFO] ---< org.codehaus.groovy-BOOTSTRAPPER:groovy-groovysh-BOOTSTRAPPER >----
DEBUG 2019-06-28 18:56:36,839: [INFO] Building groovy-groovysh-BOOTSTRAPPER 0
DEBUG 2019-06-28 18:56:36,839: [INFO] --------------------------------[ jar ]---------------------------------
DEBUG 2019-06-28 18:56:36,839: [WARNING] The POM for org.codehaus.groovy:groovy-groovysh:jar:commons-cli:org.codehaus.groovy.tools.shell.Main+commons-cli is missing, no dependency information available
DEBUG 2019-06-28 18:56:36,839: [INFO] ------------------------------------------------------------------------
DEBUG 2019-06-28 18:56:36,839: [INFO] BUILD FAILURE
DEBUG 2019-06-28 18:56:36,839: [INFO] ------------------------------------------------------------------------
DEBUG 2019-06-28 18:56:36,839: [INFO] Total time: 0.328 s
DEBUG 2019-06-28 18:56:36,839: [INFO] Finished at: 2019-06-28T18:56:36+02:00
DEBUG 2019-06-28 18:56:36,839: [INFO] ------------------------------------------------------------------------
DEBUG 2019-06-28 18:56:36,839: [ERROR] Failed to execute goal on project groovy-groovysh-BOOTSTRAPPER: Could not resolve dependencies for project org.codehaus.groovy-BOOTSTRAPPER:groovy-groovysh-BOOTSTRAPPER:jar:0: Failure to find org.codehaus.groovy:groovy-groovysh:jar:commons-cli:org.codehaus.groovy.tools.shell.Main+commons-cli in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
DEBUG 2019-06-28 18:56:36,839: [ERROR]
DEBUG 2019-06-28 18:56:36,839: [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
DEBUG 2019-06-28 18:56:36,839: [ERROR] Re-run Maven using the -X switch to enable full debug logging.
DEBUG 2019-06-28 18:56:36,839: [ERROR]
DEBUG 2019-06-28 18:56:36,839: [ERROR] For more information about the errors and possible solutions, please read the following articles:
DEBUG 2019-06-28 18:56:36,839: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
DEBUG 2019-06-28 18:56:36,839: '
As far as I can tell, the issue is that short cut expansion happens after splitting into endpoints at + in Python jgo, for example for INPUT=endpoint1+endpoint2
jgo INPUT
will be expanded to the endpoints
expand_if_shortcut(endpoint1)
expand_if_shortcut(endpoint2)
In the example provided in this issue, however, INPUT=groovy:
expand_if_shortcut(groovy)
which results in a single, mal-formed endpoint
org.codehaus.groovy:groovy-groovysh:org.codehaus.groovy.tools.shell.Main+commons-cli:commons-cli:1.3.1
Now this raises the following questions about the order of shortcut expansion and splitting into endpoints:
- Split first, then expand (as in current Python version):
- pro Shortcuts can be applied to all endpoints
- con Shortcuts cannot contain additional endpoints (
+) - con Inconsistent with
jgo.sh
- Expand first, then split (as suggested in this example):
- pro Shortcuts can contain additional endpoints (
+) - pro Consistent with
jgo.sh - con Additional endpoints cannot be created through shortcut chaining, e.g.
groovy-groovysh+commons-cli
- pro Shortcuts can contain additional endpoints (
- Expand first, then split, then expand again
- pro can do both (1) and (2)
- con probably very confusing
I do not have a strong opinion on (1) or (2) but I think we should give it some thought and then decide either way. I personally would like to rule out (3) because it will probably be very confusing and likely cause more harm than good.
I have a simple solution that would be a fourth solution: 4. Split first, expand, then split again
diff --git a/jgo/jgo.py b/jgo/jgo.py
index 0adae45..5ea9ee1 100644
--- a/jgo/jgo.py
+++ b/jgo/jgo.py
@@ -368,7 +368,7 @@ def split_endpoint_string(endpoint_string):
def endpoints_from_strings(endpoint_strings, shortcuts={}):
_logger.debug('Creating endpoints from strings %s with shortcuts %s', endpoint_strings, shortcuts)
- return [Endpoint.parse_endpoint(expand_coordinate(ep, shortcuts=shortcuts)) for ep in endpoint_strings]
+ return [Endpoint.parse_endpoint(ep2) for ep in endpoint_strings for ep2 in expand_coordinate(ep, shortcuts=shortcuts).split('+')]
def coordinates_from_endpoints(endpoints):
return [ep.get_coordinates() for ep in endpoints]
In this case, expanded shortcuts would be split again:
split: groovy -> groovy
expand: groovy -> org.codehaus.groovy:groovy-groovysh:org.codehaus.groovy.tools.shell.Main+commons-cli:commons-cli:1.3.1
split: org.codehaus.groovy:groovy-groovysh:org.codehaus.groovy.tools.shell.Main+commons-cli:commons-cli:1.3.1 -> [
org.codehaus.groovy:groovy-groovysh:org.codehaus.groovy.tools.shell.Main,
commons-cli:commons-cli:1.3.1
]
I still have to get a better intuition. Let me know what you think @ctrueden