linkis
linkis copied to clipboard
apache
[Question] PythonCodeParser.parsePythonCode() statementBuffer append error when code with Python decorator (SyntaxError: invalid syntax)
Before asking
- [X] I have read the How to ask for help (我已阅读如何寻求帮助指引文档).
Your environment
- Linkis version used: 1.3.0
- Environment name and version:
- scala-2.12.2
- jdk 1.8.0_121
- ....
Describe your questions
-
before i meet this bug, i merged this issue code to my branch https://github.com/apache/incubator-linkis/issues/2763 this issue fixed python code parser statementBuffer.last has error
-
but i found that : this has an another bug :
-
when I run these code by python engine
-
but failed
-
this error means that : python cannot compile code like this
final_code = '@timmer' compiledCode = compile(final_code, "<string>", "exec") -
then try to run PythonCodeParser.parsePythonCode ()
this is wrong statementBuffer that python cannot compile -
then i fixed it
-
ithink this is correct code and it run succed
Eureka service list
eg:
Some logs info or acctch file
linkis-xxx.log:
<!--日志文字贴到这里-->
2022-09-19 17:01:15.001 INFO Your job is accepted, jobID is IDE_jianhui_lv_python_0 and taskID is 148265 in ServiceInstance(linkis-cg-entrance, centos-bigdatalinkis-2402217.intsig.internal:9104). Please wait it to be scheduled
job is scheduled.
2022-09-19 17:01:15.001 INFO Your job is Scheduled. Please wait it to run.
Your job is being scheduled by orchestrator.
2022-09-19 17:01:15.001 INFO job is running.
2022-09-19 17:01:15.001 INFO Your job is Running now. Please wait it to complete.
2022-09-19 17:01:15.001 INFO Job with jobGroupId : 148265 and subJobId : 148224 was submitted to Orchestrator.
2022-09-19 17:01:15.001 INFO Background is starting a new engine for you,execId astJob_22964_codeExec_22964 mark id is mark_22974, it may take several seconds, please wait
2022-09-19 17:01:28.001 INFO EngineConn local log path: ServiceInstance(linkis-cg-engineconn, centos-bigdatalinkis-240227.intsig.internal:40186) /data/linkis/engineconnRootDir/jianhui_lv/20220919/python/1d5c119a-2499-4dc1-afda-d7dc7839df88/logs
centos-bigdatalinkis-240227.intsig.internal:40186_0 >> import time
centos-bigdatalinkis-240227.intsig.internal:40186_0 >> def timmer(func):
def wrapper(*args, **kwargs):
start = time.time()
func(*args, **kwargs)
stop = time.time()
print('foo运行时长:', stop - start)
return wrapper
@timmer
2022-09-19 17:01:28.904 ERROR [Linkis-Default-Scheduler-Thread-3] org.apache.linkis.manager.engineplugin.python.executor.PythonEngineConnExecutor 58 error - execute code failed! org.apache.linkis.manager.engineplugin.python.exception.PythonExecuteError: errCode: 41001 ,desc: Traceback (most recent call last):
File "/data/linkis/engineconnRootDir/jianhui_lv/20220919/python/1d5c119a-2499-4dc1-afda-d7dc7839df88/tmp/3297327555865221255.py", line 220, in <module>
compiledCode = compile(final_code, "<string>", "exec")
File "<string>", line 8
@timmer
^
SyntaxError: invalid syntax
,ip: centos-bigdatalinkis-240227.intsig.internal ,port: 40186 ,serviceKind: linkis-cg-engineconn
at org.apache.linkis.manager.engineplugin.python.executor.PythonSession.setStatementsFinished(PythonSession.scala:197) ~[linkis-engineplugin-python-1.0.3.jar:1.0.3]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_251]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_251]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_251]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_251]
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) ~[py4j-0.10.4.jar:?]
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) ~[py4j-0.10.4.jar:?]
at py4j.Gateway.invoke(Gateway.java:280) ~[py4j-0.10.4.jar:?]
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) ~[py4j-0.10.4.jar:?]
at py4j.commands.CallCommand.execute(CallCommand.java:79) ~[py4j-0.10.4.jar:?]
at py4j.GatewayConnection.run(GatewayConnection.java:214) ~[py4j-0.10.4.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_251]
2022-09-19 17:01:28.920 ERROR [Linkis-Default-Scheduler-Thread-3] org.apache.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl 58 error - PythonExecuteError: errCode: 41001 ,desc: Traceback (most recent call last):
File "/data/linkis/engineconnRootDir/jianhui_lv/20220919/python/1d5c119a-2499-4dc1-afda-d7dc7839df88/tmp/3297327555865221255.py", line 220, in <module>
compiledCode = compile(final_code, "<string>", "exec")
File "<string>", line 8
@timmer
^
SyntaxError: invalid syntax
,ip: centos-bigdatalinkis-240227.intsig.internal ,port: 40186 ,serviceKind: linkis-cg-engineconn org.apache.linkis.manager.engineplugin.python.exception.PythonExecuteError: errCode: 41001 ,desc: Traceback (most recent call last):
File "/data/linkis/engineconnRootDir/jianhui_lv/20220919/python/1d5c119a-2499-4dc1-afda-d7dc7839df88/tmp/3297327555865221255.py", line 220, in <module>
compiledCode = compile(final_code, "<string>", "exec")
File "<string>", line 8
@timmer
^
SyntaxError: invalid syntax
,ip: centos-bigdatalinkis-240227.intsig.internal ,port: 40186 ,serviceKind: linkis-cg-engineconn
at org.apache.linkis.manager.engineplugin.python.executor.PythonSession.setStatementsFinished(PythonSession.scala:197) ~[linkis-engineplugin-python-1.0.3.jar:1.0.3]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_251]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_251]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_251]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_251]
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) ~[py4j-0.10.4.jar:?]
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) ~[py4j-0.10.4.jar:?]
at py4j.Gateway.invoke(Gateway.java:280) ~[py4j-0.10.4.jar:?]
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) ~[py4j-0.10.4.jar:?]
at py4j.commands.CallCommand.execute(CallCommand.java:79) ~[py4j-0.10.4.jar:?]
at py4j.GatewayConnection.run(GatewayConnection.java:214) ~[py4j-0.10.4.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_251]
Job with execId-IDE_jianhui_lv_python_0 and subJobId : 148224 from orchestrator completed with state ErrorExecuteResponse(21304, Task is Failed,errorMsg: PythonExecuteError: errCode: 41001 ,desc: Traceback (most recent call last):
File "/data/linkis/engineconnRootDir/jianhui_lv/20220919/python/1d5c119a-2499-4dc1-afda-d7dc7839df88/tmp/3297327555865221255.py", line 220, in <module>
compiledCode = compile(final_code, "<string>", "exec")
File "<string>", line 8
@timmer
^
SyntaxError: invalid syntax
,ip: centos-bigdatalinkis-240227.intsig.internal ,port: 40186 ,serviceKind: linkis-cg-engineconn,null)
log file:
test.python__1663578193997.log
:blush: Welcome to the Apache Linkis (incubating) community!!
We are glad that you are contributing by opening this issue.
Please make sure to include all the relevant context. We will be here shortly.
If you are interested in contributing to our website project, please let us know! You can check out our contributing guide on :point_right: How to Participate in Project Contribution.
Community
| WeChat Assistant | WeChat Public Account |
|---|---|
 |
 |
Mailing Lists
| name | description | Subscribe | Unsubscribe | archive |
|---|---|---|---|---|
| [email protected] | community activity information | subscribe | unsubscribe | archive |
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Would you like to fix it? Submit pr based on the dev-1.3.1 branch to fix this problem. can I add you as a wechat friend, my wechat: casionen
ok , i will submit pr to fix it i apply wechat request , wechatId: wojiaolvjianhui , nickname is 'Jason;