marklogic-contentpump icon indicating copy to clipboard operation
marklogic-contentpump copied to clipboard

output_uri_replace causes failures if it contains backslashes

Open ndw opened this issue 7 years ago • 1 comments

Composing the output_uri_replace value from the base URI of the input_file_path is convenient, but on Windows this results in backslashes in the output_uri_replace value which causes errors. Might be reasonable to translate 's into /'s in the Windows case automatically.

ndw avatar Jul 25 '18 22:07 ndw

mlcp arguments, excluding password: [IMPORT, -output_uri_replace, C:\workspace\head\opsdirector\installer\src\main\opsdir-modules,'', -input_file_path, src/main/opsdir-modules, -output_permissions, app-user,read,app-user,execute, -database, OpsDirectorModules, -host, engrlab-130-086, -port, 8000, -username, admin] SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/C:/Users/rgupta/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.5/6edffc576ce104ec769d954618764f39f0f0f10d/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/C:/Users/rgupta/.gradle/caches/modules-2/files-2.1/org.apache.avro/avro-tools/1.8.1/361c32d4cad8dea8e5944d588e7d410f9f2a7114/avro-tools-1.8.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/C:/Users/rgupta/.gradle/caches/modules-2/files-2.1/ch.qos.logback/logback-classic/1.1.6/665e3de72f19ec66ac67d82612d7b8e6b3de3cd0/logback-classic-1.1.6.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2018-07-25 14:46:18 DEBUG ContentPump:100 - Command: IMPORT 2018-07-25 14:46:18 DEBUG ContentPump:106 - Arguments: -output_uri_replace C:\workspace\head\opsdirector\installer\src\main\opsdir-modules,'' -input_file_path src/main/opsdir-modules -output_permissions app-user,read,app-user,execute -database OpsDirectorModules -host engrlab-130-086 -port 8000 -username admin -password admin 2018-07-25 14:46:19 DEBUG ContentPump:159 - Running in: localmode 2018-07-25 14:46:19 INFO  LocalJobRunner:493 - Content type is set to MIXED.  The format of the  inserted documents will be determined by the MIME  type specification configured on MarkLogic Server. 2018-07-25 14:46:20 DEBUG MutableMetricsFactory:42 - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, always=false, sampleName=Ops, type=DEFAULT, value=[Rate of successful kerberos logins and latency (milliseconds)], valueName=Time) 2018-07-25 14:46:20 DEBUG MutableMetricsFactory:42 - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, always=false, sampleName=Ops, type=DEFAULT, value=[Rate of failed kerberos logins and latency (milliseconds)], valueName=Time) 2018-07-25 14:46:20 DEBUG MutableMetricsFactory:42 - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, always=false, sampleName=Ops, type=DEFAULT, value=[GetGroups], valueName=Time) 2018-07-25 14:46:20 DEBUG MetricsSystemImpl:231 - UgiMetrics, User and group related metrics 2018-07-25 14:46:20 DEBUG Groups:278 -  Creating new Groups object 2018-07-25 14:46:20 DEBUG NativeCodeLoader:46 - Trying to load the custom-built native-hadoop library... 2018-07-25 14:46:20 DEBUG NativeCodeLoader:55 - Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path 2018-07-25 14:46:20 DEBUG NativeCodeLoader:56 - java.library.path=C:\Program Files\Java\jdk1.8.0_72\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\cygwin\usr\local\bin;C:\cygwin\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files\Java\jdk1.8.0_72\bin;C:\Program Files\Java\jre1.8.0_72\bin;C:\ProgramData\Oracle\Java\javapath;C:\Program Files\nodejs;C:\Users\atrehan\AppData\Roaming\npm;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\PuTTY;C:\winutils\bin;C:\cygwin\lib\lapack;. 2018-07-25 14:46:20 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-07-25 14:46:20 DEBUG PerformanceAdvisory:41 - Falling back to shell based 2018-07-25 14:46:20 DEBUG JniBasedUnixGroupsMappingWithFallback:45 - Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping 2018-07-25 14:46:20 DEBUG Groups:91 - Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000 2018-07-25 14:46:20 DEBUG UserGroupInformation:209 - hadoop login 2018-07-25 14:46:20 DEBUG UserGroupInformation:144 - hadoop login commit 2018-07-25 14:46:20 DEBUG UserGroupInformation:174 - using local user:NTUserPrincipal: rgupta 2018-07-25 14:46:20 DEBUG UserGroupInformation:180 - Using user: "NTUserPrincipal: rgupta" with name rgupta 2018-07-25 14:46:20 DEBUG UserGroupInformation:190 - User entry: "rgupta" 2018-07-25 14:46:20 DEBUG UserGroupInformation:799 - UGI loginUser:rgupta (auth:SIMPLE) 2018-07-25 14:46:20 INFO  ContentPump:227 - Job name: local_1896151466_1 2018-07-25 14:46:21 DEBUG LocalJobRunner:92 - Thread pool size: 4 2018-07-25 14:46:21 INFO  FileAndDirectoryInputFormat:217 - Total input paths to process : 4 2018-07-25 14:46:21 DEBUG CombineDocumentInputFormat:99 - Total # of splits: 195 2018-07-25 14:46:21 DEBUG CombineDocumentInputFormat:100 - Total # of combined splits: 1 2018-07-25 14:46:22 DEBUG LocalJobRunner:199 - Thread Count for Split#0 : 4 2018-07-25 14:46:22 DEBUG MultithreadedMapper:182 - Running with 4 threads 2018-07-25 14:46:22 ERROR MultithreadedMapper:383 - Error running task: Illegal/unsupported escape sequence near index 18 C:\workspace\head\opsdirector\installer\src\main\opsdir-modules                   ^ 2018-07-25 14:46:22 DEBUG MultithreadedMapper:385 - java.util.regex.PatternSyntaxException: Illegal/unsupported escape sequence near index 18 C:\workspace\head\opsdirector\installer\src\main\opsdir-modules                   ^ 2018-07-25 14:46:22 ERROR MultithreadedMapper:383 - Error running task: Illegal/unsupported escape sequence near index 18 C:\workspace\head\opsdirector\installer\src\main\opsdir-modules                   ^ 2018-07-25 14:46:22 DEBUG MultithreadedMapper:385 - java.util.regex.PatternSyntaxException: Illegal/unsupported escape sequence near index 18 C:\workspace\head\opsdirector\installer\src\main\opsdir-modules                   ^ 2018-07-25 14:46:22 ERROR MultithreadedMapper:383 - Error running task: Illegal/unsupported escape sequence near index 18 C:\workspace\head\opsdirector\installer\src\main\opsdir-modules                   ^ 2018-07-25 14:46:22 DEBUG MultithreadedMapper:385 - java.util.regex.PatternSyntaxException: Illegal/unsupported escape sequence near index 18 C:\workspace\head\opsdirector\installer\src\main\opsdir-modules                   ^ 2018-07-25 14:46:22 ERROR MultithreadedMapper:383 - Error running task: Illegal/unsupported escape sequence near index 18 C:\workspace\head\opsdirector\installer\src\main\opsdir-modules                   ^ 2018-07-25 14:46:22 DEBUG MultithreadedMapper:385 - java.util.regex.PatternSyntaxException: Illegal/unsupported escape sequence near index 18 C:\workspace\head\opsdirector\installer\src\main\opsdir-modules                   ^ 2018-07-25 14:46:22 INFO  LocalJobRunner:531 -  completed 0% 2018-07-25 14:46:22 INFO  LocalJobRunner:287 - com.marklogic.mapreduce.MarkLogicCounter: 2018-07-25 14:46:22 INFO  LocalJobRunner:291 - INPUT_RECORDS: 0 2018-07-25 14:46:22 INFO  LocalJobRunner:291 - OUTPUT_RECORDS: 0 2018-07-25 14:46:22 INFO  LocalJobRunner:291 - OUTPUT_RECORDS_COMMITTED: 0 2018-07-25 14:46:22 INFO  LocalJobRunner:291 - OUTPUT_RECORDS_FAILED: 0 2018-07-25 14:46:22 INFO  LocalJobRunner:295 - Total execution time: 1 sec

jxchen-us avatar Jul 26 '18 00:07 jxchen-us