CaffeOnSpark icon indicating copy to clipboard operation
CaffeOnSpark copied to clipboard

Attribute protoFile not valid

Open qinhuaiqiang opened this issue 7 years ago • 0 comments

[root@node93 ~]# pyspark --master yarn \

--driver-library-path "${CAFFE_ON_SPARK}/caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar"
--driver-class-path "${CAFFE_ON_SPARK}/caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar"
--conf spark.driver.extraLibraryPath="${LD_LIBRARY_PATH}"
--py-files ${CAFFE_ON_SPARK}/caffe-grid/target/caffeonsparkpythonapi.zip
--files ${CAFFE_ON_SPARK}/data/caffe/caffe.so
--jars "${CAFFE_ON_SPARK}/caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar" Python 2.7.5 (default, Nov 6 2016, 00:28:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2 Type "help", "copyright", "credits" or "license" for more information. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/2.6.4.0-91/CaffeOnSpark/caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.4.0-91/spark2/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). 19/03/01 11:12:53 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 19/03/01 11:12:55 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. Welcome to ____ __ / / ___ / / \ / _ / _ `/ __/ '/ / / ._/_,
// //_\ version 2.2.0.2.6.4.0-91 /_/

Using Python version 2.7.5 (default, Nov 6 2016 00:28:07) SparkSession available as 'spark'.

from pyspark import SparkConf,SparkContext from com.yahoo.ml.caffe.RegisterContext import registerContext,registerSQLContext from com.yahoo.ml.caffe.CaffeOnSpark import CaffeOnSpark from com.yahoo.ml.caffe.Config import Config from com.yahoo.ml.caffe.DataSource import DataSource from pyspark.mllib.linalg import Vectors from pyspark.mllib.regression import LabeledPoint from pyspark.mllib.linalg import Vectors from pyspark.mllib.classification import LogisticRegressionWithLBFGS registerContext(sc) registerSQLContext(sqlContext) cos=CaffeOnSpark(sc) cfg=Config(sc) cfg.protoFile='/usr/hdp/2.6.4.0-91/CaffeOnSpark/data/lenet_memory_solver.prototxt' Attribute protoFile not valid cfg.protoFile='/user/root/lenet_memory_solver.prototxt' Attribute protoFile not valid

I dont know why the protofile is not valid

qinhuaiqiang avatar Feb 21 '19 06:02 qinhuaiqiang