hanfengatonline

@trivialfis Hi! thanks，i get [n_thread=8] from web，then use XGBoostClassifier.setNthread(n_thread) and spark config contrains [set spark.task.cpus = 8]

@wbo4958 if train data is 1 0000 0000, can i set num_workers=2000?

@wbo4958 n_thread = 8 is not working for one task?

@wbo4958 i want to use multi-threads to speed up one task , i can set one task work on one excutor, and excutor.cores=8 or 16 is ok,The only target right...

@wbo4958 thanks！ i have tried set spark.task.cpus = 1 and set num_workers=1000, because data column=1790 and rows = 1 000 0000,so here are 1000 spark task running , the param...

@wbo4958 hi, we metrics on the train process, locally we use visualVM and we have excutor metrics (online) to trace CPU usage and running thread count

@Craigacp tensorflow-core-native 1.0.0-rc.2 linux-x86_64， Is my package dependency usage incorrect？  sorry It seems that this package may not have been released with support for Java 8 version？

@Craigacp hi, i have tried jdk 11, and tensorflow-core-api / native: 1.0.0-rc.2  it seems that the results have not changed. jdk: openjdk version "11.0.23" 2024-04-16 linux: linux11,x86_64 my code:...

@Craigacp yes, there has oom, my bootstrap.sh : java -Xms18G -Xmx18G \ -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${root_dir}/dist/java_headdump.hprof \ -XX:ActiveProcessorCount=${MY_CPU_LIMIT:-2} \ -XX:+UseG1GC \ -XX:MaxGCPauseMillis=200 \ --add-opens java.base/java.lang=ALL-UNNAMED \ --add-opens java.base/java.lang.invoke=ALL-UNNAMED \ --add-opens java.base/java.lang.reflect=ALL-UNNAMED \...

@Craigacp java heap is ok, only 3GB, but total cost 25.1G by tensorflow graph and runtime tensorflow netive memory