Gyeong-In Yu
Gyeong-In Yu
Tests both cpu and gpu version of activation layers only test sigmoid activation function. We need to improve these to test ReLu, Tanh and Clipped ReLu.
A lot of researches articulate that momentum may accelerate job convergence of DNN. We should introduce momentum to our DNN codebase.
Related to #853. Our ML apps can use various kinds of optimization algorithms, including standard SGD with momentum, Adadelta, Nesterov, and so on. Recent researches report that these more sophisticated...
`NeuralNetworkParameterUpdater` requires some linear algebra operations. We should consider whether to make PS for DNN uses GPU or not.
Random matrix creation methods are not implemented in `MatrixCudaFactory` yet. We should use [cuRAND](http://docs.nvidia.com/cuda/curand/#axzz4EpFLqzYF) for random generation in GPU, and allocate & deallocate cuRAND generator in proper situation.
Aggregation Service, introduced by #348, needs some change after [REEF-402](https://issues.apache.org/jira/browse/REEF-402) is resolved. We can use list to generate static configurations for driver and evaluators, and avoid unclean encoding of `AggregationClientInfo`...
While lowering the degree of parallelism, our optimizer can make a plan which moves too much data to a single evaluator. This is not feasible, since our evaluators have memory...
Our current codebase uses `mahout`, which does not support in-place vector computation, leads to redundant memory allocation. For both performance and scalability, we should use other library supports in-place update....