ddR
ddR copied to clipboard
vertica
Standard API for Distributed Data Structures in R
Looking to [fork_driver.R](https://github.com/vertica/ddR/blob/master/ddR/R/fork_driver.R) we can see that `dmapply` is essentially `mcmapply`. So we rely on the fact that every object can be easily lazily copied from master to worker process....
I think it make sense to extend S4 method `as` instead of using `as.dlist`, `as.darray` and so on. Correct me if I missed something.
I read the article [dmapply: A functional primitive to express distributed machine learning algorithms in R](http://www.vldb.org/pvldb/vol9/p1293-ma.pdf) and found following diagram: That was surprise to see peer-to-peer communication. I started to...
I am running into a memory issue with drandomForest. I receive this message which we have all seen before: Error in drandomForest.default(m, y, ..., ntree = ntree, nExecutor = nExecutor,...
This is a question more than an issue, I would like to create a new backend using Service Fabric Stateful services, can you lead me to some examples or at...
Probably the most common operation in R is subsetting, ie. `[` and `$`. While trying to use `ddR` and looking at `ops.R` I notice that the subsetting operators all `collect`....
``` > a = as.dframe(iris) > colnames(a) [1] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width" "Species" > a$Species Species Levels: setosa versicolor virginica > iris$Species [1] setosa setosa setosa setosa setosa setosa [7]...
This is too slow: ``` > system.time({ + colnames(ds) + }) user system elapsed 16.015 41.366 206.624 ``` Here `ds` is a a `dframe` with 4 partitions. Each chunk is...
Following up on #15 here are some ideas for improvements to ddR. - [x] Makefile to automate testing, Doc build, and examples. See #15 - [x] Overhaul init and useBackend...
The changes in #15 brought this bug out. Global variables are not exported to a PSOCK cluster. This causes the kmeans example to fail. A minimal example: ``` library(ddR) globvar