Large graphs

Open ron5569 opened this issue 6 years ago • 1 comments

Hey I'm trying to use node embeding (in order to find clustreing ) Using 71607 nodes, with 75662 edges

I got: 2019-09-04 12:21:14.928295: W tensorflow/core/framework/allocator.cc:124] Allocation of 20510249796 exceeds 10% of system memory. Process finished with exit code 137 (interrupted by signal 9: SIGKILL)

It's look like due to memory error, is there anyway to overcome it?

I have about 64GB of RAM Thanks!

Sep 04 '19 12:09 ron5569

Yeah!

Could you please advise, what is the way to train bathwise? I have large sparse graph ~3,5m nodes, where the largest subgraph is 500k nodes.

And I have another problem - My graph is MultiDiGraph, and I would like firstly to stack conv2d to convert 3d sparsetensor (nnk, where n - num of nodes, k - num of edge types) to 2d. As I know, there are no way to put SparseTensor as input in tf.layers.conv2d. What is a work around?

Thank you

Sep 19 '19 18:09 mazitovs