Opt icon indicating copy to clipboard operation
Opt copied to clipboard

Dynamically change the dimensions of unknowns without re-compiling?

Open shengyenlo opened this issue 8 years ago • 16 comments

Is it possible changing the dimensions of unknowns dynamically without re-compiling (calling Opt_ProblemPlan(.) )? Thanks.

shengyenlo avatar May 04 '17 02:05 shengyenlo

Not in the current version of Opt. We are working on a larger-scale revision to how Opt handle scheduling units of work that would likely subsume this concern. Out of curiosity, what is your use case; some sort of remeshing? And how urgent is this ask?

Mx7f avatar May 15 '17 03:05 Mx7f

OK, many thanks. And Yes, I'm working on the nonrigid surface tracking, fusion and remeshing now. I'd just like to compare the performance between different solvers.

shengyenlo avatar Jun 01 '17 02:06 shengyenlo

I think I have a similar problem, I am working on non-rigid tracking and reconstruction, so after each iteration, my surface gets updated, and unknowns need to be updated. Currently, I am calling these functions to delete the problem and start new problem on each iteration

Opt_PlanFree(m_optimizerState, m_plan); Opt_ProblemDelete(m_optimizerState, m_problem);

However, after 36 iterations, I am getting following error

nUnknowns = 32304 nResiduals = table: 0x7ccd7d10 nnz = table: 0x7ccd96a0 //////////// ITERATION1 (OPT) /////////////// 0 terra (JIT) 0x00007fc01446c004 $opt.SetSolverParameter + 4 /home/aslab/shafeeq-project/new_codes/C_plus_plus/my_codes/NICP_testing/src/cmake-build-debug/Kintinuous(_ZN12ImageWarping5solveEv+0x19e) [0x804d2e]

Any workarounds for this problem?

eshafeeqe avatar Jun 02 '17 23:06 eshafeeqe

Sometimes I am getting this error message also

PANIC: unprotected error in call to Lua API (not enough memory)

eshafeeqe avatar Jun 02 '17 23:06 eshafeeqe

That sounds like a memory leak to me! I'll work on tracking that down this weekend. As for a workaround, you could define some maximum problem size and have dummy unknowns that pad out the space. Though, if you don't need to fix it before Monday I should hopefully have the leak tracked down by then.

Mx7f avatar Jun 02 '17 23:06 Mx7f

I am not in that hurry, one more thing, I have your previous version of the library (got by contacting niessner). In that, I can able to run 100-110 iterations before crashing.

eshafeeqe avatar Jun 03 '17 02:06 eshafeeqe

That's consistent with the memory leak theory; I should have a fix by tomorrow for you to try.

Mx7f avatar Jun 03 '17 02:06 Mx7f

OK thank you so much

eshafeeqe avatar Jun 03 '17 02:06 eshafeeqe

I've fixed leaking of cuda buffers. There are likely still other memory leaks, but you might be able to get more iterations. I'll work on this more next week.

Mx7f avatar Jun 03 '17 17:06 Mx7f

I tried the new version and got the crash after 36 iterations (same as before) with following error message.

PANIC: unprotected error in call to Lua API (not enough memory)

eshafeeqe avatar Jun 04 '17 00:06 eshafeeqe

I think the error is not because of memory leaks, similar error reported in other github projects

https://github.com/OpenNMT/OpenNMT/issues/26

eshafeeqe avatar Jun 04 '17 00:06 eshafeeqe

The error is we are hitting the (very low) LuaJIT memory limit; I am working on lowering our Lua memory use.

Mx7f avatar Jun 29 '17 19:06 Mx7f

Should be helped greatly by #73; give it a try if you can.

Mx7f avatar Jul 05 '17 18:07 Mx7f

Hello @Mx7f

Thanks for the update, I tried the new library, and now I am only doing Opt_PlanFree, since my problem is same only the dimension is different. I can able to run the code 520+ iterations before crashing. But the crashing problem still exists.

eshafeeqe avatar Sep 20 '17 00:09 eshafeeqe

This is definitely a wart on the current version of Opt that I will work on, but you can workaround this by always using the maximum problem size (could have a small set of problem sizes you choose from) and padding the dimensions with dummy unknowns with a signal value that you detect in the energy and set the energy to 0 when it involves said dummy unknowns.

Mx7f avatar Oct 12 '17 23:10 Mx7f

@Mx7f has there been any progress on this? i'm running into the same problem after only about 20 iterations of the solver. is it likely that the luajit memory limit will be bypassed?

dgrzech avatar Feb 01 '18 15:02 dgrzech