core icon indicating copy to clipboard operation
core copied to clipboard
verified publisher icon metacall

add base support for exporting objects and classes in node_loader

Open rxbryan opened this issue 3 years ago • 12 comments

Description

add base support for exporting objects and classes in node_loader

Fixes #(issue_no)

Type of change

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] This change requires a documentation update
  • [ ] Documentation update

Checklist:

  • [ ] I have performed a self-review of my own code.
  • [ ] I have commented my code, particularly in hard-to-understand areas.
  • [ ] I have made corresponding changes to the documentation.
  • [ ] My changes generate no new warnings.
  • [ ] I have added tests/screenshots (if any) that prove my fix is effective or that my feature works.
  • [ ] I have tested the tests implicated (if any) by my own code and they pass (make test or ctest -VV -R <test-name>).
  • [ ] If my change is significant or breaking, I have passed all tests with ./docker-compose.sh build &> output and attached the output.
  • [ ] I have tested my code with OPTION_BUILD_SANITIZER or ./docker-compose.sh test &> output and OPTION_TEST_MEMORYCHECK.
  • [ ] I have tested with Helgrind in case my code works with threading.
  • [ ] I have run make clang-format in order to format my code and my code follows the style guidelines.

If you are unclear about any of the above checks, have a look at our documentation here.

rxbryan avatar Sep 15 '22 13:09 rxbryan

Last commits look pretty well, I recommend you to implement tests in order to verify the behavior. TDD is always easier to develop. Check out metacall-java-test for having an example of how to implement the test.

viferga avatar Feb 07 '23 20:02 viferga

Check out the tests, no one is passing, try to pass them before pushing.

viferga avatar Feb 15 '23 16:02 viferga

Check out the tests, no one is passing, try to pass them before pushing.

I just figured out what's making the tests fail. I would fix it as soon as possible

rxbryan avatar Feb 20 '23 10:02 rxbryan

I node port it seems there's a bug:

2023-02-21T01:41:33.6835391Z [Tue Feb 21 00:11:52] #13550 [ 4174 | py_loader_impl_gc_print | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @Debug : Python Garbage Collector: 2023-02-21T01:41:33.6835405Z 2023-02-21T01:41:33.6835481Z ==13543==T8 TSDDtor 2023-02-21T01:41:33.6835553Z ==13543==T8 exited 2023-02-21T01:41:33.6835560Z 2023-02-21T01:41:33.6835696Z attempt to reference unallocated memory (signal SIGSEGV)

viferga avatar Feb 21 '23 17:02 viferga

Also there's this in windows:

2023-02-21T00:03:25.9422754Z ==1552==ERROR: AddressSanitizer: access-violation on unknown address 0x7ff86ac92a68 (pc 0x7ff86ac92a68 bp 0x00000000000f sp 0x0040fcbffa18 T2) 2023-02-21T00:03:25.9422849Z #0 0x7ff86ac92a67 () 2023-02-21T00:03:25.9422997Z #1 0x7ff89bfa5f24 (C:\Windows\SYSTEM32\ntdll.dll+0x180075f24) 2023-02-21T00:03:25.9423135Z #2 0x7ff89bf532e2 (C:\Windows\SYSTEM32\ntdll.dll+0x1800232e2) 2023-02-21T00:03:25.9423277Z #3 0x7ff89bf8a3dd (C:\Windows\SYSTEM32\ntdll.dll+0x18005a3dd) 2023-02-21T00:03:25.9423433Z #4 0x7ff898798ca9 (C:\Windows\System32\KERNELBASE.dll+0x180078ca9) 2023-02-21T00:03:25.9423581Z #5 0x7ff898ae2793 (C:\Windows\System32\ucrtbase.dll+0x180022793) 2023-02-21T00:03:25.9423720Z #6 0x7ff898ae2690 (C:\Windows\System32\ucrtbase.dll+0x180022690) 2023-02-21T00:03:25.9424017Z #7 0x7ff86133f579 (C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\clang_rt.asan_dbg_dynamic-x86_64.dll+0x18005f579) 2023-02-21T00:03:25.9424527Z #8 0x7ff89b607ad3 (C:\Windows\System32\KERNEL32.DLL+0x180017ad3) 2023-02-21T00:03:25.9424654Z #9 0x7ff89bf8a370 (C:\Windows\SYSTEM32\ntdll.dll+0x18005a370) 2023-02-21T00:03:25.9424665Z 2023-02-21T00:03:25.9424797Z AddressSanitizer can not provide additional info. 2023-02-21T00:03:25.9424947Z SUMMARY: AddressSanitizer: access-violation () 2023-02-21T00:03:25.9425037Z Thread T2 created by T0 here: 2023-02-21T00:03:25.9425338Z #0 0x7ff861340e38 (C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\clang_rt.asan_dbg_dynamic-x86_64.dll+0x180060e38) 2023-02-21T00:03:25.9425484Z #1 0x7ff898ae2356 (C:\Windows\System32\ucrtbase.dll+0x180022356) 2023-02-21T00:03:25.9425636Z #2 0x7ff85c4f9306 (D:\a\core\core\build\Debug\libnode.dll+0x180419306) 2023-02-21T00:03:25.9425792Z #3 0x7ff85c4f924b (D:\a\core\core\build\Debug\libnode.dll+0x18041924b) 2023-02-21T00:03:25.9426040Z #4 0x7ff8632c15d8 in node_loader_impl_initialize D:\a\core\core\source\loaders\node_loader\source\node_loader_impl.cpp:6577 2023-02-21T00:03:25.9426232Z #5 0x7ff866c26344 in loader_impl_initialize D:\a\core\core\source\loader\source\loader_impl.c:367 2023-02-21T00:03:25.9426433Z #6 0x7ff866c22093 in loader_impl_load_from_file D:\a\core\core\source\loader\source\loader_impl.c:826 2023-02-21T00:03:25.9426619Z #7 0x7ff866c1ea69 in loader_load_from_file D:\a\core\core\source\loader\source\loader.c:317 2023-02-21T00:03:25.9426811Z #8 0x7ff866c2bbc2 in metacall_load_from_file D:\a\core\core\source\metacall\source\metacall.c:348 2023-02-21T00:03:25.9427182Z #9 0x7ff7429f302d in metacall_node_event_loop_test_DefaultConstructor_Test::TestBody(void) D:\a\core\core\source\tests\metacall_node_async_resources_test\source\metacall_node_async_resources_test.cpp:45 2023-02-21T00:03:25.9427630Z #10 0x7ff742a2bc8c in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test , void (__cdecl testing::Test::)(void), char const *) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2592 2023-02-21T00:03:25.9428053Z #11 0x7ff742a2b982 in testing::internal::HandleExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test , void (__cdecl testing::Test::)(void), char const *) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2643 2023-02-21T00:03:25.9428367Z #12 0x7ff742a08eba in testing::Test::Run(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2682 2023-02-21T00:03:25.9428626Z #13 0x7ff742a099c4 in testing::TestInfo::Run(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2861 2023-02-21T00:03:25.9428886Z #14 0x7ff742a0a201 in testing::TestSuite::Run(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:3015 2023-02-21T00:03:25.9429169Z #15 0x7ff742a1082b in testing::internal::UnitTestImpl::RunAllTests(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:5855 2023-02-21T00:03:25.9429662Z #16 0x7ff742a2be1c in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl , bool (__cdecl testing::internal::UnitTestImpl::)(void), char const *) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2592 2023-02-21T00:03:25.9430150Z #17 0x7ff742a2bc02 in testing::internal::HandleExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl , bool (__cdecl testing::internal::UnitTestImpl::)(void), char const *) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2643 2023-02-21T00:03:25.9430398Z #18 0x7ff742a0a9d0 in testing::UnitTest::Run(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:5438 2023-02-21T00:03:25.9430714Z #19 0x7ff7429f29e0 in RUN_ALL_TESTS(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\include\gtest\gtest.h:2490 2023-02-21T00:03:25.9430924Z #20 0x7ff7429f1850 in main D:\a\core\core\source\tests\metacall_node_async_resources_test\source\main.cpp:27 2023-02-21T00:03:25.9431127Z #21 0x7ff742a64f88 in invoke_main D:\a_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78 2023-02-21T00:03:25.9431358Z #22 0x7ff742a64edd in __scrt_common_main_seh D:\a_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288 2023-02-21T00:03:25.9431580Z #23 0x7ff742a64d9d in __scrt_common_main D:\a_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:330 2023-02-21T00:03:25.9431791Z #24 0x7ff742a64ffd in mainCRTStartup D:\a_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:16 2023-02-21T00:03:25.9431946Z #25 0x7ff89b607ad3 (C:\Windows\System32\KERNEL32.DLL+0x180017ad3) 2023-02-21T00:03:25.9432093Z #26 0x7ff89bf8a370 (C:\Windows\SYSTEM32\ntdll.dll+0x18005a370)

But I think this has been already solved in develop, can you merge develop into your branch and push so we can see if that test passes?

viferga avatar Feb 21 '23 17:02 viferga

Also there's this in windows:

But I think this has been already solved in develop, can you merge develop into your branch and push so we can see if that test passes?

My branch is up-to-date. I don't run windows so I can't really debug the windows test

rxbryan avatar Feb 21 '23 20:02 rxbryan

I node port it seems there's a bug:

2023-02-21T01:41:33.6835696Z attempt to reference unallocated memory (signal SIGSEGV)

That's not the only problem. There's alot of load_from_file errors... look at this

Traceback (most recent call last):
  File "py_loader_impl_load_from_file_path", line 19, in load_from_path
  File "/usr/lib/python3.9/importlib/util.py", line 90, in find_spec
    fullname = resolve_name(name, package) if name.startswith('.') else name
  File "/usr/lib/python3.9/importlib/util.py", line 32, in resolve_name
    raise ImportError(f'no package specified for {repr(name)} '
ImportError: no package specified for './asd.py' (required for relative module names)

 [Tue Feb 21 00:11:38] #13550 [ 3128 | py_loader_impl_load_from_file_exception | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @Error : Python Error: Exception raised while loading the module './badrequire'
[Tue Feb 21 00:11:38] #13550 [ 4088 | py_loader_impl_error_print | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @Error : Python Error: [Type: <class 'ImportError'>]: no package specified for './badrequire' (required for relative module names)
Traceback (most recent call last):
  File "py_loader_impl_load_from_file_path", line 19, in load_from_path
  File "/usr/lib/python3.9/importlib/util.py", line 90, in find_spec
    fullname = resolve_name(name, package) if name.startswith('.') else name
  File "/usr/lib/python3.9/importlib/util.py", line 32, in resolve_name
    raise ImportError(f'no package specified for {repr(name)} '
ImportError: no package specified for './badrequire' (required for relative module names)

ception in load_from_file Error: Cannot find module './asd.ts'
    at fileResolve (/usr/local/metacall/build/bootstrap.ts:156:25)
    at /usr/local/metacall/build/bootstrap.ts:169:54
    at Array.map (<anonymous>)
    at load_from_file (/usr/local/metacall/build/bootstrap.ts:169:45)
    at /usr/local/metacall/build/bootstrap.ts:44:21
    at metacall_require (/usr/local/metacall/source/ports/node_port/index.js:189:15)
    at Module.mod.require (/usr/local/metacall/source/ports/node_port/index.js:264:11)
    at require (internal/modules/cjs/helpers.js:74:18)
    at /usr/local/metacall/source/ports/node_port/test/index.js:58:26
    at getActual (assert.js:658:5) {
  code: 'MODULE_NOT_FOUND'
}
[Tue Feb 21 00:11:34] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
Exception in load_from_file Error: Cannot find module './asd.tsx'
    at fileResolve (/usr/local/metacall/build/bootstrap.ts:156:25)
    at /usr/local/metacall/build/bootstrap.ts:169:54
    at Array.map (<anonymous>)
    at load_from_file (/usr/local/metacall/build/bootstrap.ts:169:45)
    at /usr/local/metacall/build/bootstrap.ts:44:21
    at metacall_require (/usr/local/metacall/source/ports/node_port/index.js:189:15)
    at Module.mod.require (/usr/local/metacall/source/ports/node_port/index.js:264:11)
    at require (internal/modules/cjs/helpers.js:74:18)
    at /usr/local/metacall/source/ports/node_port/test/index.js:59:26
    at getActual (assert.js:658:5) {
  code: 'MODULE_NOT_FOUND'

[Tue Feb 21 00:11:38] #13550 [ 1253 | rb_loader_impl_load_from_file | /usr/local/metacall/source/loaders/rb_loader/source/rb_loader_impl.c ] @Error : No module could be loaded
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
==13543==T53: FakeStack created: 0x7f67cb5de000 -- 0x7f67cc0e7000 stack_size_log: 20; mmapped 11300K, noreserve=0 
==13543==T53: stack [0x7f67d9f7d000,0x7f67daf7bf00) size 0xffef00; local=0x7f67daf7be1c
CSLoader loading error: File ./badrequire not found
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
[Tue Feb 21 00:11:38] #13550 [ 187 | cob_loader_impl_load_from_file | /usr/local/metacall/source/loaders/cob_loader/source/cob_loader_impl.cpp ] @Error : module 'badrequire' not found
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file

Where should ./asd.py ./asd.ts asd.cs asd.rb be located. I am able to reproduce the load_from_file errors but not the segfault. Maybe they are related.

rxbryan avatar Feb 21 '23 21:02 rxbryan

I node port it seems there's a bug: 2023-02-21T01:41:33.6835696Z attempt to reference unallocated memory (signal SIGSEGV)

That's not the only problem. There's alot of load_from_file errors... look at this

Traceback (most recent call last):
  File "py_loader_impl_load_from_file_path", line 19, in load_from_path
  File "/usr/lib/python3.9/importlib/util.py", line 90, in find_spec
    fullname = resolve_name(name, package) if name.startswith('.') else name
  File "/usr/lib/python3.9/importlib/util.py", line 32, in resolve_name
    raise ImportError(f'no package specified for {repr(name)} '
ImportError: no package specified for './asd.py' (required for relative module names)

 [Tue Feb 21 00:11:38] #13550 [ 3128 | py_loader_impl_load_from_file_exception | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @Error : Python Error: Exception raised while loading the module './badrequire'
[Tue Feb 21 00:11:38] #13550 [ 4088 | py_loader_impl_error_print | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @Error : Python Error: [Type: <class 'ImportError'>]: no package specified for './badrequire' (required for relative module names)
Traceback (most recent call last):
  File "py_loader_impl_load_from_file_path", line 19, in load_from_path
  File "/usr/lib/python3.9/importlib/util.py", line 90, in find_spec
    fullname = resolve_name(name, package) if name.startswith('.') else name
  File "/usr/lib/python3.9/importlib/util.py", line 32, in resolve_name
    raise ImportError(f'no package specified for {repr(name)} '
ImportError: no package specified for './badrequire' (required for relative module names)

ception in load_from_file Error: Cannot find module './asd.ts'
    at fileResolve (/usr/local/metacall/build/bootstrap.ts:156:25)
    at /usr/local/metacall/build/bootstrap.ts:169:54
    at Array.map (<anonymous>)
    at load_from_file (/usr/local/metacall/build/bootstrap.ts:169:45)
    at /usr/local/metacall/build/bootstrap.ts:44:21
    at metacall_require (/usr/local/metacall/source/ports/node_port/index.js:189:15)
    at Module.mod.require (/usr/local/metacall/source/ports/node_port/index.js:264:11)
    at require (internal/modules/cjs/helpers.js:74:18)
    at /usr/local/metacall/source/ports/node_port/test/index.js:58:26
    at getActual (assert.js:658:5) {
  code: 'MODULE_NOT_FOUND'
}
[Tue Feb 21 00:11:34] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
Exception in load_from_file Error: Cannot find module './asd.tsx'
    at fileResolve (/usr/local/metacall/build/bootstrap.ts:156:25)
    at /usr/local/metacall/build/bootstrap.ts:169:54
    at Array.map (<anonymous>)
    at load_from_file (/usr/local/metacall/build/bootstrap.ts:169:45)
    at /usr/local/metacall/build/bootstrap.ts:44:21
    at metacall_require (/usr/local/metacall/source/ports/node_port/index.js:189:15)
    at Module.mod.require (/usr/local/metacall/source/ports/node_port/index.js:264:11)
    at require (internal/modules/cjs/helpers.js:74:18)
    at /usr/local/metacall/source/ports/node_port/test/index.js:59:26
    at getActual (assert.js:658:5) {
  code: 'MODULE_NOT_FOUND'

[Tue Feb 21 00:11:38] #13550 [ 1253 | rb_loader_impl_load_from_file | /usr/local/metacall/source/loaders/rb_loader/source/rb_loader_impl.c ] @Error : No module could be loaded
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
==13543==T53: FakeStack created: 0x7f67cb5de000 -- 0x7f67cc0e7000 stack_size_log: 20; mmapped 11300K, noreserve=0 
==13543==T53: stack [0x7f67d9f7d000,0x7f67daf7bf00) size 0xffef00; local=0x7f67daf7be1c
CSLoader loading error: File ./badrequire not found
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
[Tue Feb 21 00:11:38] #13550 [ 187 | cob_loader_impl_load_from_file | /usr/local/metacall/source/loaders/cob_loader/source/cob_loader_impl.cpp ] @Error : module 'badrequire' not found
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file

Where should ./asd.py ./asd.ts asd.cs asd.rb be located. I am able to reproduce the load_from_file errors but not the segfault. Maybe they are related.

Those errors are fine. I am reviewing everything, you overwrote all my changes. I am going to try to solve them.

viferga avatar Feb 21 '23 22:02 viferga

I think I almost get it to work, I want still to fully review it locally.

viferga avatar Feb 21 '23 23:02 viferga

Yeah, it seems all tests passed, but I still want to fully review it. Formatting failed because I modified it from the web, but I also want to fully review all the implementation properly, I've seen things that are not clear to me and I need to understand them better and provide better feedback to you. Good job!

viferga avatar Feb 22 '23 00:02 viferga