GraphScope icon indicating copy to clipboard operation
GraphScope copied to clipboard
verified publisher icon alibaba

[BUG] Wcc failed to compile

Open hrz6976 opened this issue 8 months ago • 9 comments

Describe the bug

Wcc failed to compile.

/home/hrz/mambaforge/envs/gs/lib/python3.11/site-packages/graphscope.runtime/include/grape/analytical_apps/wcc/wcc.h:60:22: error: ‘const grape::WCC<gs::ArrowFlattenedFragment<std::basic_string<char>, long unsigned int, grape::EmptyType, grape::EmptyType> >::fragment_t’ {aka ‘const class gs::ArrowFlattenedFragment<std::basic_string<char>, long unsigned int, grape::EmptyType, grape::EmptyType>’} has no member named ‘GetOutgoingInnerVertexAdjList’
   60 |       auto es = frag.GetOutgoingInnerVertexAdjList(v);
      |                 ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: note: unrecognized command-line option ‘-Wno-unused-lambda-capture’ may have been intended to silence earlier diagnostics
cc1plus: note: unrecognized command-line option ‘-Wno-undef-prefix’ may have been intended to silence earlier diagnostics

To Reproduce

  1. Start a local server session:
import graphscope
graphscope.session(cluster_type="hosts")
  1. Start another session and connect to the server:
import graphscope
from graphscope.framework.loader import Loader
sess = graphscope.session(cluster_type="hosts", addr="127.0.0.1:38159", dangling_timeout_seconds=30000)
  1. Import the graph from a csv:
graph = sess.g(oid_type="string")
graph = graph.add_edges( Loader(
         "/fast/hrz/osscout/redis_dump_blob.csv",
         delimiter=",",
     ),
    src_label="old_blob",
     dst_label="new_blob")
  1. Run wcc:
wcc_result = graphscope.wcc(graph)

Expected behavior No compile error should occur.

Screenshots Please refer to the logs.

Environment (please complete the following information):

  • GraphScope version: v0.2.9
  • OS: Ubuntu
  • Version 24.04 LTS
  • Kubernetes Version N/A

Additional context

Input: input_head_20.csv Conda environment: environment.yml.txt

hrz6976 avatar May 21 '25 06:05 hrz6976

Thanks for reporting this issue! We will try to reproduce.

zhanglei1949 avatar May 21 '25 06:05 zhanglei1949

It seems ok as far as I tested.

import graphscope
from graphscope.framework.loader import Loader

graph = sess.g(oid_type="string",directed=False)
 graph = graph.add_edges(Loader("/mnt/zhanglei/input_head_20.csv", delimiter=",",src_label="old_blob", dst_label="new_blob")
ret = graphscope.wcc(graph)
df = ret.to_dataframe({"id": "v.id", "community": "r"})

                                          id            community
0   ad75b35f03aa9b764571f67ba25fa6e3e224fbcb                    0
1   f8989cce7f329201214483b609fb34cdde505f18                    1
2   e55e55042da8b3186dec809ee8c81fde65153e23                    2
3   0dd999ceed36a6681a8fae42ebedad0aed96ad00                    3
4   63ec6e7fcfddbb3c50febf973a6ec156774b4399                    4
5   df8cd258a3ec007c93243de8aa1e717fa685bc39                    5
6   851d1a2ed004246e57c658810683b3eba54a0e54                    6
7   763c84084ea4ddfd866fadd0e3051bf5acd37541                    7
8   58d92ed7d79af1e574e76f93fef8cf9e423c0719                    7
9   2b40e94043ee446d541ec3bf635eabe4cf7162b7                    4
10  724487727b96e00a2ee737da79ea414668ecd6bf                   10
11  e0fb240ad64632b625651d93bb1eeff504c87b6f                    5
12  1a632d26ace891e24ae88791993d05f1cd25b3c9                    6
13  d7f4866a2575dc4cff4ace9a2a4820f9f1822ca7                   13
14  89b34b8fbd52e5588a645ee8cdb4945f29ce9a92                   14
15  83f0a1526bb5673f793e53e7e6f6077f8560818f                   15
16  0bd3373315ec1e78298dfe50be59bd76623462b7                    3
17  661a456bb13415b801c2324d9298d318439c4df1                   13
18  5181ad0efd6c9c1ca8cb4b6ac6370aed49c3487a                   14
19  3fd6addd7ab28d171a2fc729e139f0808588821b                   15
20  acf82ab7afd5293de25b753f13c731b29f863029  9223372036854775811
21  200e5c9f74e744581da59b0e05c99a138c10d90c                    0
22  a4a451b0c79196274f0e7bcf162c9bf1a424b9de                    1
23  54785ed6c0ef0a742c270b8eda9c281be5cb9a3f                    2
24  1dc44cde3814ff0727092a06023c581d7bb21548                   14
25  c225b9f679b8237631f5be413cd09326aa160ca4  9223372036854775811
26  df21a3f5de326f6c8ce881246aa41636c00b7665  9223372036854775811
27  9db562379b786d59302de6e939131bc1563a1038  9223372036854775818
28  3eae70f032fea245a5520c3f6ba59c444a08f4d6                    7
29  9e724e37ebaf5d4f0b9ec6cad931675d20e7024f  9223372036854775818
30  1dee3e9b28f28bcc3b032dcb76eaf2b3a9c808a9  9223372036854775818
31  0b76b7d4676f5e31b105749b28be05e8491694f1                   10
32  27bf446c43bf010fe5c048a83b180cbb28563d58                    7

environment Linux 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux python3.10.12

pip3 install graphscope==0.29.0

zhanglei1949 avatar May 21 '25 10:05 zhanglei1949

Could you please provide additional details about the Python version and the platform or operating system you are using? Also note that wcc need a undirected graph.

zhanglei1949 avatar May 21 '25 10:05 zhanglei1949 

platform system: Linux
machine: x86_64
platform: Linux-6.8.0-60-generic-x86_64-with-glibc2.39
uname: uname_result(system='Linux', node='stor.server.osslab-pku.org', release='6.8.0-60-generic',
version='#63-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 19:04:15 UTC 2025', machine='x86_64')
kernel_ver: #63-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 19:04:15 UTC 2025
mac_ver: ('', ('', '', ''), '')
gcc_ver: b'gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0\nCopyright (C) 2023 Free Software Foundation,
Inc.\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even
for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n'
python_ver: 3.11.11

In [6]: graphscope.__version__
Out[6]: '0.29.0'

Update: gcc 14.2.0, same error.

hrz6976 avatar May 24 '25 04:05 hrz6976

Could you please try run wcc in the session where graphscope service is launched, i.e. don't create two sessions.

zhanglei1949 avatar May 26 '25 09:05 zhanglei1949

Thank you. The problem is I could not. Launching sess = graphscope.session(cluster_type="hosts") blocks the ipython session:

Image

hrz6976 avatar May 26 '25 18:05 hrz6976

Tried again with an undirected graph, gcc 11, same error:

In [17]: graph = sess.g(oid_type="string",directed=False)
    ...: graph = graph.add_edges( Loader(
    ...:          "/home/hrz/input_head_20.csv",
    ...:          delimiter=",",
    ...:      ),
    ...:     src_label="old_blob",
    ...:      dst_label="new_blob")
I20250527 03:56:11.874939 60060 /work/analytical_engine/core/grape_instance.cc:145] Loading graph, graph name: graph_KvRbTNcL, graph type: ArrowFragment, type sig: 010faa853c156b122f3d62b7c4718ad9376c4283a6f50370c2918899a21cbe59
I20250527 03:56:11.875221 60058 /work/analytical_engine/core/grape_instance.cc:145] Loading graph, graph name: graph_KvRbTNcL, graph type: ArrowFragment, type sig: 010faa853c156b122f3d62b7c4718ad9376c4283a6f50370c2918899a21cbe59
Loading empty graph:   0%|          | 0/10 [00:00<?, ?it/s]
Loading empty graph: 100%|██████████| 10/10 [00:00<00:00, 501.61it/s]
I20250527 03:56:12.020515 60058 /work/analytical_engine/core/grape_instance.cc:210] Unloading Graph graph_HePQPILn
I20250527 03:56:12.020509 60060 /work/analytical_engine/core/grape_instance.cc:210] Unloading Graph graph_HePQPILn
2025-05-27 03:56:12,100 [WARNING][graph:632]: Deducing vertex labels old_blob
2025-05-27 03:56:12,100 [WARNING][graph:636]: Deducing vertex labels new_blob
Loading edge labeled _:   0%|          | 0/10 [00:00<?, ?it/s]
Loading edge labeled _:  60%|██████    | 6/10 [00:00<00:00, 56.41it/s]
Loading edge labeled _: 100%|██████████| 10/10 [00:00<00:00, 64.42it/s]
I20250527 03:56:12.642055 60058 /work/analytical_engine/core/grape_instance.cc:210] Unloading Graph graph_KvRbTNcL
I20250527 03:56:12.642066 60060 /work/analytical_engine/core/grape_instance.cc:210] Unloading Graph graph_KvRbTNcL

In [18]: wcc_result = graphscope.wcc(graph)
I20250527 03:56:16.377355 60058 /work/analytical_engine/core/grape_instance.cc:281] Projecting graph graph_ihuzsgXC to simple graph: graph_projected_nQ3hmDTj, type sig: f60a8690a466e1d0062eab272a0ce482cae5b161cc5e4f033d08034e461254a5
I20250527 03:56:16.377338 60060 /work/analytical_engine/core/grape_instance.cc:281] Projecting graph graph_ihuzsgXC to simple graph: graph_projected_nQ3hmDTj, type sig: f60a8690a466e1d0062eab272a0ce482cae5b161cc5e4f033d08034e461254a5
2025-05-27 03:56:16,379 [WARNING][wcc:59]: WCC algorithm will output int value as component ID on graphs that has 'string' type as ID
2025-05-27 03:56:16,533 [INFO][utils:243]: app type: grape::WCC<_GRAPH_TYPE> (wcc/wcc.h), graph type: gs::ArrowFlattenedFragment<std::string,uint64_t,grape::EmptyType,grape::EmptyType> (core/fragment/arrow_flattened_fragment.h)
2025-05-27 03:56:16,536 [INFO][utils:486]: Building app library...
2025-05-27 03:56:16,581 [INFO][utils:503]: Codegened application type: cpp_pie, app header: wcc/wcc.h, app_class: grape::WCC<_GRAPH_TYPE>, vd_type: None, md_type: None, pregel_combine: None,             java_jar_path: None, java_app_class: None
2025-05-27 03:56:16,582 [INFO][utils:448]: compile on local, ["cmake . -DNETWORKX=ON -DCMAKE_PREFIX_PATH='/home/hrz/mambaforge/envs/gs/lib/python3.11/site-packages/graphscope.runtime;/home/hrz/mambaforge/envs/gs/lib/python3.11/site-packages/graphscope.runtime/openmpi' -DWCC_USE_GID=ON", 'make -j2'], /tmp/gs/builtin/4353fcbad20964dd1a480d9e9ea74e46456b82a6c37b537dd43b0f177a568afa, 4353fcbad20964dd1a480d9e9ea74e46456b82a6c37b537dd43b0f177a568afa
2025-05-27 03:56:16,582 [INFO][utils:355]: Running command: cmake . -DNETWORKX=ON -DCMAKE_PREFIX_PATH='/home/hrz/mambaforge/envs/gs/lib/python3.11/site-packages/graphscope.runtime;/home/hrz/mambaforge/envs/gs/lib/python3.11/site-packages/graphscope.runtime/openmpi' -DWCC_USE_GID=ON, cwd: /tmp/gs/builtin/4353fcbad20964dd1a480d9e9ea74e46456b82a6c37b537dd43b0f177a568afa
2025-05-27 03:56:19,087 [INFO][utils:355]: Running command: make -j2, cwd: /tmp/gs/builtin/4353fcbad20964dd1a480d9e9ea74e46456b82a6c37b537dd43b0f177a568afa
...
gInnerVertexAdjList’
   60 |       auto es = frag.GetOutgoingInnerVertexAdjList(v);
      |                 ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: note: unrecognized command-line option ‘-Wno-unused-lambda-capture’ may have been intended to silence earlier diagnostics
cc1plus: note: unrecognized command-line option ‘-Wno-undef-prefix’ may have been intended to silence earlier diagnostics
make[2]: *** [CMakeFiles/4353fcbad20964dd1a480d9e9ea74e46456b82a6c37b537dd43b0f177a568afa.dir/build.make:76: CMakeFiles/4353fcbad20964dd1a480d9e9ea74e46456b82a6c37b537dd43b0f177a568afa.dir/home/hrz/mambaforge/envs/gs/lib/python3.11/site-packages/graphscope.runtime/include/graphscope/frame/app_frame.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:110: CMakeFiles/4353fcbad20964dd1a480d9e9ea74e46456b82a6c37b537dd43b0f177a568afa.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

Env:

platform: Linux-6.8.0-60-generic-x86_64-with-glibc2.39
uname: uname_result(system='Linux', release='6.8.0-60-generic', version='#63-Ubuntu SMP PREEMPT_DYNAMIC Tue
Apr 15 19:04:15 UTC 2025', machine='x86_64')
kernel_ver: #63-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 19:04:15 UTC 2025
mac_ver: ('', ('', '', ''), '')
gcc_ver: b'gcc (conda-forge gcc 11.4.0-13) 11.4.0\nCopyright (C) 2021 Free Software Foundation, Inc.\nThis is free software; see the source for
copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n'
python_ver: 3.11.11

Are you using the official wheel on PyPI?

hrz6976 avatar May 26 '25 20:05 hrz6976

Ooops, why not try our develop image: registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:v0.24.2-amd64.

zhanglei1949 avatar May 27 '25 06:05 zhanglei1949

/cc @zhanglei1949, this issus/pr has had no activity for for a long time, could you folks help to review the status ? To suppress further notifications,

  • for issues,
    • if it is waiting for further response from the reporter/author, please help to add the label requires-further-info,
    • if you have already started working on it, please add the label work-in-progress to the issue,
    • if this issue requires further designing discussion and not in current plan, or won't be fixed, please add the label requires-further-discussion or wontfix to the issue,
  • for pull requests,
    • if you are still working on it and it is not ready for reviewing, please convert this pull request as draft PR,
    • if you have decided to hold this development on, please add the requires-further-discussion label to the pull request. Thanks!

github-actions[bot] avatar Jun 12 '25 00:06 github-actions[bot]