filecoin-project
Problems with IPNIsync
Checklist
- [X] This is not a question or a support request. If you have any boost related questions, please ask in the discussion forum.
- [X] This is not a new feature request. If it is, please file a feature request instead.
- [X] This is not an enhancement request. If it is, please file a improvement suggestion instead.
- [X] I have searched on the issue tracker and the discussion forum, and there is no existing related issue or discussion.
- [X] I am running the
Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these. - [X] I did not make any code changes to boost.
Boost component
- [X] boost daemon - storage providers
- [ ] boost client
- [ ] boost UI
- [ ] boost data-transfer
- [X] boost index-provider
- [ ] Other
Boost Version
Latest master of 13 nov 2023
Describe the Bug
Boost generates an error regenerated link from multihash lister did not match the original link; multihashes returned by the lister for the same key are not consistent causing IPNIsync to halt on Indexer side.
Logging Information
2023-11-13T12:46:10.588Z DEBUG index-provider-wrapper indexprovider/wrapper.go:575 returning piece iterator {"propCid": "bafyreicpfl4qsu4y723rhdu3njkrvvx3uh74tlpxmwr56v4mzehphz242e", "piece": "baga6ea4seaqkljwlwpcjsyol3bxudck5wvzwo3zsnxfocg6ctbgk3xdlljyseiy", "err": null}
2023-11-13T12:46:17.534Z ERROR provider/engine engine/linksystem.go:129 Regeneration of entries link from multihash iterator did not match the original link. Check that multihash iterator consistently returns the same entries for the same key. {"want": "baguqeera4v4fm56ddi4hrytmkwn2e64hd6wgwmcze4s6fph36ldv55gsvnya", "got": "baguqeeraclox2bkja373gehvxzflg2nvere6dtjagf4arhfdlzlq6mas57gq"}
2023-11-13T12:46:17.535Z ERROR dagsync/ipnisync ipnisync/publisher.go:235 Failed to load requested block {"err": "regenerated link from multihash lister did not match the original link; multihashes returned by the lister for the same key are not consistent", "cid": "baguqeera4v4fm56ddi4hrytmkwn2e64hd6wgwmcze4s6fph36ldv55gsvnya"}
Repo Steps
- Run '...'
- Do '...'
- See error '...' ...
Apart from this problem, there is a second problem emerging.
Not all deals are getting announced to the network indexer.
error while ingesting ad baguqeerazpknqo4o4ok6oax7tmpib4ebekuahg2i35gvubbpex7rwy55chhq: syncEntriesErr: failed to sync first entry while checking entries type: sync handler failed: failed to traverse requested dag: failed to load node for root cid baguqeeramden5stz3kbgzb7abpl3kastbenkruh36uo372lhrv5mrpek4rta: failed to fetch block for cid baguqeeramden5stz3kbgzb7abpl3kastbenkruh36uo372lhrv5mrpek4rta: fetch request failed: Get "http://212.6.53.92:3104/ipni/v1/ad/baguqeeramden5stz3kbgzb7abpl3kastbenkruh36uo372lhrv5mrpek4rta": net/http: request canceled (Client.Timeout exceeded while awaiting headers) (19h ago)
These timeout are blocking further ingestion of ads.
To provide some more context:
__Edit removed the screenshot due sensitive data. Please refer to the ASCII below.
This example has not made it to the indexer. Checking bafkreigjnu72cdjpzfjs55xurvif2dqip62zfbwpypohcvpe4kgettvtc4 on cid.contact gives no record for the machine with ip ending on 53.92.
In ASCII for the record:
Deal 3e500432-453f-4ea8-9b6c-75c31351d831 CreatedAt 2023-11-12 21:50:01.830 (3 days ago) Client Address f1zo3n62q6j5b3rfc4drj67d5e7hr5pqtygxy6s6i Client Peer ID 12D3KooWF3msa2qui4j5TMcK4ua7ebTscGGXmZSeG1UizLcWVnqM Signed Proposal CID bafyreiak6opod5caczdl7kgtz5vl2beypxngzx7q4gz4rumonjbd2o5an4 Label bafkreigjnu72cdjpzfjs55xurvif2dqip62zfbwpypohcvpe4kgettvtc4 Verified Yes Keep Unsealed Copy Yes Announce To IPNI Yes Piece CID baga6ea4seaqivaftkm4c2ls4ikpmrnskm7oeclnzlgcetcnpiokvfq47cxgcwpa Piece Size 32 GiB (34,359,738,368 bytes) Client Collateral 0 atto Provider Collateral 0.007 FIL Storage Price / epoch / GiB 0 atto Current Epoch 3,391,587 Start Epoch 3,404,020 (in 4 days) End Epoch 4,918,900 (in a year) Duration 1,514,880 (526 days) Storage Fee 0 atto (Price per epoch x Piece Size x Duration) Transfer Mode Online Transfer Type http Transfer Size 16.5 GiB (17,723,430,495 bytes) Transfer Params URL: https://XXXXXXXXXX.twinquasar.io/nasanex/baga6ea4seaqivaftkm4c2ls4ikpmrnskm7oeclnzlgcetcnpiokvfq47cxgcwpa.car Transferred 0 B (0 bytes) Inbound File Path /root/.boost/incoming/3e500432-453f-4ea8-9b6c-75c31351d831.download Delete After Add Piece Yes Sector ID 38763 Sector Data Offset 0 Sector Data Length 34,359,738,368 Publish Message CID bafy2bzacebyqozi2pcto3kr3k33egtka6qq4ukht4htujktsp2mb6f2cvkdkm Chain Deal ID 62,966,623 Checkpoint IndexedAndAnnounced (3 days ago) Status Sealer: Proving