docusaurus-plugin-remote-content icon indicating copy to clipboard operation
docusaurus-plugin-remote-content copied to clipboard

Errors on start with many repos

Open motionsuggests opened this issue 3 years ago • 2 comments

I have a docusaurus config with 20+ instances of this plugin with a varying amount of documents for each. On start it errors out.

Error

[ERROR] Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)

I'm using Constant Sync mode.

Sample of one of the configs:

'docusaurus-plugin-remote-content',
{
    // options here
    name: 'unique.name',
    sourceBaseUrl: 'github.enterprise.url', // some are public github urls
    outDir: 'docs/unique.name',
    documents: [
      'README.md',
      'docs/QuickStart.md',
    ], // most have one or two files and a couple have 15+ files
    modifyContent(filename, content) {
      // flattening the docs/ files and fixing some urls
    }
}

My guess is that this many connections isn't compatible with Constant Sync mode because of the number of connections.

My current work around is to use CLI Sync mode.

motionsuggests avatar Oct 26 '22 21:10 motionsuggests

Uh oh, that's not good. Didn't think this would ever be the case, as all documents are fetched at the same time, but I forgot about multiple instances of the plugin. I'll try to add a queue system later this week.

RDIL avatar Oct 26 '22 23:10 RDIL

I've got solution that works https://github.com/motionsuggests/docusaurus-plugin-remote-content/compare/main...motionsuggests:docusaurus-plugin-remote-content:axios-interceptors

It's a bit ugly. It uses axios interceptors to manage a queue. https://axios-http.com/docs/interceptors.

I'm looking to do another version using a proper queue https://github.com/sindresorhus/p-queue

motionsuggests avatar Nov 07 '22 22:11 motionsuggests