opensearchserver-php-client icon indicating copy to clipboard operation
opensearchserver-php-client copied to clipboard

Sending patterns and urls to the instance does not mean they will be crawled?

Open scaraliu opened this issue 7 years ago • 0 comments

When sending the allowed patterns and the URL list to the instance/index, does not start the crawl process on that list. I had to use the OpenSearchServer\Crawler\Web\Crawl() request to actually start the crawl process on that list of URLs. Just injecting the urls and setting the allowed patterns doesn't guarantee that those URL will be processed as configured in the running index.

I thought just by injecting urls and setting them as allowed patterns, the instance will process them. So, to index some urls I have to do as follows:

  • Inject url list OpenSearchServer\Document\Put();
  • Inject those URLs as allowed path/s OpenSearchServer\Crawler\Web\Patterns\Inclusion\Insert();
  • and also to do a crawl request on each URL OpenSearchServer\Crawler\Web\Crawl();

Am I missing something?

scaraliu avatar Jan 21 '19 22:01 scaraliu