ParallelCurl icon indicating copy to clipboard operation
ParallelCurl copied to clipboard

clear outstanding request before invoking callback

Open redcapital opened this issue 11 years ago • 0 comments

When you put new requests into queue by calling "startRequest" inside a callback, the outstanding request never gets a chance to be cleaned up from queue and the program gets stuck. Demonstrated by this simple script:

<?
require 'parallelcurl.php';
$pc = new ParallelCurl(5);

$callback = function($content, $url, $ch, $userParams) use (&$callback, $pc) {
  // Print status of finished request
  $i = $userParams['i'];
  $httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);    
  echo "Request index {$i}, got http code: $httpcode\n";

  if ($i == 20) {
    // That's enough
    echo "Finished!\n";
    return;
  }

  // Continue with next request
  // This is a common usecase when next request url depends on contents of current page
  $pc->startRequest('example.com', $callback, ['i' => $i + 1]);
};

$pc->startRequest('example.com', $callback, ['i' => 1]);
$pc->finishAllRequests(); 

Try running it without patch (script gets stuck after 5 requests) and with patch (script correctly performs 20 requests). The proposed change fixes it by first removing outstanding request from the queue and then invoking the callback.

redcapital avatar Nov 26 '14 11:11 redcapital