php curl multi 并发请求用法与原理

2020-10-26

5240 次阅读

$mh = curl_multi_init();
foreach($chs as $k => $ch){
    curl_multi_add_handle($mh, $ch);
}

// 注意以下代码在 libcurl < 7.20.0 运行良好，升级 libcurl 版本 >= 7.20.0 需要做调整、测试、开发
// https://curl.haxx.se/libcurl/c/libcurl-errors.html
// Before version 7.20.0 this could be returned by curl_multi_perform, but in later versions this return code is never used.

$active = null;
do {
    $mrc = curl_multi_exec($mh, $active);
    if ($mrc > 0) throw new Exception(curl_multi_strerror($mrc));
} while ($mrc == CURLM_CALL_MULTI_PERFORM);

// https://bugs.php.net/bug.php?id=61141:
// amoo_miki at yahoo dot com
while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($mh) == -1) {
        usleep(100);
    }
    do {
        $mrc = curl_multi_exec($mh, $active);
        if ($mrc > 0) throw new Exception(curl_multi_strerror($mrc));
    } while ($mrc == CURLM_CALL_MULTI_PERFORM);
}

$result = [];
foreach($chs as $k => $ch){
    $result[$k] = curl_multi_getcontent($ch);
    curl_multi_remove_handle($mh, $ch);
}
curl_multi_close($mh);

我的理解

以上是使用 curl multi 示例，总的来说，这个用法让人感到很疑惑。

特别是 usleep(100) 部分，pierrick@php.net 回复了这块原因：

When max_fd returns with -1, you need to wait a while and then
proceed and call curl_multi_perform anyway. How long to wait? I would suggest
100 milliseconds at least, but you may want to test it out in your own
particular conditions to find a suitable value.

参见 https://bugs.php.net/bug.php?id=61141

关于 curl multi 的并发原理，通过查阅资料，记录一下我的理解：
curl_muti_* 函数族，调用 libcurl-multi 相关 API，利用 select I/O 多路复用机制，实现并发请求。