Chris
Chris White Web Developer

Fire and Forget HTTP Requests in PHP

25 January 2019 ~2 minute read

I came across an interesting problem recently while writing some code that needed to send a high volume of HTTP requests but didn't care about the response, or even whether they were received with a 100% success rate. Sitting around and waiting for the HTTP response adds a lot of time especially when compounded over thousands of requests.

It would be ideal if we could fire off the HTTP request and continue processing the script without waiting for the response. Unfortunately PHP doesn’t have a convenient way to do this. None of the options that you can pass into the cURL extension allow PHP to continue processing before a response is returned. Guzzle does have the concept of async requests but they don’t work the way you might think. They do still wait for a response to be returned before continuing execution, but they do allow you to have multiple requests in-flight at once. See this issue for more info.

Eventually I stumbled upon a similar solution that uses sockets to send data to a remote server over UDP, but managed to use the same concept for a HTTP request over a TCP connection. Example below:

1$endpoint = 'https://enqx4fd1n8rn.x.pipedream.net';
2$postData = '{"foo": "bar"}';
3 
4$endpointParts = parse_url($endpoint);
5$endpointParts['path'] = $endpointParts['path'] ?? '/';
6$endpointParts['port'] = $endpointParts['port'] ?? $endpointParts['scheme'] === 'https' ? 443 : 80;
7 
8$contentLength = strlen($postData);
9 
10$request = "POST {$endpointParts['path']} HTTP/1.1\r\n";
11$request .= "Host: {$endpointParts['host']}\r\n";
12$request .= "User-Agent: My application v2.2.0\r\n";
13$request .= "Authorization: Bearer api_key\r\n";
14$request .= "Content-Length: {$contentLength}\r\n";
15$request .= "Content-Type: application/json\r\n\r\n";
16$request .= $postData;
17 
18$prefix = substr($endpoint, 0, 8) === 'https://' ? 'tls://' : '';
19 
20$socket = fsockopen($prefix.$endpointParts['host'], $endpointParts['port']);
21fwrite($socket, $request);
22fclose($socket);

This will send the JSON payload defined in the $postData variable to the endpoint specified in $endpoint. It also figures out the correct socket prefix, path, and port to use based on the endpoint given. As you can see, it just writes the HTTP request to the socket and then closes it and carries on.

Hand-crafting HTTP requests seemed like an unreliable method at first, but after some pretty extensive testing I can vouch for it reliably sending the requests and the remote server receiving them in full. It can be tricky to craft the request properly (respecting the line feeds that the HTTP specification expects, etc), but that’s why you have tests, right? 🙂

If you’re interested in benchmarks, I did a quick test between this method and using Guzzle to send 50 HTTP requests. Turns out to be almost a 75% decrease!

Guzzle 6 Sockets
16.49 seconds 4.28 seconds

You might be expecting it to be faster – but keep in mind that PHP still has to establish the TCP connection and negotiate the TLS handshake before it can send the request. But at least you cut out the time it takes for the remote server to prepare and send its response. The end-all solution might be to run a UDP server and send the data down a UDP socket.

Made with Jigsaw and Torchlight.