mirror of
https://github.com/moparisthebest/curl
synced 2024-12-22 08:08:50 -05:00
0f147887b0
Introducing a number of options to the multi interface that allows for multiple pipelines to the same host, in order to optimize the balance between the penalty for opening new connections and the potential pipelining latency. Two new options for limiting the number of connections: CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections to the same host. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished, so we can reuse the connection. CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished. The free connection will then be reused, if possible, or closed if the pending handle can't reuse it. Several new options for pipelining: CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a pipeline is "full" when a connection is to be reused, a new connection will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it. If not, the handle will be put in a pending state until a connection is ready (either free or a pipe got shorter). CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a transfer with a content length that is larger than this. CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a chunk larger than this. CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow pipelining. CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow pipelining. See the curl_multi_setopt() man page for details.
45 lines
1.8 KiB
Plaintext
45 lines
1.8 KiB
Plaintext
HTTP Pipelining with libcurl
|
|
============================
|
|
|
|
Background
|
|
|
|
Since pipelining implies that one or more requests are sent to a server before
|
|
the previous response(s) have been received, we only support it for multi
|
|
interface use.
|
|
|
|
Considerations
|
|
|
|
When using the multi interface, you create one easy handle for each transfer.
|
|
Bascially any number of handles can be created, added and used with the multi
|
|
interface - simultaneously. It is an interface designed to allow many
|
|
simultaneous transfers while still using a single thread. Pipelining does not
|
|
change any of these details.
|
|
|
|
API
|
|
|
|
We've added a new option to curl_multi_setopt() called CURLMOPT_PIPELINING
|
|
that enables "attempted pipelining" and then all easy handles used on that
|
|
handle will attempt to use an existing pipeline.
|
|
|
|
Details
|
|
|
|
- A pipeline is only created if a previous connection exists to the same IP
|
|
address that the new request is being made to use.
|
|
|
|
- Pipelines are only supported for HTTP(S) as no other currently supported
|
|
protocol has features resemembling this, but we still name this feature
|
|
plain 'pipelining' to possibly one day support it for other protocols as
|
|
well.
|
|
|
|
- HTTP Pipelining is for GET and HEAD requests only.
|
|
|
|
- When a pipeline is in use, we must take precautions so that when used easy
|
|
handles (i.e those who still wait for a response) are removed from the multi
|
|
handle, we must deal with the outstanding response nicely.
|
|
|
|
- Explicitly asking for pipelining handle X and handle Y won't be supported.
|
|
It isn't easy for an app to do this association. The lib should probably
|
|
still resolve the second one properly to make sure that they actually _can_
|
|
be considered for pipelining. Also, asking for explicit pipelining on handle
|
|
X may be tricky when handle X get a closed connection.
|