1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-17 23:15:08 -05:00
Commit Graph

192 Commits

Author SHA1 Message Date
Linus Nielsen Feltzing
0f147887b0 Multiple pipelines and limiting the number of connections.
Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
optimize the balance between the penalty for opening new
connections and the potential pipelining latency.

Two new options for limiting the number of connections:

CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections
to the same host. When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished, so we can reuse the connection.

CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total.
When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished. The free connection will then be reused, if possible, or
closed if the pending handle can't reuse it.

Several new options for pipelining:

CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a
pipeline is "full" when a connection is to be reused, a new connection
will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it.
If not, the handle will be put in a pending state until a connection is
ready (either free or a pipe got shorter).

CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a transfer with a content
length that is larger than this.

CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a chunk larger than this.

CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow
pipelining.

CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow
pipelining.

See the curl_multi_setopt() man page for details.
2013-03-13 23:55:24 +01:00
Yang Tse
5a053ffe80 build: fix circular header inclusion with other packages
This commit renames lib/setup.h to lib/curl_setup.h and
renames lib/setup_once.h to lib/curl_setup_once.h.

Removes the need and usage of a header inclusion guard foreign
to libcurl. [1]

Removes the need and presence of an alarming notice we carried
in old setup_once.h [2]

----------------------------------------

1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard
    up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H,
    this single inclusion guard is enough to ensure that inclusion of
    lib/setup_once.h done from lib/setup.h is only done once.

    Additionally lib/setup.h has always used __SETUP_ONCE_H macro to
    protect inclusion of setup_once.h even after commit ec691ca3, this
    was to avoid a circular header inclusion triggered when building a
    c-ares enabled version with c-ares sources available which also has
    a setup_once.h header. Commit ec691ca3 exposes the real nature of
    __SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard
    foreign to libcurl belonging to c-ares's setup_once.h

    The renaming this commit does, fixes the circular header inclusion,
    and as such removes the need and usage of a header inclusion guard
    foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl.

2 - Due to the circular interdependency of old lib/setup_once.h and the
    c-ares setup_once.h header, old file lib/setup_once.h has carried
    back from 2006 up to now days an alarming and prominent notice about
    the need of keeping libcurl's and c-ares's setup_once.h in sync.

    Given that this commit fixes the circular interdependency, the need
    and presence of mentioned notice is removed.

    All mentioned interdependencies come back from now old days when
    the c-ares project lived inside a curl subdirectory. This commit
    removes last traces of such fact.
2013-01-09 00:49:50 +01:00
Yang Tse
4a5aa6682d Revert changes relative to lib/*.[ch] recent renaming
This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:

  f871de0... build: make use of 76 lib/*.h renamed files
  ffd8e12... build: rename 76 lib/*.h files

This also reverts removal of redundant include guard (redundant thanks
to changes in above commits) done 2-12-2013, reverting 1 commit:

  c087374... curl_setup.h: remove redundant include guard

This also reverts renaming and usage of lib/*.c source files done
3-12-2013, reverting 3 commits:

  13606bb... build: make use of 93 lib/*.c renamed files
  5b6e792... build: rename 93 lib/*.c files
  7d83dff... build: commit 13606bbfde follow-up 1

Start of related discussion thread:

  http://curl.haxx.se/mail/lib-2013-01/0012.html

Asking for confirmation on pushing this revertion commit:

  http://curl.haxx.se/mail/lib-2013-01/0048.html

Confirmation summary:

  http://curl.haxx.se/mail/lib-2013-01/0079.html

NOTICE: The list of 2 files that have been modified by other
intermixed commits, while renamed, and also by at least one
of the 6 commits this one reverts follows below. These 2 files
will exhibit a hole in history unless git's '--follow' option
is used when viewing logs.

  lib/curl_imap.h
  lib/curl_smtp.h
2013-01-06 18:20:27 +01:00
Yang Tse
5b6e7927c6 build: rename 93 lib/*.c files
93 lib/*.c source files renamed to use our standard naming scheme.

This commit only does the file renaming.

----------------------------------------

  renamed:    lib/amigaos.c -> lib/curl_amigaos.c
  renamed:    lib/asyn-ares.c -> lib/curl_asyn_ares.c
  renamed:    lib/asyn-thread.c -> lib/curl_asyn_thread.c
  renamed:    lib/axtls.c -> lib/curl_axtls.c
  renamed:    lib/base64.c -> lib/curl_base64.c
  renamed:    lib/bundles.c -> lib/curl_bundles.c
  renamed:    lib/conncache.c -> lib/curl_conncache.c
  renamed:    lib/connect.c -> lib/curl_connect.c
  renamed:    lib/content_encoding.c -> lib/curl_content_encoding.c
  renamed:    lib/cookie.c -> lib/curl_cookie.c
  renamed:    lib/cyassl.c -> lib/curl_cyassl.c
  renamed:    lib/dict.c -> lib/curl_dict.c
  renamed:    lib/easy.c -> lib/curl_easy.c
  renamed:    lib/escape.c -> lib/curl_escape.c
  renamed:    lib/file.c -> lib/curl_file.c
  renamed:    lib/fileinfo.c -> lib/curl_fileinfo.c
  renamed:    lib/formdata.c -> lib/curl_formdata.c
  renamed:    lib/ftp.c -> lib/curl_ftp.c
  renamed:    lib/ftplistparser.c -> lib/curl_ftplistparser.c
  renamed:    lib/getenv.c -> lib/curl_getenv.c
  renamed:    lib/getinfo.c -> lib/curl_getinfo.c
  renamed:    lib/gopher.c -> lib/curl_gopher.c
  renamed:    lib/gtls.c -> lib/curl_gtls.c
  renamed:    lib/hash.c -> lib/curl_hash.c
  renamed:    lib/hmac.c -> lib/curl_hmac.c
  renamed:    lib/hostasyn.c -> lib/curl_hostasyn.c
  renamed:    lib/hostcheck.c -> lib/curl_hostcheck.c
  renamed:    lib/hostip.c -> lib/curl_hostip.c
  renamed:    lib/hostip4.c -> lib/curl_hostip4.c
  renamed:    lib/hostip6.c -> lib/curl_hostip6.c
  renamed:    lib/hostsyn.c -> lib/curl_hostsyn.c
  renamed:    lib/http.c -> lib/curl_http.c
  renamed:    lib/http_chunks.c -> lib/curl_http_chunks.c
  renamed:    lib/http_digest.c -> lib/curl_http_digest.c
  renamed:    lib/http_negotiate.c -> lib/curl_http_negotiate.c
  renamed:    lib/http_negotiate_sspi.c -> lib/curl_http_negotiate_sspi.c
  renamed:    lib/http_proxy.c -> lib/curl_http_proxy.c
  renamed:    lib/idn_win32.c -> lib/curl_idn_win32.c
  renamed:    lib/if2ip.c -> lib/curl_if2ip.c
  renamed:    lib/imap.c -> lib/curl_imap.c
  renamed:    lib/inet_ntop.c -> lib/curl_inet_ntop.c
  renamed:    lib/inet_pton.c -> lib/curl_inet_pton.c
  renamed:    lib/krb4.c -> lib/curl_krb4.c
  renamed:    lib/krb5.c -> lib/curl_krb5.c
  renamed:    lib/ldap.c -> lib/curl_ldap.c
  renamed:    lib/llist.c -> lib/curl_llist.c
  renamed:    lib/md4.c -> lib/curl_md4.c
  renamed:    lib/md5.c -> lib/curl_md5.c
  renamed:    lib/memdebug.c -> lib/curl_memdebug.c
  renamed:    lib/mprintf.c -> lib/curl_mprintf.c
  renamed:    lib/multi.c -> lib/curl_multi.c
  renamed:    lib/netrc.c -> lib/curl_netrc.c
  renamed:    lib/non-ascii.c -> lib/curl_non_ascii.c
  renamed:    lib/curl_non-ascii.h -> lib/curl_non_ascii.h
  renamed:    lib/nonblock.c -> lib/curl_nonblock.c
  renamed:    lib/nss.c -> lib/curl_nss.c
  renamed:    lib/nwlib.c -> lib/curl_nwlib.c
  renamed:    lib/nwos.c -> lib/curl_nwos.c
  renamed:    lib/openldap.c -> lib/curl_openldap.c
  renamed:    lib/parsedate.c -> lib/curl_parsedate.c
  renamed:    lib/pingpong.c -> lib/curl_pingpong.c
  renamed:    lib/polarssl.c -> lib/curl_polarssl.c
  renamed:    lib/pop3.c -> lib/curl_pop3.c
  renamed:    lib/progress.c -> lib/curl_progress.c
  renamed:    lib/qssl.c -> lib/curl_qssl.c
  renamed:    lib/rawstr.c -> lib/curl_rawstr.c
  renamed:    lib/rtsp.c -> lib/curl_rtsp.c
  renamed:    lib/security.c -> lib/curl_security.c
  renamed:    lib/select.c -> lib/curl_select.c
  renamed:    lib/sendf.c -> lib/curl_sendf.c
  renamed:    lib/share.c -> lib/curl_share.c
  renamed:    lib/slist.c -> lib/curl_slist.c
  renamed:    lib/smtp.c -> lib/curl_smtp.c
  renamed:    lib/socks.c -> lib/curl_socks.c
  renamed:    lib/socks_gssapi.c -> lib/curl_socks_gssapi.c
  renamed:    lib/socks_sspi.c -> lib/curl_socks_sspi.c
  renamed:    lib/speedcheck.c -> lib/curl_speedcheck.c
  renamed:    lib/splay.c -> lib/curl_splay.c
  renamed:    lib/ssh.c -> lib/curl_ssh.c
  renamed:    lib/sslgen.c -> lib/curl_sslgen.c
  renamed:    lib/ssluse.c -> lib/curl_ssluse.c
  renamed:    lib/strdup.c -> lib/curl_strdup.c
  renamed:    lib/strequal.c -> lib/curl_strequal.c
  renamed:    lib/strerror.c -> lib/curl_strerror.c
  renamed:    lib/strtok.c -> lib/curl_strtok.c
  renamed:    lib/strtoofft.c -> lib/curl_strtoofft.c
  renamed:    lib/telnet.c -> lib/curl_telnet.c
  renamed:    lib/tftp.c -> lib/curl_tftp.c
  renamed:    lib/timeval.c -> lib/curl_timeval.c
  renamed:    lib/transfer.c -> lib/curl_transfer.c
  renamed:    lib/url.c -> lib/curl_url.c
  renamed:    lib/version.c -> lib/curl_version.c
  renamed:    lib/warnless.c -> lib/curl_warnless.c
  renamed:    lib/wildcard.c -> lib/curl_wildcard.c

----------------------------------------
2013-01-03 06:13:18 +01:00
Yang Tse
13606bbfde build: make use of 93 lib/*.c renamed files
93 *.c source files renamed to use our standard naming scheme.

This change affects 77 files in libcurl's source tree.
2013-01-03 05:50:26 +01:00
Yang Tse
f871de0064 build: make use of 76 lib/*.h renamed files
76 private header files renamed to use our standard naming scheme.

This change affects 322 files in libcurl's source tree.
2012-12-28 19:37:11 +01:00
Yang Tse
a0b207164c setup_once.h: refactor inclusion of <unistd.h> and <sys/socket.h>
Inclusion of top two most included header files now done in setup_once.h
2012-12-14 17:38:18 +01:00
Daniel Stenberg
7ecd874bce Curl_write: remove unneeded typecast 2012-11-12 10:04:31 +01:00
Yang Tse
d9f686db88 remove short-lived CURL_WRITEFUNC_OUT_OF_MEMORY 2011-09-26 13:05:42 +02:00
Yang Tse
119f43360b allow write callbacks to indicate OOM to libcurl
Allow (*curl_write_callback) write callbacks to return
CURL_WRITEFUNC_OUT_OF_MEMORY to properly indicate libcurl of OOM conditions
inside the callback itself.
2011-09-25 19:05:46 +02:00
Yang Tse
a50210710a fix bool variables checking and assignment 2011-09-05 20:46:09 +02:00
Daniel Stenberg
0fd2bf3726 Curl_read_plain: indent code 2011-08-26 23:20:27 +02:00
Yang Tse
f1586cb477 stdio.h, stdlib.h, string.h, stdarg.h and ctype.h inclusion done in setup_once.h 2011-07-26 17:23:27 +02:00
Yang Tse
ef2176109f errno.h inclusion conditionally done in setup_once.h 2011-07-24 04:39:43 +02:00
Yang Tse
d30ddd9977 compiler warning: fix
Fix compiler warning: enumerated type mixed with another type
2011-05-21 14:55:10 +02:00
Daniel Stenberg
9c629e5348 RTSP: cleanups
Made several functions static

Made one function defined to nothing when RTSP is disabled to avoid
the #ifdefs in code.

Removed explicit rtsp.h includes
2011-05-05 16:53:05 +02:00
Daniel Stenberg
b903186fa0 source cleanup: unify look, style and indent levels
By the use of a the new lib/checksrc.pl script that checks that our
basic source style rules are followed.
2011-04-27 09:09:35 +02:00
Fabian Keil
1702a2c08d Fix a couple of spelling errors in lib/
Found with codespell.
2011-04-21 07:55:53 -07:00
Daniel Stenberg
c828646f60 CURL_DOES_CONVERSIONS: cleanup
Massively reduce #ifdefs all over (23 #ifdef lines less so far)
Moved conversion-specific code to non-ascii.c
2011-04-20 00:50:07 +02:00
Daniel Stenberg
13b64d7558 protocols: use CURLPROTO_ internally
The PROT_* set of internal defines for the protocols is no longer
used. We now use the same bits internally as we have defined in the
public header using the CURLPROTO_ prefix. This is for simplicity and
because the PROT_* prefix was already used duplicated internally for a
set of KRB4 values.

The PROTOPT_* defines were moved up to just below the struct definition
within which they are used.
2011-03-14 22:52:14 +01:00
Daniel Stenberg
8831000bc0 protocol handler: added flags field
The protocol handler struct got a 'flags' field for special information
and characteristics of the given protocol.

This now enables us to move away central protocol information such as
CLOSEACTION and DUALCHANNEL from single defines in a central place, out
to each protocol's definition. It also made us stop abusing the protocol
field for other info than the protocol, and we could start cleaning up
other protocol-specific things by adding flags bits to set in the
handler struct.

The "protocol" field connectdata struct was removed as well and the code
now refers directly to the conn->handler->protocol field instead. To
make things work properly, the code now always store a conn->given
pointer that points out the original handler struct so that the code can
learn details from the original protocol even if conn->handler is
modified along the way - for example when switching to go over a HTTP
proxy.
2011-03-14 22:22:22 +01:00
Daniel Stenberg
c2bfe60086 Curl_send/recv_plain: return errno on failure
When send() and recv() fail, we now store the errno value to allow the
app to access it.

Bug: http://curl.haxx.se/bug/view.cgi?id=3128121
Reported by: Yuri
2010-12-05 23:04:11 +01:00
Daniel Stenberg
1b15b31c86 sendrecv: treat all negative values from send/recv as errors
For example the libssh2 based functions return other negative
values than -1 to signal errors and it is important that we catch
them properly. Right before this, various failures from libssh2
were treated as negative download amounts which caused havoc.
2010-06-19 00:18:14 +02:00
Howard Chu
bc8fc9803f sendrecv: make them two pairs of send/recv to properly deal with FTPS
FTP(S) use two connections that can be set to different recv and
send functions independently, so by introducing recv+send pairs
in the same manner we already have sockets/connections we can
work with FTPS fine.

This commit fixes the FTPS regression introduced in change d64bd82.
2010-05-11 22:48:38 +02:00
Howard Chu
d64bd82bdc sendrecv: split the I/O handling into private handler
Howard Chu brought the bulk work of this patch that properly
moves out the sending and recving of data to the parts of the
code that are properly responsible for the various ways of doing
so.

Daniel Stenberg assisted with polishing a few bits and fixed some
minor flaws in the original patch.

Another upside of this patch is that we now abuse CURLcodes less
with the "magic" -1 return codes and instead use CURLE_AGAIN more
consistently.
2010-05-07 15:05:34 +02:00
Kamil Dudka
10977f57de qssl: reflect recent code changes in SSL interface
Reported by Guenter Knauf.
2010-04-07 10:47:19 +02:00
Kamil Dudka
ff8711135e refactorize interface of Curl_ssl_recv/Curl_ssl_send 2010-04-04 23:37:18 +02:00
Daniel Stenberg
49f3160d69 fix SFTP download hang
Matt Wixson found and fixed a bug in the SCP/SFTP area where the
code treated a 0 return code from libssh2 to be the same as
EAGAIN while in reality it isn't. The problem caused a hang in
SFTP transfers from a MessageWay server.
2010-04-01 22:39:15 +02:00
Daniel Stenberg
2309b4e330 remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
Kamil Dudka
19ca0c0fbe - Improved Curl_read() to not ignore the error returned from Curl_ssl_recv(). 2010-03-19 15:43:11 +00:00
Yang Tse
7aef172a34 fix printf-style format strings 2010-02-04 19:44:31 +00:00
Yang Tse
d1717e7c90 Fix compiler warning: conditional expression is constant 2010-02-02 16:25:07 +00:00
Daniel Stenberg
33ce0ec1f1 wrap long lines and do some indent policing 2010-01-22 23:21:39 +00:00
Daniel Stenberg
bc4582b68a Chris Conroy brought support for RTSP transfers, and with it comes 8(!) new
libcurl options for controlling what to get and how to receive posssibly
interleaved RTP data. Initial commit.
2010-01-21 13:58:30 +00:00
Daniel Stenberg
352177090f - Fabian Keil ran clang on the (lib)curl code, found a bunch of warnings and
contributed a range of patches to fix them.
2009-06-10 21:26:11 +00:00
Daniel Stenberg
3aa3d7e629 Internal cleanup: KEEP_WRITE and KEEP_READ are now called KEEP_SEND and
KEEP_RECV to better match the general terminology: receive and send is what we
do from the (remote) servers. We read and write from and to the local fs.
2009-05-11 07:53:38 +00:00
Yang Tse
33a3753c3f libcurl's memory.h renamed to curl_memory.h 2009-04-21 11:46:16 +00:00
Daniel Stenberg
bdec6f2b20 - Frank Hempel found out a bug and provided the fix:
curl_easy_duphandle did not necessarily duplicate the CURLOPT_COOKIEFILE
  option. It only enabled the cookie engine in the destination handle if
  data->cookies is not NULL (where data is the source handle). In case of a
  newly initialized handle which just had the cookie support enabled by a
  curl_easy_setopt(handle, CURL_COOKIEFILE, "")-call, handle->cookies was
  still NULL because the setopt-call only appends the value to
  data->change.cookielist, hence duplicating this handle would not have the
  cookie engine switched on.

  We also concluded that the slist-functionality would be suitable for being
  put in its own module rather than simply hanging out in lib/sendf.c so I
  created lib/slist.[ch] for them.
2009-03-09 12:21:46 +00:00
Dan Fandrich
80ffd3581f Created a CURLMIN macro to match CURLMAX 2009-01-07 19:39:35 +00:00
Daniel Stenberg
c76d939563 Christian Krause fixed a build failure when building with gss support
enabled and FTP disabled.
2008-11-16 12:42:53 +00:00
Michal Marek
c331c73ec6 - Fixed a potential data loss in Curl_client_write() when the transfer is
paused.
2008-11-13 08:20:23 +00:00
Dan Fandrich
484d549ece Marked with TODO comments a number of problems in the Kerberos code detected
while investigating the issue in http://curl.haxx.se/mail/lib-2008-09/0262.html
I'm hesitant to fix them because I have no way of testing the result.
2008-11-02 05:01:39 +00:00
Dan Fandrich
7fc4e8af0a Changed some arrays of char* to arrays of char[] to reduce data size and
run-time relocations.
2008-10-22 05:46:29 +00:00
Yang Tse
d61688923d fix compiler warning: enumerated type mixed with another type 2008-09-30 12:50:52 +00:00
Daniel Stenberg
aff5408633 made Curl_read_plain() return an 'int' instead of CURLcode since it actually
returns -1 in EAGAIN cases and that's not valid CURLcode
2008-09-29 11:13:37 +00:00
Daniel Stenberg
391e8afd1f - Made the SOCKS code use the new Curl_read_plain() function to fix the bug
Markus Moeller reported: http://curl.haxx.se/mail/archive-2008-09/0016.html

- recv() errors other than those equal to EAGAIN now cause proper
  CURLE_RECV_ERROR to get returned. This made test case 160 fail so I've now
  disabled it until we can figure out another way to exercise that logic.
2008-09-22 23:12:00 +00:00
Yang Tse
59e378f48f remove unnecessary typecasting of malloc() 2008-09-06 05:29:05 +00:00
Dan Fandrich
3acd1146f9 Use realloc when paused in Curl_client_write 2008-09-05 17:21:54 +00:00
Daniel Stenberg
4c9768565e - Introducing CURLOPT_CERTINFO and the corresponding CURLINFO_CERTINFO. By
enabling this feature with CURLOPT_CERTINFO for a request using SSL (HTTPS
  or FTPS), libcurl will gather lots of server certificate info and that info
  can then get extracted by a client after the request has completed with
  curl_easy_getinfo()'s CURLINFO_CERTINFO option. Linus Nielsen Feltzing
  helped me test and smoothen out this feature.

  Unfortunately, this feature currently only works with libcurl built to use
  OpenSSL.

  This feature was sponsored by networking4all.com - thanks!
2008-09-05 14:29:21 +00:00
Dan Fandrich
bb67388bbe Made some variables const 2008-09-04 19:43:35 +00:00