By setting PROTOPT_NOURLQUERY in the protocol handler struct, the
protocol will get the "query part" of the URL cut off before the data is
handled by the protocol-specific code. This makes libcurl adhere to
RFC3986 section 2.2.
Test 1220 is added to verify a file:// URL with query-part.
After a PORT has been issued, and the multi handle would switch to the
CURLM_STATE_DO_MORE state (which is unique for FTP), libcurl would
return the wrong fdset to wait for when curl_multi_fdset() is
called. The code would blindly assume that it was waiting for a connect
of the second connection, while that isn't true immediately after the
PORT command.
Also, the function multi.c:domore_getsock() was highly FTP-centric and
therefore ugly to keep in protocol-agnostic code. I solved this problem
by introducing a new function pointer in the Curl_handler struct called
domore_getsock() which is only called during the DOMORE state for
protocols that set that pointer.
The new ftp.c:ftp_domore_getsock() function now returns fdset info about
the control connection's command/response handling while such a state is
in use, and goes over to waiting for a writable second connection first
once the commands are done.
The original problem could be seen by running test 525 and checking the
time stamps in the FTP server log. I can verify that this fix at least
fixes this problem.
Bug: http://curl.haxx.se/mail/lib-2011-10/0250.html
Reported by: Gokhan Sengun
This workarounds old libssh2 versions not properly initializing
some ssh session variables, which would trigger memory debuggers
warnings on memory being used without having been initialized.
As I modified conn->bits.tcpconnect to become an array that holds one
bool for each potential connection all uses of that struct field must
index it correctly.
Fix compiler warning: `keycheck' might be used uninitialized in this function.
Fix compiler warning: `keybit' might be used uninitialized in this function.
Introduce an INIT state for the SSH state machine and set libssh2
non-blocking in that so that it is set properly before
libssh2_session_startup() is called.
Bug: http://curl.haxx.se/mail/archive-2011-05/0001.html
Make sure that files are closed before the post quote commands run as if
they operate on the just transferred file they could otherwise easily
fail.
Patch by: Rajesh Naganathan (edited)
The PROT_* set of internal defines for the protocols is no longer
used. We now use the same bits internally as we have defined in the
public header using the CURLPROTO_ prefix. This is for simplicity and
because the PROT_* prefix was already used duplicated internally for a
set of KRB4 values.
The PROTOPT_* defines were moved up to just below the struct definition
within which they are used.
The protocol handler struct got a 'flags' field for special information
and characteristics of the given protocol.
This now enables us to move away central protocol information such as
CLOSEACTION and DUALCHANNEL from single defines in a central place, out
to each protocol's definition. It also made us stop abusing the protocol
field for other info than the protocol, and we could start cleaning up
other protocol-specific things by adding flags bits to set in the
handler struct.
The "protocol" field connectdata struct was removed as well and the code
now refers directly to the conn->handler->protocol field instead. To
make things work properly, the code now always store a conn->given
pointer that points out the original handler struct so that the code can
learn details from the original protocol even if conn->handler is
modified along the way - for example when switching to go over a HTTP
proxy.
Some protocols have to call the underlying functions without regard to
what exact state the socket signals. For example even if the socket says
"readable", the send function might need to be called while uploading,
or vice versa. This is the case for libssh2 based protocols: SCP and
SFTP and we now introduce a define to set those protocols and we make
the multi interface code aware of this concept.
This is another fix to make test 582 run properly.
For uploads we want to use the _sending_ function even when the socket
turns out readable as the underlying libssh2 sftp send function will
deal with both accordingly. This is what the cselect_bits magic is for.
Fixes test 582.
When using the multi interface and a handle using SFTP was removed very
early on, we would get a segfault due to the code assumed data was there
that hadn't yet been setup.
Bug: http://curl.haxx.se/mail/lib-2011-03/0066.html
Reported by: Saqib Ali
When using the multi_socket API to do SFTP upload, it is important that
we set a quick expire when leaving the SSH_SFTP_UPLOAD_INIT state as
there's nothing happening on the socket so there's no read or write to
wait for, but the next libssh2 API function needs to be called to get
the ball rolling.
This is one of the problems detected by test 582.
Reported by: Henry Ludemann <misc@hl.id.au>
libssh2_knownhost_readfile() returns a negative value on error or
otherwise number of parsed known hosts - this was previously not
documented correctly in the libssh2 man page for the function.
Bug: http://curl.haxx.se/mail/lib-2011-02/0327.html
Reported by: murat
Sending "pwd" as a QUOTE command only sent the reply to the
DEBUGFUNCTION. Now it also sends an FTP-like header to the header
callback to allow similar operations as with FTP, and apps can re-use
the same parser.
It helps to prevent a hangup with some FTP servers in case idle session
timeout has exceeded. But it may be useful also for other protocols
that send any quit message on disconnect. Currently used by FTP, POP3,
IMAP and SMTP.
In libssh2 1.2.8, libssh2_session_handshake() replaces
libssh2_session_startup() to fix the previous portability problem with
the socket type that was too small for win64 and thus easily could cause
crashes and more.
It is a bad idea to use the public prefix used by another library and
now we realize that libssh2 introduces a symbol in the upcoming version
1.2.8 that conflicts with our static function named libssh2_free.
As the change in 5f0ae7a062 added a precaution against negative
file sizes that for some reason managed to get returned, this change now
introduces the same check at the second place in the code where the file
size from the libssh2 stat call is used.
This check might not be suitable for a 32 bit curl_off_t, but libssh2.h
assumes long long to work and to be 64 bit so I believe such a small
curl_off_t will be very unlikely to occur in the wild.
It is still not clarified exactly why this happens, but libssh2
sometimes report a negative file size for the remote SFTP file and that
deeply confuses libcurl (or crashes it) so this precaution is added to
avoid badness.
Reported by: Ernest Beinrohr
Bug: http://curl.haxx.se/bug/view.cgi?id=3076430
libssh2 1.2.6 and later handle >32bit file sizes properly even on 32bit
architectures and we make sure to use that ability.
Reported by: Mikael Johansson
Bug: http://curl.haxx.se/mail/lib-2010-08/0052.html
Jason McDonald posted bug report #3006786 when he found that the
SFTP code didn't timeout properly in several places in the code
even if a timeout was set properly.
Based on his suggested patch, I wrote a different implementation
that I think addressed the issue better and also uses the connect
timeout for the initial part of the SSH/SFTP done during the
"protocol connect" phase.
(http://curl.haxx.se/bug/view.cgi?id=3006786)
FTP(S) use two connections that can be set to different recv and
send functions independently, so by introducing recv+send pairs
in the same manner we already have sockets/connections we can
work with FTPS fine.
This commit fixes the FTPS regression introduced in change d64bd82.
Howard Chu brought the bulk work of this patch that properly
moves out the sending and recving of data to the parts of the
code that are properly responsible for the various ways of doing
so.
Daniel Stenberg assisted with polishing a few bits and fixed some
minor flaws in the original patch.
Another upside of this patch is that we now abuse CURLcodes less
with the "magic" -1 return codes and instead use CURLE_AGAIN more
consistently.
strlen() returns size_t, but ssh libraries are wanting 'unsigned int'. Add
explicit casts and use _ex versions of the ssh library calls.
Signed-off-by: Ben Greear <greearb@candelatech.com>
much as possible in one go, as long as it doesn't block and hasn't reached the
end of the state machine.
This avoids spurious -1 returns from curl_multi_fdset() simply because
previously it would return from this function without anything in EWOUDLBLOCK
and thus basically it wasn't actually waiting for anything!!
auth is used, as it caused a crash. I failed to repeat the issue, but still
made a change that now forces the TCP connection used for a freed SCP
session to get closed and not be re-used.
transfer.c for blocking. It is currently used only by SCP and SFTP protocols.
This enhancement resolves an issue with 100% CPU usage during SFTP upload,
reported by Vourhey.
With the curl memory tracking feature decoupled from the debug build feature,
CURLDEBUG and DEBUGBUILD preprocessor symbol definitions are used as follows:
CURLDEBUG used for curl debug memory tracking specific code (--enable-curldebug)
DEBUGBUILD used for debug enabled specific code (--enable-debug)
KEEP_RECV to better match the general terminology: receive and send is what we
do from the (remote) servers. We read and write from and to the local fs.
Chen pointed out how curl couldn't upload with resume when reading from a
pipe.
This ended up with the introduction of a new return code for the
CURLOPT_SEEKFUNCTION callback that basically says that the seek failed but
that libcurl may try to resolve the situation anyway. In our case this means
libcurl will attempt to instead read that much data from the stream instead
of seeking and that way curl can now upload with resume when data is read
from a stream!
now has an improved ability to do right when the multi interface (both
"regular" and multi_socket) is used for SCP and SFTP transfers. This should
result in (much) less busy-loop situations and thus less CPU usage with no
speed loss.
operation didn't complete properly if the EAGAIN equivalent was returned but
libcurl would simply continue with a half-completed close operation
performed. This ruined persistent connection re-use and cause some
SSH-protocol errors in general. The correction is unfortunately adding a
blocking function - doing it entirely non-blocking should be considered for
a better fix.
function when built to support SCP and SFTP that helps the library to know
in which direction a particular libssh2 operation would return EAGAIN so
that libcurl knows what socket conditions to wait for before trying the
function call again. Previously (and still when using libssh2 0.18 or
earlier), libcurl will busy-loop in this situation when the easy interface
is used!
Changed checkprefix() to use it and those instances of strnequal() that
compare host names or other protocol strings that are defined to be
independent of case in the C locale. This should fix a few more
Turkish locale problems.
remain in use as internal curl_off_t print formatting strings for the internal
*printf functions which still cannot handle print formatting string directives
such as "I64d", "I64u", and others available on MSVC, MinGW, Intel's ICC, and
other DOS/Windows compilers.
This reverts previous commit part which did:
FORMAT_OFF_T -> CURL_FORMAT_CURL_OFF_T
FORMAT_OFF_TU -> CURL_FORMAT_CURL_OFF_TU
the names of the curl_off_t formatting string directives now become
CURL_FORMAT_CURL_OFF_T and CURL_FORMAT_CURL_OFF_TU.
CURL_FMT_OFF_T -> CURL_FORMAT_CURL_OFF_T
CURL_FMT_OFF_TU -> CURL_FORMAT_CURL_OFF_TU
Remove the use of an internal name for the curl_off_t formatting string directives
and use the common one available from the inside and outside of the library.
FORMAT_OFF_T -> CURL_FORMAT_CURL_OFF_T
FORMAT_OFF_TU -> CURL_FORMAT_CURL_OFF_TU
set the attribute that has changed instead of all possible ones. Hopefully,
this will solve the "Permission denied" problem that Nagarajan Sreenivasan
reported when setting some modes, but regardless, it saves a protocol
round trip in the chmod case.
CURLINFO_APPCONNECT_TIME. This is set with the "application layer"
handshake/connection is completed (typically SSL, TLS or SSH). By using this
you can figure out the application layer's own connect time. You can extract
the time stamp using curl's -w option and the new variable named
'time_appconnect'. This feature was sponsored by Lenny Rachitsky at NeuStar.
