When a package is already partially downloaded in the cache, its download
size will only be of what's left to be downloaded. Since pkg->download_size
is what's used when calculating the total download size for the totaldl
callback, same thing apply.
However, the download progress callback was including this initial size,
which would thus lead to invalid values (and percentage) used in frontends.
That is, the progress bar could e.g. go further than 100%
In the case of pacman, there is a sanity check for different historical
reason (44a57c89), so before the possible "overflow" was noticed, the total
download size/progress reported was wrong. Once caught, the TotalDownload
option was ignored and it would use individual file download values as
fallback instead.
Signed-off-by: Olivier Brunel <jjk@jjacky.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Use of this flag causes connections to be closed on 404s -- a common
occurrence when your config sets DatabaseOptional. Handle the error
gracefully, so that the connection can be reused.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Forcing vim users to view files with a tabstop of 2 seems really
unnecessary when noet is set. I find it much easier to read code with
ts=4 and I dislike having to override the modeline by hand.
Command run:
find . -type f -exec sed -i '/vim.* noet/s# ts=2 sw=2##' {} +
Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Allan McRae <allan@archlinux.org>
Previously, we only allowed the default of responding to basic auth
challenges. Mirrors requiring authorization are far and away the edge
case, but there's no sense in preventing access to them.
Implements FS#38184.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
This commit:
-- replaces space-based indents with tabs per the coding standards
-- removes extraneous whitespace (e.g. extra spaces between function args)
-- adds missing braces for a one-line if statement
Signed-off-by: Jason St. John <jstjohn@purdue.edu>
If the server redirects from ${repo}.db to ${repo}.db.tar.gz pacman gets
this wrong: It saves to new filename and fails when accessing
${repo}.db.
We need the remote filename only when downloading remote files with
pacman's -U operation. This introduces a new field 'trust_remote_name'
to payload. If set pacman downloads to the filename given by the server.
The field trust_remote_name is set in alpm_fetch_pkgurl().
Fixes FS#36791 ([pacman] downloads to wrong filename with redirect).
[dave: remove redundant assignment leading to memory leak]
Signed-off-by: Allan McRae <allan@archlinux.org>
These references to bug numbers assume we will forever be using that bug
tracker. It is better to properly comment the code instead (which was
done in almost all cases anyway).
Signed-off-by: Allan McRae <allan@archlinux.org>
On operating systems we support, the behavior is always such that the
kernel will do the right thing as far as invalidating the file
descriptor, regardless of the eventual return value. Therefore,
potentially looping and calling close multiple times is wrong.
At best, we call close again on an invalid FD and throw a spurious EBADF
error. At worst, we might close an FD which doesn't belong to us when a
multi-threaded application opens its own file descriptor between
iterations of the loop.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
I suspect that eventually we're going to end up returning a pointer to
an allocated struct to describe the download result, but that's for
another patch when the need arises...
Fixes FS#33508.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Users have hit issues behind corporate firewalls that initially throttle
downloads to ~1B/sec.
Signed-off-by: Olivier Langlois < olivier.pis.langlois@transport.alstom.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
RFC 2616 doesn't forbid a 301 or 302 repsonse from having a body, and
servers exist in the wild that show this behavior. In order to prevent
pacman from showing a progress bar when we aren't actually downloading a
package (and merely following one of these pain in the butt redirects),
capture the server response code in the response header, rather than
waiting to peel it off the handle after the download has finished.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Reported-by: Alexandre Filgueira <alexfilgueira@cinnarch.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Apparently gcc 4.7 has decided that -Wshadow warnings aren't worth
reporting anymore even with the flag enabled. These were found on
an Ubuntu 10.04 install.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Detected by clang scan-build static code analyzer.
* Don't attempt to free an uninitialized gpgme key variable
* Initialize answer variable before asking frontend a question
* Pass by reference instead of value if uninitialized fields are
possible in download signal handler code
* Ensure we never call strlen() on NULL payload->remote_name value
Signed-off-by: Dan McGee <dan@archlinux.org>
To avoid conflicts on reusing a payload after a failed download, ensure
that we reset the filename hints in the payload struct prior to the
download operation.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
It can happen that the to-be-downloaded file cannot be created in cachedir.
For example, I am an -Sup user, and it is comfortable to set --cachedir to
/mnt/pendrive, which is a FAT filesystem, so files like
capseo-1:0.3-2-i686.pkg.tar.xz cannot be downloaded to there.
Before this patch, pacman didn't give clear output about what happens when
the download code could not create the necessary file. This can be confusing
with -Su. An example output:
***
$ sudo pacman -S capseo bochs --cachedir /c/TEMP
resolving dependencies...
looking for inter-conflicts...
Targets (2): bochs-2.4.6-1 capseo-1:0.3-2
Total Download Size: 0.61 MiB
Total Installed Size: 2.61 MiB
Proceed with installation? [Y/n]
:: Retrieving packages from extra...
warning: failed to retrieve some files from extra
bochs-2.4.6-1-i686 611.5 KiB 118K/s 00:05 [------------------] 97%
error: failed to commit transaction (unexpected error)
Errors occurred, no packages were upgraded.
***
After the patch, pacman will give more informative error message (and
pm_errno is set properly):
***
error: could not open file '/c/TEMP/capseo-1:0.3-2-i686.pkg.tar.xz.part': Invalid argument
error: failed to commit transaction (failed to retrieve some files)
***
Unfortunately, the "could not open file" error message is printed for
every mirror (that can be dozens of lines), which is ugly, but at least
informative... Without modifying the download logic (for example, by
introducing -2 return value for _alpm_download() to indicate giving up),
this ugliness cannot be eliminated.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is particularly important in the case of FTP control connections,
which may be closed by rogue NAT/firewall devices detecting idle
connections on larger transfers which may take 5-10+ minutes.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We lost this logic somewhere between the libfetch and libcurl
transition, as it existed in the internal downloader, but was pulled
back only into the sync workflow. Add a helper function that will let us
check for existance in the filecache prior to calling the downloader.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Ensures that config.h is always ordered correctly (first) in the
includes. Also means that new source files get this for free without
having to remember to add it.
We opt for -imacros over -include as its more portable, and the
added constraint by -imacros doesn't bother us for config.h.
This also touches the HACKING file to remove the explicit mention of
config.h as part of the includes.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We had a 16 KiB limit on database signatures, we should do the same here
too to have a slight sanity check, even if we can't do so for the
package itself yet.
Signed-off-by: Dan McGee <dan@archlinux.org>
Replacing the strdup when after the first NULL check assures that we get
continue with payload->remote_name defined.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Dan: fix mask calculation, add it to the success/fail block instead.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This adds a logger to the CURLE_OK case so we can always know the return
code if it was >= 400, and debug log it regardless. Also adjust another
logger to use the cURL error message directly, as well as use fstat()
when we have an open file handle rather than stat().
Signed-off-by: Dan McGee <dan@archlinux.org>
This is in the realm of "probably not going to happen", but if someone
were to translate "disk" to a string longer than 256 characters, we
would have a smashed/corrupted stack due to our unchecked strcpy() call.
Rework the function to always length-check the value we copy into the
hostname buffer, and do it with memcpy rather than the more cumbersome
and unnecessary snprintf.
Finally, move the magic 256 value into a constant and pass it into the
function which is going to get inlined anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
Similar to what we did in edd9ed6a, disconnect the relationship with our
stack allocated error buffer from the curl handle. Just as an FTP
connection might have some network chatter on teardown causing the
progress callback to be triggered, we might also hit an error condition
that causes curl to write to our (now out of scope) error buffer.
I'm unable to reproduce FS#26327, but I have a suspicion that this
should fix it.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a poor place for it, and it will likely move again in the
future, but it's better to have it here than as a static variable.
Initialization of this variable is now no longer necessary as its
zeroed on creation of the payload struct.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This was done to squash a memory leak in the sync database download
code. When we downloaded a database and then reused the payload struct,
we could find ourselves calling get_fullpath() for the signatures and
overwriting non-freed values we had left over from the database
download.
Refactor the payload_free function into a payload_reset function that we
can call that does NOT free the payload itself, so we can reuse payload
structs. This also allows us to move the payload to the stack in some
call paths, relieving us of the need to alloc space.
Signed-off-by: Dan McGee <dan@archlinux.org>
Rather than always initializing it on any handle creation. There are
several frontend operations (search, info, etc.) that never need the
download code, so spending time initializing this every single time is a
bit silly. This makes it a bit more like the GPGME code init path.
Signed-off-by: Dan McGee <dan@archlinux.org>
In the sync code, we explicitly allocated a string for this field, while
in the dload code itself it was filled in with a pointer to another
string. This led to a memory leak in the sync download case.
Make remote_name non-const and always explicitly allocate it. This patch
ensures this as well as uses malloc + snprintf (rather than calloc) in
several codepaths, and eliminates the only use of PATH_MAX in the
download code.
Signed-off-by: Dan McGee <dan@archlinux.org>
This function doesn't exist on OSX. Since there aren't any other
candidates in alpm for which this function would make sense to use,
simply replace the function call with a loop that does the equivalent.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
if ~/.netrc exists and has credentials for the hostname requested in a
download, they will be provided in an http auth request. This can still
be overridden by explcitly declaring user:pass in the URL.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This gives us some amount of room to grow in case we ever find another
reason that we might return with an error from the progress callback.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
We lost some of this output in the fetch->curl conversion, but I also
noticed in FS#25852 that we just lack some of this useful information
along the way.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
