This is particularly important in the case of FTP control connections,
which may be closed by rogue NAT/firewall devices detecting idle
connections on larger transfers which may take 5-10+ minutes.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Was able to get my hands on one of these boxes today, so add yet another
new way of doing this. I'm glad these calls are so standardized. This
was compile tested on Linux and Illumos and seems to still be working in
both places.
Signed-off-by: Dan McGee <dan@archlinux.org>
Rework the frontend and backend to allow passing a ratio value in for
UseDelta rather than having a hardcoded #define-d 0.7 value always used.
This is useful for those with fast connections, who would likely benefit
from tuning this ratio to lower values; it is also useful for general
testing purposes.
The libalpm API changes for this, but we do support the old config file
format with a no-value 'UseDelta' option; in this case we simply use the
old default of 0.7.
We clamp the ratio values to a sane range between 0.0 and 2.0, allowing
ratios above 1.0 for testing purposes.
Signed-off-by: Dan McGee <dan@archlinux.org>
The entry's name is only used when not "." or ".." so only print the
string then.
Signed-off-by: Olivier Brunel <i.am.jack.mail@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We lost this logic somewhere between the libfetch and libcurl
transition, as it existed in the internal downloader, but was pulled
back only into the sync workflow. Add a helper function that will let us
check for existance in the filecache prior to calling the downloader.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We don't need to open the data to be checked if we don't have a
signature to check against, so postpone that open until we know we have
either the base64_data or a valid signature file.
Signed-off-by: Dan McGee <dan@archlinux.org>
"invalid" in this case simply means files that may or may not be
archives. Discovered via a `pacman -Sc` operation with delta files in
the package cache directory, but can be triggered if any file is passed
to `pacman -Ql` that isn't an archive, for instance, or if the sync
database file is not an archive.
Fix it up so we are more careful about calling archive_read_finish()
only on archives that are valid and have not already been closed, and
teach our archive open function to set the returned archive to NULL if
we aren't going to be returning something valid anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
In both cases we can go with the slightly leaner <stdint.h> header
include since we aren't using the print macros.
Signed-off-by: Dan McGee <dan@archlinux.org>
A look at what this does on 64 bit systems since we were using the
unnecessarily large 'unsigned long' type before even though it was 64
bits wide:
$ ~/bin/bloat-o-meter libalpm.so.old lib/libalpm/.libs/libalpm.so
add/remove: 0/0 grow/shrink: 0/4 up/down: 0/-10412 (-10412)
function old new delta
md5_finish 370 356 -14
sha2_finish 547 531 -16
md5_process 3762 2643 -1119
sha2_process 20356 11093 -9263
The code size is nearly halved in the sha2 case (44% smaller code size),
and md5 gets a nice size reduction (27% smaller) as well.
We also move base64 code to <stdint.h> types as well; we can use
'uint32_t' rather than 'unsigned long' for at least two variables in the
decode function. This doesn't net the same size benefit as the hash code
case, but it is more proper.
Signed-off-by: Dan McGee <dan@archlinux.org>
PGP keyservers are pieces of sh** when it comes to searching for
subkeys, and only allow it if you submit an 8-character fingerprint
rather than the recommended and less chance of collision 16-character
fingerprint.
Add a second remote lookup for the 8-character version of a key ID if we
don't find anything the first time we look up the key. This fixes
FS#27612 and the deficiency has been sent upstream to the GnuPG users
mailing list as well.
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch changes a variety of small things related to our pkghash
implementation with an eye toward performance, especially on native
32-bit systems.
* Use `unsigned int` rather than `size_t` for hash sizes. We already
return ERANGE for any attempted creation of a hash greater than 1
million elements, so unsigned int is more than large enough for our
purposes. Switching to this type allows 32 bit systems to do native
math without helper functions from libgcc.
* _alpm_pkghash_create() now internally adds extra padding for
additional array elements, rather than that being the responsibility of
the caller.
* #define values are moved into static const values in pkghash.c; a new
`stride` value is also extracted (but remains set at 1).
* Division and modulus operators are removed from the normal find and
add paths if possible. We store the upper limit of the number of
elements in the hash so we no longer need to calculate this every
element addition. When doing wraparound position calculations, we only
apply the modulus operator if the value is greater than the number of
buckets.
Signed-off-by: Dan McGee <dan@archlinux.org>
be_package.c: In function 'parse_descfile':
be_package.c:181:28: error: comparison between signed and unsigned
integer expressions [-Werror=sign-compare]
ptr - key + 2 is guaranteed to be > 0 so we can cast to size_t
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Now that filelist arrays know their own size, we don't need to do the
bookkeeping we used to do when they were linked lists. Remove some of
the counter variables and use math instead.
Signed-off-by: Dan McGee <dan@archlinux.org>
This reduces the number of functions we call by log(n) in this function,
and the inlined version is trivial and barely increases the size of the
function.
Signed-off-by: Dan McGee <dan@archlinux.org>
This reduces the number of regcomp() calls when parsing delta entries in
the database from once per entry to once for the entire context handle
by storing the compiled regex data on the handle itself. Just as we do
with the cURL handle, we initialize it the first time it is needed and
free it when releasing the handle.
A few other small tweaks to the parsing function also take place,
including using the stack to store the transient and short file size
string while parsing it.
When parsing a sync database with 1378 delta entries, this reduces the
time of a `pacman -Sl deltas` operation by 50% from 0.22s to 0.12s.
Signed-off-by: Dan McGee <dan@archlinux.org>
In commit 4c5e7af32f, we changed this code to use the regex gathered
substrings. However, we failed to correctly store the delta file name
(leaking memory), as well as freeing the temporary string used to hold
the file size string.
Signed-off-by: Dan McGee <dan@archlinux.org>
More than likely the compiler will do the three operation breakdown we
had here before (2 shifts + subtraction), but let the compiler do the
optimizations and make the actual operation more obvious. This actually
slightly shrinks the function binary size, likely due to instruction
reordering or something.
Signed-off-by: Dan McGee <dan@archlinux.org>
This eliminates the need for strtrim() usage completely, instead relying
on the fact that the only allowed delimiter between key and value is the
" = " string.
Signed-off-by: Dan McGee <dan@archlinux.org>
Extract the actual unlinking of files into a new method, which
eliminates a goto used for flow control. Also fix up a few small issues
in the code:
* Unnecessary (unsigned long) cast, use '%zd' instead
* Total up errors returned from unlink_file calls and return to caller
* Be consistent with scriptlets- we run pre_remove on dbonly, so we
should also run post_remove. Both can be disabled by way of the
--noscriptlet argument.
* Don't pass an invalid pointer to oldpkg to the event callbacks;
instead call the callback before we free the object.
Signed-off-by: Dan McGee <dan@archlinux.org>
Used in alpm_compute_md5sum() and alpm_compute_sha256sum().
Signed-off-by: Diogo Sousa <diogogsousa@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Ensures that config.h is always ordered correctly (first) in the
includes. Also means that new source files get this for free without
having to remember to add it.
We opt for -imacros over -include as its more portable, and the
added constraint by -imacros doesn't bother us for config.h.
This also touches the HACKING file to remove the explicit mention of
config.h as part of the includes.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Mostly a waste of time. Sure, we no longer make sure your pacman
database partition has enough space, but if you are using this option
you better know what you are doing anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
It is quite easy to hoist this potentially repeated computation out of
the loop; even if we don't end up using it, it is super cheap to do it
only once.
Signed-off-by: Dan McGee <dan@archlinux.org>
The return should probably be checked to ensure its not longer than
PATH_MAX, but I have no idea what the correct behavior is when that
happens.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
As per HACKING file, we use 'CTRL(' rather than 'CTRL ('
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Since we know the length of the line, we can use this all the way
through and do a cheaper operation than strdup() by just invoking malloc
and memcpy directly.
Signed-off-by: Dan McGee <dan@archlinux.org>
We were using the size of a pointer, not the size of the whole
archive_read_buffer struct. Thanks to Clang/LLVM 3.0 and Allan/Dave in
IRC for finding this one.
Signed-off-by: Dan McGee <dan@archlinux.org>
We had a 16 KiB limit on database signatures, we should do the same here
too to have a slight sanity check, even if we can't do so for the
package itself yet.
Signed-off-by: Dan McGee <dan@archlinux.org>
We do this in several of the package duplication steps; add a helper
function for doing so to reduce some of the repetitive code.
Also add a free_deplist function for our repeated depend list free calls
of both the data and the list.
Signed-off-by: Dan McGee <dan@archlinux.org>
Made existing documentation more consistent and added
documentation where there was none. One function still
needs documentation and is marked with 'TODO'.
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
