File paths are resolved if necessary during inter-package conflict
checks so that packages carrying the same effective file due to
directory symlinks on the filesystem are flagged as conflicting.
Signed-off-by: Allan McRae <allan@archlinux.org>
If a filename isn't resolved, the original can be used instead of strdup()ing
it.
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
The _alpm_filelist_resolve function takes a filelist and creates
a list with any symlinks in directory paths resolved.
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Add an array to hold the resolved paths of the files in alpm_filelist_t.
When the file name and its resolved file name are identical, the pointer
to the original file name is used to avoid duplicate memory allocation.
Signed-off-by: Allan McRae <allan@archlinux.org>
Detect a conflict between a file/symlink in one package and a directory
in another when both are being installed at once.
A side effect is the creation of conflicts between a directory symlink
and a real directory (e.g lib -> usr/lib in pkg1 and /lib in pkg2).
Given we can not guarantee pkg1 is installed before pkg2, this is a
genuine conflict.
Signed-off-by: Allan McRae <allan@archlinux.org>
This is redundant, and any usage of -D should belong to CPPFLAGS.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
GPG signatures have a timestamp which is checked and if it's in the
future, verification will fail.
Dan: slight wording change.
Signed-off-by: Florian Pritz <bluewind@xinu.at>
Given the message is repeated for each repo, it is a good idea to
print the repo name in the output.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
To improve conflict checking, we will need to make these functions
diverge to an extent where having two separate functions will be
preferable.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We have a few of these and might as well gather them together. This also
cleans up the code a bit by using an enum instead of integer values, as
well as makes a "search for file in filelist" function public so
frontends can do better than straight linear search of the filelists.
Signed-off-by: Dan McGee <dan@archlinux.org>
74274b5dc3 which added the real_line_size to the buffer struct
didn't properly account for what happens when archive_fgets has to loop
more than once to find the end of a line. In most cases, this isn't a
problem, but could potentially cause a longer line such as PGP signature
to be improperly read.
This patch fixes the oversight and focuses on only calculating the line
length when we hit the end of line marker. The effective length is then
calculated via pointer arithmetic as:
(start_of_last_read + read_length) - start_of_line
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Take advantage of the fact that our filelists are arrays sorted by
filename with a known length and use a binary search. This should speed
up file conflict checking, particularly when larger packages are
involved.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This may very well be a no-op, but better safe than sorry.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
On the assumption that these arrays are already mostly sorted, use the
standard quicksort method to sort the files arrays. The files_msort
function name is tweaked to give it a more general name to reflect this
change.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
When checking if a package owns a directory, it is important to check
not only that all the files in the directory are part of the package,
but also if the directory is part of a package. This catches empty
subdirectories during conflict checking for directory to file/symlink
replacements.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
When two packages own an empty directory, pacman finds no conflict when
one of those packages wants to replace the directory with a file or a
symlink. When it comes to actually extracting the new file/symlink,
pacman sees the directory is still there (we do not remove empty
directories if they are owned by a package) and refuses to extract.
Detect this potential conflict early and bail. Note that it is a
_potential_ conflict and not a guaranteed one as the other package owning
the directory could be updated or removed first which would remove
the conflict. However, pacman currently can not sort package installation
order to ensure this, so this conflict requires manual upgrade ordering.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Only load filesystem details for the mount points that we're actually
going to write to. This reduces our syscall count considerably. In the
case of installation, we would actually stat every mountpoint twice (an
extra round for download diskspace) which means (on my system) a total
of 60 syscalls to write to 3 partitions when installing the kernel
package. This change reduces the 60 syscalls down to the expected 3.
A slight debug output change is added here to discern between a
mountpoint added to our linked list versus when we actually load the fs
info.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
add mount_point_load_fsinfo() for platforms using getmntent().
Dan: move the #ifdef slightly so we don't have unused functions on
certain platforms (e.g., OS X).
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Similar to the case for makedepends, it is useful to be able to
access this information without parsing a PKGBUILD.
Signed-off-by: Allan McRae <allan@archlinux.org>
If known, callers can pass the line size to this function in order to
avoid an strlen call. Otherwise, they simply pass 0 and
_alpm_strip_newline will do the call instead.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We inevitably call strlen() or similar on the line returned from
_alpm_archive_fgets(), so include the line size of the interesting line
in the struct.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
With lazy loading in place, it's now quite obvious that we aren't
necessarily checking the right mountpoint for necessary download space.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
This is useful for tools that automatically rebuild packages and
thus require to generate a build order. These entries are skipped
by pacman.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Apparently gcc 4.7 has decided that -Wshadow warnings aren't worth
reporting anymore even with the flag enabled. These were found on
an Ubuntu 10.04 install.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This path is rarely (read: never) taken in any normal run of the code,
so injecting the fprintf() call everywhere with the macro is a bit
overkill. Instead, add a lightweight _alpm_alloc_fail() function that
gets called instead.
This does have a reasonable effect on the size of the generated code;
most places using the macros provided by util.c have their code size
reduced.
Signed-off-by: Dan McGee <dan@archlinux.org>
Increment the strlen() provided value by 1 for the NULL byte so we use
the right value in all three places we later reference it.
Signed-off-by: Dan McGee <dan@archlinux.org>
There is little reason here to grab 4K from the heap only to return it a
few lines later. Instead, just use the stack to hold the returned value
saving ourselves the malloc/free cycle.
Signed-off-by: Dan McGee <dan@archlinux.org>
No one seems to do this "correctly", but for the sake of having an easy
method of detecting the presence and version of libalpm on a given
system, we provide a straightforward .pc file.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
There isn't a whole lot of reason other than code clarity for this, but
it makes it a bit more obvious where multivalued attributes start.
Signed-off-by: Dan McGee <dan@archlinux.org>
Detected by clang scan-build static code analyzer.
* Don't attempt to free an uninitialized gpgme key variable
* Initialize answer variable before asking frontend a question
* Pass by reference instead of value if uninitialized fields are
possible in download signal handler code
* Ensure we never call strlen() on NULL payload->remote_name value
Signed-off-by: Dan McGee <dan@archlinux.org>
Not sure why this one wasn't showing up on x86_64, but this fixes the
compile on i686.
diskspace.c: In function 'calculate_removed_size':
diskspace.c:247:4: error: assuming signed overflow does not occur when negating a division [-Werror=strict-overflow]
cc1: all warnings being treated as errors
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes a bunch of small issues in order to enable a clean
successful build with a crazy number of GCC warning flags. A lot of
these changes are covered by -Wshadow, -Wformat-security, and
-Wstrict-overflow=5.
Signed-off-by: Dan McGee <dan@archlinux.org>
Continue the trend of not touching the environment CFLAGS, ensuring that
the user always has the final say.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
- handle gpgme libs and cflags separately rather than appending to
CFLAGS and LDFLAGS
- be consistent in AC_LINK_IFELSE check for gpgme 1.3.0 (though this is
irrelephant since we don't actually run)
- be consistent with usage of "have" and "with" variables (this
actually ends up reducing SLOC)
- when voluntary detection fails, unset GPGME_CFLAGS and GPGME_LIBS
- when requested support fails the version check, complain about the min
version.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Failure isn't always due to the package file location not existing;
permission issues can also play a part on something like a FUSE-based
filesystem inaccessible to root.
Signed-off-by: Dan McGee <dan@archlinux.org>
The initial patch to implement this achieved nothing apart from
adding a configure option. This patch makes that configure option
do what it advertises.
Note that specifing any shell apart from /bin/sh causes testsuite
failures as /bin/sh is the only shell in the testing environment.
Bug-found-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Dan was right. This should have been FREE(), not free().
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Pull updates from transifex, run update-po on all files, fix a few
errors, and push them back to Transifex.
Signed-off-by: Dan McGee <dan@archlinux.org>
For key searches only, gpg2 will fail to lookup any and all keys that
are not prefixed with 0x.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Using fputs should be faster as no format string parsing is required. It
also prevents silly errors related to unescaped '%' signs, and removes
the need to double them up in a lot of places.
Signed-off-by: Dan McGee <dan@archlinux.org>
This removes a call to _alpm_local_db_pkgpath() as well as an access()
call when reading the local database. This appears to be code from 2006
that has stuck around. We don't need it because:
1) We never use this path except to check it via access(); however, we
are already in a readdir() loop so it exists, or at least did at the
time of the call.
2) The fopen() and other calls will fail on accessing the database files
anyway, and we need to check those for errors.
Signed-off-by: Dan McGee <dan@archlinux.org>
In case we have a mirror failure, unlink_on_fail would remain set,
causing an interrupt in a successive download attempt to be wrongly
unlinked.
This also fixes a memory leak in the url member, as we would allocate
over the previous, unfreed URL.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
To avoid conflicts on reusing a payload after a failed download, ensure
that we reset the filename hints in the payload struct prior to the
download operation.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We had one stubbed out so we didn't require a translation update, and
the other is more a code style issue.
Signed-off-by: Dan McGee <dan@archlinux.org>
Unify the output for local and sync packages by only printing a
list of possible validation types for sync packages. This also
has the advantage of not printing the very long sha256 checksum
which line wrapped on a standard width terminal.
Signed-off-by: Allan McRae <allan@archlinux.org>
When installing a package, store information on which validation
method was used and output this on "pacman -Qi" operations.
e.g.
Validated By : SHA256 Sum
Possible values are Unknown, None, MD5 Sum, SHA256 Sum, Signature.
Dan: just a few very minor tweaks.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
No new behaviour introduced, everything should work exactly as before.
Dan: refactored to use the single alpm_depend_t structure.
Signed-off-by: Benedikt Morbach <benedikt.morbach@googlemail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the first step in parsing and handling optdepends. There is no
behavior change introduced in this commit; however, depends that contain
a ": " string will now be parsed as having a description and it will be
stored in the depend structure. Later patches will utilize this new
field as appropriate.
This is heavily based on the work of Benedikt, who did something similar
but introduced a new type for this rather than only a new field to the
existing type.
Heavily-influenced-by: Benedikt Morbach <benedikt.morbach@googlemail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
* it updates to all translations
* minor fr, pt_BR, de, lt, sk and uk updates
* add new strings in pacman translation catalog
Signed-off-by: Dan McGee <dan@archlinux.org>
If we begin to create a file list when loading a package, but abort
because of an error to one of our goto labels, the memory used to create
the file list will leak. This is because we use a set of local variables
to hold the data, and thus _alpm_pkg_free() cannot clean up for us.
Use the file list struct on the package object as much as possible to
keep state when building the file list, thus allowing _alpm_pkg_free()
to clean up any partially built data.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is easily triggered via a `pacman -Sc` operation when it attempts
to open a delta file as a package- we end up leaking loads of memory
due to us never freeing the archive object. When you have upwards of
1200 delta files in your sync database directory, this results in a
memory leak of nearly 1.5 MiB.
Also fix another memory leak noticed at the same time- we need to call
the internal _alpm_pkg_free() function, as without the origin data being
set the public free function will do nothing.
Signed-off-by: Dan McGee <dan@archlinux.org>
It seems that if we pass the permissions that we want the created
directory to have, then we should probably use it...
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Add 2012 to the copyright range for all libalpm and pacman source files.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Simplify the implementation:
- allocate and manipulate a copy of the passed in path rather than
building out a path as the while loop progresses
- use simple pointer arithmetic to skip uninteresting cases
- use mkdir(3)'s return value and errno to detect failure
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
It can happen that the to-be-downloaded file cannot be created in cachedir.
For example, I am an -Sup user, and it is comfortable to set --cachedir to
/mnt/pendrive, which is a FAT filesystem, so files like
capseo-1:0.3-2-i686.pkg.tar.xz cannot be downloaded to there.
Before this patch, pacman didn't give clear output about what happens when
the download code could not create the necessary file. This can be confusing
with -Su. An example output:
***
$ sudo pacman -S capseo bochs --cachedir /c/TEMP
resolving dependencies...
looking for inter-conflicts...
Targets (2): bochs-2.4.6-1 capseo-1:0.3-2
Total Download Size: 0.61 MiB
Total Installed Size: 2.61 MiB
Proceed with installation? [Y/n]
:: Retrieving packages from extra...
warning: failed to retrieve some files from extra
bochs-2.4.6-1-i686 611.5 KiB 118K/s 00:05 [------------------] 97%
error: failed to commit transaction (unexpected error)
Errors occurred, no packages were upgraded.
***
After the patch, pacman will give more informative error message (and
pm_errno is set properly):
***
error: could not open file '/c/TEMP/capseo-1:0.3-2-i686.pkg.tar.xz.part': Invalid argument
error: failed to commit transaction (failed to retrieve some files)
***
Unfortunately, the "could not open file" error message is printed for
every mirror (that can be dozens of lines), which is ugly, but at least
informative... Without modifying the download logic (for example, by
introducing -2 return value for _alpm_download() to indicate giving up),
this ugliness cannot be eliminated.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Mostly a waste of time. Sure, we no longer make sure your pacman
database partition has enough space, but if you are using this option
you better know what you are doing anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
(cherry picked from commit ee96900605)
If one had a mountpoint at '/e' (don't ask), a file being installed to
'/etc' would map to it incorrectly. Ensure we do more than just prefix
matching on paths by doing some more sanity checks once the simple
strncmp() call succeeds.
Signed-off-by: Dan McGee <dan@archlinux.org>
This reverts commit f3fa77bcf1 along with
making other necessary changes to fully back this (mis)feature out until
we can do it correctly.
The quick summary here is this was not implemented correctly; provides
are not fully taken into account in this logic, and making that happen
exposes a lot of other flaws in this code that are covered up later on
in the dependency resolving process by several other pieces of
convoluted and conditional logic.
Tests have been adjusted accordingly. Some test EXISTS conditions have
been removed as we already know the package is installed locally, and we
also are checking the VERSION condition anyway.
With these two related revert commits, we do have some changes in test
pass/fail results:
* upgrade078.py: does not pass, this is due to --recursive getting
removed for -U/-S operations after this commit.
* sync302.py: the version checks have been disabled, so this test
continues to pass but has been scaled back in scope.
* sync303.py: now passes, was failing before.
* sync304.py: still failing, was failing before.
* sync305.py: now passes, was failing before.
* sync306.py: still passes, was passing before.
Signed-off-by: Dan McGee <dan@archlinux.org>
Set errno to 0 at the start of _alpm_open_archive as it is not set when
archive_read_open_fd fails. This can result in _alpm_pkg_load_internal
thinking errno == ENOENT and setting the wrong pm_errno. e.g.
Before:
> testpkg pacman-4.0.1-4-i686.pkg.tar.gz.sig
error: could not open file pacman-4.0.1-4-i686.pkg.tar.gz.sig: Unrecognized archive format
Cannot find the given file.
After:
> testpkg pacman-4.0.1-4-i686.pkg.tar.gz.sig
error: could not open file pacman-4.0.1-4-i686.pkg.tar.gz.sig: Unrecognized archive format
Cannot open the given file.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Some distributions insist on using bash specific commands in their
install scripts under the assumption that "sh" is a symlink to bash.
This can causes issues if (e.g.) their users what to change sh to
point at another shell, such as dash, that does not support these
features. Add a configure option to explicitly set the shell being
used to run install scripts.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
pacman -U <pkg> returns a bogus "could not find or read package" if the
file is on a fuse file system that doesn't allow root access. Debug
output isn't very helpful here either so we should log why the access
check failed.
The other 2 checks already log something when failing so logging a more
specific error won't hurt either.
Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Dan McGee <dan@archlinux.org>
The max filesize for a delta download must be the full size of the delta
file, not just what's remaining.
Fixes FS#28345
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is after some manual massaging to fix issues with newlines in some
translations of the script catalogs.
Signed-off-by: Dan McGee <dan@archlinux.org>
This makes several small adjustments to our exposed method names, and in
one case, parameters. The justification here is to make methods less odd
in their naming convention. If a method takes an alpm_db_t argument, the
method should be named 'alpm_db_*', but perhaps more importantly, if it
doesn't take a database as the first parameter, it should not.
Summary of changes:
alpm_db_register_sync -> alpm_register_syncdb
alpm_db_unregister_all -> alpm_unregister_all_syncdbs
alpm_option_get_localdb -> aplpm_get_localdb
alpm_option_get_syncdbs -> aplpm_get_syncdbs
alpm_db_readgroup -> alpm_db_get_group
alpm_db_set_pkgreason -> alpm_pkg_set_reason
All methods keep the same argument list except for alpm_pkg_set_reason;
there we drop the 'handle' argument as it can be retrieved from the
passed in package object.
Signed-off-by: Dan McGee <dan@archlinux.org>
Don't use trailing commas in enums if people really want to use a strict
C89 compiler, and document why on earth one particular enum uses bitmask
values when it doesn't seem necessary.
With comments, shoot for more consistency. When something is a
one-liner, keep it that way and move the whole /** sequence */ to one
line. When it needs more than one line, ensure we format most of them in
a similar fashion.
Two minor function signature adjustments are made that don't change
anything other than matching the parameter name (name -> filename)
and fitting in with our coding style (type* var -> type *var).
Signed-off-by: Dan McGee <dan@archlinux.org>
The pacman-scripts catalog is omitted here due to various newline errors
I don't have the time to fix right now.
Signed-off-by: Dan McGee <dan@archlinux.org>
Very rarely a segfault would occur when removing a number of packages
due to a corrupted list for the local database (FS#27805, FS#28195).
This was caused by the alpm_list_msort function not correctly dealing
with the two new head node's prev values.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This moves the code for removal of local database entries right into
be_local.c, which was the last user of the rmrf() function we had in our
utility source file. We can simplify the implementation and make it
non-recursive as we know the structure of the local database entries.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is particularly important in the case of FTP control connections,
which may be closed by rogue NAT/firewall devices detecting idle
connections on larger transfers which may take 5-10+ minutes.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Was able to get my hands on one of these boxes today, so add yet another
new way of doing this. I'm glad these calls are so standardized. This
was compile tested on Linux and Illumos and seems to still be working in
both places.
Signed-off-by: Dan McGee <dan@archlinux.org>
Rework the frontend and backend to allow passing a ratio value in for
UseDelta rather than having a hardcoded #define-d 0.7 value always used.
This is useful for those with fast connections, who would likely benefit
from tuning this ratio to lower values; it is also useful for general
testing purposes.
The libalpm API changes for this, but we do support the old config file
format with a no-value 'UseDelta' option; in this case we simply use the
old default of 0.7.
We clamp the ratio values to a sane range between 0.0 and 2.0, allowing
ratios above 1.0 for testing purposes.
Signed-off-by: Dan McGee <dan@archlinux.org>
The entry's name is only used when not "." or ".." so only print the
string then.
Signed-off-by: Olivier Brunel <i.am.jack.mail@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We lost this logic somewhere between the libfetch and libcurl
transition, as it existed in the internal downloader, but was pulled
back only into the sync workflow. Add a helper function that will let us
check for existance in the filecache prior to calling the downloader.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We don't need to open the data to be checked if we don't have a
signature to check against, so postpone that open until we know we have
either the base64_data or a valid signature file.
Signed-off-by: Dan McGee <dan@archlinux.org>
"invalid" in this case simply means files that may or may not be
archives. Discovered via a `pacman -Sc` operation with delta files in
the package cache directory, but can be triggered if any file is passed
to `pacman -Ql` that isn't an archive, for instance, or if the sync
database file is not an archive.
Fix it up so we are more careful about calling archive_read_finish()
only on archives that are valid and have not already been closed, and
teach our archive open function to set the returned archive to NULL if
we aren't going to be returning something valid anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
In both cases we can go with the slightly leaner <stdint.h> header
include since we aren't using the print macros.
Signed-off-by: Dan McGee <dan@archlinux.org>
A look at what this does on 64 bit systems since we were using the
unnecessarily large 'unsigned long' type before even though it was 64
bits wide:
$ ~/bin/bloat-o-meter libalpm.so.old lib/libalpm/.libs/libalpm.so
add/remove: 0/0 grow/shrink: 0/4 up/down: 0/-10412 (-10412)
function old new delta
md5_finish 370 356 -14
sha2_finish 547 531 -16
md5_process 3762 2643 -1119
sha2_process 20356 11093 -9263
The code size is nearly halved in the sha2 case (44% smaller code size),
and md5 gets a nice size reduction (27% smaller) as well.
We also move base64 code to <stdint.h> types as well; we can use
'uint32_t' rather than 'unsigned long' for at least two variables in the
decode function. This doesn't net the same size benefit as the hash code
case, but it is more proper.
Signed-off-by: Dan McGee <dan@archlinux.org>
PGP keyservers are pieces of sh** when it comes to searching for
subkeys, and only allow it if you submit an 8-character fingerprint
rather than the recommended and less chance of collision 16-character
fingerprint.
Add a second remote lookup for the 8-character version of a key ID if we
don't find anything the first time we look up the key. This fixes
FS#27612 and the deficiency has been sent upstream to the GnuPG users
mailing list as well.
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch changes a variety of small things related to our pkghash
implementation with an eye toward performance, especially on native
32-bit systems.
* Use `unsigned int` rather than `size_t` for hash sizes. We already
return ERANGE for any attempted creation of a hash greater than 1
million elements, so unsigned int is more than large enough for our
purposes. Switching to this type allows 32 bit systems to do native
math without helper functions from libgcc.
* _alpm_pkghash_create() now internally adds extra padding for
additional array elements, rather than that being the responsibility of
the caller.
* #define values are moved into static const values in pkghash.c; a new
`stride` value is also extracted (but remains set at 1).
* Division and modulus operators are removed from the normal find and
add paths if possible. We store the upper limit of the number of
elements in the hash so we no longer need to calculate this every
element addition. When doing wraparound position calculations, we only
apply the modulus operator if the value is greater than the number of
buckets.
Signed-off-by: Dan McGee <dan@archlinux.org>
be_package.c: In function 'parse_descfile':
be_package.c:181:28: error: comparison between signed and unsigned
integer expressions [-Werror=sign-compare]
ptr - key + 2 is guaranteed to be > 0 so we can cast to size_t
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Now that filelist arrays know their own size, we don't need to do the
bookkeeping we used to do when they were linked lists. Remove some of
the counter variables and use math instead.
Signed-off-by: Dan McGee <dan@archlinux.org>
This reduces the number of functions we call by log(n) in this function,
and the inlined version is trivial and barely increases the size of the
function.
Signed-off-by: Dan McGee <dan@archlinux.org>
This reduces the number of regcomp() calls when parsing delta entries in
the database from once per entry to once for the entire context handle
by storing the compiled regex data on the handle itself. Just as we do
with the cURL handle, we initialize it the first time it is needed and
free it when releasing the handle.
A few other small tweaks to the parsing function also take place,
including using the stack to store the transient and short file size
string while parsing it.
When parsing a sync database with 1378 delta entries, this reduces the
time of a `pacman -Sl deltas` operation by 50% from 0.22s to 0.12s.
Signed-off-by: Dan McGee <dan@archlinux.org>
In commit 4c5e7af32f, we changed this code to use the regex gathered
substrings. However, we failed to correctly store the delta file name
(leaking memory), as well as freeing the temporary string used to hold
the file size string.
Signed-off-by: Dan McGee <dan@archlinux.org>
More than likely the compiler will do the three operation breakdown we
had here before (2 shifts + subtraction), but let the compiler do the
optimizations and make the actual operation more obvious. This actually
slightly shrinks the function binary size, likely due to instruction
reordering or something.
Signed-off-by: Dan McGee <dan@archlinux.org>
This eliminates the need for strtrim() usage completely, instead relying
on the fact that the only allowed delimiter between key and value is the
" = " string.
Signed-off-by: Dan McGee <dan@archlinux.org>
Extract the actual unlinking of files into a new method, which
eliminates a goto used for flow control. Also fix up a few small issues
in the code:
* Unnecessary (unsigned long) cast, use '%zd' instead
* Total up errors returned from unlink_file calls and return to caller
* Be consistent with scriptlets- we run pre_remove on dbonly, so we
should also run post_remove. Both can be disabled by way of the
--noscriptlet argument.
* Don't pass an invalid pointer to oldpkg to the event callbacks;
instead call the callback before we free the object.
Signed-off-by: Dan McGee <dan@archlinux.org>
Used in alpm_compute_md5sum() and alpm_compute_sha256sum().
Signed-off-by: Diogo Sousa <diogogsousa@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Ensures that config.h is always ordered correctly (first) in the
includes. Also means that new source files get this for free without
having to remember to add it.
We opt for -imacros over -include as its more portable, and the
added constraint by -imacros doesn't bother us for config.h.
This also touches the HACKING file to remove the explicit mention of
config.h as part of the includes.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Mostly a waste of time. Sure, we no longer make sure your pacman
database partition has enough space, but if you are using this option
you better know what you are doing anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
It is quite easy to hoist this potentially repeated computation out of
the loop; even if we don't end up using it, it is super cheap to do it
only once.
Signed-off-by: Dan McGee <dan@archlinux.org>
The return should probably be checked to ensure its not longer than
PATH_MAX, but I have no idea what the correct behavior is when that
happens.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
As per HACKING file, we use 'CTRL(' rather than 'CTRL ('
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Since we know the length of the line, we can use this all the way
through and do a cheaper operation than strdup() by just invoking malloc
and memcpy directly.
Signed-off-by: Dan McGee <dan@archlinux.org>
We were using the size of a pointer, not the size of the whole
archive_read_buffer struct. Thanks to Clang/LLVM 3.0 and Allan/Dave in
IRC for finding this one.
Signed-off-by: Dan McGee <dan@archlinux.org>
We had a 16 KiB limit on database signatures, we should do the same here
too to have a slight sanity check, even if we can't do so for the
package itself yet.
Signed-off-by: Dan McGee <dan@archlinux.org>
We do this in several of the package duplication steps; add a helper
function for doing so to reduce some of the repetitive code.
Also add a free_deplist function for our repeated depend list free calls
of both the data and the list.
Signed-off-by: Dan McGee <dan@archlinux.org>
Made existing documentation more consistent and added
documentation where there was none. One function still
needs documentation and is marked with 'TODO'.
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This moves the common setup code of about 5 different callers into one
method. Error messages will now be common and shared in all places;
several paths did not have any messages at all before.
In addition, we now pick an ideal block size for the archive read based
off the larger value of our default buffer size or the st.st_blksize
field. For a filesystem such as NFS, this is often much larger than the
default 8192- values such as 32768 and 131072 are common.
Signed-off-by: Dan McGee <dan@archlinux.org>
When doing a bare -U operation on a local package that doesn't pull in
any dependencies from the sync databases, we can get away with missing
database files. This makes the check conditional on no sync targets
found in the target list. This is not the prettiest code here so we have
a bit of hackish behavior required to straighten both the behavior and
the nonsensical error message out.
Addresses FS#26899.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is consistent with the other enums and structs, and should be
slightly more readable.
Signed-off-by: Jonathan Conder <jonno.conder@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Bump the version, update the translation template files, and fill in
NEWS with relevant commits and changes since 4.0.0.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is work originally provided by Sascha Kruse on FS#20360 with only
minor adjustments to the implementation. It's been expanded to cover:
NoUpgrade, NoExtract, IgnorePkg, IgnoreGroup.
Adds tests ignore008, sync139, sync502, and sync503.
Also satisfies FS#18988.
Original-work-by: Sascha Kruse <knopwob@googlemail.com>
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
This is a simple change that allows comparions to be more in line with
how other checks are done. It will be necessary for ensuing patchwork
that implements fnmatch for comparing and assumes a specific argument
ordering.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
The point of this early compare to NULL byte check was so we could bail
early and skip the strcmp() call. Given we weren't doing the check
right, this never exited early. Fix it to work as intended.
Noticed-by: Pepe Juárez <trulustapa@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a trivial operation that doesn't require calling a function over
and over- just do some math and indexing into a character array.
Signed-off-by: Dan McGee <dan@archlinux.org>
This gives us a bit more control and over the archive reading process,
and a bit less is done behind the scenes. It also allows us to use
fstat() in preference to stat(), which should avoid some potential race
conditions.
Some reorganization is necessary to move the stat calls after the open()
calls. Error handling and cleanup in general is also improved, as we had
several potential memory and file handle leaks before in some error
paths.
Signed-off-by: Dan McGee <dan@archlinux.org>
This removes an unnecessary level of buffering. We are not doing
line-based I/O here, so we can read in blocks of 8K at a time directly
from the file.
Signed-off-by: Dan McGee <dan@archlinux.org>
Replacing the strdup when after the first NULL check assures that we get
continue with payload->remote_name defined.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Dan: fix mask calculation, add it to the success/fail block instead.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This takes the place of three previously used constants:
ARCHIVE_DEFAULT_BYTES_PER_BLOCK, BUFFER_SIZE, and CPBUFSIZE.
In libarchive 3.0, the first constant will be no more, so we can ensure
we are forward-compatible by removing our usage of it now. The rest are
unified for consistency.
By default, we will use the value of BUFSIZ provided by <stdio.h>, which
is 8192 on Linux. If that is undefined, a default value is provided.
Signed-off-by: Dan McGee <dan@archlinux.org>
There aretwo seperate issues in the same block of file conflict
checking code here:
1) If realpath errored, such as when a symlink was broken, we would call
'continue' rather than simply exit this particular method of
resolution. This was likely just a copy-paste mistake as the previous
resolving steps all use loops where continue makes sense. Refactor
the check so we only proceed if realpath is successful, and continue
with the rest of the checks either way.
2) The real problem this code was trying to solve was canonicalizing
path component (e.g., directory) symlinks. The final component, if
not a directory, should not be handled at all in this loop. Add a
!S_ISLNK() condition to the loop so we only call this for real files.
There are few other small cleanups to the debug messages that I made
while debugging this problem- we don't need to keep printing the file
name, and ensure every block that sets resolved_conflict to true prints
a debug message so we know how it was resolved.
This fixes the expected failures from symlink010.py and symlink011.py,
while still ensuring the fix for fileconflict007.py works.
Signed-off-by: Dan McGee <dan@archlinux.org>
There is some pecular behavior going on here when a package is loaded
that has no files, as is very common in our test suite. When we enter
the realloc/sort code, a package without files will call the following:
files = realloc(NULL, 0);
One would assume this is a no-op, returning a NULL pointer, but that is
not the case and valgrind later reports we are leaking memory. Fix the
whole thing by skipping the reallocation and sort steps if the pointer
is NULL, as we have nothing to do.
Note that the package still gets marked as 'files loaded', becuase
although there were none, we tried and were successful.
Signed-off-by: Dan McGee <dan@archlinux.org>
First, use fstat() in preference to stat() since we already have an open
file handle. This also removes the need to check for a symlink as that
is not possible when a file is opened.
Next, use archive_entry_mode() rather than archive_entry_stat() as we
only use the mode portion of the stat struct and the call is much
cheaper. Also delay it until it is necessary.
Signed-off-by: Dan McGee <dan@archlinux.org>
Extend the return values of compute_download_size to allow callers to
know that a .part file exists for the package.
This extra value isn't currently used, but it'll be needed later on.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This adds a logger to the CURLE_OK case so we can always know the return
code if it was >= 400, and debug log it regardless. Also adjust another
logger to use the cURL error message directly, as well as use fstat()
when we have an open file handle rather than stat().
Signed-off-by: Dan McGee <dan@archlinux.org>
Create a new static function called 'download_single_file' which
iterates over the servers for each payload.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is done by both the delta and regular file code, so we can extract
a little helper method. Done mostly to satisfy my "why are we repeating
code here" itch.
Signed-off-by: Dan McGee <dan@archlinux.org>
Break out the logic of finding payloads into a separate static function
to avoid nesting mayhem. After gathering all the records, download them
all at once.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
The absolutely terrible part about this is the failure on GPGME's part
to distinguish between "key not found" and "keyserver timeout". Instead,
it returns the same silly GPG_ERR_EOF in both cases (why isn't
GPG_ERR_TIMEOUT being used?), leaving us helpless to tell them apart.
Spit out a generic enough error message that covers both cases;
unfortunately we can't provide much guidance to the user because we
aren't sure what actually happened.
Signed-off-by: Dan McGee <dan@archlinux.org>
This logic is reused in both diskspace and downloadspace check
functions, so pull it out into its own static method.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This function determines if the given cachedir has at least the given
amount of free space on it. This will be later used in the sync code to
preemptively halt downloads.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We can take a large shortcut here that saves us a lot of time,
especially when upgrading packages with lots of directories. Obviously
iterating the full file list of every single package to determine if
this directory was present in any other package can take quite some time
on a system with many packages installed. We don't need to remove a
directory at all if we are upgrading a package and the version we are
moving to still had the directory.
Also make a small optimization on the package comparsion- we really only
care about equality here, not the result of the compare, so we can
shortcut using our name_hash.
What kind of benefit does this give us? Oh, only a reduction from 295.7
million to 1.4 million strcmp() calls (99.5% fewer) during a
`pacman -S linux libreoffice-common` operation.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is in the realm of "probably not going to happen", but if someone
were to translate "disk" to a string longer than 256 characters, we
would have a smashed/corrupted stack due to our unchecked strcpy() call.
Rework the function to always length-check the value we copy into the
hostname buffer, and do it with memcpy rather than the more cumbersome
and unnecessary snprintf.
Finally, move the magic 256 value into a constant and pass it into the
function which is going to get inlined anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
Allan McRae