When we switched to a file object and not just a simple string, we missed an
update along the way here in target-target conflicts. This patch looks
large, but it really comes down to one errant (char *) cast before that has
been reworked to explicitly point to the alpm_file_t object. The rest is
simply code cleanup.
Signed-off-by: Dan McGee <dan@archlinux.org>
A few parameters were outdated or wrongly named, and a few things were
explicitly linked that Doxygen wasn't able to resolve.
Signed-off-by: Dan McGee <dan@archlinux.org>
We returned the right error code but never set the flags accordingly.
Also, now that we can bail early, ensure we set the error code.
Signed-off-by: Dan McGee <dan@archlinux.org>
This adds calls to gpgme_op_import_result() which we were not looking at
before to ensure the key was actually imported. Additionally, we do some
preemptive checks to ensure the keyring is even writable if we are going
to prompt the user to add things to it.
Signed-off-by: Dan McGee <dan@archlinux.org>
This also fixes a segfault found by dave when key_search is
unsuccessful; the key_search return code documentation has also been
updated to reflect reality.
Signed-off-by: Dan McGee <dan@archlinux.org>
Because we aren't using gpgv and a dedicated keyring that is known to be
all safe, we should honor this flag being set on a given key in the
keyring to know to not honor it. This prevents a key from being
reimported that a user does not want to be used- instead of deleting,
one should mark it as disabled.
Signed-off-by: Dan McGee <dan@archlinux.org>
Add two new static methods, key_search() and key_import(), to our
growing list of signing code.
If we come across a key we do not have, attempt to look it up remotely
and ask the user if they wish to import said key. If they do, flag the
validation process as a potential 'retry', meaning it might succeed the
next time it is ran.
These depend on you having a 'keyserver hkp://foo.example.com' line in
your gpg.conf file in your gnupg home directory to function.
Signed-off-by: Dan McGee <dan@archlinux.org>
This moves the result processing out of the validation check loop itself
and into a new loop. Errors will be presented to the user one-by-one
after we fully complete the validation loop, so they no longer overlap
the progress bar.
Unlike the database validation, we may have several errors to process in
sequence here, so we use a function-scoped struct to track all the
necessary information between seeing an error and asking the user about
it.
The older prompt_to_delete() callback logic is still kept, but only for
checksum failures. It is debatable whether we should do this at all or
just delegate said actions to the user.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is for eventual use by the PGP key import code. Breaking this into
a separate commit now makes the following patches a bit easier to
understand.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows a frontend program to query, at runtime, what the library
supports. This can be useful for sanity checking during config-
requiring a downloader or disallowing signature settings, for example.
Signed-off-by: Dan McGee <dan@archlinux.org>
This takes the libraries hidden default out of the equation: hidden in
the sense that we can't even find out what it is until we create a
handle. This is a chicken-and-egg problem where we have probably already
parsed the config, so it is hard to get the bitmask value right.
Move it to the frontend so the caller can do whatever the heck they
want. This also exposes a shortcoming where the frontend doesn't know if
the library even supports signatures, so we should probably add a
alpm_capabilities() method which exposes things like HAS_DOWNLOADER,
HAS_SIGNATURES, etc.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows us to do all delta verification up front, followed by
whatever needs to be done with any found errors. In this case, we call
prompt_to_delete() for each error.
Add back the missing EVENT(ALPM_EVENT_DELTA_INTEGRITY_DONE) that
accidentally got removed in commit 062c391919.
Remove use of *data; we never even look at the stuff in this array for
the error code we were returning and this would be much better handled
by one callback per error anyway, or at least some strongly typed return
values.
Signed-off-by: Dan McGee <dan@archlinux.org>
If siglist->results wasn't a NULL pointer, we would try to free it
anyway, even if siglist->count was zero. Only attempt to free this
pointer if we had results and the pointer is valid.
Signed-off-by: Dan McGee <dan@archlinux.org>
This one can be overwhelming when reading debug output from a very large
package. We already have the output of each extracted file so we
probably can do without this in 99.9% of cases.
Signed-off-by: Dan McGee <dan@archlinux.org>
This adds two new static methods, check_validity() and load_packages(),
to sync.c which are simply code fragments pulled out of our
do-everything sync commit code.
Signed-off-by: Dan McGee <dan@archlinux.org>
In reality, there is no retrying that happens as of now because we don't
have any import or changing of the keyring going on, but the code is set
up so we can drop this in our new _alpm_process_siglist() function. Wire
up the basics to the sync database validation code, so we see something
like the following:
$ pacman -Ss unknowntrust
error: core: signature from "Dan McGee <dpmcgee@gmail.com>" is unknown trust
error: core: signature from "Dan McGee <dpmcgee@gmail.com>" is unknown trust
error: database 'core' is not valid (invalid or corrupted database (PGP signature))
$ pacman -Ss missingsig
error: core: missing required signature
error: core: missing required signature
error: database 'core' is not valid (invalid or corrupted database (PGP signature))
Yes, there is some double output, but this should be fixable in the
future.
Signed-off-by: Dan McGee <dan@archlinux.org>
This adds a some new callback event and progress codes for package
loading, which was formerly bundled in with package validation before.
The main sync.c loop where loading occurred is now two loops running
sequentially. The behavior should not change with this patch outside of
progress and event display; more changes will come in following patches.
Signed-off-by: Dan McGee <dan@archlinux.org>
_alpm_pkg_load_internal() was becoming a monster. Extract the top bit of
the method that dealt with checksum and signature validation into a
separate method that should be called before one loads a package to
ensure it is valid.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is always true at the end since we return early if we couldn't
create the tmpdir, so it is totally unnecessary.
Signed-off-by: Dan McGee <dan@archlinux.org>
We shouldn't be going through the accessor that does a bunch of
unnecessary legwork, including potentially loading the pkgcache right
before we free it.
Signed-off-by: Dan McGee <dan@archlinux.org>
Rather than using a string-based path, we can restore the working
directory via a file descriptor and use of fchdir().
From the getcwd manpage:
Opening the current directory (".") and calling fchdir(2) to
return is usually a faster and more reliable alternative when
sufficiently many file descriptors are available.
Signed-off-by: Dan McGee <dan@archlinux.org>
We did a lot of both malloc-ing and stack printing to form some paths in
this code. Attempt to unify it all into the one get_pkgpath() method by
adding an optional third "filename" parameter, and form the necessary
path string all in one go.
Signed-off-by: Dan McGee <dan@archlinux.org>
1. Don't run it if something failed in package removal- this mirrors
what we already do in sync transactions.
2. Don't run it if we are invoking it for the replaces removal bit of a
sync transaction- it doesn't make sense to run ldconfig halfway through
a sync install; we should only run it once at the end.
Signed-off-by: Dan McGee <dan@archlinux.org>
We add them to this list with the root path not appended; we should be
searching for them this way as well.
Signed-off-by: Dan McGee <dan@archlinux.org>
We checked the (fgets == NULL and !feof) case, but never actually bailed
out of the loop if we were at the end of the file, causing infinite
looping.
Signed-off-by: Dan McGee <dan@archlinux.org>
This shouldn't really be declared with const, and causes a compile error
when -Wcast-qual is used. Remove the const specifier from the function
specification and all implementations.
Also fix one other trivial -Wcast-qual warning in _alpm_db_cmp().
Signed-off-by: Dan McGee <dan@archlinux.org>
We never ended up using or really needing this; kill it for now knowing
it is in git history if ever needed again.
Signed-off-by: Dan McGee <dan@archlinux.org>
This function doesn't exist on OSX. Since there aren't any other
candidates in alpm for which this function would make sense to use,
simply replace the function call with a loop that does the equivalent.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Similar to an earlier commit which accounts for .part files for full
packages, calculate the download_size for deltas keeping mind the
possibility of a partial transfer.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Check for the existance of a partial download of a package file before
jumping to delta calculations. Currently, if there were 10MiB remaining
in a 100MiB the values passed to the front end do not reflect this.
Refactored from an old patch originally by Dan.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
if ~/.netrc exists and has credentials for the hostname requested in a
download, they will be provided in an http auth request. This can still
be overridden by explcitly declaring user:pass in the URL.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This adjusts type usage to match POSIX provided types from
<sys/types.h> rather than assuming everything will fit in a long or
unsigned long. Use fsblkcnt_t (unsigned) and blkcnt_t (signed) as
appropriate. These are affected the same way off_t is on 32 bit
platforms, where the types are extende to 64 bits if large file support
is enabled.
Because most numbers here are block counts, this isn't
near as pressing as using a 32-bit variable for file sizes where
anything over 2GiB can burn you; we likely can support files at least
512 but mainly 4096 times larger.
Signed-off-by: Dan McGee <dan@archlinux.org>
Changes in commit dc3336c277 caused this to stop working as expected for
sync packages, due to the way the logic is structured. Ensure we always
enter the signature code if the bitflag is flipped on to check
signatures for packages. Rename 'use_sig' to 'has_sig' for clarity.
Signed-off-by: Dan McGee <dan@archlinux.org>
This gives us some amount of room to grow in case we ever find another
reason that we might return with an error from the progress callback.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
We lost some of this output in the fetch->curl conversion, but I also
noticed in FS#25852 that we just lack some of this useful information
along the way.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
_alpm_filecache_setup() destroys the list of cachedirs when it finds no
writeable directories in the config. This put us in an awkward situation
where _alpm_filecache_find() would locate a downloaded file in a r/o
cachedir, but then fail to install it after _alpm_filecache_setup() is
called (with a NULL argument). Change this behavior to merely prepend
the temporary directory to the list of available cachedirs.
Dan exposed it in e07547ee4e, as now a package can be found in a
directory we may not be able to actually store packages in.
Reported-by: Rémy Oudompheng <remy@archlinux.org>
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Neither deltas nor filename attributes are ever present in the local
database, so we can remove all of the indirection for accessing these
attributes.
Signed-off-by: Dan McGee <dan@archlinux.org>
Put all the callback stuff in alpm.h in one spot, and make the following
renames for clarity with the new structure:
ALPM_TRANS_EVT_* --> ALPM_EVENT_*
ALPM_TRANS_CONV_* --> ALPM_QUESTION_*
ALPM_TRANS_PROGRESS_* --> ALPM_PROGRESS_*
alpm_option_get_convcb() --> alpm_option_get_questioncb()
alpm_option_set_convcb() --> alpm_option_set_questioncb()
Signed-off-by: Dan McGee <dan@archlinux.org>
This was just disgusting before, unnecessary to limit these to only
usage in a transaction. Still a lot of more room for cleanup but we'll
start by attaching them to the handle rather than the transaction we may
or may not even want to use these callbacks.
Signed-off-by: Dan McGee <dan@archlinux.org>
The call to apply was tucked inside validate, and the EVENT callbacks
were done outside the function rather than inside. Reorganize things a
bit to make more sense.
Signed-off-by: Dan McGee <dan@archlinux.org>
We currently have csize, isize, and size concepts, and sometimes the
difference isn't clear. Ensure the following holds:
* size (aka csize): always the compressed size of the package; available
for everything except local packages (where it will return 0)
* isize: always the installed size of the package; available for all
three package types
Signed-off-by: Dan McGee <dan@archlinux.org>
This upgrades the simple 15/17 scaling by package number we used before
to package sized based scaling, which is much more accurate. Addresses
some of the issues raised in FS#25817.
Signed-off-by: Dan McGee <dan@archlinux.org>
We were using atol(), which on 32 bit, cannot handle values greater than
2GiB, which is fail.
Switch to a strtoull() wrapper function tailored toward parsing off_t
values. This allows parsing of very large positive integer values. off_t
is a signed type, but in our usages, we never parse or have a need for
negative values, so the function will return -1 on error.
Before:
$ pacman -Si flightgear-data | grep Size
Download Size : 2097152.00 K
Installed Size : 2097152.00 K
After:
$ ./src/pacman/pacman -Si flightgear-data | grep Size
Download Size : 2312592.52 KiB
Installed Size : 5402896.00 KiB
Signed-off-by: Dan McGee <dan@archlinux.org>
This reduces from 5 to 3 the number of searches needed on the string
looking for a comparison operator, since we can so a second quick
comparison looking for '=' if we find '<' or '>'. It also makes every
search doable with strchr() or memchr() rather than the slower strstr()
method.
In testing, only 10% of splitdep calls (~1600 / 16000) during an -Ss
database load found a version comparison operator, so optimizing the not
found path to be require less work makes sense.
Signed-off-by: Dan McGee <dan@archlinux.org>
If we fell through to the ALPM_DEP_MOD_ANY case, ptr would be NULL, and
we would pass (0 - <str>), which is a rather large negative number or
bogus positive number, depending on signed/unsigned. Just use strdup in
the case where we don't have a ptr available.
Signed-off-by: Dan McGee <dan@archlinux.org>
Hard to believe there was still more room to improve on this, but I
found an easily correctable oversight tonight. Our databases (both sync
and local) contain many blank lines, and we were not moving onto the
next line right away in these cases; instead we would proceed through
our strcmp() conditional checks as normal.
Some local numbers follow to show the effects of this patch:
Sync `-Ss foobarbaz`:
71,709 blank lines skipped early
~1,505,889 strcmp() calls avoided (21 per line)
~15% speed improvement (.210 --> .179 sec)
Local `-Qs foobarbaz`:
6,823 blank lines skipped early
115,991 strcmp() calls avoided (17 per line)
~6% speed improvement (.080 -> .071 sec)
Signed-off-by: Dan McGee <dan@archlinux.org>
Add new alpm_pkg_get_origin() method, use it in the front end now that
the enum constants are publicly available.
Signed-off-by: Dan McGee <dan@archlinux.org>
libalpm now exports type alpm_pkgfrom_t in alpm.h, which may be used
by frontends.
Pacman now uses alpm_pkgfrom_t instead of replicating that type (pkg_from
as was in src/pacman/package.h)
Updated API change in README.
Signed-off-by: Diogo Sousa <diogogsousa@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
There are many other ways to fail a package load other than "file not
found". We should also use the correct error code in this case. Clean it
up a bit in the various callers.
Signed-off-by: Dan McGee <dan@archlinux.org>
Just pass the entire sync package in if we have it; that way we can do
any necessary operations involving it rather than have a parameter list
growing endlessly.
Signed-off-by: Dan McGee <dan@archlinux.org>
I was trying to take a shortcut and not introduce a wrapper struct for
the signature results, so packed it all into alpm_sigresult_t in the
first iteration. However, this is painful when one wants to add new
fields or only return information regarding a single signature.
Refactor the type into a few components which are exposed to the end
user, and will allow a lot more future flexibility. This also exposes
more information regarding the key to the frontend than was previously
available.
The "private" void *data pointer is used by the library to store the
actual key object returned by gpgme; it is typed this way so the
frontend has no expectations of what is there, and so we don't have any
hard gpgme requirement in our public API.
Signed-off-by: Dan McGee <dan@archlinux.org>
Free "md5sum" if md5_file() fails in alpm_compute_md5sum(). Free
"sha256sum" if sha2_file() fails in alpm_compute_sha256sum().
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
Do all the checks at once; this also avoids the 'return' call after we
have allocated memory for "pkgpath" as well as tweaked the umask.
Signed-off-by: Dan McGee <dan@archlinux.org>
Free "syncpath" and restore umask if we fail to grab a lock.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
After commit 2e7d002315, we use off_t rather than long variables.
Use the _LARGE variants of the methods to indicate we are passing off_t
sized variables, and cast using (curl_off_t) accordingly.
Signed-off-by: Dan McGee <dan@archlinux.org>
There were many cases where the string coming in was a blank line, e.g.
"\n\0", length 1. The trim routine starts by trimming leading spaces,
thus trimming everything. We would then proceed to do a memmove of the
NULL byte, which is completely worthless as we can just assign it
instead.
Signed-off-by: Dan McGee <dan@archlinux.org>
This one wasn't all that necessary as we only used it in one place in
the function, which can be checked easily enough at the call site.
Signed-off-by: Dan McGee <dan@archlinux.org>
This handles the no Content-Length header problem as stated in the
comments of FS#23413. We add a quick check to the callback that will
force an abort if the downloaded data exceeds the payload size, and then
check for this error in the post-download cleanup code.
Signed-off-by: Dan McGee <dan@archlinux.org>
Beautiful of libcurl to use floating point types for what are never
fractional values. We can do better, and we usually want these values in
their integer form anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
Improved alpm_list_mmerge() performance by removing an extra
pass to obtain the tail node.
This was actually suggested by a TODO comment.
Signed-off-by: Diogo Sousa <diogogsousa@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We had this interesting set of facts conundrum, according to vercmp
return values:
2.0a < 2.0
2.0 < 2.0.a
2.0a == 2.0.a
This introduces a code change that ensures '2.0a < 2.0.a' as would be
expected by the first two comparisons. Unfortunately this stays us a bit
further from upstream RPM code, but those are the breaks (in RPM, the
versions involving 'a' do in fact compare the same, but they are both
greater than the bare '2.0').
Signed-off-by: Dan McGee <dan@archlinux.org>
Not sure how or why some of this differed, but it is easy enough to set
it back to how it was so it is easier to diff.
Signed-off-by: Dan McGee <dan@archlinux.org>
This avoids error spam when no servers are configured for a repo and a
sync operation is performed:
Proceed with installation? [Y/n] y
:: Retrieving packages from testing...
warning: failed to retrieve some files from testing
warning: failed to retrieve some files from testing
warning: failed to retrieve some files from testing
warning: failed to retrieve some files from testing
warning: failed to retrieve some files from testing
warning: failed to retrieve some files from testing
warning: failed to retrieve some files from testing
warning: failed to retrieve some files from testing
warning: failed to retrieve some files from testing
warning: failed to retrieve some files from testing
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Since we store this directly in the download function, just rework
mask_signal() to take a pointer to a location to store the original.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a precursor to a following patch which will move the setting of
options to a separate function. With the open mode as part of the
struct, we can avoid modifying stack allocated variables.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is more in line with the menagerie of file name members that we now
have on the payload struct.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
These are private to the download operation already, so glob them onto
the struct. This is an ugly rename patch, with the only logical change
being that destfile and tempfile are now freed by the payload_free
function.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This was a vestige leftover from the libfetch days of yore.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
In the case of a non-operation (e.g. DNS resolver error), delete the
leftover 0 byte .part file.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a far more accurate description of what this is, since it's more
than likely not really a filename at all, but the name after a final
slash on a URL.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a refactor and refresh of the code used to find where we should
download packages.
* Incorporate suggestions from FS#25435 to use TMPDIR from the
environment if set, otherwise fall back to /tmp as before.
* Make the writability tests a bit more in depth. We now do a three part
check consisting of:
- S_ISDIR(): is this even a directory
- access(W_OK): is this directory writable by the current user.
Unfortunately for root, this almost always returns that it is, but
in the case of a RO mount or NFS share inaccessible to root, this
check will exclude the directory.
- mode & (any write bit): is there a writable bit set on this
directory. This makes it possible to enforce a read-only cache
directory by setting permissions to 0555, for example.
Signed-off-by: Dan McGee <dan@archlinux.org>
This changes the signature of _alpm_pkg_dup() to return an integer error
code and provide the new package in a passed pointer argument. All
callers are now more robust with checking the return value of this
function to ensure a fatal error did not occur.
We allow load failures to proceed as otherwise we have a chicken and egg
problem- if a 'desc' local database entry is missing, the best way of
restoring said file is `pacman -Sf --dbonly packagename`. This patch
fixes a segfault that was occurring in this case.
Fixes the segfault reported in FS#25667.
Signed-off-by: Dan McGee <dan@archlinux.org>
ftp and http both define >=400 as being "something bad happened"
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Let callers of _alpm_download state whether we should delete on fail,
rather than inferring it from context. We still override this decision
and always unlink when a temp file is used.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
$ pacman -S cronie
resolving dependencies...
looking for inter-conflicts...
:: cronie and fcron are in conflict (@.). Remove fcron? [y/N] n
$ sudo pacman -S pacman
resolving dependencies...
looking for inter-conflicts...
:: pacman and pacman-git are in conflict (pKÈ). Remove pacman-git? [y/N]
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
The whole first loop is trying to check literals only, so teach it to do
so. Also, reorder operations to make more sense by putting the strcmp()
first in the literal loop, and using a very cheap name_hash check first
in the second loop.
Signed-off-by: Dan McGee <dan@archlinux.org>
When we switched to using alpm_depcmp() in resolving replacments, we had
some interesting behavior with regard to providers and packages not
found in repositories. Teach the replacement resolving code to not look
at provisions at all to be slightly more sane.
Signed-off-by: Dan McGee <dan@archlinux.org>
This omits the finding of matching provisions and only checks the
package itself against the provided dep.
Signed-off-by: Dan McGee <dan@archlinux.org>
The functions alpm_db_get_name(), alpm_pkg_get_name(), and
alpm_pkg_get_version() are not necessary at all, so remove the calling
and indirection when used in the backend, which makes things slightly
more efficient and reduces code size.
Signed-off-by: Dan McGee <dan@archlinux.org>
* Move is_local standalone field to status enum
* Create VALID/INVALID flag pair
* Create EXISTS/MISSING flag pair
With these additional fields, we can be more intelligent with database
loading and messages to the user. We now only warn once if a sync
database does not exist and do not continue to try to load it once we
have marked it as missing.
The reason for the flags existing in pairs is so the unknown case can be
represented. There should never be a time when both flags in the same
group are true, but if they are both false, it represents the unknown
case. Care is taken to always manipulate both flags at the same time.
Signed-off-by: Dan McGee <dan@archlinux.org>
Use the STRDUP macro instead of strdup() for the sake of better error
handling on memory allocation failures.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
Return with ALPM_ERR_WRONG_ARGS instead of causing a potential segfault
if alpm_fetch_pkgurl() is invoked with a NULL URL.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
This moves all the delete-on-fail logic to under cleanup label. This
also implies should_unlink when a payload is received that doesn't allow
resuming.
Fixes .db.sig.part files leftover in the sync dir.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This leverages earlier work that avoids a rename when destfile is unset.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
stat()'s behaviour is undefined if the first argument is NULL and might
be prone to segfault. Add an additional check to skip the stat()
invocation if no destfile is used.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
Avoid a potential segfault that may occur if we use a temporary file and
fail to build the destination file name from the effective URL.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
This function is used regardless of whether gpgme support is enabled, so
make sure it is always accessible.
Signed-off-by: Dan McGee <dan@archlinux.org>
This reverts some hacky behavior from 5fc3ec and resets the handle's
pm_errno where it should be reset -- prior to each download. This
prevents a transaction with a download from being aborted when a package
is successfully grabbed from a secondary server.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
The precedence goes as follows: signature > sha256sum > md5sum
Add some logic and helper methods to check what we have available when
loading a package, and then only check what is necessary to verify the
package. This should speed up sync database verifies as we no longer
will be doing both a checksum and a signature validation.
Signed-off-by: Dan McGee <dan@archlinux.org>
It would prevent compilation of pacman on FreeBSD, and possibly other
systems.
Signed-off-by: Rémy Oudompheng <remy@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Previously, the behavior was such that if a sync operation required
packages from multiple repos, a download error in the first repo would
cause a hard repo, ignoring the remainder of the repositories. Change
this behavior so that we do a better job of fetching as many packages as
possible before aborting the transaction.
There's a little bit of refactoring mixed in here to get rid of some
useless variables. Since we now depend heavily on the value of
handle->pm_errno being accurate the determine the function's return
value, we clear it when the transaction state is set.
Fixes FS#25532.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
A partial fix for this was in commit 7de92cb22, but this should fix the
remaining cases. There are still several issues dealing with "provision
as replacement" selection however.
Addresses FS#25538 and FS#25527.
Signed-off-by: Dan McGee <dan@archlinux.org>
We did this with depends way back in commit c244cfecf6 in 2007. We
can do it with these fields as well.
Of note is the inclusion of provides even though only '=' is supported-
we'll parse other things, but no guarantees are given as to behavior,
which is more or less similar to before since we only looked for the
equals sign.
Also of note is the non-inclusion of optdepends; this will likely be
resolved down the road.
The biggest benefactors of this change will be the resolving code that
formerly had to parse and reparse several of these fields; it only
happens once now at load time. This does lead to the disadvantage that
we will now always be parsing this information up front even if we never
need it in the split form, but as these are uncommon fields and our
parser is quite efficient it shouldn't be a big concern.
Signed-off-by: Dan McGee <dan@archlinux.org>
We may end up allocating 1 or 2 extra bytes this way, but it is worth it
to simplify the method and not have to call base64_decode() a second
time. Use the hueristic that base64 encoding produces 3 bytes of decoded
data for every 4 bytes of encoded data.
Signed-off-by: Dan McGee <dan@archlinux.org>
These items are never present in anything but sync databases, nor do we
even try to load them from the local database. Remvoe the indirection
meant to allow the caching layer to work since it will never do anything
anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
This prevents iteration through the remainder of the current tree, with
pacman claiming that they're all replacements to the original
replacement candidate.
:: Synchronizing package databases...
allanbrokeit is up to date
testing is up to date
core is up to date
extra is up to date
community-testing is up to date
community is up to date
:: Starting full system upgrade...
:: Replace util-linux-git with core/util-linux? [Y/n] n
:: Replace util-linux-git with core/vi? [Y/n] n
:: Replace util-linux-git with core/vpnc? [Y/n] n
:: Replace util-linux-git with core/wget? [Y/n] n
:: Replace util-linux-git with core/which? [Y/n] n
:: Replace util-linux-git with core/wireless-regdb? [Y/n] n
:: Replace util-linux-git with core/wireless_tools? [Y/n] n
:: Replace util-linux-git with core/wpa_actiond? [Y/n] n
:: Replace util-linux-git with core/wpa_supplicant? [Y/n] n
:: Replace util-linux-git with core/xfsprogs? [Y/n] n
:: Replace util-linux-git with core/xinetd? [Y/n] n
:: Replace util-linux-git with core/xz? [Y/n] n
:: Replace util-linux-git with core/zd1211-firmware? [Y/n] n
:: Replace util-linux-git with core/zlib? [Y/n] n
there is nothing to do
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This adds a field in the package struct for this checksum type as well
as allowing access via the API to it. The frontend is now able to
display any read value. Note that this does not implement any use or
verification of the value internally.
Signed-off-by: Dan McGee <dan@archlinux.org>
adds a new API method: alpm_pkg_get_base64_sig
[Dan: don't use a new header string in frontend]
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Take this opportunity to refactor the if/then/else logic into a
switch/case which is likely going to be needed to fine tune more
exceptions in the future.
Fixes FS#25531
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This closely matches what we had before for -R --recursive. Basically,
when specifying a target (e.g., pacman), we can now recursively pull all
dependencies, regardless of version specifiers and whether they are
already satisfied in the local database. This could be used to update
pacman on a system with an old glibc, for example, as both pacman and
glibc would get pulled into the transaction.
This is most useful with --needed to prevent needless reinstalls as
described in the man page changes.
The end goal of this change is to wire it into SyncFirst and have it be
the default mode of operation there, but that belongs in a separate
changeset.
Signed-off-by: Dan McGee <dan@archlinux.org>
This prevents possible null dereferences in FTP transfers when the
progress callback is touched during connection teardown.
http://curl.haxx.se/mail/lib-2011-08/0128.html
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is done extremely crudely and is not very efficient, but it does
push us down the path of being closer to right, as one additional test
now passes.
Signed-off-by: Dan McGee <dan@archlinux.org>
* Add *_hash fields to conflict struct and populate them
* Remove unnecessary backwards string comparisons
Signed-off-by: Dan McGee <dan@archlinux.org>
This also pulls in some early translations we had entered in Transifex
in the last day so those would not be lost. The diffstat is huge and not
very telling as usual, as all sorts of fuzzyness switches happened this
time around for some reason.
Signed-off-by: Dan McGee <dan@archlinux.org>
If the string was zero-length to begin with, or consists of only newline
characters, nothing stopped us from incrementing right off the front of
the string. Ensure len stays above zero the whole time.
Signed-off-by: Dan McGee <dan@archlinux.org>
Noticed in my PowerPC Linux VM:
cc1: warnings being treated as errors
dload.c:45: error: 'get_filename' defined but not used
make[3]: *** [dload.lo] Error 1
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a fairly valid assumption at this point, or at least as good of
one as assuming packages all have names.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a bit of a mess, due to the fact that we have a progress meter
running. It is also ironic that we are in the midst of a method named
"commit" when we haven't done a damn thing yet, and can still fail hard
if either a checksum or signature is invalid or unrecognized.
Adapt the former test_md5sum method to be invoked for any of the various
failure types, which at least gives the user some indication of what
packages are failing. A second patch will be needed to actually show
worthwhile error codes, but this is going to involve modifying the
actual data passed with the callback.
Signed-off-by: Dan McGee <dan@archlinux.org>
This moves us toward staring translations for the 4.0.0 release,
although this should not be interpreted as a string freeze by any means.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is standard procedure elsewhere and cuts down on translations that
won't be seen (and we don't want if we need English debug output
anyway).
Signed-off-by: Dan McGee <dan@archlinux.org>
If we are missing a local database file, we get repeated messages over
and over telling us the same thing, rather than being sane and erroring
only once. This package adds an INFRQ_ERROR level that is added to the
mask if we encounter any errors on a local_db_read() operation, and
short circuits future calls if found in the value. This fixes FS#25313.
Note that this does not make any behavior changes other than suppressing
error messages and repeated code calls to failure cases; we still have
more to do in the "local database is hosed" department.
Also make a small update to the wrong but unused flags set in
be_package; using INFRQ_ALL there was not totally correct.
Signed-off-by: Dan McGee <dan@archlinux.org>
We did a good job checking this in add.c, but not necessarily anywhere
else. Fix this up by adding checks into dload.c, remove.c, and conf.c in
the frontend. Also add loggers where appropriate and make the message
syntax more consistent.
Signed-off-by: Dan McGee <dan@archlinux.org>
We don't write with extra or unknown whitespace, so there is little
reason for us to trim it when reading either. This also fixes the
hopefully never encountered "paths that start or end with spaces" issue,
for which two pactests have been added. The tests also contain other
evil characters that we have encountered before and handle just fine,
but it doesn't hurt to ensure we don't break such support in the future.
Signed-off-by: Dan McGee <dan@archlinux.org>
This ensures we are actually making correct use of the information gpgme
is returning to us. Marginal being allowed was obvious before, but
Unknown should deal with trust level, and not the presence or lack
thereof of a public key to validate the signature with.
Return status and validity information in two separate values so check
methods and the frontend can use them independently. For now, we treat
expired keys as valid, while expired signatures are invalid.
Signed-off-by: Dan McGee <dan@archlinux.org>
Immediately jump to the cleanup code after setting the return code to -1
in case rename() fails. Otherwise, it will be reset to 0 right after we
leave the if branch.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
This accomplishes quite a few things with one rather invasive change.
1. Iteration is much more performant, due to a reduction in pointer
chasing and linear item access.
2. Data structures are smaller- we no longer have the overhead of the
linked list as the file struts are now laid out consecutively in
memory.
3. Memory allocation has been massively reworked. Before, we would
allocate three different pieces of memory per file item- the list
struct, the file struct, and the copied filename. What this resulted
in was massive fragmentation of memory when loading filelists since
the memory allocator had to leave holes all over the place. The new
situation here now removes the need for any list item allocation;
allocates the file structs in contiguous memory (and reallocs as
necessary), leaving only the strings as individually allocated. Tests
using valgrind (massif) show some pretty significant memory
reductions on the worst case `pacman -Ql > /dev/null` (366387 files
on my machine):
Before:
Peak heap: 54,416,024 B
Useful heap: 36,840,692 B
Extra heap: 17,575,332 B
After:
Peak heap: 38,004,352 B
Useful heap: 28,101,347 B
Extra heap: 9,903,005 B
Several small helper methods have been introduced, including a list to
array conversion helper as well as a filelist merge sort that works
directly on arrays.
Signed-off-by: Dan McGee <dan@archlinux.org>
As noted by Allan, we failed pretty hard if gpgme was compiled out. With
these changes, only sign001.py fails. This can/will be fixed later once
we beef up the test suite with more signing tests anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
If we can't read the keyring, gpgme will output confusing debug
information and fail to verify the signature, so we should log some
debug information.
Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a wrapper function for access() which logs some debug
information and eases handling in case of split directory and filename.
Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Dan McGee <dan@archlinux.org>
This addresses FS#25141. We shouldn't remove every empty directory we
come across during the removal process unless it is truly not known to
any other package. This will prevent removal of essential directories
such as '/var/lock/'.
This is accomplished by first checking the empty/non-empty status of a
directory, which was previously done implicitly by calling rmdir() and
ignoring errors. We do this to avoid the next (new) check in most cases,
which is to look at all local packages to see if the to-be-removed
directory is present in another packages' filelist. If we do not find it
anywhere, then we remove it, else we keep the file around.
The pactest has been updated to test more cases, as well as finding a
flaw in the original expected to fail case- we need separate DIR and
FILE based EXIST rules.
Signed-off-by: Dan McGee <dan@archlinux.org>
This can only ever operate on the local database, and a local package at
that. Change the function signature to take a handle and package object,
add the relevant asserts, and ensure the frontend can detect the package
not found condition when finding packages to pass to this method.
Signed-off-by: Dan McGee <dan@archlinux.org>
The bulk of this commit is adding new tests to ensure the new behavior
works without disrupting old behavior. This is a relatively sane maneuver
when a package adds a conf file (e.g. '/etc/mercurial/hgrc') that was
not previously in the package, but it is placed in the backup array. In
essence, we can treat the existing file as having always been a part of
the package and do our normal compare/install as pacnew logic checks.
Signed-off-by: Dan McGee <dan@archlinux.org>
This code duplication has always been a rather clumsy casuality of
fixing some past upgrade issues. Unify the removal code across upgrade
and remove operations into a new _alpm_remove_single_package() method
wihch makes it very clear how we handle upgrade and remove differently,
via several conditionals on newpkg.
This commit highlights interesting behavior such as the fact that the
implicit removal in every package upgrade never gets transaction events
or progress callbacks.
Signed-off-by: Dan McGee <dan@archlinux.org>
Fixes "error: no previous prototype for '_alpm_raw_cmp'
[-Werror=missing-prototypes]" warnings, and also prevents someone from
getting the prototypes and functions out of sync.
Signed-off-by: Dan McGee <dan@archlinux.org>
Restore some sanity to the number of arguments passed to _alpm_download
and curl_download_internal.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
This means creating a new struct which can pass more descriptive data
from the back end sync functions to the downloader. In particular, we're
interested in the download size read from the sync DB. When the remote
server reports a size larger than this (via a content-length header),
abort the transfer.
In cases where the size is unknown, we set a hard upper limit of:
* 25MiB for a sync DB
* 16KiB for a signature
For reference, 25MiB is more than twice the size of all of the current
binary repos (with files) combined, and 16KiB is a truly gargantuan
signature.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
URLs might end with a slash and follow redirects, or could be a
generated by a script such as /getpkg.php?id=12345. In both cases, we
may have a better filename that we can write to, taken from either
content-disposition header, or the effective URL.
Specific to the first case, we write to a temporary file of the format
'alpmtmp.XXXXXX', where XXXXXX is randomized by mkstemp(3). Since this
is a randomly generated file, we cannot support resuming and the file is
unlinked in the event of an interrupt.
We also run into the possibility of changing out the filename from under
alpm on a -U operation, so callers of _alpm_download can optionally pass
a pointer to a *char to be filled in by curl_download_internal with the
actual filename we wrote to. Any sync operation will pass a NULL pointer
here, as we rely on specific names for packages from a mirror.
Fixes FS#22645.
Signed-off-by: Dave Reisner <d@falconindy.com>
The supposed safety blanket of this function is better handled by
explicit length checking and usages of strlen() on known NULL-terminated
strings rather than hoping things fit in a buffer. We also have no need
to fully fill a PATH_MAX length variable with NULLs every time as long
as a single terminating byte is there. Remove usages of it by using
strcpy() or memcpy() as appropriate, after doing length checks via
strlen().
Signed-off-by: Dan McGee <dan@archlinux.org>
We can readily detect the first node in a list by checking if
node->prev->next is NULL. So there is no need to pass the head
of the list to this function and its prototype now looks like
all the other item accessors.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
The only thing this accessor did was remove the const qualifier
given our entire list implementation requires passing around the
head anyway.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
They are placeholders, but important for things like trying to re-sync a
database missing a signature. By using the alpm_db_validity() method at
the right time, a client can take the appropriate action with these
invalid databases as necessary.
In pacman's case, we disallow just about anything that involves looking
at a sync database outside of an '-Sy' operation (although we do check
the validity immediately after). A few operations are still permitted-
'-Q' ops that don't touch sync databases as well as '-R'.
Signed-off-by: Dan McGee <dan@archlinux.org>
Show output in -Qip for each package signature, which includes the UID
string from the key ("Joe User <joe@example.com>") and the validity of
said key. Example output:
Signatures : Valid signature from "Dan McGee <dpmcgee@gmail.com>"
Unknown signature from "<Key Unknown>"
Invalid signature from "Dan McGee <dpmcgee@gmail.com>"
Also add a backend alpm_sigresult_cleanup() function since memory
allocation took place on this object, and we need some way of freeing
it.
Signed-off-by: Dan McGee <dan@archlinux.org>
The error code is in fact a bitmask value of an error code and an error
source, so use the proper function to get only the relevant bits. For
the no error case, this shouldn't ever matter, but it bit me when I was
trying to compare the error code to other values and wondered why it
wasn't working, so set a good example.
Signed-off-by: Dan McGee <dan@archlinux.org>
This gives us more granularity than the former Never/Optional/Always
trifecta. The frontend still uses these values temporarily but that will
be changed in a future patch.
* Use 'siglevel' consistenly in method names, 'level' as variable name
* The level becomes an enum bitmask value for flexibility
* Signature check methods now return a array of status codes rather than
a simple integer success/failure value. This allows callers to
determine whether things such as an unknown signature are valid.
* Specific signature error codes mostly disappear in favor of the above
returned status code; pm_errno is now set only to PKG_INVALID_SIG or
DB_INVALID_SIG as appropriate.
Signed-off-by: Dan McGee <dan@archlinux.org>
Now that the filelists capture mode and size information, we can read
the data from there and prevent having to loop through and uncompress
every archive to check required diskspace usage.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows us to capture size and mode data when building filelists
from package files. Future patches will take advantage of this newly
available information, and frontends can use it as well.
Signed-off-by: Dan McGee <dan@archlinux.org>
This saves replicating the potentially large list of files in a package
that is being removed.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We passed in 'line', but not 'buf.line'. In addition, the macros
building off of READ_NEXT() assume variable names anyway. Since we only
use these macros in one function, might as well simplify them.
Signed-off-by: Dan McGee <dan@archlinux.org>
Change the check into a loop over all signatures present and returned by
GPGME. Also modify the return values and checks slightly now that I know
a little bit more about what type of values are returned.
Signed-off-by: Dan McGee <dan@archlinux.org>
There is little need to expose the guts of this function even within the
library. Make it static in be_local.c, and clean up a few other things
since we know exactly where it is being called from:
* Remove unnecessary origin checks in _cache_get_*() methods- if you are
calling a cache method your package type will be correct.
* Remove sanity checks within local_db_read() itself- packages will
always have a name and version if they get this far, and the package
object will never be NULL either.
The one case calling this from outside the backend was in add.c, where
we forced a full load of a package before we duplicated it. Move this
concern elsewhere and have pkg_dup() always force a full package load
via a new force_load() function on the operations callback struct.
Signed-off-by: Dan McGee <dan@archlinux.org>
Some of these are legit (the backup hash NULL checks), while others are
either extemely unlikely or just impossible for the static code
analysis to prove, but are worth adding anyway because they have little
overhead.
Signed-off-by: Dan McGee <dan@archlinux.org>
Modifying prefix caused tmp directories to be left behind after
running scriptlets, and the path '/' to be passed to _alpm_rmrf. Broken
in f01c6f.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This avoids, probably among other things, leaving the lock file in place
after a SIGINT'd sync DB update.
Fixes regression introduced in 4f8ae2b.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
The following function renames take place for the same reasoning as
the previous commit:
_alpm_grp_new -> _alpm_group_new
_alpm_grp_free -> _alpm_group_free
_alpm_db_free_grpcache -> _alpm_db_free_groupcache
_alpm_db_get_grpfromcache -> _alpm_db_get_groupfromcache
Signed-off-by: Allan McRae <allan@archlinux.org>
Using grp instead of group is a small saving at the cost of clarity.
Rename the following functions:
alpm_option_get_ignoregrps -> alpm_option_get_ignoregroups
alpm_option_add_ignoregrp -> alpm_option_add_ignoregroup
alpm_option_set_ignoregrps -> alpm_option_set_ignoregroups
alpm_option_remove_ignoregrp -> alpm_option_remove_ignoregroup
alpm_db_readgrp -> alpm_db_readgroup
alpm_db_get_grpcache -> alpm_db_get_groupcache
alpm_find_grp_pkgs -> alpm_find_group_pkgs
Signed-off-by: Allan McRae <allan@archlinux.org>
Only one of these looked like a real red flag, in find_requiredby(), but
it doesn't hurt to fix several of them up anyway.
Unfortunately, we can't turn this on universally due to things like the
sync(), remove(), etc. builtins which we often use as variable names.
Signed-off-by: Dan McGee <dan@archlinux.org>
We have just looped through the list of files, so might as well get
the count as we go.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This addresses FS#24904. In a normal upgrade case, this replacement
seems to work just fine. However, when doing a sync "replace" type
upgrade, we weren't properly handling this edge case due to path
comparison not ignoring trailing slashes. Fix this by pruning any
trailing slashes past a certain point of file conflict resolution where
we no longer need them, which allows us to safely detect cases such as
now tested in the new pactest.
Signed-off-by: Dan McGee <dan@archlinux.org>
While researching the root cause of FS#24904, I couldn't help but clean
up some of the cruft in here. A few whitespace/line-wrapping issues, but
also fix shadowed variables and add some const where applicable.
Signed-off-by: Dan McGee <dan@archlinux.org>
We can reorganize things a bit to not require reading a directory-only
entry first (or at all). This was noticed while working on some pactest
improvements, but should be a good step forward anyway.
Also make _alpm_splitname() a bit more generic in where it stores the
data it parses.
Signed-off-by: Dan McGee <dan@archlinux.org>
Discovered this when doing some pactest rewrite work to generate
archives in memory only. If a sync database file or PKGINFO file is
missing a newline on the final line, the text from that line gets tossed
aside and never read into the package struct. This is pretty critical
when that last line is a depend or something.
Signed-off-by: Dan McGee <dan@archlinux.org>
These operate on the handle, and the state is stored on the handle, so
move them where they belong. Up until now only the transaction stuff
calls them, but this will soon change and alpm_db_update() will handle
locking all on its own.
Signed-off-by: Dan McGee <dan@archlinux.org>
Start by converting all of our flags to a 'status' bitmask (pkgcache
status, grpcache status). Add a new 'valid' flag as well. This will let
us keep track if the database itself has been marked valid in whatever
fashion.
For local databases at the moment we ensure there are no depends files;
for sync databases we ensure the PGP signature is valid if
required/requested. The loading of the pkgcache is prohibited if the
database is invalid.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is another step toward doing both local database validation
(ensuring we don't have depends files) and sync database validation (via
signatures if present) when the database is registered.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the ideal place to do it as all clients should be checking the
return value and ensuring there are no errors. This is similar to
pkg_load().
We also add an additional step of validation after we download a new
database; a subsequent '-y' operation can potentially invalidate the
original check at registration time.
Note that this implementation is still a bit naive; if a signature is
invalid it is currently impossible to refresh and re-download the file
without manually deleting it first. Similarly, if one downloads a
database and the check fails, the database object is still there and can
be used. These shortcomings will be addressed in a future commit.
Signed-off-by: Dan McGee <dan@archlinux.org>
For the files count when loading from a package, we can keep a counter.
The two in the frontend were completely useless due to the fact that if
sync_dbs is non-NULL, alpm_list_count() will always be greater than 0.
Signed-off-by: Dan McGee <dan@archlinux.org>
This doesn't fix the real (bigger) problem of failing to parse sync
databases without directory entries, but it does prevent the parser from
segfaulting when the first desc file encountered did not have a
directory entry, among other conditions.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the first step at separating the pacman message catalog and the
scripts message catalog. Makefiles, configure.ac, and other such files
are adjusted accordingly, as well as renaming files. The TEXTDOMAIN of
scripts is also adjusted.
Note that no actual pot or po files get changed here; these will get
pruned in a future commit so each catalog contains only the necessary
messages.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is for the eventual 4.0.0 release, but more importantly to
logically separate new translations and strings from the PO split about
to happen between pacman and scripts.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows us to separate the name and hash elements in one place and
not scatter different parsing code all over the place, including both
the frontend and backend.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is an unfortunate chain of events. RET_ERR and RET_ERR_VOID will
eventually call CHECK_HANDLE, which resets the handle's pm_errno member.
Dan probably had a reason for doing this, so we merely switch the order
of operations in the RET_ERR macros to avoid stomping on our pm_errno.
Signed-off-by: Dave Reisner <d@falconindy.com>
* Check the return value of canonicalize_path() for non-NULL
* Use ASSERT and RET_ERR as appropriate
* Make remove_cachedir() use same path munge logic as add_cachedir()
Signed-off-by: Dan McGee <dan@archlinux.org>
Documented the _alpm_download() function in dload.c
Signed-off-by: Kerrick Staley <mail@kerrickstaley.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Added a line to the top of each of be_local.c, be_package.c, and
be_sync.c indicating their purposes.
Signed-off-by: Kerrick Staley <mail@kerrickstaley.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We were using copy_data before; this works for the struct itself but not
the strings contained within. Fix it up by duplicating all the data as
we do with our other structures.
Signed-off-by: Dan McGee <dan@archlinux.org>
Calling get_logcb() here would reset any previous setting of
handle->pm_errno due to the CHECK_HANDLE() macro contained within. This
would make error setting a bit funny if one set pm_errno before calling
_alpm_log(), such as in the RET_ERR() macro.
Signed-off-by: Dan McGee <dan@archlinux.org>
This removes the need to write accessor methods for every type we have,
and simplifies the API. Any type that doesn't need magic* can be
converted in this fashion to make it easier for frontend applications to
use, as well as make it less of a pain to introduce new such structs in
the future.
* "magic" meaning something like pmpkg_t where values can be lazy loaded.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is more in line with reality and what we have our makepkg, etc.
options named anyway.
Original-patch-by: Kerrick Staley <mail@kerrickstaley.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
* Don't name static methods with a gpgme_ prefix to avoid confusion with
methods provided by the library. These are static and local to our
file so just give them sane non-prefixed names.
* Rework sigsum_test_bit() to not require assignment.
* Don't balk if there is more than one signature available (for now,
only check the first).
* Fix error codes in publicly visible methods to return -1, not 0, if pkg
or db are not provided.
Signed-off-by: Dan McGee <dan@archlinux.org>
We didn't do due diligence before and ensure prior pm_errno values
weren't influencing what happened in further ALPM calls. I observed one
case of early setup code setting pm_errno to PM_ERR_WRONG_ARGS and that
flag persisting the entire time we were calling library code.
Add a new CHECK_HANDLE() macro that does two things: 1) ensures the
handle variable passed to it is non-NULL and 2) clears any existing
pm_errno flag set on the handle. This macro can replace many places we
used the ASSERT(handle != NULL, ...) pattern before.
Several other other places only need a simple 'set to zero' of the
pm_errno field.
Signed-off-by: Dan McGee <dan@archlinux.org>
* Move several variables into better scope
* const-ify a few variables
* Avoid duplicating filelists if it is unnecessary
* Better handling out out of memory condition when adding file conflicts
to our list
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows callers to retrieve it from wherever is convenient, which
may or may not be on the package object itself.
Signed-off-by: Dan McGee <dan@archlinux.org>
This method is old, it doesn't adequately check for a NULL server list,
and can easily be done using better API method we provide these days.
All former users of this method can get similar results by calling
alpm_db_get_servers() and using the data from the returned server list.
Signed-off-by: Dan McGee <dan@archlinux.org>
Note that is a bit different than the normal _alpm_db_path() method; the
caller is expected to free the result.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the last user of our global handle object. Once again the diff
is large but the functional changes are not.
Signed-off-by: Dan McGee <dan@archlinux.org>
This makes these functions consistent with the rest of the transaction
related API calls. We do an additional assert to ensure the handle
attached to the package is the same as the handle passed in.
Signed-off-by: Dan McGee <dan@archlinux.org>
The siglevel field of a newly created pmdb_t struct is now
initialized when it is created in _alpm_db_new().
Signed-off-by: Kerrick Staley <mail@kerrickstaley.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
A few of these snuck in as of late, some from the table display patches
that were using the previous format before we changed it after the 3.5.X
major release.
Noticed-by: Kerrick Staley <mail@kerrickstaley.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Commit e68f5d9a30 did something a bit silly and changed the
scriptlet calls to use 'newpkg->handle' rather than the 'handle'
argument passed in. Use the handle directly.
Signed-off-by: Dan McGee <dan@archlinux.org>
Begin enforcing the need to pass a handle. This allows us to remove one
more extern handle declaration from the backend.
Signed-off-by: Dan McGee <dan@archlinux.org>
This requires a lot of line changes, but not many functional changes as
more often than not our handle variable is already available in some
fashion.
Signed-off-by: Dan McGee <dan@archlinux.org>
The few remaining instances were utilized for buffers in calls to
snprintf() and realpath(). Both of these functions will always ensure
the returned value is padded with '\0', so there is no need for the
extra byte.
Signed-off-by: Dan McGee <dan@archlinux.org>
The vast majority of the time we will just be passing the same string
value on to the lstat() call. The only time we need to duplicate it is
if the path ends in '/'. In one run using a profiler, only 400 of the
200,000 calls (0.2%) required the string to be copied first.
Signed-off-by: Dan McGee <dan@archlinux.org>
Due to the way we set up the graph structure, we don't always have good
parent information. The changes made in dd8cf0c12d assumed this, so
back them out and just live with the dead pointers being there in the
memory while we are cleaning up after ourselves.
Signed-off-by: Dan McGee <dan@archlinux.org>
These new method signatures return and take handle objects to operate on
so we can move away from the idea of one global handle in the API. There
is also another important change and that deals with the setting of root
and dbpaths. These are now done at initialization time instead of using
setter methods. This allows the library to operate more safely knowing
that paths won't change underneath it.
Signed-off-by: Dan McGee <dan@archlinux.org>
This keeps duplicate code to a minimum. This will come in more handy as
we refactor some of these option setters away.
Signed-off-by: Dan McGee <dan@archlinux.org>
When only downloading a package, pacman can produce some incorrect
output.
> pacman -Sddw nvidia-utils
warning: nvidia-utils-270.41.19-1 is up to date -- reinstalling
This line is now now silenced when using -Sw.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows us to not require the context (e.g. handle) when calling
this function. Also beef up the checks in the two callers of this
function to bail if the last return code is not ARCHIVE_EOF, which is
the expected value.
This requires a change to one of the pactest return codes and the
overall result of the test, but results in a much safer operating
condition whereby invalid database entries will stop the operation.
Signed-off-by: Dan McGee <dan@archlinux.org>
This kills a lot more global handle business off. sync.c still requires
the handle declaration for one reference that can't be changed yet; it
will be removed in a future patch which isolates all of the necesary API
changes.
Signed-off-by: Dan McGee <dan@archlinux.org>
This will make the patching process less invasive as we start to remove
this variable from all source files.
Signed-off-by: Dan McGee <dan@archlinux.org>
Similar to what we just did for the database; this will make it easy to
always know what handle a given package originated from.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the first step in a long process to remove our dependence on the
global handle variable we currently share in libalpm, with the goal to
make things a bit more thread-safe and re-entrant.
Signed-off-by: Dan McGee <dan@archlinux.org>
These are simple accessor functions for a struct; the handle never even
comes into play when calling these functions.
Signed-off-by: Dan McGee <dan@archlinux.org>
The usefulness of this is rather limited due to it not being compiled
into production builds. When you do choose to see the output, it is
often overwhelming and not helpful. The best bet is to use a debugger
and/or well-placed fprintf() statements.
Signed-off-by: Dan McGee <dan@archlinux.org>
Callers to curl_download_internal now tell us if its okay to continue a
transfer, so obey this instead of using a heuristic.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
If a connection drops below 1kb/s for 10s, curl will kill the transfer
and we'll report failure. This is the average transfer speed over the
delta defined by CURLOPT_LOW_SPEED_TIME, so setting a low value here
shouldn't bother folks using 14.4k dial-up.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This addresses FS#24292. If one does the bad thing of not checking
pm_errno after calling set_dbpath(), you may not realize the
initialization process went wrong and calling trans_init() resulted in a
segfault. If we don't have a lockfile path, bail out and have
trans_init() fail.
Also remove a ALPM_LOG_FUNC call that was causing pm_errno to return "no
handle"; this was due to a log call in the handle setup (whereby the log
attempts to use a callback attached to the handle).
Signed-off-by: Dan McGee <dan@archlinux.org>
Issue FS#24230. If a symlink is broken and included in the removal
process of a package, we blew up and segfaulted due to
alpm_compute_md5sum() returning NULL and then performing a strcmp()
operation.
Signed-off-by: Dan McGee <dan@archlinux.org>
The addition of the DB version check introduces a lag time between the
lockfile creation and the transaction initialization. In cases where the
local DB is large enough and/or the user's disk is slow enough, this
time is significant enough that its possible for a user to send a SIGINT
and leave behind a db.lck file.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
It's your own damn fault if you do this, and this code is remnants from
an old time when we weren't very good at coding.
Signed-off-by: Dan McGee <dan@archlinux.org>
This needlessly prevents the easiest way available of clearing any of these
values. We can also do the same for the 'arch' value.
Signed-off-by: Dan McGee <dan@archlinux.org>
Currently we have one call that has all sorts of crazy behavior and doesn't
make a whole lot of sense. Go from one method to the normal four methods we
have for all of our other lists we use in the library to make it a lot
easier for a frontend to manipulate server lists.
Signed-off-by: Dan McGee <dan@archlinux.org>
The switch from FUNCTION to DEBUG was ill-advised inside the local
database load. Instead, add a DEBUG level logger to both local and sync
database loads that shows the number of packages processed.
Signed-off-by: Dan McGee <dan@archlinux.org>
We were erroring out in the case where a first (possibly bogus) mirror
would cause the download process to return a failure code, even though
subsequent servers had the file.
Signed-off-by: Dan McGee <dan@archlinux.org>
This started off removing the "(void)foo" hacks to work around
unused function parameters and ended up fixing every warning
generated by -Wunused-parameter.
Dan: rename to UNUSED.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We had a lot of similar looking code that we can collapse down into a
function. This also fixes errors seen when turning on some gcc warnings
and implicitly casting away the const-ness of the string. Free the list
when we are done with it as well.
Also, fix a logic error where we should be checking with &&, not ||.
Signed-off-by: Dan McGee <dan@archlinux.org>
This makes it possible to omit usage of -lgpgme, just as we can do for
-lcurl and -lcrypto.
Thanks to Rémy Oudompheng for an initial stab at this.
Signed-off-by: Dan McGee <dan@archlinux.org>
Add some lookup functions for nice names for the various types used by
the library, and remove some fields that are of little use to us in the
debug output. This should make looking at key loading and verification a
bit easier, especially in determining what makes up our good and bad
criteria.
Signed-off-by: Dan McGee <dan@archlinux.org>
Both md5sum verification and PGP verification can and should be done at
package load time. This allows verification to happen as early as
possible for packages provided by filename and loaded in the frontend,
and moves more stuff out of sync_commit that doesn't really belong
there. This should also set the stage for simplified parallel loading of
packages later down the road.
Signed-off-by: Dan McGee <dan@archlinux.org>
More stuff going on in the pre-committing stage that can be in a static
method to make things a bit more clear.
Signed-off-by: Dan McGee <dan@archlinux.org>
This part is almost completely self-contained, except building the list
of delta filenames that we use later to check their md5sums. Refactor it
into a static method so we can bring most of the code in sync_commit
closer to the method name.
Signed-off-by: Dan McGee <dan@archlinux.org>
Given that we offer no transparency into the pmpgpsig_t type, we don't
really need to expose it outside of the library, and at this point, we
don't need it at all. Don't decode anything except when checking
signatures. For packages/files not from a sync database, we now just
read the signature file directly anyway.
Also push the decoding logic down further into the check method so we
don't need this hanging out in a less than ideal place. This will make
it easier to conditionally compile things down the road.
Signed-off-by: Dan McGee <dan@archlinux.org>
Rather than go through all the hassle of doing this ourselves, just let
GPGME handle the work by passing it a file handle.
Signed-off-by: Dan McGee <dan@archlinux.org>
The various "level" values were a bit crazy to decipher, and we were
doing some very interesting comparisons in certain places. Break it out
into two parameters instead so we can seperate the type from the extra
information display, and do things accordingly.
Nothing changes with the display of any of the five types we currently
show: -Si, -Sii, -Qi, -Qii, -Qip.
Something to note- we should expose the PKG_FROM enum type somehow, this
patch leaves the door open to do that quite easily.
Signed-off-by: Dan McGee <dan@archlinux.org>
There's a lot of related moving parts here:
* Iteration through mirrors is moved back to the calling functions. This
allows removal of _alpm_download_single_file and _alpm_download_files.
* The download function gets a few more arguments to influence behavior.
This allows several different scenarios to customize behavior:
- database
- database signature (req'd and optional)
- package
- package via direct URL
- package signature via direct URL (req'd and optional)
* For databases, we need signatures from the same mirror, so structure
the code accordingly.
Some-inspiration-from: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
The allow_resume is the start of the fix to the "don't ever resume
database downloads" problem, as well as being useful for '.sig'
downloads as well. For now, we say "always allow resume", but this will
eventually get pushed down as necessary.
Error checks are reworked in order to correctly error out when a file is
not found on the remote end and reports 0 bytes downloaded. In addition,
the two error messages printed are now different as one reports a more
specific error message provided via the cURL error buffer.
Some example output from an -Sy run with [testing], [community],
[community2], [eee], and [nonexistant] defined as repos. [community2]
and [nonexistant] are both invalid, one using FTP and one using HTTP.
:: Synchronizing package databases...
testing is up to date
community is up to date
error: failed retrieving file 'community2.db' from ftp.archlinux.org : Given file does not exist
error: failed to update community2 (FTP: couldn't retrieve (RETR failed) the specified file)
eee is up to date
error: failed retrieving file 'nonexistant.db' from code.toofishes.net : The requested URL returned error: 404
error: failed to update nonexistant (HTTP response code said error)
Signed-off-by: Dan McGee <dan@archlinux.org>
The value PM_PGP_VERIFY_UNKNOWN is reserved to error cases,
now that the signature verification level defaults to the
globally set level. The only error case is when handle == NULL,
which is false in the context of _alpm_sync_commit().
Signed-off-by: Rémy Oudompheng <remy@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This does touch a lot of things, and hopefully doesn't break things on
other platforms, but allows us to also clean up a bunch of crud that no
longer needs to be there.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the standard, and we have had a few of these introduced lately
that should not be here.
Done with:
find -name '*.c' | xargs sed -i -e 's#if (#if(#g'
find -name '*.c' | xargs sed -i -e 's#while (#while(#g'
Signed-off-by: Dan McGee <dan@archlinux.org>
* add _alpm_db_get_sigverify_level
* add alpm_option_{get,set}_default_sigverify
And set the default verification level to OPTIONAL if not set otherwise.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This makes it absolutely dead easy to ensure off_t has the same length
in all compilation units. I just spent 2.5 hours bashing my head on an
issue related to this so damn it I'm fixing it for good.
Signed-off-by: Dan McGee <dan@archlinux.org>
Currently the only error case then when handle == NULL.
However several handle functions return -1 on this error,
and a uniform API makes things simpler.
Signed-off-by: Rémy Oudompheng <remy@archlinux.org>
For a package to be loaded from any of our backends, these two fields
are always required upfront. Due to this fact, we don't need them to be
backend-specific operations and can just refer to the field directly.
Additionally, our static (and thus private) cache package accessors had
a NULL check on pkg before returning the relevant field. Eliminate this
since they only way they are ever called is via the packages attached
callback struct, which would have caused the NULL pointer dereference in
the first place.
Signed-off-by: Dan McGee <dan@archlinux.org>
So we only need one copy in the final library, not one copy per time
used. Ensure all necessary includes are in place (especially to get the
right size of off_t each time it is compiled) by including "config.h" in
the new graph.c.
One small adjustment here makes the graph_free code more robust- ensure
we don't have invalid pointers after each iteration by looking at the
parents and children and adjusting accordingly.
Signed-off-by: Dan McGee <dan@archlinux.org>
Ensure we only have one- this looks like the result of a bad merge from
old 2008 signing code with the current stuff which has changed quite a
bit.
Originally-seen-by: Rémy Oudompheng <remyoudompheng@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
The current state of the code does not allow to see immediately
that it returns a list of pmdepmissing_t structures.
Signed-off-by: Dan McGee <dan@archlinux.org>
Some systems, like FreeBSD might define both statfs
and statvfs: however if statvfs exists whereas getmntinfo()
uses a statfs struct, the current ifdefs would select the wrong
line of code.
Signed-off-by: Rémy Oudompheng <remy@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
There is no reason to not support versions of libarchive that lack
ARCHIVE_COMPRESSION_UU. Distributions should work properly without
this.
Signed-off-by: Rémy Oudompheng <remy@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
That's a funny one, building with optimization levels (with both gcc and
clang) caused open_mode to always be set to "ab", which worked.
This was spotted both with clang-analyzer, and by Jakob who reported a
segfault as he was using an un-optimized build.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
So we don't segfault when calling this on be_sync loaded packages. They
return logical values as much as possible for indicating there is no
changelog available.
Signed-off-by: Dan McGee <dan@archlinux.org>
We didn't do this sanity check before trying to open an archive. If
the alpm dbpath wasn't set, the sync database dbpath would be NULL,
causing us to hang indefinitely in archive_read_open_filename() rather
than erroring out.
We already have a corresponding check in local_db_populate().
The following program will test this case, and hangs before this patch
without the call to set_dbpath:
int main(int argc, char *argv[]) {
alpm_initialize();
// alpm_option_set_dbpath("/var/lib/pacman/");
pmdb_t *core = alpm_db_register_sync("core");
pmpkg_t *pkg = alpm_db_get_pkg(core, "pacman");
return 0;
}
Signed-off-by: Dan McGee <dan@archlinux.org>
We had two functions that were oh so similar but slightly different. We
can combine them and add some conditional operation stuff to decide what
to return.
Signed-off-by: Dan McGee <dan@archlinux.org>
Many alpm_option_get/set_*() functions already check this
and set pm_errno to the right value, but not all, so
this improves consistency.
Signed-off-by: Rémy Oudompheng <remy@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This greatly simplifies the cleanup fallthrough in our download function
and we'll be able to reuse this for signatures.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Based on the fact that localf always points to the same file, there's no
need to code in multiple fopen calls with varying results. Instead,
track the desired file open mode and make a single call to fopen.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Create a more general function that allows appending a suffix to a
filepath.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This lets us determine the real size of the file on disk so that we can
properly bump the progress bar when we're resuming a download.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This addresses FS#23424. The -dd backend code was introduced in commit
b6ec9019d7, and unfortunately the munged depend used for comparison did
not carry through to the eventual display of this version. To fix this,
we undo some of the depcmp_tolerant() business introduced, and instead
make a new pmdepend_t object if necessary when the no dependency version
flag is set. This results in the correct depend being copied to the
missing depend passed onto the frontend.
Signed-off-by: Dan McGee <dan@archlinux.org>
pacman 3.5.0 removed alpm_db_register_local, so calling
alpm_db_unregister_all leaves the front end in a position where there's
no local db, and no way to re-register it.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
After updating a database, remove the old signature to prevent it
being used in validation if the new signature fails to download.
Signed-off-by: Allan McRae <allan@archlinux.org>
If signature verification is needed, attempt to download a signature
file for a repo when it is updated. Return an error if unable to
download signature only when checking is mandatory, or if signature is
invalid.
TODO: At the moment the database signature is only checked on download.
Should we do anything with a database if it fails to be verified to prevent
its future usage?
Signed-off-by: Allan McRae <allan@archlinux.org>
Add a pmpgpsig_t struct to the database entry struct and functions for
the lazy loading of database signatures. Add a function for checking
database signatures, reusing (and generalizing) the code currently used
for checking package signatures.
TODO: The code for reading in signature files from the filesystem is
duplicated for local packages and database and needs refactoring.
Signed-off-by: Allan McRae <allan@archlinux.org>
Our curl callback does a whole lot of work for nothing if the front end
never defined a callback to receive the data we'd calculate for it.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
CURLINFO_HTTP_CODE is deprecated in favor of CURLINFO_RESPONSE_CODE.
Both yield the same values.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
The files we transfer are generally compressed already, so this just
adds unnecessary overhead.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Use a static variable to effectively track the initialization state of
the progress callback via the last byte amount reported as downloaded by
libcurl.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
* introduces new macro in util.h (DOUBLE_EQ) for properly comparing
floating point values
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Move the (possibly still temporary) output generated during signature
checking into the --debug output.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Once we do this, add support for VerifySig to pactest. We just check if
the repo name contains Always, Never or Optional to determine the value
of VerifySig. The default is Never. pacman uses Always by default but
this is not suitable for pactest.
Original-work-by: shankar <jatheendra@gmail.com>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We pass in a db object, so no need to go looking for it in the list on
the handle. This is a remnant of when we passed in a treename, more than
likely.
Signed-off-by: Dan McGee <dan@archlinux.org>
This will serve as the home directory we pass to GPGME when making calls so
we can have a libalpm-utilized keyring.
Signed-off-by: Dan McGee <dan@archlinux.org>
If a .sig file sits side-by-side on the filesystem with a package archive,
read it in during the package struct creation process so we can verify it at
a later time if necessary.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Add a new field to the package struct to hold PGP information and
instruct db_read to pick it up from the database. It is currently unused
internally but this is the first step.
Due to the fact that we store the PGP sig as binary data, we need to store
both the data and the length so we have a small utility struct to assist us.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
We will need these for GPG functionality (decoding the base64 encoded
signature stored in the databases).
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Implements FS#23103. Also modify libalpm so it ignores this value
without any warning as we know it is likely to exist.
Signed-off-by: Dan McGee <dan@archlinux.org>
Wrap lines of long length, noticed while creating and messing around
with some of the other maint branch patches.
Signed-off-by: Dan McGee <dan@archlinux.org>
Noticed with the openoffice/libreoffice replacement scheme where many
packages are listed as replacements to one package, thus electing it for
removal multiple times. Ensure a given package is not already present
before placing it in the removal list.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a rather serious data corruption issue that luckily manifested
itself today in a noticable way. A package in testing had replaces
entries read in as ["%RE pkgname", "%RE"] which was clearly wrong. This
happens when we hit the end of an archive block, do not have a newline,
and have to continue reading from the next block to complete the line.
Signed-off-by: Dan McGee <dan@archlinux.org>
Drawing progress bars before calling curl_easy_perform() is needless as
the curl progress callback is called with zero progress before actually
downloading the file anyways. Fixes display of "0%" progress bars when
sync'ing package databases that are already up to date.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
This was discussed and more or less agreed upon on the mailing list. A
huge checkin, but if we just do it and let people adjust the pain will
end soon enough. Rebasing should be relatively straighforward for anyone
that sees conflicts; just be sure you use the new return style if
possible.
The following semantic patch was used to do the change, along with some
hand-massaging in order to preserve parenthesis where appropriate:
The semantic match that finds this problem is as follows, although some
hand-massaging was done in order to keep parenthesis where appropriate:
(http://coccinelle.lip6.fr/)
// <smpl>
@@
expression a;
@@
- return(a);
+ return a;
// </smpl>
A macros_file was also provided with the following content:
Additional steps taken, mainly for ASSERT() macros:
$ sed -i -e 's#return(NULL)#return NULL#' lib/libalpm/*.c
$ sed -i -e 's#return(-1)#return -1#' lib/libalpm/*.c
Signed-off-by: Dan McGee <dan@archlinux.org>
We erroniously dropped the call to _alpm_delta_parse() when macro-izing,
causing segfaults for repos that provide deltas. Addresses FS#23314.
Signed-off-by: Dan McGee <dan@archlinux.org>
Partially addresses the "why doesn't CheckSpace work in a chroot" issue.
We can't make it work, but we can at least detect when it won't work by
checking for a partition for our given installation root. If we can't
determine the mountpoint for this, bail out with an error.
Signed-off-by: Dan McGee <dan@archlinux.org>
this is just some debuggery to allow pacman to operate with both fetch
and curl at the same time. use the PACMANDL variable to control which
library is used.
Signed-off-by: Dave Reisner <d@falconindy.com>
This is a feature complete re-implementation of the fetch based internal
downloader, with a few improvements:
* support for SSL
* gzip and deflate compression on HTTP connections
* reuses a single connection over the entire session for lower resource
usage.
Signed-off-by: Dave Reisner <d@falconindy.com>
Add PM_ERR_LIBCURL to error enum and handle case in error.c by returning
curl_easy_strerror() based on the error number carried by the gloabl alpm
handle.
Signed-off-by: Dave Reisner <d@falconindy.com>
no actual code changes here. change preprocessor logic to include
get_tempfile, get_destfile, signal handler enum, and the interrupt
handler logic when either HAVE_LIBCURL or HAVE_LIBFETCH are defined.
Signed-off-by: Dave Reisner <d@falconindy.com>
Do this in preparation for implementing similar curl based
functionality. We want the ability to test these side by side.
Signed-off-by: Dave Reisner <d@falconindy.com>
Adding the CURLcode is necessary in order to return an error string from
pm_error. Unlike libfetch, curl returns numerical error numbers and does
not maintain a staticly allocated string with the last error generated.
Adding the curl object itself to the handle is advantageous (and
encouraged by curl_easy_perform(3)) because the handle is reusable for
successive operations. This cuts back on overhead when downloading
multiple files in a single transaction.
Signed-off-by: Dave Reisner <d@falconindy.com>
repo-add can add a "files" entry into the sync db. Currently we
do nothing with this file, so explicitly skip it to prevent
unknown database file warnings.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Our keywords were all screwed up in this regard. Fix it so our
ngettext() shortcut calls are actually recognized and respected.
Signed-off-by: Dan McGee <dan@archlinux.org>
Fixes FS#23090, a rather serious problem where the user was completely
unable to read the local database. Even if entry->d_type is available,
the given filesystem providing it may not fill the contents, in which
case we should fall back to a stat() as we did before. In this case, the
filesystem was XFS but there may be others.
Signed-off-by: Dan McGee <dan@archlinux.org>
A lot of these were places that should have used the same message but
didn't, or were very easy to convert to using the same message and
letting some of the burden off of the translators.
Signed-off-by: Dan McGee <dan@archlinux.org>
Ensure we have a local DB version that is up to par with what we expect
before we go down any road that might modify it. This should prevent
stupid mistakes with the 3.5.X upgrade and people not running
pacman-db-upgrade after the transaction as they will need to.
Signed-off-by: Dan McGee <dan@archlinux.org>
We only call these from the transaction init and teardown, so move them
to that file, mark them static, and push more of the logic of handle
manipulation into these functions.
Signed-off-by: Dan McGee <dan@archlinux.org>
According to FOPEN(3), using fclose on an fdopen'd file stream also
closes the underlying file descriptor. This happened in _alpm_lckmk
(util.c), which meant that when alpm_trans_release closed it again, the
log file (which reused the original file descriptor) was closed instead.
Signed-off-by: Jonathan Conder <jonno.conder@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
There's no API functions exposed which allow manipulation of this type,
so remove it from public view. Also, rename the public and private
alpm_db_get_pkgcache symbol to alpm_db_get_pkgcache_has.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows error messages emitted by the frontend to be a bit more
descriptive and not have the annoying "well why didn't you tell me that
the first time" problem. If a package had multiple missing deps, we
would bail on the first one before rather than finish processing all
missing dependencies, and only print one error message. Instead,
continue through this entire set of missing deps and append all eventual
errors.
The added pactest tests this case, as the to be installed package has
two missing dependencies. However, pactest does not actually test or see
the difference in output from before and after, so it passes in both
cases, but it is clearly visible in the logs.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is old code that has since gone stale; we no longer ever add
anything to this list so no need to keep it around and check the
contents during extraction.
Signed-off-by: Dan McGee <dan@archlinux.org>
* Make conflict_isin() static; it is used nowhere else.
* Remove does_conflict(): it turns out to be replaceable by a single call to
_alpm_depcmp(). By pushing it up, we can reduce calls to _alpm_splitdep()
from 60,368 to 16,940 during one test -Su operation I ran.
Signed-off-by: Dan McGee <dan@archlinux.org>
* Use stat() and not lstat(); we don't care for the size of the symlink if
it is one, we want the size of the reference file.
* FS#22896, fix local database estimation on platforms that don't abide by
the nlink assumption for number of children.
* Fix a missing newline on an error message.
Signed-off-by: Dan McGee <dan@archlinux.org>
Change _alpm_graph_new() to use CALLOC to avoid explicit zeroing out of fields
in pmgraph_t.
Signed-off-by: Pang Yan Han <pangyanhan@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Turn it into an enum rather than a boolean, and use a bitmask like we do for
reading DB entries. The relevant flag is turned on in our two calculate
loops, and anything reading the used flag later can decided which flag (or
either) is relevant.
This will allow the read-only partition code to be triggered on a
remove-only operation, e.g. if /boot was read-only and one tried to remove
grub in a sync transaction. Of course, right now, we don't actually run the
diskspace check code in the '-R' codepath.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a bit of a stopgap solution for the problem, but an easier one than
revamping the file conflict checking code to support the same stuff. Using
some more gross autoconf magic, figure out which struct field we need to
look at to determine read-only status and store that on our mountpoint
struct. If we find out we needed this partition after calculating size
requirements, then toss an error.
Signed-off-by: Dan McGee <dan@archlinux.org>
In the getmntinfo() section, the local variable mnt doesn't exist; this
would have caused a compile error if I had tested the code on such a
platform. Unify both codepaths to just run strlen() on the already copied
mount path instead.
Signed-off-by: Dan McGee <dan@archlinux.org>
In packages, our description file contains:
key = value is here
type entries, and we passed "key " and " value is here" to our strtrim
function, causing us to always memmove the value portion to remove the
space. Since this is a throwaway buffer, do the advancing on our own before
trimming to save the need to shift memory around; "value is here" will now
be passed and strtrim will be responsible for trailing whitespace.
Signed-off-by: Dan McGee <dan@archlinux.org>
We did this in some but not all cases, assuming the 0 value coming out of
libarchive would not be a problem. However, this does not work for "fake"
filesystems such as rpc_pipefs, which reports a free block and total block
count of zero.
Fix this by not ever counting symlinks or directories, and adding a note
explaining that if we someday do count directories, their size needs to be
attributed to the proper place.
This patch also includes a few cleanups/performance tweaks- avoid calling
strlen() on the mountpoint directory string as much by storing this size in
our mountpoint struct, and push the snprintf() call up to the calculate
functions since we were already doing it here in the remove case.
Signed-off-by: Dan McGee <dan@archlinux.org>
The overlapping code in _alpm_pkghash_add() and _alpm_pkghash_add_sorted()
are now in a new static function pkghash_add_pkg(). This function has a
third flag parameter which determines whether the package should be added in
sorted order.
Signed-off-by: Pang Yan Han <pangyanhan@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
In sync_db_populate() and local_db_populate(), a NULL db->pkgcache is not
caught, allowing the functions to continue instead of exiting.
A later alpm_list_msort() call which uses alpm_list_nth() will thus traverse
invalid pointers in a non-existent db->pkgcache->list.
pm_errno is set to PM_ERR_MEMORY as _alpm_pkghash_create() will only return
NULL when we run out of memory / exceed max hash table size. The local/sync
db_populate() functions are also exited.
Signed-off-by: Pang Yan Han <pangyanhan@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
When reading the "desc" file in _alpm_local_db_read(), some
strings are trimmed and checked for length > 0 before their
use/duplication subsequently. They are then trimmed again
when there is no need to.
The following code snippet should illustrate it clearly:
while(fgets(line, sizeof(line), fp) &&
strlen(_alpm_strtrim(line))) {
char *linedup;
STRDUP(linedup, _alpm_strtrim(line), goto error);
info->groups = alpm_list_add(info->groups, linedup);
}
This patch removes the redundant _alpm_strtrim() calls in
_alpm_local_db_read() such as the one inside the STRDUP shown
above.
Signed-off-by: Pang Yan Han <pangyanhan@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
PM_ERR_WRITE is defined in alpm.h but not handled in
alpm_strerror(). This patch corrects that.
Signed-off-by: Pang Yan Han <pangyanhan@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We located files in a few places but didn't check if they were files or
directories. Ensure they are actually files using stat() and S_ISREG(); this
showed itself when trying to download to the directory name itself in
FS#22645.
Signed-off-by: Dan McGee <dan@archlinux.org>
Rather than potentially move every item to the next NULL, attempt to move at
most one item at a time by iterating backwards from the NULL location in the
hash array. If we move an item, we repeat the process on the now shorter
"chain" until no more items need moving.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
This takes in the list and a list item, and does the pointer dance necessary
to remove it from the list regardless of whether it is first, last, or
somewhere in the middle. It is useful for callers that already know what
item needs to be removed and have a pointer to it rather than doing a search
by data that the plain alpm_list_remove() does.
Refactor alpm_list_remove() to use this function as well.
Signed-off-by: Dan McGee <dan@archlinux.org>
Fully removes a package from the hash. Also unify prototype with
removal from an alpm_list_t, fixing issues when removing a package
from the pkgcache.
Signed-off-by: Allan McRae <allan@archlinux.org>
While probably still not optimal in terms of everyday usage in
pacman, this reduces the absolute size increase to "more reasonable"
levels. For databases greater than 5000 in size, the minimum size
increase is used which is still on the order of a 10% increase.
Signed-off-by: Allan McRae <allan@archlinux.org>
Check that the requested size of a pkghash is not beyond the maximum
prime. Also check for successful creation of a new hash before
rehashing.
Signed-off-by: Allan McRae <allan@archlinux.org>
Since the sync database never changes size once we initialize it, we
allow it to be filled a bit more. This reduces the overall memory
footprint needed by the hash table.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows us to get through the rehash required by smoke001 and pass
all pactests. It is by no means the best or most efficient
implementation but it does do the job.
Signed-off-by: Dan McGee <dan@archlinux.org>
Read the package information for sync/local databases into a pmpkghash_t
structure.
Provide a alpm_db_get_pkgcache_list() method that returns the list from
the hash object. Most usages of alpm_db_get_pkgcache are converted to
this at this stage for ease of implementation. Review whether these are
better accessing the hash table directly at a later stage.
Signed-off-by: Allan McRae <allan@archlinux.org>
This works for both local and sync databases in slightly different ways. For
the local database, we can use the directory hard link count on the local/
folder. For sync databases, we use the archive size coupled with some
computed average per-package sizes to determine an estimate.
This is currently a dead assignment once calculated, but could be used to
set the initial size of a hash table.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Noted in FS#22697. When I factored out _alpm_parsedate() into a common
function, I didn't move the <locale.h> include properly, causing a build
failure when NLS is disabled and this header isn't automatically included
everywhere.
Signed-off-by: Dan McGee <dan@archlinux.org>
This flag allows to disable version checking in dependency resolving
code.
depcmp_tolerant respects the NODEPVERSION flag but we still keep the
original strict depcmp. The idea is to reduce the impact of the
NODEPVERSION flag by using it in fewer places.
I replaced almost all depcmp calls by depcmp_tolerant in deps.c (except
in the public find_satisfier used by deptest / pacman -T), but I kept
depcmp in sync.c and conflict.c
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
It's likely that these interfaces will break sooner or later, now that
pacman no longer uses them.
So better force the two people who use them to migrate their code to the
new add_pkg/remove_pkg interface, which is very easy anyway.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Old interface is marked as deprecated:
int alpm_sync_target(char *target);
int alpm_sync_dbtarget(char *db, char *target);
int alpm_add_target(char *target);
int alpm_remove_target(char *target);
New recommended interface:
int alpm_add_pkg(pmpkg_t *pkg);
int alpm_remove_pkg(pmpkg_t *pkg);
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
For consistency with alpm_add_pkg.
The new recommended interface is alpm_add_pkg / alpm_remove_pkg, all
others interfaces are deprecated.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
This group function is meant to help group handling from frontend : it
scans all dbs, handling ignored packages and duplicate members (the
first repo where a member is found has the priority).
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
This new function is meant to deprecate all existing
sync/add target functions :
int alpm_sync_target(char *target);
int alpm_sync_dbtarget(char *db, char *target);
int alpm_add_target(char *target);
Rather than dropping these 3 interfaces, it might be better to rewrite
them using alpm_add_pkg for now.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
This is a public interface for resolvedep. It looks nicer to expose it
this way rather than through sync_target.
This function can also be helpful for external tools as it should give
good results close to how pacman select a package for satisfying a given
dep.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
If there are multiple providers in one db, pacman used to just stop at
the first one (both during dependency resolution or for pacman -S
'provision' which uses the same code).
This adds a new conversation callback so that the user can choose which
provider to install. By default (user press enter or --noconfirm), the
first provider is still chosen, so for example the behavior of sync402
and 403 is preserved. But at least the user now has the possibility to
make the right choice in a manual run.
If one of the provider is already installed, it is picked for
reinstall/upgrade, so that provision 002/003 pactest now pass.
$ pacman -S community/smtp-server
:: There are 3 providers available for smtp-server:
1) courier-mta 2) esmtp 3) exim
Which one do you want to install?
Enter a number (default=1):
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Perform the cheap struct and string setup of the local DB at handle
initialization time to match the teardown we do when releasing the handle.
If the local DB is not needed, all real initialization is done lazily after
DB paths and other things have been configured anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
We explicitly place 'pkgbase' (and used to place 'force') fields inside
PKGINFO files, so ignore them silently instead of printing an error for
them. Also make the error message for unknown keys actually contain the key.
Signed-off-by: Dan McGee <dan@archlinux.org>
We were returning a package error code rather than a DB one, and we
would leak the archive memory if the database file didn't exist.
Signed-off-by: Dan McGee <dan@archlinux.org>
Instead, go the same route we have always taken with version-release in
libalpm and treat it all as one piece of information. Makepkg is the only
script that knows about epoch as a distinct value; from there on out we will
parse out the components as necessary.
This makes the code a lot simpler as far as epoch handling goes. The
downside here is that we are tossing some compatibility to the wind;
packages using force will have to be rebuilt with an incremented epoch to
keep their special status.
Signed-off-by: Dan McGee <dan@archlinux.org>
Adapting from RPM, follow the [epoch:]version[-release] syntax. We can also
borrow some of their parsing code for our purposes (thanks!). Add some new
tests to our vercmp shell script tester for epoch comparisons, and then make
the code work with these newfangled epoch specifiers.
Signed-off-by: Dan McGee <dan@archlinux.org>
Since it is the same string. Done with some bash looping and sed magic.
for src in po/*.po; do
echo $src
newtrans=$(grep -A1 "msgid.*$1" $src | tail -n1)
newtrans=${newtrans//\\/\\\\}
echo "$newtrans"
fname=${src##*/}
dest=lib/libalpm/po/$fname
sed -i -e "/msgid.*$1/{N; s/msgstr.*$/$newtrans/}" $dest
done
Signed-off-by: Dan McGee <dan@archlinux.org>
In most (all?) cases, we will process all files for a given sync database
entry sequentially. The code currently does an _alpm_pkg_find() for every
file in the database, but we had the "current" package readily available.
Shift some local variables around a bit to expose this to sync_db_read() and
use it if the package is the correct one.
On my system, this cuts calls to _alpm_pkg_find() from 20,769 to 10,349
calls during a -Qu operation, and results in a ~30% speedup of the same
operation (0.35 sec -> 0.27 sec). This benefit should be apparent anywhere
we read in the full contents of the sync databases.
Signed-off-by: Dan McGee <dan@archlinux.org>
We don't need to create a temporary copy of the string if we are smart with
our pointer manipulation and string copying. This saves a bunch of string
duplication during database parsing, both local and sync.
Signed-off-by: Dan McGee <dan@archlinux.org>
Remove the need for an unconditional string duplication by using pointer
arithmetic instead, and strndup() instead of an unspecified-length strdup().
This should reduce memory churn a fair amount as this is called pretty
frequently during database loads.
Signed-off-by: Dan McGee <dan@archlinux.org>
When installing packages from a file, the integrity check count
stays at (0/x) complete. This ensures it is bumped to (x/x) at
the end of the process.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is essentially a backport/cherry-pick of commit 33240e87b9 from
master, but has to be done by hand because the DB format has diverged. Read
more in the commit message used there, which follows.
Due to the way we funk around with package data loading, we had a condition
where the filelist got doubled up because it was loaded twice.
Packages are originally loaded with INFRQ_BASE. In an upgrade/sync, the
package is checked for file conflicts next, leaving us in an "INFRQ_BASE |
INFRQ_FILES" state. Later, when committing a single package, we have an
explicit call to _alpm_local_db_read() with INFRQ_ALL as the level. Because
the package's level did not match this, we skipped over our previous "does
the incoming level match where I'm at" shortcut, and continued to load
things again, because of a lack of fine-grained checking for each of DESC,
FILES, and INSTALL.
The end result is we loaded the filelist twice, causing our remove logic to
iterate twice over the installed files, spewing a bunch of "cannot find file
X" messages.
Fix the problem by doing a bit more bitmasking logic throughout the load
method, and also fix the sanity check at the beginning of the function- this
should *only* be used for local packages as opposed to the "not a package"
check that was there before.
Signed-off-by: Dan McGee <dan@archlinux.org>
After all the debate as to what to do on maint, we are going to end up just
incorporating epoch into the version string, so we don't need this separate
field at all. Revert commit 5c8083baa4 and also kill the force flag we were
recording here as well.
Signed-off-by: Dan McGee <dan@archlinux.org>
Due to the way we funk around with package data loading, we had a condition
where the filelist got doubled up because it was loaded twice.
Packages are originally loaded with INFRQ_BASE. In an upgrade/sync, the
package is checked for file conflicts next, leaving us in an "INFRQ_BASE |
INFRQ_FILES" state. Later, when committing a single package, we have an
explicit call to _alpm_local_db_read() with INFRQ_ALL as the level. Because
the package's level did not match this, we skipped over our previous "does
the incoming level match where I'm at" shortcut, and continued to load
things again, because of a lack of fine-grained checking for each of DESC,
FILES, and INSTALL.
The end result is we loaded the filelist twice, causing our remove logic to
iterate twice over the installed files, spewing a bunch of "cannot find file
X" messages.
Fix the problem by doing a bit more bitmasking logic throughout the load
method, and also fix the sanity check at the beginning of the function- this
should *only* be used for local packages as opposed to the "not a package"
check that was there before.
A debug log message was added to upgraderemove as well to match the one
already in the normal remove codepath.
Signed-off-by: Dan McGee <dan@archlinux.org>
libarchive eventually calls it anyway, but backtraces make a lot more sense
if we call it, as well as matching our precedent from alpm_pkg_load().
Signed-off-by: Dan McGee <dan@archlinux.org>
This is helpful anyway to the user, and should also be helpful to us if we
see problems cropping up in the check during development.
Also add a missing ->used = 0 initialization in the code path less taken.
Signed-off-by: Dan McGee <dan@archlinux.org>
Although they won't be the same in the gettext catalog because of the '\n'
we should still use the same text.
Signed-off-by: Dan McGee <dan@archlinux.org>
This can take a while too, and it is really easy to add the necessary
callback stuff for adding a progressbar.
Signed-off-by: Dan McGee <dan@archlinux.org>
These were just two small things I came across today and found could be
fixed or helpful, so I've added them and I'm not sure what else to bundle
them with. commit_count++
Signed-off-by: Dan McGee <dan@archlinux.org>
We were checking if a package existed locally, but then using the
incoming package to calculate removed size rather than the currently
installed package.
Also adjust the local variable in the replaces loop to make it more
clear that we are always dealing with local packages here.
Signed-off-by: Dan McGee <dan@archlinux.org>
All of these can be done with integer division; the only slightly
interesting part is ensuring we round up like before with calling the
ceil() function.
We can also remove the math library from requirements; now that the only
ceil() calls are gone, we don't need this anymore.
Signed-off-by: Dan McGee <dan@archlinux.org>
file_pkg_ops can be a static struct like in other backends, we just need
to initialize it at some point.
Dan: add initialization flag.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
None of these warn at the normal "-Wall -Werror" level, but casts do occur
that we are fine with. Make them explicit to silence some warnings when
using "-Wconversion".
Signed-off-by: Dan McGee <dan@archlinux.org>
There is a lot of swtiching between size_t and int for alpm_list sizes
in the codebase. Start converting these to all be size_t by adjusting
the return type of alpm_list_count and fixing all additional warnings
given by -Wconversion that are generated by this change.
Dan: a few more small changes to ensure things compile, adjusting some
printf format string characters to accommodate the larger size on x86_64.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
All functions that are limited to the local translation unit are
declared static. This exposed that the _pkg_get_deltas declaration
in be_local.c was being satified by the function in packages.c which
when declared static caused linker failures.
Fixes all warnings with -Wmissing-{declarations,prototypes}.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We use PATH_MAX everywhere by including limits.h so there is no
point in doing a check for it in a different header when dealing
with FreeBSD's libfetch.
Also, remove autoconf check for strings.h header as it is not used
anywhere.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
POSIX does not require PATH_MAX be defined when there is not actual
limit to its value. This affects HURD based systems. Work around
this by defining PATH_MAX to 4096 (as on Linux) when this is not
defined.
Also, clean up inclusions of limits.h and remove autoconf check for
this header as we do not use macro shields for its inclusion anyway.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Just like we did for package name comparsions, if we add a depend name_hash
field on depend struct initialization, we can use it instead of doing a
string name comparison, saving us a lot of checks in the depcmp code.
Signed-off-by: Dan McGee <dan@archlinux.org>
Noticed when tweaking testdb, when we run _alpm_depcmp in loops and call it
seven million times, the strdup()/free() combo can add up. Remove the need
for any string duplication by some pointer manipulation and use of strncmp
instead of strcmp. Also kill the function logger and add an escape so we
don't needlessly retrieve the list of provides.
Signed-off-by: Dan McGee <dan@archlinux.org>
The old function was written in a time before we relied on it for nearly
every operation. Since then, we have switched to the archive backend and now
fast parsing is a big deal.
The former function made a per-character call to the libarchive
archive_read_data() function, which resulted in some 21 million calls in a
typical "load all sync dbs" operation. If we instead do some buffering of
our own and read the blocks directly, and then find our newlines from there,
we can cut out the multiple layers of overhead and go from archive to parsed
data much quicker.
Both users of the former function are switched over to the new signature,
made easier by the macros now in place in the sync backend parsing code.
Performance: for a `pacman -Su` (no upgrades available),
_alpm_archive_fgets() goes from being 29% of the total time to 12% The time
spent on the libarchive function being called dropped from 24% to 6%.
This pushes _alpm_pkg_find back to the title of slowest low-level function.
Signed-off-by: Dan McGee <dan@archlinux.org>
The amount of diskspace needed for a transaction can be less than
zero. Only test this against the available disk space if it is
positive, which avoids a comparison being made between signed and
unsigned types (-Wsign-compare).
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Always declare a function with (void) rather than () when we expect
no arguements. Fixes all warnings with -Wstrict-prototypes.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This simplifies a lot of the repetative code and makes it obvious where the
tricky or different ones are (e.g. depends, dates). It also makes it
significantly easier to change the way this code works in the future.
There should be no functional change with this patch.
Signed-off-by: Dan McGee <dan@archlinux.org>
Saves a few bytes due to padding (256 -> 248 bytes), especially on x86_64,
so we get the overhead of our new hash field right back.
Signed-off-by: Dan McGee <dan@archlinux.org>
This results in huge gains to a lot of our codepaths since this is the most
frequent method of random access to packages in a list. The gains are seen
in both profiling and real life.
$ pacman -Sii zvbi
real: 0.41 sec -> 0.32 sec
strcmp: 16,669,760 calls -> 473,942 calls
_alpm_pkg_find: 52.73% -> 26.31% of time
$ pacman -Su (no upgrades found)
real: 0.40 sec -> 0.50 sec
strcmp: 19,497,226 calls -> 524,097 calls
_alpm_pkg_find: 52.36% -> 26.15% of time
There is some minor risk with this patch, but most of it should be avoided
by falling back to strcmp() if we encounter a package with a '0' hash value
(which we should not via any existing code path). We also do a strcmp once
hash values match to ensure against hash collisions. The risk left is that a
package name is modified once it was originally set, but the hash value is
left alone. That would probably result in a lot of other problems anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is prepping for the addition of a hash field to each package to greatly
speed up the string comparisons we frequently do on package name in
_alpm_pkg_find.
Signed-off-by: Dan McGee <dan@archlinux.org>
Rather than error out, this is easy enough. Looks quite similar to the code
in be_local for creating the local directory.
Signed-off-by: Dan McGee <dan@archlinux.org>
Whenever depends is needed from the local db, so is desc. The only
disadvantage to merging them is the additional time taken to read the
depends entries when they are not needed. As depends is in general
relatively small, the additional time taken to read it in will be
negligable. Also, merging these files will speed up local database
access due to less file seeks.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This should hopefully address some of the concerns raised in FS#11639 with
regards to continuing after filling the disk up.
Add some more checks and passing of error conditions between our functions.
When a libarchive warning is encountered, check if it is due to lack of disk
space and if so upgrade it to an error condition. A review of other
libarchive warnings suggests that these are less critical and can remain as
informative warning messages at this stage.
Note the presence of errors after extraction of an entire package is complete.
If so, we abort the transaction to be on the safe side and keep damage to a
minimum.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: make ENOSPC warning into an error]
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Rather than hiding these warnings, show them to the user as they happen.
This will prevent things such as hiding full filesystem errors (ENOSPC) from
the user as seen in FS#11639.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: adjust warning wording and add gettext calls]
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Whether it be "desc", "depends", or "deltas", it really doesn't matter-
treat them all the same and have the ability to read any data from any file
in that list. This continues the work in a44c7b8956.
Signed-off-by: Dan McGee <dan@archlinux.org>
* Use our normal return() function syntax
* Rework a few things to reduce number of casts
* Fix void function argument declaration
* Add missing gettext _() call
* Remove need for seperate malloc() of statvfs/statfs structure
* Unify argument order of static functions- mountpoints now always
passed first
* Count all files that start with '.' in a package against the DB
* Rename db to db_local in check_diskspace to clarify some code
* Fix some line wrapping to respect 80 characters
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Turn it into a configure-type typedef, which allows us to reduce the
amount of duplicated code and clean up some #ifdef magic in the code
itself. Adjust some of the other defined checks to look at the headers
available rather than trying to pull in the right ones based on
configure checks.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Checking disk space needed for a transaction can take a while so add
an informative progress bar.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Disk space checking is likely to be an unnecessary bottleneck to
people with reasonable partition sizes so add a configuration option
to allow it to be disabled/enabled as wanted.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Pull together the work of the previous commits to implement a check
for enough free space before performing an install transaction. Abort
if there is not enough free space with an appropriate pm_errno..
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Two helper function are added to calculate the disk usage from packages
that are either currently installed on the system or from a package
archive.
Some minor approximations have been made:
1. Size for directories is not considered when removing a package from the
filesystem to avoid multiple counting across packages. Also, these are
reported to take zero size while installing.
2. Symlinks are reported to contribute zero size towards removal as
libarchive reports them to have zero size for install.
3. Package data files (.PKGINFO, .INSTALL, .CHANGELOG) are counted towards
usage on dbpath on install, but their size is not counted on package
removal.
4. No handling of extra size needed for .pacsave/.pacnew files.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
For a given file, determine which mount point it is on or will be
installed to. Take into account that we might be using an alternative
installation root.
Add additional helper function added to sort mount point list for easier
matching.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Add a mount_point_list() function that attempts to portably obtain
a list of system mount points and a struct to hold needed mount point
information.
Abort the transaction if we are unable to determine the mount points.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Very basic prototyping for adding functionality to check free disk
space before performing package installs.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This macro is deemed unnecessary by even the autoconf guys, so we really
don't need to use it.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We were including the header in a lot of places it is no longer used.
Additionally, use the correct autoconf macro for determining whether
d_type is available as a member: HAVE_STRUCT_DIRENT_D_TYPE.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a delta specific definition so it makes more sense to put
it in the delta specific header file.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This will allow us to eventually combine the depends and desc entries
within the sync database.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
When a -Sk or -Uk operation induced a removal of an existing local
package, --dbonly was not in effect and the files were all removed.
Fixing this behavior was already marked as TODO in database012 pactest
------------
TODO: I honestly think the above should NOT delete the original les, it
hould upgrade the DB entry without touching anything on the file stem.
E.g. this test should be the same as:
pacman -R --dbonly dummy && pacman -U --dbonly dummy.pkg.tar.gz
------------
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
[Dan: small coding style touchup]
Signed-off-by: Dan McGee <dan@archlinux.org>
Either we expose all low levels function dealing with pmdepend_t
(splitdep and depfree come to mind), or we don't.
Since none of the tools use depcmp, I chose to remove it. In the future,
we might want to expose higher level functions such as
alpm_find_satisfier, or just lower level functions like splitdep and
depfree together with depcmp.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This has been replaced by the more flexible alpm_find_satisfier
function, and alpm_deptest was completely unsused now.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
whatprovides and splitdep were removed, so depcmp alone is quite useless
now without splitdep, and deptest is not flexible enough.
Introduce a new alpm_find_satisfier which is hopefully more flexible,
this should make implementation of deptest very easy, and also help alpm
tools such as pactree.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch is only meant for 3.4.x. It prepares the place for the future
epoch-aware release.
All force packages that get reinstalled or upgraded will get an EPOCH
entry in the local database, and thus the new pacman with epoch won't
reinstall them by mistake on the first -Su.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
If we have a corrupted database, a package can come through without an arch,
causing the code to blow up when making strcmp() calls. It might even be
possible with perfectly valid database entries lacking an 'arch =' line.
This behavior was seen as at least one of the problems in FS#21668.
Ensure pkgarch is not null before doing anything further.
Signed-off-by: Dan McGee <dan@archlinux.org>
$ pacman -Rd kde-meta
Remove (15): kde-meta-kdewebdev-4.5-1 [0.00 MB] kde-meta-kdeutils-4.5-1 [0.00 MB]
kde-meta-kdetoys-4.5-1 [0.00 MB] kde-meta-kdesdk-4.5-1 [0.00 MB]
kde-meta-kdeplasma-addons-4.5-1 [0.00 MB] kde-meta-kdepim-4.5-1 [0.00 MB]
kde-meta-kdenetwork-4.5-1 [0.00 MB] kde-meta-kdemultimedia-4.5-1 [0.00 MB]
kde-meta-kdegraphics-4.5-1 [0.00 MB] kde-meta-kdegames-4.5-1 [0.00 MB]
kde-meta-kdeedu-4.5-1 [0.00 MB] kde-meta-kdebase-4.5-1 [0.00 MB]
kde-meta-kdeartwork-4.5-1 [0.00 MB] kde-meta-kdeadmin-4.5-1 [0.00 MB]
kde-meta-kdeaccessibility-4.5-1 [0.00 MB]
Total Removed Size: 0.06 MB
Do you want to remove these packages? [Y/n]
( 1/15) removing kde-meta-kdewebdev [------------------------] 100%
$ it stopped here..
On one side, libalpm did not initialize the progress bar at 0 percent.
So with meta-packages that have 0 files, there was only one progress bar
call with percent == 100.
On the other side, pacman callback kept track of the last percent that
it received. When there are only meta-packages, we always received only
100, so pacman believed the progress bar needed not update. Thus only
the first package was actually displayed.
A proper fix for the callback would be to keep track of last package
name to make sure the recorded prev percent applies.
But since we now specify that both Add and Remove should at least send
percent=0 at beginning and percent=100 at the end, there is no need
for that.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We still need to read force entry in epoch-aware pacman, so that when we
install an old force package, EPOCH gets written to the local db.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This will allow for better control of what was previously the 'force' option
in a PKGBUILD and transferred into the built package.
Signed-off-by: Dan McGee <dan@archlinux.org>
Remove unnecessary parsing of fields not found in local desc files.
Leave %FORCE% parsing as this likely will make an appearance in desc
files in the future.
Signed-off-by: Allan McRae <allan@archlinux.org>
The splitname function is a general utility function and so is better
suited to util.h. Rename it to _alpm_splitname to indicate it is an
internal libalpm function as was the case prior to splitting local and
sync db handling.
Signed-off-by: Allan McRae <allan@archlinux.org>
These functions are only needed by be_local and were only promoted
to db.{h,c} as part of the splitting of handling the local and sync
dbs. Move them into be_local.c and make them static again.
Signed-off-by: Allan McRae <allan@archlinux.org>
Read in list of packages for sync db from tar archive.
Breaks reading in _alpm_sync_db_read and a lot of pactests (which
is expected as they do not handle sync db in archives...).
Signed-off-by: Allan McRae <allan@archlinux.org>
Put the db_operations struct to use and completely split the handling
of the sync and local databases.
Signed-off-by: Allan McRae <allan@archlinux.org>
The file be_files.c is "split" to be_local.c and be_sync.c in order
to achieve separate handling of sync and local databases.
Some basic clean-up of functions that are only of use for local or
sync databases has been performed and some rough function renaming
in duplicated code has been performed to prevent compilation errors.
However, most of the clean-up and final separation of sync and local
db handling occurs in following patches.
Signed-off-by: Allan McRae <allan@archlinux.org>
These will be needed for the handling of both local and sync database
caches, so put them in a common location.
Signed-off-by: Allan McRae <allan@archlinux.org>
Move splitname, checkdbdir, get_pkgpath into db.{h,c} as these will be
needed to parse both the local and sync databases during the initial
splitting. They will be moved out of db.{h,c} at to more appropriate
locations at a later stage.
Signed-off-by: Allan McRae <allan@archlinux.org>
It doesn't do a whole lot yet, but these type of operations will
potentially be different for the DBs we load.
Signed-off-by: Dan McGee <dan@archlinux.org>
Cache bullshit only has relevance to be_files, so move it there.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: BIG rebase]
Signed-off-by: Allan McRae <allan@archlinux.org>
Hopefully we've finally arrived at package handling nirvana, or at least
this commit will get us a heck of a lot closer. The former method of getting
the depends list for a package was the following:
1. call alpm_pkg_get_depends()
2. this method would check if the package came from the cache
3. if so, ensure our cache level is correct, otherwise call db_load
4. finally return the depends list
Why did this suck? Because getting the depends list from the package
shouldn't care about whether the package was loaded from a file, from the
'package cache', or some other system which we can't even use because the
damn thing is so complicated. It should just return the depends list.
So what does this commit change? It adds a pointer to a struct of function
pointers to every package for all of these 'package operations' as I've
decided to call them (I know, sounds completely straightforward, right?). So
now when we call an alpm_pkg_get-* function, we don't do any of the cache
logic or anything else there- we let the actual backend handle it by
delegating all work to the method at pkg->ops->get_depends.
Now that be_package has achieved equal status with be_files, we can treat
packages from these completely different load points differently. We know a
package loaded from a zip file will have all of its fields populated, so
we can set up all its accessor functions to be direct accessors. On the
other hand, the packages loaded from the local and sync DBs are not always
fully-loaded, so their accessor functions are routed through the same logic
as before.
Net result? More code. However, this code now make it roughly 52 times
easier to open the door to something like a read-only tar.gz database
backend.
Are you still reading? I'm impressed. Looking at the patch will probably be
clearer than this long-winded explanation.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: rebase and adjust]
Signed-off-by: Allan McRae <allan@archlinux.org>
