_alpm_pkg_load_internal() was becoming a monster. Extract the top bit of
the method that dealt with checksum and signature validation into a
separate method that should be called before one loads a package to
ensure it is valid.
Signed-off-by: Dan McGee <dan@archlinux.org>
This shouldn't really be declared with const, and causes a compile error
when -Wcast-qual is used. Remove the const specifier from the function
specification and all implementations.
Also fix one other trivial -Wcast-qual warning in _alpm_db_cmp().
Signed-off-by: Dan McGee <dan@archlinux.org>
Neither deltas nor filename attributes are ever present in the local
database, so we can remove all of the indirection for accessing these
attributes.
Signed-off-by: Dan McGee <dan@archlinux.org>
We currently have csize, isize, and size concepts, and sometimes the
difference isn't clear. Ensure the following holds:
* size (aka csize): always the compressed size of the package; available
for everything except local packages (where it will return 0)
* isize: always the installed size of the package; available for all
three package types
Signed-off-by: Dan McGee <dan@archlinux.org>
libalpm now exports type alpm_pkgfrom_t in alpm.h, which may be used
by frontends.
Pacman now uses alpm_pkgfrom_t instead of replicating that type (pkg_from
as was in src/pacman/package.h)
Updated API change in README.
Signed-off-by: Diogo Sousa <diogogsousa@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Just pass the entire sync package in if we have it; that way we can do
any necessary operations involving it rather than have a parameter list
growing endlessly.
Signed-off-by: Dan McGee <dan@archlinux.org>
This changes the signature of _alpm_pkg_dup() to return an integer error
code and provide the new package in a passed pointer argument. All
callers are now more robust with checking the return value of this
function to ensure a fatal error did not occur.
We allow load failures to proceed as otherwise we have a chicken and egg
problem- if a 'desc' local database entry is missing, the best way of
restoring said file is `pacman -Sf --dbonly packagename`. This patch
fixes a segfault that was occurring in this case.
Fixes the segfault reported in FS#25667.
Signed-off-by: Dan McGee <dan@archlinux.org>
These items are never present in anything but sync databases, nor do we
even try to load them from the local database. Remvoe the indirection
meant to allow the caching layer to work since it will never do anything
anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
This adds a field in the package struct for this checksum type as well
as allowing access via the API to it. The frontend is now able to
display any read value. Note that this does not implement any use or
verification of the value internally.
Signed-off-by: Dan McGee <dan@archlinux.org>
This accomplishes quite a few things with one rather invasive change.
1. Iteration is much more performant, due to a reduction in pointer
chasing and linear item access.
2. Data structures are smaller- we no longer have the overhead of the
linked list as the file struts are now laid out consecutively in
memory.
3. Memory allocation has been massively reworked. Before, we would
allocate three different pieces of memory per file item- the list
struct, the file struct, and the copied filename. What this resulted
in was massive fragmentation of memory when loading filelists since
the memory allocator had to leave holes all over the place. The new
situation here now removes the need for any list item allocation;
allocates the file structs in contiguous memory (and reallocs as
necessary), leaving only the strings as individually allocated. Tests
using valgrind (massif) show some pretty significant memory
reductions on the worst case `pacman -Ql > /dev/null` (366387 files
on my machine):
Before:
Peak heap: 54,416,024 B
Useful heap: 36,840,692 B
Extra heap: 17,575,332 B
After:
Peak heap: 38,004,352 B
Useful heap: 28,101,347 B
Extra heap: 9,903,005 B
Several small helper methods have been introduced, including a list to
array conversion helper as well as a filelist merge sort that works
directly on arrays.
Signed-off-by: Dan McGee <dan@archlinux.org>
This gives us more granularity than the former Never/Optional/Always
trifecta. The frontend still uses these values temporarily but that will
be changed in a future patch.
* Use 'siglevel' consistenly in method names, 'level' as variable name
* The level becomes an enum bitmask value for flexibility
* Signature check methods now return a array of status codes rather than
a simple integer success/failure value. This allows callers to
determine whether things such as an unknown signature are valid.
* Specific signature error codes mostly disappear in favor of the above
returned status code; pm_errno is now set only to PKG_INVALID_SIG or
DB_INVALID_SIG as appropriate.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows us to capture size and mode data when building filelists
from package files. Future patches will take advantage of this newly
available information, and frontends can use it as well.
Signed-off-by: Dan McGee <dan@archlinux.org>
There is little need to expose the guts of this function even within the
library. Make it static in be_local.c, and clean up a few other things
since we know exactly where it is being called from:
* Remove unnecessary origin checks in _cache_get_*() methods- if you are
calling a cache method your package type will be correct.
* Remove sanity checks within local_db_read() itself- packages will
always have a name and version if they get this far, and the package
object will never be NULL either.
The one case calling this from outside the backend was in add.c, where
we forced a full load of a package before we duplicated it. Move this
concern elsewhere and have pkg_dup() always force a full package load
via a new force_load() function on the operations callback struct.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows us to separate the name and hash elements in one place and
not scatter different parsing code all over the place, including both
the frontend and backend.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows callers to retrieve it from wherever is convenient, which
may or may not be on the package object itself.
Signed-off-by: Dan McGee <dan@archlinux.org>
Similar to what we just did for the database; this will make it easy to
always know what handle a given package originated from.
Signed-off-by: Dan McGee <dan@archlinux.org>
Both md5sum verification and PGP verification can and should be done at
package load time. This allows verification to happen as early as
possible for packages provided by filename and loaded in the frontend,
and moves more stuff out of sync_commit that doesn't really belong
there. This should also set the stage for simplified parallel loading of
packages later down the road.
Signed-off-by: Dan McGee <dan@archlinux.org>
Given that we offer no transparency into the pmpgpsig_t type, we don't
really need to expose it outside of the library, and at this point, we
don't need it at all. Don't decode anything except when checking
signatures. For packages/files not from a sync database, we now just
read the signature file directly anyway.
Also push the decoding logic down further into the check method so we
don't need this hanging out in a less than ideal place. This will make
it easier to conditionally compile things down the road.
Signed-off-by: Dan McGee <dan@archlinux.org>
This makes it absolutely dead easy to ensure off_t has the same length
in all compilation units. I just spent 2.5 hours bashing my head on an
issue related to this so damn it I'm fixing it for good.
Signed-off-by: Dan McGee <dan@archlinux.org>
For a package to be loaded from any of our backends, these two fields
are always required upfront. Due to this fact, we don't need them to be
backend-specific operations and can just refer to the field directly.
Additionally, our static (and thus private) cache package accessors had
a NULL check on pkg before returning the relevant field. Eliminate this
since they only way they are ever called is via the packages attached
callback struct, which would have caused the NULL pointer dereference in
the first place.
Signed-off-by: Dan McGee <dan@archlinux.org>
Add a pmpgpsig_t struct to the database entry struct and functions for
the lazy loading of database signatures. Add a function for checking
database signatures, reusing (and generalizing) the code currently used
for checking package signatures.
TODO: The code for reading in signature files from the filesystem is
duplicated for local packages and database and needs refactoring.
Signed-off-by: Allan McRae <allan@archlinux.org>
Add a new field to the package struct to hold PGP information and
instruct db_read to pick it up from the database. It is currently unused
internally but this is the first step.
Due to the fact that we store the PGP sig as binary data, we need to store
both the data and the length so we have a small utility struct to assist us.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Instead, go the same route we have always taken with version-release in
libalpm and treat it all as one piece of information. Makepkg is the only
script that knows about epoch as a distinct value; from there on out we will
parse out the components as necessary.
This makes the code a lot simpler as far as epoch handling goes. The
downside here is that we are tossing some compatibility to the wind;
packages using force will have to be rebuilt with an incremented epoch to
keep their special status.
Signed-off-by: Dan McGee <dan@archlinux.org>
Saves a few bytes due to padding (256 -> 248 bytes), especially on x86_64,
so we get the overhead of our new hash field right back.
Signed-off-by: Dan McGee <dan@archlinux.org>
This will allow for better control of what was previously the 'force' option
in a PKGBUILD and transferred into the built package.
Signed-off-by: Dan McGee <dan@archlinux.org>
Hopefully we've finally arrived at package handling nirvana, or at least
this commit will get us a heck of a lot closer. The former method of getting
the depends list for a package was the following:
1. call alpm_pkg_get_depends()
2. this method would check if the package came from the cache
3. if so, ensure our cache level is correct, otherwise call db_load
4. finally return the depends list
Why did this suck? Because getting the depends list from the package
shouldn't care about whether the package was loaded from a file, from the
'package cache', or some other system which we can't even use because the
damn thing is so complicated. It should just return the depends list.
So what does this commit change? It adds a pointer to a struct of function
pointers to every package for all of these 'package operations' as I've
decided to call them (I know, sounds completely straightforward, right?). So
now when we call an alpm_pkg_get-* function, we don't do any of the cache
logic or anything else there- we let the actual backend handle it by
delegating all work to the method at pkg->ops->get_depends.
Now that be_package has achieved equal status with be_files, we can treat
packages from these completely different load points differently. We know a
package loaded from a zip file will have all of its fields populated, so
we can set up all its accessor functions to be direct accessors. On the
other hand, the packages loaded from the local and sync DBs are not always
fully-loaded, so their accessor functions are routed through the same logic
as before.
Net result? More code. However, this code now make it roughly 52 times
easier to open the door to something like a read-only tar.gz database
backend.
Are you still reading? I'm impressed. Looking at the patch will probably be
clearer than this long-winded explanation.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: rebase and adjust]
Signed-off-by: Allan McRae <allan@archlinux.org>
Implement this seemingly simple change in package.h:
typedef enum _pmpkgfrom_t {
- PKG_FROM_CACHE = 1,
- PKG_FROM_FILE
+ PKG_FROM_FILE = 1,
+ PKG_FROM_LOCALDB,
+ PKG_FROM_SYNCDB
} pmpkgfrom_t;
which requires flushing out several assumptions from around the codebase
with regards to usage of the PKG_FROM_CACHE value. Make some changes where
required to allow the switch, and now the correct value should be set (via a
crude hack) depending on whether a package was loaded as an entry in a local
db or a sync db.
This patch underwent some big rebasing from Allan and Dan.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
The combination of tabs and spaces is annoying in any editor that
does not use a tab width of 2 spaces.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
After our recent screwup with size_t and ssize_t in the download code, I
found the `-Wsign-conversion` flag to GCC to see if we were doing anything
else boneheaded. I didn't find anything quite as bad, but we did have some
goofups- most of our public unsigned methods would return -1 on error, which
is a bit odd in an unsigned context.
Signed-off-by: Dan McGee <dan@archlinux.org>
The main purpose of this function to make our code more readable.
It frees transaction specific fields of pmpkg_t. (It is used when a package
is removed from the target list.)
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
pmsyncpkg_t data sructure was removed:
1. pmpkg_t.reason is used instead of pmsyncpkg_t.newreason. (The target
packages come from sync repos, so we can use this field without any
problems. Upgrade transaction also uses this field to store this info.)
2. pmsyncpkg_t.removes was moved to pmpkg_t.removes.
This step requires careful programming, because we don't duplicate packages
when we add them to trans->packages. So we modify sync pkgcache when we
add this transaction-only info to our package. Hence it is important to
free this list when we remove any package from the target list
(remove_unresolvable, remove_conflicts, trans_free), otherwise this could
confuse the new sync transactions (with non-pacman GUI).
Overall, our code became ~100 line shorter, and we can call our helper
functions directly on trans->packages in sync.c, we don't need to maintain
parallel package lists.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
We have been using unsigned long as a file size type for a while, which
works but isn't quite correct and could easily break. Worse was probably our
use of int in the download callback functions, which could be restrictive
for packages > 2GB in size.
Switch all file size variables to use off_t, which is the preferred type for
file sizes. Note that at least on Linux, all applications compiled against
libalpm must now be sure to use large file support, where _FILE_OFFSET_BITS
is defined to be 64 or there will be some weird issues that crop up.
Signed-off-by: Dan McGee <dan@archlinux.org>
Actually, just rename _alpm_versioncmp to alpm_pkg_vercmp and get rid of the
need for a wrapper since it did nothing anyway.
Signed-off-by: Dan McGee <dan@archlinux.org>
Any real call of this function doesn't specify a name or version ahead of
time, so just kill that functionality off. Now to remove those dummy
packages...
Signed-off-by: Dan McGee <dan@archlinux.org>
* remove obsolete and unused *_cmp helper functions like deppkg_cmp and
_alpm_grp_cmp
* new alpm_list_remove_str function, used 6 times in handle.c
* remove _alpm_prov_cmp / _alpm_db_whatprovides and replace them by
a more general alpm_find_pkg_satisfiers with a cleaner implementation.
before: alpm_db_whatprovides(db, targ)
after: alpm_find_pkg_satisfiers(alpm_db_getpkgcache(db), targ)
* remove satisfycmp and replace alpm_list_find + satisfycmp usage by
_alpm_find_dep_satisfiers.
before : alpm_list_find(_alpm_db_get_pkgcache(db), dep, satisfycmp)
after : _alpm_find_dep_satisfiers(_alpm_db_get_pkgcache(db), dep)
* remove _alpm_pkgname_pkg_cmp, which was used with alpm_list_remove, and
use _alpm_pkg_find + alpm_list_remove with _alpm_pkg_cmp instead.
This commit actually get rids of all complicated and asymmetric _cmp
functions. I first thought these functions were worth it, be caused it
allowed us to reuse list_find and list_remove. But this was at the detriment
of the clarity and also the ease of use of these functions, dangerous
because of their asymmetricity.
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Now the syntax is coherent with alpm_list_find and alpm_sync_find.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Dan McGee