1
0
mirror of https://github.com/moparisthebest/pacman synced 2024-12-23 08:18:51 -05:00
Commit Graph

1512 Commits

Author SHA1 Message Date
Allan McRae
c9820ec97b Slightly more efficient rehash size selection
While probably still not optimal in terms of everyday usage in
pacman, this reduces the absolute size increase to "more reasonable"
levels.  For databases greater than 5000 in size, the minimum size
increase is used which is still on the order of a 10% increase.

Signed-off-by: Allan McRae <allan@archlinux.org>
2011-02-04 09:55:45 +10:00
Allan McRae
d843c86b7b Error handling for maximum database size
Check that the requested size of a pkghash is not beyond the maximum
prime.  Also check for successful creation of a new hash before
rehashing.

Signed-off-by: Allan McRae <allan@archlinux.org>
2011-02-04 09:55:45 +10:00
Dan McGee
021085624e Change default sync hash table sizing to 66% full
Since the sync database never changes size once we initialize it, we
allow it to be filled a bit more. This reduces the overall memory
footprint needed by the hash table.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-04 09:55:45 +10:00
Dan McGee
ce54715112 Implement a quick and dirty rehash function
This allows us to get through the rehash required by smoke001 and pass
all pactests. It is by no means the best or most efficient
implementation but it does do the job.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-04 09:55:45 +10:00
Allan McRae
f8fdce6cb0 Read pkgcache into hash
Read the package information for sync/local databases into a pmpkghash_t
structure.

Provide a alpm_db_get_pkgcache_list() method that returns the list from
the hash object.  Most usages of alpm_db_get_pkgcache are converted to
this at this stage for ease of implementation.  Review whether these are
better accessing the hash table directly at a later stage.

Signed-off-by: Allan McRae <allan@archlinux.org>
2011-02-04 09:55:45 +10:00
Dan McGee
5dae577a87 Get estimated package count when populating databases
This works for both local and sync databases in slightly different ways. For
the local database, we can use the directory hard link count on the local/
folder. For sync databases, we use the archive size coupled with some
computed average per-package sizes to determine an estimate.

This is currently a dead assignment once calculated, but could be used to
set the initial size of a hash table.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-02-04 09:55:45 +10:00
Allan McRae
e17b0446bd Add a hash table for holding packages
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-02-04 09:55:45 +10:00
Dan McGee
87240dae6d Fix locale.h/setlocale inclusion with --disable-nls
Noted in FS#22697. When I factored out _alpm_parsedate() into a common
function, I didn't move the <locale.h> include properly, causing a build
failure when NLS is disabled and this header isn't automatically included
everywhere.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-01 09:59:12 -06:00
Xavier Chantry
b6ec9019d7 alpm/depcmp: new NODEPVERSION flag
This flag allows to disable version checking in dependency resolving
code.

depcmp_tolerant respects the NODEPVERSION flag but we still keep the
original strict depcmp. The idea is to reduce the impact of the
NODEPVERSION flag by using it in fewer places.

I replaced almost all depcmp calls by depcmp_tolerant in deps.c (except
in the public find_satisfier used by deptest / pacman -T), but I kept
depcmp in sync.c and conflict.c

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-31 10:49:39 -06:00
Xavier Chantry
ed1aef7bc5 libalpm: fix db_update documentation
return codes were mixed up

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-31 09:36:30 -06:00
Xavier Chantry
e263cf7231 alpm: drop old target interfaces
It's likely that these interfaces will break sooner or later, now that
pacman no longer uses them.

So better force the two people who use them to migrate their code to the
new add_pkg/remove_pkg interface, which is very easy anyway.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
2011-01-29 19:40:08 +01:00
Xavier Chantry
2a90dbe3a8 alpm: deprecate old interface
Old interface is marked as deprecated:
int alpm_sync_target(char *target);
int alpm_sync_dbtarget(char *db, char *target);
int alpm_add_target(char *target);
int alpm_remove_target(char *target);

New recommended interface:
int alpm_add_pkg(pmpkg_t *pkg);
int alpm_remove_pkg(pmpkg_t *pkg);

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
2011-01-29 19:39:06 +01:00
Xavier Chantry
953e0d48d7 alpm: new alpm_remove_pkg interface
For consistency with alpm_add_pkg.

The new recommended interface is alpm_add_pkg / alpm_remove_pkg, all
others interfaces are deprecated.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
2011-01-29 19:33:16 +01:00
Xavier Chantry
1767a569c6 alpm: add alpm_find_grp_pkgs
This group function is meant to help group handling from frontend : it
scans all dbs, handling ignored packages and duplicate members (the
first repo where a member is found has the priority).

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
2011-01-29 19:33:16 +01:00
Xavier Chantry
fb7e1b4b9b alpm: new alpm_add_pkg interface
This new function is meant to deprecate all existing
sync/add target functions :
int alpm_sync_target(char *target);
int alpm_sync_dbtarget(char *db, char *target);
int alpm_add_target(char *target);

Rather than dropping these 3 interfaces, it might be better to rewrite
them using alpm_add_pkg for now.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
2011-01-29 19:33:15 +01:00
Xavier Chantry
b8590ed634 alpm/dep: add alpm_find_dbs_satisfier
This is a public interface for resolvedep. It looks nicer to expose it
this way rather than through sync_target.

This function can also be helpful for external tools as it should give
good results close to how pacman select a package for satisfying a given
dep.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
2011-01-29 19:33:15 +01:00
Xavier Chantry
4097c98c1e Add interactive provider selection
If there are multiple providers in one db, pacman used to just stop at
the first one (both during dependency resolution or for pacman -S
'provision' which uses the same code).

This adds a new conversation callback so that the user can choose which
provider to install. By default (user press enter or --noconfirm), the
first provider is still chosen, so for example the behavior of sync402
and 403 is preserved. But at least the user now has the possibility to
make the right choice in a manual run.

If one of the provider is already installed, it is picked for
reinstall/upgrade, so that provision 002/003 pactest now pass.

$ pacman -S community/smtp-server
:: There are 3 providers available for smtp-server:
   1) courier-mta  2) esmtp  3) exim

Which one do you want to install?
Enter a number (default=1):

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
2011-01-29 19:33:15 +01:00
Dan McGee
ba97a22ce1 Merge branch 'maint' 2011-01-29 12:16:11 -06:00
Dan McGee
ef86da97f5 Remove need to explicitly register the local DB
Perform the cheap struct and string setup of the local DB at handle
initialization time to match the teardown we do when releasing the handle.
If the local DB is not needed, all real initialization is done lazily after
DB paths and other things have been configured anyway.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-29 12:13:56 -06:00
Dan McGee
9b876fff09 Ignore known but unused package descfile fields
We explicitly place 'pkgbase' (and used to place 'force') fields inside
PKGINFO files, so ignore them silently instead of printing an error for
them. Also make the error message for unknown keys actually contain the key.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-28 17:34:06 -06:00
Thomas Bächler
272f7cf25e libalpm: Fix a missing "nicht" (not) in German translation.
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-28 16:41:21 -06:00
Dan McGee
fe76c353af Fix memory leak and error code in DB reading
We were returning a package error code rather than a DB one, and we
would leak the archive memory if the database file didn't exist.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-24 21:46:42 -06:00
Dan McGee
36ea02cc48 Compute download size for sync packages only
Neither packages from files nor packages from the local database will ever
have a download size.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-22 11:11:29 -06:00
Dan McGee
c4332c8091 Merge branch 'maint' 2011-01-22 10:12:46 -06:00
Dan McGee
bf46e04614 Remove epoch as an independent field
Instead, go the same route we have always taken with version-release in
libalpm and treat it all as one piece of information. Makepkg is the only
script that knows about epoch as a distinct value; from there on out we will
parse out the components as necessary.

This makes the code a lot simpler as far as epoch handling goes. The
downside here is that we are tossing some compatibility to the wind;
packages using force will have to be rebuilt with an incremented epoch to
keep their special status.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-21 19:30:45 -06:00
Dan McGee
5c46ba14f7 Allow version comparison to contain epoch specifier
Adapting from RPM, follow the [epoch:]version[-release] syntax. We can also
borrow some of their parsing code for our purposes (thanks!).  Add some new
tests to our vercmp shell script tester for epoch comparisons, and then make
the code work with these newfangled epoch specifiers.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-21 19:30:45 -06:00
Dan McGee
76735e6519 Copy new backend translation over from frontend translation
Since it is the same string. Done with some bash looping and sed magic.

    for src in po/*.po; do
        echo $src
        newtrans=$(grep -A1 "msgid.*$1" $src | tail -n1)
        newtrans=${newtrans//\\/\\\\}
        echo "$newtrans"
        fname=${src##*/}
        dest=lib/libalpm/po/$fname
        sed -i -e "/msgid.*$1/{N; s/msgstr.*$/$newtrans/}" $dest
    done

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-21 11:04:52 -06:00
Dan McGee
3863e48788 Add a likely_pkg hint argument to sync_db_read
In most (all?) cases, we will process all files for a given sync database
entry sequentially. The code currently does an _alpm_pkg_find() for every
file in the database, but we had the "current" package readily available.
Shift some local variables around a bit to expose this to sync_db_read() and
use it if the package is the correct one.

On my system, this cuts calls to _alpm_pkg_find() from 20,769 to 10,349
calls during a -Qu operation, and results in a ~30% speedup of the same
operation (0.35 sec -> 0.27 sec). This benefit should be apparent anywhere
we read in the full contents of the sync databases.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-21 10:11:59 -06:00
Dan McGee
c86ff120c8 Improve splitname memory allocation
We don't need to create a temporary copy of the string if we are smart with
our pointer manipulation and string copying. This saves a bunch of string
duplication during database parsing, both local and sync.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-21 09:14:47 -06:00
Dan McGee
01c8f39ab8 Improve depends string parsing
Remove the need for an unconditional string duplication by using pointer
arithmetic instead, and strndup() instead of an unspecified-length strdup().
This should reduce memory churn a fair amount as this is called pretty
frequently during database loads.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-21 09:14:47 -06:00
Dan McGee
aff3e63c45 Add strndup fallback function to libalpm util
The same fallback we are currently using in the pacman frontend.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-21 09:14:47 -06:00
Dan McGee
f65edb7f29 Fix integrity check status when installing from file
When installing packages from a file, the integrity check count
stays at (0/x) complete.  This ensures it is bumped to (x/x) at
the end of the process.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-18 13:19:25 -06:00
Dan McGee
30f53cfe8d Fix double read issue in maint releases
This is essentially a backport/cherry-pick of commit 33240e87b9 from
master, but has to be done by hand because the DB format has diverged. Read
more in the commit message used there, which follows.

Due to the way we funk around with package data loading, we had a condition
where the filelist got doubled up because it was loaded twice.

Packages are originally loaded with INFRQ_BASE. In an upgrade/sync, the
package is checked for file conflicts next, leaving us in an "INFRQ_BASE |
INFRQ_FILES" state. Later, when committing a single package, we have an
explicit call to _alpm_local_db_read() with INFRQ_ALL as the level. Because
the package's level did not match this, we skipped over our previous "does
the incoming level match where I'm at" shortcut, and continued to load
things again, because of a lack of fine-grained checking for each of DESC,
FILES, and INSTALL.

The end result is we loaded the filelist twice, causing our remove logic to
iterate twice over the installed files, spewing a bunch of "cannot find file
X" messages.

Fix the problem by doing a bit more bitmasking logic throughout the load
method, and also fix the sanity check at the beginning of the function- this
should *only* be used for local packages as opposed to the "not a package"
check that was there before.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-12 09:17:22 -06:00
Dan McGee
9e8af82c97 Back out anticipated epoch changes
After all the debate as to what to do on maint, we are going to end up just
incorporating epoch into the version string, so we don't need this separate
field at all. Revert commit 5c8083baa4 and also kill the force flag we were
recording here as well.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-11 23:15:44 -06:00
Dan McGee
dd26592155 Merge branch 'diskspace-fixes' 2011-01-11 21:20:12 -06:00
Dan McGee
57c5afd69c Merge branch 'fix-double-load' 2011-01-11 21:20:10 -06:00
Dan McGee
33240e87b9 Fix double filelist issue when upgrading a package
Due to the way we funk around with package data loading, we had a condition
where the filelist got doubled up because it was loaded twice.

Packages are originally loaded with INFRQ_BASE. In an upgrade/sync, the
package is checked for file conflicts next, leaving us in an "INFRQ_BASE |
INFRQ_FILES" state. Later, when committing a single package, we have an
explicit call to _alpm_local_db_read() with INFRQ_ALL as the level. Because
the package's level did not match this, we skipped over our previous "does
the incoming level match where I'm at" shortcut, and continued to load
things again, because of a lack of fine-grained checking for each of DESC,
FILES, and INSTALL.

The end result is we loaded the filelist twice, causing our remove logic to
iterate twice over the installed files, spewing a bunch of "cannot find file
X" messages.

Fix the problem by doing a bit more bitmasking logic throughout the load
method, and also fix the sanity check at the beginning of the function- this
should *only* be used for local packages as opposed to the "not a package"
check that was there before.

A debug log message was added to upgraderemove as well to match the one
already in the normal remove codepath.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-11 21:16:39 -06:00
Dan McGee
25fab402c7 Call archive_read_data_skip() while checking diskspace
libarchive eventually calls it anyway, but backtraces make a lot more sense
if we call it, as well as matching our precedent from alpm_pkg_load().

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-11 21:15:24 -06:00
Dan McGee
6942bba75d Add error message stating which partition is full
This is helpful anyway to the user, and should also be helpful to us if we
see problems cropping up in the check during development.

Also add a missing ->used = 0 initialization in the code path less taken.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-11 21:15:24 -06:00
Dan McGee
55bff19b76 Unify two free diskspace error messages
Although they won't be the same in the gettext catalog because of the '\n'
we should still use the same text.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-11 21:15:24 -06:00
Dan McGee
fe6e90c21f Add a progressbar for package integrity checking
This can take a while too, and it is really easy to add the necessary
callback stuff for adding a progressbar.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-11 21:15:10 -06:00
Dan McGee
9a82cb92a4 Small fix to download size lookup and a logger
These were just two small things I came across today and found could be
fixed or helpful, so I've added them and I'm not sure what else to bundle
them with. commit_count++

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-11 21:15:04 -06:00
Dan McGee
842cbc9ea4 Ensure we use local package when calculating removed size
We were checking if a package existed locally, but then using the
incoming package to calculate removed size rather than the currently
installed package.

Also adjust the local variable in the replaces loop to make it more
clear that we are always dealing with local packages here.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-10 10:50:03 -06:00
Dan McGee
d03b57f459 Remove need for floating point division in backend
All of these can be done with integer division; the only slightly
interesting part is ensuring we round up like before with calling the
ceil() function.

We can also remove the math library from requirements; now that the only
ceil() calls are gone, we don't need this anymore.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-10 10:49:55 -06:00
Xavier Chantry
281a4c0a4f libalpm/be_package.c: fix small memleak
file_pkg_ops can be a static struct like in other backends, we just need
to initialize it at some point.

Dan: add initialization flag.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-07 21:27:39 -06:00
Dan McGee
62f5da3779 Fix some more simple conversion "errors"
None of these warn at the normal "-Wall -Werror" level, but casts do occur
that we are fine with. Make them explicit to silence some warnings when
using "-Wconversion".

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-07 21:15:47 -06:00
Allan McRae
f966f3a834 Use size_t for alpm_list sizes
There is a lot of swtiching between size_t and int for alpm_list sizes
in the codebase.   Start converting these to all be size_t by adjusting
the return type of alpm_list_count and fixing all additional warnings
given by -Wconversion that are generated by this change.

Dan: a few more small changes to ensure things compile, adjusting some
printf format string characters to accommodate the larger size on x86_64.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-07 21:15:46 -06:00
Dan McGee
4bc6ed56aa Refactor old date parsing into single method
We've managed to duplicate this four times at this point, so make it a
method in util.c instead.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-07 20:55:05 -06:00
Allan McRae
d288240426 Update copyright years for 2011
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-07 18:47:37 -06:00
Dan McGee
26652768d6 Remove FORCE reading from local DB
We never wrote it here, so no need to read it in either.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-05 20:56:06 -06:00
Dan McGee
c41edf49be Fix function indentation
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-02 12:25:33 -06:00
Dan McGee
08d885fda5 Merge branch 'maint'
Conflicts:
	lib/libalpm/sync.c
	test/pacman/tests/ignore007.py
2010-12-30 09:41:46 -06:00
Allan McRae
351250adb4 Declare all local functions static
All functions that are limited to the local translation unit are
declared static.  This exposed that the _pkg_get_deltas declaration
in be_local.c was being satified by the function in packages.c which
when declared static caused linker failures.

Fixes all warnings with -Wmissing-{declarations,prototypes}.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-30 09:39:23 -06:00
Jakob Gruber
6ddc115c7f Respect Ignore{Pkg,Group} for group members
Fixes FS#19854.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-30 09:35:03 -06:00
Jakob Gruber
df360b791d Move group code to separate function
This makes the following commits more readable.
No logic was changed in this commit.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-30 09:35:03 -06:00
Jakob Gruber
cb7ba4e4e5 Add const to some ALPM function signatures
char * -> const char *.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-30 09:25:16 -06:00
Allan McRae
d227771464 Use limits.h for PATH_MAX
We use PATH_MAX everywhere by including limits.h so there is no
point in doing a check for it in a different header when dealing
with FreeBSD's libfetch.

Also, remove autoconf check for strings.h header as it is not used
anywhere.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-29 19:26:21 -06:00
Allan McRae
81dd9d3ebc Detect undefined PATH_MAX
POSIX does not require PATH_MAX be defined when there is not actual
limit to its value.  This affects HURD based systems.  Work around
this by defining PATH_MAX to 4096 (as on Linux) when this is not
defined.

Also, clean up inclusions of limits.h and remove autoconf check for
this header as we do not use macro shields for its inclusion anyway.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-29 19:24:13 -06:00
Allan McRae
fcc09bd7e3 Correct type for hash value storage
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-29 19:10:49 -06:00
Dan McGee
a7972625e3 Merge branch 'depcmp-perf' 2010-12-29 18:43:47 -06:00
Dan McGee
735a197fc2 Use name hashes in depends to avoid strcmp calls
Just like we did for package name comparsions, if we add a depend name_hash
field on depend struct initialization, we can use it instead of doing a
string name comparison, saving us a lot of checks in the depcmp code.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-21 17:25:34 -06:00
Dan McGee
34a78d935a Remove need for memory allocation in _alpm_depcmp
Noticed when tweaking testdb, when we run _alpm_depcmp in loops and call it
seven million times, the strdup()/free() combo can add up. Remove the need
for any string duplication by some pointer manipulation and use of strncmp
instead of strcmp. Also kill the function logger and add an escape so we
don't needlessly retrieve the list of provides.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-21 17:25:02 -06:00
Dan McGee
f2dff08600 Overhaul archive fgets function
The old function was written in a time before we relied on it for nearly
every operation. Since then, we have switched to the archive backend and now
fast parsing is a big deal.

The former function made a per-character call to the libarchive
archive_read_data() function, which resulted in some 21 million calls in a
typical "load all sync dbs" operation. If we instead do some buffering of
our own and read the blocks directly, and then find our newlines from there,
we can cut out the multiple layers of overhead and go from archive to parsed
data much quicker.

Both users of the former function are switched over to the new signature,
made easier by the macros now in place in the sync backend parsing code.

Performance: for a `pacman -Su` (no upgrades available),
_alpm_archive_fgets() goes from being 29% of the total time to 12% The time
spent on the libarchive function being called dropped from 24% to 6%.

This pushes _alpm_pkg_find back to the title of slowest low-level function.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-21 14:58:17 -06:00
Allan McRae
c78a808c49 Only check diskspace availability if needs more than zero
The amount of diskspace needed for a transaction can be less than
zero.  Only test this against the available disk space if it is
positive, which avoids a comparison being made between signed and
unsigned types (-Wsign-compare).

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-20 18:03:34 -06:00
Allan McRae
a611879318 Always specify arguement type in function delcarations
Always declare a function with (void) rather than () when we expect
no arguements.  Fixes all warnings with -Wstrict-prototypes.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-20 18:03:21 -06:00
Dan McGee
8ac7f7e6e5 Use macros in sync DB parsing
This simplifies a lot of the repetative code and makes it obvious where the
tricky or different ones are (e.g. depends, dates). It also makes it
significantly easier to change the way this code works in the future.

There should be no functional change with this patch.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-20 18:02:47 -06:00
Dan McGee
ab9c0814d2 Add a cushion for diskspace checking
It is the minimum of 5% of disk capacity or 20 MiB on a per-partition basis.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-15 00:41:41 -06:00
Dan McGee
c7d332a26a Reorganize fields in package struct
Saves a few bytes due to padding (256 -> 248 bytes), especially on x86_64,
so we get the overhead of our new hash field right back.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-14 17:46:02 -06:00
Dan McGee
919bb6c9e0 Used hashed package name in _alpm_pkg_find
This results in huge gains to a lot of our codepaths since this is the most
frequent method of random access to packages in a list. The gains are seen
in both profiling and real life.

    $ pacman -Sii zvbi
    real: 0.41 sec -> 0.32 sec
    strcmp: 16,669,760 calls -> 473,942 calls
    _alpm_pkg_find: 52.73% -> 26.31% of time

    $ pacman -Su (no upgrades found)
    real: 0.40 sec -> 0.50 sec
    strcmp: 19,497,226 calls -> 524,097 calls
    _alpm_pkg_find: 52.36% -> 26.15% of time

There is some minor risk with this patch, but most of it should be avoided
by falling back to strcmp() if we encounter a package with a '0' hash value
(which we should not via any existing code path). We also do a strcmp once
hash values match to ensure against hash collisions. The risk left is that a
package name is modified once it was originally set, but the hash value is
left alone. That would probably result in a lot of other problems anyway.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-14 12:36:02 -06:00
Dan McGee
c2a73ba989 When setting package name, set hash value as well
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-14 12:06:31 -06:00
Dan McGee
dbf59a6b14 Add hash_sdbm function
This is prepping for the addition of a hash field to each package to greatly
speed up the string comparisons we frequently do on package name in
_alpm_pkg_find.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-14 11:56:32 -06:00
Dan McGee
d1d163c5a3 Use _alpm_pkg_find in deps search
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-14 11:50:50 -06:00
Dan McGee
a5e43b1605 Correctly force load of package reason
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-13 21:25:14 -06:00
Dan McGee
580fe21065 Abstract has_scriptlet() to package ops struct
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-13 21:25:14 -06:00
Dan McGee
c00e05992e Remove non-public functions from header
And rename accordingly.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-13 21:16:47 -06:00
Dan McGee
94d3d665f0 Mark sync_db_read() as static
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-13 21:14:03 -06:00
Bruno Widmann
5f36523af9 Abort db_populate if dbpath is not set
Rather than segfault. Fixes FS#21345.

Signed-off-by: Bruno Widmann <bruno.widmann@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-13 16:45:22 -06:00
Dan McGee
1ff8118212 Create sync/ DB directory if it does not exist
Rather than error out, this is easy enough. Looks quite similar to the code
in be_local for creating the local directory.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 22:41:02 -06:00
Allan McRae
21833d90e2 Merge desc and depends files in local db
Whenever depends is needed from the local db, so is desc.  The only
disadvantage to merging them is the additional time taken to read the
depends entries when they are not needed.  As depends is in general
relatively small, the additional time taken to read it in will be
negligable.  Also, merging these files will speed up local database
access due to less file seeks.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 21:42:41 -06:00
Dan McGee
209d0643e5 Attempt to stop installation when we encounter problems
This should hopefully address some of the concerns raised in FS#11639 with
regards to continuing after filling the disk up.

Add some more checks and passing of error conditions between our functions.
When a libarchive warning is encountered, check if it is due to lack of disk
space and if so upgrade it to an error condition.  A review of other
libarchive warnings suggests that these are less critical and can remain as
informative warning messages at this stage.

Note the presence of errors after extraction of an entire package is complete.
If so, we abort the transaction to be on the safe side and keep damage to a
minimum.

Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: make ENOSPC warning into an error]
Signed-off-by: Allan McRae <allan@archlinux.org>

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 21:42:41 -06:00
Dan McGee
b276a76dc9 Turn libarchive warnings into libalpm warnings
Rather than hiding these warnings, show them to the user as they happen.
This will prevent things such as hiding full filesystem errors (ENOSPC) from
the user as seen in FS#11639.

Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: adjust warning wording and add gettext calls]
Signed-off-by: Allan McRae <allan@archlinux.org>

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 21:42:41 -06:00
Dan McGee
3030542d10 Make reading from any file possible in sync DB
Whether it be "desc", "depends", or "deltas", it really doesn't matter-
treat them all the same and have the ability to read any data from any file
in that list. This continues the work in a44c7b8956.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:38:20 -06:00
Dan McGee
b7015af0fc diskspace style cleanups and small fixes
* Use our normal return() function syntax
* Rework a few things to reduce number of casts
* Fix void function argument declaration
* Add missing gettext _() call
* Remove need for seperate malloc() of statvfs/statfs structure
* Unify argument order of static functions- mountpoints now always
  passed first
* Count all files that start with '.' in a package against the DB
* Rename db to db_local in check_diskspace to clarify some code
* Fix some line wrapping to respect 80 characters

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2010-12-12 20:32:05 -06:00
Dan McGee
ec136784d4 Refactor statfs/statvfs type check
Turn it into a configure-type typedef, which allows us to reduce the
amount of duplicated code and clean up some #ifdef magic in the code
itself. Adjust some of the other defined checks to look at the headers
available rather than trying to pull in the right ones based on
configure checks.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2010-12-12 20:31:51 -06:00
Allan McRae
24684a616e Display progress bar for disk space checking
Checking disk space needed for a transaction can take a while so add
an informative progress bar.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:30:11 -06:00
Allan McRae
e22aa23c8f Add configuration option to control disk space checking
Disk space checking is likely to be an unnecessary bottleneck to
people with reasonable partition sizes so add a configuration option
to allow it to be disabled/enabled as wanted.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:30:05 -06:00
Allan McRae
3f0d98c124 Implement disk space checking
Pull together the work of the previous commits to implement a check
for enough free space before performing an install transaction. Abort
if there is not enough free space with an appropriate pm_errno..

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:29:59 -06:00
Allan McRae
e527699ddd Add functions to calculate approximate disk usage by packages
Two helper function are added to calculate the disk usage from packages
that are either currently installed on the system or from a package
archive.

Some minor approximations have been made:

1. Size for directories is not considered when removing a package from the
   filesystem to avoid multiple counting across packages. Also, these are
   reported to take zero size while installing.

2. Symlinks are reported to contribute zero size towards removal as
   libarchive reports them to have zero size for install.

3. Package data files (.PKGINFO, .INSTALL, .CHANGELOG) are counted towards
   usage on dbpath on install, but their size is not counted on package
   removal.

4. No handling of extra size needed for .pacsave/.pacnew files.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:29:54 -06:00
Allan McRae
695656d252 Add function to match file to mount point
For a given file, determine which mount point it is on or will be
installed to.  Take into account that we might be using an alternative
installation root.

Add additional helper function added to sort mount point list for easier
matching.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:29:49 -06:00
Allan McRae
f4e9deb6d7 Add function for listing system mount points
Add a mount_point_list() function that attempts to portably obtain
a list of system mount points and a struct to hold needed mount point
information.

Abort the transaction if we are unable to determine the mount points.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:29:43 -06:00
Allan McRae
adb10c3ab2 Prototype disk space checking functionality
Very basic prototyping for adding functionality to check free disk
space before performing package installs.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:29:37 -06:00
Dan McGee
f0051a7678 Remove AC_TYPE_SIGNAL usage
This macro is deemed unnecessary by even the autoconf guys, so we really
don't need to use it.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:29:30 -06:00
Dan McGee
fc74ef93b6 dirent usage cleanup
We were including the header in a lot of places it is no longer used.
Additionally, use the correct autoconf macro for determining whether
d_type is available as a member: HAVE_STRUCT_DIRENT_D_TYPE.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:29:20 -06:00
Allan McRae
8e9a69e8f6 Move MAX_DELTA_RATIO definition
This is a delta specific definition so it makes more sense to put
it in the delta specific header file.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:29:05 -06:00
Allan McRae
a44c7b8956 Combine reading from depends and desc in sync db
This will allow us to eventually combine the depends and desc entries
within the sync database.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:25:35 -06:00
Xavier Chantry
abefa23341 alpm/remove.c : respect --dbonly during remove-upgrade
When a -Sk or -Uk operation induced a removal of an existing local
package, --dbonly was not in effect and the files were all removed.

Fixing this behavior was already marked as TODO in database012 pactest
------------
TODO: I honestly think the above should NOT delete the original les, it
hould upgrade the DB entry without touching anything on the file stem.
E.g. this test should be the same as:
   pacman -R --dbonly dummy && pacman -U --dbonly dummy.pkg.tar.gz
------------

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
[Dan: small coding style touchup]
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:19:24 -06:00
Xavier Chantry
d0d8f605d5 alpm: don't expose alpm_depcmp
Either we expose all low levels function dealing with pmdepend_t
(splitdep and depfree come to mind), or we don't.

Since none of the tools use depcmp, I chose to remove it. In the future,
we might want to expose higher level functions such as
alpm_find_satisfier, or just lower level functions like splitdep and
depfree together with depcmp.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:08:54 -06:00
Xavier Chantry
d5a7dc67d9 alpm: kill alpm_deptest
This has been replaced by the more flexible alpm_find_satisfier
function, and alpm_deptest was completely unsused now.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:08:27 -06:00
Xavier Chantry
a03daad073 alpm: add new alpm_find_satisfier function
whatprovides and splitdep were removed, so depcmp alone is quite useless
now without splitdep, and deptest is not flexible enough.

Introduce a new alpm_find_satisfier which is hopefully more flexible,
this should make implementation of deptest very easy, and also help alpm
tools such as pactree.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:07:58 -06:00
Xavier Chantry
c2cce1f46a Fix a few problems reported by clang-analyzer
One missing NULL-check and 3 dead assignments.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-12 20:07:15 -06:00