1
0
mirror of https://github.com/moparisthebest/pacman synced 2024-11-15 22:05:02 -05:00
Commit Graph

71 Commits

Author SHA1 Message Date
Dan McGee
bb3dada871 Convert package filelists to an array instead of linked list
This accomplishes quite a few things with one rather invasive change.

1. Iteration is much more performant, due to a reduction in pointer
   chasing and linear item access.
2. Data structures are smaller- we no longer have the overhead of the
   linked list as the file struts are now laid out consecutively in
   memory.
3. Memory allocation has been massively reworked. Before, we would
   allocate three different pieces of memory per file item- the list
   struct, the file struct, and the copied filename. What this resulted
   in was massive fragmentation of memory when loading filelists since
   the memory allocator had to leave holes all over the place. The new
   situation here now removes the need for any list item allocation;
   allocates the file structs in contiguous memory (and reallocs as
   necessary), leaving only the strings as individually allocated. Tests
   using valgrind (massif) show some pretty significant memory
   reductions on the worst case `pacman -Ql > /dev/null` (366387 files
   on my machine):

   Before:
     Peak heap:   54,416,024 B
	 Useful heap: 36,840,692 B
	 Extra heap:  17,575,332 B

   After:
     Peak heap:   38,004,352 B
	 Useful heap: 28,101,347 B
	 Extra heap:   9,903,005 B

Several small helper methods have been introduced, including a list to
array conversion helper as well as a filelist merge sort that works
directly on arrays.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-21 15:04:30 -05:00
Dan McGee
7af0ab1cde signing: move to new signing verification and return scheme
This gives us more granularity than the former Never/Optional/Always
trifecta. The frontend still uses these values temporarily but that will
be changed in a future patch.

* Use 'siglevel' consistenly in method names, 'level' as variable name
* The level becomes an enum bitmask value for flexibility
* Signature check methods now return a array of status codes rather than
  a simple integer success/failure value. This allows callers to
  determine whether things such as an unknown signature are valid.
* Specific signature error codes mostly disappear in favor of the above
  returned status code; pm_errno is now set only to PKG_INVALID_SIG or
  DB_INVALID_SIG as appropriate.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:13:20 -05:00
Dan McGee
1ce7f39ad7 Merge remote-tracking branch 'allan/ALPM'
Conflicts:
	lib/libalpm/be_local.c
	lib/libalpm/be_package.c
	lib/libalpm/conflict.c
	lib/libalpm/diskspace.c
	lib/libalpm/dload.c
	lib/libalpm/remove.c
2011-07-03 14:44:57 -05:00
Dan McGee
6a6fc3107f Move alpm filelists to a struct object
This allows us to capture size and mode data when building filelists
from package files. Future patches will take advantage of this newly
available information, and frontends can use it as well.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 14:29:30 -05:00
Allan McRae
afc96f2ab3 Prefix _alpm_errno_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:39 +10:00
Allan McRae
ca43fdd92f Prefix alpm_loglevel_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:38 +10:00
Allan McRae
cd1e39ba62 Rename pmbackup_t to alpm_backup_t
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-06-28 23:28:23 +10:00
Allan McRae
9540dfc4d9 Rename pmdepend_t to alpm_depend_t
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-06-28 23:28:23 +10:00
Allan McRae
8a04bc25a1 Rename pmpkg_t to alpm_pkg_t
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-06-28 23:26:39 +10:00
Allan McRae
64c1cf7921 Rename pmhandle_t to alpm_handle_t
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-06-28 14:04:00 +10:00
Dan McGee
f12ead2cf2 Remove three unnecessary usages of alpm_list_count()
For the files count when loading from a package, we can keep a counter.
The two in the frontend were completely useless due to the fact that if
sync_dbs is non-NULL, alpm_list_count() will always be greater than 0.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-24 01:36:47 -05:00
Dan McGee
54ef162a1a Convert backup list to new pmbackup_t type
This allows us to separate the name and hash elements in one place and
not scatter different parsing code all over the place, including both
the frontend and backend.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-22 12:31:12 -05:00
Kerrick Staley
d980bd403d Documented purpose of be_*.c
Added a line to the top of each of be_local.c, be_package.c, and
be_sync.c indicating their purposes.

Signed-off-by: Kerrick Staley <mail@kerrickstaley.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-20 00:17:54 -05:00
Dan McGee
ee015f086f Ensure handle is valid and pm_errno is reset when calling into API
We didn't do due diligence before and ensure prior pm_errno values
weren't influencing what happened in further ALPM calls. I observed one
case of early setup code setting pm_errno to PM_ERR_WRONG_ARGS and that
flag persisting the entire time we were calling library code.

Add a new CHECK_HANDLE() macro that does two things: 1) ensures the
handle variable passed to it is non-NULL and 2) clears any existing
pm_errno flag set on the handle. This macro can replace many places we
used the ASSERT(handle != NULL, ...) pattern before.

Several other other places only need a simple 'set to zero' of the
pm_errno field.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-14 10:01:08 -05:00
Dan McGee
ba63e31cc7 Small handle related cleanups
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-14 08:39:02 -05:00
Dan McGee
52bffd2457 Switch all logging to use handle directly
This is the last user of our global handle object. Once again the diff
is large but the functional changes are not.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-13 19:41:16 -05:00
Dan McGee
e2aa952689 Move pm_errno onto the handle
This involves some serious changes and a very messy diff, unfortunately.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-13 19:38:38 -05:00
Dan McGee
d76341297a Require handle for alpm_pkg_load()
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-13 19:35:22 -05:00
Dan McGee
bcd442761b Fix memory leak if package sig was invalid
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-13 17:44:49 -05:00
Dan McGee
17a6ac5675 Require handle argument to all alpm_option_(get|set)_*() methods
This requires a lot of line changes, but not many functional changes as
more often than not our handle variable is already available in some
fashion.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-09 14:24:45 -05:00
Dan McGee
cc25576f8b Use standard errno codes in return from _alpm_archive_fgets
This allows us to not require the context (e.g. handle) when calling
this function. Also beef up the checks in the two callers of this
function to bail if the last return code is not ARCHIVE_EOF, which is
the expected value.

This requires a change to one of the pactest return codes and the
overall result of the test, but results in a much safer operating
condition whereby invalid database entries will stop the operation.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-03 14:37:51 -05:00
Dan McGee
de36c5fac4 Push down extern handle variable to files that need it
This will make the patching process less invasive as we start to remove
this variable from all source files.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-03 12:23:32 -05:00
Dan McGee
7f98460e37 Add handle attribute to pmpkg_t struct
Similar to what we just did for the database; this will make it easy to
always know what handle a given package originated from.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-03 12:13:22 -05:00
Dan McGee
78cbc045c1 Remove ALPM_LOG_FUNC macro
The usefulness of this is rather limited due to it not being compiled
into production builds. When you do choose to see the output, it is
often overwhelming and not helpful. The best bet is to use a debugger
and/or well-placed fprintf() statements.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-03 11:48:24 -05:00
Allan McRae
7680f46157 Deal with unused function parameters correctly
This started off removing the "(void)foo" hacks to work around
unused function parameters and ended up fixing every warning
generated by -Wunused-parameter.

Dan: rename to UNUSED.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-05-04 15:53:49 -05:00
Dan McGee
4d63ebe2fb Perform package verification at package load time
Both md5sum verification and PGP verification can and should be done at
package load time. This allows verification to happen as early as
possible for packages provided by filename and loaded in the frontend,
and moves more stuff out of sync_commit that doesn't really belong
there. This should also set the stage for simplified parallel loading of
packages later down the road.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-24 10:48:34 -05:00
Dan McGee
31e55b8049 signing: let GPGME handle loading signatures from files
Rather than go through all the hassle of doing this ourselves, just let
GPGME handle the work by passing it a file handle.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-24 10:48:33 -05:00
Dan McGee
3c5661ec3c Form the signature file location in one place
Since we do this for all cases anyway.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-20 20:10:27 -05:00
Dan McGee
791928dc48 Header inclusion cleanup
This does touch a lot of things, and hopefully doesn't break things on
other platforms, but allows us to also clean up a bunch of crud that no
longer needs to be there.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-20 20:09:13 -05:00
Dan McGee
4af6c72d79 syntax: if/while statements should have no trailing space
This is the standard, and we have had a few of these introduced lately
that should not be here.

Done with:
  find -name '*.c' | xargs sed -i -e 's#if (#if(#g'
  find -name '*.c' | xargs sed -i -e 's#while (#while(#g'

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-20 19:47:39 -05:00
Dave Reisner
91594a1ef8 style cleanup: cast as (type *) not (type*)
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-20 19:04:53 -05:00
Dave Reisner
fdcfcf28a2 lib: remove dead code in be_local and be_package
Signed-off-by: Dave Reisner <d@falconindy.com>
2011-04-20 19:04:27 -05:00
Dan McGee
efd8ae483f Merge branch 'maint'
Conflicts:
	lib/libalpm/alpm.h
	lib/libalpm/trans.c

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-15 18:41:49 -05:00
Dan McGee
8df7208d7b Don't create two pmpkg_t objects in be_package
Ensure we only have one- this looks like the result of a bad merge from
old 2008 signing code with the current stuff which has changed quite a
bit.

Originally-seen-by: Rémy Oudompheng <remyoudompheng@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-11 15:56:34 -05:00
Rémy Oudompheng
6ebb6fec8b Move documentation for public package function to alpm.h 2011-04-09 22:36:43 +02:00
Dan McGee
4a3cd364d0 Temporary fix for new warnings from gcc 4.6
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-03-27 20:48:26 -05:00
Dan McGee
9a3325a56d Refactor signature loading code into common function
We can use this for both standalone package signatures as well as
standalone database signatures.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-03-23 22:56:54 -05:00
Dan McGee
88746ec067 Read in .sig files when opening a package file
If a .sig file sits side-by-side on the filesystem with a package archive,
read it in during the package struct creation process so we can verify it at
a later time if necessary.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-03-23 02:22:00 -05:00
Dan McGee
0303b26b1e Style change: return(x) --> return x
This was discussed and more or less agreed upon on the mailing list. A
huge checkin, but if we just do it and let people adjust the pain will
end soon enough. Rebasing should be relatively straighforward for anyone
that sees conflicts; just be sure you use the new return style if
possible.

The following semantic patch was used to do the change, along with some
hand-massaging in order to preserve parenthesis where appropriate:

The semantic match that finds this problem is as follows, although some
hand-massaging was done in order to keep parenthesis where appropriate:
(http://coccinelle.lip6.fr/)

// <smpl>
@@
expression a;
@@
- return(a);
+ return a;

// </smpl>

A macros_file was also provided with the following content:

Additional steps taken, mainly for ASSERT() macros:
$ sed -i -e 's#return(NULL)#return NULL#' lib/libalpm/*.c
$ sed -i -e 's#return(-1)#return -1#' lib/libalpm/*.c

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-03-20 19:49:45 -05:00
Dan McGee
cfa7602032 Don't generate filelist unless we are going to use it
We throw it away if !full, so no need to waste time creating the list in the
first place.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-16 20:57:07 -06:00
Dan McGee
06cbb516c3 Avoid a memmove by advancing value pointer
In packages, our description file contains:
key = value is here
type entries, and we passed "key " and " value is here" to our strtrim
function, causing us to always memmove the value portion to remove the
space. Since this is a throwaway buffer, do the advancing on our own before
trimming to save the need to shift memory around; "value is here" will now
be passed and strtrim will be responsible for trailing whitespace.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-11 10:23:43 -06:00
Dan McGee
87240dae6d Fix locale.h/setlocale inclusion with --disable-nls
Noted in FS#22697. When I factored out _alpm_parsedate() into a common
function, I didn't move the <locale.h> include properly, causing a build
failure when NLS is disabled and this header isn't automatically included
everywhere.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-02-01 09:59:12 -06:00
Dan McGee
9b876fff09 Ignore known but unused package descfile fields
We explicitly place 'pkgbase' (and used to place 'force') fields inside
PKGINFO files, so ignore them silently instead of printing an error for
them. Also make the error message for unknown keys actually contain the key.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-28 17:34:06 -06:00
Dan McGee
bf46e04614 Remove epoch as an independent field
Instead, go the same route we have always taken with version-release in
libalpm and treat it all as one piece of information. Makepkg is the only
script that knows about epoch as a distinct value; from there on out we will
parse out the components as necessary.

This makes the code a lot simpler as far as epoch handling goes. The
downside here is that we are tossing some compatibility to the wind;
packages using force will have to be rebuilt with an incremented epoch to
keep their special status.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-21 19:30:45 -06:00
Dan McGee
9a82cb92a4 Small fix to download size lookup and a logger
These were just two small things I came across today and found could be
fixed or helpful, so I've added them and I'm not sure what else to bundle
them with. commit_count++

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-11 21:15:04 -06:00
Xavier Chantry
281a4c0a4f libalpm/be_package.c: fix small memleak
file_pkg_ops can be a static struct like in other backends, we just need
to initialize it at some point.

Dan: add initialization flag.

Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-07 21:27:39 -06:00
Dan McGee
4bc6ed56aa Refactor old date parsing into single method
We've managed to duplicate this four times at this point, so make it a
method in util.c instead.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-07 20:55:05 -06:00
Allan McRae
d288240426 Update copyright years for 2011
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-01-07 18:47:37 -06:00
Allan McRae
351250adb4 Declare all local functions static
All functions that are limited to the local translation unit are
declared static.  This exposed that the _pkg_get_deltas declaration
in be_local.c was being satified by the function in packages.c which
when declared static caused linker failures.

Fixes all warnings with -Wmissing-{declarations,prototypes}.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-30 09:39:23 -06:00
Dan McGee
f2dff08600 Overhaul archive fgets function
The old function was written in a time before we relied on it for nearly
every operation. Since then, we have switched to the archive backend and now
fast parsing is a big deal.

The former function made a per-character call to the libarchive
archive_read_data() function, which resulted in some 21 million calls in a
typical "load all sync dbs" operation. If we instead do some buffering of
our own and read the blocks directly, and then find our newlines from there,
we can cut out the multiple layers of overhead and go from archive to parsed
data much quicker.

Both users of the former function are switched over to the new signature,
made easier by the macros now in place in the sync backend parsing code.

Performance: for a `pacman -Su` (no upgrades available),
_alpm_archive_fgets() goes from being 29% of the total time to 12% The time
spent on the libarchive function being called dropped from 24% to 6%.

This pushes _alpm_pkg_find back to the title of slowest low-level function.

Signed-off-by: Dan McGee <dan@archlinux.org>
2010-12-21 14:58:17 -06:00