Commit Graph

3979 Commits

Author SHA1 Message Date
Dave Reisner 6dc71926f9 lib/dload: prevent large file attacks
This means creating a new struct which can pass more descriptive data
from the back end sync functions to the downloader. In particular, we're
interested in the download size read from the sync DB. When the remote
server reports a size larger than this (via a content-length header),
abort the transfer.

In cases where the size is unknown, we set a hard upper limit of:

* 25MiB for a sync DB
* 16KiB for a signature

For reference, 25MiB is more than twice the size of all of the current
binary repos (with files) combined, and 16KiB is a truly gargantuan
signature.

Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05 22:58:55 -04:00
Dave Reisner 6c9b82e72a dload: handle irregular URLs
URLs might end with a slash and follow redirects, or could be a
generated by a script such as /getpkg.php?id=12345. In both cases, we
may have a better filename that we can write to, taken from either
content-disposition header, or the effective URL.

Specific to the first case, we write to a temporary file of the format
'alpmtmp.XXXXXX', where XXXXXX is randomized by mkstemp(3). Since this
is a randomly generated file, we cannot support resuming and the file is
unlinked in the event of an interrupt.

We also run into the possibility of changing out the filename from under
alpm on a -U operation, so callers of _alpm_download can optionally pass
a pointer to a *char to be filled in by curl_download_internal with the
actual filename we wrote to. Any sync operation will pass a NULL pointer
here, as we rely on specific names for packages from a mirror.

Fixes FS#22645.

Signed-off-by: Dave Reisner <d@falconindy.com>
2011-07-05 22:58:27 -04:00
Dave Reisner 44889da5b7 dload: rearrange code to avoid extra cpp block
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05 17:18:23 -04:00
Dave Reisner 6cce517f1a lib/rawstr: borrow raw string functions from curl
We'll need these functions to do locale agnostic and case insensitive
string comparisons.

Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05 17:18:23 -04:00
Allan McRae dfc532668d makepkg: update --pkg desciption in man page
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:22:45 -05:00
Allan McRae 77a93328cf Add library files to POTFILES.in
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:22:37 -05:00
Allan McRae bfd6d22be2 parse_options: accept multiple arguments
Allow command-line options to accept multiple arguments without
additional quoting by taking the list of arguments until one
starting with a "-" is reached.

The only current use of this is the --pkg option in makepkg.  This
allows (e.g.)

makepkg --pkg foo bar

and packages "foo" and "bar" will be built.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:22:31 -05:00
Allan McRae 87ee38d8b3 parse_options: implement optional arguments
This allows options specified with a trailing "::" to optionally
take arguments.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:22:25 -05:00
Allan McRae ddb8617d96 parse_options: add missing newlines
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:22:20 -05:00
Allan McRae 24324ff0e1 Simplify alpm_list_previous
We can readily detect the first node in a list by checking if
node->prev->next is NULL. So there is no need to pass the head
of the list to this function and its prototype now looks like
all the other item accessors.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:22:11 -05:00
Allan McRae 97103f860d Remove alpm_list_first
The only thing this accessor did was remove the const qualifier
given our entire list implementation requires passing around the
head anyway.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:22:05 -05:00
Dan McGee c748eadc80 Allow invalid sync DBs to be returned by the library
They are placeholders, but important for things like trying to re-sync a
database missing a signature. By using the alpm_db_validity() method at
the right time, a client can take the appropriate action with these
invalid databases as necessary.

In pacman's case, we disallow just about anything that involves looking
at a sync database outside of an '-Sy' operation (although we do check
the validity immediately after). A few operations are still permitted-
'-Q' ops that don't touch sync databases as well as '-R'.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:13:20 -05:00
Dan McGee 07502f2d82 Allow frontend access to signature verification information
Show output in -Qip for each package signature, which includes the UID
string from the key ("Joe User <joe@example.com>") and the validity of
said key. Example output:

Signatures     : Valid signature from "Dan McGee <dpmcgee@gmail.com>"
                 Unknown signature from "<Key Unknown>"
                 Invalid signature from "Dan McGee <dpmcgee@gmail.com>"

Also add a backend alpm_sigresult_cleanup() function since memory
allocation took place on this object, and we need some way of freeing
it.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:13:20 -05:00
Dan McGee e8443b1685 Correctly check the GPG error codes
The error code is in fact a bitmask value of an error code and an error
source, so use the proper function to get only the relevant bits. For
the no error case, this shouldn't ever matter, but it bit me when I was
trying to compare the error code to other values and wondered why it
wasn't working, so set a good example.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:13:20 -05:00
Dan McGee 7af0ab1cde signing: move to new signing verification and return scheme
This gives us more granularity than the former Never/Optional/Always
trifecta. The frontend still uses these values temporarily but that will
be changed in a future patch.

* Use 'siglevel' consistenly in method names, 'level' as variable name
* The level becomes an enum bitmask value for flexibility
* Signature check methods now return a array of status codes rather than
  a simple integer success/failure value. This allows callers to
  determine whether things such as an unknown signature are valid.
* Specific signature error codes mostly disappear in favor of the above
  returned status code; pm_errno is now set only to PKG_INVALID_SIG or
  DB_INVALID_SIG as appropriate.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05 10:13:20 -05:00
Dan McGee 1ce7f39ad7 Merge remote-tracking branch 'allan/ALPM'
Conflicts:
	lib/libalpm/be_local.c
	lib/libalpm/be_package.c
	lib/libalpm/conflict.c
	lib/libalpm/diskspace.c
	lib/libalpm/dload.c
	lib/libalpm/remove.c
2011-07-03 14:44:57 -05:00
Dan McGee c839415e3f conflict code tweaks and cleanups
We can take advantage of a few things on our new and improved filelist
in this code.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 14:29:30 -05:00
Dan McGee 26195f8da1 diskspace: remove all libarchive usage
Now that the filelists capture mode and size information, we can read
the data from there and prevent having to loop through and uncompress
every archive to check required diskspace usage.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 14:29:30 -05:00
Dan McGee 6a6fc3107f Move alpm filelists to a struct object
This allows us to capture size and mode data when building filelists
from package files. Future patches will take advantage of this newly
available information, and frontends can use it as well.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 14:29:30 -05:00
Dan McGee a2995f586e pactest: add a few more checks to fileconflict checks
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 13:44:45 -05:00
Dan McGee 06840f14b4 Fix debug logger without a newline
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 13:44:26 -05:00
Allan McRae a7b02d07f4 Do not replicate files list when removing packages
This saves replicating the potentially large list of files in a package
that is being removed.

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 13:42:43 -05:00
Allan McRae ab79b13079 Add alpm_list_previous method
Helper function to get the previous item in a list

Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 13:42:24 -05:00
Dan McGee f612e5ede7 checkdeps: remove unnecessary list join and copy
We can just perform the same search operation on both lists.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 13:39:13 -05:00
Dan McGee 925d74f38d be_local: use macros in database loading similar to be_sync
This removes some of the repetition in the code for reading and parsing
database file lines.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 13:37:04 -05:00
Dan McGee de8b9a85a5 be_sync: make READ_NEXT() a no-arg macro
We passed in 'line', but not 'buf.line'. In addition, the macros
building off of READ_NEXT() assume variable names anyway. Since we only
use these macros in one function, might as well simplify them.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03 13:07:19 -05:00
Allan McRae afc96f2ab3 Prefix _alpm_errno_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:39 +10:00
Allan McRae bd88a8d551 Prefix alpm_transprog_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:39 +10:00
Allan McRae 495ba26e63 Prefix alpm_transconv_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:39 +10:00
Allan McRae 3189d3bc4a Prefix alpm_transevt_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:39 +10:00
Allan McRae 39262acab6 Prefix alpm_transflag_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:39 +10:00
Allan McRae ca43fdd92f Prefix alpm_loglevel_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:38 +10:00
Allan McRae d796d1cdda Prefix alpm_fileconflicttype_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:38 +10:00
Allan McRae f818f570c5 Prefix alpm_depmod_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:38 +10:00
Allan McRae eb39a9482b Prefix alpm_pkgreason_t members with ALPM
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02 02:01:38 +10:00
Dan McGee cf1401a04d signing: check validity of all available signatures
Change the check into a loop over all signatures present and returned by
GPGME. Also modify the return values and checks slightly now that I know
a little bit more about what type of values are returned.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 17:25:53 -05:00
Dan McGee 23a2d2c16a Make alpm_db_get_sigverify_level() public
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 16:16:41 -05:00
Dan McGee 68284da0d7 Add an alpm_db_get_valid() public function
This allows one to check if a database is valid or invalid.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 16:05:20 -05:00
Dave Reisner 6633b8e5c2 move proto files to new subdirectory, proto/
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 13:37:58 -05:00
Dave Reisner 98a2fc8deb pacman: return with 128+signum on signaled exit
This is a convention that is widely followed in *nix and posix-ish
environments. We should follow it, too.

Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 12:43:57 -05:00
Dan McGee 35ffe6af2d pactest: remove no longer necessary newline hacks
libalpm can now cope with this as of commit 719e0d3ddb.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 12:02:33 -05:00
Dan McGee 0ed848a9ea pactest: create packages in memory
This is similar to what was just done for the sync databases. Move a few
pieces around so we never need to actually write out the filesystem to
create a package, and simply stream the tarfile out from the data we've
collected.

Once again, a few newline addition hacks and other things have to be
left in place in order not to break everything; this time however most
of the assumptions are in pactest and not libalpm.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 11:55:00 -05:00
Dan McGee 13235ba65a Make local_db_read() private to the local backend
There is little need to expose the guts of this function even within the
library. Make it static in be_local.c, and clean up a few other things
since we know exactly where it is being called from:

* Remove unnecessary origin checks in _cache_get_*() methods- if you are
  calling a cache method your package type will be correct.
* Remove sanity checks within local_db_read() itself- packages will
  always have a name and version if they get this far, and the package
  object will never be NULL either.

The one case calling this from outside the backend was in add.c, where
we forced a full load of a package before we duplicated it. Move this
concern elsewhere and have pkg_dup() always force a full package load
via a new force_load() function on the operations callback struct.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 11:51:49 -05:00
Dan McGee b94e8ecd1f Fix a few warnings pointed out via clang scan-build
Some of these are legit (the backup hash NULL checks), while others are
either extemely unlikely or just impossible for the static code
analysis to prove, but are worth adding anyway because they have little
overhead.

Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 11:51:36 -05:00
Florian Pritz 9efd10cd2a fix vim syntax highlighting of .sh files
vim recognises what type of shell script it's dealing with by looking at
the shebang. If detection fails it falls back to sh which doesn't
support some bash features. Adding a normal, possibly broken, shebang
which gets fixed by the Makefile allows vim to detect bash syntax.

Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 10:44:45 -05:00
Dan McGee c2cce4f3f5 Merge remote-tracking branch 'dave/makepkg' 2011-06-30 10:41:49 -05:00
Dan McGee ad577b3cb4 Merge remote-tracking branch 'allan/breakshit' 2011-06-30 10:37:08 -05:00
Dave Reisner 0f4aaeee42 lib/util: modify entry_prefix, not prefix
Modifying prefix caused tmp directories to be left behind after
running scriptlets, and the path '/' to be passed to _alpm_rmrf. Broken
in f01c6f.

Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 10:34:32 -05:00
Dave Reisner 002d2fda7a lib/alpm: unlock the handle before freeing it
This avoids, probably among other things, leaving the lock file in place
after a SIGINT'd sync DB update.

Fixes regression introduced in 4f8ae2b.

Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 10:33:24 -05:00
Dave Reisner 2860ade2f5 repo-add.sh.in: avoid being clever with repo repacking
Revert to the old behavior that 6f5a90 attempted to simplify and go with
the original proposed solution of using "ugly" bash to detect empty
directories.

Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30 10:32:15 -05:00