Sorry for this being such a huge patch, but I believe it is necessary for
quite a few reasons which I will attempt to explain herein. I've been
mulling this over for a while, but wasn't super happy with making the
download interface more complex. Instead, if we carefully order things in
the internal download code, we can actually make the interface simpler.
1. FS#15657 - This involves `name.db.tar.gz.part` files being left around the
filesystem, and then causing all sorts of issues when someone attempts to
rerun the operation they canceled. We need to ensure that if we resume a
download, we are resuming it on exactly the same file; if we cannot be
almost postive of that then we need to start over.
2. http://www.mail-archive.com/pacman-dev@archlinux.org/msg03536.html - Here
we have a lighttpd bug to ruin the day. If we send both a Range: header and
If-Modified-Since: header across the wire in a GET request, lighttpd doesn't
do what we want in several cases. If the file hadn't been modified, it
returns a '304 Not Modified' instead of a '206 Partial Content'. We need to
do a stat (e.g. HEAD in HTTP terms) operation here, and the proceed
accordingly based off the values we get back from it.
3. The mtime stuff was rather ugly, and relied on the called function to
write back to a passed in reference, which isn't the greatest. Instead, use
the power of the filesystem to contain this info. Every file downloaded
internally is now carefully timestamped with the remote file time. This
should allow the resume logic to work. In order to guarantee this, we need
to implement a signal handler that catches interrupts, notifies the running
code, and causes it to set the mtimes on the file. It then rethrows the
signal so the pacman signal handler (or any frontend) works as expected.
4. We did a lot of funky stuff in trying to track the DB last modified time.
It is a lot easier to just keep the downloaded DB file around and track the
time on that rather than in a funky dot file. It also kills a lot of code.
5. For GPG verification of the databases down the road, we are going to need
the DB file around for at least a short bit of time anyway, so this gets us
closer to that.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Xav: fixed printf with off_t]
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
* It makes the code clearer to read/understand
* Cppcheck tool doesn't show this anymore: [./util.c:215]: (error) Resource leak: fd
[Dan: don't change the coding style]
Signed-off-by: Dan McGee <dan@archlinux.org>
For some packages, generally the 'any' arch ones, a build step is not
required and therefore can be skipped. In these cases, a package()
function without a build() one is sufficient.
As a side effect, this commit makes meta packages without any function
at all in the PKGBUILD possible.
Fixes FS#15147.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Since commit fb97d32, which brought in this test, support for split
PKGBUILDs was added, and therefore, all values of pkgname and also
pkgbase have to be checked now.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
makepkg looks for a package() function when building a single package
but package_$pkgname() style package functions when building a split
package. This patch allows the use of a package_$pkgname() function
when building a single package for consistency. This is achieved by
having makepkg consider a non-split package with a package_$pkgname()
function as a split package (creating just the one package).
Fixes FS#16622.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
* FS#16623, second half of makepkg
* Includes stuff like -o to ||, -a to &&, etc.
* if [ $(type ... preserved due to a bash bug with [[ and set -e and ERR traps
Signed-off-by: Isaac Good <pacman@isaac.otherinbox.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
* FS#16623, first half of makepkg
* Includes stuff like -o to ||, -a to &&, etc.
* if [ $(type ... preserved due to a bash bug with [[ and set -e and ERR traps
Signed-off-by: Isaac Good <pacman@isaac.otherinbox.com>
[Dan: made commit message useful]
Signed-off-by: Dan McGee <dan@archlinux.org>
FS#16623 suggested this change for makepkg; this patch applies it to the
remaining files in the scripts directory.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Dan McGee <dan@archlinux.org>
The source command triggers / might trigger the ERR trap which makes
makepkg abort right after a successful installation of missing
dependencies.
Thanks to Xavier Chantry <shiningxc@gmail.com> for finding this
solution.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Fixes FS#16871 and makes the pkgdesc workaround obsolete.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Thanks to Laszlo Papp <djszapi@archlinux.us> for the following catch:
opendir(path)) == (DIR *)-1;
is maybe the result of misunderstanding the manpage. If an opendir() call
isn't successful it returns NULL rather than '(DIR *)-1'.
Noticed-by: Laszlo Papp <djszapi@archlinux.us>
Signed-off-by: Dan McGee <dan@archlinux.org>
Currently makepkg takes the commandline arguments, assigns them to a
variable and passes that variable to the next makepkg call (within
fakeroot).
Use a comination of quotes and arrays in this process to ensure any
arguments passed within quotes and containing spaces stay as a single
argument during the second makepkg call.
Thanks to Dan for figuring out how to get this working.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
alpm_pkg_get_name() gives us little benefit in backend code besides a NULL
check on the package passed in; we could do that ourself if necessary. By
changing to direct references in the cases where we are sure we have a valid
package, we save a function call each time we need a package name. This
function can't be inlined because it is externally accessible.
This cuts the calls to get_name() from 1.3 million times in a
pacman -Qu operation to around 2400.
Signed-off-by: Dan McGee <dan@archlinux.org>
In doing so, it is possible to get rid of all the tests for colored
messages except for one global one.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
pkgbase is used in the following rm calls, and since pkgname can be
present when pkgbase is not, it is safer to check for pkgbase.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Commit c7e4d10d introduced a small error in the testing of whether
a package is already built.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows building a mixture of binary and arch=any packages.
Fixes FS#15955.
The value of CARCH is no longer overridden to "any" in when arch=any
is used and the assigning of the "any" arch is delayed to during the
packaging stage. Adjustments were required to fix installing and
checking for pre-built packages of varing arches.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Fixing a single package within a split package requires the overriding
or pkgrel. In very rare (but existing) cases, it is useful to
override pkgver. Partial fix for FS#15955.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes the following valgrind warning :
==26831== Syscall param rt_sigaction(act->sa_flags) points to uninitialised
byte(s)
==26831== at 0x4282547: __libc_sigaction (in /lib/libc-2.10.1.so)
==26831== by 0x403C693: download_internal (dload.c:152)
==26831== by 0x403D0E4: _alpm_download_single_file (dload.c:311)
==26831== by 0x4033B72: alpm_db_update (be_files.c:319)
==26831== by 0x805205E: pacman_sync (sync.c:257)
==26831== by 0x804EE54: main (pacman.c:1120)
==26831== Address 0xbec6cc04 is on thread 1's stack
==26831==
==26831== Syscall param rt_sigaction(act->sa_restorer) points to
uninitialised byte(s)
==26831== at 0x4282547: __libc_sigaction (in /lib/libc-2.10.1.so)
==26831== by 0x403C693: download_internal (dload.c:152)
==26831== by 0x403D0E4: _alpm_download_single_file (dload.c:311)
==26831== by 0x4033B72: alpm_db_update (be_files.c:319)
==26831== by 0x805205E: pacman_sync (sync.c:257)
==26831== by 0x804EE54: main (pacman.c:1120)
==26831== Address 0xbec6cc08 is on thread 1's stack
==26831==
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This also removes the awk dependency from makepkg and repo-add.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Pacman's fgets function in the API used hardcoded numbers to identify the size.
This is not good practice, so replace them with sizeof handling.
Signed-off-by: Laszlo Papp <djszapi@archlinux.us>
Signed-off-by: Dan McGee <dan@archlinux.org>
Pacman's long option parsing used hardcoded numbers to identify them.
This is not good practice, so replace them with enumeration constants.
Signed-off-by: Laszlo Papp <djszapi@archlinux.us>
Signed-off-by: Dan McGee <dan@archlinux.org>
Currently, a changelog is added to a package if a specific file with a
hardcoded name exists in the PKGBUILD's directory. This approach is not
pretty and also inconsistent with the handling of install files, but it
works.
With the introduction of split PKGBUILDs, however, a drawback in this
old behavior has arisen: you only have the possibility to include one
specific changelog file in either every package defined in the PKGBUILD
or in none.
The use of an additional variable, `changelog`, works around this issue
and makes it possible to include a changelog in only some of the
packages, and besides, each package of the PKGBUILD can have its own
changelog file.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This offers a cleaner way to deal with constant in enum and allow easy
maintainance
Signed-off-by: solsTiCe d'Hiver <solstice.dhiver@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
When redirecting both stderr and stdout and using the 2>&1 construct,
you have to redirect stdout first. Otherwise stderr will be redirected to
the 'old' stdout and not to the new resource.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Dan McGee <dan@archlinux.org>
If the -d switch was invoked with -S (or -U), the removes list was simply
lost, because trans->remove was computed in an
"if(!(trans->flags & PM_TRANS_FLAG_NODEPS))" block.
I've added a new pactest file, sync045.py (derived from sync043.py) to test
this.
Additionally, I did some other minor cleanups in sync_prepare:
* preferred list is not needed anymore
* I removed a needless alpm_list_remove_dupes line (the target list should
not contain dupes at all)
* I moved alpm_list_free(remove); to cleanup part to eliminate a possible
memleak
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is more efficient than alpm_list_diff since it assumes the two lists
are sorted. And also we get the two sides of the diff.
Even sorting should more efficient than the current list_diff. Sorting the
two lists should be O(n*log(n)+m*log(m)) while the current list_diff is
O(n*m). So I also reimplemented list_diff using list_diff_sorted.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
update download -> fetch
This just meant that we used XferCommand even if internal download was
available, no big deal.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
