Sorry for this being such a huge patch, but I believe it is necessary for
quite a few reasons which I will attempt to explain herein. I've been
mulling this over for a while, but wasn't super happy with making the
download interface more complex. Instead, if we carefully order things in
the internal download code, we can actually make the interface simpler.
1. FS#15657 - This involves `name.db.tar.gz.part` files being left around the
filesystem, and then causing all sorts of issues when someone attempts to
rerun the operation they canceled. We need to ensure that if we resume a
download, we are resuming it on exactly the same file; if we cannot be
almost postive of that then we need to start over.
2. http://www.mail-archive.com/pacman-dev@archlinux.org/msg03536.html - Here
we have a lighttpd bug to ruin the day. If we send both a Range: header and
If-Modified-Since: header across the wire in a GET request, lighttpd doesn't
do what we want in several cases. If the file hadn't been modified, it
returns a '304 Not Modified' instead of a '206 Partial Content'. We need to
do a stat (e.g. HEAD in HTTP terms) operation here, and the proceed
accordingly based off the values we get back from it.
3. The mtime stuff was rather ugly, and relied on the called function to
write back to a passed in reference, which isn't the greatest. Instead, use
the power of the filesystem to contain this info. Every file downloaded
internally is now carefully timestamped with the remote file time. This
should allow the resume logic to work. In order to guarantee this, we need
to implement a signal handler that catches interrupts, notifies the running
code, and causes it to set the mtimes on the file. It then rethrows the
signal so the pacman signal handler (or any frontend) works as expected.
4. We did a lot of funky stuff in trying to track the DB last modified time.
It is a lot easier to just keep the downloaded DB file around and track the
time on that rather than in a funky dot file. It also kills a lot of code.
5. For GPG verification of the databases down the road, we are going to need
the DB file around for at least a short bit of time anyway, so this gets us
closer to that.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Xav: fixed printf with off_t]
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
* It makes the code clearer to read/understand
* Cppcheck tool doesn't show this anymore: [./util.c:215]: (error) Resource leak: fd
[Dan: don't change the coding style]
Signed-off-by: Dan McGee <dan@archlinux.org>
Thanks to Laszlo Papp <djszapi@archlinux.us> for the following catch:
opendir(path)) == (DIR *)-1;
is maybe the result of misunderstanding the manpage. If an opendir() call
isn't successful it returns NULL rather than '(DIR *)-1'.
Noticed-by: Laszlo Papp <djszapi@archlinux.us>
Signed-off-by: Dan McGee <dan@archlinux.org>
Pacman's fgets function in the API used hardcoded numbers to identify the size.
This is not good practice, so replace them with sizeof handling.
Signed-off-by: Laszlo Papp <djszapi@archlinux.us>
Signed-off-by: Dan McGee <dan@archlinux.org>
Pacman's long option parsing used hardcoded numbers to identify them.
This is not good practice, so replace them with enumeration constants.
Signed-off-by: Laszlo Papp <djszapi@archlinux.us>
Signed-off-by: Dan McGee <dan@archlinux.org>
Yes, it was that bad :P
We still have memleaks left because we cannot free the error data returned
by libalpm, but pacman has the same issue.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Previously we only looked at the first cache directory returned by the
library. This allows us to look at all cache directories for cleaning.
In addition, change the way we do a full (-Scc) cache cleaning operation.
Instead of removing the parent directory, remove each package one-by-one as
in the -Sc case. This would be ideal for someone mounting a cache directory
over NFS, as it ensures we don't wipe out the mountpoint from underneath the
directory.
Signed-off-by: Dan McGee <dan@archlinux.org>
We didn't look at the return status of sync_cleandb() in sync_cleandb_all().
Make it do so and return it up the call chain.
Signed-off-by: Dan McGee <dan@archlinux.org>
Commit 34e1413d75 attempted to implement lazy loading of package databases.
Although it took care of my main complaint (creating the database directory
if it didn't exist), it didn't allow sync repos to be registered before
alpm_option_set_dbpath() had been called.
With this patch, we no longer compute the individual repository DB paths
until necessary, allowing full lazy loading to work as intended, and
allowing us to drop the extra setlibpath() calls from the frontend. This
allows the changes introduced in a2cd48960 (but later reverted) to be added
back in again.
Signed-off-by: Dan McGee <dan@archlinux.org>
After commit 0da96abc, pacman always asks user confirmation for -U, so it is
more coherent to doing that for -R, too.
Btw, most users use -Rs always, so they won't notice any change. In the old
code the -Ru operation was forgotten: Though it is a not "dangerous" operation,
but the target list can be changed by that, too.
Non-interactive scripts should always use --noconfirm (unexpected questions can
be asked by all transactions). [That's why we should always default to the
safest answers.]
I've also added a pkglist != NULL sanity check (because -Ru can empty target
list in trans_prepare part).
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
It was probably a bad idea to modify the target directly in case of
repo/pkg syntax.
Duplicating it also allows us to keep the original target string, which
is more informative when printing errors.
Also remove a duplicated error message from libalpm, and improve the
message already returned to the frontend.
$ pacman -S foo/bar
before
error: repository 'foo' not found
error: 'bar': no such repository
after
error: 'foo/bar': could not find repository for target
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
./src/pacman/package.c:
- small typo fix
./src/pacman/pacman.c:
- strdup is changed to strndup, because it's safer like in
case of config option
Signed-off-by: Laszlo Papp <djszapi2@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Add more untranslated strings, improve consistency, etc.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This doesn't quite work, as can be seen by the pactest results:
Total = 179
Pass = 108 ( 60.34%)
Expected Fail = 5 ( 2.79%)
Unexpected Pass = 0 ( 0.00%)
Fail = 66 ( 36.87%)
If you peek inside '_alpm_db_new' when it gets called for the sync
databases, the base dbpath is still at the default value, causing things
like pactest to fail miserably. We need some further work to do fully lazy
loading, and that belongs on master, not maint.
This reverts commit a2cd48960e.
Signed-off-by: Dan McGee <dan@archlinux.org>
-int alpm_trans_sysupgrade(int enable_downgrade);
-int alpm_trans_sync(char *target);
-int alpm_trans_add(char *target);
-int alpm_trans_remove(char *target);
+int alpm_sync_sysupgrade(int enable_downgrade);
+int alpm_sync_target(char *target);
+int alpm_sync_dbtarget(char *db, char *target);
+int alpm_add_target(char *target);
+int alpm_remove_target(char *target);
* functions renaming
* add new sync_dbtarget which allows to specify the db
* repo/ syntax handling is moved to frontend
( should implement FS#15141)
* group handling is moved to backend
( see http://www.archlinux.org/pipermail/pacman-dev/2009-June/008847.html )
This basically started with this change :
/* Transaction */
struct __pmtrans_t {
- pmtranstype_t type;
pmtransflag_t flags;
pmtransstate_t state;
- alpm_list_t *packages; /* list of (pmpkg_t *) */
+ alpm_list_t *add; /* list of (pmpkg_t *) */
+ alpm_list_t *remove; /* list of (pmpkg_t *) */
And then I have to modify all the code accordingly.
This patch utilizes the power of sync.c to fix FS#3492 and FS#5798.
Now an upgrade transaction is just a sync transaction internally (in alpm),
so all sync features are available with -U as well:
* conflict resolving
* sync dependencies from sync repos
* remove unresolvable targets
See http://www.archlinux.org/pipermail/pacman-dev/2009-June/008725.html
for the concept.
We use "mixed" target list, where PKG_FROM_FILE origin indicates local
package file, PKG_FROM_CACHE indicates sync package. The front-end can add
only one type of packages (depending on transaction type) atm, but if alpm
resolves dependencies for -U, we may get a real mixed trans->packages list.
_alpm_pkg_free_trans() was modified so that it can handle both target types
_alpm_add_prepare() was removed, we use _alpm_sync_prepare() instead
_alpm_add_commit() was renamed to _alpm_upgrade_targets()
sync.c (and deps.c) was modified slightly to handle mixed target lists,
the modifications are straightforward. There is one notable change here: We
don't create new upgrade trans in sync.c, we replace the pkgcache entries
with the loaded package files in the target list (this is a bit hackish) and
call _alpm_upgrade_targets(). This implies a TODO (pkg->origin_data.db is
not accessible anymore), but it doesn't hurt anything with pacman front-end,
so it will be fixed later (otherwise this patch would be huge).
I updated the documentation of -U and I added a new pactest, upgrade090.py,
to test the syncdeps feature of -U.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
When a conflict is detected, pacman asks if the user wants to remove
the conflicting package. In many cases this is a bad idea. e.g.
udev conflicts with initscripts (initscripts<2009.07).
Remove initscripts [Y/n]
This changes the query to [y/N].
The --noconfirm behavior has been also changed, because it chooses the
default answer. Since the yes answer is more interesting in our pactests
dealing with conflicts, I inserted '--ask=4' to all of them with one
exception: sync042.py tests the no answer.
(I also fixed a typo in sync043.py)
Original-work-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
This re-implements the --ask option which was removed in commit
1ff8e7f364.
This option does not have to be exposed to the user (help,doc,etc), but is
very very useful for pactest if we want to have more coverage there.
This was rewritten in a smarter way, without code duplication. And with a
different behavior : this option is now only used to inverse default
behavior to questions.
We still use bit operations based on the following struct :
/* Transaction Conversations (ie, questions) */
typedef enum _pmtransconv_t {
PM_TRANS_CONV_INSTALL_IGNOREPKG = 0x01,
PM_TRANS_CONV_REPLACE_PKG = 0x02,
PM_TRANS_CONV_CONFLICT_PKG = 0x04,
PM_TRANS_CONV_CORRUPTED_PKG = 0x08,
PM_TRANS_CONV_LOCAL_NEWER = 0x10,
PM_TRANS_CONV_REMOVE_PKGS = 0x20,
} pmtransconv_t;
for each conv matched, the default answer is inversed.
--ask 0 : all default answers are preserved
--ask 4 : only conflict question is inversed
--ask 63 : all questions are inversed (63 == 1+2+4+8+16+32)
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Sometimes "foo conflicts with bar" information is not enough, see this
thread: http://bbs.archlinux.org/viewtopic.php?id=77647. That's why I added
a new reason field to our pmconflict_t struct that stores the packager-
defined conflict that induced the fact that package1 conflicts with
package2.
I modified the front-end (in callback.c, sync.c, upgrade.c) to print this
new information as well.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
The main reason for this change is that scripts could not catch the removed
targets with -S --noconfirm (the return value was 0). So the effect of a
pacman command may have differed from the expected one. Moreover, for my
taste the default no answer is better (I wanted to install the specified
targets, not a subset of them).
I had to change some pactest files as well, because now the default behavior
is not to remove unresolvable targets. In fact, the only pactest file that
tested this feature was ignore005.py.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This implements FS#15581
'-Su foo' should be more or less equivalent do '-Su ; -S foo'
Note : I moved a block of code to a new process_target function
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Lazy opening of databases is supported since 34e1413d75.
We don't need that setlibpath call each time we register a database.
Besides this caused a memleak in case setlibpath failed, because setlibpath
exit directly and we did not do the cleanup part (section string was not
freed, and a file descriptor remained open).
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
* just do one malloc call
* p = realloc(p, new_size) was not good
(see http://www.iso-9899.info/wiki/Why_not_realloc)
* use more efficient strncpy instead of strncat
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
this operation was re-implemented using static strings, instead of using the
existing strreplace function
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
For example when we are not in a tty, there is no point in wrapping the
output. This actually makes the job harder for scripts.
$ pacman -Si binutils | grep Desc
Description : A set of programs to assemble and manipulate binary and
The description was cut because the rest was on the following line.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
[Dan: use printf everywhere]
Signed-off-by: Dan McGee <dan@archlinux.org>
similarly to the $repo variable, Server can now contain $arch, which will be
automatically replaced by the appropriate architecture.
This allows us to have one universal mirrorlist file, for both i686 and x86_64,
woohoo!
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
- fix one memleak if get_filename failed
- cleanup according to Joerg's feedback:
"url_for_string: If fetchParseURL returned successful, you should always
have a scheme set. The logic for anonftp should only be needed for very
broken server -- do you know of any such?
download_internal:
Specifying 'p' is now a nop -- it is tried by default first with
fall-back to active FTP."
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
[Dan: remove from pacman.conf and pacman.conf.5]
Signed-off-by: Dan McGee <dan@archlinux.org>
For example, if no package is outdated, -Qu will return 1.
This implements FS#15938
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Skip the SyncFirst dialog, if all the -S packages are SyncFirst packages.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes: FS#15675 - pacman can not determine ownership of dangling
symlinks
Using lstat seems more correct than stat for the -Qo operation anyway.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows a frontend to define its own download algorithm so that the
libfetch dependency can be omitted without using an external process.
The callback will be used when if it is defined, otherwise the old
behavior applies.
Signed-off-by: Sebastian Nowicki <sebnow@gmail.com>
[Dan: minor cleanups]
Signed-off-by: Dan McGee <dan@archlinux.org>
