This is done extremely crudely and is not very efficient, but it does
push us down the path of being closer to right, as one additional test
now passes.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a bit of a mess, due to the fact that we have a progress meter
running. It is also ironic that we are in the midst of a method named
"commit" when we haven't done a damn thing yet, and can still fail hard
if either a checksum or signature is invalid or unrecognized.
Adapt the former test_md5sum method to be invoked for any of the various
failure types, which at least gives the user some indication of what
packages are failing. A second patch will be needed to actually show
worthwhile error codes, but this is going to involve modifying the
actual data passed with the callback.
Signed-off-by: Dan McGee <dan@archlinux.org>
Restore some sanity to the number of arguments passed to _alpm_download
and curl_download_internal.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
This means creating a new struct which can pass more descriptive data
from the back end sync functions to the downloader. In particular, we're
interested in the download size read from the sync DB. When the remote
server reports a size larger than this (via a content-length header),
abort the transfer.
In cases where the size is unknown, we set a hard upper limit of:
* 25MiB for a sync DB
* 16KiB for a signature
For reference, 25MiB is more than twice the size of all of the current
binary repos (with files) combined, and 16KiB is a truly gargantuan
signature.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
URLs might end with a slash and follow redirects, or could be a
generated by a script such as /getpkg.php?id=12345. In both cases, we
may have a better filename that we can write to, taken from either
content-disposition header, or the effective URL.
Specific to the first case, we write to a temporary file of the format
'alpmtmp.XXXXXX', where XXXXXX is randomized by mkstemp(3). Since this
is a randomly generated file, we cannot support resuming and the file is
unlinked in the event of an interrupt.
We also run into the possibility of changing out the filename from under
alpm on a -U operation, so callers of _alpm_download can optionally pass
a pointer to a *char to be filled in by curl_download_internal with the
actual filename we wrote to. Any sync operation will pass a NULL pointer
here, as we rely on specific names for packages from a mirror.
Fixes FS#22645.
Signed-off-by: Dave Reisner <d@falconindy.com>
They are placeholders, but important for things like trying to re-sync a
database missing a signature. By using the alpm_db_validity() method at
the right time, a client can take the appropriate action with these
invalid databases as necessary.
In pacman's case, we disallow just about anything that involves looking
at a sync database outside of an '-Sy' operation (although we do check
the validity immediately after). A few operations are still permitted-
'-Q' ops that don't touch sync databases as well as '-R'.
Signed-off-by: Dan McGee <dan@archlinux.org>
This gives us more granularity than the former Never/Optional/Always
trifecta. The frontend still uses these values temporarily but that will
be changed in a future patch.
* Use 'siglevel' consistenly in method names, 'level' as variable name
* The level becomes an enum bitmask value for flexibility
* Signature check methods now return a array of status codes rather than
a simple integer success/failure value. This allows callers to
determine whether things such as an unknown signature are valid.
* Specific signature error codes mostly disappear in favor of the above
returned status code; pm_errno is now set only to PKG_INVALID_SIG or
DB_INVALID_SIG as appropriate.
Signed-off-by: Dan McGee <dan@archlinux.org>
Using grp instead of group is a small saving at the cost of clarity.
Rename the following functions:
alpm_option_get_ignoregrps -> alpm_option_get_ignoregroups
alpm_option_add_ignoregrp -> alpm_option_add_ignoregroup
alpm_option_set_ignoregrps -> alpm_option_set_ignoregroups
alpm_option_remove_ignoregrp -> alpm_option_remove_ignoregroup
alpm_db_readgrp -> alpm_db_readgroup
alpm_db_get_grpcache -> alpm_db_get_groupcache
alpm_find_grp_pkgs -> alpm_find_group_pkgs
Signed-off-by: Allan McRae <allan@archlinux.org>
We didn't do due diligence before and ensure prior pm_errno values
weren't influencing what happened in further ALPM calls. I observed one
case of early setup code setting pm_errno to PM_ERR_WRONG_ARGS and that
flag persisting the entire time we were calling library code.
Add a new CHECK_HANDLE() macro that does two things: 1) ensures the
handle variable passed to it is non-NULL and 2) clears any existing
pm_errno flag set on the handle. This macro can replace many places we
used the ASSERT(handle != NULL, ...) pattern before.
Several other other places only need a simple 'set to zero' of the
pm_errno field.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows callers to retrieve it from wherever is convenient, which
may or may not be on the package object itself.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the last user of our global handle object. Once again the diff
is large but the functional changes are not.
Signed-off-by: Dan McGee <dan@archlinux.org>
This requires a lot of line changes, but not many functional changes as
more often than not our handle variable is already available in some
fashion.
Signed-off-by: Dan McGee <dan@archlinux.org>
This kills a lot more global handle business off. sync.c still requires
the handle declaration for one reference that can't be changed yet; it
will be removed in a future patch which isolates all of the necesary API
changes.
Signed-off-by: Dan McGee <dan@archlinux.org>
This will make the patching process less invasive as we start to remove
this variable from all source files.
Signed-off-by: Dan McGee <dan@archlinux.org>
The usefulness of this is rather limited due to it not being compiled
into production builds. When you do choose to see the output, it is
often overwhelming and not helpful. The best bet is to use a debugger
and/or well-placed fprintf() statements.
Signed-off-by: Dan McGee <dan@archlinux.org>
We were erroring out in the case where a first (possibly bogus) mirror
would cause the download process to return a failure code, even though
subsequent servers had the file.
Signed-off-by: Dan McGee <dan@archlinux.org>
Both md5sum verification and PGP verification can and should be done at
package load time. This allows verification to happen as early as
possible for packages provided by filename and loaded in the frontend,
and moves more stuff out of sync_commit that doesn't really belong
there. This should also set the stage for simplified parallel loading of
packages later down the road.
Signed-off-by: Dan McGee <dan@archlinux.org>
More stuff going on in the pre-committing stage that can be in a static
method to make things a bit more clear.
Signed-off-by: Dan McGee <dan@archlinux.org>
This part is almost completely self-contained, except building the list
of delta filenames that we use later to check their md5sums. Refactor it
into a static method so we can bring most of the code in sync_commit
closer to the method name.
Signed-off-by: Dan McGee <dan@archlinux.org>
Given that we offer no transparency into the pmpgpsig_t type, we don't
really need to expose it outside of the library, and at this point, we
don't need it at all. Don't decode anything except when checking
signatures. For packages/files not from a sync database, we now just
read the signature file directly anyway.
Also push the decoding logic down further into the check method so we
don't need this hanging out in a less than ideal place. This will make
it easier to conditionally compile things down the road.
Signed-off-by: Dan McGee <dan@archlinux.org>
There's a lot of related moving parts here:
* Iteration through mirrors is moved back to the calling functions. This
allows removal of _alpm_download_single_file and _alpm_download_files.
* The download function gets a few more arguments to influence behavior.
This allows several different scenarios to customize behavior:
- database
- database signature (req'd and optional)
- package
- package via direct URL
- package signature via direct URL (req'd and optional)
* For databases, we need signatures from the same mirror, so structure
the code accordingly.
Some-inspiration-from: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
The value PM_PGP_VERIFY_UNKNOWN is reserved to error cases,
now that the signature verification level defaults to the
globally set level. The only error case is when handle == NULL,
which is false in the context of _alpm_sync_commit().
Signed-off-by: Rémy Oudompheng <remy@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This does touch a lot of things, and hopefully doesn't break things on
other platforms, but allows us to also clean up a bunch of crud that no
longer needs to be there.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the standard, and we have had a few of these introduced lately
that should not be here.
Done with:
find -name '*.c' | xargs sed -i -e 's#if (#if(#g'
find -name '*.c' | xargs sed -i -e 's#while (#while(#g'
Signed-off-by: Dan McGee <dan@archlinux.org>
* add _alpm_db_get_sigverify_level
* add alpm_option_{get,set}_default_sigverify
And set the default verification level to OPTIONAL if not set otherwise.
Signed-off-by: Dave Reisner <d@falconindy.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Once we do this, add support for VerifySig to pactest. We just check if
the repo name contains Always, Never or Optional to determine the value
of VerifySig. The default is Never. pacman uses Always by default but
this is not suitable for pactest.
Original-work-by: shankar <jatheendra@gmail.com>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Wrap lines of long length, noticed while creating and messing around
with some of the other maint branch patches.
Signed-off-by: Dan McGee <dan@archlinux.org>
Noticed with the openoffice/libreoffice replacement scheme where many
packages are listed as replacements to one package, thus electing it for
removal multiple times. Ensure a given package is not already present
before placing it in the removal list.
Signed-off-by: Dan McGee <dan@archlinux.org>
This was discussed and more or less agreed upon on the mailing list. A
huge checkin, but if we just do it and let people adjust the pain will
end soon enough. Rebasing should be relatively straighforward for anyone
that sees conflicts; just be sure you use the new return style if
possible.
The following semantic patch was used to do the change, along with some
hand-massaging in order to preserve parenthesis where appropriate:
The semantic match that finds this problem is as follows, although some
hand-massaging was done in order to keep parenthesis where appropriate:
(http://coccinelle.lip6.fr/)
// <smpl>
@@
expression a;
@@
- return(a);
+ return a;
// </smpl>
A macros_file was also provided with the following content:
Additional steps taken, mainly for ASSERT() macros:
$ sed -i -e 's#return(NULL)#return NULL#' lib/libalpm/*.c
$ sed -i -e 's#return(-1)#return -1#' lib/libalpm/*.c
Signed-off-by: Dan McGee <dan@archlinux.org>
A lot of these were places that should have used the same message but
didn't, or were very easy to convert to using the same message and
letting some of the burden off of the translators.
Signed-off-by: Dan McGee <dan@archlinux.org>
Read the package information for sync/local databases into a pmpkghash_t
structure.
Provide a alpm_db_get_pkgcache_list() method that returns the list from
the hash object. Most usages of alpm_db_get_pkgcache are converted to
this at this stage for ease of implementation. Review whether these are
better accessing the hash table directly at a later stage.
Signed-off-by: Allan McRae <allan@archlinux.org>
It's likely that these interfaces will break sooner or later, now that
pacman no longer uses them.
So better force the two people who use them to migrate their code to the
new add_pkg/remove_pkg interface, which is very easy anyway.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
This group function is meant to help group handling from frontend : it
scans all dbs, handling ignored packages and duplicate members (the
first repo where a member is found has the priority).
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
When installing packages from a file, the integrity check count
stays at (0/x) complete. This ensures it is bumped to (x/x) at
the end of the process.
Signed-off-by: Dan McGee <dan@archlinux.org>
This can take a while too, and it is really easy to add the necessary
callback stuff for adding a progressbar.
Signed-off-by: Dan McGee <dan@archlinux.org>
There is a lot of swtiching between size_t and int for alpm_list sizes
in the codebase. Start converting these to all be size_t by adjusting
the return type of alpm_list_count and fixing all additional warnings
given by -Wconversion that are generated by this change.
Dan: a few more small changes to ensure things compile, adjusting some
printf format string characters to accommodate the larger size on x86_64.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
POSIX does not require PATH_MAX be defined when there is not actual
limit to its value. This affects HURD based systems. Work around
this by defining PATH_MAX to 4096 (as on Linux) when this is not
defined.
Also, clean up inclusions of limits.h and remove autoconf check for
this header as we do not use macro shields for its inclusion anyway.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Checking disk space needed for a transaction can take a while so add
an informative progress bar.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Disk space checking is likely to be an unnecessary bottleneck to
people with reasonable partition sizes so add a configuration option
to allow it to be disabled/enabled as wanted.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Very basic prototyping for adding functionality to check free disk
space before performing package installs.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We were including the header in a lot of places it is no longer used.
Additionally, use the correct autoconf macro for determining whether
d_type is available as a member: HAVE_STRUCT_DIRENT_D_TYPE.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Either we expose all low levels function dealing with pmdepend_t
(splitdep and depfree come to mind), or we don't.
Since none of the tools use depcmp, I chose to remove it. In the future,
we might want to expose higher level functions such as
alpm_find_satisfier, or just lower level functions like splitdep and
depfree together with depcmp.
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Cache bullshit only has relevance to be_files, so move it there.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: BIG rebase]
Signed-off-by: Allan McRae <allan@archlinux.org>
Implement this seemingly simple change in package.h:
typedef enum _pmpkgfrom_t {
- PKG_FROM_CACHE = 1,
- PKG_FROM_FILE
+ PKG_FROM_FILE = 1,
+ PKG_FROM_LOCALDB,
+ PKG_FROM_SYNCDB
} pmpkgfrom_t;
which requires flushing out several assumptions from around the codebase
with regards to usage of the PKG_FROM_CACHE value. Make some changes where
required to allow the switch, and now the correct value should be set (via a
crude hack) depending on whether a package was loaded as an entry in a local
db or a sync db.
This patch underwent some big rebasing from Allan and Dan.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
We no longer use these anywhere outside of sync.c, so do the rename and add
static to their definition to meet our coding standards.
Signed-off-by: Dan McGee <dan@archlinux.org>
As reported in FS#20221, we don't always do the right thing when installing
a group and using the --needed option. This was due to the code pulling
packages based on what was already in the transaction's add list, but
completely ignoring the fact that we may have already seen and skipped this
same package in an earlier repository.
Add a list to the private _alpm_sync_pkg() function that allows us to have
this extra information so we don't mistakenly downgrade a package when using
--needed.
Signed-off-by: Dan McGee <dan@archlinux.org>
Rather than say we can't find the target after saying "No, I guess I don't
want to install this", we should make sure the ignored status gets passed
all the way through. This fixes FS#19866.
Pactest is also included that failed before due to the fact that we normally
treat an unfound package as a reason to exit with a non-zero status.
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch fixes the phonon/qt issue, if all to-be-upgraded packages are
explicit targets (ie. only not-yet-installed packages are pulled by
resolvedeps). This condition covers the most common situations, for example
it should hold with every -Su operation.
After this patch sync405.py passes, but sync406.py doesn't.
The work is inspired by the patch of Henning Garus, thanks for his work:
http://mailman.archlinux.org/pipermail/pacman-dev/2010-February/010429.html
(I moved the alpm_list_diff computation to sync.c in order to compute it
only once.)
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
After commit df99495b82 pacman downloaded files from the first repo only,
and reported corrupted packages for all files from other repos.
The download_size was set to 0 for _all_ transaction packages after
downloading some files from the first repo. This code-block was moved to its
correct place.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
If the -d switch was invoked with -S (or -U), the removes list was simply
lost, because trans->remove was computed in an
"if(!(trans->flags & PM_TRANS_FLAG_NODEPS))" block.
I've added a new pactest file, sync045.py (derived from sync043.py) to test
this.
Additionally, I did some other minor cleanups in sync_prepare:
* preferred list is not needed anymore
* I removed a needless alpm_list_remove_dupes line (the target list should
not contain dupes at all)
* I moved alpm_list_free(remove); to cleanup part to eliminate a possible
memleak
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
It was probably a bad idea to modify the target directly in case of
repo/pkg syntax.
Duplicating it also allows us to keep the original target string, which
is more informative when printing errors.
Also remove a duplicated error message from libalpm, and improve the
message already returned to the frontend.
$ pacman -S foo/bar
before
error: repository 'foo' not found
error: 'bar': no such repository
after
error: 'foo/bar': could not find repository for target
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
-int alpm_trans_sysupgrade(int enable_downgrade);
-int alpm_trans_sync(char *target);
-int alpm_trans_add(char *target);
-int alpm_trans_remove(char *target);
+int alpm_sync_sysupgrade(int enable_downgrade);
+int alpm_sync_target(char *target);
+int alpm_sync_dbtarget(char *db, char *target);
+int alpm_add_target(char *target);
+int alpm_remove_target(char *target);
* functions renaming
* add new sync_dbtarget which allows to specify the db
* repo/ syntax handling is moved to frontend
( should implement FS#15141)
* group handling is moved to backend
( see http://www.archlinux.org/pipermail/pacman-dev/2009-June/008847.html )
This basically started with this change :
/* Transaction */
struct __pmtrans_t {
- pmtranstype_t type;
pmtransflag_t flags;
pmtransstate_t state;
- alpm_list_t *packages; /* list of (pmpkg_t *) */
+ alpm_list_t *add; /* list of (pmpkg_t *) */
+ alpm_list_t *remove; /* list of (pmpkg_t *) */
And then I have to modify all the code accordingly.
This patch utilizes the power of sync.c to fix FS#3492 and FS#5798.
Now an upgrade transaction is just a sync transaction internally (in alpm),
so all sync features are available with -U as well:
* conflict resolving
* sync dependencies from sync repos
* remove unresolvable targets
See http://www.archlinux.org/pipermail/pacman-dev/2009-June/008725.html
for the concept.
We use "mixed" target list, where PKG_FROM_FILE origin indicates local
package file, PKG_FROM_CACHE indicates sync package. The front-end can add
only one type of packages (depending on transaction type) atm, but if alpm
resolves dependencies for -U, we may get a real mixed trans->packages list.
_alpm_pkg_free_trans() was modified so that it can handle both target types
_alpm_add_prepare() was removed, we use _alpm_sync_prepare() instead
_alpm_add_commit() was renamed to _alpm_upgrade_targets()
sync.c (and deps.c) was modified slightly to handle mixed target lists,
the modifications are straightforward. There is one notable change here: We
don't create new upgrade trans in sync.c, we replace the pkgcache entries
with the loaded package files in the target list (this is a bit hackish) and
call _alpm_upgrade_targets(). This implies a TODO (pkg->origin_data.db is
not accessible anymore), but it doesn't hurt anything with pacman front-end,
so it will be fixed later (otherwise this patch would be huge).
I updated the documentation of -U and I added a new pactest, upgrade090.py,
to test the syncdeps feature of -U.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Sometimes "foo conflicts with bar" information is not enough, see this
thread: http://bbs.archlinux.org/viewtopic.php?id=77647. That's why I added
a new reason field to our pmconflict_t struct that stores the packager-
defined conflict that induced the fact that package1 conflicts with
package2.
I modified the front-end (in callback.c, sync.c, upgrade.c) to print this
new information as well.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
If the user switches from unstable repo to a stable one, it is quite hard to
sync its system with the new repo (the user will see many "Local is newer
than stable" messages, nothing more). That's why I introduced -Suu, which
treats a sync package like an upgrade, iff the package version doesn't match
with the local one's.
I added a new pactest (sync104.py) to test this, and I updated the
documentation of -Su.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
[Dan: slight doc reword]
Signed-off-by: Dan McGee <dan@archlinux.org>
$ sudo pacman -S mc
Old output:
***********
:: mc conflicts with mc-mp. Remove mc-mp? [Y/n] y
...
(1/1) checking for file conflicts [################] 100%
(1/1) installing mc [################] 100%
New output:
***********
:: mc conflicts with mc-mp. Remove mc-mp? [Y/n] y
...
(1/1) checking for file conflicts [################] 100%
(1/1) removing mc-mp [################] 100%
(1/1) installing mc [################] 100%
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes FS#14899. When running an -Sp operation without servers
configured for a repository, we would segfault, so add an assert to the
backend method returning the first server preventing a null pointer
dereference.
In addition, add a new error code to libalpm that indicates we have no
servers configured for a repository. This makes -Sy and -S <package>
operations fail gracefully and helpfully when a repo is set up with no
servers, as the default mirrorlist in Arch is provided this way.
Signed-off-by: Dan McGee <dan@archlinux.org>
The main purpose of this function to make our code more readable.
It frees transaction specific fields of pmpkg_t. (It is used when a package
is removed from the target list.)
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
There is apparently no need to handle the re-compression manually when
applying a xdelta patch in case of bzip2 or xz.
Only gzip needs to be handled specifically for disabling timestamp with the
-n option.
After this patch, if xdelta is enhanced with xz support (1-line patch), it
will be transparent from pacman side.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
"Foo replaces bar" simply means that "foo is a new version of bar".
So this patch refactors the code to use this "rule".
_alpm_sync_sysupgrade now does the following for each local package [bar]
(pseudo-code):
for db in syncdbs {
if(db contains bar) {check if db/bar is an upgrade; break;}
replacers = find(bar replacers in db);
if(replacers!=NULL) {ask_user; break;}
}
Note:
1. Replacers are grouped per repo. If more than one package replace bar in
a repo, all of them are considered ("package set of bar replacers").
2. If repo1/foo1 and repo2/foo2 both replaces bar, only repo1/foo1 is
considered (if repo1 stands before repo2 in pacman.conf). FS#11737 is fixed.
3. It can happen that pacman doesn't consider any replacer, if it found a
literal "earlier", so sync132.py modified accordingly (btw, that situation
should not appear irl).
The new sysupgrade code doesn't use sync_newversion(), so I removed the
"local is newer than repo" message, which was annoying with -Qu and
SyncFirst.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes a bug introduced by my previous changes which changes the
behavior of IgnorePkg/IgnoreGroup to allow the user to remove unresolvable
packages from the transaction. The bug is that the target-list was no
longer being consulted first to resolve dependencies, which means that if
two packages in the sync database satisfied a dependency, and the user
explicitly requested one of those two packages in the sync, the other
package was still being pulled in.
A new test was added, sync993.py, to verify the desired behavior.
Signed-off-by: Bryan Ischo <bji-keyword-pacman.3644cb@www.ischo.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
pmsyncpkg_t data sructure was removed:
1. pmpkg_t.reason is used instead of pmsyncpkg_t.newreason. (The target
packages come from sync repos, so we can use this field without any
problems. Upgrade transaction also uses this field to store this info.)
2. pmsyncpkg_t.removes was moved to pmpkg_t.removes.
This step requires careful programming, because we don't duplicate packages
when we add them to trans->packages. So we modify sync pkgcache when we
add this transaction-only info to our package. Hence it is important to
free this list when we remove any package from the target list
(remove_unresolvable, remove_conflicts, trans_free), otherwise this could
confuse the new sync transactions (with non-pacman GUI).
Overall, our code became ~100 line shorter, and we can call our helper
functions directly on trans->packages in sync.c, we don't need to maintain
parallel package lists.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
From now on _alpm_db_find_fileconflicts() works with upgrade and remove
target lists (like checkdeps), which makes it transaction independent
(we still need a trans param because of the progressbar). This is a small
step towards the universal transaction. So we call this function directly
from sync.c before commiting the remove transaction. This is much safer,
but we can get false fileconflict error alarms in some tricky cases
("symlinks puzzle" etc).
The patch on find_fileconflict looks complex, but it is mainly an
"indent-patch", the new code-part can be found after the
/* check remove list ... */ comment, and I modified something around the
"file has changed hand" case (see comment modifications in the code).
Unfortunately sync.c became more ugly, because we have to create 2 parallel
internal transactions: to avoid duplicated work, upgrade transaction is
used to load package data (filelists). This problem will disappear, when
we finally get rid of internal transactions.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Resolvedeps reports error when it cannot resolve some dependencies, puts
them into the *data list, and set pm_errno. If user removes the
unresolvable packages from the target list, we have no error anymore,
so from now on we free *data list (we eliminate a memleak) and unset
pm_errno.
(Additionally I removed two needless lines from the code, unresolvable list
is always freed in cleanup.)
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch fixes FS#12059.
Now sync_addtarget can return with PM_ERR_PKG_IGNORED, which indicates that
although the requested package was found it is in ignorepkg, so alpm could
not add it to the transaction. So the front-end can decide what to do.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
1) The changes to sync.c look big but there are mostly caused by
the indentation. Fix a bug where download_size == 0 because the packages and
deltas are already in the cache, but we still need to build the deltas list
and apply the deltas to create the final package.
2) Fix the gzip / md5sum issue by switching to xdelta3, disabling external
recompression and using gzip -n in pacman, and disable bsdtar compression
and using gzip -n in makepkg.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
The from_md5 and to_md5 fields were a nice extra safety, which would avoid
trying to apply deltas on corrupted package files. However, they are not
strictly necessary, since xdelta should be able to detect that on its own.
The main problem is that it is impossible to compute these informations from
the delta only. So repo-add would not be able to compute the delta entry
based on just the delta file.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Don't prompt the user for unignore of IgnorePkg/IgnoreGroup packages,
except for packages explicitly listed for sync by the user. This
eliminates many unnecessary prompts when IgnorePkg/IgnoreGroup is
used.
Signed-off-by: Bryan Ischo <bryan@ischo.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Enabled a new prompt to ask the user if they'd like to remove
unresolvable packages from the transaction rather than failing it.
Many pactest tests that used to fail now return success codes, because
pacman now issues a prompt allowing the user to cancel rather than
failing many transactions, and the pactest scripts always choose to
cancel with no error rather than failing. The only net effect is that
the return status of pacman is now 0 in cases where it used to be
nonzero.
Signed-off-by: Bryan Ischo <bryan@ischo.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This change reorganizes the internal code so that packages are
resolved one at a time instead of all at once from a list. This will
allow a future checkin to prompt the user to see if they'd rather
remove unresolvable packages from the transaction and continue, or
fail the transaction. This change does not affect the actual behavior
of libalpm and all tests pass without changes.
Signed-off-by: Bryan Ischo <bryan@ischo.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
From now on -Qu is an "outdated package" filter on local database.
(This is a behaviour change.)
This patch fixes some memleaks and makes the code cleaner, for details see
my comment on FS#7884.
FS#11868 is implemented.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
These two functions now take directly a package list rather than a database.
checkdbconflicts was renamed to checkconflicts.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
This patch kills one of our hackish pseudo transactions: PRINTURIS.
(The other one is -Sw)
From now on, front-end must not call trans_commit in case of -Sp,
it should print the uris of target packages "by hand" instead.
PRINTURIS flag was removed, NOCONFLICTS flag can be passed to skip
conflict checks.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
* Change the return values to be more informative.
It was previously boolean, only indicating if a sync package was newer than
a local package.
Now it is a simple wrapper to vercmp, handling the force flag.
* Remove the verbose output from _alpm_pkg_compare_versions.
The "force" message is not so useful.
The "package : local (v1) is newer than repo (v2)" message can be moved to
-Su operation.
For the -S operation, it is better to have something like :
"downgrading package from v1 to v2"
* Don't display the "up to date -- skipping" and "up to date -- reinstalling"
messages, when the local version is newer than the sync one.
* Fix the behavior of --needed option to not skip a target when the local
version is newer, and clarify its description.
* Add a new alpm_pkg_has_force function
This allows us to access the pkg->force field like any other package fields.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Debug messages were removed from _alpm_sync_find, because it is a general
purpose function; debug messages should be placed in the caller function.
I inserted "adding package foo-1.0-1 to the transaction targets" debug
message to find_replacements and sync_sysupgrade.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Now '-S provision' handling is done in the back-end.
In case of multiple providers, the first one is selected (behavior change:
deleted provision002.py). The old processing order was: literal, group,
provision; the new one: literal, provision, group. This is more rational,
but "pacman -S group" will be slower now. "pacman -S repo/provision" also
works. Provision was generalized to dependencies, so you can resolve deps by
hand: "pacman -S 'bash>2.0'" or "pacman -S 'core/bash>2.0'" etc. This can be
useful in makepkg dependency resolving. The changes were documented in
pacman manual.
alpm_find_pkg_satisfiers and _alpm_find_dep_satisfiers functions were
removed, since they are no longer needed.
I added some verbosity to "select provider instead of literal" and
"fallback to group".
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
I divided resolvedeps into 2 functions. The new _alpm_resolvedep function
will resolve one dependency, for example the 'foo>=1.0-1' dependency. It
can be useful in sync_addtarget refactoring.
The resolvedeps parameters were changed, to be coherent with recursedeps:
* the target-list is an alpm_list* instead of alpm_list**. This is OK,
because alpm_list_add == alpm_list_add_last
* syncpkg param was removed. list contains the to-be-installed packages,
resolvedeps will add the required dependencies into this list
* trans param was removed, it was used in QUESTION() only, which can be used
on the main (handle->trans) transaction only (because the front-end cannot
access our pseudo-transactions at all!).
The patch fixes some wrong dynamic pmdepmissing_t usage.
I did a behavior change (and sync1003.py was modified accordingly), which
needs some explanation: The old resolvedeps didn't elect packages from
'remove' list. I've dropped this because I don't want that 2nd excluding
list param. In fact, in real life, we ~never need this rule. Resolvedeps is
called before checkconflicts, so only -Su's %REPLACES% packages are sitting
in 'remove' list. This means, that we have the replacement packages in our
target list. Usually "foo replaces bar" means, that bar isn't in our repos
any more, so resolvedeps *cannot* elect it; but usually it won't try it at
all, because foo is in the target list, and it is expected to satisfy
'bar>=1.0-1'-like dependencies too. Since checkdeps and checkconflicts is
done after resolvedeps, this cannot cause any harm.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Add a new totaldlcb callback function to libalpm and make pacman utilize it
when the TotalDownload option is enabled. This callback function is pretty
simple- it is meant to be called once at the beginning of a "list download"
action, and once at the end (with value 0 to indicate the list has been
finished). The frontend is responsible for keeping track of adding
individual file download amounts to the total xfered amount in order to
display some sort of overall progress.
Signed-off-by: Dan McGee <dan@archlinux.org>
We have been using unsigned long as a file size type for a while, which
works but isn't quite correct and could easily break. Worse was probably our
use of int in the download callback functions, which could be restrictive
for packages > 2GB in size.
Switch all file size variables to use off_t, which is the preferred type for
file sizes. Note that at least on Linux, all applications compiled against
libalpm must now be sure to use large file support, where _FILE_OFFSET_BITS
is defined to be 64 or there will be some weird issues that crop up.
Signed-off-by: Dan McGee <dan@archlinux.org>
* remove obsolete and unused *_cmp helper functions like deppkg_cmp and
_alpm_grp_cmp
* new alpm_list_remove_str function, used 6 times in handle.c
* remove _alpm_prov_cmp / _alpm_db_whatprovides and replace them by
a more general alpm_find_pkg_satisfiers with a cleaner implementation.
before: alpm_db_whatprovides(db, targ)
after: alpm_find_pkg_satisfiers(alpm_db_getpkgcache(db), targ)
* remove satisfycmp and replace alpm_list_find + satisfycmp usage by
_alpm_find_dep_satisfiers.
before : alpm_list_find(_alpm_db_get_pkgcache(db), dep, satisfycmp)
after : _alpm_find_dep_satisfiers(_alpm_db_get_pkgcache(db), dep)
* remove _alpm_pkgname_pkg_cmp, which was used with alpm_list_remove, and
use _alpm_pkg_find + alpm_list_remove with _alpm_pkg_cmp instead.
This commit actually get rids of all complicated and asymmetric _cmp
functions. I first thought these functions were worth it, be caused it
allowed us to reuse list_find and list_remove. But this was at the detriment
of the clarity and also the ease of use of these functions, dangerous
because of their asymmetricity.
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Now the syntax is coherent with alpm_list_find and alpm_sync_find.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
We removed one too many FREELIST() calls when trying to fix some memleaks,
and add a safety/sanity check to ensure filename is set, as packages in old
DBs are likely to not have this field.
Signed-off-by: Dan McGee <dan@archlinux.org>
Reference : FS#9547.
The get_filename function first tries to get the filename field from the
database, and if it doesn't find it, it tries to guess it based on the name,
version and arch.
This field was introduced in 3.0, but there are still many old entries in
the official databases without it. So the databases need to be regenerated
first before this patch can be applied.
There is a second problem with the delta code, which needs the filename for
locally installed packages too, but this field is not present in the local
db. So the delta code needs to be fixed first.
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
As Nathan noticed, the new informations in the delta struct allows us to
get rid of this list :
http://www.archlinux.org/pipermail/pacman-dev/2008-February/011163.html
So I rewrote apply_deltas for that. The previous apply_deltas also had a
limitation: it assumed that the initial package and the deltas were in the
first cache dir, which is not necessarily the case. That situation is
supported now.
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Using the graph structures that Nagy set up for dependency sorting, we now
do a similar process for deltas. Load up all of the deltas into a graph
object on which we can then apply Dijkstra's algorithm, using the new weight
field of graph struct.
We initialize the nodes weight using the base files that we can use in our
filecache (both filename and md5sum must match). The algorithm then picks
the best path among those that can be resolved.
Note that this algorithm has a few advantages over the old one:
1. It is completely file agnostic. These delta chains do not have to consist
of package files- this could be adopted to do delta-fied DBs.
2. It does not use the local_db anymore, or even care if a package or file
is currently installed. Instead, it only looks in the filecache for files
and packages that match delta chain entries.
Original-work-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
Start to move the delta struct away from an assumed package name scheme and
towards something that is package (or even filename) agnostic. This will
allow us much greater flexibility in the usage of deltas (maybe even sync
DBs some day) as well as allowing code outside of delta.h/delta.c to be much
cleaner with less of a need for snprintf() calls.
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch should avoid duplicated target names in the backend.
1. sync_loadtarget will return with PM_ERR_TRANS_DUP_TARGET when trying to
add a duplicated target
2. sysupgrade never pulls duplicated targets
3. resolvedeps won't pull duplicated targets anymore
A pulled list was introduced in sync_prepare to improve the
pmsyncpkg_t<->pmpkg_t list conversion by making it more direct.
Also replace sync1005 and sync1006 by the sync1008 pactest, which is
similar but more interesting (the provisions are dependencies instead of
explicit targets).
sync1005 didn't work as expected anyway. It was expecting that pacman
failed, and pacman indeed failed, but not for the good reason. It didn't
fail during the preparation step because of conflicting targets, but during
the commit step, because of a md5 error...
And sync1006 didn't pass and was not really worth fixing. We have already
enough failing pactests more important than these two.
sync1008 pass with this patch.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
Its implementation was quite broken:
* add_loadtarget() might have silently filtered out some targets when
replacing an older version.
* This was used in sync.c to determine whether a target is implicit or not,
which is incorrect behavior. Before this patch we silently removed user
confirmed replacements; now we always warn on a replacement.
* remove001.py behavior was quite odd in adding same target 5 times to the
target list, we can change this behavior to be a failure.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
[Xav: changed remove001 pactest accordingly]
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
[Dan: rewrote commit message]
Signed-off-by: Dan McGee <dan@archlinux.org>
test_delta_md5sum and test_pkg_md5sum were simple wrappers to test_md5sum,
and only used once, so not very useful. I removed them.
Also, test_md5sum and alpm_pkg_checkmd5sum functions were a bit duplicated,
so I refactored them with a new _alpm_test_md5sum function in libalpm/util.c
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
Remove what was a pretty weird abstraction in the libalpm backend. Instead
of parsing server URLs as we get them (of which we don't usually use more
than a handful anyway), wait until they are actually used, which allows us
to store them as a simple string list instead. This allows us to remove a
lot of code, and will greatly simplify the continuing refactoring of the
download code.
Signed-off-by: Dan McGee <dan@archlinux.org>
Add new stub functions that work by calling the existing (terrible) download
forreal function, which needs a serious overhaul. Hide the existing
functions and switch all former users to the new functions.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the first in what will be a series of patches to clean up the
current download code in libalpm. Start by moving download code out of
server.c and into download.c.
Signed-off-by: Dan McGee <dan@archlinux.org>
