Detect a conflict between a file/symlink in one package and a directory
in another when both are being installed at once.
A side effect is the creation of conflicts between a directory symlink
and a real directory (e.g lib -> usr/lib in pkg1 and /lib in pkg2).
Given we can not guarantee pkg1 is installed before pkg2, this is a
genuine conflict.
Signed-off-by: Allan McRae <allan@archlinux.org>
It turns out when you set the filelist for a package to include
"usr/lib/foo" in the pactest suite, it thinks there is only the
file "usr/lib/foo" in there... No "usr/" or "usr/lib/" directory.
This makes life difficult when testing code that scrolls through
a filelist looking for directory entries.
Signed-off-by: Allan McRae <allan@archlinux.org>
When using the --valgrind flag with the pactest.py script, the path
to the suppression file relies on the script being called from the
source root directory. Construct the path from the scripts location
to allow it to be called from directory.
Dan: style cleanup.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
If a pacakge has a directory symlink (e.g. /lib -> usr/lib), do not
allow it to be removed if any other package is trying to install a
file into that path (e.g. /lib/foo). This is because the local
database can become invalidated if the symlink is removed after the
package with file /lib/foo is installed (sync702.py). If the symlink
is removed before the file is installed (sync701.py), the upgrade is
actually a success, but we can not guarantee the ordering so both
cases should fail.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
If we have a symlink to a directory (e.g. /lib -> usr/lib), then
we can not remove it if a local package thinks it has files in the
symlink directory (e.g. /lib/foo), because this will invalidate the
local file database.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Note failure to detect a conflict between files having the same
effective path across packages due to a directory symlink.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Note failure to detect a conflict between a directory in one package
and a file in the other when the directory is not currently on the
filesystem.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
The wrong test file was inadvertantly committed in 44e9fdd0. Add the
correct test and tidy up the test which was committed.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
When checking if a package owns a directory, it is important to check
not only that all the files in the directory are part of the package,
but also if the directory is part of a package. This catches empty
subdirectories during conflict checking for directory to file/symlink
replacements.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
When two packages own an empty directory, pacman finds no conflict when
one of those packages wants to replace the directory with a file or a
symlink. When it comes to actually extracting the new file/symlink,
pacman sees the directory is still there (we do not remove empty
directories if they are owned by a package) and refuses to extract.
Detect this potential conflict early and bail. Note that it is a
_potential_ conflict and not a guaranteed one as the other package owning
the directory could be updated or removed first which would remove
the conflict. However, pacman currently can not sort package installation
order to ensure this, so this conflict requires manual upgrade ordering.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a bash wrapper around an awk function that parses human readable
sizes and returns their representative values in bytes, as a string. A
small test harness is added to validate the functionality.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Check for an exact match when querying ownership of files in the root.
Previously, our test was too simple and would match the the basename of
package files against the query parameter, e.g.
$ pacman -Qo config
/config is owned by cower-git 20120614-1
Adds a new test to verify this behavior, query007.py.
Fixes FS#30388.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This will replace our current options parser used in pacman-key,
makepkg, and ideally elsewhere. It follows heuristics closer to that of
GNU getopt long (and thus pacman itself), with the exception that it
does not allow for options with optional arguments. Due to the way this
parser will be used, this sort of functionality will not be needed.
Instead of relying on eval+set, options are normalized into an array,
OPTRET, which callers should expect to be populated after returning from
parseopts. This avoids problems with quotes and spaces in arguments,
assuming that the user quotes properly when passing into the
application.
A new test harness for parseopts is added in test/scripts.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
This has outlived its usefulness and causes more problems than it
solves. It has historically only ever been used to install pacman first.
That should not be needed given we provide the vercmp utility (which has
no library dependencies) and so calling pacman in install scripts is a
sign of poor packaging.
Work-duplicated-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Python PEP-394 states that all python code should point at the
python2 or python3 symlinks at maintain cross-distro compatibility.
Note that this does not matter when calling these scripts using
"make check" as they are explictly called using the detected python
version. As this only affects manually calling these scripts, I
have not had configure/make replace the shebangs.
Signed-off-by: Allan McRae <allan@archlinux.org>
This reverts commit f3fa77bcf1 along with
making other necessary changes to fully back this (mis)feature out until
we can do it correctly.
The quick summary here is this was not implemented correctly; provides
are not fully taken into account in this logic, and making that happen
exposes a lot of other flaws in this code that are covered up later on
in the dependency resolving process by several other pieces of
convoluted and conditional logic.
Tests have been adjusted accordingly. Some test EXISTS conditions have
been removed as we already know the package is installed locally, and we
also are checking the VERSION condition anyway.
With these two related revert commits, we do have some changes in test
pass/fail results:
* upgrade078.py: does not pass, this is due to --recursive getting
removed for -U/-S operations after this commit.
* sync302.py: the version checks have been disabled, so this test
continues to pass but has been scaled back in scope.
* sync303.py: now passes, was failing before.
* sync304.py: still failing, was failing before.
* sync305.py: now passes, was failing before.
* sync306.py: still passes, was passing before.
Signed-off-by: Dan McGee <dan@archlinux.org>
This reverts commit 0903452032.
Tests affected by this revert have been adjusted; additionally a few
EXIST tests have been removed where there is already a VERSION test
doing the job for us.
Signed-off-by: Dan McGee <dan@archlinux.org>
Add 1 failing for the -Su case, and the same case using -S (and
passing).
This is based on a real (current) issue of upgrading staging chroots
with the new pacman in staging for a libarchive build, and a new
toolchain in testing.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This converts our script generation to use the built-in AM_V_GEN macro,
which honors the V= setting passed to make and allows one to see the
full command if they truly desire. The AM_V_at macro is also used in
place of an explicit @ so verbose-mode compiles show all commands being
run.
We can also use these two macros in doc generation to quiet it down to
the level we expect.
Other minor changes:
* a pointless test call is removed in test/pacman/tests/
* sed is used instead of dos2unix as we depend on it anyway
* consecutive chmod calls are reduced to a single call (e.g., '+x,a-x')
Signed-off-by: Dan McGee <dan@archlinux.org>
Both currently marked as failing.
* sync303.py encapsulates the broken behavior reported in FS#27214.
* sync304.py shows how packages depending on a specific version of a
package in SyncFirst can cause breakage of the dependency resolver.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is work originally provided by Sascha Kruse on FS#20360 with only
minor adjustments to the implementation. It's been expanded to cover:
NoUpgrade, NoExtract, IgnorePkg, IgnoreGroup.
Adds tests ignore008, sync139, sync502, and sync503.
Also satisfies FS#18988.
Original-work-by: Sascha Kruse <knopwob@googlemail.com>
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
There aretwo seperate issues in the same block of file conflict
checking code here:
1) If realpath errored, such as when a symlink was broken, we would call
'continue' rather than simply exit this particular method of
resolution. This was likely just a copy-paste mistake as the previous
resolving steps all use loops where continue makes sense. Refactor
the check so we only proceed if realpath is successful, and continue
with the rest of the checks either way.
2) The real problem this code was trying to solve was canonicalizing
path component (e.g., directory) symlinks. The final component, if
not a directory, should not be handled at all in this loop. Add a
!S_ISLNK() condition to the loop so we only call this for real files.
There are few other small cleanups to the debug messages that I made
while debugging this problem- we don't need to keep printing the file
name, and ensure every block that sets resolved_conflict to true prints
a debug message so we know how it was resolved.
This fixes the expected failures from symlink010.py and symlink011.py,
while still ensuring the fix for fileconflict007.py works.
Signed-off-by: Dan McGee <dan@archlinux.org>
These should all prevent installation, and yet two of the three tests
currently fail. Not good.
The best way to see what is going on here is to diff the three new tests
side by side- there is only a small difference between the three tests,
and that is in the destination of the symlink in question that should
never be overwritten.
symlink010.py: myprogsuffix -> myprog
symlink011.py: myprogsuffix -> broken
symlink012.py: myprogsuffix -> otherprog
Signed-off-by: Dan McGee <dan@archlinux.org>
This is not something that should be used on a frequent basis, and
giving it a short option encourages use without making the drawbacks
obvious. For the 1% of situations that require it, the 5 extra
keystrokes are a fair price to pay.
Signed-off-by: Dan McGee <dan@archlinux.org>
Make the output into a single block and add separators at the end
so that they do not merge into each other.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Since c51b9ca, ldconfig.stub is required by pactest so we need to
include it as part of the dist tarball.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This removes the last usages of this rule that aren't explicitly looking
at real output from pacman. Notably, these tests depended on one
particular debug logger not ever being changed, which is too fragile,
not to mention doesn't work at all with --nolog.
Signed-off-by: Dan McGee <dan@archlinux.org>
This will work fine on x86_64 (or any platform that has a 64 bit long),
but currently fails on i686. This test also stresses the recent changes
to accommodate package size values greater than a 32 bit UINT_MAX.
Signed-off-by: Dan McGee <dan@archlinux.org>
This actually does something in a scriptlet we can check with our normal
set of rules, rather than relying on pacman debug output.
Signed-off-by: Dan McGee <dan@archlinux.org>
We had this interesting set of facts conundrum, according to vercmp
return values:
2.0a < 2.0
2.0 < 2.0.a
2.0a == 2.0.a
This introduces a code change that ensures '2.0a < 2.0.a' as would be
expected by the first two comparisons. Unfortunately this stays us a bit
further from upstream RPM code, but those are the breaks (in RPM, the
versions involving 'a' do in fact compare the same, but they are both
greater than the bare '2.0').
Signed-off-by: Dan McGee <dan@archlinux.org>
When we switched to using alpm_depcmp() in resolving replacments, we had
some interesting behavior with regard to providers and packages not
found in repositories. Teach the replacement resolving code to not look
at provisions at all to be slightly more sane.
Signed-off-by: Dan McGee <dan@archlinux.org>
These are equivalent. Use the autoconf macro for consistency.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
We can't just check for LIBS as curl won't be listed. Instead, look at
the length of the LIBCURL var from the Makefile.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is done extremely crudely and is not very efficient, but it does
push us down the path of being closer to right, as one additional test
now passes.
Signed-off-by: Dan McGee <dan@archlinux.org>
It turns out we have a few problems here which are best tackled
independently. The first is simply parsing replacements as dep strings;
the second will be dealing with replaces when the original package name
still exists in the repository.
Signed-off-by: Dan McGee <dan@archlinux.org>
We were doing some really silly stuff before and abusing the os.walk()
call, having to walk the entire local database for every single PKG
rule. We really only need top level directories, and we can cache any
generated package since calls to db_read() are well-defined and only
happen in one place.
This speeds up the running of tests that may want to add 100 PKG_VERSION
rules at once, where before we had to limit how many we used in order to
not put a serious cramp in the speed of the test suite run.
Signed-off-by: Dan McGee <dan@archlinux.org>
Note that this is meant to exercise pacsort more than the underlying
version comparsion; that is better left to the standalone vercmptest.sh
test script.
Signed-off-by: Dan McGee <dan@archlinux.org>
Add code to conf.c that parses the new SigLevel directive. An
overwhelming number of options are presented, but most users will still
be fine with the Never/Optional/Required trio. More advanced users can
combine these or any of the other options on a 'SigLevel = ' line, which
is parsed in a left-to-right fashion and flags turned on and off
accordingly. For example, all three of these will net the same config:
SigLevel = Required PackageOptional
SigLevel = Optional DatabaseRequired
SigLevel = DatabaseRequired PackageOptional
Additionally, database-specific lines assume you wish to start with any
global default that has been set. For example, if any of the above lines
were in the [options] section, something such as:
SigLevel = PackageRequired PackageAllowMarginal
Would continue to enforce required database signatures.
Inspiration-by: Kerrick Staley <mail@kerrickstaley.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We don't write with extra or unknown whitespace, so there is little
reason for us to trim it when reading either. This also fixes the
hopefully never encountered "paths that start or end with spaces" issue,
for which two pactests have been added. The tests also contain other
evil characters that we have encountered before and handle just fine,
but it doesn't hurt to ensure we don't break such support in the future.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is more in line with standard Python practice, and makes keyboard
interrupts behave a lot more sanely. It also prevents the useless
spawning of a shell as well as simplifies the command building and
working directory stuff.
Signed-off-by: Dan McGee <dan@archlinux.org>
This addresses FS#25141. We shouldn't remove every empty directory we
come across during the removal process unless it is truly not known to
any other package. This will prevent removal of essential directories
such as '/var/lock/'.
This is accomplished by first checking the empty/non-empty status of a
directory, which was previously done implicitly by calling rmdir() and
ignoring errors. We do this to avoid the next (new) check in most cases,
which is to look at all local packages to see if the to-be-removed
directory is present in another packages' filelist. If we do not find it
anywhere, then we remove it, else we keep the file around.
The pactest has been updated to test more cases, as well as finding a
flaw in the original expected to fail case- we need separate DIR and
FILE based EXIST rules.
Signed-off-by: Dan McGee <dan@archlinux.org>
The bulk of this commit is adding new tests to ensure the new behavior
works without disrupting old behavior. This is a relatively sane maneuver
when a package adds a conf file (e.g. '/etc/mercurial/hgrc') that was
not previously in the package, but it is placed in the backup array. In
essence, we can treat the existing file as having always been a part of
the package and do our normal compare/install as pacnew logic checks.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is similar to what was just done for the sync databases. Move a few
pieces around so we never need to actually write out the filesystem to
create a package, and simply stream the tarfile out from the data we've
collected.
Once again, a few newline addition hacks and other things have to be
left in place in order not to break everything; this time however most
of the assumptions are in pactest and not libalpm.
Signed-off-by: Dan McGee <dan@archlinux.org>
This addresses FS#24904. In a normal upgrade case, this replacement
seems to work just fine. However, when doing a sync "replace" type
upgrade, we weren't properly handling this edge case due to path
comparison not ignoring trailing slashes. Fix this by pruning any
trailing slashes past a certain point of file conflict resolution where
we no longer need them, which allows us to safely detect cases such as
now tested in the new pactest.
Signed-off-by: Dan McGee <dan@archlinux.org>
Start by converting all of our flags to a 'status' bitmask (pkgcache
status, grpcache status). Add a new 'valid' flag as well. This will let
us keep track if the database itself has been marked valid in whatever
fashion.
For local databases at the moment we ensure there are no depends files;
for sync databases we ensure the PGP signature is valid if
required/requested. The loading of the pkgcache is prohibited if the
database is invalid.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is the ideal place to do it as all clients should be checking the
return value and ensuring there are no errors. This is similar to
pkg_load().
We also add an additional step of validation after we download a new
database; a subsequent '-y' operation can potentially invalidate the
original check at registration time.
Note that this implementation is still a bit naive; if a signature is
invalid it is currently impossible to refresh and re-download the file
without manually deleting it first. Similarly, if one downloads a
database and the check fails, the database object is still there and can
be used. These shortcomings will be addressed in a future commit.
Signed-off-by: Dan McGee <dan@archlinux.org>
Sync database are no longer exploded on the filesystem. Rework the logic
used to generate our test databases so we can create them completely in
memory without having to write the individual files to disk at all. The
local database is unaffected.
Note that several shortcomings in libalpm parsing were discovered by
this change, which have since been temporarily patched around in this
test suite:
* archive_fgets() did not properly handle a file that ended in a
non-newline, and would silently drop the data in this line.
* sync database with only the file entries and not the directories would
fail to parse properly, and even cause segfaults in some cases.
Signed-off-by: Dan McGee <dan@archlinux.org>
test/pacman/README mentioned the -A flag, which no longer exists.
Signed-off-by: Kerrick Staley <mail@kerrickstaley.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We were testing whether there were any values in the array, rather than
looking if the values contained anything.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows us to not require the context (e.g. handle) when calling
this function. Also beef up the checks in the two callers of this
function to bail if the last return code is not ARCHIVE_EOF, which is
the expected value.
This requires a change to one of the pactest return codes and the
overall result of the test, but results in a much safer operating
condition whereby invalid database entries will stop the operation.
Signed-off-by: Dan McGee <dan@archlinux.org>
Issue FS#24230. If a symlink is broken and included in the removal
process of a package, we blew up and segfaulted due to
alpm_compute_md5sum() returning NULL and then performing a strcmp()
operation.
Signed-off-by: Dan McGee <dan@archlinux.org>
Don't call os.stat() when we should be using os.lstat(); this allows us
to actually test dead symlinks that don't have a corresponding file. Add
a new LINK_EXIST rule that complements FILE_EXIST for a similar purpose.
Signed-off-by: Dan McGee <dan@archlinux.org>
This moves the generated root/ directory into /tmp, or at least a path
returned by tempfile.mkdtemp(), by default. This can make test runs
significantly faster if done when /tmp is a tmpfs.
If you are debugging a failed test, use the new --keep-root option to
not clean up and pactest will print the location of the generated root/
test directory.
Signed-off-by: Dan McGee <dan@archlinux.org>
This does touch a lot of things, and hopefully doesn't break things on
other platforms, but allows us to also clean up a bunch of crud that no
longer needs to be there.
Signed-off-by: Dan McGee <dan@archlinux.org>
The vercmptest script needs to be invoked as a bash script for this to
be valid; the -p operator is interpreted as an argument to look up by
sh. This goes way back to commit 3bf9448943, done to solve
http://mailman.archlinux.org/pipermail/pacman-dev/2008-July/007180.html.
Saw this problem running in a virtual machine where sh is not bash, but
in fact dash:
user@debian-powerpc:~/projects/pacman$ ./test/util/vercmptest.sh
src/util/vercmp-p: not found
src/util/vercmp is src/util/vercmp
vercmp binary (src/util/vercmp) could not be located
Signed-off-by: Dan McGee <dan@archlinux.org>
Once we do this, add support for VerifySig to pactest. We just check if
the repo name contains Always, Never or Optional to determine the value
of VerifySig. The default is Never. pacman uses Always by default but
this is not suitable for pactest.
Original-work-by: shankar <jatheendra@gmail.com>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Noted in FS#23342. When the user attempts to install an ignored package
and answers no when asked whether to install it, pacman bails out with:
"error: target not found: packagename"
This is because satisfiers are not found for the package and execution
continues to process_group(), where the package is treated as a group
(which does not exist).
In addition, test ignore006.py is updated with PACMAN_RETCODE=0 since
saying no to installing an ignored package should not be considered an
error.
Signed-off-by: Pang Yan Han <pangyanhan@gmail.com>
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
I managed to just make deptest001.py fail by changing a DEBUG-level
logger in commit b12be99c89. This should not be this fickle. Enhance the
OUTPUT rule to use an actual Python re object when looking for matches,
and make a lot of the rules use stronger patterns to match with.
Signed-off-by: Dan McGee <dan@archlinux.org>
This case currently fails, but highlights a failure in our install
process I experienced the other day. Because we don't do replacement
uninstalls inline with the rest of the upgrade uninstalls, we can have a
time on our system where a critical package is not installed.
I hope no one ever renames glibc.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows error messages emitted by the frontend to be a bit more
descriptive and not have the annoying "well why didn't you tell me that
the first time" problem. If a package had multiple missing deps, we
would bail on the first one before rather than finish processing all
missing dependencies, and only print one error message. Instead,
continue through this entire set of missing deps and append all eventual
errors.
The added pactest tests this case, as the to be installed package has
two missing dependencies. However, pactest does not actually test or see
the difference in output from before and after, so it passes in both
cases, but it is clearly visible in the logs.
Signed-off-by: Dan McGee <dan@archlinux.org>
