Read in list of packages for sync db from tar archive.
Breaks reading in _alpm_sync_db_read and a lot of pactests (which
is expected as they do not handle sync db in archives...).
Signed-off-by: Allan McRae <allan@archlinux.org>
Put the db_operations struct to use and completely split the handling
of the sync and local databases.
Signed-off-by: Allan McRae <allan@archlinux.org>
The file be_files.c is "split" to be_local.c and be_sync.c in order
to achieve separate handling of sync and local databases.
Some basic clean-up of functions that are only of use for local or
sync databases has been performed and some rough function renaming
in duplicated code has been performed to prevent compilation errors.
However, most of the clean-up and final separation of sync and local
db handling occurs in following patches.
Signed-off-by: Allan McRae <allan@archlinux.org>
These will be needed for the handling of both local and sync database
caches, so put them in a common location.
Signed-off-by: Allan McRae <allan@archlinux.org>
Move splitname, checkdbdir, get_pkgpath into db.{h,c} as these will be
needed to parse both the local and sync databases during the initial
splitting. They will be moved out of db.{h,c} at to more appropriate
locations at a later stage.
Signed-off-by: Allan McRae <allan@archlinux.org>
It doesn't do a whole lot yet, but these type of operations will
potentially be different for the DBs we load.
Signed-off-by: Dan McGee <dan@archlinux.org>
Cache bullshit only has relevance to be_files, so move it there.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: BIG rebase]
Signed-off-by: Allan McRae <allan@archlinux.org>
Hopefully we've finally arrived at package handling nirvana, or at least
this commit will get us a heck of a lot closer. The former method of getting
the depends list for a package was the following:
1. call alpm_pkg_get_depends()
2. this method would check if the package came from the cache
3. if so, ensure our cache level is correct, otherwise call db_load
4. finally return the depends list
Why did this suck? Because getting the depends list from the package
shouldn't care about whether the package was loaded from a file, from the
'package cache', or some other system which we can't even use because the
damn thing is so complicated. It should just return the depends list.
So what does this commit change? It adds a pointer to a struct of function
pointers to every package for all of these 'package operations' as I've
decided to call them (I know, sounds completely straightforward, right?). So
now when we call an alpm_pkg_get-* function, we don't do any of the cache
logic or anything else there- we let the actual backend handle it by
delegating all work to the method at pkg->ops->get_depends.
Now that be_package has achieved equal status with be_files, we can treat
packages from these completely different load points differently. We know a
package loaded from a zip file will have all of its fields populated, so
we can set up all its accessor functions to be direct accessors. On the
other hand, the packages loaded from the local and sync DBs are not always
fully-loaded, so their accessor functions are routed through the same logic
as before.
Net result? More code. However, this code now make it roughly 52 times
easier to open the door to something like a read-only tar.gz database
backend.
Are you still reading? I'm impressed. Looking at the patch will probably be
clearer than this long-winded explanation.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Allan: rebase and adjust]
Signed-off-by: Allan McRae <allan@archlinux.org>
Implement this seemingly simple change in package.h:
typedef enum _pmpkgfrom_t {
- PKG_FROM_CACHE = 1,
- PKG_FROM_FILE
+ PKG_FROM_FILE = 1,
+ PKG_FROM_LOCALDB,
+ PKG_FROM_SYNCDB
} pmpkgfrom_t;
which requires flushing out several assumptions from around the codebase
with regards to usage of the PKG_FROM_CACHE value. Make some changes where
required to allow the switch, and now the correct value should be set (via a
crude hack) depending on whether a package was loaded as an entry in a local
db or a sync db.
This patch underwent some big rebasing from Allan and Dan.
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Move almost all of the caching related stuff into a single #define
(which should maybe even just be a static function) so we don't
duplicate logic all over the place. This also makes the code a heck of a
lot shorter and means further changes to this stuff don't have to touch
each and every getter function.
Signed-off-by: Dan McGee <dan@archlinux.org>
We weren't reading this in from our packages, thus causing us not to write
it out to our local database. Adding this now will help ease the upgrade
path for epoch later and not require reinstallation of all force packages.
Signed-off-by: Dan McGee <dan@archlinux.org>
On Linux and OS X, we can determine if an entry obtained through a readdir()
call is a directory without also having to stat it. This can save a
significant number of syscalls. The performance increase isn't dramatic, but
it could be on some platforms (e.g. Cygwin) so it shouldn't hurt to use this
unconditionally where supported.
Signed-off-by: Dan McGee <dan@archlinux.org>
I don't know what I tested in commit 3e7b90ff69, but it definitely wasn't
working as advertised. Fix the checks in the source code itself to match the
right define (HAVE_LIBFETCH), as well as make sure the configure check
defaults to looking for the library but not bailing if it could not be
found.
Signed-off-by: Dan McGee <dan@archlinux.org>
It touched up these a bit after it ran, so might as well check the changes
in so we don't have to deal with them again later.
Signed-off-by: Dan McGee <dan@archlinux.org>
Model it after the new OpenSSL check, and have it be a bit more useful. If
you do not explicitly pass a command line option, it will be linked if
available but will not error out if it is missing. Also bump the version to
that where connection caching was introduced as we use these new features in
the codebase.
Signed-off-by: Dan McGee <dan@archlinux.org>
I've noticed my Atom-powered laptop is dog-slow when doing integrity checks
on packages, and it turns out our MD5 implementation isn't near as good as
that provided by OpenSSL. Using their routines instead provided anywhere
from a 1.4x up to a 1.8x performance benefit over our built-in MD5 function.
This does not remove the MD5 code from our codebase, but it does enable
linking against OpenSSL to get their much faster implementation if it is
available on whatever platform you are using. At configure-time, we will
default to using it if it is available, but this can be easily changed by
using the `--with-openssl` or `--without-openssl` arguments to configure.
Signed-off-by: Dan McGee <dan@archlinux.org>
This gave at least a 10% improvement on a few tested platforms due to the
reduced number of read calls from files when computing the md5sum. It really
is just a precursor to another patch to come which is to use MD5 functions
that do the job a lot better than anything we can do.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is being checked in as 'pt' rather than 'pt_PT' as that is what
Transifex seems to want, and it is also the dominant choice of packages
already installed on my system when doing a count of the files located in
the /usr/share/locale translation directories.
Thanks for the new translation!
Signed-off-by: Dan McGee <dan@archlinux.org>
Fixes FS#18770, and hopefully an occasional deadlock in my frontend as well.
For simplicity it redirects all scriptlet output through SCRIPTLET_INFO, and
all callbacks in the child process have been replaced for thread-safety.
Signed-off-by: Jonathan Conder <j@skurvy.no-ip.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
The combination of tabs and spaces is annoying in any editor that
does not use a tab width of 2 spaces.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
From the fgets manpage:
fgets() reads in at most one less than size characters from stream and
stores them into the buffer pointed to by s. Reading stops after an EOF
or a newline. If a newline is read, it is stored into the buffer. A
'\0' is stored after the last character in the buffer.
This means there is no need at all to do 'size - 1' math. Remove all of that
and just use sizeof() for simplicity on the buffer we plan on reading into.
Signed-off-by: Dan McGee <dan@archlinux.org>
This will allow downloads to reuse connections if possible, which could make
big differences on perceived FTP speed as the connection won't have to be
reestablished each time. For the most part, HTTP requests wouldn't be using
keep alive anyway so this won't have an effect there.
I'm not enthused about having to do this with the library initialization,
but there isn't a much better place due to the fact that the loop over
databases occurs on the frontend and not the backend.
Signed-off-by: Dan McGee <dan@archlinux.org>
We no longer use these anywhere outside of sync.c, so do the rename and add
static to their definition to meet our coding standards.
Signed-off-by: Dan McGee <dan@archlinux.org>
As reported in FS#20221, we don't always do the right thing when installing
a group and using the --needed option. This was due to the code pulling
packages based on what was already in the transaction's add list, but
completely ignoring the fact that we may have already seen and skipped this
same package in an earlier repository.
Add a list to the private _alpm_sync_pkg() function that allows us to have
this extra information so we don't mistakenly downgrade a package when using
--needed.
Signed-off-by: Dan McGee <dan@archlinux.org>
The sync db should be stored in the sync/ folder. This cleans up
DBPath to only have local/ and sync/ directories in it.
A nice side effect is that the db are now in the right place so we
can implement directly reading from them.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
With commit 5dffef78, the repo database always has a symlink
of the form reponame.db. Use that filename and let libarchive
determine the compression type.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Prevents compiler warnings when building with -D_FORTIFY_SOURCE=2
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Check that writing to destination file actually occurs in
_alpm_copyfile. Required adding a new error (PM_ERR_WRITE)
as none of the others appeared appropriate.
Prevents compiler warning when using -D_FORTIFY_SOURCE=2.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
The casting of nread is safe as it is tested to be >0 when it is
initally assigned. It is also being implicitly cast in the fwrite
call in the line above.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
Rather than say we can't find the target after saying "No, I guess I don't
want to install this", we should make sure the ignored status gets passed
all the way through. This fixes FS#19866.
Pactest is also included that failed before due to the fact that we normally
treat an unfound package as a reason to exit with a non-zero status.
Signed-off-by: Dan McGee <dan@archlinux.org>
Follow the HACKING guidelines and always use != 0 or == 0 rather
than negation within conditional statements to improve clarity.
Most of these are !strcmp usages which is the example of what not
to do in the HACKING document.
Signed-off-by: Allan McRae <allan@archlinux.org>
Use strcoll to compare package names to provide output sorted
according to a users LC_COLLATE settings.
Signed-off-by: Andres P <aepd87@gmail.com>
[Allan: added commit message]
Signed-off-by: Allan McRae <allan@archlinux.org>
Fix the '\t' characters that got introduced by the last update of this
translation that should not have been there.
Signed-off-by: Dan McGee <dan@archlinux.org>
This puts us more in line with other projects that don't attach the country
code to the language code.
$ du -sh /usr/share/locale/nb*/LC_MESSAGES
3.5M /usr/share/locale/nb/LC_MESSAGES
132K /usr/share/locale/nb_NO/LC_MESSAGES
Signed-off-by: Dan McGee <dan@archlinux.org>
This puts us more in line with other projects that don't attach the country
code to the language code.
$ du -sh /usr/share/locale/sv*/LC_MESSAGES
7.2M /usr/share/locale/sv/LC_MESSAGES
60K /usr/share/locale/sv_SE/LC_MESSAGES
Signed-off-by: Dan McGee <dan@archlinux.org>
Thanks to CalimeroTeknik <calimeroteknik@free.fr> for providing many
corrections !
Signed-off-by: Xavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch fixes the phonon/qt issue, if all to-be-upgraded packages are
explicit targets (ie. only not-yet-installed packages are pulled by
resolvedeps). This condition covers the most common situations, for example
it should hold with every -Su operation.
After this patch sync405.py passes, but sync406.py doesn't.
The work is inspired by the patch of Henning Garus, thanks for his work:
http://mailman.archlinux.org/pipermail/pacman-dev/2010-February/010429.html
(I moved the alpm_list_diff computation to sync.c in order to compute it
only once.)
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
After commit df99495b82 pacman downloaded files from the first repo only,
and reported corrupted packages for all files from other repos.
The download_size was set to 0 for _all_ transaction packages after
downloading some files from the first repo. This code-block was moved to its
correct place.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
The request of FS#12950 is implemented.
On the backend side, I introduced a new function, alpm_db_set_pkgreason(),
to modify the install reason of a package in the local database. On the
front-end side, I introduced a new main operation, -D/--database, which has
two options, --asdeps and --asexplicit. I documented this in pacman manual.
I've created two pactests to test -D: database001.py and database002.py.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
It isn't really necessary here and it helps us get rid of some link
pollution so we can have a slim vercmp binary.
Signed-off-by: Dan McGee <dan@archlinux.org>
This will facilitate using this object file on its own in the vercmp tool
which will be done in a future commit. The net impact on the generated
binaries should not be noticeable after this commit.
Signed-off-by: Dan McGee <dan@archlinux.org>
Just as we do in -Qi, we can compute required by information for sync
database packages. The behavior seems sane; for a given package, the -Sii
required by will show all packages in *any* sync database that require it.
Implements FS#16244.
Signed-off-by: Dan McGee <dan@archlinux.org>
Basically I'm the idiot that thought I could make it better and completely
forgot how freeing the contents of the original lists would screw up our
nice little diff extraction lists. This caused segfaults among other
problems. Last time I try to do that...
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff627ce26 in strcmp () from /lib/libc.so.6
(gdb) bt
Signed-off-by: Dan McGee <dan@archlinux.org>
Follow-up to the previous "Only extract new DB entries" patch; move the
partial extraction code inside one side of the loop so we can use the same
code for actually doing file extraction.
Signed-off-by: Dan McGee <dan@archlinux.org>
This implements FS#15198. The idea apparently came from Csaba Henk
<csaba-ml <at> creo.hu> which submitted a patch to Frugalware, so thanks to
him, even though I did not look at the code :)
The idea is to only extract folders for new packages into the package
database and clean up the old directories. This is essentially implementing
Xyne's "rebase" script within pacman.
If using -Syy, just remove and extract everything.
If using -Sy :
1. Generate list of directories in DB
2. Generate list of directories in archive
3. Compare both
4. Clean up old directories
5. Extract new directories
Original-work-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
[Dan: fix compile error, s/int/size_t/]
Signed-off-by: Dan McGee <dan@archlinux.org>
We free'd the handle but didn't NULL out the global variable, leading to
problems if you try to reinitialize the library. Make sure we clean up after
ourselves.
Signed-off-by: Dan McGee <dan@archlinux.org>
download_internal is supposed to always set pm_errno but did not in many
cases.
The most important (and tested) change is the one concerning fetchStat. This
is typically where the code will fail when the network is down for example.
Before commit d2dbb04a9a, this fetchStat call did not exist and the
same kind of errors would be encountered in the fetchXGet call that follows.
I just copied the error printing to restore the old behavior.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Some users reported duplicated database entries in /var/lib/pacman/local/,
for example, both foo-1.0-1 and foo-2.0-1 subdirectories existed. (Bogus
3rd-party scripts, backup?) In this case pacman reported no error and its
behaviour was mysterious.
From now on, pacman detects this situation and prints an error message.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a bit embarrassing. For example:
$ pacman -Qi mesa
...
Required By : mesa mesa mesa mesa mesa mesa
Something is clearly not right, and the problem was introduced in commit
0bc961. Fix the issue by getting the package name off the correct variable.
Signed-off-by: Dan McGee <dan@archlinux.org>
Sorry for this being such a huge patch, but I believe it is necessary for
quite a few reasons which I will attempt to explain herein. I've been
mulling this over for a while, but wasn't super happy with making the
download interface more complex. Instead, if we carefully order things in
the internal download code, we can actually make the interface simpler.
1. FS#15657 - This involves `name.db.tar.gz.part` files being left around the
filesystem, and then causing all sorts of issues when someone attempts to
rerun the operation they canceled. We need to ensure that if we resume a
download, we are resuming it on exactly the same file; if we cannot be
almost postive of that then we need to start over.
2. http://www.mail-archive.com/pacman-dev@archlinux.org/msg03536.html - Here
we have a lighttpd bug to ruin the day. If we send both a Range: header and
If-Modified-Since: header across the wire in a GET request, lighttpd doesn't
do what we want in several cases. If the file hadn't been modified, it
returns a '304 Not Modified' instead of a '206 Partial Content'. We need to
do a stat (e.g. HEAD in HTTP terms) operation here, and the proceed
accordingly based off the values we get back from it.
3. The mtime stuff was rather ugly, and relied on the called function to
write back to a passed in reference, which isn't the greatest. Instead, use
the power of the filesystem to contain this info. Every file downloaded
internally is now carefully timestamped with the remote file time. This
should allow the resume logic to work. In order to guarantee this, we need
to implement a signal handler that catches interrupts, notifies the running
code, and causes it to set the mtimes on the file. It then rethrows the
signal so the pacman signal handler (or any frontend) works as expected.
4. We did a lot of funky stuff in trying to track the DB last modified time.
It is a lot easier to just keep the downloaded DB file around and track the
time on that rather than in a funky dot file. It also kills a lot of code.
5. For GPG verification of the databases down the road, we are going to need
the DB file around for at least a short bit of time anyway, so this gets us
closer to that.
Signed-off-by: Dan McGee <dan@archlinux.org>
[Xav: fixed printf with off_t]
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
* It makes the code clearer to read/understand
* Cppcheck tool doesn't show this anymore: [./util.c:215]: (error) Resource leak: fd
[Dan: don't change the coding style]
Signed-off-by: Dan McGee <dan@archlinux.org>
Thanks to Laszlo Papp <djszapi@archlinux.us> for the following catch:
opendir(path)) == (DIR *)-1;
is maybe the result of misunderstanding the manpage. If an opendir() call
isn't successful it returns NULL rather than '(DIR *)-1'.
Noticed-by: Laszlo Papp <djszapi@archlinux.us>
Signed-off-by: Dan McGee <dan@archlinux.org>
alpm_pkg_get_name() gives us little benefit in backend code besides a NULL
check on the package passed in; we could do that ourself if necessary. By
changing to direct references in the cases where we are sure we have a valid
package, we save a function call each time we need a package name. This
function can't be inlined because it is externally accessible.
This cuts the calls to get_name() from 1.3 million times in a
pacman -Qu operation to around 2400.
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes the following valgrind warning :
==26831== Syscall param rt_sigaction(act->sa_flags) points to uninitialised
byte(s)
==26831== at 0x4282547: __libc_sigaction (in /lib/libc-2.10.1.so)
==26831== by 0x403C693: download_internal (dload.c:152)
==26831== by 0x403D0E4: _alpm_download_single_file (dload.c:311)
==26831== by 0x4033B72: alpm_db_update (be_files.c:319)
==26831== by 0x805205E: pacman_sync (sync.c:257)
==26831== by 0x804EE54: main (pacman.c:1120)
==26831== Address 0xbec6cc04 is on thread 1's stack
==26831==
==26831== Syscall param rt_sigaction(act->sa_restorer) points to
uninitialised byte(s)
==26831== at 0x4282547: __libc_sigaction (in /lib/libc-2.10.1.so)
==26831== by 0x403C693: download_internal (dload.c:152)
==26831== by 0x403D0E4: _alpm_download_single_file (dload.c:311)
==26831== by 0x4033B72: alpm_db_update (be_files.c:319)
==26831== by 0x805205E: pacman_sync (sync.c:257)
==26831== by 0x804EE54: main (pacman.c:1120)
==26831== Address 0xbec6cc08 is on thread 1's stack
==26831==
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Pacman's fgets function in the API used hardcoded numbers to identify the size.
This is not good practice, so replace them with sizeof handling.
Signed-off-by: Laszlo Papp <djszapi@archlinux.us>
Signed-off-by: Dan McGee <dan@archlinux.org>
This offers a cleaner way to deal with constant in enum and allow easy
maintainance
Signed-off-by: solsTiCe d'Hiver <solstice.dhiver@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
If the -d switch was invoked with -S (or -U), the removes list was simply
lost, because trans->remove was computed in an
"if(!(trans->flags & PM_TRANS_FLAG_NODEPS))" block.
I've added a new pactest file, sync045.py (derived from sync043.py) to test
this.
Additionally, I did some other minor cleanups in sync_prepare:
* preferred list is not needed anymore
* I removed a needless alpm_list_remove_dupes line (the target list should
not contain dupes at all)
* I moved alpm_list_free(remove); to cleanup part to eliminate a possible
memleak
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is more efficient than alpm_list_diff since it assumes the two lists
are sorted. And also we get the two sides of the diff.
Even sorting should more efficient than the current list_diff. Sorting the
two lists should be O(n*log(n)+m*log(m)) while the current list_diff is
O(n*m). So I also reimplemented list_diff using list_diff_sorted.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We don't need to count the number of packages left once per file when
removing; we only need to do it once per package. Also move a variable into
the correct scope.
Signed-off-by: Dan McGee <dan@archlinux.org>
After our recent screwup with size_t and ssize_t in the download code, I
found the `-Wsign-conversion` flag to GCC to see if we were doing anything
else boneheaded. I didn't find anything quite as bad, but we did have some
goofups- most of our public unsigned methods would return -1 on error, which
is a bit odd in an unsigned context.
Signed-off-by: Dan McGee <dan@archlinux.org>
A NULL list element triggered an infinite loop. Not cool :)
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
fetchIO_read returns -1 in case of error, and the return type is
ssize_t, not size_t ! So we converted -1 to an unsigned, which led to
huge file write.
The rest is just changing the error return a bit.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Get them a bit more standardized across the board, as they were quite a
mess. Also note the two new translations we received for 3.3.1.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is for 3.3.0, not for 3.3.1. But since there are only like 10 messages
missing, it seems worth including now.
Signed-off-by: Christian Larsson <congacx@gmail.com>
Signed-off-by: Laszlo Papp <djszapi2@archlinux.us>
[Dan: fix some busted translation strings]
Signed-off-by: Dan McGee <dan@archlinux.org>
Commit 34e1413d75 attempted to implement lazy loading of package databases.
Although it took care of my main complaint (creating the database directory
if it didn't exist), it didn't allow sync repos to be registered before
alpm_option_set_dbpath() had been called.
With this patch, we no longer compute the individual repository DB paths
until necessary, allowing full lazy loading to work as intended, and
allowing us to drop the extra setlibpath() calls from the frontend. This
allows the changes introduced in a2cd48960 (but later reverted) to be added
back in again.
Signed-off-by: Dan McGee <dan@archlinux.org>
It was probably a bad idea to modify the target directly in case of
repo/pkg syntax.
Duplicating it also allows us to keep the original target string, which
is more informative when printing errors.
Also remove a duplicated error message from libalpm, and improve the
message already returned to the frontend.
$ pacman -S foo/bar
before
error: repository 'foo' not found
error: 'bar': no such repository
after
error: 'foo/bar': could not find repository for target
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Add more untranslated strings, improve consistency, etc.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
After commit 774c252 the --debug output shows 5-6 "syntax error..." lines
for each package. After this patch pacman recognizes makepkgopt as a valid
key, but doesn't do anything.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
-int alpm_trans_sysupgrade(int enable_downgrade);
-int alpm_trans_sync(char *target);
-int alpm_trans_add(char *target);
-int alpm_trans_remove(char *target);
+int alpm_sync_sysupgrade(int enable_downgrade);
+int alpm_sync_target(char *target);
+int alpm_sync_dbtarget(char *db, char *target);
+int alpm_add_target(char *target);
+int alpm_remove_target(char *target);
* functions renaming
* add new sync_dbtarget which allows to specify the db
* repo/ syntax handling is moved to frontend
( should implement FS#15141)
* group handling is moved to backend
( see http://www.archlinux.org/pipermail/pacman-dev/2009-June/008847.html )
This basically started with this change :
/* Transaction */
struct __pmtrans_t {
- pmtranstype_t type;
pmtransflag_t flags;
pmtransstate_t state;
- alpm_list_t *packages; /* list of (pmpkg_t *) */
+ alpm_list_t *add; /* list of (pmpkg_t *) */
+ alpm_list_t *remove; /* list of (pmpkg_t *) */
And then I have to modify all the code accordingly.
This patch utilizes the power of sync.c to fix FS#3492 and FS#5798.
Now an upgrade transaction is just a sync transaction internally (in alpm),
so all sync features are available with -U as well:
* conflict resolving
* sync dependencies from sync repos
* remove unresolvable targets
See http://www.archlinux.org/pipermail/pacman-dev/2009-June/008725.html
for the concept.
We use "mixed" target list, where PKG_FROM_FILE origin indicates local
package file, PKG_FROM_CACHE indicates sync package. The front-end can add
only one type of packages (depending on transaction type) atm, but if alpm
resolves dependencies for -U, we may get a real mixed trans->packages list.
_alpm_pkg_free_trans() was modified so that it can handle both target types
_alpm_add_prepare() was removed, we use _alpm_sync_prepare() instead
_alpm_add_commit() was renamed to _alpm_upgrade_targets()
sync.c (and deps.c) was modified slightly to handle mixed target lists,
the modifications are straightforward. There is one notable change here: We
don't create new upgrade trans in sync.c, we replace the pkgcache entries
with the loaded package files in the target list (this is a bit hackish) and
call _alpm_upgrade_targets(). This implies a TODO (pkg->origin_data.db is
not accessible anymore), but it doesn't hurt anything with pacman front-end,
so it will be fixed later (otherwise this patch would be huge).
I updated the documentation of -U and I added a new pactest, upgrade090.py,
to test the syncdeps feature of -U.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Sometimes "foo conflicts with bar" information is not enough, see this
thread: http://bbs.archlinux.org/viewtopic.php?id=77647. That's why I added
a new reason field to our pmconflict_t struct that stores the packager-
defined conflict that induced the fact that package1 conflicts with
package2.
I modified the front-end (in callback.c, sync.c, upgrade.c) to print this
new information as well.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
This function is unused since commit
358cc5804a.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
[Dan: also kill from util.h]
Signed-off-by: Dan McGee <dan@archlinux.org>
- fix one memleak if get_filename failed
- cleanup according to Joerg's feedback:
"url_for_string: If fetchParseURL returned successful, you should always
have a scheme set. The logic for anonftp should only be needed for very
broken server -- do you know of any such?
download_internal:
Specifying 'p' is now a nop -- it is tried by default first with
fall-back to active FTP."
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
[Dan: remove from pacman.conf and pacman.conf.5]
Signed-off-by: Dan McGee <dan@archlinux.org>
libfetch supports checking mtime so we do not need to do it manually.
when the databases are already up-to-date, initiating a connection with
fetchXGet and closing it right after with fetchIO_close took a very long
time (up to 10min!) on some network.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
(cherry picked from commit d7675e393f)
We had 10000 as our timeout value, assuming it was expressed in ms. This is
false after looking at the current code, so reset it back to 10 seconds.
Addresses FS#15369.
Signed-off-by: Dan McGee <dan@archlinux.org>
I assume the loop was never iterated more than once, because the write
location was not updated at each loop iteration (buffer instead of buffer +
nwritten), yet we never had reports of corrupted download.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
libfetch supports checking mtime so we do not need to do it manually.
when the databases are already up-to-date, initiating a connection with
fetchXGet and closing it right after with fetchIO_close took a very long
time (up to 10min!) on some network.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
If /sbin is not in the PATH and sudo is used, ldconfig cannot be found. So
use /sbin/ldconfig instead. The code checked for the existence of
/sbin/ldconfig anyway..
Signed-off-by: Marc - A. Dahlhaus <mad@wol.de>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
After commit 30c4d53ce5, get_destfile and
get_tempfile are only used for internal download, so move these two
functions inside the ifdef
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
'make distcheck' had issues with this one and reformatted it. In addition,
it found a fuzzy message which is now fixed due to an inadvertent msgid
edit.
Signed-off-by: Dan McGee <dan@archlinux.org>
This allows a frontend to define its own download algorithm so that the
libfetch dependency can be omitted without using an external process.
The callback will be used when if it is defined, otherwise the old
behavior applies.
Signed-off-by: Sebastian Nowicki <sebnow@gmail.com>
[Dan: minor cleanups]
Signed-off-by: Dan McGee <dan@archlinux.org>
If the user switches from unstable repo to a stable one, it is quite hard to
sync its system with the new repo (the user will see many "Local is newer
than stable" messages, nothing more). That's why I introduced -Suu, which
treats a sync package like an upgrade, iff the package version doesn't match
with the local one's.
I added a new pactest (sync104.py) to test this, and I updated the
documentation of -Su.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
[Dan: slight doc reword]
Signed-off-by: Dan McGee <dan@archlinux.org>
This happens for example if you install a new package, and one of its
backup config file is already on the file system.
If the local file was different, it was saved to .pacorig which is fine.
However if the local file and pkg file were the same, the pkg file
(temporarily extracted as .paccheck) was left on the system.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes FS#15546
Also fix the interface of unlink_file which was really stupid..
(alpm_list_t used with only one element)
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
A package can now replace symdir->dir by dir without fileconflicts.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
When one package wants to replace a directory by a file, we check that all
files in that directory were owned by that package.
Additionally pacman can be more verbose when the extraction of the symlink
(or file) fails. The patch to add.c looks more complex than it is, I just
moved and reindented code to handle cases 10 and 11 together.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes FS#15294.
The code to run a command inside a chroot was refactored from the
_alpm_runscriptlet function to _alpm_run_chroot.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
See FS#14642- this allows -Qs output to be fed back into pacman without
problems or having to strip off the 'local/' prefix manually.
Signed-off-by: Dan McGee <dan@archlinux.org>
See FS#13099. This makes sense especially for the pacman frontend, as we
show groups in the search output.
Signed-off-by: Dan McGee <dan@archlinux.org>
$ sudo pacman -S mc
Old output:
***********
:: mc conflicts with mc-mp. Remove mc-mp? [Y/n] y
...
(1/1) checking for file conflicts [################] 100%
(1/1) installing mc [################] 100%
New output:
***********
:: mc conflicts with mc-mp. Remove mc-mp? [Y/n] y
...
(1/1) checking for file conflicts [################] 100%
(1/1) removing mc-mp [################] 100%
(1/1) installing mc [################] 100%
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes FS#14899. When running an -Sp operation without servers
configured for a repository, we would segfault, so add an assert to the
backend method returning the first server preventing a null pointer
dereference.
In addition, add a new error code to libalpm that indicates we have no
servers configured for a repository. This makes -Sy and -S <package>
operations fail gracefully and helpfully when a repo is set up with no
servers, as the default mirrorlist in Arch is provided this way.
Signed-off-by: Dan McGee <dan@archlinux.org>
The main purpose of this function to make our code more readable.
It frees transaction specific fields of pmpkg_t. (It is used when a package
is removed from the target list.)
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
There is apparently no need to handle the re-compression manually when
applying a xdelta patch in case of bzip2 or xz.
Only gzip needs to be handled specifically for disabling timestamp with the
-n option.
After this patch, if xdelta is enhanced with xz support (1-line patch), it
will be transparent from pacman side.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This flag indicates that the front-end will not call alpm_trans_commit(),
so the database needn't be locked. This is the first step toward fixing
FS#8905.
If this flag is set, alpm_trans_commit() does nothing and returns with
an error (that has new code: PM_ERR_TRANS_NOT_LOCKED).
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
"Foo replaces bar" simply means that "foo is a new version of bar".
So this patch refactors the code to use this "rule".
_alpm_sync_sysupgrade now does the following for each local package [bar]
(pseudo-code):
for db in syncdbs {
if(db contains bar) {check if db/bar is an upgrade; break;}
replacers = find(bar replacers in db);
if(replacers!=NULL) {ask_user; break;}
}
Note:
1. Replacers are grouped per repo. If more than one package replace bar in
a repo, all of them are considered ("package set of bar replacers").
2. If repo1/foo1 and repo2/foo2 both replaces bar, only repo1/foo1 is
considered (if repo1 stands before repo2 in pacman.conf). FS#11737 is fixed.
3. It can happen that pacman doesn't consider any replacer, if it found a
literal "earlier", so sync132.py modified accordingly (btw, that situation
should not appear irl).
The new sysupgrade code doesn't use sync_newversion(), so I removed the
"local is newer than repo" message, which was annoying with -Qu and
SyncFirst.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes a bug introduced by my previous changes which changes the
behavior of IgnorePkg/IgnoreGroup to allow the user to remove unresolvable
packages from the transaction. The bug is that the target-list was no
longer being consulted first to resolve dependencies, which means that if
two packages in the sync database satisfied a dependency, and the user
explicitly requested one of those two packages in the sync, the other
package was still being pulled in.
A new test was added, sync993.py, to verify the desired behavior.
Signed-off-by: Bryan Ischo <bji-keyword-pacman.3644cb@www.ischo.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
pmsyncpkg_t data sructure was removed:
1. pmpkg_t.reason is used instead of pmsyncpkg_t.newreason. (The target
packages come from sync repos, so we can use this field without any
problems. Upgrade transaction also uses this field to store this info.)
2. pmsyncpkg_t.removes was moved to pmpkg_t.removes.
This step requires careful programming, because we don't duplicate packages
when we add them to trans->packages. So we modify sync pkgcache when we
add this transaction-only info to our package. Hence it is important to
free this list when we remove any package from the target list
(remove_unresolvable, remove_conflicts, trans_free), otherwise this could
confuse the new sync transactions (with non-pacman GUI).
Overall, our code became ~100 line shorter, and we can call our helper
functions directly on trans->packages in sync.c, we don't need to maintain
parallel package lists.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
From now on _alpm_db_find_fileconflicts() works with upgrade and remove
target lists (like checkdeps), which makes it transaction independent
(we still need a trans param because of the progressbar). This is a small
step towards the universal transaction. So we call this function directly
from sync.c before commiting the remove transaction. This is much safer,
but we can get false fileconflict error alarms in some tricky cases
("symlinks puzzle" etc).
The patch on find_fileconflict looks complex, but it is mainly an
"indent-patch", the new code-part can be found after the
/* check remove list ... */ comment, and I modified something around the
"file has changed hand" case (see comment modifications in the code).
Unfortunately sync.c became more ugly, because we have to create 2 parallel
internal transactions: to avoid duplicated work, upgrade transaction is
used to load package data (filelists). This problem will disappear, when
we finally get rid of internal transactions.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Resolvedeps reports error when it cannot resolve some dependencies, puts
them into the *data list, and set pm_errno. If user removes the
unresolvable packages from the target list, we have no error anymore,
so from now on we free *data list (we eliminate a memleak) and unset
pm_errno.
(Additionally I removed two needless lines from the code, unresolvable list
is always freed in cleanup.)
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
After commit f57f8d3386 pacman *silently*
ignores packages from IgnorePkg/IgnoreGroup during dependency resolving,
if prompt == 0. This behavior is changed to "give warning + ignore".
(Otherwise the user is not informed about the fact that the package
resolving was blocked by ignorepkg.)
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch fixes FS#12059.
Now sync_addtarget can return with PM_ERR_PKG_IGNORED, which indicates that
although the requested package was found it is in ignorepkg, so alpm could
not add it to the transaction. So the front-end can decide what to do.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This adds alpm_db_update() to the alpm_databases Doxygen group. The function
is described in more detail and a code example is given.
Signed-off-by: Dan McGee <dan@archlinux.org>
1) The changes to sync.c look big but there are mostly caused by
the indentation. Fix a bug where download_size == 0 because the packages and
deltas are already in the cache, but we still need to build the deltas list
and apply the deltas to create the final package.
2) Fix the gzip / md5sum issue by switching to xdelta3, disabling external
recompression and using gzip -n in pacman, and disable bsdtar compression
and using gzip -n in makepkg.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
This reverts commit 9558639d80.
This change was wrong, popen does require /bin/sh in a subchroot.
1) pacman -S lilo -r root
Notice no error
2) rm root/bin/sh ; pacman -S lilo -r root
Notice an error :
error: scriptlet failed to execute correctly
Actually, we already get an explicit error here, when popen is run, so there
is no need to check for bin/sh explicitely.
Besides this check was problematic in some cases. For example, bash itself
has a scriptlet, but only post_install and post_upgrade, no pre_install and
pre_upgrade. However, since bash has a scriptlet, runscriptlet will also be
called before bash is installed. It won't do anything since the scriptlet
has no pre_install function. But if we keep the check, we will still get
"error : no /bin/sh".
Conflicts:
lib/libalpm/trans.c
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a bug I noticed 2 years ago :
http://www.nabble.com/Re%3A-logging-output-crazy-to11437357.html#a11479679
I thought I fixed it with 57d77eab32
But the bug was still here. Reading man fork, this part caught my attention:
* The child inherits copies of the parent's set of open file
descriptors. Each file descriptor in the child refers to the same open
file description (see open(2)) as the corresponding file descriptor in the
parent. This means that the two descriptors share open file status
flags, current file offset, and signal-driven I/O attributes (see the
description of F_SETOWN and F_SETSIG in fcntl(2)).
Since the open file descriptors are inherited, it is probably a good idea to
flush them before forking.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
The from_md5 and to_md5 fields were a nice extra safety, which would avoid
trying to apply deltas on corrupted package files. However, they are not
strictly necessary, since xdelta should be able to detect that on its own.
The main problem is that it is impossible to compute these informations from
the delta only. So repo-add would not be able to compute the delta entry
based on just the delta file.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Don't prompt the user for unignore of IgnorePkg/IgnoreGroup packages,
except for packages explicitly listed for sync by the user. This
eliminates many unnecessary prompts when IgnorePkg/IgnoreGroup is
used.
Signed-off-by: Bryan Ischo <bryan@ischo.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Enabled a new prompt to ask the user if they'd like to remove
unresolvable packages from the transaction rather than failing it.
Many pactest tests that used to fail now return success codes, because
pacman now issues a prompt allowing the user to cancel rather than
failing many transactions, and the pactest scripts always choose to
cancel with no error rather than failing. The only net effect is that
the return status of pacman is now 0 in cases where it used to be
nonzero.
Signed-off-by: Bryan Ischo <bryan@ischo.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This change reorganizes the internal code so that packages are
resolved one at a time instead of all at once from a list. This will
allow a future checkin to prompt the user to see if they'd rather
remove unresolvable packages from the transaction and continue, or
fail the transaction. This change does not affect the actual behavior
of libalpm and all tests pass without changes.
Signed-off-by: Bryan Ischo <bryan@ischo.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
We don't need to create the directories when local or sync dbs are
registered. For example, if a sync db does not exist, we cannot even do
"pacman -Q" as an user.
Instead, we can create the local db if needed during the db_prepare
operation, and sync dbs on db_update.
Also remove some more useless abstractions in db_update and switch to a much
more efficient way to remove a sync db : rm -rf.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
These db_open and db_close looked quite useless. And they caused the db
directory to be opened on a simple registering of a database. This is
totally unneeded, this opening can be delayed to when we actually need it.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Instead of appending the prefix to each entry name, we can chdir to the
prefix before extracting, and restoring when it is done.
This seems to work better with the strange and special case of FS#12148
where an archive contained the "./" entry.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
This patch fixes FS#12148 ('unstable' regular file).
I also changed the other archive_entry_set_mode usage in add.c to
archive_entry_set_perm.
Since I cannot find any relevant info in libarchive manual, I quote
Tim Kientzle (the author of libarchive) here, and I say thank you for
his help.
*** Tim Kientzle wrote *************************************
This is the problem in libalpm/util.c:
323 if(S_ISREG(st->st_mode)) {
324 archive_entry_set_mode(entry, 0644);
325 } else if(S_ISDIR(st->st_mode)) {
326 archive_entry_set_mode(entry, 0755);
327 }
Your example unstable.db.tar.gz is not empty. It has
one entry in it, called "./". That entry is marked
as a directory. But, when you call archive_entry_set_mode(),
you are changing the file type! archive_read_extract()
then creates the file /var/unstable as you requested.
(archive_read_extract() will replace an empty directory
with a file.)
You should either set the mode value correctly:
323 if(S_ISREG(st->st_mode)) {
324 archive_entry_set_mode(entry, IFREG | 0644);
325 } else if(S_ISDIR(st->st_mode)) {
326 archive_entry_set_mode(entry, IFDIR | 0755);
327 }
Or use archive_entry_set_perm(), which does not change
the file type:
323 if(S_ISREG(st->st_mode)) {
324 archive_entry_set_perm(entry, 0644);
325 } else if(S_ISDIR(st->st_mode)) {
326 archive_entry_set_perm(entry, 0755);
327 }
************************************************************
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Clearly the old code was more elegant (NULL cache indicated "not loaded"),
but it had some drawbacks, so from now on we indicate the state of caches
explicitly.
Old drawbacks:
When we had an empty database (unstable), libalpm called db_populate after
every pkgcache access, because NULL pkgcache indicated "not loaded" state.
This is not a common case, but the same situation can happen with grpcache,
which is more problematic: If the user had a custom repo with no groups,
grpcache was always NULL. (grpcache is also loaded per database.) Thus
every get_grpcache call induced a load_grpcache operation, so the benefits
of grpcache was completely lost.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch introduces the following function name convention:
_compute_ in function name: the return value must be freed.
_get_ in function name: the return value must not be freed.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
The HoldPkg feature is even more important when the packages to be held are
pulled automatically by pacman, in a -Rc and -Rs operation. Before, it only
applied when the packages were explicitly requested by the user to be
removed. This patch extends holdpkg to -Rc and -Rs by doing the HoldPkg
check just before trans_commit.
Additionally, the whole HoldPkg stuff was moved to the front-end.
I changed the default behavior to "don't remove", so I modified remove030.py
pactest as well.
See also: FS#9173.
Original-work-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Dan McGee <dan@archlinux.org>
Aaron said to consider libdownload a dead project so libdownload support was
removed to more easily fix libfetch one (otherwise many ifdef needed).
There was no direct replacement for ferror to detect an error while
downloading. So instead, I added a check at the end to see if the file was
fully downloaded, which is just a small chunk of code taken from here:
http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/net/libfetch/files/fetch.c?only_with_tag=MAIN
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
