As noted in FS#20498, if an absolute path is used for specifying the
database when invoking repo-add, the symlink generated will point to the
absolute path instead of being relative to the directory. Fix this for
the two linking cases, but leave the copy untouched so that will still
work.
Signed-off-by: Dan McGee <dan@archlinux.org>
We were seeing some issues when trying to create our new database alias
using symlinks on certain filesystems (see FS#19907). Have a fallback method
in place where we first try a symlink, then a hard link, then just copy the
database if all else fails.
Signed-off-by: Dan McGee <dan@archlinux.org>
This is a small step towards allowing pacman to handle databases
with variable compression types.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
FS#16623 suggested this change for makepkg; this patch applies it to the
remaining files in the scripts directory.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Dan McGee <dan@archlinux.org>
This also removes the awk dependency from makepkg and repo-add.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
When redirecting both stderr and stdout and using the 2>&1 construct,
you have to redirect stdout first. Otherwise stderr will be redirected to
the 'old' stdout and not to the new resource.
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
Signed-off-by: Dan McGee <dan@archlinux.org>
-f/--force has been dead for a while, so kill it off. In addition, the
check for > 2 args is pretty useless when you do something like:
repo-add -q -q
or a more legit:
repo-add -q /path/to/mine.db.tar.gz
So instead make repo-add just return 1 when it doesn't do anything with
the database which seems to make more sense.
Signed-off-by: Dan McGee <dan@archlinux.org>
Rather than creating no database at all, create an empty zipped tar archive
in its place. This keeps the download side of repositories a bit more sane
as a DB will always exist, and pacman handles this empty case just fine.
For this to be fully transparent, we also need to make sure repo-add and
repo-remove accept an empty "DB" as an argument, which in reality is a
completely void of files .tar.{gz,bz2,xz} archive.
Signed-off-by: Dan McGee <dan@archlinux.org>
With split packages, the pkgbase variable provides a useful way to
find out which packages were build from the same PKGBUILD. Add it
to the packages .PKGINFO file and the repo database only when
package splitting is used.
Original-patch-by: Pierre Schmitz <pierre@archlinux.de>
[Allan: restrict to including only with spilt packages
and include after pkgname]
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Dan McGee <dan@archlinux.org>
When unzipping packages and the database archives, we don't need to look
through the entire archive to do what we need to do. For packages, .PKGINFO
should only be found once and should be the first file in the package. For
the database check, we only really need to look for one desc file.
The bsdtar -q option is very similar to the GNU tar --occurrence=1 option.
Example of speedup:
$ time repo-add junkdb.db.tar.gz *.pkg.tar.gz >/dev/null
real 0m16.159s
user 0m14.836s
sys 0m2.277s
$ time ./scripts/repo-add junkdb.db.tar.gz *.pkg.tar.gz >/dev/null
real 0m4.949s
user 0m3.730s
sys 0m2.093s
Signed-off-by: Dan McGee <dan@archlinux.org>
Before this commit, the repo creation could fail after all packages have
been added to the database. Now this will be detected before adding
anything.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This simple patch adds support for the xz archive format to makepkg and repo-
add.
Xz can be used as source, package and package db file type.
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
[Dan: fixed a few alignment issues]
Signed-off-by: Dan McGee <dan@archlinux.org>
The modification time on depends and desc file were changed to match the
modification time of the package file. I don't see why and we are actualling
losing information here. If we want to know the date of the package file, we
can just look inside the depends file. If we want to know when the entry was
created, we should not alter the modification time of depends and desc.
Besides, this had the non-obvious and undocumented side effect that the
depends file was always created, even if it was empty. And pacman actually
does require that. So I added a "touch depends" to always create the file.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Weird things could happen if several repo-add were run concurrently on the
same database. The introduced locking system will prevent this to happen.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
- arch was missing
- backup is not used by repo-add. However makepkg still needs to put it in
PKGINFO because pacman uses it
- startdir is no longer used after the new delta implementation
- the declaration of group, depend, backup, etc is not needed because these
variables are always declared before being used :
declare $var="$val"
case "$var" in
group) _groups="$_groups$group\n" ;;
- reorder the variables declaration to follow the same order than they are
written to the depends and desc file, for making future checks easier
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Use the correct database format
Use xdelta3 to get the source and destination files from the delta itself
Allow delta files to be added with repo-add just like package files. delta
files can also be removed with repo-remove. This is simply done by looking
for a .delta extension in the arguments, and calling the appropriate
db_write_delta or db_remove_delta functions.
Example usage:
repo-add repo/test.db.tar.gz repo/libx11-1.1.99.2-2-x86_64.pkg.tar.gz
repo-add repo/test.db.tar.gz repo/libx11-1.1.5-2_to_1.1.99.2-2-x86_64.delta
repo-remove repo/test.db.tar.gz libx11-1.1.5-2_to_1.1.99.2-2-x86_64.delta
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
The current implementation has several problems :
Wrong database format
All the info is taken from the filename, which is a bit ugly
It looks for .delta files in the current directory when adding a package,
which is not very flexible
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
* report when a package entry to be removed is not found
* backup and restore eventual "deltas" files
* slight optimization when looking for an entry : only look at the entries
starting with $pkgname
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Rework slightly db_write_entry so that $pkgfile is no longer referenced
from the temporary dir. This means $pkgfile can be a relative path and does
not need to be converted with realpath anymore.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
REPO_DB_FILE does not need to be an absolute path anymore so no need to
call realpath.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This function was used only once, was basically just one line, and was also
called with an unused argument.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
eval was ugly and dirty, and bit us here. Instead, use a safer form of
variable declaration to ensure quotes don't foil us in pkgdesc or any other
fields.
This fixes FS#10837.
Signed-off-by: Dan McGee <dan@archlinux.org>
Considering one can easily run:
repo-add .... >/dev/null
to get only warnings and errors, the -q flag is mostly useless.
Make the -q flag silence only level 2 messages.
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
This is similar to the change we made in makepkg so it is cross-platform
compatible and doesn't require coreutils.
Signed-off-by: Dan McGee <dan@archlinux.org>
DB_COMPRESSION was only used in repo-add and DB_CHECKSUMS was not used
anywhere.
This also removes the dependency on makepkg.conf in repo-add, so repo-add no
longer needs to source makepkg.conf
And instead of DB_COMPRESSION, it seems better to just check the extension
of the repository file. It does not make sense to have a tar.gz file with a
tar.bz2 extension or whatever.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Commit 149839c539 introduced a small behavior regression as a drawback
for a better portability. repo-add now includes the approximate size (to the
nearest KB) rather than an exact size due to the switching of the du command
to a more portable form. Instead of sacrificing the exact size, use
configure to help us determine a valid command to acquire our filesize and
place it in the sync database.
Signed-off-by: Dan McGee <dan@archlinux.org>
repo-add didn't handle whitespaces nicely in fields value, and this has hurt
us several times, first with provision version (FS#9171) and then with
optdepends (FS#10630), so it is time to fix it.
Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
* change ln -s to ln -sf in the Makefile to prevent a failure when the link
already exists.
* make test_repo_db_file simpler and more natural, move the complexity out
of it.
* remove one $cmd = repo-remove check that wasn't needed
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
They shared about 75% of their code, so there is no real reason we should
maintain them separately. Merge the differences accordingly and add a check
based on the basename of the command used to decide what behavior to follow.
Signed-off-by: Dan McGee <dan@archlinux.org>
Linux coreutils provides readlink, and BSD systems tend to have realpath
available. Both commands provide similar functionality but of course have
different names. Add a check for either and use what is available.
While doing this, also unify some of the differences that have cropped up
between repo-add and repo-remove.
Signed-off-by: Dan McGee <dan@archlinux.org>
This fixes FS#10459.
There is apparently no portable ways to get the apparent size of a file,
like du -b does. So the best compromise seems to get the block size in kB,
and then convert that to byte so that we keep compatibility.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
They are pretty noisy scripts in their normal course of operations, so allow
all messages to be squashed except for warning and error messages with this
new flag.
Signed-off-by: Dan McGee <dan@archlinux.org>
Address the issue of our scripts not working so great when gettext is not
available. This has come up in multiple bug reports, and is relatively easy
to address by adding a simple check and a stub function if gettext was not
found that simply echos the original message.
Addresses concerns from FS#9214 and FS#9607.
Signed-off-by: Dan McGee <dan@archlinux.org>
The force option should only be specified in the PKGBUILD with
options=(force). This information should be handled like any other meta
info, and there is no need to have a special repo-add option for it.
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
[Dan: fix up a few more references in contrib/ scripts, etc]
Signed-off-by: Dan McGee <dan@archlinux.org>
Fix for FS#9176.
A previous commit (6e8daa553b) already forced all database files to 644.
Now the directories are also forced to 755.
Additionally, repo-add now sets the umask to 022, just like makepkg does, to
fix the problem at its root.
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Update the GPL boilerplate to direct people to the GNU website for a copy of
the license, as well as bump all of Judd's copyrights to 2007.
Signed-off-by: Dan McGee <dan@archlinux.org>
repo-add was adding the delta files of any package that started with the
same package name plus a hyphen.
In this example, deltatest-libs delta files were added to the deltatest
package:
$ repo-add deltatest.db.tar.gz deltatest-1.0.2-1-i686.pkg.tar.gz
==> Extracting database to a temporary location...
==> Adding package 'deltatest-1.0.2-1-i686.pkg.tar.gz'
-> Removing existing package 'deltatest-1.0.2-1'...
-> Creating 'desc' db entry...
-> Computing md5 checksums...
-> Creating 'depends' db entry...
-> Creating 'deltas' db entry...
-> Added delta 'deltatest-1.0.2rc3-1_to_1.0.2-1-i686.delta'
-> Added delta 'deltatest-libs-1.0.0-1_to_1.0.1-1-i686.delta'
==> Creating updated database file /tmp/deltatest.db.tar.gz
Signed-off-by: Nathan Jones <nathanj@insightbb.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
