Holy inefficient batman! For a pacman -Qt operation (when we are using
compute_requiredby and not database entries), splitdep was being called ~1.3
million times on my local database. By splitting when we read the DB, we
drop this number to around 1700 and save a LOT of time in doing so (a 5x
increase in pacman -Qt speed here).
Note that the depends alpm_list_t in the package struct is no longer a
string list, but a list of pmdepent_t objects.
Signed-off-by: Dan McGee <dan@archlinux.org>
Instead of using the often-busted REQUIREDBY entries in the pacman database,
compute them each time they are required. This should help many things:
1. Simplify the codebase
2. Prevent future database corruption
3. Ensure when we do use requiredby, it is always correct
4. Shrink the pmpkg_t memory overhead
Signed-off-by: Dan McGee <dan@archlinux.org>
This option acts as if IgnorePkg was set on each package in the group.
This closes FS#1592.
Signed-off-by: Nathan Jones <nathanj@insightbb.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Commit 47622eef4d introduced localized times
in the metadata by way of storing the UNIX epoch value instead of a hard
coded date string. However, it missed a few things:
* If we weren't in the C/POSIX/en_US locale, the date parsing would fail
as it tried to use the abbreviations of the locale being used. Fix this
by switching the LC_TIME value before we parse a date.
* We used ctime to print the date value, which is always the C locale
string. Instead, use strftime to print a localized date string.
Signed-off-by: Dan McGee <dan@archlinux.org>
These macros take the place of the common 4 or 5 line blocks of code we had
in most places that called malloc or calloc. This should reduce some code
duplication and make memory allocation more standard in libalpm.
Highlights:
* Note that the MALLOC macro actually uses calloc, this is just for safety
so that memory is initialized to 0. This can be easily changed in one
place.
* One malloc call was completely eliminated- it made more sense to do it
on the stack.
* The use of RET_ERR in public functions (mainly the alpm_*_new functions)
was standardized, this makes sense so pm_errno is set.
Signed-off-by: Dan McGee <dan@archlinux.org>
The code didn't match the following comment :
"A depends on B through n depends <=> A listed in B's requiredby n times"
It stopped at n=1 with a break.
I was surprised to see this case happens in real, that's how I noticed the
bug: wine depends on both freeglut and glut, while freeglut provides glut.
So when installing wine, the update_depends function listed wine twice in
freeglut's requiredby. But the compute_requiredby function (used when
installing freeglut, and used by testdb) listed wine only once in
freeglut's requiredby. That made testdb unhappy.
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Some of the error messages in _alpm_pkg_load failed to use the pkgname
value when printing, which made error messages rather hard to decode.
Signed-off-by: Dan McGee <dan@archlinux.org>
Packages and DBs now support using the UNIX epoch (seconds since Jan 1, 1970)
for use in builddate and installdate. This will only affect newly built
packages. Old existing packages with the text format are still supported, but
this is deprecated.
In the case of removal of text time support, this code will fail gracefully,
returning the start of the epoch for broken packages.
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
In most cases, we want to fully scan a package when we load it, which serves
as a integrity verification check. However, there are times when it is only
desired to read the metadata and nothing else, so allow the caller of pkg_load
to choose the behavior they need.
This pays big dividends in speeding up pacman cache cleaning functionality.
Old (729 packages):
real 1m43.717s
user 1m20.785s
sys 0m2.993s
New (729 packages):
real 0m25.607s
user 0m19.389s
sys 0m0.543s
Signed-off-by: Dan McGee <dan@archlinux.org>
Previously, package names must match a specified scheme or they will cause
pacman add operations to fail. This is not a very intelligent or necessary
way to act, so remove the dependency on the name of the package to be
installed and read all relevant information from the metadata instead.
This does have one causality to be addressed later- pacman cache cleaning
functionality, which has never been phenomenal, just lost most capability.
Signed-off-by: Dan McGee <dan@archlinux.org>
Add some alpm functions for getting optdepends, have makepkg include them
in the PKGINFO file, and have a pacman -Qi operation display the raw string
as stored by libalpm.
Signed-off-by: Dan McGee <dan@archlinux.org>
installdate should never be present in a package descfile, so get rid of it.
With the last commit, we also don't need the util strtoupper function.
Signed-off-by: Dan McGee <dan@archlinux.org>
This fix FS#8081. The tr_TR locale has known issue with case insensitive
comparisons, mostly because upper(i) != I.
So the .PKGINFO files generated by makepkg MUST contain all keywords in lowercases now.
This was already done, but was not mandatory.
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
This allows us to remove the hack in the frontend where we added a newline
to everything coming out of the pm_printf functions, and instead let the
developer put newlines where they want them. This should be the last hangover
of that auto-newline stuff.
Signed-off-by: Dan McGee <dan@archlinux.org>
A bunch of changes related to my first "real" install of pacman-git into
/usr/local and trying to use it.
* Shift some uses of free -> FREE in libalpm.
* Move stat and sanity checks of config paths into libalpm from the
config and argument parsing in pacman.c.
* Fix issue where dbpath still was not defined early enough due to its
requirement for being used in alpm_db_register. This should be rewritten
so it doesn't have this dependency, but this will work for now.
Signed-off-by: Dan McGee <dan@archlinux.org>
This patch cleans up and fix the _alpm_trans_update_depends function
and ensure that all requiredby fields are filled in case of multiple satisfiers
(previously, the handling of mutliple satisfiers in that function was inconsistent).
This makes a special case handling of requiredby in commit_single_pkg() obsolete,
and so allows cleaning that code as well.
Also fixed upgrade056 pactest because :
1) the requiredby fields were wrong, and this wouldn't happen with the fixed _alpm_trans_update_depends().
2) this is a very unusual case anyway (and handling all corner cases combined to a broken database
seems nearly impossible to achieve).
References :
http://www.archlinux.org/pipermail/pacman-dev/2007-July/008919.htmlhttp://www.archlinux.org/pipermail/pacman-dev/2007-July/008920.html
Signed-off-by: Chantry Xavier <shiningxc@gmail.com>
In order to best resolve bug 6404, move some cachedir handling stuff out of
sync.c and into util.c and create two new functions: filecache_find and
filecache_setup. sync.c was rewritten to use these, and alpm_fetch_pkgurl
now also uses these routines.
Signed-off-by: Dan McGee <dan@archlinux.org>
There's no need for a second hashing algorithm. MD5 serves the purpose
of verifying that a package file hasn't been corrupted during download.
Signed-off-by: Andrew Fyfe <andrew@neptune-one.net>
Signed-off-by: Dan McGee <dan@archlinux.org>
We were using a void *data element in pmpkg_t before, which is unsafe by its
nature of being untyped. Reimplement data as origin_data being a union that
can hold either a path to a package file or a pointer to a cache database,
and make the other necesary updates in the code to reflect this. See package.h
for details.
Signed-off-by: Dan McGee <dan@archlinux.org>
Remove the commented desc_localized stuff, we can find it later in version
control. Also remove some unnecessary includes of the stat header and
use -fstack-protector-all which is a bit more broad.
Signed-off-by: Dan McGee <dan@archlinux.org>
Remove versioncmp.c by moving all functions to locations that make sense.
Move replacement functions (for building without glibc) into util.c where
they belong, and do proper checks for them instead of using __sun__, etc.
Signed-off-by: Dan McGee <dan@archlinux.org>
Remove unused buildtype field from pmpkg_t struct and anything associated
with it, as it is unused at the moment. If we need to readd it, it is an
easy revert of this commit.
Signed-off-by: Dan McGee <dan@archlinux.org>
There is no real reason to burden our translators with these messages, as
anyone helping to debug these will probably want them in English.
Signed-off-by: Dan McGee <dan@archlinux.org>
This should hopefully allow multiple cache dirs to be specified in
pacman.conf and/or on the command line, and allow pacman to test
each one for the package file. The first one found to be writeable is
used as the download cache.
Signed-off-by: Dan McGee <dan@archlinux.org>
Add some 'const' keywords all over the code to make it a bit more strict on
what you can and can't do with data. This is especially important when we
return pointers to the pacman frontend- ideally this would always be
untouchable data.
Signed-off-by: Dan McGee <dan@archlinux.org>
alpm.h is the only "publically viewable" file, so there is no reason to have
functions in alpm.c that belong in package.c, db.c, etc. Move the functions
where they belong and leave only the library init functions in alpm.c.
Signed-off-by: Dan McGee <dan@archlinux.org>
This change allows us to use all autoconf specified paths, most notably
$(localstatedir). It is quite a change and touches a lot of files, as
all references to the DB and cache were done with the ROOTDIR as a prefix.
* add --lock command-line option to pacman to specify the location of the
lockfile (this can now be specified at configure time by setting the
$localstatedir path).
* Rip quite a few settings out of configure.ac as they are now picked by
setting the paths during configure or make.
* Fix bug with /tmp fallback for sync downloads not working correctly
(related to root location, now the system tmp dir is used).
* Simplified the parameters to some libalpm functions, and added get/set
for the new lockfile option.
* Renamed several of the DEFS to names without the PM_ prefix.
Signed-off-by: Dan McGee <dan@archlinux.org>
Reorder package version checking so there is no output when a
package is listed in IgnorePkg.
Closes FS#7111
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
The old function did too much on its own; let alpm_depcmp do the hard
work. This will allow for future versioned provisions if necessary.
Signed-off-by: Dan McGee <dan@archlinux.org>
We had many unnecessary casts, most of them dealing with malloc and
other memory allocations. The variable type should take care of it;
no need to do it explicitly. In addition, I caught a const error while
removing the casts.
Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove some unnecessary conditional compiling in util.h- move the
functions tha required it to trans.c (along with a bunch of new header
includes).
* Clean up util.h a bit- remove some header includes, remove universal
libarchive include and only put it in the files that need it.
Signed-off-by: Dan McGee <dan@archlinux.org>
Remove inclusion of libintl.h from all files, because we can do it once
in util.c where the _() macro is defined.
Signed-off-by: Dan McGee <dan@archlinux.org>
reordering and adding ones that were forgotten (noticed when trying to
compile after reordering).
* Updated the HACKING file to include information on #include usage.
* print -> vprint in "making dir" function in pactest.
it. Eventually we'll make progress.
* Rewrote the _alpm_splitdep function to behave more like all our other
function calls. Use heap instead of stack allocation for the depend struct,
so now it needs to be freed by the caller.
* Moved entirely to alpm_pkg_get_* accessors, to read data on demand
* Mostly removed the INFRQ_ parameters from outside the be_files backend (making
the backend more extensible in the long run)
* packages created from _alpm_db_scan now have the db and origin set (making
accessors actually work for these packages)
* removed _alpm_db_ensure_pkgcache
* totally revamped the _alpm_checkconflicts function, making it cleaner and
easier to read (and thus fix in the long run) - maintainable code ftw
NOTE: feel free to rename the functions... I couldn't think of anything better
* removed an extra loop in sync.c:find_replacements - no sense in looping over
an entire DB while strcmp'ing the name, when we have get_pkgfromcache
Other:
* package struct "license" -> "licenses"
* Created _alpm_sync_find (duplicate code in some places, find_pkginsync
* Minor const correctness changes along the way
* fixed a couple extra '/' pathing issues (non-issues really)
* removed a duplicate pkg_cmp function
allows for the same bit operators to be used across the board on pkg_get
operations.
* Changed name of INFRQ_NONE -> INFRQ_BASE to more clearly reflect what it
does (loads pkg name and version).
* Added a few missing things on package functions, such as SYMEXPORT and
ALPM_LOG_FUNC.
* Slight updates to pmenv to print 'pass' and 'fail' instead of 'passed' and
'failed'. Keeps output a bit more concise.
* Fixed a doxygen comment spelling error. :P
about bigger DB changes later, but lets not screw anything up for release.
* Removed some weird uses of "not ... ==" usage in pactest- correct me if I'm
wrong, but isn't "!=" a lot more clean and concise?
* Print description of failed tests in the pactest summary. This could get
dirty with a lot of failed tests though, so watch out.
* Unified some functions names "package" -> "pkg" for consistency
* Removed the goofy 'faketarget' stuff used for dep testing
* Renamed alpm_pkg_isin -> alpm_pkg_find
* Renamed alpm_db_readpkg -> alpm_db_get_pkg
* Fixed a segfault in the conflict checking code
* Added an automatic failure in the case of -A/-U for a replacement of an
existing package. This requires a large amount of work and is postponed for
now. Example:
If ncmpc is installed, pacman -U /path/to/ncmpc-svn.pkg.tar.gz will fail
with and appropriate error message
- the code should be clearer, more organized, commented, and have worthwhile
variable names now
- proactive backup=()s now work. That is, adding a file to a backup array
does what it should on the upgrade to that package, no longer forcing you to
wait a full upgrade cycle for it to take effect
* ldconfig was being run twice on an upgrade operation - fixed
* fixed another pm_fprintf/printf output corruption with the progress bars
* refactored some duplicate code for adjusting 'requiredby' lists
* Added config.rpath to .cvsignore
unbelievable amount of strcmp() calls (25 million) due to the list searching.
This has been reimplemented with a set-intersection scheme, due to the fact
that file lists are always ordered. - NEEDS TESTING
* Minor clean up, "globalized" the str_cmp helper to match the alpm comparison
signature, so we can use it elsewhere.
descriptions, nor do we actually build them with the newest makepkg. The actual
implementation in here seems like it could be done better, and it soaks a large
amount of CPU time according to profilers.
from the PKGINFO, and size is the stat() size of the archive
* Removed the useless 'output' param from package.c:parse_descfile
* Installation progress
- Call progress callback once at 0% for initialization
- 'needdisp' was useless
- alpm_list_count is called an excessive amount in these nested loops. Now we
only call it once per iteration
- Use the compressed sizes for PROGRESS calcs as uncompressed (isize) is not
exact (it is missing metadata sizes), and thus produces > 100% numbers
packages to be upgraded in a -Su operation. Much of the code is duplicated from
sync.c.
TODO: move the implementation to upgrades.c, and reimplement the sync_sysupgrade
function in terms of this:
trans->packages = alpm_get_upgrades();
operations (now aborts on a corrupt archive)
* Fixed the pm_fprintf newline error that was plaguing us. It seems a line
resetting 'neednl' was removed a while back (by me). This causes all the
output errors we've been seeing
* Added alpm_list_remove_node for single list node removal
* Proper error checking/output for failed db_read/db_write (missing files)
* Invalid packages (missing files) are now removed from the package cache
* -Qs and -Ss output now look the same
* config.rpath causes errors on one machine I had, so I added it to CVS
* Fixed a "clobbered memory" issue when installing groups - only the outer list
should be free'd, not the contained data
optimizations in order to make the code much more readable and
type-checkable. Every enum in the library now has it's own type that
should be used instead of the generic 'unsigned char'. In addition,
several #define statements dealing with constants were converted to
enums.
Signed-off-by: Dan McGee <dpmcgee@gmail.com>
* renamed pmlist_t -> alpm_list_t
* made alpm_list_t a public type (alpm_list.h header)
* removed additional storage for registered DBs in pacman source
* some code cleanup
* removed duplicate (pm)list_display functions from pacman source
* misc code cleanup
* Removed some unnecessary headers and library links
* Made things static if possible
* Cleaned up makefiles a bit
* Fixed some old comments in the code
* Fixed some errors the static code checker splint pointed out
* Backwards arguments in a memset call in _alpm_db_read (could have been worse)
* Other various small fixes
Other:
* Default to 80 columns when getcols cannot determine display width
* Removal of ._install as a valid install file in packages
* Modified some dependancy checking
* Changed "performing local database upgrade" message to be more clear
* Change 'usize' to 'isize' in database files
* Scriptlet output is now sent to pacman's log file
* Limited some debugging output to be more clear
* libalpm api changes - move from a _getinfo(p, WHAT_WE_WANT) scheme to a
typesafe _get_what_we_want(p) scheme [not 100% complete yet]
* some const correctness changes
* removal of PM_* types in alpm.h in favor of the pm*_t types used throughout
libalpm
* Addition of hacky architecture check in the _splitname function
* Removal of libfetch from the archlinux proper - it has been renamed to
libdownload and can be found at http://phraktured.net/libdownload
* Merge of _some_ of the Frugalware makepkg change - this may still be
incomplete
* Removal of libftp from cvs proper
* PKGBUILD manpage now says 'PKGBUILD' instead of FrugalBuild (he he)
* Furthered the "lazy caching" to force the pkgcache to read nothing
(INFRQ_NONE) by default. Anything requiring package data should now check
the infolevel of each package and attempt to update it. This could be
ironed out a bit more later (by using the front-end get_info function
* Switched to libfetch. Drastic changes to the download code and the callback
progress bar functions. Also fixed the return value of
_alpm_downloadfiles_forreal. Downloading now supports http, ftp, https, and
files urls, along with 'mtime's and numerous other fancy features from
libfetch.
* Removed the PMList typedef, in favor of the same naming scheme other
structs use 'pmlist_t'
* Added a time stamp on debug output, to make it more informational
* Moved alpm_db_register to _alpm_db_register, making the public function
not take a callback parameter
