----------- Introduction, copyright, license debdelta is an application suite designed to compute changes between Debian packages. These changes (that we will call 'deltas') are similar to the output of the "diff" program in that they may be used to store and transmit only the changes between Debian packages. This suite contains 'debdelta-upgrade', that downloads deltas and use them to create all Debian packages needed for an 'apt-get upgrade'. debdelta is Copyright (C) 2006-2011 Andrea Mennucci debdelta is free software. See the file COPYING for copying conditions. debdelta uses 'minizgip', that is a simplified version of /usr/share/doc/zlib1g-dev/examples/minigzip.c.gz minigzip is released with a permissive license, (see minigzip.c , or debian/copyright) minigzip is Copyright (C) 1995-2002 Jean-loup Gailly ---------- Description The debdelta application suite is really composed of different applications. ---- debdelta 'debdelta' computes the delta, that is, a file that encodes the difference between two Debian packages. Example: $ a=/var/cache/apt/archives $ debdelta -v $a/emacs-snapshot-common_1%3a20060512-1_all.deb $a/emacs-snapshot-common_1%3a20060518-1_all.deb /tmp/emacs.debdelta the result is: deb delta is 12.5% of deb ; that is, 15452kB would be saved ---- debpatch 'debpatch' can use the delta file and a copy of the old Debian package to recreate the new Debian package. (This process is called "applying the delta file"). If the old Debian package is not available, but is installed in the host, it can use the installed data; in this case, '/' is used in lieu of the old .deb. Example: $ debpatch -A /tmp/emacs.debdelta / /tmp/emacs.deb ----- debdeltas 'debdeltas' can be used to generate deltas for many debs at once. It will generate delta files with names such as package_old-version_new-version_architecture.debdelta and put them in the directory where the new .deb is. If the delta exceeds ~70% of the deb, 'debdeltas' will delete it and leave a stamp of the form package_old-version_new-version_architecture.debdelta-too-big Example usages are in the man page; see also the scripts /usr/share/debdelta/debmirror-delta-security and /usr/share/debdelta/debmirror-deltas ----------- debdelta-upgrade This command will download necessary deltas and apply them to create debs for an 'apt-get upgrade' . The deltas are available for upgrades in 'stable' , 'stable-security' , 'testing', 'unstable' and 'experimental', for i386 and amd64. Example usage: # apt-get update && debdelta-upgrade && apt-get upgrade If run by a non-root user, debs are saved in /tmp/archives : do not forget to move them in /var/cache/apt/archives debdelta-upgrade will also download .debs for which no delta is available (this is done in parallel to patching, to maximize speed). See the explanation of "debdelta-upgrade --deb-policy" in the man page for more info and customization on which debs get downloaded. More info are in README.upgrade ------------- TODO Prepare an APT method so that 'apt-get upgrade' would actually use deltas. Some code is already written. It would be nice if debdelta-upgrade would actually choose if - download a delta and use it to create the .deb - download the deb depending on which one would be faster. Unfortunately, this decision must depend on a good model to predict the speed of patching... and this I still cannot achieve. -------------- Statistics and tests Tests and comparisons on backend binary delta difference compressors are available in http://debdelta.debian.net/run/tests see README.txt Logs of the server that creates deltas for 'debdelta-upgrade' are in http://debdelta.debian.net/run/log ------------ exact patching and exact recompression When debdelta recreates a .deb, it must be identical to the desired one (otherwise APT will refuse it when checking signatures). Suppose a .deb has inside a huge file /usr/share/doc/foobar/document.info.gz and this starts with a RCS tag ... then each time it is released, the file will be different even though just few bytes were changed. Another examples are manpages that start with the header containing the version of the command. So , to get good compression of the difference, I had to be able to gunzip those files, diff them, and gzip back them *exactly identical*. For this reason, I studied gzip formats, and I wrote in debdelta some python code that does the trick (90% of the times...). -------------- Speed Warning: this section is referred to experiments where the backend for delta encoding was 'xdelta'. On a desktop with CPU Athlon64 3000 and a average hard disk, $ debdelta mozilla-browser_1.7.8-1sarge3_i386.deb mozilla-browser_1.7.8-1sarge6_i386.deb /tmp/m-b.debdelta processes the 10Mb of mozilla-browser in ~11sec, that is a speed of ~900kB per second. Then debpatch applies the above delta in 16sec, at a speed of ~600kB per second. Numbers drop in a old PC, or in a notebook (like mine, that has a Athlon 1600MHz and slow disks), where data are chewed at ~200kB per second. Still, since I have a ADSL line that downloads at max 80kB per second, I have a benefit downloading deltas. In a theoretical example, indeed, to download a 80MB package, it would take 1000seconds; whereas to download a delta that is 20% of 80MB it takes 200seconds, and then 80MB / (200kB/sec) = 400seconds to apply it, for a total of 600seconds. So I may get a "virtual speed" of 80MB / 600sec = 130kB/sec . Note that delta downloading and delta patching is done in parallel: if 4 packages as above have to be downloaded, then the total time for downloading of full debs would be 4000seconds, while the time for parallel-download-patch-apply-patch may be as low as 1400seconds. This is a real example of running 'debdelta-upgrade' : Looking for a delta for libc6 from 2.3.6-9 to 2.3.6-11 Looking for a delta for udev from 0.092-2 to 0.093-1 Patching done, time: 22sec, speed: 204kB/sec, result: libc6_2.3.6-11_i386.deb Patching done, time: 4sec, speed: 57kB/sec, result: udev_0.093-1_i386.deb Delta-upgrade download time 28sec speed 21.6k/sec total time: 53sec; virtual speed: 93.9k/sec. (Note that the "virtual speed" of 93.9k/sec , while less than the 130kB/sec of the theoretical example above, is still more than the 80kB that my ADSL line would allow). Of course the above is even better for people with fast disks and/or slow modems. Actually, an apt delta method may do a smart decision of how many deltas to download, and in which order, to optimize the result, (given the deltas size, the packages size, the downloading speed and the patching speed).