debdelta

What

debdelta is an application suite designed to compute changes between Debian packages. These changes (that we will call 'deltas') are similar to the output of the "diff" program in that they may be used to store and transmit only the changes between Debian packages. This suite contains 'debdelta-upgrade', that downloads deltas and use them to create all Debian packages needed for an 'apt-get upgrade'.
Debdelta is available as a Debian package. Debdelta is written and mantained by this guy. The source is made available using git, use

 git clone http://debdelta.debian.net/debdelta-suite.git

to obtain a copy. The source code is copyright A Mennucci 2006-2011 and licensed GPL-v2.

Documentation

• A quick FAQ for starting.
• The general idea and usage is in the documentation

debdelta-upgrade service

There is a server that generates deltas for upgrades of packages in the Debian repository archives.
The server generates deltas by looking at the current state of the archives and at the state before (precisely, up to 40snapshots before, ~10days) and it generates all deltas needed to upgrade. It also (daily) deletes all deltas that would upgrade to a package version that is not present in the archive any more, and all deltas older than 50 days. (The script that manages all this is /usr/share/debdelta/debmirror-marshal-deltas, distributed in debdelta >= 0.41).
Summarizing, an end user has a window (of 10 to 50 days) of opportunity to upgrade using debdelta-upgrade.
More info specific to 'debdelta-upgrade' are in the documentation.

To download deltas, debdelta-upgrade looks into /etc/debdelta/sources.conf; this is what is available at each stanza (as of Mar 2011).

• The stanza

  [main debian archive]
  Origin=Debian
  Label=Debian
  delta_uri=http://debdeltas.debian.net/debian-deltas
  

  will provide deltas for upgrading of
  • distributions=squeeze squeeze-updates wheezy sid experimental
  • architectures=i386 amd64
  • sections=main contrib non-free
  (As of Mar 2011, these deltas amount to 6.5GB of data).
• The stanza

  [stable security debian archive]
  Origin=Debian
  Label=Debian-Security
  delta_uri=http://debdeltas.debian.net/debian-security-deltas
  

  refers to deltas for security updates of Debian/stable (currently Squeeze) for the architectures i386,amd64.

debdelta-upgrade service news

• Oct 2012 : Marco was so kind as to send me 3 months of apache logs from debdeltas.debian.net. Some statistics.
  People using "debdelta-upgrade" downloaded approximatively 17860 MB / month of deltas. In 3 months they used 2744 unique IP addresses; they asked for ~561000 deltas, but ~198000 times the delta was not present (that is 35%, I should investigate this, yes).
  People using "cupt" downloaded approximatively 1060 MB / month of deltas; in 3 months they used 140 unique IP addresses.
• Jun 2012 : unfortunately ftp.linux.it AKA debdeltas.debian.net suffers intermittent unknown problems
• 15th June 2012: the UPS died; now the server is on its own
• 14th June 2012: I found a bug in the XZ parameters autodetection; now the server fails only on 12/22315 deltas (8 of them are dbg files, that are too large to process in my small server)
• 13th May 2012: the dates inside the server log are UTC, but the filename of the log was using CEST for the date part: corrected it, now it is all UTC.
• Apr 2012: added a new external disk for the ever-growing mirror (unfortunately it seems to be a bit unstable... sometimes it hangs and needs a power-cycle)
• Oct 2011: a new battery in the UPS should improve reliability
• 24-28th Aug 2011: added support for data.tar.xz (only when default compression is used); it is in 0.44
• Aug/Sep 2011: some days no deltas were generated , due to repeated blackouts, network problems, etc etc.
• 26th Jun 2011: the Squeeze point release filled the server HD, so on 26th no delta was generated; I found some additional space, and now it is back on track (for the record: to catch up, the server ran for 5hours, and processed 16GB of data)
• 7th Jun 2011: corrected a bug when dealing with doubly compressed files (debdelta would uncompress them once and pass them to xdelta3 that then would uncompress them again, and recompress them on arrival introducing some small difference)
• May 2011: enlarge time window to 40 snapshots
• 7th May 2011: New documentation.
• the debdelta repository has a new home, thanks to 'md'
• 24th Apr 2011: some interesting ideas for the future of debdelta
• 18th Apr 2011: I found out that the server debdeltas.debian.net is not really mirroring the whole repository of deltas , so I am looking for a solution.
• 13th Mar 2011: added squeeze-updates
• 12th Mar 2011: The change from gcc 4.4 to gcc 4.5 has adversely affected the quality of deltas: binaries and libraries cannot be effectively delta-ed, and so the deltas are quite large. See for example the bibledit package
• Mar 2011: after the release of Squeeze, a change in the archives' signatures broke 'debmirror' (see bug #614383); and the new debmirror 2.6 broke my 'trash' patch that I was using to create deltas; I eventually rewrote the server scripts to use the new 'debmirror --marshal' method.
• Mar 2011: after the release of Squeeze, the 'debdelta for security' service is generating deltas for security updates for Squeeze only, and not for Lenny. If you miss it, please drop me a note.
• 23 Sep 2010: due to a combination of two problems, no deltas where generated since 15th Sep. Now it should work.
• 17 July 2007: a change in zlib1g breaks debdelta, needs workaround
• 31 May 2006: announcement of debdelta and debdelta-upgrade in debian-devel
• 20 May 2006: first release

The server that creates the deltas keeps logs and statistics.