Slashdot Mirror


Use apt-p2p To Improve Ubuntu 9.04 Upgrade

An anonymous reader writes "With Jaunty Jackalope scheduled for release in 12 days on April 23, this blog posting describes how to switch to apt-p2p in preparation for the upgrade. This should help significantly to reduce the load on the mirrors, smooth out the upgrade experience for all involved, and bypass the numerous problems that have occurred in the past on Ubuntu release day. Remember to disable all third-party repositories beforehand."

55 of 269 comments (clear)

  1. Website and Warning by Daengbo · · Score: 4, Informative

    The site doesn't have much information, but other sources I have read state that apt-p2p is very experimental. Use at your own peril!

    1. Re:Website and Warning by drinkypoo · · Score: 4, Insightful

      The site looks badly outdated. The caveat I would add to your warnings is that the upload speed is uncapped by default. You'll want to limit this unless you want the world to be able to leech you hard. If I left this unlimited my ISP would fucking kill me.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    2. Re:Website and Warning by drinkypoo · · Score: 2, Informative

      Easily found from apt-p2p's main page: protocol... please don't ask me to browse the web for you again, kthxbye.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    3. Re:Website and Warning by strstrep · · Score: 2, Informative

      I don't know about Ubuntu, but Debian uses GPG to sign all their packages, so I'd guess that Ubuntu does the same.

    4. Re:Website and Warning by mrsteveman1 · · Score: 2, Informative

      If we're talking about package security, there is already signing of the packages themselves.

      Getting them from a different source shouldn't matter as long as the signing method is secure, and i believe with deb it is GPG so, yea.

    5. Re:Website and Warning by blueg3 · · Score: 2, Informative

      You do realize that there are no extant MD5 or SHA1 attacks that can produce data of a specified length that matches a specified hash, right? (For that matter, there isn't such an attack when the length isn't specified.) You would need such an attack to poison something like BitTorrent with false data.

      (This protocol, and BitTorrent, both use SHA1.)

      The existence of a type of attack on MD5 doesn't even imply that MD5 is rendered useless, much less SHA1. There's only a risk where that type of attack can be employed.

    6. Re:Website and Warning by NFN_NLN · · Score: 3, Funny

      Easily found from apt-p2p's main page: protocol... please don't ask me to browse the web for you again, kthxbye.

      What are you a leprechaun? You're bound by ancient laws to comply with any mortal request, the only loophole being that you can bitch all you want :).

      Just to be sure, can you please post your credit card number, name, address and CID.... oh and naked pictures of your leprechaun wife too and any daughters over the age of 250 (I'm not falling for pedochaun's trap again).

    7. Re:Website and Warning by smallfries · · Score: 2, Interesting

      Good point, I had of course forgotten that the blocks are constant size. That would require a much better attack than those currently available.

      You are aware that there is an attack for MD5 when the length isn't specified though? There is a demo that will produce forged pdf documents with a given md5 hash.

      --
      Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
    8. Re:Website and Warning by Mozk · · Score: 2, Insightful

      In any case, even if somebody could produce a file that matched the checksum, you would have to download the entire file from that one source, as any discrepancies in the data would produce an incorrect checksum. Typically with p2p (though I'm not sure of apt-p2p's particular method), you are downloading from multiple sources, and it's unlikely that they would all have that file unless they themselves were downloading it only from that one source.

      --
      No existe.
  2. Alternate CD by elwin_windleaf · · Score: 5, Informative

    You can also upgrade Ubuntu with an alternate install CD. These can be downloaded via bittorrent, and usually trigger an "automatic update" prompt as soon as they are inserted into an existing Ubuntu system.

    1. Re:Alternate CD by drinkypoo · · Score: 2, Insightful

      That will help a lot, but you're still going to have a lot to get from the mirrors on a typical system. Odds are, many of the packages in the ISO will be outdated by the time you get it :P I'm running apt-get update on my apt-p2p'd system and so far, so good.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    2. Re:Alternate CD by QuantumRiff · · Score: 2, Interesting

      No, as he stated, you can get the alternate disc from bittorrent as well. Then use that to upgrade to 9.04. That would DRASTICALLY reduce the load on the mirrors..

      --

      What are we going to do tonight Brain?
    3. Re:Alternate CD by drinkypoo · · Score: 2, Insightful

      The person who modded you up obviously misunderstood my comment and/or the situation vis-a-vis updating your system in the same way you did: every package on your system is unlikely to be represented on the alternate install CD, and even some of those which are will be outdated by the time you get the ISO, so you will still be downloading numerous packages from the repositories. I didn't say it wouldn't help. You either didn't read my comment, didn't understand my comment, or don't understand the relationship between the repo on the CD and the live repo.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  3. good idea but... by mrphoton · · Score: 5, Interesting

    I had wondered for a while why yum and apt did not do this by default. It would seem a great ideal. However.... I recently tried to down load fedora 11 alpha via bit torrent using a BT internet connection in the UK. It worked great for about 10Mb (@90-100kb/s), then the download speed gradually ground to a halt. (5kb/s) When I tried a direct download of the same iso the speed bumped back up to a steady 100kb/s. I concluded BT was throttling my bit torrent connection of a legal download to a very slow speed.
    So my point is sounds like a great idea but if it is enabled by default it had better have some way to detect bandwidth throttling of p2p networks and revert to http transfer.

    1. Re:good idea but... by nurb432 · · Score: 3, Interesting

      I had wondered for a while why yum and apt did not do this by default. *snip*

      Because it would be wrong to default to forcing a person to share their limited resources.

      --
      ---- Booth was a patriot ----
    2. Re:good idea but... by Jurily · · Score: 2, Insightful

      They are in bed with the RIAA and MPAA, but they give massive bandwidth, and do not block any ports or filter p2p.

      So, they're letting you do whatever you want, to make sure they maximize the amount of money they can sue you for?

    3. Re:good idea but... by FluffyWithTeeth · · Score: 3, Informative

      This isn't how it works in the UK. If BT has phone lines going somewhere, then you have dozens of ISPs to choose from.

      They can be buying direct from BT wholesale, or own anything quite a bit further up the chain. Noone should really be touching the BT consumer ISP for any reason.

    4. Re:good idea but... by Fruit · · Score: 2, Insightful

      It could be that your uploading is killing your download speed. See one of the other comments for instructions on how to limit upload speed if you hadn't already.

    5. Re:good idea but... by turbidostato · · Score: 2, Informative

      Please undo moderation to parent post. Signed packages anyone?

    6. Re:good idea but... by vadim_t · · Score: 4, Informative

      Ubuntu packages are signed. The signature certifies that the package was mirrored as-is and not modified in any way.

    7. Re:good idea but... by Kjella · · Score: 5, Informative

      All packages are signed, the repository is just a convienient way of getting them. If you add a third party repository they usually also ask you to add their public key to the trusted package signers. That's also why you have all the local mirrors - I doubt Canonical operates very many of them. Same thing in companies, set one machine to download and the 100 others to download from the local machine, you don't need to put any trust in that machine as it's just passing signed packages. So you download the package from P2P or whatever, apt checks the signature and if's Genuine Canonical(tm) it'll install the package otherwise it'll complain. Didn't you notice the repositories are all http? No certificates or security checks there, anyone can give you any garbage data but it won't have the right signature.

      --
      Live today, because you never know what tomorrow brings
    8. Re:good idea but... by RalphSleigh · · Score: 2, Interesting

      As far as I can tell, British Telecom's retail ISP (BT) throttle Bittorrent (BT) to around 10 kb/sec down during peak times, but leaving torrents on overnight works well, as they unthrottle around midnight, and I can usually max out my 8Mb/sec ADSL with bittorrent overnight. This is the only limitation I have come across so am pretty happy with them as an ISP.

      --
      Come as you are, do what you must, be who you will.
  4. Slashdotted... by Anonymous Coward · · Score: 3, Informative

    mirror here: http://74.125.77.132/search?q=cache:3gY3Bq4EKnMJ:blog.chenhow.net/os/linux/ubuntu/using-apt-p2p-for-faster-upgrades-from-intrepid-to-jaunty/+http://blog.chenhow.net/os/linux/ubuntu/using-apt-p2p-for-faster-upgrades-from-intrepid-to-jaunty&cd=1&hl=nl&ct=clnk&gl=nl

  5. Deterrent by senorpoco · · Score: 5, Funny

    I have yet to have an Ubuntu distro update smoothly, ever. But that won't stop me, onward I will plunge headlong into it with abandon. I don't like my data anyway.

  6. I'm upgrading to 8.10 by a09bdb811a · · Score: 2, Insightful

    Currently on 8.04, I'll be upgrading to 8.10 sometime after 9.04 is released.

    Staying 6 months behind is a reasonable compromise. Let the lab rats (er, enthusiasts!) debug the new stuff first. Last time I checked 8.10 in a VM there was something like 320MB worth of updated packages.

    As for the packages themselves, run a local apt proxy like approx, especially if you have more than one Debian or Ubuntu system. It keeps a copy of every .deb you download, and automatically purges the ones that are outdated.

    1. Re:I'm upgrading to 8.10 by drinkypoo · · Score: 2, Insightful

      I've never had an Ubuntu upgrade completely kill my filesystem or hard drive.

      "I know what you're asking yourself..."

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  7. Slower to start by Nomaxxx · · Score: 2, Interesting

    I've used apt-p2p as an apt-get replacement for a short time. It often downloads faster than the standard method but is slower to start downloading. So it's not great when you have many small packages to install. But for a full system upgrade I guess it's a good alternative. Especially on (or close to) launch date when you're sure that update manager will go idle midway through the upgrade. Other alternative is to wait for a week or too after release date when servers are less busy.

  8. Good citizenship by AlecC · · Score: 3, Insightful

    What I like about this is not so much the potentially faster upgrade as the ability to contribute a bit to others. The six-monthly upgrades are are rate enough that I don't mind if they are a bit slow - not that they have been. But I am very conscious that I am using other people's freely given bandwidth and I am pleased to be able to give some back.

    Does anybody know if I can force my various machines to cross-peer from each other? If I update one first, I don't want the others searching the Net for peers - they should just copy from the first.

    --
    Consciousness is an illusion caused by an excess of self consciousness.
    1. Re:Good citizenship by Daengbo · · Score: 3, Informative

      You should just set up an apt-cache on one and direct the others to fetch from the first. There are several to choose from. Search for "apt proxy."

  9. Bandwidth usage by Chris_Jefferson · · Score: 5, Interesting

    I'm concerned that after reading the article, and apt-p2p's FAQ page, that I can't find any guide to how much upload bandwidth this thing will use. While I'm all for sharing, I find it important to cap my upload speed so my connection performs well on other stuff I'm doing, and also stop uploading once I'm at 1:1 sharing or so. Some of us pay if we use too much bandwidth!

    --
    Combination - fun iPhone puzzling
    1. Re:Bandwidth usage by Mr_Perl · · Score: 3, Informative

      Can't help you with the paying for extra bandwidth, but the wondershaper has helped my limited speed home network remain responsive during downloads.

      --

      My poetry site welcomes the unusual.
    2. Re:Bandwidth usage by nurb432 · · Score: 3, Informative

      It will obliterate your monthly use cap.

      This mode of distribution only works in a perfect world, which few of us live in now.

      --
      ---- Booth was a patriot ----
    3. Re:Bandwidth usage by stevied · · Score: 2, Interesting

      Just installed it, there's an option in /etc/apt-p2p.conf to limit the upload bandwidth. I haven't tested it yet, however ..

    4. Re:Bandwidth usage by eldepeche · · Score: 4, Funny

      The option is called "not using apt-p2p." I don't remember the exact syntax, but I think there's a switch in the file /etc/apt/this/is/the/default.behavior

  10. No thanks, im no criminal by wjh31 · · Score: 5, Funny

    p2p is a method used exclusively by criminals, there's no way im going to be using this method.

  11. Slashdotted? by drinkypoo · · Score: 5, Informative

    It worked for me. But in case it really is slashdotted here's the story, from memory (let's test those theories eh?)

    1. apt-get install apt-p2p (Not in Hardy and older repos IIRC... for you late/sporadic upgraders)
    2. Back up your /etc/apt/sources.list and then edit the file, s/\/\//\/\/localhost:9977\// (hope I got that right -- Guess I could have just used # or something eh?)
    3. Not in the guide: edit /etc/apt-p2p/apt-p2p.conf and set UPLOAD_LIMIT ... just in case. :) You probably have to /etc/init.d/apt-p2p restart after that.
    4. apt-get update
    5. Then make the update... But it's not time for that yet.
    --
    "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  12. More Linux mirrors needed by Antique+Geekmeister · · Score: 4, Insightful

    Many primary Linux download sites wind up taking an unreasonable amount of traffic from default setups. If you want to contribute back to the OS's and packages that you find so useful, consider setting up a local mirror to share with the world at large. If you can't justify that, at least consider setting up an internal rsync mirror anytime you have a dozen or more boxes to make updates and downloads much faster for your site, and configure your local machines to point to that local mirror.

    This turns out to be especially useful for PXE installaters and cluster setups, for any Linux or other OS. There's nothing like having 100 internal Linux machines all trying to update OpenOffice at the same time from an external primary site, through a corporate DSL line, to ensure that many of the updates will fail.

    1. Re:More Linux mirrors needed by scientus · · Score: 3, Insightful

      its reasonable, but yes you should have one computer set to upgrade and hour before the rest, and large deployments could use a local mirror. Its smart unlike a regular proxy server.

  13. Why upgrade? by wiresquire · · Score: 4, Insightful

    If it works, why upgrade at all?

    Ubuntu 8.04 is a Long Term Support (LTS) release. It will have any security patches until the next LTS release, which is typically every 18 months. So, why not just wait for 9.10?

    ws

    --

    So does Anonymous Coward have good karma?

    1. Re:Why upgrade? by Aladrin · · Score: 2, Informative

      For the same reason that you'll upgrade to 9.10 instead of waiting for 11.04: Features.

      Sure, it'll have all the bugfixes for years, but it won't have any of the new features.

      (In case anyone has forgotten, LTS are supported for 3 years on the desktop, so there's no 'need' to upgrade every 18 months.)

      --
      "If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
    2. Re:Why upgrade? by WormholeFiend · · Score: 4, Funny

      AFAIK, women prefer men who have all the latest upgrades

  14. If you want to get the new release when it's hot by badpazzword · · Score: 2, Interesting

    I honestly suggest to upgrade when the RC is out (1). That's one week before the actual release date, or in other words Thursday. FYI, when I upgraded to the Alpha 6 I had to download 1.3 GBs; torrenting as much is still going to take a lot of time.

    The Release Candidate is typically identical to the "gold" release; also you will help Canonical in testing everything runs as good as it should. If you install apt-p2p (2) you'll even get the warm fuzzy feeling of being a seed for the new packages. :D

    The upgrade process is identical -- the only difference is in starting it. Hit Alt-F2 and use "update-manager -d" then hit "Upgrade".

    (1) Or hell, upgrade /right now/. I'm using the beta and it is rather stable and experience tells me the beta is always pretty near to what goes gold.
    (2) I wouldn't use apt-p2p to upgrade to a dev version as you will find far less peers. However installing it afterwards should let you act as a seed for those packages.

    --
    When ideas fail, words become very handy.
  15. Irony by digitalderbs · · Score: 4, Informative

    that a site advising the use of p2p to prevent the meltdown of servers has itself been slashdotted.

    On a side note : web data and pages themselves could be p2p distributed too, no? Say a peer gets a webpage's hash (containing html and images) and the date/time of expiry for a webpage from a server. If other peers have that page (html+images), and it's up to date, you could download their copy. Otherwise, the server sends a fresh copy to you, and you seed it for others. Not being in computer science, I'm sure this has been proposed before and that there are glaring shortcomings I have missed.

    1. Re:Irony by slashdotmsiriv · · Score: 2, Informative
  16. Partitions are your friend by mangu · · Score: 3, Insightful

    I have yet to have an Ubuntu distro update smoothly, ever.

    Me too. Often it's quicker to do a full install from scratch.

    But that won't stop me, onward I will plunge headlong into it with abandon. I don't like my data anyway.

    That's why my systems always have at least two different partitions: one for "/" and another for "/home". I can reformat my system partition and still have my data intact.

    1. Re:Partitions are your friend by rincebrain · · Score: 2, Informative

      Not going to help you - most filesystems are growable but not shrinkable online.

      --
      It's only an insult if it's not true.
  17. Re:8.10 upgrade glitch: downclocking by vadim_t · · Score: 2, Informative

    ondemand actually happens to be the best governor.

    In theory, "powersave", by keeping the CPU frequency at a minimum would save some power in comparison. In practice, it doesn't. This is because doing anything at all prevents the CPU from entering the lowest power using modes (which go beyond simply dropping in frequency).

    So it's more efficient to make the CPU run at full blast, do whatever needs to be done, then go to sleep (C3, not suspend to RAM), than to do the same work at a lower clock speed, keeping the CPU active 3 or 4 times longer. By C2 the clock isn't active anymore, which is a huge gain on anything the "powersave" governor can provide.

  18. Re:What about deltas? by stevied · · Score: 2, Informative

    More promising is some sort of system built on zsync - there are some ideas here.

  19. Re:Mirror anxiety by turbidostato · · Score: 4, Interesting

    "Is it just me or is the fun game of "pick your closest mirror" not very fun at all? Just download the damn thing at best possible speed. I don't care where you get it from. "

    You are aware that "closest" in this context means "faster", aren't you?

    "As if I'm in a position to pick the best site where to download something from. Give me a break. Apologies to the power users who can lick their Ethernet cable and tell which site will have the best download performance and availability."

    Probably is too much a power user the one able to install the package "apt-spy" which will build a sources.list for you based on bandwith probes, isn't it?

    Oh, and please, don't let parent post at +Insigthful when it's plain -Nonsense.

  20. Re:What about deltas? by cheftw · · Score: 2, Informative

    $diff slashface-1.1.deb slashface-1.2.deb> slashface1.1-1.2.debdelta

    --
    Always back up, never back down. ---- Think you're cool 'cos your uid is prime? Take mine, modulo the one digit integers
  21. Re:Hey mods... by harry666t · · Score: 3, Funny

    It's not funny and it's not sarcasm. It's the same cliche meme repeated a thousandth time. LOL, p2p is helping terrorism, mod me funny. I could write a python one-liner that would produce more varied content than most of these +5, funnies all over here on /.

    I even actually wrote it:

    $ python -c 'print (lambda words, random: " ".join([random.choice(words)[:-1] for i in range(random.randint(1,10))]))(open("/usr/share/dict/words").readlines(), __import__("random"))+"."'

    angoras lawgiver's Father's approbations uninteresting inferring Antonio's Clotho's chlorine.

  22. Re:Mirror anxiety by blake182 · · Score: 4, Insightful

    You are aware that "closest" in this context means "faster", aren't you?

    Is your point that a host that's connected via T-1 that's a mile away is faster than a host that's connected on an OC-3 3,000 miles away? That is, based on knowing the geographic location of a host, you're saying it's somehow an indication of how fast my download will complete? That's the only thing that matters to me -- when will I have my completed bits. My only point here is that the information given in mirror selection is not enough to pick the "fastest" way to get what I want. It lists the geographical location and that's it.

    Probably is too much a power user the one able to install the package "apt-spy" which will build a sources.list for you based on bandwith probes, isn't it?

    Yes. Yes it is. If such a list can be generated, then why not just generate it in normal operation or list the mirrors based on the output of that tool? Though I do appreciate the tip, and I will try it. Obviously bandwidth and availability varies on a day by day basis. So taking a snapshot at one point in time seems like it will get stale.

    Oh, and please, don't let parent post at +Insigthful when it's plain -Nonsense.

    It's a legitimate end-user concern. "Which mirror should I select" should not be a user problem. The user wants his bits as soon as possible, which is a technical problem that has allegedly been solved with apt-spy. If that's the case, we should probably integrate that with the mirror selection process, and then you don't have to put up with all the "Nonsense".

  23. howtoforge by lems1 · · Score: 2, Informative
    --
    This sig can be distributed under the LGPL license
  24. apt-spy considered dangerous by kostmo · · Score: 2, Informative

    and according to this bug, "apt-spy is no longer in the Ubuntu repository for releases newer than feisty."

  25. Re:Is upgrading worthwhile? by 3vi1 · · Score: 2, Insightful

    Upgrades of Linux distributions work much better than Windows distributions because of the library structure and package dependancy system.

    If you try to upgrade a library on Linux to a new version that cannot coexist with a previous version that other apps depend on, the new package will be set up so that it tells you it needs to remove the old library and its dependent apps if you really want to proceed.

    There's also not going to be a lot of garbage hanging around in a "registry". If a package doesn't work because of settings, they're easily removed from /etc (or a '.' directory in your home directory, for personal settings) and can in no way be harming unrelated apps.

    On Windows.... I agree that clean installs are the better policy.