Slashdot Mirror


Linux 3.4 Released

jrepin writes with news of today's release (here's Linus's announcement) of Linux 3.4: "This release includes several Btrfs updates: metadata blocks bigger than 4KB, much better metadata performance, better error handling and better recovery tools. There are other features: a new X32 ABI which allows to run in 64 bit mode with 32 bit pointers; several updates to the GPU drivers: early modesetting of Nvidia Geforce 600 'Kepler', support of AMD RadeonHD 7xxx and AMD Trinity APU series, and support of Intel Medfield graphics; support of x86 cpu driver autoprobing, a device-mapper target that stores cryptographic hashes of blocks to check for intrusions, another target to use external read-only devices as origin source of a thin provisioned LVM volume, several perf improvements such as GTK2 report GUI and a new 'Yama' security module."

32 of 385 comments (clear)

  1. btrfs needed the work by bzipitidoo · · Score: 4, Interesting

    I tried btrfs, and ended up going back to ext4. Hoped btrfs might be a good choice for a small hard drive, and it is-- it uses space more efficiently. But it's not a good choice for a slow hard drive or the obsolete computer that the small size goes with.

    Firefox ran especially poorly on btrfs. I was told this is because Firefox does lots of syncs, and btrfs had very poor performance on syncs. Maybe this improvement in performance on metadata is just the thing to fix that?

    --
    Intellectual Property is a monopolistic, selfish, and defective concept. It is "tyranny over the mind of man"
    1. Re:btrfs needed the work by Baloroth · · Score: 5, Informative

      Checksumming, built-in RAID support, snapshotting, transparent compression, online volume resizing, et alia. Basically, a lot of stuff that is very interesting at the enterprise level and to serious nerds who like to do strange things with their volume management, but nothing particularly important to the average user. It's basically a non-Oracle-owned version of ZFS, if you know what that is.

      --
      "None can love freedom heartily, but good men; the rest love not freedom, but license." --John Milton
    2. Re:btrfs needed the work by Tacticus.v1 · · Score: 5, Informative

      well comparing it to lvm ignores a significant amount of what btrfs is
      you would compare it with the entire stack
      mdadm + lvm +ext 3/4

      btrfs gets you:
      Checksums on data
      mirrored metadata on a single disk
      lots of flexibility (online resizing and reshaping(single disk to raid 1 to 0 to single disk (or some variant of it) ( additionally raid5/6 like systems are coming)
      easy striping and mirroring across different sized disks
      snapshots
      and probably more go check https://btrfs.wiki.kernel.org/

    3. Re:btrfs needed the work by Gothmolly · · Score: 4, Interesting

      Fix Firefox? Why does it "need" to do a lot of syncs?

      --
      I want to delete my account but Slashdot doesn't allow it.
    4. Re:btrfs needed the work by Anonymous Coward · · Score: 4, Informative

      Actually, ironically Oracle "owns" btrfs! But it is Open Source.

    5. Re:btrfs needed the work by oakgrove · · Score: 4, Insightful

      One really nice thing about ext4 vs. previous versions is the dreaded 20 minute fsck every 40th or so boot only takes about 10-20 seconds now.

      --
      The soylentnews experiment has been a dismal failure.
    6. Re:btrfs needed the work by smellotron · · Score: 5, Insightful

      Fix Firefox? Why does it "need" to do a lot of syncs?

      Sync (or fsync) is the way to ensure that files are committed to disk and not just cached somewhere. This is a precondition for reliable "restore session" and similar functionality. However, application developers cannot rely on the OS to sync data in the background, because e.g. on a laptop where frequent disk access is both expensive (battery life) and risky (physical motion), the OS will cache as much as possible. If FF did not sync, the OS might delay writes for hours, which means a computer crash leads to lost hours of browsing history for the user. It doesn't sound like a big deal, but I can tell you that it is infuriating as a user to see a browser say, "whoops, I lost your tabbed windows, hope you weren't using the WWW for anything important!". Not having looked at the source myself, I don't know if it's possible to optimize FF's sync behavior; but I do know that it's impossible to eliminate it.

    7. Re:btrfs needed the work by Tacticus.v1 · · Score: 4, Insightful

      no because if you lose a disk in a striped array you lose everything. (perhaps you are thinking raid1 in which case it protects you from disk failure but does not provide backups)

      but soon they will be working on a btrfs send\receive system so you would be able to take snapshots and push to another disk

      IMO there are a number of different failure states that you must cater for.
      1. Human failures (the oh shit I deleted something): a snap shot capable file system helps protect you from these (not perfect but fairly good)
      2. Hardware failures (disks are dead): traditional backup systems work here (or btrfs\zfs send\receive) disk failures can have reduced impact due to mirroring your data (or strip plus parity) checksums and COW help defend against silent failure
      3. Software failures (the OS is hosed, partition table is dead): traditional backup systems work here (or btrfs\zfs send\receive) (though COW file systems and marking shit read-only helps)
      4. oh shit the building burnt down: Hope you do offsite backups

      BTRFS helps in the first 3 by bringing awesome features to the table (snapshots, COW(so you can walk back up the tree to recover) and mirroring your data on multiple disks) but is only something that can supplement a backup system not replace it at all

      only a good backup system helps in the 4th situation.

    8. Re:btrfs needed the work by Tacticus.v1 · · Score: 4, Informative

      GPL for ever.

      early in the development of BTRFS commits were sourced from vocal and stubborn devs that would protect it from being re-licensed source: http://www.youtube.com/watch?v=hxWuaozpe2I

    9. Re:btrfs needed the work by Jonner · · Score: 4, Informative

      Checksumming, built-in RAID support, snapshotting, transparent compression, online volume resizing, et alia. Basically, a lot of stuff that is very interesting at the enterprise level and to serious nerds who like to do strange things with their volume management, but nothing particularly important to the average user. It's basically a non-Oracle-owned version of ZFS, if you know what that is.

      Checksumming is useful to anyone who doesn't like corrupt data. Transparent compression is useful to anyone who likes to fit more stuff on their drives and access it faster. Btrfs is technically superior to ZFS though currently less mature. For better or worse, Btrfs is largely developed by Oracle employees so they do own part of it. Oracle could simply stop paying people to develop it but they can't take it away from Linux. Both ZFS and and Btrfs are available under Free and Open Source licenses though the licenses are are not compatible which is the primary reason ZFS cannot be included as part of Linux.

    10. Re:btrfs needed the work by Tacticus.v1 · · Score: 5, Informative

      >like RAID support that doesn't cover RAID5
      Is on the way targeted for 3.5 (was held for the fast offline check code)
      >no online file system check
      btrfs scrub start /blah

    11. Re:btrfs needed the work by shaitand · · Score: 5, Insightful

      "Journaling makes sense for servers; not so much for personal boxes."

      I'm sorry my friend but you must be insane. I don't go uncleanly powering off my boxes intentionally but it still happens a couple times over the course of a month for various reasons (power flickers and the like). In my experience ext2 will fsck its way back to functionality 4 or 5 times tops before it won't fix or the data lost in the fixing is something critical.

      Linux was a fun toy and nothing more before ext3 because ext2 is the most destructible filesystem on earth. Don't get me wrong, I played with that toy but that is all it was.

    12. Re:btrfs needed the work by Jonner · · Score: 4, Insightful

      Wow. Am I out of the loop, or what? We're up to ext*4* now? I'm still using (happily) ext2. Yeah, I've heard of btrfs, but why change if what you're using works? Journaling makes sense for servers; not so much for personal boxes.

      Yes, you are way behind. Ext3 became part of Linux eleven years ago and added journaling to ext2. Some of us have been using superior journaling file systems like Reiserfs3, XFS, JFS and Reiserfs4 for many years. Journaling is a good idea for all file systems because it allows much stronger metadata and sometimes data consistency guarantees. In other words, though hardware failures and unexpected shutdowns can cause data loss on any file system, journaled ones are more likely to know which data are corrupt and which aren't. Btrfs improves on that by also checksumming everything so no corruption can ever go unnoticed. This is increasingly important as disks get bigger and errors become more likely. Another thing that's perhaps especially nice for desktop and laptop systems is that journeled filesystems can generally be checked for consistency very quickly, meaning you much less oftend need to do a lengthy fsck.

    13. Re:btrfs needed the work by shutdown+-p+now · · Score: 4, Insightful

      This is known as featuritis, and is anathema to the Unix way, where each part should do just one thing, and do it extremely well.

      How do you make a file system out of parts, though? e.g. how do you tackle, say, snapshotting, or online volume resizing, efficiently onto an existing FS that was not designed with those features in mind?

    14. Re:btrfs needed the work by waveclaw · · Score: 4, Interesting

      This is known as featuritis, and is anathema to the Unix way, where each part should do just one thing, and do it extremely well.

      All btrfs does is manage a B-tree filesystem. All grep does is apply a regular expression to a string.

      However, the UNIX way is not always even a good thing.

      It is also the UNIX way to duplicate a single thing a hundred times for each little feature variation (grep, egrep, fgrep, most of Perl.) That can also be unpleasant for the end user (xterm, gnome-terminal, kterm, gterm, LXterm, terminator, editing Perl.) Great for a system administrator who is expert at their particular tool and only that tool but horrible for everyone else.

      That's without getting into the UNIX Way for (lack of) documentation. Or how that one thing is so often the wrong thing so it doesn't matter how well that one tool does it.

      btrfs is famously called a rampant layering violation. The roll-up of filesystem-management features in one place actually lets the developers avoid duplicating code (which may actually be about as non-UNIXy as you can get in some ways.) Code that now knows about certain information normally hidden from it can do things differently. This is sometimes better (rapid mkfs) or worse (fsck tool was apparently hard to write.)

      In my opinion, it's not interesting for enterprise because you get mediocre features, like RAID support that doesn't cover RAID5, no online file system check

      In my opinion, if your enterprise system depends on fsck and not good backups then you don't have an enterprise system. Yes, xfs_repair can do amazing things to mostly trashed disks. But one day your data will take a good fscking where only surviving copy will be the backup copy.

      RAID5 implementation from Intel is in the tree, but waiting until after the fsck is done. And btrfsck has been around since, oh, February? And the btrfs-progs you should be using with the 3.4 kernel have btrfsctl included?

      I was hoping the RAID5 code was going to land in 3.4, actually. Reading the pull request says that RAID5/6 should be in 3.5. Oh, well.

      Of course, if you have enough money to buy an "enterprise" solution, your SAN/NAS should do the thing doing RAID for you anyway.

      My major criticism of btrfs is the horrid sync performance. Hosting virtual machines tends to require lots of small writes to disk that make btrfs incredibly non-performant.

      btrfs has many sexy, sexy features for a world of enterprise SAN storage and virtual machine hosting. It has thin disks, balanced meta-data, flexible storage, SSD optimized modes, multiple snapshot layers, checksummed data on disk. All of this just because it does one thing and does it well: manage a B-Tree database.

      Today it's is just not there in the I/O department, sadly. Probably good for inside the virtual machine guests, though. Only testing will tell.

      My money is on NILFS, if nothing else because Oracle gives people a bad taste in their mouths, but ICBW.

      Wow, speaking of niche file systems. Log file systems have quite a long history. Of horrible performance and fragmentation. But if we all end up on SSDs, that won't matter. Underlying any file system you put on it, an SSD implements storage as a circular log and performance is fast enough to not depend on huge uncommitted disk caches.

      --

      "You cannot have a General Will unless you have shared experiences. You cannot be fair to people you don't know."
    15. Re:btrfs needed the work by petermgreen · · Score: 4, Informative

      This is known as featuritis, and is anathema to the Unix way, where each part should do just one thing, and do it extremely well.

      The problem with conventional raid is it has no way of knowing which redundant copy of the information is correct and indeed it may well end up overwriting the correct copy with the bad copy during a resync. So it protects against drives that fail but it doesn't protect against drives that quietly return bad data.

      In theory you could implement a raid layer with strong checksums so it knew which copy was bad, but the problem then becomes where to put those checksums (without creating a load of extra seeks).

      By implementing raid techniques as part of the filesystem the checksums can be stored with the existing metadata. Implementing raid as part of the filesystem also allows different redundancy policies to be applied to different data.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    16. Re:btrfs needed the work by KiloByte · · Score: 4, Informative

      Yes, there is a massive speed difference. Unpacking a particular tarball takes 10 seconds on btrfs, 124 seconds on ext4.

      The problem is with certain broken programs that fsync after every single write. But that's a problem with those programs, not btrfs. The words "fsync" and "performance" don't belong in the same sentence. fsync may be legitimately uses when durability is actually needed, but in almost all use cases the programs want only consistency.

      An example: there's a file where a transaction consists of appending a few bytes to the file then updating something in the header. Let's say it's four bytes in both writes. The program can handle power loss between transactions or if it happens between the first write and a header update, but not any other reordering. The disk in question has 100MB linear speed and 10ms seek time. Even with some kind of a barrier syscall, ext4 would need to alternate writes anyway, plus it needs to write to the journal and inode. This gives you a whooping 25 transaction per second. btrfs and log-structured filesystems with atomic flushes get 25M instead (assuming infinitely fast CPU).

      The primary offender here is Firefox which fsyncs all the time. This not only slows down writes but also causes insane fragmentation. The data it protects is not vital in the first place (mostly browsing history), and if it used sqlite in WAL mode on relevant platforms instead it wouldn't have to fsync for consistency almost at all.

      --
      The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
  2. Re:How RedHat's Linux Can Defeat Micr$oft's Windoz by Anonymous Coward · · Score: 5, Funny

    As much as Linux is doing rather well despite the plethora of different versions and security risk from the open code base, using it is rather risky for legal reasons as well. Red Hat stole much of Linux from SCO's Caldera, and are distributing it without paying royalties, meaning users could be on the hook for several hundred dollars a license and casting the future of Red Hat's offerings in jeopardy.. Litigation is ongoing now, and experts expect SCO to win a crushing verdict any day now. Linux has some neat features, but there's a lot of fear, uncertainty, and doubt in the community about its legal future.

  3. Re:Yes, 3.4 BUT... by Anonymous Coward · · Score: 4, Informative

    Yes! In the last RC! I'm not having that problem any more at all and 3.4 is rock solid on my systems.

  4. Re:yes but... by Clarious · · Score: 5, Informative

    It's a common FUD. Nowaday Linux audio works just fine, PulseAudio as a sound server (mixer) and ALSA to talk to the hardware, the rest (OpenAL, gstreamer, OSS, ESD) are either obsolete or totally different stuff unessential to audio playback. Earlier problems related to closed source softwares (Flash, Skype) or badly written HW drivers are mostly fixed.

  5. Re:They fix the sound bullshit yet? by Clarious · · Score: 4, Informative

    "Another" audio subsytem? Today standard is PulseAudio on ALSA, and that it has been like that for at least 4 years. Before ALSA there was OSS but Linux developers disagree with how OSS do the sound mixing and resampling in kernel space (for better latency, they said) and OSS went closed source for awhile. PulseAudio is an effort to unite all the sound server/mixer (ESD from GNOME, aRTs from KDE or ALSA's own dmix) plus some nifty features like better battery life (less wake ups per second).
    Update your FUD once in awhile, please.

  6. Re:Yes, 3.4 BUT... by deek · · Score: 5, Funny

    Achievement Unlocked

    Most gratuitous use of the word "fuck" in a serious Slashdot post.

  7. Most programs don't need a 64-bit address space by Myria · · Score: 5, Informative

    The new x86-64 ABI with 32-bit pointers is cool because it allows you to get the architecture improvements of x86-64, such as extra registers and RIP-relative addressing, without increasing memory usage substantially due to larger data structures. Also, 64-bit operations will just use the 64-bit registers. The vast majority of programs simply do not need the extra address space.

    One reason that this ABI works so well is that the majority of the x86-64 instruction set uses 32-bit operations. Some operations involving pointers can be done in one instruction without using a temporary register to load a 64-bit constant.

    Windows actually also can support this, in theory, but you're on your own in trying to communicate with the Win32 API. The linker option /LARGEADRESSAWARE:NO causes the NT kernel to limit your program's address space to 2^31 bytes.

    --
    "Screw Sun, cross-platform will never work. Let's move on and steal the Java language." - Visual J++ Product Manager
    1. Re:Most programs don't need a 64-bit address space by Mr+Z · · Score: 4, Interesting

      Yes, but so what? A system that supports x32 should also support x86-64. So, if you're relying on ASLR for security purposes, compile those sensitive apps as x86-64.

      Granted, the potential attack surface grows as you consider larger and larger threats. For example, a GCC compiled as x32 makes a fair bit of sense. What about Open/Libre Office? Well, that depends on if you open untrusted documents that might try to exploit OOo / LO. (Odds seem pretty low, though.) And what about Firefox? Far less to trust on the web...

      So, at some point, you have to make a tradeoff between the marginal benefit of increased performance/better memory footprint in x32 mode vs. increased security against certain overflow attacks that ASLR offers. For most people in most situations, the former likely wins for anything with a decent memory footprint. For people building hardened Internet-facing servers, the latter probably wins.

    2. Re:Most programs don't need a 64-bit address space by Narishma · · Score: 4, Insightful

      The problem is not the memory but the CPU cache. No reason to clog it with bloated 64 bits pointers when 32 bits pointers will do.

      --
      Mada mada dane.
  8. Who cares why it needs it? by Sycraft-fu · · Score: 5, Insightful

    It is something the FS should handle. The "Just fix the program," is a bad answer because while maybe one could change Firefox, you'll find another program that can't be changed because the nature of what it does requires many syncs.

    The low level systems should be robustly written to do what apps need, they shouldn't be telling apps "You can't do that."

  9. kernel 3.2 was released only 5 months ago by Anonymous Coward · · Score: 4, Insightful

    What is the rationale for moving up to 3.4 so soon?

    Obviously big tech companies, as well as the Mozilla Foundation play the versioning game aggressively, but the Linux kernel always had a reputation of being conservative.

    1. Re:kernel 3.2 was released only 5 months ago by shaitand · · Score: 4, Informative

      "I'd say too conservative, if they were only updating the third digit every few months."

      I beg to differ. This is the kernel not some userland app or even a daemon. Stable releases are supposed to be reliable enough to trust with billions of dollars in data flow and human life support systems on the day of release.

  10. GNU? by unixisc · · Score: 5, Informative

    Why is the GNU logo the one that marks this story, when it's specifically about the Linux kernel, and not GNU userland? Among the keywords, GNU shouldn't even be there for this story, and the logo for this story should have just been the penguin logo.

  11. Re:yes but... by adolf · · Score: 4, Funny

    Prior to the advent of pulse audio on my distros of choice, I had audio issues from time to time. Usually small stuff like audio randomly stopping working or not being able to play certain applications simultaneously.

    [Old Man mode]: I remember a time before PulseAudio, and before JACK, and before ALSA: The Linux kernel had some built-in drivers ("OSS-Free"?) which supported adequate functionality for every sound card/chip on the list, and if you wanted more features or support you could just pay 4front for a better driver (and they were always worth the minimal price).

    And: Everything. Just. Worked. Always. Hardware settings (back when sound cards still had configurable analog sections(!)) were deterministic and reliable, and getting excellent sound from *random_app* was a foregone conclusion.

    Much fun was had, for instance, with "cat /dev/audio > /dev/st0" to dump a radio show (reliably! without problems! in the plain-and-simple way that Unix is supposed to be!) to DDS tape.

    Now, this was 17 (or so) years ago. Anything involving further difficulty, at any stage of the game on a user level, on the Linux sound front is a step backward.

    Now, get the fuck off my lawn.

    [/Old Man mode]

  12. Oct, 1991, comp.os.minix by Taco+Cowboy · · Score: 5, Funny

    I still remember that message, on Oct 1991, from a guy by the name of Linus Benedict Torvalds on comp.os.minix

    "Do you pine for the nice days of minix-1.1, when men were men and wrote
    their own device drivers? Are you without a nice project and just dying
    to cut your teeth on a OS you can try to modify for your needs? Are you
    finding it frustrating when everything works on minix? No more all-
    nighters to get a nifty program working? Then this post might be just
    for you :-)
    "

    --
    Muchas Gracias, Señor Edward Snowden !
  13. Re:yes but... by hey! · · Score: 4, Insightful

    It's a common FUD. Nowaday Linux audio works just fine

    Well, sometimes getting audio to work is beyond the control of the Linux kernel. If the system has integrated audio on the motherboard (e.g. a laptop) the ACPI DSDT (Differentiated System Description Table) supplied by the manufacturer in the ROM can instruct the hardware to behave differently under different operating systems, or provide different descriptions of the hardware (e.g. audio inputs and outputs) to different operating systems. That's why it's common to have little glitches in Linux audio, like not having the right mixer controls.

    The DSDT is written in a language called ACPI Source Language (ASL). Intel and Microsoft both provide compilers for ASL, but the MS compiler accepts buggy, non-compliant DSDTs. Since for some vendors (Toshiba) the job is considered done when stuff works under the current version of windows, they ship their laptops with DSDTs that won't work under anything but Windows and might not work in future versions of Windows.

    Since the kernel writers have no way of knowing what specific hardware is in your machine except what your machine tells the kernel, they can't fix this. It's entirely the manufacturer's fault, although users blame Linux because everything works in Windows. Getting stuff working isn't exactly a nightmare, but it's beyond most users' capability. You extract the DSDT from ROM, decompile it, fix the bugy ASL, compile it, then put the fixed DSDT in your initramfs (remembering to do this again every time you install a new kernel). Sometimes using a linux boot parameter to masquerade as Windows to the hardware works.

    So to recap: the Linux audio system may be fine, the hardware drivers may be fine, but if the manufacturer fails to supply a correct description of what the hardware contains to the Linux kernel, audio might not work.

    Disclaimer -- this information is a few years out of date, as I've stopped using Toshiba laptops and use Asus instead. However I'm fairly sure it still exists with certain manufacturer's laptops, which have worked flawlessly for me under Linux.

    --
    Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.