Slashdot Mirror


Linux 4.18 Preparing Many New Features While Dropping 100k+ Lines of Code (phoronix.com)

An anonymous reader writes: Linux 4.18 development is going strong with recent 4.18-rc1 release. This kernel cycle has dropped 107,210 lines of code so far but Linux 4.18 is adding many new features. The kernel is coming in lighter as a result of the LustreFS code being removed and other code cleanups. On the feature front, Phoronix reports, "ew AMDGPU support improvements, mainlining of the V3D DRM driver, initial open-source work on NVIDIA Volta GV100 hardware, merging of the Valve Steam Controller kernel driver, merging of the BPFILTER framework, ARM Spectre mitigation work, Speck file-system encryption support, removal of the Lustre file-system, the exciting restartable sequences system call was merged, the new DM writecache target, and much more."

45 of 105 comments (clear)

  1. mainlining by descil · · Score: 1

    Yeah man, gimme another hit of that v3d drm

    1. Re:mainlining by descil · · Score: 1

      *sighs* yeah man. i was joking about how the kernel is called "mainline" and thus features get "mainlined." Mainlining is a drug term appropriated for the kernel. Arguments claiming that mainline is a plumbing or railway term are false: that term is "main line," and it's not a term that needs definition, it means what it says, the main (most used) line. Nothing to do with the word "mainline." And yet the kernel is called mainline.

      Additionally using the TLA "DRM" in your device driver is the kind of horrible idea someone could only come up with while having some of that sweet sweet black juice running through their veins.

      Furthermore calling it V3D is extremely misleading, as it's not a 3d driver, nor is it virtual reality.

      So all in all it seemed like a good opportunity to call out all of these idiocies appearing at once. But thanks to you the joke is ruined. Congratulations.

    2. Re:mainlining by descil · · Score: 1

      :)

  2. Lustre dead? by afidel · · Score: 3, Interesting

    With Intel shutting down their commercial support business last year and now LustreFS being removed from the mainline kernel is Lustre dead as a common solution? What is replacing it as a scalable FS in HPC applications?

    --
    There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    1. Re:Lustre dead? by Anonymous Coward · · Score: 1

      Glusterfs

    2. Re:Lustre dead? by Anonymous Coward · · Score: 2, Informative

      The Lustre devs basically never played nice with the Linux Kernel devs, and Lustre never left the STAGING sub-section of the Linux Kernel because they never (in almost a decade) cleaned up the code to pass Linux Kernel code reviews.

      Lustre's most recent full release was just this last April, it's very much under heavy development, but in the HPC world having your code 'upstream' is basically a non-issue because they're used to having to do a million minor adjustments to eek out another 1% performance, since at the scale of computing they're doing that tiny improvement makes a noticeable difference. They don't run 'out of the box' Linux distros compared to a web-server or the like.

      - WolfWings, too lazy to login to /. in far too long.

    3. Re:Lustre dead? by Wrath0fb0b · · Score: 2

      The Lustre devs basically never played nice with the Linux Kernel devs, and Lustre never left the STAGING sub-section of the Linux Kernel because they never (in almost a decade) cleaned up the code to pass Linux Kernel code reviews.

      This is a major downside of the "everything in the same source tree" Linux philosophy.

      In a different universe, we could have pluggable filesystem modules that could be built against each kernel source[1] without having to get it all into the same tree. If the kernel API changed, the FS devs would have to adapt to be compatible, but at least they wouldn't have to fight with the maintainers and the maintainers wouldn't have to be bothered by their patches. It wouldn't solve a technical problem -- after all, the build system hardly gives two shits where the source comes from -- but it would solve a major human problem.

      Oh well, we can dream right?

      [1] Specifically not wanting ABI compatibility here. First, because down that road madness lies and second because it's not too much to ask that FS developers recompile when the kernel changes and adopt changing API. It also cuts off the crazy assertion that pluggable modules would stagnate the kernel code. It would, unless we just stipulate that kernel changes are absolutely allows to break any plugin and that the plugin has no right to a stable API.

      This doesn't replace in-tree, by the way. If a contributor and the maintainer agree, then by all means put it in-tree and be done with it. This is even beneficial for the contributor because it means that anyone changing the API must go and fix/shim all his code.

    4. Re:Lustre dead? by Hylandr · · Score: 1

      I have recently switched to this over NFS very recently. Native fault tolerance is a good thing!

      --
      ~ People that think they are better than anyone else for any reason are the cause of all the strife in the world.
    5. Re:Lustre dead? by Bruce+Perens · · Score: 1

      The problem with pluggable filesystem modules is that we still have a lot of innovation to do in the low-level kernel handling of I/O, thus the interface remains in flux. Significant performance gains have been realized and more are possible, and architecture for systems with significantly greater parallelism is still in progress. Taking advantage of new hardware capabilities has also required changes in interfaces.

      One of the unique things about Linux is that the developers are allowed to innovate. It's a feature, not a bug.

      I, for one, wouldn't really want a filesystem that hadn't passed review by the kernel team.

    6. Re:Lustre dead? by sjames · · Score: 1

      We have that now. You build your out of tree module and then modprobe it.

    7. Re:Lustre dead? by DamnOregonian · · Score: 1

      I compile modules out-of-tree all the time....
      Never a filesystem module, I'll admit, but they can function fine as a module afaik... Am I wrong?

    8. Re:Lustre dead? by Guy+Smiley · · Score: 1

      Lustre is most definitely *not* dead. The removal from staging was a bit of an unfortunate event, but work on getting it cleaned up for re-submission to the kernel is continuing.

    9. Re:Lustre dead? by MAXOMENOS · · Score: 2
      The Register suggests that other FS's can pick up the slack. Quoting:

      When Lustre emerged in the year 2003 it had little competition for creation of large-scale filesystems. Nearly 15 years on and Red Hat offers Gluster, IBM’s Spectrum Scale (aka the GPFS General Parallel File System) and scale-out NFS can all do plenty of what made Lustre useful. HDFS has emerged, too, for big data workloads.

      Source: Linux literally loses its Lustre – HPC filesystem ditched in new kernel. The Register, 18 Jun 2018 at 01:29

    10. Re:Lustre dead? by nctritech · · Score: 1

      https://www.mjmwired.net/kerne...

      Executive Summary: You think you want a stable kernel interface, but you really do not, and you don't even know it. What you want is a stable running driver, and you get that only if your driver is in the main kernel tree. You also get lots of other good benefits if your driver is in the main kernel tree, all of which has made Linux into such a strong, stable, and mature operating system which is the reason you are using it in the first place.

    11. Re:Lustre dead? by afidel · · Score: 2

      He's wrong, I prefer a stable API, unstable APIs and bad vendors is why we have the mess we do with Android and why Treble is the kinda solution, by freezing on a LTSR kernel they've effectively created an artificially stable API which they backport everything else around. In an idealized world, yes I'd love to have every driver in mainline so that everything gets updated together and the kernel devs were free to monkey with things to their hearts content, in the real world I live in a stable API allows vendors to dump a driver once and for it to be usable for a decade or more (see most Windows drivers from Vista still working on 10 today).

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    12. Re:Lustre dead? by Wrath0fb0b · · Score: 1

      Indeed. So I would stipulate that the kernel is absolutely allowed to break the plugin API at any point for any reason.

      It would be courteous to announce those on some list in advance, but that's all I would even ask those guys to provide.

    13. Re:Lustre dead? by Wrath0fb0b · · Score: 1

      I disagree. Stable API are good. Being able to innovate at all layers is good.

      My $0.02 is that you should avoid breaking it, but sometimes you gotta. When Windows moved to the new graphics model, they broke a ton of shit. But they also greatly improved platform stability.

    14. Re:Lustre dead? by brantondaveperson · · Score: 1

      An interface that you can change at will, provided you change everything that calls it, doesn't really deserve the moniker "interface". It's just a bunch of function calls that can change whenever you want. The entire point of interfaces is to divide development tasks using them, so that one does not have to have everything in the same source tree.

    15. Re:Lustre dead? by sjames · · Score: 1

      That's what LTS kernels are for. Unless you have a compelling reason, you shouldn't be updating every time Linus releases a kernel. Instead, wait for your distro to backport any critical fixes.

    16. Re:Lustre dead? by Wrath0fb0b · · Score: 1

      Can does not imply should.

      My view on things was that the kernel devs should be entitled to change the API and that the plugin vendors should not be entitled to complain about the API change. This was meant only to pre-resolve any dispute over who is responsible for breakage.

      Given the good reputation of kernel devs, I expect they. would not do so for light and transient reasons, but would change the API when there was a good reason to. It would be stable enough, but power to decide when improvements require a rework would lie with them.

    17. Re:Lustre dead? by _merlin · · Score: 1

      Repeating it doesn't make it true. Having a module in the main kernel tree only guarantees it will continue to build, because there's no way for the kernel maintainers to actually test drivers for often esoteric and expensive peripherals, specialised filesystems, or other unusual use-cases. I do want stable kernel ABIs, so drivers continue to work and I'm not at the mercy of people having to play constant catch-up when things are shuffled.

    18. Re:Lustre dead? by cas2000 · · Score: 1

      In a different universe, we could have pluggable filesystem modules that could be built against each kernel source[1] without having to get it all into the same tree.

      We have that in this universe.

      If the kernel API changed, the FS devs would have to adapt to be compatible, but at least they wouldn't have to fight with the maintainers and the maintainers wouldn't have to be bothered by their patches. It wouldn't solve a technical problem -- after all, the build system hardly gives two shits where the source comes from -- but it would solve a major human problem.

      Oh well, we can dream right?

      You don't have to dream, you can have this right now with ZFS on Linux. And several other filesystems too. It works, there's nothing actually "wrong" with it, but it's a lot less desirable than you think.

      Almost every time there's a new kernel release, the ZFS modules won't compile against it. There's usually a delay of a few days to a few weeks before ZoL is patched to compile with the new kernel.

      It's a minor hassle, but an annoying one. The features offered by ZFS are more than worth the hassle.

      The solution is to not install new kernels until you know for sure that ZFS is updated to work with them.

      In debian terms, 'apt-mark hold' the linux-image-* and linux-headers-* packages so that an apt upgrade wont auto-upgrade them. Upgrade them manually with 'apt-get install' after the zfs-dkms & spl-dkms & other packages have been upgraded, and then remember to hold them with apt-mark again.

      IMO it's best to also hold all the ZFS-related packages too, and manually upgrade them immediately before manually upgrading the kernel. Then hold them again. Otherwise you could end up with the userland portions of zfs (libs and zfs/zpool/etc utils) that are different (and possibly incompatible) versions to what is running in the kernel. Mostly harmless, most of the time, but not something I want to do with fileystems that contain the bulk of my files.

  3. Causing more problems down the line by xack · · Score: 1

    By removing features from kernels you force people to either use older kernels or force them to back port back to the current kernels. This creates a mess of patches and security holes. With Linux deployments now in the billions due to Android it is a bad idea to remove features as someone is using it somewhere.

    1. Re:Causing more problems down the line by Anonymous Coward · · Score: 5, Informative

      In this case what was removed was mostly components in the staging sub-section that had been there for years (over half a decade in many cases) but hadn't completed appreciable work towards exiting the Staging sub-section. So they're finally getting the boot back out of kernel since they had never graduated to 'full kernel support' due to lack of action on their devs part. Lustre in particular basically stopped contributing to the staging branch since they felt it slowed them down too much versus working on their out-of-tree version of the code instead.

  4. Those ridges are stoney and bristol... by cre1mer · · Score: 1

    The older AMD Stoney Ridge and Bristol Ridge AMD APUs finally have temperature reporting support.

    Stoney Ridge is old. Bristol Ridge is available on AMD's newest platform, AM4.

  5. Re:bug remaining? by bmimatt · · Score: 1

    Have a similar problem/question. Need Ryzen support for a Dell Insipron 1700. Just spent a weekend trying to get it to work, ending up frustrated/disappointed. Can anyone shed some light on the Ryzen support?

  6. Re:bug remaining? by SiChemist · · Score: 2

    I thought I had that bug, but It was fixed by switching wireless PCI cards. The ATH9K driver was the culprit on my system.

    It was pretty weird because the symptoms were the same-- Only crash at idle or near idle. As long as the CPU was under heavy load, the system ran flawlessly. I tried everything I could find online to mitigate it including the "rcu_nocbs=0-15" parameter.

    I finally noticed some ath9k messages in the logs near the time of the crash, so I bought an intel PCI wireless adapter and replaced my old one. Haven't had a crash since (that I recall). This system is up 24/7 except for kernel updates and is running Xubuntu 18.04 now. (It was on Xubuntu 16.04 at the time of the problems and my solution).

  7. Looks like you still need the bios setting by raymorris · · Score: 1

    It looks like anything you do in the kernel is a workaround rather than a fix. So I wouldn't expect it to be "fixed" in the kernel, since it's not really a kernel problem.

    From what I can gather, until AMD does a firmware fix (or hardware?) the best option is to select "typical current idle" in bios.

    Of course, if the rcu workaround is working for you, you might just leave it at that until AMD really fixes the issue.

  8. You need zenstates by webnut77 · · Score: 4, Informative

    How can I tell if the bug that was crashing Ryzen processors at idle has been fixed?

    In 4,15, Ryzen users had to put in a boot option

    "rcu_nocbs=0-15

    (this case for 8 cores =0-11 for a six core...)

    I had this problem too and the rcu_nocbs=0-15 didn't fix entirely the problem. You need zenstates to turn off the C6 power saving state.
    Here's the systemd unit file:

    [Unit]
    Description=Turn off power saving C6 state

    [Service]
    Type=oneshot
    StandardOutput=syslog
    ExecStart=/usr/bin/python /usr/local/src/ZenStates-Linux-master/zenstates.py --c6-disable

    [Install]
    WantedBy=basic.target

    1. Re: You need zenstates by sjames · · Score: 4, Informative

      Yes, I'm sure she'd rather see error 0×657EA6556778B4732FFED56546

      That way she can fire up her hex editor and manually patch the Windows kernel to fix the issue.

    2. Re:You need zenstates by brantondaveperson · · Score: 1

      It doesn't matter. The point is that the file is declarative, not procedural, which prevents the boot sequence from becoming a Turing complete system.

    3. Re: You need zenstates by tepples · · Score: 1

      Why would your grandmad do it manually? Instead of you running a script over SSH, like a COMPUTER user.

      Running a script over SSH doesn't help if the problem is in the computer's networking configuration, or if the computer is behind an ISP that applies carrier-grade network address translation (CGNAT) because it lacks enough IPv4 addresses for all of its home subscribers.

    4. Re:You need zenstates by datavirtue · · Score: 1

      The trend is simplified config files over XML/JSON. So yeah...back to Windows ini files it is.

      --
      I object to power without constructive purpose. --Spock
  9. "ew AMDGPU" by TeknoHog · · Score: 4, Funny

    OK, I get it that you're an Nvidia guy, but no need to rub it in my face.

    --
    Escher was the first MC and Giger invented the HR department.
  10. Re:Comment vs. "SysAdmin" scriptkiddies by sjames · · Score: 2

    No, I'm just tired of squinting at attached screenshots rather than the user with a problem just being able to cut paste 160 characters of text into an email.

    That and clearly telling the user to click on the thing that looks like a squished beetle and they keep clicking on the deformed grasshopper.

    Give us a call after you spend some time maintaining a server you have never actually touched or seen on the other side of the country, then tell us how much you like the gooey interface.

  11. Re:Speck file-system encryption?? by sjames · · Score: 1

    But be careful, the NSA also provided advice to NIST that resulted in significantly weakening their crypto standards.

  12. Re:slashdot is a showcase for narcissists by samwichse · · Score: 2

    Whoosh

  13. Re: bug remaining? by Zero__Kelvin · · Score: 1

    Unless you tried a known good Atheros chip based NIC of the same model and manufacturer you don't know if it was a driver issue or a hardware issue. Most of the times my drivers were "having issues" it turned out to be hardware issues. That obviously doesn't mean this isn't a driver issue either.

    --
    Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
  14. Re:You I'll reply to & why? Ok... apk by sjames · · Score: 2

    You obviously haven't reached a skill level where the significant advantages of text based admin come in. Sometimes enterprise means resilliant and effective, but quite often it means brittle and catering to the lowest common denominator.

    It''s fine if users use a GUI, but the system itself should be text based so a skilled admin has a chance of actually fixing it when it's not working right. Even MS is starting to recognize that.

    As for LAN/WAN, step one when there is a problem is ditch the GUI and get a command line. If you actually understand the network and how the routers work, that will give you the tools you actually need to solve the problem. Use the GUI to check status when things are working well. If you need a picture of a thermometer to know when the temperature is too high, you'be already lost.

  15. Hash in counter mode is a stream cipher by tepples · · Score: 1

    Except that SHA is a hash, not an encryption scheme.

    Any hash function can be turned into a stream cipher by running it in counter mode. This construction, called "Snuffle" by Daniel J. Bernstein, was the insight that led to the United States loosening its cryptography export regulations in the late 1990s. See Bernstein v. US .

  16. Re:bug remaining? by datavirtue · · Score: 1

    What is it about deleting reams of code that arouses me? This is a story I can cheer.

    --
    I object to power without constructive purpose. --Spock
  17. Re:Yah right. by Tough+Love · · Score: 1

    This kernel cycle has dropped 107,210 lines of code so far but Linux 4.18 is adding many new features. The kernel is coming in lighter as a result of the LustreFS code being removed

    Yah right. This is nothing to be proud of. It is a failure of the community, to be unable to maintain Lustre. Now it goes more proprietary than it already was. Shame on Linus and his hangers-on.

    Some weeny with mod points thinks Linus is perfect. No, far from it. Linus is great, but far from perfect, and the whole Lustre thing was obviously mismanaged, including by Linus and hangers-on like Gregkh.

    --
    When all you have is a hammer, every problem starts to look like a thumb.
  18. Re:Yah right. by Tough+Love · · Score: 1

    Or the Lustre guys could have gotten their shit together and it could have been put in the mainline kernel.

    They could have been encouraged to do that, instead of just throwing the baby out with the bathwater which is what idiot Gregkh did, because he has not got a fucking clue that this is just as important as the stuff he actually understands. Problem is, he's a bit of a dunce. You can see that in a lot of stuff he does. Crappy coder too.

    --
    When all you have is a hammer, every problem starts to look like a thumb.
  19. Re: bug remaining? by SiChemist · · Score: 1

    Late reply, but this Atheros card came out of the predecessor of my Ryzen machine (Intel core i5). It worked in that setup without issue with the same OS.

    After I replaced the wireless card in my Ryzen machine, I switched my old Atheros card back to the Intel computer where it works flawlessly.

  20. Re: bug remaining? by SiChemist · · Score: 1

    Almost forgot-- Unloading the ath9k kernel module without replacing the hardware also solved the problem.