CFQ In Linux Gets BFQ Characteristics

← Back to Stories (view on slashdot.org)

CFQ In Linux Gets BFQ Characteristics

Posted by timothy on Friday February 5, 2016 @05:47AM from the omfg dept.

jones_supa writes: Paolo Valente from University of Modena has submitted a Linux kernel patchset which replaces CFQ (Completely Fair Queueing) I/O scheduler with the last version of BFQ (Budget Fair Queuing, a proportional-share scheduler). This patchset first brings CFQ back to its state at the time when BFQ was forked from CFQ. Paolo explains: "Basically, this reduces CFQ to its engine, by removing every heuristic and improvement that has nothing to do with any heuristic or improvement in BFQ, and every heuristic and improvement whose goal is achieved in a different way in BFQ. Then, the second part of the patchset starts by replacing CFQ's engine with BFQ's engine, and goes on by adding current BFQ improvements and extra heuristics." He provides a link to the thread in which it is agreed on this idea, and a direct link to the e-mail describing the steps.

65 comments

Min score:

Reason:

Sort:

Neat! by Anonymous Coward · 2016-02-05 05:53 · Score: 0

This will probably improve compatibility with windows games like Batman and Sky Rim
1. Re: Neat! by Anonymous Coward · 2016-02-05 06:45 · Score: 0
  
  Arkham Night is Linux native on steam
2. Re:Neat! by aliquis · 2016-02-05 06:47 · Score: 1
  
  Only with "DFQ?"
  In other news Batman Arkham Knight _won't_ be ported to Linux.
Nope by Anonymous Coward · 2016-02-05 05:53 · Score: 0

I still use deadline.
Why? by cephalien · 2016-02-05 05:55 · Score: 2

Sorry, but I just don't understand what the purpose is, and it isn't stated in the thread linked -- other than a few ... (maybe) benchmarks that don't cover many real-world use cases.

--
If firefighters fight fire, and crimefighters fight crime, what do freedom fighters fight? - George Carlin
1. Re:Why? by slacka · 2016-02-05 06:06 · Score: 3, Insightful
  
  I just don't understand what the purpose is, and it isn't stated in the thread linked -- other than a few ... (maybe) benchmarks that don't cover many real-world use cases.
  How can you complain that you don't have any benchmarks to judge it by and then claim that if those benchmarks existed they do not cover real-world use cases? If your claim is true, this sounds unnecessary. But my gut trusts the kernel devs more than some close-minded Slashdot commenter afraid of change.
2. Re:Why? by Anonymous Coward · 2016-02-05 06:12 · Score: 5, Informative
  
  Scheduling is an area many people don't really understand, as it is mostly behind the scenes.
  It's also a very debated topic, as the optimal scheduling configuration often depends on what exactly a given machine is doing.
  A server, for instance, may have many more processes running than a desktop, but they are largely the same type of processes. For this, a completely 'fair' type of scheduler may be the best choice.
  A desktop however might be running far less processes at once, but they are vastly different. A mix of background processes and real-time applications, who's timing requirements vary between processes and possibly within a process at different times or situations. For this, a more heuristic based scheduler might be more appropriate, providing an overall 'more responsive' feeling experience.
3. Re:Why? by Anonymous Coward · 2016-02-05 06:14 · Score: 1
  
  [...] some close-minded Slashdot commenter afraid of change.
  New here, are you?
4. Re:Why? by Allasard · 2016-02-05 06:22 · Score: 2
  
  And why are they going thorough the trouble of removing improvements from CFQ?
  There is already a choice of schedulers in the kernel. Why not just make an addition one named BFQ?
  That seems much safer than mucking with the current default scheduler and potentially breaking performance for a type of workload.
5. Re:Why? by Anonymous Coward · 2016-02-05 06:31 · Score: 0
  
  But my gut trusts the kernel devs more than some close-minded Slashdot commenter afraid of change.
  Since they change the scheduler almost every other year, why would you trust them?
6. Re:Why? by Anonymous Coward · 2016-02-05 06:41 · Score: 1
  
  "Low latency for interactive applications
  According to our results, regardless of the actual background
  workload, for interactive tasks the storage device is virtually as
  responsive as if it was idle. For example, even if one or more of the
  following background workloads are being executed:
  - one or more large files are being read or written,
  - a tree of source files is being compiled,
  - one or more virtual machines are performing I/O,
  - a software update is in progress,
  - indexing daemons are scanning filesystems and updating their
  databases,
  starting an application or loading a file from within an application
  takes about the same time as if the storage device was idle. As a
  comparison, with CFQ, NOOP or DEADLINE, and in the same conditions,
  applications experience high latencies, or even become unresponsive
  until the background workload terminates (also on SSDs)."
  This sounds like heaven to me, I still don't have to money to replace all my spinning rust.
7. Re:Why? by aliquis · 2016-02-05 06:43 · Score: 1
  
  But my gut trusts the kernel devs more than some close-minded Slashdot commenter afraid of change.
  Just call him racist and be done with it.
8. Re:Why? by Anonymous Coward · 2016-02-05 06:48 · Score: 2, Informative
  
  This video shows the improvement
9. Re:Why? by Tough+Love · 2016-02-05 07:18 · Score: 1
  
  Enjoy.
  
  --
  When all you have is a hammer, every problem starts to look like a thumb.
10. Re:Why? by Tough+Love · 2016-02-05 07:26 · Score: 5, Informative
  
  And why are they going thorough the trouble of removing improvements from CFQ?
  CFQ was never very good, Lots of quirky behaviour, often being worse than the NOOP scheduler and sometimes stuffing up completely. This is a nice polite way of taking it out behind the barn and shooting it. The new one turns in massive improvements in read latency, respectable improvement in other loads, and little to no regression on any load, besides being thought through and 1,000,000 times better documented than the old steaming pile.
  
  --
  When all you have is a hammer, every problem starts to look like a thumb.
11. Re:Why? by Anonymous Coward · 2016-02-05 07:29 · Score: 0
  
  Who's timing requirements? Were you ill the day they taught the difference between "who's" and "whose" in your school? It is not a concept all that difficult to understand, you know.
12. Re:Why? by Anonymous Coward · 2016-02-05 08:13 · Score: 0
  
  I apologize for the grammatical error. Hopefully it does not distract too much from the discussion at hand.
13. Re:Why? by cephalien · 2016-02-05 09:19 · Score: 1
  
  Thank you; that's what I meant by 'I don't understand', not some sort of half-arsed attack against the 'trusted linux developers' (whoever decides that particular subset, anyway).
  
  --
  If firefighters fight fire, and crimefighters fight crime, what do freedom fighters fight? - George Carlin
14. Re:Why? by sumdumass · 2016-02-05 10:38 · Score: 1
  
  I don't think he is complaining about benchmarks but about not being able to derive the point from the article or the benchmarks.
  I too suffer the same ignorance as i wasn't aware that there was an issue needing fixed or why this fix seems to be address it. Anyone have a idiots guide to the back story on this?
15. Re:Why? by Foresto · 2016-02-05 10:55 · Score: 1
  
  Thank goodness. It's embarrassing how long my desktop machines have had the habit of looking completely locked up whenever they're asked to copy a large file in the background. It's especially bad when a slow-ish NAS server is involved. I've tried the existing optional IO schedulers, and they don't fix the problem.
16. Re:Why? by davester666 · 2016-02-05 12:11 · Score: 1
  
  why is that a problem? are you claiming that we already found the 'one-true-algorithm' for doing this, and that it's impossible to do any better, and these guys are just screwing around with the code because they have commit rights?
  Actually, if anything, it's possible that making it a configuration option would be useful, as computers used for specific tasks, one algorithm would work better than others. Or you would just build the distribution to use that algorithm.
  
  --
  Sleep your way to a whiter smile...date a dentist!
17. Re:Why? by davester666 · 2016-02-05 12:15 · Score: 1
  
  Probably is an active ISIS combatant.
  
  --
  Sleep your way to a whiter smile...date a dentist!
18. Re: Why? by Anonymous Coward · 2016-02-05 14:01 · Score: 0
  
  Just today, I started copying a big file locally while watching an online video in a quarter-screen window and the video was skipping until the copy finished. I just thought "that's Linux" and then thought about just how much brokenness I sweep under that carpet every day. To be fair, it's usually KDE or the GUI apps which are at fault but I lump them all together as Linux.
  If this is a fix for such problems, why hasn't it been sorted out years ago? A scheduler is such a fundamental part of an o/s and should be able to cope with any workload automatically.
  Will this patch noticeably improve scheduling for my desktop?
  Will it appear as a normal update in the repos or do I have to install and tinker with the patch?
19. Re: Why? by Anonymous Coward · 2016-02-05 14:23 · Score: 0
  
  Linux is fixable, therefore it's also breakable. Microsoft doesn't give you the option to monkey with the internals. There's only one distro and if they have their way, there will be only one version as well. Pick your poison.
20. Re:Why? by Anonymous Coward · 2016-02-05 14:36 · Score: 0
  
  Probably is an active ISIS combatant.
  Can't be very active if they sit here.
  Maybe it's smartphone hour. /aliquis
21. Re:Why? by evilviper · 2016-02-05 14:51 · Score: 2
  
  Sorry, but I just don't understand what the purpose is, and it isn't stated in the thread linked -- other than a few ... (maybe) benchmarks that don't cover many real-world use cases.
  With CFQ, an high disk-IO task will block every other process on the system from getting any time. This can be a big file cp, but I see it most often when writing to slow USB thumb drives... Queue up a copy/rsync/etc. of a few GBytes of data to a slow thumb drive, and after your RAM/buffer cache is filled, your system will be almost completely unresponsive.
  Change your scheduler from CFQ to deadline and your system will spring back to life. I don't specifically know that BFQ does any better, but it couldn't possibly be worse... CFQ is crap.
  
  --
  Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
22. Re: Why? by Anonymous Coward · 2016-02-05 15:15 · Score: 1
  
  NOOP is a good scheduler, it proves just focusing on minimizing overhead is often better than trying to manage things.
23. Re:Why? by Anonymous Coward · 2016-02-06 00:06 · Score: 0
  
  You being a prick is not a good contribution.
24. Re: Why? by Anonymous Coward · 2016-02-06 02:10 · Score: 0
  
  Especialy if you have a high end raid controller which is trying to be smart too also vm's should always use noop.
25. Re:Why? by allo · 2016-02-07 03:14 · Score: 3, Interesting
  
  Sorry, but what it shows is that gnome terminal sucks.
  I know it's slow. And the startup time is not the biggest problem. It has tabs, if you want to.
  But let some text scroll fastly there and look at the cpu usage.
  Then use urxvt and compare startup time (instant) and text scrolling (almost no additional cpu, the program producing the text dominates).
  Thats the reason against gnome-terminal. When a low cpu program occupies a whole cpu core, because the terminal drawing the output sucks, youre doing it wrong.
CFQ gets BFQy by Anonymous Coward · 2016-02-05 06:00 · Score: 1

And this is a BFD?
1. Re:CFQ gets BFQy by Anonymous Coward · 2016-02-05 10:07 · Score: 0
  
  No, this is Patrick
WTF title? by paulpach · 2016-02-05 06:08 · Score: 4, Insightful

The title says: "CFQ In Linux Gets BFQ Characteristics"
CFQ is not getting BFQ characteristics, it is simply being replaced by BFQ in this patchset, in several steps.
This is nothing new, BFQ has been proposed for the kernel before.
LOL ... WTF? by gstoddart · 2016-02-05 06:15 · Score: 4, Funny

Yeah, well I'm taking my AFQ, combining it with my DFQ, and I'm going to EFQ the FFQ out of here.
Take that.

--
Lost at C:>. Found at C.
1. Re:LOL ... WTF? by TimSSG · 2016-02-05 09:26 · Score: 1
  
  I hope you did all that PDQ. Tim S.
  
  Yeah, well I'm taking my AFQ, combining it with my DFQ, and I'm going to EFQ the FFQ out of here.
  Take that.
Let me be the first to say... by Anonymous Coward · 2016-02-05 06:17 · Score: 0

OMGWTFBBQ!!!
KPH
Chain of authorship by tepples · 2016-02-05 06:53 · Score: 3, Interesting

And why are they going thorough the trouble of removing improvements from CFQ?
My guess is to establish a chain of authorship, so that that those things that BFQ shares with CFQ can be correctly attributed to the author of CFQ. Chain of authorship is very important to the Linux project. It dates back to the SCO lawsuit, which ends up being why Git has the --signoff option.

Why not just make an addition one named BFQ?
That might be the ultimate plan: duplicate CFQ, producing a second scheduler identical to CFQ, then apply the heuristic removal patch and the BFQ patch to "Copy of CFQ".
Use of heuristics by postmortem · 2016-02-05 06:57 · Score: 1

Is that admission that original algorithm does not work (in some cases) ?
1. Re:Use of heuristics by Anonymous Coward · 2016-02-05 07:07 · Score: 0
  
  Generalized thing doesn't work as well as one designed for a specific case. STOP THE PRESSES.
  Also, do you schedule for throughput or responsiveness, background or foreground tasks, even or odd procids... Each of those is a tradeoff. A generalized schedule has to at least work decently to mitigate the downsides to their choice of where they fall on the spectrum.
2. Re:Use of heuristics by Tough+Love · 2016-02-05 07:28 · Score: 1
  
  Pretty much.
  
  --
  When all you have is a hammer, every problem starts to look like a thumb.
PDQ and ASAP by JustAnotherOldGuy · 2016-02-05 07:00 · Score: 1

Stupendous! Get the ECN on this sent PDQ so the IUB can patch the IRQ DMOB and regain the benefits of CSR ASAP!!

--
Just cruising through this digital world at 33 1/3 rpm...
1. Re:PDQ and ASAP by Anonymous Coward · 2016-02-05 15:49 · Score: 0
  
  JustAnotherOldBlowhardBULLSHITTER why'd ya lie about working for Microsoft? Can ya prove that statement? No? Thought not. Must be an "NDA" that ya signed with yer fantasyland fake name here online, right? Hahahahaha (the bullshit and foam spewing from JustAnotherOldBLOWHWARD's piehole will ensue - stay tuned, keep yer seatbelts on everyone! Hilarity will ensue, guaranteed!)
2. Re: PDQ and ASAP by Anonymous Coward · 2016-02-06 09:24 · Score: 0
  
  APK, is that you?
Re:Anyone check with Lennart? by Anonymous Coward · 2016-02-05 07:25 · Score: 0

-1? Jeez, you ladies can't take a joke...
BFQ is AWESOME... by BrookHarty · 2016-02-05 07:54 · Score: 5, Informative

Been running PF-Kernel for a few years, which has a bunch of patches, including BFQ, ck patch set with BFS, Tux in ice, UKSM, and grayskys gcc kernel patch. I normally just use the PF Github repo
Love it. With BFQ, you no longer get system pauses on your desktop. I can listen to music or play a video, run a few vms, while a compiler runs in the background, and x-windows doesn't pause, typing doesn't pause, its how a system should act. Your system seems more fluid with no pausing.
PF-Kernel seems to be for Arch/Gentoo/Rpm based, but I've used it on Ubuntu systems. Pf-Kernel isn't the only one, there are other kernels out there that include more performance patches, Xanmod and Liquorix
Re: Anyone check with Lennart? by Earthquake+Retrofit · 2016-02-05 08:06 · Score: 1

Lay off will ya... I've already apologised.

--
Fifty years of Yippie! 1968-2018
Will it help the swap I/O deadlock during memory r by Anonymous Coward · 2016-02-05 11:11 · Score: 0

Will it help the swap I/O deadlock during memory run outs?
Currently when you run out of memory, Linux begins to swap out the old data which makes the machine almost completely unresponsive. If the process wants to allocate a large memory block or you run out of memory due to a continuous memory leak, you are toasted. You probably won't be able to kill the process and end up hard-rebooting your computer.
This issue should be solved.
Such process should just hang waiting for free memory, and swapping off should not cause the machine to go unresponsive.
Re:Will it help the swap I/O deadlock during memor by Anonymous Coward · 2016-02-05 14:41 · Score: 0

Disable memory overcommit with:
sysctl -w vm.overcommit_memory=2
With this, the damn processes just get killed...
Android is way ahead by Anonymous Coward · 2016-02-05 15:11 · Score: 0

Devs experiment with schedulers in Android waaay more than on the workstation Linux side.
I'd think schedulers like Alucard or Wheatly might be even better than BFQ on some systems.
1. Re:Android is way ahead by epyT-R · 2016-02-05 18:47 · Score: 1
  
  mobile != desktop workload.
Re:Anyone check with Lennart? by Anonymous Coward · 2016-02-05 18:32 · Score: 0

That's 'womyn' to you, comrade..