Slashdot Mirror

← Back to Stories (view on slashdot.org)

Distributed Statistical Debugging

Posted by michael on from the data-crunching dept.

Luis Villa writes "The Cooperative Bug Isolation Project at UC Berkeley and Stanford is working on statistical debugging techniques to report, find, and fix the bugs that drive the most users crazy every day. A handful of outside bug volunteers have been running the project's special feedback builds for a few weeks, and that has generated some really interesting data. But for strong results they need more runs. /. has been known to generate those kinds of big numbers ;) Their site has feedback builds of several open source applications, and the entire project is open sourced. Read more about it, then install some applications, and help them make our free software better for everyone. I'm really looking forward to the end results."

12 of 103 comments (clear)

  1. RTFA by jbellis · · Score: 2, Insightful

    this isn't the same thing at all.

    the "stanford bug checker" is a static source analysis program. this is something else entirely, and arguably more useful.

  2. Fix bugs...? by jabbadabbadoo · · Score: 2, Funny
    "statistical debugging techniques to report, find, and fix the bugs"

    A debugger that FIX BUGS?

    What's next? A compiler that brews beer?

  3. Other architectures? by gouldtj · · Score: 2, Interesting

    From my reading of this, it sounds like the data is going to be architecture specific (i.e. x86, PPC). So that means that those hundreds of thousands of samples that are needed, would be needed for every architecture.

    Hmph, mean they probably won't figure out why some programs are seemingly less stable on PPC :( But, I guess many bugs effect every architecture, so I can be happy about that ;)

    1. Re:Other architectures? by Ed+Avis · · Score: 5, Insightful

      Yes, but if you fix a crash-causing bug (such as a buffer overrun) on one architecture, it's likely that the same bug was present on others. Or rather, would also cause a crash on other platforms - since a bug is a property of the source code, the meaning of the program, and the fact that a program happens to work on Tuesday doesn't mean the bug is not there.

      Consider: projects such as openssh and Apache do not issue separate vulnerability warnings for different platforms. If a memory-corruption bug exists, it is probably exploitable on most platforms given the right conditions.

      Still, it's very likely that some particular bug cause crashes often on PPC, but is less likely to be tickled on x86. Then it might not get fixed, because it's less likely to be reported. But if a PPC user does report it, the x86 users benefit from the fix too, even if for them it was a fairly obscure and not-often-noticed bug.

      --
      -- Ed Avis ed@membled.com
  4. Number 1 on the list by one9nine · · Score: 2, Funny

    Duplicates on Slashdot. ;-)

  5. Re:Bugs.... by Benoni · · Score: 2, Informative

    cybermace5 asked:

    How can they tell which bugs are the culprits driving the users crazy every day?

    It's a numbers game. We're looking for statistical trends in large numbers of runs. That means we will learn the most, the most quickly, about the bugs that are happening most often to the most users. Bug triage falls out as a natural consequence of the sparse sampling and statistical modeling approach.

    That said, you suggestion about measuring head-into-keyboard smacks isn't half bad. There are some groups here at Berkeley that work on haptic (touch-based) interfaces; perhaps I should pass that suggestion along to them. :-)

  6. Microsoft's Distributed Statistical Debugging by twoslice · · Score: 2, Interesting
    Microsoft has been using this feature in XP and it apparently does not work. One of our applications crashes frequently for no reason at all. I have told all of our users (1600+) to just click the "SEND" button when it asks to send the "encountered a problem report" back to Microsoft.

    In my calculations Microsoft must have gotten at least 30,000 reports of this bug and it is still not fixed yet...

    --

    From excellent karma to terible karma with a single +5 funny post...
    1. Re:Microsoft's Distributed Statistical Debugging by MrMr · · Score: 2, Insightful

      In my calculations Microsoft must have gotten at least 30,000 reports of this bug and it is still not fixed yet...

      Yes, but what makes you think the feature doesn't work?

    2. Re:Microsoft's Distributed Statistical Debugging by Koyaanisqatsi · · Score: 2, Insightful

      I would click SEND more often myself if the app would give you back a "support ticket number", or a bug ID in the DB so that you could look back later and see how many other entries have been logged for the same error and if a patch is in the making ...

      Now that would be cool! (and very much like an automated bugzilla interface compiled w/ the code ...)

  7. Re:TO America: by 91degrees · · Score: 2, Insightful

    Freedom of speech should not be extended to those who seek to curtail the freedom of speech

    Why the hell not!? Freedom of speech is a right. Not a luxury good to be traded.

  8. Re:Bugs.... by Benoni · · Score: 2, Informative

    In the future we may add the ability for users to manually report non-crash application misbehavior. I know that this is something that The GIMP's developers are very interested in, and we are already doing controlled experiments along similar lines. The underlying statistical debugging techniques still apply.

    But in the current public deployment you are quite right that we only pick up on crash bugs. Consider that the more general project name gives us something to aspire to. :-)

  9. Re:Bugs.... by Benoni · · Score: 2, Informative

    If you're a raving ignoramus, you're a raving ignoramus who asks good questions. :-)

    In our current deployment, we only learn about problems that crash the application. These are important bugs, but they are certainly not the only bugs.

    We are considering ways to let users manually report non-crash misbehavior. I know that this is something that The GIMP's developers are very interested in, and we are already doing controlled experiments along similar lines. The underlying statistical debugging techniques still apply.

    Or to put things into academic terms, "FORMAT MY *$^&$* TABLE CORRECTLY" is future work. :-)