Slashdot Mirror
Taking the Pain Out of Debugging With Live Programming
angry tapir writes "'Everyone knows that debugging is twice as hard as writing a program in the first place,' Brian Kernighan once wrote (adding: 'So if you're as clever as you can be when you write it, how will you ever debug it?') However, Sean McDirmid, a researcher at Microsoft, has been working to remove some of the pain from debugging. McDirmid, based at Microsoft Research Asia, has been studying ways of implementing usable live programming environments: a solution that is less intrusive than classical debuggers. The idea is to essentially provide a programming environment in which editing of code and the execution of code occur simultaneously — and in the same interface as code editing — with tools to track the state of variables in a more or less live manner."
I've personally fallen in love with Visual Studio. It's definitely the planets most feature-rich and capable IDE. It supports wide array of languages, has great debugging options and performs really well. The best thing is that Visual Studio Express is completely free.
It's also great to see Microsoft Research's latest offerings. They have always been the monolith research club of the industry. It's nice to see that Microsoft is really dedicated to support research.
That being said, I'm quite sure we will see the fruits of this research in an even better Visual Studio based product. Microsoft really cares about programmers and developers and helps all of them write efficient and clean code. Helping the debugging process is just second part of it.
I would, however, also love to see better support for debuggers like OllyDbg. It's basically assembler level debugger for programs that have been already compiled. It's binary code analysis is unmatched in the industry. Compiling these two will lead to synergies.
All in all, Microsoft Visual Studio keeps getting better!
This will no doubt become an ad hoc, informally-specified, bug-ridden, slow implementation of half of a REPL.
"'Everyone knows that debugging is twice as hard as writing a program in the first place,' Brian Kernighan once wrote (adding: 'So if you're as clever as you can be when you write it, how will you ever debug it?')
And Stroustroup said, "sounds like weakness! Here, have some templates!"
I've been using this amazing technique for a long time now, IDEs like Eclipse make it fairly easy to do that. TFS doesn't explain what is it that he is doing differently.
You can't handle the truth.
Old pascal compilers had a crude version of this baked right in. You could watch lines of code blink and change color as they ran.
OO makes this kind of idea almost infinitely more complex, however. It was a nice feature for procedural programming, saved me a lot of time.
... this should be in addition to good coding/style standards, proper design, proper source revision control, proper code reviews, and continuous testing/integration. Without any of the former, using this tool does not provide that much information: You first want to know whether your code does what you think it should do, whether it is thread safe, whether it is leaking memory, etc., etc., etc.
Support a Europe-related section on Slashdot!
Forget that, will this be reinventing, dunno, "MFC .NET", where software is effortlessly implemented, tested, and documented using a mix of object classes, C++ templates, custom extensions to C++ that break portability, "wizard" (i.e. obscure template) generated code, a virtual machine, and calls between "managed code" on the virtual machine and native code that break security and prompt stern security scoldings when your code is on a virtual drive or "network share"?
Nothing pisses me off more than programs written dependent on .NET except perhaps every government website that requires you to use IE instead of pretty much every other browser. On further thought, the fact that government websites require you to use effing Lotus to submit forms instead of PDF also sucks.
Prograph did this in the early 1990s. Way to innovate, Microsoft!
and it wants Smalltalk back.
https://en.wikipedia.org/wiki/Smalltalk#Image-based_persistence
They've reinvented Smalltalk. Let's party like it's 1980.
Though not quite the same I like an IDE (like Eclipse) for Live Syntax Checking. What makes it better than a text editor is that It can check syntax on the fly, and remove some of the edit/compile/broken-compile/swear/re-edit/recompile loop.
No, it's not as cool as potential Live Running, but it helps a lot to keep your mind on code flow rather than language syntax.
An approach to address debugging of defects that are easily addressable by other means, albeit with less in the way of eye candy. Wake me up when this approach effectively addresses timing issues.
So, it's a spreadsheet, basically? It isn't a bad idea at all.
Still a dream today .....
VIM editing Perl code in one window, another for an execution trace, and a third to run the program. Ugly and basic but it gets the job done.
how would it integrate data/network/user input?
how does it know when to start live testing, as pieces of interdependent code are being written?
My God can beat up your God. Just kidding...don't take offense. I know there's no God.
"a programming environment in which editing of code and the execution of code occur simultaneously" is commonly called an interpreter, welcome to 1975
Just because you're paranoid doesn't mean they aren't out to get you
For I have been looking for one, lo, these past 27 years.
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
I do not code or develop code any more, but I'm great for finding bugs in code which irks my developers to no end. But conversely, they really like when I test their code, especially when they want to have bugs found. I know how to recall what I did when creating my bug report. I almost never report something as a bug until I can repeat the issue and then I usually re-create the issue two more times to ensure I recall just how I got to that bugged state. I document step by step instructions and follow it up with screen shots. If weird occurrences are also observed, I note those as well. I also try to note down what version of software I am running, because that also helps to narrow down an issue.
While all of the info I provide doesn't always help in debugging all code, it certainly simplifies the process by the developer to figure out what's wrong and where it's taking place. In essence, my good testing and regurgitation of information help cut down debugging efforts by a large percent.
God knows it's better than "I clicked a button and the program blew up".
Life takes interesting turns, but the most interest is when you're off the beaten path.
How is the implementation going to dynamically show me the output of my 1million LOC 3D game? This sort of stuff is useful when scripting for games in Lua, Python or JS but a fibonacci example is not really proving a thing.
While I'm writing code is the easiest time to think about how it might misbehave.
I put in debugging in the form of asserts.
Debugging often consists of just making a strongly typed, picky compiler happy and then clearing the asserts.
Sometimes it takes a week or so before there is enough surrounding code to actually debug anything.
Without the asserts, I have get back to the frame of mind I had early in the week.
With the asserts, I don't.
As a bonus, the asserts are still around for regression testing for later code mods.
The model they propose (simultaneous coding and debugging) doesn't work until you have a 'quorum' of code to do incremental improvements on.
If you want to stop an spend a week restructuring, same story.
But for incremental improvements, sounds great.
Again, you are in the best possible frame of mind to think about how code can misbehave while you are writing it.
Already done, much better, by Bret Victor. See http://vimeo.com/36579366 at time 3:36.
Very nice, but its usefulness is really limited to functions where the parameters are simple objects or native types. Complex objects would require too much initialization to make the unit test worthy.
Here's what would be more useful for anyone out there working on debuggers to consider:
Typical step debugging has a really pathetic interface with the following interface:
- break
- step
- step over
- step out
- resume
I've been using that now for 20 years. What would be nice would be to add the following:
A resume with a reduced speed option. In other words, start stepping with a slight delay, with visual feedback, so the developer can see the code as it is stepped through. Allow the developer to control the speed of the stepping (mouse-wheel or some dial) as well as dynamically prompt the debugger to step out (climb back up the stack) when the debugger gets too far down into weeds.
Replay. Vmware is making this happen with their tools, but it would be nice if this were more ubiquitous. The developer should be able to start "recording", and every memory/register change from then on is tracked, such that the developer can rewind instead of only being able to move forward.
And for anyone working on language development, my 20 years experience has shown that the null-pointer exception is the most common problem that people run into. It would be nice if coding languages enforced natively the concept of a object reference/pointer parameter not being null at compile time. The C/C++ reference (&) comes close, but you can still circumvent it by simply de-referencing the pointer. You can get around this with templates/generics/etc but it is a hassle.
Please implement.
Thanks
Fred Brooks noted this one years ago, and it's still true: The reason programming is complicated is frequently because the real-world problem you're trying to model in software is complicated. The reason debugging is even more complicated is that not only do you need to understand what you're trying to model, you also need to understand exactly how the existing software modeled the real-world problem.
I am officially gone from
INSERT...
No!!!!!!!!!!
At first glance, I just want to say, "Hey, Basic Interpreter from the 70's, you're finally getting some respect!"
But there are some deep issues that are, even with these techniques, going to be very hard to debug:
1) Large I/O situations -- real time data collection and display. Weather reporting, gaming, etc. How is debugging on the GPU going to be helped by this, would be one of my first questions.
2) Networked, distributed code -- with clients on multiple platforms (CPU, browser, Java version, etc.), and clustered virtual servers, where do you debug? Just locating which side is in error (wrong message being sent, wrong message being heard, wrong result being stored/displayed) is a challenge, and real-time transactions (for instance, race conditions on a resource that could be anything from an interrupt bit to a piano in inventory) further complicate this.
If they ever come up with a debugger that will let me hook into both the client and the server at the same time, that would be brilliant.
Design for Use, not Construction!
Cool, I knew this would make a comeback from the 60s or something.
I want to play Free Market with a drowning Libertarian.
Pretty sure these were features of VB6. I remember hacking out code, using the immediate window to trace values, setting break points, stepping through the code, modifying in the middle of execution, and then resuming execution. The language itself may have had a lot of issues and performance issues, but the IDE and development environment had some very nice features.
The problem is that so many programmers need debuggers in the first place. That is a sign of low competency levels. I found that using methods from design by contract basically eliminate the need for debuggers as they lead to much, much better code and give you errors that are directly meaningful. Of course, this requires thinking about what you want to program before doing so, as you are actually implementing a check of its implementation before you implement the actual code. And it can be done in any language once you have understood the concept. The only time I use a debugger is when diagnosing obscure problems, but that requires a lot of thinking about the state I see anyways, and DDD is perfectly fine for it.
This tool does nothing than making programming seemingly accessible to programmers of even lower skill. Writing code that does what you want is easy, but writing it as simply as possible (maintainability), with clear and clean interfaces, robust, aware of special cases, fast, etc. is not. Hence the only thing this tool will is to decrease code quality overall.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I just want the ability to detect a fault and then run my entire system backwards to figure out where the problem came in. Wind River Simics can do this, but it's expensive and time-consuming to get a model of your hardware unless it happens to be something they already support. It's also slow to run (as you'd expect).
Hah, this techinque is called PHP+println() and is not new by any means. I programmed PHP live with 50-100 clients ready to kill me if i clicked Ctrl-S a bit too early. Noobs.
Can I light a sig ?
This looks soooo much like the way smalltalk engines where implemented in the 80's it's boring.
One can always count on MS to bring the latest in software innovation
I'm trying to get modded "Interesting Flamebait Informative and Insightful Redundant Troll" *-* Please Help *-*
I need to read up on this but my first reaction is that you wouldn't run production code in this new debug environment, so how do you know code that works in this new debug environment will work after being compiled (to .NET or native code - remember MS is pushing C++ native code again, and this new debug environment sure wouldn't do much with C++!) and put into production?
But the most difficult bugs are not the crashes or randomization introduced by the thread completion order. What is difficult is the bug that happens after several hours of computation. Longer it takes for the bug to manifest itself, harder it is to debug. Of course it is my luck I end up with coding for such products. Not that I am complaining. It needs a special skill have such products humming along, and it takes a long time on the job to develop it. The pay, benefits and job security are good. So I am not threatened ...
BRB
What Bob? Boss wants to see me? Why is she having these gentlemen from security and the HR blond boy with her?
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Good work. Great to see R&D money is not going to waste.
Might want to talk to the VB6 guys if there are any left.
My first reaction was:
I've been doing this in python for at least 5 years, and it's probably been around longer.
Then I thought for a second about a CS prof I had who had a MIPS course where they did this in emacs back about 2000...
Then I thought about EMACS.... oh yeah... LISP, SCHEME, rep loops...
Then I thought on LISP -- oh yeah, MIT built fucking hardware and o/s that did this shit in the 70's.
I may find the idea of LISP fairly distasteful... but this idea is nothing new under the sun. It's in fact older than I am, and older than some of my friends that have kids...
McDirmid notes that there are limitations to his prototype. For example, the approach used by LPX won't work for "interactive programs whose inputs consist of time-ordered events and whose outputs can vary over time
You mean, such as programs which offer a user interface, and which both write to and read from some kind of persistence layer or document storage on the backend?
Well, at least there can't be too many of those....
Koans and fables for the software engineer
Why would anyone try to write a program all at once, then debug? I've always written software in pieces, and each piece gets compiled and tested before moving further along. That irons out 99% of the bugs, and using an actual debugger is unnecessary (printed messages from testing can reveal any bug). The remaining 1% of bugs jump out along the way (or after finishing), and looking at the code with a problem in mind is all it takes to find and fix them.
No debugger or elaborate runtime compiler necessary.
What's funny about this article is it's focused on a very limited text based debugging system where the author is already apologizing for bugs while demo'ing it.
It mentions a quote from an Apple guy on the same topic. Wait a minute... Apple is working on this too? So you click the link and find a much better article with a similar system that's way more advanced and live connects the graphics with the code.
Just kind of sad, I RTFA and think "Huh, that's interesting, someday" then check out the link inside the article and find a much more informative and interesting story that I'm still reading. Read THAT article instead. Looking forward to seeing this creep into Xcode updates.
Cwm, fjord-bank glyphs vext quiz
This sounds like Light Table for Clojure:
http://www.lighttable.com/
It's supposed to help you see the flow of data through your program, as you're coding, by executing things and showing you the results in the IDE.
What's wrong with very short write-compile-test-fix work patterns? Instead of writing code for weeks on end, do very short write-save-compile cycles, to make sure your code at least compiles most of the time, then, when you think your subfeature is testable, take a break from spilling code and test that what you wrote does what you think it does, then fix it until it actually does what you meant it to do. While at it, you'll also notice that you can take some the code you just wrote and move it into its own reusable module.
Oh, right... you need to understand what the hell you are doing :)
VB6 had very functional edit-and-continue - it took MS a lot of versions of .NET to get back up to the same general functionality.
Why not check if you're right before randomly gainsaying someone?
My first suggestion would be is for them to implement WinDbg in an actual usable way. The cryptic commands and setting up the directories for the source and .pdb files.....sheesh.
The idea is good, but not new : http://vimeo.com/36579366 (Bret Victor's Inventing On Principle)
I'm not sure who remembers the still somewhat LISP-centric Light Table IDE by Chris Granger, who was looking for kickstarter funding last year. One of the selling points of his IDE was that it's all real-time execution/evaulation, so you see your output in real time.
I'm not entirely familiar with it, but they've got Python and Javascript integration as well, and as Chris stated, any language with a dynamic runtime is a viable option.
Funny enough, he was in charge of some parts of the Visual Studio IDE, like the C# and Visual Basic 'experience', as well as managing?contributing? other bits. I wonder if this is just 3-years-later trickle down of his ideas?
From www.lighttable.com: Light Table is a new interactive IDE that lets you modify running programs and embed anything from websites to games. It provides the real time feedback we need to not only answer questions about our code, but to understand how our programs really work.
Not for sure I agree that debugging is harder than writing code.
Writing GOOD code is hard. Debugging good code is easy.
I've always enjoyed debugging and assuming the code is
decently written, I usually find it fairly easy to track down and
fix a bug.
The idea is to essentially provide a programming environment in which editing of code and the execution of code occur simultaneously — and in the same interface as code editing — with tools to track the state of variables in a more or less live manner."
Not that I'm a Luddite resisting new tools and ideas, but wouldn't a REPL be good enough for most cases? Most of my professional life I worked with separate editing, compiling and debugging sessions, a necessary PITA for the tools at my disposal. But ever since I started using/playing with languages that come with a REPL (Python, Ruby, Scala), it's been like a productivity blessing where one can break portions of the code into (semi)independent, coherent groups of lines that can be tried out again and again and again.
The side effect of this (when I'm a liberty to use a tool with a REPL) is that my code (already designed for testing in mind) becomes even more testable, and firing a full-blown debugger is something I have to do only for very difficult-to-find bugs. I certainly hate when I have to work with an environment (C and Java, which is what I mostly do) without it (even though that is how I've been working most of my professional life).
So I'm wondering what else can one get with the new approach that is substantially better than a REPL-based development environment.
Anybody have an explanation about how this is different than what LLDB and LLVM can do? You can debug running code and modify the executable on the fly already.
Very little good code gets written. Most code is created by people with far more enthusiasm than either skill or knowledge. Sadly, while the lack of skill is forgiveable (not everyone has innate ability), the lack of knowledge is not. Computer Science enjoys a massive library of very well written resources (as well as a sea of badly composed trash).
A bad programmer (or a lazy one, or one that works in a job where they don't give a damn) loves the formal debug tools, where one does almost nothing while the debugger runs automatically, spewing out megabytes of 'data' that the programmer can convince his 'pointy-haired boss' will take weeks of careful examination before the project can move forward. No good programmer EVER relies on these tools (and many never use them at all).
This article is about something else. The 'fantasy' of modular programming, and the ease with which a module can be turned into 'live' code for testing. Good programmers already do a lot of this where practical, but such a practise CANNOT be turned into a universal model for programming. We know this because the idea of modular programming has existed for ever, and the dream of solving the idea of correct code this way has existed for just as long. The pile of s**t programming language ADA was created from just such a fallacy.
In reality, any complex project will have significant amounts of bottom-up, AND top-down design. Bottom-up we can now consider as 'libraries', the closest thing to universally testable code-engines. In practise, 'libraries' prove to have as many enduring bugs as any other code-base, when the library is anything other than trivial. This happens because of 'common path' bias- the bias that means any library has commonly used elements likely to be rapidly corrected, and obscure options that may possibly get no common use whatsoever. Anyone using core open-source libraries will know their buginess is legendary, if one dares to attempt to use any of the less popular functions.
Modular coding using requires the creation of test-bed datasets- a programming task in itself. The language used to create the module is usually not appropriate for easily creating the test-data- an issue live programming concepts completely ignore. Being able to trivially run code is likely to lead to lazy coding habits, not superior ones. Live coding is good for prototype hacking, but every real programmer knows that all too frequently, prototype code that seems to work ends up being used as finished code.
Code is made good by an act of will. A discipline that many coding environments will not encourage. Ideally, code should be tested close to its time of creation by unrelated people with a high incentive to identify issues. As a project grows, the mathematical permutations that describe the near infinite ways code blocks can interact with one another allows a 'bad' project to excuse complete failure. After all, how can you debug the 'infinite'? Good projects have top-down designers who stay on top of the issues by having expert knowledge of computer science (appropriate algorithms and data structure methods), and understand the capabilities and limitations of the hardware used. The problem is that to managers, good and bad designers/programmers can sound the same- people who spew a sea of buzzwords with confidence.
In the end, programmers do need better tools, but every tool tends to make this boast. Access to modular programming methods are very important, but programmers should ALWAYS be encouraged to work with multiple languages at the same time. Good test data will only be produced/used if the creation of said data is a 'piece of cake', and that almost always requires the use of languages different from the ones used to do the core coding. Debugging by tracking the state of variables in a brain-dead automatic way is a joke. Real programmers have no issue inserting 'print' commands that emit their output to some 'terminal' device.
Didn't Chuck Moore implement this already 20 years ago?
So, now the developer has this great tool that runs on their desktop that helps them debug the program. Now what happens when it fails in the field? Did the developers spend the time to create sufficient tracing/logging functions so that the end user can report the failures?
The largest problem isn't "debugging" during development, its "debugging" post failure analysis of field problems. Its such a large problem that nearly all "software engineering" and related process is focused on moving the debugging aspects to the developers machine, rather than getting the end customer involved.
When I first started writing code, I spent a lot of time in step/trace debuggers. Now I would say its probably less than 1% of my development time. That is for a number of reasons, the least of which is that I can almost always "see" the bugs in the code without having to run it. For code I'm writing, the output is almost always sufficient to find the bug without really having to think about it. That is something I couldn't do 15 years ago.
Yeah, if you're as clever as you can be in programming, no wonder debugging is hell. Years ago, I decided I wanted *elegant* code, not "clever" code. When it's 16:15 on a Friday, or 02:10 any night, and I get a phone call about a problem, I want to be able to leave on time Friday, or go back to sleep that night, not spend *hours* trying to figure out how "clever" I or someone else had been a year and a half ago, and if it was someone else, they don't work here any more....
mark
Here's at thought. Why not work at writing good code to start with. I know, it's hard, and you actually have to work at it, but if you don't write crap to begin with then debugging it becomes 10 times easier.
And idea very similar to this is already in development. It's called Light Table
http://www.lighttable.com/
So, he's reinvented FORTH 50 years later and using 5000x more lines of code, and FORTH still does it better! ROFLMAO! Ah, Charlie Moore, I doubt you read /. but you will never be dethrowned as the author of the most elegant code ever. Thx!
"Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
Its 100x better for this than any of the languages you mention, higher performance, generally, and achieves VERY high rates of internal reusability to boot. I literaly wrote 4.5 million lines of fully debugged functioning FORTH code in 5 years. It certainly isn't a FASHIONABLE tool, but its frighteningly effective.
"Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
Why
In the name of God
Are you writing like this?
Live programming has been available for decades and is the common working mode of any Smalltalk programmer. (Or Forth for that matter) Talk about nothing new under the sun. We've wondered why anyone would work any other way, for decades.
If your programming is that clever, that you cannot completely understand it at execution (debugging) time, then you are not that clever. The code should always be so simple, that you can understand it fully. That you can understand the border cases.
It looks like he made one of the more basic pieces of software shown by Bret in this video from a year ago: http://vimeo.com/36579366 Check out his more advanced content creation demo @29:10 in this video.
Many have noted other examples of these kinds of capabilities. I first saw them debugging on a Multics system.
Researcher from Microsoft refuting the God of good programming? I can totally trust this guy!
Debugging is for the weak
Again, APL solved this in the 1960s: any crash saved a debug workspace with all state preserved, halted at the point the error was detected. You could cut back the stack and step through to the error (assuming the code was written with sufficient modularity).
"Inventing on Principle"
http://www.youtube.com/watch?v=PUv66718DII
In this guy's system, one doesn't even have to stop execution while editing.
I've been working in G2 for most of the past 15 years, and this article describes the strengths of that environment, even 15 years ago. The "inspect" interface is like a troubleshooters wet dream. I don't know why something similar hasn't been in every platform for the last decade.
Uh, yah any unix/windows machine can store core dumps. That gives you one piece of the puzzle, but is rarely sufficient, especially if its an error in processing or whatnot and the system continues to run due to error handling.
Java and IntelliJ IDEA. Best IDE there is, in my experience. Makes refactoring code very easy compared to pretty much everything else out there. (And I've used Visual Studio, Eclipse, and others, so I do have a basis for comparison.) You can improve some of the others with plugins like ReSharper, though.
Note, by the way, that being able to edit code and update a running program with it is fairly standard for Java IDEs. The Java VM has supported this for a long time.
Wow, I never noticed this development ( back while I was travelling for two years I guess )
Glad to see there are Linux options with full integration into a gui (Eclipse ). They show forward and back stepping through code but I'm not sure it can do the reverse memory watch like the vmware video shows?
http://undo-software.com/
any others we should be aware of?
In other words, are reinventing live kernel runtimes like the ones on some lisp, smalltalk or oberon
Hell, i used to do every point in their bullet list... for free! on windows! years ago!
in the freeware Dolphin SmallTalk Comunity Edition or in the open source
BlackBox Component Pascal ( some kind of derivative Oberon2 Framework )
On dotnet? maybe with zonnon
..is to avoid bugs when coding. There was a long time, maybe 40 year, since the last "new" type of bug appeared. So you know what type of bugs to avoid. It's just your time and /or lazyness that causes bugs. And, of course, lack of knowledge. Which you should have researched before started coding.
Mundus Vult Decipi
... a typical LISP environment from the 1990s, such as a Symbolics machine. How much we forget!
"Everyone knows that debugging is twice as hard as writing a program in the first place." I write software for a living, I disagree very strongly with that statement. Debugging even the most heinous inherited codebase is monkey work compared to designing good software. When you're debugging, you know exactly what's wrong, you almost always choose the least invasive change that will fix it, and then you test it and verify that it's fixed. It is orders of magnitude more challenging to have the complete freedom to write whatever you want while trying to balance simplicity with flexibility and deadlines with quality.
FTFY I did.
As usual, Microsoft thinks they invented something that's actually already been around awhile. Back to your lives, citizens.
We could call it "Smalltalk-80"!