Slashdot Mirror
The Law of Leaky Abstractions
Joel Spolsky has a nice essay on the leaky abstractions which underlie all high-level programming. Good reading, even for non-programmers.
← Back to Stories (view on slashdot.org)
← Back to Stories (view on slashdot.org)
Because the first step to solving any problem is always to create more problems.
-E
http://almostsmart.com
Great! I'll print off a hardcopy and stick it on my refrigerator! I'm sure my wife will love it!
Although I used to program as a hobby, my eyes bugged out when I saw this article. It's actually quite interesting; I finally realize why the hell people program in lower level languages.
One point that I think could be addressed is backward compatibilty. I really know nothing about this, but don't the versions of the abstractions have to be fairly compatible with each other, especially on a large, distributed system? This extra abstraction of an abstraction has to be orders of magnitude more leaky. The best example I can think of is Windows.
I'm of the idea that the whole premise that high-level tools and high level abstraction coupled with encasulation are the biggest bane of the software industry. We have these high level tools which most programmers really don't understand and are taught that they don't need to understand in order to build these sophisticated products.
Yet, when something goes wrong with the underlying technology they are unable to properly fix their product because all they know is some basic java or VB and they don't understand anything about sockets or big-endian/little endian byte alignment issues. It's no wonder todays software is huge and slow and doesn't work as advertised.
The one shining example of this is FreeBSD, which is based totally on low level C programs and they stress using legacy program methodologies in place of the fancy schmancy new ones which are faulty. The proof is in the pudding, as they say, when you look at the speed and quality if FreeBSD, as opposed to some of the slow ponderous OS's like Windows XP or Mac OSX.
Warmest regards,
--Jack
Wagner LLC Consulting Co. - Getting it right the first time
+3 Informative? Trust me, this is information that I don't need...
Well I wouldn't say that it's reliable "because there are timeouts". AAMOF, timeouts just compicate things. So you timeout waiting for packet N, you request a resend of it, and in the interim, guess what, packet N shows up, now you have two N's. Your code is now more complex in having to deal with this situation. Timeouts are just another parameter used adjust the behaviour of the algorithms that control the protocol. Getting deterministic results from an undeterministic foundation involves making observations, accepting some compromises, making some simplifying assumptions, and then writing code that takes all those things into account to come up with something that usually works.
"Reliable" means "always works", it doesn't mean "always obeys the spec". (Unless you use a circular definition)
A timeout is a legal result by the TCP specification, but it's not reliable, because your data didn't make it through.
By the IP specification, your data might not make it either- and that's a legal result because the spec allows it to drop packets for any reason at all. That doesn't mean IP is reliable, just that it obeys its own definition.
Of course, no real protocol can ever meet this restrictive definition of reliable. Some maniac can always cut through your wires or incinerate your CPUs. Calling TCP a "reliable protocol" is just a shorthand for "as much more reliable than the underlying protocols as we could manage"
The timeout you mention does make TCP more reliable than IP, because it alerts you to the data loss, where the application can possibly take steps to retransmit it sometime in the future.
But its not as if TCP could ever achieve the perfect reliablity that the simplest, most abstract description of it would imply. Which is why, as the author says, those who rely on the abstractions can get bitten later.
Is our own bodies.
I'm studying to be a bioinformatics guy with the university of melbourne and have just had the misfortune of looking into the enzymatic reactions that control oxygen based metabolism in the human body.
I tried to do a worst case complexity analysis and gave up about half way through the krebs cycle.
When you think about it, most of basic science, some religeon and all of medicine has been about removing layers of abstraction to try and fix things when they go wrong.
Unfortunately, his Slashdotted server is proving that to us right now.
As a VB programmer, I've *lived* leaky abstractions. Nowhere has it been more obvious than in the gigantic VB app our team is responsible for maintaining. 262 .frm files, 36 .bas modules, 25 .cls classes, and a handful of .ctl's.
Much of our troubles, though, come from a single abstraction leak: the Sheridan (now called Infragistics) Grid control.
Like most VB controls, the Sheridan Grid is designed to be a drop-in, no-code way to display database information. It's designed to be bound to a data control, which itself is a drop-in no-code connection to a database using ODBC (or whatever the flavor of the month happens to be).
The first leak comes in to play because we don't use the data control. We generate SQL on the fly because we need to do things with our queries that go beyond the capabilities of the control, and we don't save to the database until the client clicks "OK". Right away, we've broken the Sheridan Grid's paradigm, and the abstraction started to leak. So we put in buckets -- bucketfuls of code in obscure control events to buffer up changes to be written when the form closes.
Just when things were running smoothly, Sheridan decided to take that kid with his finger in the dike and send him to an orphanage. They "upgraded" the control. The upgrade was designed to make the control more efficient, of course... but we don't use the data control! It completely broke all our code. Every single grid control in the application -- at least one and usually more in each of 200+ forms -- had to have all-new buckets installed to catch the leaks.
You may be wondering by now why we haven't switched to a better grid control. Sure enough, there are controls out there now that would meet 95% of our needs... but 1) that 5% has high client visibility and 2) the rest of the code works, by golly! No way we're going to rip it out unless we're absolutely forced to.
By the way, our application now compiles to a svelte 16.9 MEG...
Stressed? Me? Of course not. Stress is what a rubber band feels before it breaks, silly.
FIne it's relaibale becasue of acks, timeouts, adaptive re-transmit timeouts that take statistical averages of RTT times, exponential back-off and slow start, window acks which keep track of what bytes are received, etc.
So in your case of timing out N, re-tx'ing N, and then getting the repsonse to the first N back after sending the second N, you do two things:
1) Good! You got yr packet!
2) keep track of how many bytes you have received thsu far (TCP is not sending messages, it is sending a stream)
3) when you get the response from your second request, discard it, becuase you already received those bytes from the stream.
4) since you timed out, DON'T use the Round TRip Time for that reponse: slow down your expected RTT time, and THEN start measuring.
And guess what? If I unplug the NIC of the other machine, there is no reliable way of transmitting that data (assuming your destination machine isn't dual homed)- so I keep streaming bytes to a TCP socket and I don't find out my peer is gone for approx. 2 minutes.
WOW. There's nothing reliable about that boundary condition!
my point is TCP is reliable ENOUGH. But I wouldn't equate it with a Maytag warranty. It is not a panacea. Infact, for a closed homogenous network I wouldn't even consider it the best option. But if the boundary conditions fall within the acceptible fudge range (remember Real Time human grade systems are not 100% reliable, only 99.99999% and much of that is achieved through redundancy) your leaks are ok.
In the future, I would want to not be isolated from my friends in the Space Station.
The market rewards abstractions because they help create high level tools that get products on the market faster. Classic case in point is WordPerfect. They couldn't get their early assembler-based product out on a competitive schedule with Word or other C based programs.
Proper abstractions avoid unintended side-effects by presenting a clean view of the intent and function of a given interface, and not just a collection of methods or structures.
When I read what Joel wrote about "leaky abstractions" i saw a peice complaining about "unintended side-effects". I don't think the problem is with abstractions themselves, but rather the implementation.
He lists some examples:
1. TCP - This is a common one. Not only does TCP itself have peculiar behavior in less than ideal conditions, but it is also interfaced with via sockets, which compound the problem with an overly complex API.
If you were to improve on this and present a clean reliable stream transport abstraction is would likely have a simple connection establishment interface and some simple read/write functionality. Errors would be propagated up to a user via exceptions or event handlers. But the point I want to make is that This problem can be solved with a cleaner abstraction.
2. SQL - This example is a straw man. The problem with SQL is not the abstraction it provides, but the complexity of dealing with unknown table sizes when you are trying to write fast generic queries. There is no way to ensure that a query runs fastest on all systems. Every system and environment is going to have different amounts and types of data. The amount of data in a table, the way it is indexed, and the relationship between records is what determines a queries speed. There will always be manual performance tweaking of truly complex SQL simply because every scenario is different and the best solution will vary.
3. C++ string classes. I think this is another straw man. Templates and pointers in C++ are hard. That is all there is too it. Most Visual Basic only coders will not be able to wrap their minds around the logic that is required to write complex c++ template code. No matter how good the abstractions get in C++, you will always have pointers, templates, and complexity. Sorry Joel, your VB coders are going to have to avoid c++ forever. There is simply no way around it. This abstraction was never meant to make things simple enough for Joe Programmer, but rather to provide an extensible, flexible tool for the programmer to use when dealing with string data. Most of the time this is simpler, sometimes it is more complex (try writing your own derived string class - there are a number of required constructors you must implement which are far from obvious) but the end result is that you have a flexible tool, not a leaky abstraction.
There are some other examples, but you see the point. I think Joel has a good idea brewing regarding abstractions, complexity, and managing dependencies and unintended side-effects, but I do not think the problem is anywhere near as clear cut as he presents. As a discipline software engineering has a horrible track record of implementing arcane and overly complex abstractions for network programming (sockets and XTI) generic programming (templates, ref counting, custom allocators) and even operating systems API's (POSIX).
Until we can leave behind all of the cruft and failed experiments of the past, start new with complete and simple abstractions that do not mask behavior, but rather recognize it and provide a mechansim to handle it gracefully, we will run into these problems.
Luckily, such problems are fixable - just write the code. If joel were right and complex abstractions were fundamentally flawed, that would be a dark picture indeed for the future of software engineering (it is only going to grow ever more complex from here kids - make no mistake about it).
The problem that this article points to is a byproduct of large scale software development primarily being an exercise in complexity management. Abstraction is the foremost tool available in order to reduce complexity.
In practice a person can keep track of between 4 and 11 different concepts at a time. The median lands around 5 or 6. If you want to do a self-experiment have someone write down a list of twenty words, then spend 30 seconds looking at them without using memnonic devices such as anagrams to memorize them then put the list away. After thirty more seconds write down as many as you can recall.
This rule applies equally when attempting to manage a piece of software - you can only really keep track of between 4 and 11 "things" at the same time, so the most common practice is to abstract away complexity - you reduce an array of characters terminated by a null characters and a set of functions designed to operate on that array to a String. You went from half a dozen functions, a group of data pieces, and a pointer to a single concept - freeing up slots to pay attention to something else.
The article is completely correct in its thesis that abstractions gloss over details and hide problems - they are designed to. Those details will stop you from being productive because the complexity in the project will rapidly outweigh your ability to pay attention to it.
This range of attention sneaks into quite a few places in software development:
other schemes exist for managing complexity, but abstraction is decided human - you don't open a door, rotate, sit down backwards, rotate again, bend legs, position your feet, extend left arm, grasp door, pull door shut, insert key in iginition, extend right arm above left shoulder, grasp seatbelt, etc... you start the car. Software development is no different.
There exist peopel that can track vast amounts of information in their heads at one time - look at Emacs - iirc RMS famously wrote it as he did because he could keep track of what everythign did, no one else can though. There also exist memnonic devices aside from abstraction for managing complexity - naming conventions, taxonomies, making notes, etc.
-Frums
We have examples of massively complex systems that work very reliably day-in and day-out. Jet airplanes, for one; the national communications infrastructure, for another. Airplanes are, on the whole, amazingly reliable. The communications infrastructure, on the other hand, suffers numerous small faults, but they're quickly corrected and we go on. Both have some obvious leaky abstractions.
The argument works out to be pessimism, pure and simple -- and unwarrented pessimism to boot. If it were true that things were all that bad, programmers would all _need_ to understand, in gruesome detail, the microarchitectures they're coding to, how instructions are executed, the full intricacies of the compiler, etc. All of these are leaky abstractions from time to time. They'd also need to understand every line of libc, the entire design of X11 top to bottom, and how their disk device driver works. For almost everyone, this simply isn't true. How many web designers, or even communications applications writers, know -- to the specification level -- how TCP/IP works? How many non-commo programmers?
The point is that sometimes you need to know a _little bit_ about the place where the abstraction can leak. You don't need to know the lower layer exhaustively. A truly competant C programmer may need to know a bit about the architecture of their platform (or not -- it's better to write portable code) but they surely do not need to be a competant assembly programmer. A competant web designer may need to know something about HTML, but not the full intricacies of it. And so forth.
Yes, the abstractions leak. Sometimes you get around this by having one person who knows the lower layer inside and out. Sometimes you delve down into the abstraction yourself. And sometimes, you say that, if the form fails because it needs JavaScript and the user turned off JavaScript, it's the user's fault and mandate JavaScript be turned on -- in fact, a _good_ high-level tool would generate defensive code to put a message on the user's screen telling them that, in the absence of JavaScript, things will fail (i.e. the tool itself can save the programmer from the leaky abstraction).
What Ian Malcolm says, when you boil it all down, is that complex systems simply can't work in a sustained fashion. We have numerous examples which disprove the theory. That doesn't mean that we don't need to worry about failure cases, it means we overengineer and build in failsafes and error-correcting logic and so forth. What Joel Spolsky says is that you can't abstract away complexity because the abstractions leak. Again, there are numerous examples where we've done exactly that, and the abstraction has performed perfectly adequately for the vast majority of users. Someone needs to understand the complex part and maintain the abstraction -- the rest of us can get on with what we're doing, which may be just as complex, one layer up. We can, and do, stand on the shoulders of giants all the time -- we don't need to fully understand the giants to make use of their work.
Neal Stephenson talks about something similar in In the Beginning was the Command Line. He calls it interface shear; he's specificially referring the the UI as an abstraction (an interesting idea in itself). His take on it was that abstractions are metaphors, and that "interface shear"/"leaky abstractions" occur in regions where the metaphors break down.
Interesting stuff...
I think the article is great. And this principle can also be applied to Math. Theorems are much like library function calls. You can use them in your own proofs, without caring about how they are proved, because someone has already taken care of that for you. You prove that the hypothesis are true, and you get a result which is guaranteed to be true.
The problem is that in real Math, you often need a slightly different result, or you cannot prove that the hypothesis are true in your situation. The solution often involves understanding what's "under the hood" in the theorem, so that you can modify the proof a little bit and use it.
Every professional mathematician knows how to prove the theorems that he/she uses. There is no such thing as a "high-level mathematician", that doesn't really know the basics, but only uses sophisticated theorems in top of each other. The same should be true in programming, and this is what the article is about.
The solution? Good education. If anyone wants to be considered a professional programmer, he/she should have a basic understanding of digital electronics, micro-processor design, assembly language (at least one), OS architechture, C, some object oriented language, databases... and should be able to understand the relationship between all those things, because when things go wrong, you may have to go to any of the levels.
It's a lot of things to learn, but there is no other way out. Building software is a difficult task and whoever sells you something else lies.
In physics the abstractions leak. Newton's laws leak like crazy. Einstein's theories leak. Presently there are no fundamental theories in physics which don't leak like crazy when quantum mechanics and gravity interact.
In sports the abstractions leak. That's how we get players like Gretzky and pay a lot of money to watch what they do.
And how about the reason why didn't C++ didn't define a native string type. Because there isn't any way to implement a string class that serves all possible applications. The premise of C++ is not being stuck with someone else's choice on what part of the abstraction should leak. Because C++ doesn't define a native string type, the user is free to replace the default standard string implementation with any other string implementation and have it integrate with the language on an equal footing with the standard string type.
If a language is imposes standard abstractions it only takes one abstraction you can't live with to make that choice of language untenable. Which is how C++ has been so successful despite being the worst of all possible languages (except for all the others).
I agree with Joel, but some people seem to be taking it as a call to stop abstracting. That's silly.
Humans form abstractions. That's what we do. If you abstractions are leaking with detrimental consequences, then it could be because the programming language implementation you're using is deficient, not because you shouldn't be abstracting.
Try a high-performnce Common Lisp compiler some time. Strong dynamic typing and optional static typing, macros, first class functions, generic-function OO, restartable conditions, first class symbols and package systems make abstraction much easier and less prone to arbitrary decisions and problems that are really:
(i) workarounds for methods-in-once-class-rule of "ordinary" single-dispatch OO
(ii) workarounds for the association of what an object is with the name of the object rather than it itself (static typing is really saying "this variable can only hold this type of object", dynamic typing is saying "the object is of this type". Some languages mix these issues up, or fail to recognise the distinction.
(iii) workarounds for the fact that most languages, unlike forth and lisp, are not themselves extensible for new abstractions
(iv) workarounds for the fact that one cannot pass functions as parameters to functions in some languages (doesn't apply to C, thanks to function pointers - here's where the odd fact that low level languages are often easier to form new abstractions in comes in)
(v) workarounds for namespace issues
(vi) workarounds for crappy or nonexistent exception processing
Plus, Common Lisp's incremental compile cycle means faster development, and it's defined behaviours for in place modifications to running programs makes it good for high-availability systems
"1) Good! You got yr packet!"
It seems like your keyboard keeps dropping packets. Could we have a repost of this comment?
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
While I usually like Joel's work, I'm pissed about the random jab at C++. For those he didn't read the article, he says something along the lines of
"A lot of the stuff the C++ committe added to the language was to support a string class. Why didn't they just add a built-in string type?"
It's good that a string class wasn't added, because that lead to templates being added! And templates are the greatest thing, ever!
The comment shows a total lack of understanding of post-template, modern C++. People are free not to like C++ (or aspects of it) and to disagree with me about templates, of course, and in that case I'm fine with them taking stabs at it. But I get peeved when people who have just given the language a cursory glance try to fault it. If you haven't used stuff like Loki or Boost, or taken a look at some of the fascinating new design techniques that C++ has enabled, then you're in no place to comment about the language. At least read something like the newer editions of D&E or "The C++ Programming Language" then read "Modern C++" before spouting off.
PS> Of course, I'm not accusing the author of being unknowledgable about C++ or anything of the sort. I'm just saying that this particular comment sounded rather n00b'ish, so to speak.
A deep unwavering belief is a sure sign you're missing something...
Loved this article. Sent it on to my manager and a co-worker.
.NET effectively. Unless one understands some of the underpinnings of this NEW technology, you actually can't take advantage of it. Throw in the generated code issues and the IDE, an abstraction of an abstraction, really is disadvantageous.
One thing I liked especially is the danger of the Shiny New Thing. It may be neat and cool and save time, but knowing how to use it does not mean that you can do anything else - or function outside of it.
Right now I'm on an ASP.NET project - and some ASP.NET stuff I actually like. But the IDE actually makes it harder to program responsibly, and even utilize
A friend of mine just about strangled some web developers he worked with as they ONLY use tools (and they love all the Shiny New Ones) and barely know what the tools produce. This has led to hideous issues of having to configure servers and designs to work with their products as opposed to them actually knowing how they work. The guy's a saint, I swear.
I think managers and employers need to be aware of how abstract things can get, and realize good programmers can "drill down" from one layer to another to fix things. A Shiny New Thing made with Shiny New Things does NOT mean the people who did it are talented programmers, or that they can haul your butt out of a jam when the Shiny New Thing looses its shine.
"The Sage treasures Unity and measures all things by it" - Lao Tzu
Maybe I'm an old fashioned has-been but people doign software development should understand the fundamentals of how computers work. That means that they should understand things like memor management, they should understand what a pointer is, they should undertsand about how tight loops versus unrolled loops might affect the performance of the caches on their system. I meet so many "programmers" that have no understanding that there are architectural constraints on what they can and can't do. Software runs on hardware. If you're going to write software and treat the hardware as a black box, you're not going to write it as well, or as efficiently as you could be doing it.
Looks like he just discovered and renamed the basic idea that "all models are incomplete". Any scientist could tell you that one! I remember a quote that goes something like this: The greatest scientific accomplishment of the 19th century was the discovery that everything could be described by equations. The greatest scientific accomplishment of the 20th century is that nothing can be described by equations.
That's all an abstraction is: a model. Just like Newtonian physics, supply and demand under perfect competition, and every other hard or soft scientific model. Supply and demand breaks down at the low end (you can't be a market participant if you haven't eaten in a month) and the high end (if you are very wealthy, you can change the very rules of the game). Actually, supply and demand breaks down in many ways, all the time. Physics breaks down at the very large or very small scales. Planetary orbits have wobbles that can only be explained by more complex theories. Etc.
No one should pretend that the models are complete. Or even pretend that complete models are possible. However, the models help you understand. They help you find better solutions (patterns) to problems. They help you discuss and comprehend and write about a problem. They allow you to focus on invariants (and even invariants break down).
All models are imperfect. It's good that computer science folks can understand this, however, I don't think Joel should use a term like "leaky abstraction". Calling it that implies the existence of "unleaky abstraction", which is impossible. These are all just "abstractions" and the leaks are unavoidable.
Example: if I unplug the computer and drop it out of a window, the software will fail. That's a leak, isn't it? Think of how you would address that in your model: maybe another computer watches this one so it can take over if it dies..etc..more complexity, more abstractions, more leaks....
He also points out that, basically, computer science isn't exempt from the complexity, specialization, and growing body of understanding that accompanies every scientific field. Yeah, these days you have to know quite a bit of stuff about every part of a computer system in order to write truly reliable programs and understand what they are doing. And it will only get more complex as time goes on.
But what else can we do, go back to the Apple II? (actually that's not a bad idea. That was the most reliable machine I've ever owned!)
You don't, and in fact can't, deal with page faults in your Java program. Nonetheless, your java program will suffer a performance hit when it page faults. Thats a leaky abstraction.
While you are slamming ``liberal arts'' -- a term you seem not to understand -- you highlight the need for it. Liberal arts does not imply a non-scientific, non-technological education. It implies a broad education, including science, mathematics, and engineering along with the ``traditional'' topics of history, literature, languages, politics, economics, and arts. For politics, governance, and management, I want people who are conversant in all of those topics.
Unfortunately, the subjects you list have all grown to the point that no human can obtain even a BASIC understanding of all of them before he's too old to have a useful carreer left.
It was once possible to be a "Rennisance Man" - a master of ALL the sciences and arts reduced to teachability. No more. It's just too bloody large. (I say this as someone who attended a univerdity that claims to try to produce such people - centuries after the last of them is dead. B-) )
Unfortunately, "Liberal Arts" schools have, over much of the last century, been filled with the mathematically and technically illiterate - both because the students without the necessary skills gravitated there, and because the faculties themselves were so disabled, and in turn disparaged the skills they were incompetent to teach.
The engineering/scientific/biologic/technical cirriculum had constant feedback from the real world about what was true and what was false. But the "Arts Schools" taught classes where what was "right" was ONLY a matter of opinion - and grades solely a measure of how well you could regurgitate your Prof's pet bonnet-bees. (This DESPITE the fact that SOME of these theories could be TESTED - if only the academics understood, and/or believed in, things like the scientific method, statistics, and sampling methods.)
Yes the "Social 'Sciences'" are hard. But the bulk of their credentialed practitioners used this as an excuse to drop "science" from their methodologies. (This despite that fact that mathematics departments were generally part of the art, rather than the engineering, side of the school organization.)
I've been out of academia for a while now. I can hope that things have improved, as you seem to claim. But I have not personally seen any sign of such from the outside (other than your claim).
In my school days, too, many students on the Arts side of the wall knew tech, math, and the like. (Students are generally young, and still hunting for their muse.) But they would generally transfer out to some field more conducive to clear thought, drop out to use it in the real world, or (if they stayed in LS&A) suppress it or flunk out.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
No one would cut a lawn with scissors
:)
You'd be surprised what people will cut lawns with. In Brasilia (Capital of Brasil) the standard method of trimming lawns is to use a machete. No, I'm not talking about hacking down waist-high grass, I'm talking about trimming 3-inch high grass down to two inches by hacking repeatedly at it with a machete, trying to swing parallel to the ground as best you can. No, you don't do this yourself, you hire someone to do it. And if you're a salaried groundskeeper, it makes sure that you always have something to do - you woldn't want to be found slacking off during the day. On rare occasions I've seen people using hedge trimmers (aka big scissors) instead. My family was the only one I knew about in our neighborhood that even owned an American-style lawn mower. My parents were too cheap to hire a full-time groundskeeper, and I have lots of brothers and sisters who work for free
Moral of the story; if it works and fits the requirements better, someone will do it.
"Space Exploration is not endless circles in low earth orbit." -Buzz Aldrin
many high level abstractions simply do not exist in assembly language.
Consider the following assembly language code:
Okay, so this is a little snippet of some assembly language I've just recently worked on. Here's the declaration for the input file:
That's it. Is this readable? Is it abstracted at a level high enough? The primary difference between assembly and a HLL is that in assembly one must invent their own logical abstractions for a real world problem, where languages such as C/C++ simply provide them.
You've probably noticed that I'm using a lot of macros. In fact, classes, polymorphism, inheritance, and virtual functions are all easily implemented with macros. I'm using NASM right now (though I'm using my own macro processor), and it works very well. Because I understand both the high-level concepts and low level details, I can code rather high-level abstractions in a relatively low level language such as assembler. I get the best of both worlds: the ease of HLL abstraction with the power of low level coding.
Please tell me what you think of this - I would honestly like to know. For the past few years, I've been working on macro sets and libraries that make coding in assembly seem more like a HLL. I've also set rules for function calls, like a function must preserve all registers, except those which are used to pass parms. With a well developed library of classes and routines, I've found that I can develop applications quickly and painlessly. Because I stick to coding standards, I'm able to reuse quite a bit (> 50%) of my assembly code.
You might be tempted to ask, "Why not just write in a HLL then?" I do. In fact, I prefer to write in C++. But when the need arises, it's nice to be able to apply the same abstractions of a HLL in assembly. It just so happens that the need has arisen - I'm working on a project that will last a few weeks, and my boss doesn't consider it fiscally responsible to buy a $1200 compiler that will be used for such a short time.
Interestingly, the use of assembly has made me a better programmer. Assembly forces one to think about what one is doing before coding the solution, which usually results in better code. Assembly forces me to come up with new abstractions and solutions that fit the problem, rather than fitting the problem into any given HLL's logical paradigm. Once I prove that the abstract algorithm will indeed solve the problem, I'm then free to convert the algorithm into assembly. Notice that this is the opposite of the way most HLL coders go about writing code - they find a way in which to squeeze a real world problem into the paradigm of the language used. Which leaves them at a loss when "leaky abstractions" occur. Assembly has the flexibility to adapt to the solution best suited to a problem, where as HLL's, while very good at solving the particular problem for which they were designed, perform very poorly for solving problems outside of their logical paradigms. While assembly is easily surpassed by C/C++, Java, or VB for many problems, there are simply some problems that cannot be solved without it. But even if one never uses assembly professionally, learning it forces one to learn to develop logical abstractions on their own - which in turn, increases their general problem solving ability, regardless of the language in which they write.
I see the key difference between a good assembly coder and a HLL coder is that an assembly language coder must invent high level abstractions, where as the HLL coder simply learns and uses them. So assembly is a bit more mental work.
The society for a thought-free internet welcomes you.
It's useful to distinguish between performance-related leaks and correctness leaks. SQL offers an abstraction for which the underlying database layout is irrelevant except for performance issues. The performance issues may be major, but at least you don't have to worry about correctness.
C++ is notorious for this; the language adds abstractions with "gotchas" inside. If you try to get the C++ standards committee to clean things up, you always hear 1) that would break some legacy code somewhere, even if we can't find any examples of such code anywhere in any open source distro or Microsoft distro, or 2) that only bothers people who arent "l33t".
Hardware people used to insist that everything you needed to know to use a part had to be on the datasheet. This is less true today, because hardware designers are so constrained on power, space, heat, and cost all at once.
The abstraction is a reliable byte stream, which of course isn't really possible due to phenomena that can only be affected by interfaces beneath TCP. A leak that's documented is still a leak.
... machine code itself is an abstraction in the first place. This is especially true for modern processors that reorder instructions, execute them in parallel, and in extreme cases convert them into an entirely different instruction set.