Slashdot Mirror


Programmers and the "Big Picture"?

FirmWarez asks: "I'm an embedded systems engineer. I've designed and programmed industrial, medical, consumer, and aerospace gear. I was engineering manager at a contract design house for a while. The recent thread regarding the probable encryption box of the Columbia brought to mind a long standing question. Do Slashdot readers think that the theories used to teach (and learn) programming lead to programmers that tend to approach problems with a 'black box', or 'virtual machine' mentality without considering the entire system? That, in and of itself, would explain a lot of security issues, as well as things as simple as user interface nightmares. Comments?"

"Back working on my undergrad (computer engineering) I remember getting frustrated at the comp-sci profs that insisted machines were simply 'black boxes' and the underlying hardware need not be a concern of the programmer.

Of course in embedded systems that's not the case. When developing code for a medical device, you've got to understand how the hardware responds to a software crash, etc. A number of Slashdot readers dogmatically responded with "security through obscurity" quotes about the shuttle's missing secret box. While that may have some validity, it does not respect the needs of the entire system, in this case the difficulty of maintaining keys and equipment across a huge network of military equipment, personnel, installations."

96 of 390 comments (clear)

  1. In general... yes by Anonymous Coward · · Score: 4, Interesting

    I don't have as much experience as some, but I've always wondered about coders who restrain themselves in the 'world' their code runs in. It overlaps, I think, with the problems of sysadmins who leave systems/gateways/firewalls and whatnot wide open to the world.

    If a coder isn't ignoring the fact their code isn't going to be running on the exact same shell as they are, they're ignoring that it won't always be running in the exact same OS, or exact same network. Tragically, when it breaks it can then break BIG.

    Note I also don't have enough experience to offer a solution other than "get a clue!". It's more work until you embed it in your habits to take notice of these possibilities.

    1. Re:In general... yes by Goth+Biker+Babe · · Score: 2, Insightful

      I'm in software engineering in the real world as essentially a software architect for embedded systems and yes hearing about CS courses where the lecturers tell students not to worry about the hardware "If your software won't run on the specified system then you need to get the hardware engineers to give you more memory, or more MIPs, or whatever" does make me cringe. This is just not practical when you are building a product to a budget in the embedded world. Get yourself the schematics and learn the system you are going to code for but... ...coding too directly to the hardware is also bad. A company which has, for say customer reasons, use different processors and operating systems in it's products will end up with lots of teams which are all reinventing the wheel for their particular hardware. That doesn't bode well for reuse and potential requires rewrites even for cost down versions of products.

      Understanding the hardware doesn't prevent you from using black box methods. Just use heirachical ones. The system as a box, subsystems as boxes, their objects as boxes and so on. So if we take a DVD player (European market one) for example without knowing the hardware you know it's going to have the following subsystems: storage (dvd mechanism say), user i/o (front panel, remote and display), a/v routing (either video from the box or passthrough, analogue audio or digital audio - possibly optical), memory and decode (codecs etc). For storage you could split that into device driver (low and high level), io stream etc. All of these have software components that can be reused even as basic as the on screen widget set in the OSD.

      If you do use a black box/component approach then you can never have too much documentation. Components should be considered projects in their own rights. Do your requirements analysis and find out what it's needed to do, model it, document it formally, explain how it's supposed to be used. If there are areas of ambiguity then it hasn't been documented properly.

      The art of desiging the system is knowing when to reuse and when to reinvent. When to componentize and when you shouldn't. I don't see black box engineering and knowing the system as mutually exclusive as an engineer you must consider how much and at what level you need to know.

  2. Probably by nizcolas · · Score: 4, Insightful

    Most programmers who are going to come across a "black box" have enough experience to be able code for the situation. Isn't that skill a trait of a good programmer?

    Then again maybe Im missing the point :)

    --
    If you get an error, type "OVERRIDE" or "SECURITY OVERRIDE" and then try the optimize command again.
    1. Re:Probably by ackthpt · · Score: 5, Interesting
      Most programmers who are going to come across a "black box" have enough experience to be able code for the situation. Isn't that skill a trait of a good programmer?

      I think it's more than a skill, it's an attitude. I've encountered a number of programmers (just out of school/training) who are oblivious to external concerns, including interface design (traditionally what users complain most about and programmers lack any standard to follow.) Generally it takes little effort to break programs written by very skilled programmers, but blind to anything outside their scope. I was probably as bad when I first started, but recently an analyst complained angrily why I went beyond the scope of the project by including an error/warning log (most likely because the errors/warnings accounted for any untrapped logic and revealed how incomplete the spec was and how little the analyst, and some of the higher-ups, knew of the business function) I felt there were too many things unaccounted for and added the log, when it produced 1,000+ entries things got a little heated. I stuck to my guns though and see a general lack of interest in review of why there are gaps in the spec or knowledge (by the very people who should know.

      --

      A feeling of having made the same mistake before: Deja Foobar
    2. Re:Probably by ryochiji · · Score: 5, Interesting
      >programmers that tend to approach problems with a 'black box', or 'virtual machine' mentality without considering the entire system?

      I think there's a lot of truth in this. For example, how many programmers think about writing software from the standpoint of a support technician? In fact, how many programmers even have experience as a support technician? I've never even heard anyone even talk about writing supportable software, yet, when considering the overall costs or quality of a system, I think it's important to consider how heavily the introduction of that system will tax the support department. Whether you're shipping or deploying the system, lower support needs will lower over all costs and vastly improve the reputation of the system.

      The same applies for security and usability. It's really not a question of programming/technical ability, but a question of mentality. I think programmers need to have a specific (or perhaps not-so-specific) mindset to get a bigger picture, and not very many programmers are willing to do that. Part of it may be inherent to programmer-types, but it also might be cultural (the whole "us vs. them" elitist attitude).

    3. Re:Probably by oconnorcjo · · Score: 3, Insightful
      The same applies for security and usability. It's really not a question of programming/technical ability, but a question of mentality. I think programmers need to have a specific (or perhaps not-so-specific) mindset to get a bigger picture, and not very many programmers are willing to do that. Part of it may be inherent to programmer-types, but it also might be cultural (the whole "us vs. them" elitist attitude).

      You ALMOST have it except it is not inherent in the programmer but in how programming departments are managed.

      Management usually puts an emphasis on more features and fast timelines instead of security and stability. Programmer's must prioritize the demands given to them and when management's views are skewed, so are the employees.

      Good management would have code reviews of all programmers code on a periodic basis (no matter how much experience they have) and system designers would have meetings with the programmers (including every senior to junior programmer involved in building the system) and explain why and what their system is supposed to do.

      Instead most companies give out specs and nobody knows what the hell their piers are doing either because management is incompetent or lazy and thus leave code reviews and design meetings in a dusty book that could be called "good practices that most don't do".

      One of the reasons why the code in open source software is often of a higher quality than commercial software is that: 1. programmers write their code KNOWING that somebody might be looking at it later (and often getting good suggestions back from other developers). 2. Open source projects have developer mailing lists where developers explain what/how they are designing/redesigning something new in the project.

      But most company management's are very short sighted and impatient like the rest of society.

      --
      I miss the Karma Whores.
  3. Involvement in the SDLC by seanmcelroy · · Score: 5, Insightful

    I think the problem increases as programmers are less and less a part of the complete systems development life cycle and are contracted to work on individual components of an overall system. Especially during the maintenance phases of a system's life, the inexperience of new programmers on a project is probably more to blame than 'training' per say to think in a black-box mentality.

    --
    Be very, very careful what you put into that head, because you will never, ever get it out. -Thomas Cardinal Wolsey
  4. Huh? by twofidyKidd · · Score: 5, Funny

    I don't know what you're trying to say here man, but no amount of programming or "Fatal Error: Wing no longer attached to craft" terminal prompts would've saved them from what happened.

    If you're trying to make a case for programming paradigm shifts based on security procedures, it isn't working in this context.

    --


    Hades, PoD: Official Advocate
  5. Black Box....yes, but....... by jlk_71 · · Score: 5, Insightful

    I am taking courses toward my degree and I must say that in my intro to programming course, the instructor was constantly stressing the need for 'black box' programming. In addition though, he also stressed that while keeping things black box, you also need to keep your mind on the whole project, always watching out for possibly security problems, etc.
    I believe that some people tend to get tunnel vision and concentrate wholely on the bb theory, without taking into consideration the whole program. This does usually lead to problems and errors in the code.

    Regards,

    jlk

    1. Re:Black Box....yes, but....... by plumby · · Score: 4, Insightful
      There's nothing wrong with developing to a black box model. This is what design by contract and component development are all about. Each method on each component should describe, with pre- and post-conditions, what that method requires in order to work, and what changes it will make to the external environment that it is operating in. Beyond that, the inner workings of the component should be a black box. I don't care how your component does what it says it will do, just that it does exactly that (and nothing else).

      As the developer of that component, you will know exactly what the internals do, but then you treat the rest of the world as a black box, to be talked to through clearly defined interfaces.

      It is the lack of a black box approach that often leads to unexpected side-effects.

  6. Sometimes It's Impossible by syntap · · Score: 5, Insightful

    Many times, management is the cause of preventing developers to see the "big picture". Sometimes it's "Here, code this" and you don't get a lot of opportunity to ask the questions you know need to be asked. Sometimes you have to hope resolutions to these types of issues are built into the requirements specification or will be ironed out in quality assurance measures.

    The developer is only one in a group of responsible parties in any given system, and his/her output depends largely in input from others. If a developer is kept "out of the loop" on things (or is lazy and stays out of the loop opn purpose), you're going to see these problems.

    Often it's like blaming clogged fuel injectors _for_ cheap gasoline instead of _on_ it.

    1. Re:Sometimes It's Impossible by Anonvmous+Coward · · Score: 4, Insightful

      "Many times, management is the cause of preventing developers to see the "big picture". Sometimes it's "Here, code this" and you don't get a lot of opportunity to ask the questions you know need to be asked..."

      Don't forget the "make it work by the next trade show" mentality.

    2. Re:Sometimes It's Impossible by aziegler · · Score: 2, Informative

      It's not just management. I'm on a contract right now where I've advised that the client do things in particular ways, and the management itself is reasonably amenable to the concept, but the programmer to whom I report is, well, set in her ways.

      Developers should know what the big picture is so that they can have a sense of direction in the development process. They don't need to worry abuot it, perhaps, but they should know what it is nonetheless.

      -austin

      --
      Ni bhionn an rath achx mar a mbionn an smacht (There is no Luck without Discipline)
    3. Re:Sometimes It's Impossible by wwwdsummers · · Score: 2, Funny

      Or even worse, "Uhm, hi. We're at a trade show right now and killer 'feature' Q isn't working in our live demo. Fix. Now."

    4. Re:Sometimes It's Impossible by LadyLucky · · Score: 2, Insightful

      Anyone else here got massively overworked and stressed while working towards HIMSS? Geeeeeeeeeeeeez. So glad the work for that is over.

      --
      dominionrd.blogspot.com - Restaurants on
  7. The "Big Picture" is TOO big for most people by Entrope · · Score: 5, Insightful

    Keeping the "big picture" in mind is a good thing for managers and designers. For people implementing the finer details, though, it can be a distraction and a poor use of their time. Someone implementing or verifying flow control in a ground-to-space network link does not need to know much about the format of data carried over the link. Someone doing circuit layout or design for a cockpit control widget does not need to worry about reentry dynamics and airflow. Similar examples can be found in any large system design.

    It is the responsibility of the higher level designers and managers to encapsulate the big picture into digestible, approachable chunks. To the extent possible, they should be aware of and codify the assumptions and requirements that cross multiple domains -- when those are documented, it is easier to test the system for correctness or robustness, as well as to diagnose problems.

    When everyone on the project tries to orient their work to what they each perceive as the big picture, you end up with enough different perceptions that people work against each other. Breaking down the system into smaller, more defined, chunks combats that tendency.

    1. Re:The "Big Picture" is TOO big for most people by MCZapf · · Score: 2, Interesting
      Yes, not everyone is a "big picture" person. I myself am a big picture person, and I've found that it has hampered my ability to be productive at a lower level, such as coding, because I keep worrying about larger issues in the development.

      On the other hand, if you aren't aware of at least some of the big picture, you may end up doing things with consequences that you didn't anticipate.

    2. Re:The "Big Picture" is TOO big for most people by Anonvmous+Coward · · Score: 2, Insightful

      "Someone doing circuit layout or design for a cockpit control widget does not need to worry about reentry dynamics and airflow."

      I think this example is debatable and can possibily be used against you. One could argue that reentry dynamics and airflow could make for a bumpy ride, thus the designers need to be aware of the journey this vessel's going to go on.

      That's besides the point, though. I'm not interested in debating that detail. Instead, I want to offer my insight from observing both poles of this dicussion: having strictly one point of view or the other is bad. If you're overly broad, you over-design software. If you're overly narrow, you design yourself into a corner.

      I'm radically oversimplifying this problem, but it's true. Everybody has their own perspective. A good manager places them where they're useful. My company has a nice mixture of personality types in engineering. They're all placed where they fit best. If we were to polarize all the sudden, I really think the project would collapse.

    3. Re:The "Big Picture" is TOO big for most people by pmz · · Score: 4, Informative

      When everyone on the project tries to orient their work to what they each perceive as the big picture, you end up with enough different perceptions that people work against each other. Breaking down the system into smaller, more defined, chunks combats that tendency.

      This is why good managers are worth their weight in gold. Bad managers are worse than worthless.

    4. Re:The "Big Picture" is TOO big for most people by Entrope · · Score: 4, Insightful

      One could argue that reentry dynamics and airflow could make for a bumpy ride, thus the designers need to be aware of the journey this vessel's going to go on.

      That actually occurred to me while I was writing my post, and I considered it to be an instance where my second paragraph bears true: if the ride will be bumpy, or flown upside down, or whatever, then those cases should be documented (or at least known) to the designers of the cockpit widgets.

      Yes, you need to avoid both over- and under-design. Yes, you need to know things beyond your piece of the work. But no, you do not need to consider the whole system and all parts of it when you do implementation or even some of the design.

      A good designer knows how far away the interaction horizon should be, and can analyze the effects of everything within that horizon. If the collective effects are too many to analyze, it is a sign that the design needs to be refined or reworked.

    5. Re:The "Big Picture" is TOO big for most people by GileadGreene · · Score: 4, Insightful
      This is why good managers are worth their weight in gold. Bad managers are worse than worthless.

      No. This is why good systems engineers are worth their weight in gold. Dealing with the big picture, and designing large, complex systems using an engineering approach is why systems engineering came into being in the first place.

      Managers are trained to deal with schedule and budget. Not with designing complex systems. Systems engineers are trained to design complex systems, and to make sure that all the pieces interact in such a way that the overall system acheives whatever goal it was designed for.

      That said, decent systems engineers seem to be somewhat rare these days, or at least they seem to get overruled by management. Many of the well-known engineering blunders in recent years can be chalked up to poor systems engineering.

    6. Re:The "Big Picture" is TOO big for most people by pmz · · Score: 2, Insightful

      Managers are trained to deal with schedule and budget. Not with designing complex systems.

      I agree with what you said; however, schedule/budget managers cannot be ignorant of what good systems engineering requires. Also, the lead engineer on a project is a kind of manager. It isn't uncommon on in a small company or project for the manager and the lead engineer to be the same person, which I guess makes their job even harder.

  8. IMHO by Em+Emalb · · Score: 5, Funny

    People tend to focus exclusively on their area of expertise.

    Otherwise they become managers :D

    --
    Sent from your iPad.
    1. Re:IMHO by hendridm · · Score: 5, Funny

      > Otherwise they become managers :D

      At my last job you were allowed to make 3 major mistakes, then they made you a manager.

    2. Re:IMHO by GileadGreene · · Score: 2, Insightful
      People tend to focus exclusively on their area of expertise.
      Otherwise they become managers :D

      Or systems engineers...

  9. What does that have to do with UI design? by 192939495969798999 · · Score: 2, Insightful

    If i write a component that takes in X1 and outputs X2, isn't it the designer's job to make it look pretty? I mean, supposedly they were the ones that came up with needing the component in the first place, to accomplish some function or other, and thus make the user happy.

    --
    stuff |
  10. Abstraction is necessary! by Dr.+Manhattan · · Score: 5, Insightful
    Being able to abstract chunks of a program or system out and not worry about implementation is utterly vital. No human, however gifted, is capable of understanding the entirety of more than a trivial system at once.

    Now, the amount of abstraction possible does differ depending on what you're doing. Embedded systems programming is hard, and you do have to know details of the machine. But I ask you - do you insist on a gate-level understanding of the embedded CPU, or will you settle for knowing the opcodes and their timing characteristics?

    Because, in embedded programming, you need to know more about the device, it's proportionately harder to do. That's one reason, apart from power and cost considerations, that embedded systems tend to be simple - the simpler the system, the easier it is to think about, to prove correctness or to at least enumerate possible pathways and handle them.

    But even in that case, you need to be able to ignore some implementation issues or you can't do it at all.

    --
    PHEM - party like it's 1997-2003!
    1. Re:Abstraction is necessary! by dboyles · · Score: 4, Insightful

      I agree, and would like to add my thoughts.

      One of the most likeable things about programming is that on a low enough level, it's always predictable. This kind of goes hand-in-hand with the fact that computers don't make mistakes, humans do. As a programmer, it's very comforting (for lack of a better word) to have a chunk of code and know that, given X input, you'll get Y output. You can write a subroutine, document it well, and come back to it later, knowing how it will behave. Of course, other programmers can do the same with your code, without having to have intricate knowledge of how the code goes about returning the output.

      But of course, there's a catch. It's probable that the programmer who wrote the subroutine initially didn't envision some special case, and therefore didn't write the code to handle it. If everybody is lucky, the program will hiccup and the second programmer will see the problem. The worse situation is when the error is seemingly minor, and goes unnoticed: when that floating point number gets converted to an integer and nobody notices.

      I know this isn't some groundbreaking new look on abstraction in code, but it is pretty interesting to think about.

      --
      -- "Complacency is a far more dangerous attitude than outrage." -Naomi Littlebear
    2. Re:Abstraction is necessary! by Entrope · · Score: 2, Insightful

      That's not entirely true. Right brained people can, thing is..


      Being right-brained does not magically grant someone the ability to keep ten thousand things in their head at once. At best, it allows one the ability to easily defocus on things that are not important for what they do want to focus on -- in other words, to think abstractly. :)


      The hard part of being a really good programmer is not learning how to do job X, Y or Z. The hard part is learning where to draw the interfaces, where you should use existing abstractions, and where you need to extend them or create new abstractions. This is true at most layers of design -- like Dr. Manhattan said, you can find abstraction layers and idealized interfaces at many different places in a complex (especially a computerized) system.

    3. Re:Abstraction is necessary! by Dr.+Manhattan · · Score: 2, Insightful
      Uhh, and John Carmack does not understand every ounce of the quake code. And Linus torvalds does not understand every ounce of the Linux core.

      No, they don't. Those codebases are split into well-defined modules, and they are able to understand how those modules fit together. And they can look inside one of those modules and know how it's put together. That's why you have a core engine that can have software, Glide, and OpenGL renderers; or a filesystem core that can work with ext2, ext3, reiserFS, XFS, etc.

      But neither of even these prodigiously talented gentlemen can visualize the entire state of their respective systems. Else why would you have, e.g., the Quake physics bugs or any number of kernel bugs? Their horizons may be quite a bit broader than average but they are still limited.

      --
      PHEM - party like it's 1997-2003!
    4. Re:Abstraction is necessary! by ltkije · · Score: 2, Insightful
      Now, the amount of abstraction possible does differ depending on what you're doing. Embedded systems programming is hard, and you do have to know details of the machine. But I ask you - do you insist on a gate-level understanding of the embedded CPU, or will you settle for knowing the opcodes and their timing characteristics?

      I work on embedded products. A typical design has about 30,000 lines of C code, but the amount of assembly language is 1% of that and dropping. So there is little need for most of the programmers on a team to know something as concrete as assembly.

      Our code runs on 16-bit single-chip microcontrollers rated at about 5 MIPS. The chips are typical of what $5 buys today. The application, which Slashdot readers would recognize instantly, has soft real-time requirements. We could probably run most everything on 20 ms timer ticks and get good responsiveness. There are plenty of spare CPU cycles even at peak loading. Yet there are people in my company who want to read the assembly code generated by our ANSI-standard C compiler, and turn off all the compiler optimizations. Some still insist on writing their own memset() functions.

      Contrast this with the fact that it takes us 18 months to develop each new product. The 2003 version is about 80% the same as the 2001 version, about 20% of the code handles product differences from the older version, and there's maybe 5% new code for new features. What's wrong with this picture?

      One obvious answer is simple: we're probably using the wrong level of abstraction, or just the wrong abstractions, in our design. We'd do much better to:

      Abstract away hardware specifics wherever possible.

      Trade off a little performance for shorter project schedules.

      Profit!

      This is not to say we should never open the black box -- just that we should be smart about when to dig deeply into the underlying hardware and CPU cycles. And being able to debug with an oscilliscope is still sometimes a valuable talent. As others have said here, the art of engineering lies in knowing when to do these things.

      Successful black-box design can produce amazing results. For instance, look at Pure-Systems, whose initial product generates an optimized embedded kernel, written in C++, that's small enough to run on an Atmel AVR chip.

  11. Not the theories themselves.... by keyslammer · · Score: 5, Insightful

    ... but the lack of experience.

    Programmers have to consider subsystems as abstractions: there's a limit to how many things the brain can deal with at one time. We know that this kind of thinking produces cleaner designs which are less susceptible to bugs and security holes.

    Knowing the limitations of the "black box" and what will break the abstraction is the product of lots and lots of experience. I don't believe there's any way to teach this - it's something that you just have to live through.

    That's why senior level developers can continue to justify their existence (and higher price tags)!

  12. When I was programming the temperature control ... by burgburgburg · · Score: 3, Funny

    on the new Death Star, I found that trying to envision the "Big Picture" interfered with the specific requirements of my task. I needed control mechanisms smart enough to deal with Storm Trooper suits, regular Empire uniforms, robots with various temperature ranges, Wookies. It needed to be able to maintain a comfortable temperature range in the beam tunnel vicinity even during firing. And it needed to be efficient enough that they wouldn't shift power from the exhaust port shields to the jacuzzi heaters like they did on the old Death Star.

  13. Experience by wackysootroom · · Score: 4, Insightful

    The only thing that school prepares you for is to get an entry level job where you can gain the experience to write reliable software.

    School will get you up to speed on new terms and concepts, but the only thing that will make you better at writing good code is to read good code, write your own code and compare it to the good code, notice the difference, and adjust your approach until your code is "good".

    1. Re:Experience by Anonymous Coward · · Score: 2, Insightful

      So then how does better code get developed if everyone is merely attempting to match their coding style to the "good" style they see?

      *Someone* had to provide that good code in the first place.

    2. Re:Experience by Oink.NET · · Score: 5, Insightful
      the only thing that will make you better at writing good code is to read good code

      Because code is the most direct way to communicate wisdom between geeks? I would submit that unless you get the analysis and design right, your approach to writing good code just teaches you how to make a more polished turd.

      As far as getting better at the mechanics of coding, I would suggest reading Steve McConnell's Code Complete.

    3. Re:Experience by Surreal_Streaker · · Score: 2, Funny
      The only thing that school prepares you for is to get an entry level job where you can gain the experience to write reliable software.

      -1 Pandering.

    4. Re:Experience by pmz · · Score: 2, Interesting

      Because code is the most direct way to communicate wisdom between geeks?

      Not really. I've gained much more by reading books like The Mythical Man Month and good object-oriented analysis and data modeling books. Managing complexity through good data modeling is the most important (and hardest) part of a program to get right.

      The worst applications I've had to work with were designed piece-meal by a high-turnover team of inexperience people (read: really ugly data that resulted in nasty bloated unmaintainable code).

  14. From a programming point of view, it depends. by Dthoma · · Score: 3, Interesting

    Programming classes may encourage a 'black box' approach to programming, depending on what language you use. The reason for this is that it all relies on how high-level the language is; if you're using PHP then chances are you won't be worrying nearly as much about the hardware of your system than if you're using C, assembler, or machine code.

    --

    Note to M1-ers: a curt but otherwise insightful message is not "Flamebait" or "Troll".

  15. We don't need anymore black boxes by jj_johny · · Score: 4, Interesting
    I think that the programmer who thinks of things in a black box mentality is usually going to be involved in failed program. I have run into so many programmers who know nothing of the many parts that their program touches. They seem to believe that their software does not work within a wider system and a wider world.

    The problem with these programmers is that they rarely understand what can and does go wrong with the outside world. It is always amazing to me that there are people out there that assume that everyone has a 100BaseT Ethernet hub between the front end and the back end or other stupid assumptions.

    The issue that crops up most when programmers think in black box terms is that today's software is not spec'd out enough so that the end user does not get what they wanted but the programmer did not solve it by asking. Too often the problem is very fuzzy and thus the programmer is there to help clarify not just implement.

    Without a well rounded programmer looking at the overall system (or his/her boss), you will wind up with chatty, buggy applications that was what the user asked for but not what they needed.

    1. Re:We don't need anymore black boxes by sconeu · · Score: 2, Interesting

      It is always amazing to me that there are people out there that assume that everyone has a 100BaseT Ethernet hub between the front end and the back end or other stupid assumptions.

      Actually, I've found myself doing the reverse several times. For many years, I worked in what I guess would be semi-embedded systems. We did special purpose computers for the military. The thing was, we had our own RTE, and I got into the habit of coding for that, and assuming that the target environment essentially had nothing.

      Now that I'm coding for standard OSen (Linux), I find it hard to get used to the concept that it's already there, I don't have to roll my own.

      Don't get me wrong, I believe in reuse. I think it's wonderful to have the tools. It's just difficult to "rewire" myself after 15 years of a particular mindset (but I'm working at it!).

      --
      General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
    2. Re:We don't need anymore black boxes by (trb001) · · Score: 2, Interesting

      Without a well rounded programmer looking at the overall system (or his/her boss)

      This is what the lead programmer/designer or the PM is for. Depending on the project, you should have 1 of these for each section of the project. If the project is sufficiently large, have 1 LP for each sub-section and have them report to a primary LP. The primary should know how all the subs interface and the subs should know how every component under them interfaces together. There are few projects that I've ever seen that required more than a few LPs (and I've worked on projects with 250+ developers) because they worked in a multi-tiered environment...each LP knew their own section.

      Individual programmers need to look at things as a black box, it will make them much more efficient. Granted, you need to have sufficient requirements for them to do this effectively (very, very few projects have these). Ideally, programmers shouldn't even be hired until at least the first draft of requirements are out. The LP should be God before that, dictating what's doable, feasible and what makes sense from a technical perspective. He then needs to hire on programmers that can produce what he's envisioned. It amazes me the number of programs that staff up before requirements are out...how can you effectively staff up if you don't know what your staff is going to be doing? That's why sub-contractors are important, they can be staffed up anytime and (should!) have domain knowledge on what you're doing.

      --trb

    3. Re:We don't need anymore black boxes by Arandir · · Score: 2, Insightful

      I'm programming on a very complex system. I simply cannot know about all the parts that my code touches. I would need three engineering degrees just to understand it all. I have to program in a black box because the white box is too big.

      This is why I demand complete requirements and specifications, and invite all relevant parties to my design and code reviews. And when I'm the one called on to write the reqs and specs, I make sure I get a sign off absolving me of responsibility for it (you would be surprised how much closer the docs are inspected when you start demanding stuff like that).

      The problem isn't the developers writing black boxes, but the upper management buying into the party line of Microsoft, thinking that snap-together black box components will reduce the resource needs for the project.

      --
      A Government Is a Body of People, Usually Notably Ungoverned
  16. That's the difference between CS and CE. by Anonymous Coward · · Score: 4, Insightful

    Computer science professors and courses are more concerned with the methods, ideas, and logic of computer programming and design. The idea is to create a totally abstract system, hardware or software, that can then be implemented on any system. This is the purpose of "black box" programming.

    While I agree with you that programmers should understand the hardware they are writing for, any knowledge of that hardware is biasing their creation of a system to run on that hardware and further removing itself from computer science's notion of total abstraction.

  17. UI issues: left hand/right hand by Illserve · · Score: 5, Insightful

    I recently installed the recent version of the accursed RealOne player to watch an rm file. I hate Real player more than can be described by words and it just seems to be getting worse.

    So I pop it up to view the file, and what happens? I get the movie playing in a window on top of the Real advertising/browser thing. It spontaneously pops up a "help" balloon giving me a tip for how to use the browser window. The balloon is sitting RIGHT ON TOP OF THE GODDAMNED MOVIE IMAGE. It goes away after a few seconds of frantic clicking, but the point is clear, these programs are often a monstrous brew, created by too many chefs. They just throw in features, and there doesn't seem to be someone sitting at the top, deciding whether these features actually contribute to improving the final product, or just make it worse.

    Then there's Office, which, by default will turn 2-21-95 into 2/21/95. ????? I have to dig through numerous help pages to figure out what subpanel of the preferences menu will deactivate this. Worse, I enter 23 hello on a new line in Word, and hit enter, it auto indents, adds a 24 and positions the cursor after. !?!?!!?!?!?!?!!?
    How many times I've had to fight this particular feature I can't tell you.

    And it's certainly not just a closed source thing either, if anything, some open source GUI packages are even worse. Although, to be fair, I don't expect as much from open source stuff, because noone's getting paid. But when a program created by paid programmers is just badly done, I get infuriated at the incompetence, at the hours wasted taking a usable product and making it actually worse by throwing in garbage features.

    It's been said a million times, but if we made cars the way we make software, noone would get anywhere.

  18. Re:Probably (levels) by $$$$$exyGal · · Score: 3, Interesting
    Programming Levels:
    1. Microsoft Frontpage
    2. Raw HTML
    3. CGI/PHP/etc.
    4. Servlets/Mod-perl/etc.
    5. Object-Oriented black boxes
    6. Documented API's
    7. Public Documented API's
    8. Performance
    9. "The Big Picture" - Architects

    --sex

    --
    Very popular slashdot journal for adul
  19. Of course by Scarblac · · Score: 5, Insightful

    It is essential that every programmer in a big system only thinks about his own problem, and uses the other parts as a black box.

    Say I want to use some library. Then it has a documented API, which explains how I can use it. I don't need to know more. For me as a programmer, that means:

    • Simplicity - it is a limit on what I need to understand.
    • Compatibility - if a new version comes out, which changes implementation details but leaves the API intact, programs that don't make assumptions about these details won't break.
    • Portability - if there is a new implementation of the same API by another vendor, I can (theoretically) just change to that implementation and nothing changes.

    I'm certain that without these black boxes, no big software engineering project would be possible. The human mind can't keep track of everything in a whole system at once (except for some simple cases - like embedded systems, perhaps).

    It is done sometimes - I believe perl looks inside a file struct when reading/writing files on some platforms to get faster I/O than standard C, for example. But that's only as an optimization after coding the general case, and even then I don't believe it's a good idea.

    For hardware, the story is much the same. Any speedups specific for the hardware are optimizations, and they should only be looked at when the program works, after profiling, when there's a speed problem, and the algorithm can't be improved.

    Remember the rules of optimization: 1) Don't do it. 2) (for experts only) Don't do it yet.

    Black boxes in software engineering are your friend.

    --
    I believe posters are recognized by their sig. So I made one.
    1. Re:Of course by _xeno_ · · Score: 3, Interesting
      Block boxes can be your friend - but there are issues with them from time to time. One of the most annoying things I never knew about Java is because of the black box that is the java.lang.String class. It turns out that String.substring(...) creates a new String object that keeps a reference to the entire original sequence of characters that made up the original String object. In other words, if I had a 1024-character long string, and wanted only 5 characters from it, I would end up with a String object that presented through its black box just those 5 characters but maintained internally all 1024 characters.

      There's a way around it. Doing new String(string.substring(0,5)) allocates a new String that only contains the five required characters. But the documentation for the black box warns against doing that: "Unless an explicit copy of [the original string] is needed, use of this constructor is unnecessary since Strings are immutable."

      Well, yes, they are - but using the constructor can also be required to get around the fact that the entire array of original characters is maintained.

      As it turns out, this is a "speed hack" (ie, only one array of characters is maintained at any given time - a new array for the substring is not allocated, and the original is used). However, this implementation assumes that everyone using the black box is also going to need the parent string or is going to dereference both full string and substring (and hence allow to be gced) at about the same time, preventing memory and time from being wasted on the substring.

      Unfortunately, I had written a program that read in a list of strings, keeping certain substrings and throwing out the rest - or so I thought. (Think "comments" in the text file - they would removed, the remaining characters would be kept.) This meant I was wasting quite a bit of space by the end of the run due to characters that were no longer accessible being kept around indefinately since the "substring" objects kept a reference to the array of characters for the full string. Fixing it requires doing the new String(string) thing, which as it turns out does allocate a new buffer and is there expressly for that purpose (if you read the source code).

      My point is this - black boxes can be dangerous. A black box is a very useful abstraction - assuming that important details about implementation side effects are documented. In the given example, the Java developers implement an algorithm that is useful in many cases - but there are those cases where it would be useful not to use such an implementation.

      I think that the idea of "black boxes" are important, but that a developer also has to be aware that something happens in the black box and be prepared to learn about it if the need arises. Likewise, when creating a black box, care should be taken to either fully specify what a given implementation does and that there are no side effects to the environment (like maintaining a 1024-element array and only allowing access to 5 elements). There are pitfalls - and both users of black boxes and designers must ensure that such issues are addressed.

      (Especially because if the next String blackbox "fixes" the issue above, my code will start doing a useless extra step to get around the problems in the implementation I saw of the black box...)

      --
      You are in a maze of twisty little relative jumps, all alike.
  20. Welcome to API's and OOP by DakotaSandstone · · Score: 4, Insightful
    I'm also an embedded systems engineer. Two huge concepts I got as an undergrad in CS were "APIs" and "object oriented programming." By their very nature, these things inspire black box thinking.

    And heck, I don't know. I mean, is it great that I can now call malloc(1000000); and get a valid pointer that's just usable? Yeah probably. In DOS, I wasn't shielded from the memory manager as much, and to do something like that, I had to write my own EMS memory swapping code! That was a PITA, and kept me from the true task I was trying to solve.

    So a modern 32-bit malloc() is a black box for me. Cool. It's empowered me very nicely.

    However, something like WinSock has become a big black box for people too. Okay, great. So it's really easy, in 5 function calls, to open a socket across the internet and send data. But you've missed the nourances of security. So now your app is unsafe, because you weren't forced to know more about what's going on in the "black box."

    Well, that's all I can really say in my post. Black boxes are a darn complex issue to talk about. Anyone who attempts to distill this down to a "yes" or "no" answer is probably missing a lot of the completixy at hand in the issue.

    Very interesting question to put forth, though. Good topic.

    --
    Nothing is so smiple that it can't get screwed up.
  21. As a 3rd Year CS Student... by calebp · · Score: 2, Interesting

    I concur that there is a lot of this kind of niavete out there, but on the other hand, there are always the few that will go above and beyond. While my schooling tends to focus on a more abstract approach with emphasis on OOP, I have also started working on embedded systems in my own time.

    It seems apparent enough to me, that any passionate CS student will not be satisfied with a mystically based understanding of computer architecture; and will in turn educate themself. I propose then, that any kind of 'black-box' mentality is more a reflection of the students' drive then their education.

    --
    ________________
    "A man prepared who hesitates, is lost." -Dante The Divine Comedy: Inferno Canto XXVIII, 99
  22. Oh like in the grand scheme of things ... by nicodaemos · · Score: 5, Funny

    Okay the "big picture" college profs should be showing you is this one.

  23. Not yet, anyways by captredballs · · Score: 2, Interesting

    Is black box programming a pipe dream? I wouldn't go that far, as software engineering/compsci is a relatively new "science". At any rate, I know that I am very reliant on knowledge of the underlying platform that my code is running on. When a piece of software (especially one that I didn't write) doesn't work, I often resort to tools like truss/strace, lsof, netcat, /proc,etc... to help me determine what is going on "under the hood". I can figure out what ports, files, dlls, and logs the software is using in a matter of seconds, instead of resorting to a dubugger or printf's.

    I'm no superstar engineeer, but I find this methodology (my window into the black box) so valuable that I'm often frustrated by collegues who refuse to learn more about an OS/VM/interpreter and make use of it. It is also what most frustrates me about troubleshooting in windows.

    While it's true that I don't know much about windows, I get the feeling that these kind of observation tools that are so common on unix-ish machines aren't quite so prominently available on winderboxen. Sure, you can figure out a lot about a problem using MSDEV (what I remember from college, where VC++ wouldn't stop opening everytime netscape crashed), but it isn't available on ANY machine that I ever troubleshoot.

    Hell, even when I'm programming java I use truss to figure out what the hell is wrong with my classpath ;-)

    --

    I suppose I'm not too threatening, presently, but wait till I start Nautilus
  24. School Doesn't Prepare You For Real World Coding by Carnage4Life · · Score: 2, Insightful

    There are many things computer science education does not teach the average student about programming. This is burdened by the fact that programming can vary significantly across areas of CS (i.e. networking vs. database implementation) and even within the same area (GUI programming on Windows vs. GUI programming on Apple computers).

    When I was at GA Tech the administration prided themselves on creating students that could learn quickly about technologies thrown at them and had broad knowledge about various areas of CS. There was also more focus on learning how to program in general than specifics. This meant that there was no C++ taught in school even at the height of the language's popularity because its complexity got in the way of teaching people how to program.

    Students were especially thought to learn how to think 'abstractly' which especially with the advent of Java meant not only ignoring how hardware works but also how things like memory management work as well. In the general case, one can't be faulted for doing this while teaching students. Most of my peers at the time were getting work at dotcoms doing Java servlets or similar web/database programming so learning how things like how using linked lists vs. arrays for a data structure affects the number of page faults the system makes were not things that they would really have to concern themselves with considering how things like the virtual machine and database server would be more significantly affect their application than any code they wrote.

    Unfortunately for the few people who ended up working on embedded systems where failure is a life or death situation (such as at shops like Medtronic) this meant they sometimes would not have the background to work in those environments. However some would counter that the training they got in school would give them the aptitude to learn what they needed.

    I believe the same applies for writing secure software. Few schools teach people how to write secure code not even simple things like why not to use C functions like gets() or strcpy(). However I've seen such people become snapped into shape when exposed to secure programming practices.

  25. I think everyone is missing the point. by Anonymous Coward · · Score: 2, Insightful

    I've been doing pda programming for both the pocketpc and the palm os.

    The application for both is intended to be identical, but the api is different for each device.

    I designed the app originally for the palm, but now I am porting it over to the pocketpc. Unfortunately, the api is different enough that little of the code is portable.

    If I had known I would be coding for both, I would have tried to design the code to be more portable. Knowing the requirements of both systems might have allowed me to factor out the device-specific sections.

  26. All Systems Are Embedded by gnetwerker · · Score: 4, Interesting

    I started my career (long ago, in a galaxy far away) developing embedded systems, and much later, when running an R&D lab, came to the conclusion that, excepting (importantly) user-interface design, embedded systems were the best crucible in which to learn the right balance between modularity and holism in systems design and implementation.

    It's easy for programmers who have only worked on PCs to lose sight of the notion that programs affect the world, but when you are controlling big machines that, improperly instructed, will destroy themselves and the people around them, you begin to think twice about your coding tricks, your testing, and the interaction of your component in the system as a whole.

    But there is an underlying assumption in the question that modular design and system holism are mutually exclusive, and I don't accept that either. I also except user-interface design, which is more sociology and psychology and neurology than computer science.

    You are correct, however, in supposing that security is particularly vulnerable.

    Here's one (true) story, which I will deliberately leave unattributed: a programmer is writing code to control the dual vertical bandsaw in a sawmill -- two huge saws, each 12 inches of high-tensile stainless steel with 3-inch teeth, stretched tight between two six-foot diameter wheels and running at 10,000rpm. A log is pulled on a chain through the middle, so a cut can be made on both sides. Logs enter the system, are measured with a laser scanner, and a queued (physically and in the control program) before entering the bandsaw.

    The old fart programmers used to simply store log data in an array of sufficient size to hold the maximum number of logs that could ever be in the system, but are cognizant of the problem of "phantom logs" when a log falls off the belt or otherwise leaves the system in an uncontrolled way. The clever young programmer decides to use newly-learned techniques of memory allocation and linked-list design, and build a replacement.

    During mill installation the system is tested and appears to run well. At the end of the shift, however, as the last log is about to be run through the system, the operator discovers that there is no data in the queue for the last log, but decides to run it anyway. The computer dereferences a null pointer, grabs garbage data, and tells the bandsaw to set to an impossible position.

    Because the mill is still being installed, the stops on the bandsaw have not been adjusted, and the saws set to position "0" -- and run into the chainguide in the middle. High-stress stainless at great speed meets six inches of fixed steel, and the saw blades explode, burying foot-long shards of stainless steel sawblades up to four inches deep in the walls of the mill, destroying the operator's booth, and causing tens of thousands of dollars damage to the mill.

    Whose fault was it? The operator, for running the phantom log? The hardware installation guys, for not setting the stops on the mill? Or the programmer, for not constraining the output of his program, testing more completely, and using simpler techniques. Answer: all of the above. Better modules would have forestalled the problem, and better systems holism would have forestalled it as well. A combination would have given an even better margin of error.

    This has led me to the following conclusion: in order to get a CS degree, every programmer must write code that will lower a 10-ton machine press a maximum speed to within inches of his chest, and then stop it. We would have more careful programmers if this were the case. If they went on to write security code, we would have fewer holes.

    gnet

  27. Necessary requirements by (trb001) · · Score: 3, Insightful

    Don't laugh, but this is one of the reasons why it's important to have solid requirements BEFORE you being coding anything. Most projects don't, I know, but something as complicated as the space shuttle would need to be completely spec'd out beforehand. After proper requirements and specs are laid down, the programmer should then approach the system as if it were a black box...with a lot of restrictions.

    The idea behind black box development is that you don't need to know what the rest of the system does...your component takes input and delivers output. That's a Good Thing (tm). Requirements are what tell you how to design and implement your black box, ie, you can't have more than 1ms latency between input/output, you can't assume some system variable is going to be out there, you can't assume your process won't be interrupted. Given these sorts of requirements, your part of the system SHOULD be a black box...someone else should send you the inputs and know what kind of outputs they're getting out. Assuming you correctly followed the requirements (that's what QA testing is for), they know what they're getting.

    --trb

    1. Re:Necessary requirements by Tony · · Score: 2, Insightful

      The idea behind black box development is that you don't need to know what the rest of the system does...your component takes input and delivers output. That's a Good Thing (tm).

      This is a Good Thing (tm) only when the black boxes are true black boxes. The problem with treating software as engineering problems with Black Boxes is that there is no such thing, in software. This is the reason object oriented programming has not been the panacea we were promised.

      In construction engineering (architecture, for instance) the behavior of all the pieces are known beforehand. A steel I beam is a steel I beam (yeah, I know there are different strengths, but these are specified by the engineering firm). A steel I beam has only two APIs: welding, and riveting.

      In programming, every black box has unique characteristics. Even if two ungif "black boxes" have the exact same API, the behavior of those routines are probably different. Even worse, in all non-trivial projects, connections among Black Boxes produce a complex system; the interrelations among those Black Boxes will change the behavior of the system, often in unexptected ways.

      The case for understanding all your Black Boxes is represented in one of the most dramatic engineering failures ever: the Tacoma Narrows bridge disaster. The bridge was well-designed. However, strong winds blowing down the narrows set up harmonics in the swaying of the bridge. This was a case of an engineering firm who did not understand the whole system, with devastating results.

      The problem with OO programming is it encourages the thinking that you only need to understand your component, and not the whole system. This is a Very Bad Thing (tm) in my book. There are too many Tacoma Narrows software projects out there already.

      Anyway, that's just my opinion. I could be wrong.

      --
      Microsoft is to software what Budweiser is to beer.
  28. I bless you.... by FirstNoel · · Score: 2, Insightful

    Ah, I think we have someone here who DOES see the big picture.

    There are lots of times, at least in my experience, where it's not the programmer's fault in how the program works.

    I've seen specs come down from higher-ups who have no idea what they are asking for. I'm a little bit luckier though. The analysts we have here tend to spot these problems long before I get to program. But occasionally some do slip through. I have loads of fun ripping these things to shreds. I feel like a professor at college with my little red pen. "Ah, wrong...can't do that! What does that say? etc, etc ..."

    Aside:
    That is also usually a good stall tactic. If I'm swamped with other projects, I'll send them a flurry of notes, overwell them with spec questions. It usually gives me a few days.

    It's tough to think outside the box, when there is no box.

    Sean D.

    --
    "Hmm. I am to metaphor cheese as metaphor cheese is to transitive verb crackers!"
  29. Re:School Doesn't Prepare You For Real World Codin by mrtroy · · Score: 2

    I believe the opposite.
    I am IN school where they mainly teach us abstract concepts and not specific programming languages.
    What good would it be to learn a specific programming language in the rapidly changing technological world? Keep your Cobol and teach me recursive binary tree algorithms.
    If you cant go and teach yourself how to apply these concepts to specific languages, either you arent meant to be coding or your school did not help teach the second step of algorithms which is the application
    Basically, since we were mainly taught algorithms, our assignments would be in a random language like C, java, etc, and we would code portions of these algorithms. Sink or swim, but it worked, I know everyone in my graduating class can code in any language efficiently and effectively.
    *go waterloo*

    --
    [I can picture a world without war, without hate. I can picture us attacking that world, because they'd never expect it]
  30. Controlling complexity by praetorian_x · · Score: 3, Insightful

    Do Slashdot readers think that the theories used to teach (and learn) programming lead to programmers that tend to approach problems with a 'black box', or 'virtual machine' mentality without considering the entire system?

    Well, programming is, at its root, about controlling complexity. A good program (not that *I've* written one) will have sub-components within it that largely act as black boxes to one another. It is a great and rare skill to recognize where the boundries are in your program and establish them early, to avoid painful refactoring later.

    In my experience, it is when something *isn't* a black box that things can get seriously fsked. "What? I set a global variable and now the app seg faults when I click that drop down?" Ahhh, not "black boxy" enough.

    My perspective is from a higher level than embedded though. Embedded is a whole different game, although the "controlling complexity" insight of higher level programming languages no doubt sill applies (as far as it can go)

    Cheers, prat
  31. Level of abstraction by Hirofyre · · Score: 2, Insightful

    The correct level of abstraction for a project is often hard to find, even for very experienced programmers. Sometimes you have to raise the level of abstraction of an overall system, or you will never get to the point where you can move forward on your piece of the process. Generally, I've found that the problems lie in the areas where the pieces don't quite fit together properly; namely where one person's code doesn't follow the contract a second person was expecting. A lot of the time, even for mid-size problems, it would be impossible for one developer (or a team) to have an end-to-end understanding of the problem space.

    As far as I can tell, there is only one way to avoid the "black-box" problem, and that is to have one person code the whole thing, which is very likely infeasable. The further you get from "your" part of the system, the more abstract it is going to get. If your abstraction is faulty, there is going to be trouble, but I wouldn't say it was caused by treating the problem as a black box.

  32. The only thing that school prepares you for by shortscruffydave · · Score: 2, Funny

    In high school I did a 2 year Computer Studies course.

    During that period, one night I went to a heavy party and then spent the following day trying to write functional code whilst suffering a hangover.

    This was the only experience from the course which mirrored anything which happened to me since I started programming professionally

  33. True, but you need an expanded view by nosilA · · Score: 3, Insightful

    In order to do a good job on your module, you need a solid understanding of how the components you directly interact with function. In addition, a superficial understanding of other components is useful.

    For example, let's say you are working on the software for automatic transmission control in the car. You need an intimate understanding of the hardware you are running on, that's directly related to your job.

    However, you also need a solid understanding of how the automatic transmission works. Understanding the mechanics of the gear change is important to understanding timing issues, errors that can occur, and how to deal with them.

    It is very useful to have a good understanding of how a car works in general, to get an idea of how your product will be used. This allows you to optomize your product for likely scenarios.

    Sometimes, for personal satisfaction, it is nice to know how the windshield wiper mechanism works, but it doesn't help you in any way to make your automatic transmission control better.

    -Alison

  34. Oh this is kind of crap... by dasmegabyte · · Score: 4, Insightful

    The "Black Box" design theory abounds because of the freedom it offers programmers from the dark ages of having to know the underlying hardware intimately before anything could be accomplished. It's what allows programmers to devote all of their time to doing what matters rather than pouring over volumes of errata and arcana.

    The reason Windows became so popular, for example, is because its API offered programmers a way to manipulate graphics without having to make graphics calls. Variation from driver to driver was of no concern, and shouldn't be -- that's an IT issue which can be repaired without redoing the entire application.

    And in a perfect world, there's no problem. If a driver hooks into an API properly and documents any disperity, then the black box theory holds true. Problem is, driver aren't perfect. A lot of them are designed for bare bones functionality, and only optimized as necessary (hence how Nvidia's still squeezing substantial horsepower out of my ancient GeForce GTS with every new driver release). Obscure hardware cases always cause trouble, which is why Dells are (sometimes) more reliable than "no name" machines with "better" hardware. Dell has the clout to make sure the drivers are as seamless as possible.

    What's the solution for embedded developers? Design and test the drivers in house, so the black box coders have a shoulder to cry on when hardware doesn't act properly. But it should not be the core developer's job to know what goes on with the hardware. That kind of thinking bloats budgets, increases the complexity of the project and ultimately the cost. Modularity, even though it makes things more difficult to map in total, makes things easier to deal with on a micro level. If the application works when unit tested but fails on the release machine, then it's the driver's fault. Much easier to fix than it is to perfectly replicate the release in your tests.

    Expecting EVERY software developer to be an electrical engineer as well is absurd unless you intend to pay them for both degrees. Better to keep it modular and put the pressure on the hardware abstractor to do a good job of catching the tiger's tail.

    --
    Hey freaks: now you're ju
    1. Re:Oh this is kind of crap... by Lodragandraoidh · · Score: 2, Interesting

      I have been given projects where I had to interface with some existing POS (such as windows) - and I did not have access to the source, and thus needed to approach the project from the standpoint of a blackbox.

      In general this works fine - until some future date, when the owner of the blackbox decides to change the API such that your code breaks.

      Blackboxes, other than in an academic setting, imply a closed proprietary system. Open systems, on the other hand, are not truely black boxes because you do have access to the source and all the underlying APIs (no Eastereggs waiting to create undocumented interactions).

      --

      Lodragan Draoidh
      The more you explain it, the more I don't understand it. - Mark Twain
    2. Re:Oh this is kind of crap... by Lodragandraoidh · · Score: 2, Interesting

      Better to put the responsibility for fixing a driver or api call on somebody who is both libel to me and who has experience in doing so.

      - dasmegabyte



      To quote the FOCUS Magazine interview with Bill Gates [October 23, 1995]:

      "FOCUS:
      Every new release of a software which has less bugs than the older one is also more complex and has more features...

      Gates:
      No, only if that is what'll sell!

      FOCUS:
      But...

      Gates:
      Only if that is what'll sell! We've never done a piece of software unless we thought it would sell. That's why everything we do in software ... it's really amazing: We do it because we think that's what customers want. That's why we do what we do.

      FOCUS:
      But on the other hand - you would say: Okay, folks, if you don't like these new features, stay with the old version, and keep the bugs?

      Gates:
      No! We have lots and lots of competitors. The new version - it's not there to fix bugs. That's not the reason we come up with a new version.

      FOCUS:
      But there are bugs an any version which people would really like to have fixed.

      Gates:
      No! There are no significant bugs in our released software that any significant number of users want fixed.

      FOCUS:
      Oh, my God. I always get mad at my computer if MS Word swallows the page numbers of a document which I printed a couple of times with page numbers. If I complain to anybody they say "Well, upgrade from version 5.11 to 6.0".

      Gates:
      No! If you really think there's a bug you should report a bug. Maybe you're not using it properly. Have you ever considered that?

      FOCUS:
      Yeah, I did...

      Gates:
      It turns out Luddites don't know how to use software properly, so you should look into that. -- The reason we come up with new versions is not to fix bugs. It's absolutely not. It's the stupidest reason to buy a new version I ever heard. When we do a new version we put in lots of new things that people are asking for. And so, in no sense, is stability a reason to move to a new version. It's never a reason."


      So, you can see that your assumption is incorrect. YOU CAN NOT DEPEND ON YOUR VENDOR TO FIX IT. We found this out the hard way at my job - after spending millions of dollars; now we have an open architecture system where we can plug and play different vendor solutions easily, and use open source next to vendor supplied applications.

      On the other hand, I have written the maintainer of a famous development environment [don't want to drop names - not good form] - and he returned my email the same day with an answer to my question. My experience tells me your basic understanding does not jive with reality.

      --

      Lodragan Draoidh
      The more you explain it, the more I don't understand it. - Mark Twain
  35. The biggest lesson I learned... by scottfitch · · Score: 2, Insightful
    The best lesson that I learned on my first large development project was that there is a big difference in the need for abstractions and black boxes in implementation versus design. All code should be written using a black box approach... no matter whether you're programming in SmallTalk or Assembly. (Though some languages make it easier than others :-).

    The big difference is that when you are actually designing and coding (verbs!) you have to look into those black boxes. If you don't understand the subsystems/objects/subroutes that your code interfaces with, you won't know what boundary conditions to test, what assumptions the other subsystems are making, etc.

    So now I always write well abstracted code (just like your Comp Sci 101 prof taught), but design with the big picture in mind.

  36. The funniest post I've read yet! by K'tohg · · Score: 5, Funny

    Dude. I have never laughed so hard! I can only imagine a bad cartoon:

    "Frank!!! I think we have a problem!!!"
    "What?... I'll check the console logs..... Oh No!!"
    "What's wrong?"
    "Theres a 'Fatel Error: Wing no longher attached!' message!!"
    "What do we do Frank?!??"
    "I don't know... um... I'll open up the debugger!"

    Sick man, sick. I am very proud of you. ^_^

    --
    > SELECT * FROM brain_cells WHERE synaptic_rate > 0
    0 row returned
  37. Engineering Black Box and Whole System by mrs+clear+plastic · · Score: 2, Interesting

    Here are some thoughts that go beyond programming and include engineering as well. And not just systems vs black block, but concepts as well.

    Here are some random thoughts.

    Take the slide rule. Back in the days before destops, calculators, and palmtops, we had slide rules to do division and multiplication. You slide the rule for the numerator over the denominator
    (I think, its' been so long). You then look at the
    result.

    The thing is, you can see how 'close' the result is to whatever you desire (in a circuit or system). You can intuit how close thing are. You can easily 'play with the numbers' with a slide rule in some cases. Slide it a little to see what it would take to get the desired results. A teeny amount, alot; whatever.

    With digital calculators, it's a harder (for me) to see the changes visualy. All you see is a quanitive value. I can't look at the physical distances on a slide rule and make inferences.

    I can remember doing the same intuiting with meters. In the days before digitization and computers, we had analog meters. A needle would point to the value (voltage, amperage, whatever). Often the 'movement' of the needle is almost more important than the actual value itself.

    Take the tuning of a final output circuit in a radio transmitter. You dip the plate and tune for
    proper power. With an analog meter, you can see the needle do a quick dip. Sometimes with a digital meter, you can miss the dip, espcially if the circuit has a high Q value. The motion of the needle of the meter controls the speed at which I turn the various knobs.

    With a digital meter, I feel removed from the process of tuning.

    Monitoring the electrical service for a facility, whether it be a radio transmitter facility, or even a computer room; I am much more comfortable with an analog voltmeter and amp-probe. It's far easier for me to watch for hiccups (needles jumping rapidly or slowly) to indicate something is happening.

    I feel that all of these examples are important in my desire to be a part of the overall system, rather than being only a blind black box. I use my overall knowlege of what is happening in the system as a whole to get a 'feel' if what is happening right there and then.

    With only abstract figures and a blind black box interface, I would feel much alone and out of touch with the reality of the system.

    I think the same can be said about programming. In all of the projects I have been involved with, I have been fortunate enough to see the overall picture of the system at a high enough level to be able to able to be a 'part of the system' rather than a disconnected black box'. This is certainly true in my background in writing scripts to monitor the health of databases and operating systems.

    Mark

    --
    Cleara
  38. Not Black box, but *PERFECT* black box by wowbagger · · Score: 5, Insightful

    I think the problem is not so much the "black box" mindset, but rather the perfect black box mindset.

    Being an EE who now does software design myself, I try to decompose a problem into smaller problems, and decompose the solution into smaller parts. However, I don't make the mistake of thinking that my smaller parts are each perfect - I try to ask "Now, if component X malfunctions, what effects will it have on this higher level assembly Y?"

    The problem is that many time CS folks are not taught that the system can be imperfect, so by exclusion they believe it to be perfect - one plus one will always come back two, disk writes will always succeed if there is enough space for them, and so on. Folks are not taught that sometimes 1.0 + 1.0 != 2.0 (rounding errors), that disks sometimes fail (sector not found - abort, retry, cancel), and so on.

    In Circuits 1, an EE-to-be is taught the idea of the perfect op-amp - infinite gain, infinite bandwidth, infinite possible output voltage, infinite input impedance. He is taught to use this model to analyze a circuit.

    He is then IMMEDIATELY taught that the model is BS, and starts to add to it - finite input impedance, finite gain, finite bandwidth, finite offset voltage, finite output impedance. The EE-to-be is taught to apply those non-ideal behaviors when needed, and taught to judge when they can be ignored.

    Sometimes I think the best thing in the world would be if CS and EEs had to work with robotics as part of their job. When they have to deal with sticky steppers, dust-clogged optics, and misfiring soleniods they will learn to be a bit more paranoid.

  39. Hard Software Engineering by PhaserBlast · · Score: 2, Informative

    What you are talking about has been a serious concern for a number of researchers. In particular David L. Parnas has been working on this problem for many years now. The basic problem is that software specification often lacks in precision required to make appropriate decisions when writing the software. This is compounded by the lack of precise specification of the OS itself.

    Take a look at the CRL, SERG and SQRL reserach documents at

    http://www.cas.mcmaster.ca/serg/serg.publications. html
    In particular read the CRL Report 241 - "Predicate Logic for Software Engineering" - it covers some of the fundamentals. Then read up on CRL Report 259 "Formal Documentation of Well-structured Programs". There is tons of other interesting reports there that address some aspects of this very issue.
  40. Roman bridges by giampy · · Score: 4, Interesting

    This reminds me of how the romans used to test their bridges: they put the designer under the bridge while marching over it with the entire legion.

    Of course, a bridge i a MUCH simpler thing than a program, but, hey, 2000 years, all the bridges are still there !!!

    --
    We learn from history that we learn nothing from history - Tom Veneziano
  41. Absolutely! by casmithva · · Score: 5, Interesting
    I've been quite frustrated over the years, interviewing recent college graduates whose software development abilities seem to be limited to problem-solving. They didn't know about requirements, design, configuration management, testing, lifecycles. They didn't put as much thought into how others would use their libraries or classes as they should've, eventually causing some serious redesign to be done to make overall integration easier. Only after a couple of years of having design documents ripped apart and pissed upon, having CM staff threaten them with dismemberment, having QA people file a ton of defect reports against their work, and having their phone ring in the wee hours of the night did they understand the bigger picture.

    I took a couple of CS courses in college as part of my Math major. They were full-blown CS courses, not courses that had been altered for us Math majors. And they were nothing more than problem-solving courses -- and the problems being solved were so utterly asinine that it was laughable. However, when I studied in Germany I took a CS practicum course where we were assigned the task of creating a graphics program in X Windows on SunOS 4. The class was divided into groups: GUI, backend algorithms, SCM, QA, and requirements and management. There were design sessions and reviews, unit and integration testing, etc, etc, etc. It's the closest I'd ever seen to the real world in academia. I've never heard of any American college or university offering such a course, and no one I've interviewed ever had such a course. That's not to say that it's not offered somewhere, but it just doesn't seem all that common. And that's a real shame.

  42. Definitely! by jhouserizer · · Score: 4, Insightful

    While there's definite benefits of treating software components as "black boxes", I agree with the asker of the question that there are some definite negative side-effects.

    For instance, we've got a couple of developers that just don't know how to work with the team, and figure that they can go sit in their dark cubes and code away their component as a black box that with simply fit in with everybody elses stuff. Common problems that arise are:

    • Different logging schemes
    • Different configuration schemes
    • Different admin-alerting mechanisms
    • Components that don't match the design pattern that all other components follow - thus making them harder to understand.
    • Components that expect some type of "global" data to exist, that simply doesn't.
    These issues have led to no end of grief for those of us who do communicate with each other about what they're doing.

    Abstraction is great!, but you still need to make sure everything fits together correctly, and not just at the interface level.

    1. Re:Definitely! by SWPadnos · · Score: 2, Insightful

      I'm not sure that this problem is with the black box mentality - it seems to be with those coders.

      Logging scheme: Should be part of the interface definition (the format of log messages should be part of the spec). The logging functionality should be another black box module (with a suitable interface for all portions of the project).

      Configuration scheme: Should also be part of the spec. If it's done wrong, then the module doesn't meet spec, and the programmer(s) should be reminded that their paychecks are dependent on writing modules to spec.

      Admin Alerting: Like logging, if there is a specific format / function to use, then this should be part of the spec.

      Design pattern: The spec should incorporate any company coding standards (by reference, if it's too long :). It is then up to the programmers to follow this standard. (I see that as a different issue than the black box thing, though)

      Global data: This should never be touched, unless the access is defined in the spec.

      It looks to me like:
      1) the specs are not complete; and 2) that the programmers in question don't adequately communicate when they encounter problems coding to the spec. (It's perfectly valid to discuss changes to a black box specification if problems are encountered).

      --
      - The Sigless Wonder
  43. Re:GET SOME PRIORITIES!!! by cybergibbons · · Score: 2, Insightful

    No, when something bad happens, it normally highlights smaller problems that all are part of the situation. So, instead of mourning the loss of 7 people who you probabky didn't know, which is futile, get something out of it.

    I also don't understand why this is worse than other people who die in air crashes, as soldiers, in cars, or whatever. They've done something with their lives. Get over it.

    It's no worse than the Challenger disaster, and certainly no worse than the Apollo fire. Those men died horrible, slow deaths, for no reason.

  44. Engineers can see all an action's consequences by Minna+Kirai · · Score: 4, Funny

    If you're really an engineer, then you shouldn't have any trouble seeing the big picture.

    Unlike, say, managers or interns, Engineers are trained to think through all the consequences of an action.

    If you can't predict the effects of your software code on not just the rest of the project, but the economy and society as a whole, then I guess you've been slacking off.

    (nobody flame me without reading the cartoon)

  45. A happy medium somewhere by agaznog · · Score: 2, Interesting

    It is true that most, if not all, abstractions are leaky. But it is still essential to be able to work in "black box" mode to contain complexity when necessary. It is just as important to be able to flip back and forth between levels of "nested black boxes" when necessary. Of course no single person can learn everything, which is why there are specialized developers, and management software engineers. Meaning at higher or lower levels of abstraction can be preserved (ie. abstraction leak prevention) when working from a particular level: To ensure that everything is sound and complete in other levels, you usually have superior ranking software engineers looking over the shoulders of the code monkeys. So, if software fails because of naivete on the part of a particular developer, it's most likely an engineering management and/or software architecture problem. You can't blame a single developer for not knowing everything. You might (and probably should) blame his/her managing engineering for not ensuring that everything fits together at higher levels. Or you might the software analysts/architects for not designing everything to fit together properly. If you ranting against the general usage of abstraction in CS, you are naive. Everything humans know is an abstraction. Computer engineering is an abstraction. Electrical engineering is an abstraction. Biology, chemistry, physics and everything in between are abstractions. Mathematics is perhaps the ultimate abstraction of all. Unless you are suggesting that we all should attain some sort of zen like state where all the semantic levels converge into a giant mass, you cannot escape the "black box mentality". (Trying to suggest that programmers need to code in only machine? Or maybe raw electrical impulses?) Rod

  46. Yes and no by brandonY · · Score: 2, Interesting

    On the one hand, you should be able to look at a computer as a black box. If it's not an operating system, and it's not a driver, you shouldn't have to know what sort of system your code is running on. Portability is a wonderful, wonderful thing. On the other hand, you should always take into account what system your program will run primarily on, and you should always be aware of how the systems under your program probably work so that you don't duplicate functionality, try to out-guess the compiler, or make all sorts of horrendously expensive blocking calls that you don't need to make. I'm an undergrad at Georgia Tech, and I've found that one of the big differences between a solid degree in computer science and a weak one is that the better programs open the black box as much as possible, especially later in the program. Sure, the early classes are taught in pseudocode and java and such, but the farther along the education gets, the more we have to take classes like ECE 2030 (which explains transisters up to CPUs) and Design of Operating Systems (which explains printf down to the CPU). Another big difference is theory and knowledge of design paradigms, from the simple, like hash tables, to the more unusual, like factory classes. It makes a big difference to see the big picture, but then again it's quite possible to write perfectly acceptable code without the slightest idea how the API works. Otherwise nobody could write Windows software. Caching and pipelining and all that stuff is useful to remember, but there's a reason most of it is completely transparent -- so you don't have to know it's there.

  47. Stuff that's bigger than any of us. by DoraLives · · Score: 2, Interesting
    I think the problem increases as programmers are less and less a part of the complete systems development life cycle and are contracted to work on individual components of an overall system.

    ANYTHING of sufficient size and complexity is by definition something that no one of us can comprehend in its entirety. This being the case, there's no hope of ever seeing to it that everything from minor annoyances to catastrophic failures will be abolished.

    My experience with this sort of thing isn't in software, it's large scale construction projects. Launch pads, to be precise. The basic goal is "build something that the Space Shuttle can successfully fly off of on launch day." In the real world, NOBODY knows exactly what this is going to involve down to the finest detail, and the possibility for malinteractions of that detail.

    Fortunately, the launch pad, once built, more or less just stays put and continues doing the same job over and over. With software development, no such stability is feasable. We're still learning more and more about how computers work, both hard and soft ware. In this phantasmogoric landscape, with things morphing from this to that with bewildering speed and little overall pattern, the guys who have to grind out the code (and all their bosses right up to the CEO) have no prayer of ever getting it right. Don't be so hard on yourselves, it's a situation that you can NOT control fully. Just do the best you can and let Charles Darwin sort out the mistakes.

    --
    Is it fascism yet?
  48. top-down-bottom-up teaching by Diphthong · · Score: 2, Informative

    In the lovely concrete blob where I went to college, I think they more or less got it right.

    First they teach the basics of programming (variables, environments, scoping, addition, subtraaction, subroutines, etc.) and all of that happy stuff.

    Then they teach you how to build a NAND gate using transistors.

    Then they teach you how to build a CPU using NAND gates.

    Then they teach you how to call subroutines using RAM and registers and pointers and that CPU you just built.

    Then they go back to the basics of programming, now that you have a full appreciation for everything that happens when you add two numbers together, and proceed from there into the nether regions of algorithmia.

    To me it seems to be a good approach -- it gives you the basic destination first, then it gives you the foundations to get there, and then it goes onwards. It means you've been exposed to every level of what you're working with (to a point -- we never doped silicon to build NAND gates!), so you know what's down there, and you also see the importance of ignoring what's down there when it doesn't matter.

  49. All Systems Need Forth? by Baldrson · · Score: 3, Insightful
    Bit serially, although Forth isn't the be-all and end-all of programming environments, it does have the elegant simplicity that should be sought by whole systems.

    One of the priemere embedded systems languages, Forth was invented by Chuck Moore. I like Chuck Moore's 1% Code Page. His introduction:

    I've studied many C programs in the course of writing device drivers for colorForth. Some manufacturers won't make documentation available, instead referring to Linux open source.

    I must say that I'm appalled at the code I see. Because all this code suffers the same failings, I conclude it's not a sporadic problem. Apparently all these programmers have copied each others style and are content with the result: that complex applications require millions of lines of code. And that's not even counting the operating system required.

    Sadly, that is not an undesirable result. Bloated code does not just keep programmers employed, but managers and whole companies, internationally. Compact code would be an economic disaster. Because of its savings in team size, development time, storage requirements and maintainance cost.

  50. only to a point by dwk123 · · Score: 2, Interesting

    This only works to a point, and the fundamental problem as others have pointed out is that the 'black boxes' are almost never specified to a level of detail adequate to fully describe their behavior. Things like side effects, performance criteria/guarantees, behavior on edge conditions etc are very frequently overlooked. Your statements are idealized versions that might have been taken from some software Methodology book - they completely ignore the real-world problems like bugs, partial/incomplete implementations, outdated specs/documentation and a whole host of others.
    This doesn't mean that you always have to be explicitly focused on these issues, but the overall success of the system as a whole can be critically dependent on them. In a perfect world, of course, this wouldn't be an issue. But assuming the viability of black-box treatment in most real-world projects is the source of many problems, and the truth is that a relatively small portion of the population is capable of maintaining a sufficiently broad view of a system to be able to effectively respond..

  51. Abstaction: powerful, necessary, and often misused by tmoertel · · Score: 3, Insightful
    As systems become increasingly complex, the practicality of (and even the feasibility of) treating them as a monolithic entities becomes a legitimate concern. One method of addressing this concern is via abstraction, where suitable models are used as representations of their real-world counterparts. Indeed, abstraction is one of the fundamental principles of engineering and has been used for thousands of years to great success in the construction of complex systems.

    For example, in electrical engineering, it is common to simplify complex circuits by breaking them into smaller circuits, analyzing each of the smaller circuits, and then replacing the smaller circuits with appropriate black-box equivalents. With these black boxes in place, the original circuit becomes much easier to understand.

    However, as in any modeling exercise, it is crucial to choose appropriate models and to understand the limitations of the models chosen. While a simple resistance model might be a good substitute for DC and low-frequency circuits, it would be inappropriate as a substitute for higher-frequency circuits where capacitance effects come into play.

    So, returning to the original poster's questions:

    Do Slashdot readers think that the theories used to teach (and learn) programming lead to programmers that tend to approach problems with a 'black box', or 'virtual machine' mentality without considering the entire system?
    Yes, in these days too much stock is placed in the idea of letting somebody else worry about the complexity. This is especially so in mainstream industry, where one of the key selling points of software development systems is that with Magic DevStationPro X you no longer have to worry about the details but instead just use some brilliant Wizard or API to work at "a higher level." This applies not only to software development but also to user-level domains such as operating systems and applications. For example, a common notion in industry is that by using Microsoft operating systems on servers, administrators no longer need to know how to administer servers; rather, they need only know how to use the GUI administration tools. In other words, the pitch is that you need not concern yourself that the GUI tools present a mere model of the underlying system. Let the model be the system and reap the rewards.

    That's hogwash. The model approximates the system, no more. In engineering, this is well understood, and I suspect that in good CS programs the same can be said.

    Abstraction is a powerful tool. It is widely applicable, effective, and well founded -- when used appropriately. It probably ought to be used more often. Nevertheless, it is not a substitute for rational thought. Nor is it a replacement for being responsible for the entirety of the systems we build.

    That, in and of itself, would explain a lot of security issues, as well as things as simple as user interface nightmares. Comments?"
    It would certainly explain some of these problems, but I suspect that far more errors are the result of interface errors. One of the tools that goes with abstraction is composition -- breaking things into pieces, treating the pieces individually (at fine granularity), and combining the pieces (at a larger granularity) to yield a system. The risk of combining pieces is that in order to put them together properly, the boundaries where they coincide -- their interfaces -- must be well understood and compatible. Since the individual pieces make natural units for delegation, pieces are often assigned to different people who may have slightly differing understandings of the boundary conditions. As a result, "interface mismatches" are a significant source of error in software systems.

    Certainly abstraction plays a role here. Each piece can be thought of as a black-box model. The limitations of that model, and the assumptions under which the model is valid, are certainly important characteristics of the piece's interface with the world. Yet, these characteristics are frequently neglected in documentation and often go uncommunicated across delegation boundaries. This sad fact makes interface mismatches an especially harmful side effect of using abstraction and composition in common software development practice.

    Nevertheless, abstraction is a powerful and genuinely useful tool. It is also a necessary tool if we are to build increasingly complex systems. Like any tool, its uses and limitations must be understood if it is to be applied effectively. Thus, getting back to the original poster's question about whether the use of black boxes is harmful, my answer is, No.

    The problem isn't abstraction, the problem is improper use of abstraction.

  52. Re:Supportable software? by mobiGeek · · Score: 4, Insightful
    Seriously, screw the next guy. Just follow your companies procedures (no more no less) and move on your merry way.
    When you said "seriously", you were joking, right?

    I mean, c'mon! The great U.S. of A. is gonna remain on top of the world by "obscurity" ?

    The reason the guy in India/Mexico/Nebraska/whereever makes $5/hr and is worth it is because he does a job that is only worth $5/hr. If the job is worth $55/hr, then a $55/hr person will get that job.

    Do you honestly believe that you are able to hold onto a higher paying position because your code is not documented? Do you honestly believe that this is the way to stay ahead of the game?

    It is this mentality that has stifled innovation. People spend all sorts of time trying to figure out how-in-the-hell they got to where they are instead of trudging forward down avenues unknown.

    Unions in the Western World are doing just this: pressure tactics to avoid "outsourcing" of work. The reason the work is being outsourced is that it is no longer work worthy of a high-paying (and supposedly higher-quality) employee. The work is being shipped off elsewhere because someone else can do it cheaper (possibly at a lower quality or a lower efficiency, but if the resultant product effectively meets the desired goal, then it is the right move for the company and, ultimately, ITS CUSTOMERS).

    The Unions need to work with the companies to find ways to take these higher-productive workers and make the company even higher returns. But Unions don't "work with" management and fight all change.

    Oh, don't get me wrong! I know that management has just as many boobs running the show too. They make plenty of mistakes themselves. But the nature of "management vs. union" mentality keeps the unions from effectively working with (upper) management to enable change that makes better use of all resources and, in turn, makes the company more profitable.



    Seriously (no, I'm not kidding...I mean seriously), you cannot believe that documenting your code, outlining your procedures, using effective architectural designs, and improving the company procudures is threatening your job, can you? If you do, then realize that a single, competent manager is all it takes to tear down your warped house of cards...

    --

    ...Beware the IDEs of Microsoft...

  53. Re:Probably (levels) by oconnorcjo · · Score: 3, Informative

    Programming Levels:

    # Microsoft Frontpage
    # Raw HTML
    # CGI/PHP/etc.
    # Servlets/Mod-perl/etc.
    # Object-Oriented black boxes
    # Documented API's
    # Public Documented API's
    # Performance
    # "The Big Picture" - Architects



    Another elitest post without a real clue. A good programmer knows how to get a job done and should ALWAYS have a big picture view of how things work around them. It does not matter whether they are working on a web site or writing a backend database app or a game engine for the latest and greatest game. Somebody writing good PHP code could probably write good backend C++ code. You are associating the tasks people do with how capable they are. Languages and programs are TOOLS and a programmer should be able to quickly learn to use new tools whether it is a new language, interacting with a new API or using a performance profiler. A good programmer really should not care HOW they get things done- ONLY that they DO get them done.

    --
    I miss the Karma Whores.
  54. Damn you, Clippy. by Mr.+Bad+Example · · Score: 3, Funny

    It looks like you're trying to fly a craft with only one wing. Would you like to:

    * Plummet to the ground

    * Stick an arm out the window and flap vigorously

    * Visit the Morton Thikol technical support website?

  55. Black boxes can be expensive too by vrassoc · · Score: 2, Insightful

    ... some people tend to get tunnel vision and concentrate wholely on the bb theory, ... This does usually lead to problems and errors in the code.

    Not only errors: black box routines can be expensive on performance too. Take database programming as an example: black box programming teaches us that we must break the problem down into its smallest (easily) solveable (and reusable) parts, create a routine for each one and then work our way up. In a program that does disk reads to solve the problem this could mean many, many more reads from disk than what is necessary, if you're not careful.

    IMHO part of what makes a good programmer an even better programmer, is to know which routines to black box, taking into consideration performance and resource availability, and which ones not to.

    High level languages are no less than a library of black box routines. And there are often parts of a program that are best written at the lowest level, for efficiency's sake.

    A good programmer will constanly be weighing up the pros and cons of his or her methodology in order to provide a system that is sufficiently practical for the underlying architechture while taking into consideration all the other constraints of the project such as budget and deadline.

  56. Is it really Embedded Vs Virtual Machine? by that+_evil+_gleek · · Score: 2, Interesting

    Did you do you're embedded programming in C? Or Forth? I'm thinking C because you seem to put as an 'either or. ' I think the difference is how much you, have to do, yourself, to get to something the resembles a 'virtual machine.' More, "Who provides the virtual machine?" than "Is it a virtual machine?"

    One thing to remember is that the only thing some programmers learn from school
    is how to misuse elements of CS to rationalize away the fact that they suck.
    IGNORE sophistical arguments, instead of buying into the BS and getting the wrong idea
    about whatever they're using as an excuse.
    It sounds like you've heard 'blackbox coding' where I've heard 'implementation details'.
    Like:
    Me: What if?---
    Them: I'm not worring about 'implementation details' [with a tone that suggest that they are above it all]

    But for you, perhaps, it was like:
    You: What happens when?
    Them: We don' have to worry about that because we're coding to a virtual machine.
    You: Yeah I know, but ---
    Them: Haven't you taken object orientied programming and design?
    You: Yeah
    Them: Ah then you see... [start long winded lection about OO that isn't germane ].

    There are two types of people: People who don't know everything, and people who don't admit that they don't know everything. If we had been dealing with the former. It would have gone like:
    Us: What about?
    Them: Well, if that happens we're fscked. That's something we'd /have to/ deal with
    if we were doing embedded systems programming like in medical equipment, but for now for the purposes of CS201 we implement the 'ostrich algorithm'.

  57. Yes, Black Boxes are the norm by Gleef · · Score: 2, Insightful

    I studied Computer Science in College, and currently work as a Programmer/Analyst for a non-profit organization (Desktop, Web and Server-based systems). Yes, all of the above encourage a "black box" model to design and coding. Furthermore, I am guilty of perpetuating this to the people forced to listen to me blather, and will continue to do so until I see a better way.

    I understand that it hides some bugs. I don't like this. On the other hand, we can never have enough staff to make sure we have people expert on not only each system used but on each interface between the system to do a good integrated system.

    So what we do is take some premade components (eg. hardware, OS kernel, C library, certain widget libraries, web server, etc), and say "OK, assume these work according to these specifications, we're going to work on adding a piece that does this". When the premade component deviates from the specifications, we fix the component or update the specs.

    As much as possible, we make use of open standards and free software so that if we need to, we can open up the black box and fix something. However, the more we can assume that a component is a black box that will just do what it's supposed to do, the faster we can develop the "interesting" bits.

    The bottom line for us is to manage complexity. The more complexity that we can abstract away, the faster we can work on the custom stuff unique to our organization. A "black box" model works well for us, but yes, it does cause some bugs that need to get cleaned up after the fact. Most organizations I've seen make a similar design choice (or blunder into it blindly), and most schools teach their courses with a similar mindset.

    If we were to develop a truly critical system, one that lives or big bucks depended on, we ought to take a different approach for that system, but we aren't likely to work on such a system for a while.

    --

    ----
    Open mind, insert foot.
  58. Managers think more code, better coder by lukme · · Score: 2, Insightful

    Some managers think that you are doing nothing unless you can say, yup, wrote 3000 line of code today. They don't really seem to care that you have just created a mantenance problem.
    I was just at a job interview where my comment that I have done several projects in which I mentioned that I have replced around 5000 lines of code with 500. Due to his obsession with lines of code, this got missinterpeted as, I have only done mantenance.
    Note: the 500 line of code did the more than the 5000, with fewer bugs and was alot faster (ie, 1 scan though the data as opposed to multiple).
    Every job that I have had, I have found areas of code bloat either done due to the pressures of meeting a deadline or though incompentence. In either case, the best thing to do is to just clean them up and go forward.

  59. The boxes have to be black by revbob · · Score: 2, Insightful
    My business card says "Embedded Software Engineer" and my current job is member of the software architecture team for the Common Operating Environment of perhaps the largest System of Systems project ever.

    I see preliminary designs for databases of objects that magically exist in pure object-land (i.e., they don't actually do anything) and yet somehow the work gets done.

    By training and disposition, whenever I don't smell silicon, I become deeply suspicious, so my first reaction is that such designs are nonsense. Perhaps it will not always be this way -- for instance, perhaps the designers of those very systems will get around to saying who actually does something and how they do it.

    But I've grown to realize that I must accept a certain amount of nonsense (subject always to good engineering judgment and a demonstration that some of these fanciful schemes can actually work) because the "how" absolutely must not enter into the design.

    If I have to say to someone writing the software for communicating between commanders and various kinds of "things" (I'm going to apply some severe declassification here) that to talk to a big orange truck you have to stick a 32-bit word into a mailbox interrupt register at such-and-such and address, while to talk to a little red truck you have to send "HELLO, WORLD!" to port 80, they're going to say to me, "Just what the hell have you been doing in your software architecture group for the past six months?"

    This is a gross example -- but the less obvious examples are nearly as bad, from my point of view.

    For instance, since one of the requirements for this SoS is that communications not be of the form, "Let's tell the enemy what we're going to do", and since communications security is best done by people who know what they're doing, we will not train every engineer to manage communications security everywhere in his application, but rather layer the architecture so that, to the greatest extent possible, engineers will not even know it's happening.

    Indeed, I expect the architecture our team develops to survive several iterations of "how"s. The first implementation better not work as well as the final implementation, or somebody's wasting money.

    In short, we'll use elementary principles of engineering in order to define common objects that communicate with one another in precisely defined ways at a level of abstraction that's appropriate for the objects themselves. That some objects will have precise real-world counterparts (e.g., big orange truck) is merely evidence that the architecture is sane. And if some of those objects have functions associated with them, that's because in the real world functions aren't performed by spirits and demons, but by (now let's not always see the same hands) objects!

    This ain't rocket science, people. If you've written an API that you can't jack up, haul out the Yugo that's underneath, and replace it with a Viper with no one the wiser except the customer who appreciates how fast he's going, you've screwed up. You've let the "how" creep into your "what".

    (Hoping some people will return my phone calls and answer their email so I can stop talking about this and get back to doing it).

  60. Were not all talking about the same thing here. by small_box_of_stuff · · Score: 2, Insightful

    After reading many of these posts, I think were getting a bit confused here. In part, the original poster is a bit confused, or at least imprecise in his wording.

    When coding a module or piece of program for use in a wider system, such as a library or module, black box thinking is a good thing. In this context, a black box means something that does not expose its internals, and provides an abstraction to the user/programmer.

    The STL or the standard c library is a black box. TCPIP and the sockets API is a black box. the java standard class library is a black box. If you were required to know all the interal details of all these systems to be able to use them, you wouldnt get very far.

    Abstractions, which is what we are talkign about when we use the term black box as above, are absolutely required to write decent software. You simply cant reliably keep that many details inplay at once and expect to get it to work right.

    As useful as the above black box libraries are, it is very possible to create a library that is unusable. Making the user of the library know all the impelentation details from inside is one way to do this, but the other is to use the wrong abstractions, make the wrong assuptions about usage, etc.

    In short, what the original poster and many others are complaining about when they say they dont like black boxes is that they are using bad or incompatible boxes!

    Having to fight with a library that makes assuptions that are invalid, provides the wrong level of abstraction, or is not implemented very well is not an indictment of black boxes, but of black boxes that suck. The fact that their internals are hidden is not the problem. the fact that their interface to these internals sucks is the problem.

    So dont confuse the problems of using bad black boxes with a fundamental problem with black boxes. A bad impelemtation doesnt mean the concept is bad. A poorly designed system well may provide a black box that no one can use, or at least not in all situations. but making that box white, and exposing all its internals, isnt the solution. designing the module to work right is.

    Further, one persons perfect abstraction is anothers miserable pile of junk. Just because the black box doesnt provide the interface you need/want doesnt mean it sucks, it could just mean your using the wrong tool for the job. Complaining about how hard threads make it to hammer in a screw doesnt make much sense. Use a nail.

  61. Proper delegation by ajole · · Score: 2, Interesting

    I've always beleived that a proper system, from an oo perspective, has a top-down hierarchy with no links going upwards. That is never possible, but if you have tools T that are used by users U, and U are used by Senior users S, then T will never use U, and U will never use S. Heck, it works in companies...
    Now I know that sounds elementary and naive, but I still beleive that if you are making [too many] links up, it might be possible to re-work your design.

    --
    -P ...and the boy pulled open his bleary eyes an discovered the python he always knew he was.
  62. Re:Supportable software? by cmacb · · Score: 2, Insightful

    I can understand the original posters "screw them" attitude. It is very frustrating for people who actually want to do quality work to be ultimately punished for trying to do so.

    Process improvement methodologies that proliferated in the 90's seem to be falling by the wayside. I think part of the problem is that at about the same time RAD techniques promised upper management almost instant turnaround on their requests. The RAD techniques, and the tools that go with them have turned out reams of bad and unmaintainable code that companies will either have to live with, or replace the old fashioned way (which involves thinking about business rules, a process quite foreign to most of the people involved).

    Hopefully the current generation of programmers who were obsessed with pretty graphics will be replaced by a new generation who take those things for granted and can get back to solving actual problems.

    Regarding the original story I do find it odd that the existence of that box in the wild represents a threat. One would HOPE that these transmitions use encryption techniques that depend on large keys for their security and not on the obscurity of the algorithm itself

    On the other hand I've read that a lot of this code is written in Ada and NASA is going to be walking on eggshells trying to modernize anything that uses this old software.

    While it may be completely debugged, transition of this software via new hardware and a new Ada compiler, or a re-write to a more modern language is bound to generate errors.

  63. A good example of this... by shylock0 · · Score: 2, Interesting
    A while ago I submitted to Ask Slashdot about command line/GUI interfunctionality (you can look up the post for yourself, it's a while old -- but look through my info and you'll probably find it).

    Anyway, I think that the issue of the GUI is a great example. Programmers got carried away with the GUI, and now applications and OSes are completely over-GUIed. The mouse is much, much slower than they keyboard when it comes to many tasks. I use graphic design programs on a regular basis, and I would give an arm and a leg to have a quick and easy command line interface in, say, Adobe Illustrator, for precise object manipulation. Same goes for Photoshop. AutoCAD and other programs have a decent implementation of the CLI, but it could get much better.

    I would love to see programmers get out of the object-oriented point-and-click mode that they've been stuck in since the invention of the original Macintosh.

    GUIs are great for representing data, and they are great for the visual manipulation of data. But visual manipulation is often imprecise. For precise data manipulation, the CLI is still necessary -- clicking through a menu and two dialog boxes to finally find a text box with the field to rotate an object by 20 degrees, or add a 2nd column to the page, or fix page margins; that's absolutely ludicrous. There should be a simple, (preferably standardized) command line that's accessible from all applications. Remember the ~ in the original Quake? That was a huge step forward. We need it in more applications. How much productivity has been lost by over-mousing? -Shylock0

    Questions and comments welcome. Flames ignored. Post responsibly.

    --
    Statistically speaking, there's a 99.998% chance that my IQ is higher than yours. Get over it.