This is why you should keep the pure programmers to the higher level functions and let the HW engineers write the hardware interface.
They still need to write debug code to test the hardware (Unless you enjoy sending prototype hardware between SW and HW engineers just to find out if the problem is a SW or HW bug.) so they might just as well add the extra glue to make the hardware abstraction.
This way the HW bugs are discovered early in the prototype stage and the right person is given an opportunity to do a proper fix before it leaves his desk.
You still want the SW-guys to be involved in specifying the interface to the HW access functions, otherwise you will get an extra level of bloat when they try to adapt their code around the access functions.
An excellent approach. It forces the hardware guys to think about the software. It's also good for a team if at least some of the software people have a good understanding of how to write low level code to talk to the hardware. Even if they don't do most of it, it helps them to specify reasonable interfaces.
No, you send e-mail about it immediately. If you get blocked 10 minutes after the day's stand-up has ended, you're doing to wait 23+ hours to bring it up? Seriously?
Yes, seriously. That's what good Agile drones are supposed to do. No talking about things outside of class, and no talking unless the whole class is there, lest someone feel left out. Leads/managers can pretend they know what's going on by attending the 15 minute roll call. The more I hear Agilistas justify it, the more Agile sounds like a good way to run a kindergarten class. If people can't manage themselves a week at a time, you really really need better people.
The daily Party meeting sounds like a way to attempt to herd people who are too inexperienced to know when to report/consult/discuss something, or to compensate for a culture that otherwise discourages that, or too irresponsible to be allowed to work for more than a day by themselves, or who are not good team players, because good team players will report/consult/discuss whenever it's desirable. To deal with the fact that it's never perfect, and some greater formality and general sharing is required, a weekly meeting should be more than good enough. If it's not, find better managers and/or programmers.
And hourly would be even closer. Perhaps people should learn, and be allowed to, talk with each other outside of approved Party meetings.
From your slashdot id, use of language, and over reliance on the wisdom of others I would say you are new to the game.
If I'd known that one was required to obtain a Slashdot ID before starting in this business, I would have done so immediately. "Use of language" is too vague to address. I'm also puzzled where I showed an "over reliance on the wisdom of others", though I confess I don't think I invented everything myself.
As for new to the game, definitely newer than Grace Hopper was. So far I only have about 30 years. Long enough to have seen many fads and buzzwords come and go. "Agile" is just another buzzword to sell books and for "gurus" to suck up consulting and speaking fees. To the extent that it contains ideas that are useful in certain situations, there is nothing new about it. Software technology changes but most "new" program management practices are at best old wine in new bottles.
Oh, us poor beleaguered software engineers, everything gets dumped on us! Hardware screws up and we have to fix it.
Gimme a break. I see it from both sides. Yes I have an advantage of sorts when I get to design both the hardware and the software because I'm eating my own dog food (not so bad if you're a good cook), but if hardware gets designed without consultation with software then you're working in a disfunctional organization, period. Believe me, the hardware people in your organization have plenty of complaints about software too.
BTW, that "hardware gets fixed in software" line especially doesn't cut it when designing analog/RF or even digital that just runs too fast for software intervention.
You've fallen into the trap of using their terminology. As soon as 'the problem' is defined in terms of 'upfront design', you've already lost half the ideological battle.
'The problem' (with methodology) is that people want to avoid the difficult work of thinking hard about the business/customer's problem and coming up with solutions that meet all their needs. But there isn't a substitute for thinking hard about the problem and almost certainly never will be.
We're getting buzzwordy here. To me "thinking hard about the business/customer's problem and coming up with solutions that meet all their needs" is the first and most important part of the upfront design, though I could care less what you call it.
That's the flexibility needed in the real world, which is what overly rigid advocates of the Big Plan don't get. Real projects always have with a certain unavoidable level of changes and chaos. Agile is designed to be pure chaos.
This is my evidence that "proper software engineering" *can* work. The fact that most businesses (and their customers) are willing to save money by accepting less from their software is not the fault of software engineering. We could and did build buildings much faster than we do today, if you are willing to make more mistakes and pay more in human lives. If established industries and their customers began demanding software at that higher standard and were willing to pay for it like it was real engineering, then maybe it would happen more often.
Impressive stuff, and not unique to the space shuttle. Fly-by-wire systems are the same way. You're talking DO-178B Level A stuff. It works, and it's very very expensive. If it was only 10x the cost of normal software development I'd be amazed. I agree that way too much software is poorly planned and implemented crap, and part of the reason is that nobody wants realistic cost estimates or to make the difficult decisions about what it's supposed to do up-front. But what you're talking about is aerospace quality. You couldn't afford a car or even a dishwasher made to those standards.
Until the thing is built or the software is shipped there are many options and care should be taken that artificial administrative constraints don't remove too many of them.
Exactly, and as someone who does both hardware and software I can tell you that that's better understood by Whoever Controls The Great Spec in hardware than in software. Hardware is understood to have physical constraints, so not every change is seen as the result of a screw-up. It's a mentality. I'll also admit that there is a tendency to get sloppy in software specs because it is easier to make changes. Hardware, with the need to order materials, have things fabbed, tape out a chip, whatever, imposes a certain discipline that's lacking when you know you can change the source code at anytime. Being both, I'm not saying this is because hardware engineers are virtuous and software engineers are sloppy, but because engineers are human (at least some of them).
So you're saying if a fellow team member is doing something in a way you think could be done better, you just stay silent? If a team member is planning to do something that you think isn't actually necessary because of something you're doing, you just stay silent? If a team member is struggling with a problem you have the skill set to help out with, you just stay silent?
It's true, for Agile to work, you need to have a team.
There is a difference between a team and trying to create a hive mind. If everybody has to talk about this crap every day, I fall asleep. We have a 0.5-1 hour meeting every week where everyone talks about what they've done, will do, what problems they face, and how they think they can fix it. That seems about right. It isn't necessary for everyone to go into excruciating detail about everything they do. Good people generally know when they've got things under control by themselves and when they have a big worries or need multiple people working on an issue. The daily confessionals seem like something out of 1984. In addition to the weeklies we have either impromptu or sometimes official meetings whenthings come up. We're also trying a new technology called "email".
"Proper software engineering" does work, its called "Systems Engineering", is well established and successfully used for large-scale mission-critical projects in almost every industry outside of IT - which seems to be blind to anything invented outside of IT.
True, but the problem is when the Great Plan becomes too rigid. It's necessary to change it as warranted. This actually seems to be easier in hardware, when you say "sure I could get that last 1dB of dynamic range you wanted, but it'll double the price and power consumption", or "yes we could put that function in module X as planned, but it'd be half the cost and complexity if we move it to module Y". Good systems engineers know this. Of course the headache of being a systems engineer is fighting with the implementers - half the changes or spec relaxations they ask for make sense, and the other half are about laziness or incompetence.
I do hardware and software in a 50/50 split, so I see both sides. Hardware is more easily understood to have physical constraints, but every thing in software that isn't exactly as planned is seen as a failure of the implementers.
You're right, but you're going to the other extreme. The problem with all methodologies, or processes, or whatever today's buzzword is, is that too many people want to practice them in their purest form. Excessive zeal in using any one approach is the enemy of getting things done.
On a sufficiently large project, some kind of upfront design is necessary. Spending too much time on it or going into too much detail is a waste though. Once you start to implement things, you'll see what was overlooked or why some things won't work as planned. If you insist on spinning back every little change to a monstrously detailed Master Design Document, you'll move at a snail's pace. As much as I hate the buzzword "design patterns", some pattern is highly desirable. Don't get bent out of shape though when someone has a good reason for occasionally breaking that pattern or, as you say, you'll wind up with 500 SLOC's to add 2+2 in the approved manner.
Lastly, I agree that there is no substitute for good engineers who actually talk to and work with each other. Also don't require that every 2 bit decision they make amongst themselves has to be cleared, or even communicated, to the highest levels. If you don't trust those people to make intelligent decisions (including about when things do have to be passed up) then you've either got the wrong people or a micromanagement fetish. Without good people you'll never get anything decent done, but with good people you still need some kind of organization.
The problem the article refers to about an upfront design being ironclad promises is tough. Some customers will work with you, and others will get their lawyers and "systems" people to waste your time complaining about every discrepancy, without regard to how important it is. Admittedly bad vendors will try and screw their customers with "that doesn't matter" to excuse every screw-up and bit of laziness. For that reason I much prefer working on in-house projects, where "sure we could do exactly what we planned" gets balanced with the cost and other tradeoffs.
The worst example of those problems is defense projects. As someone I used to work with said: In defense everything has to meet spec, but it doesn't have to work. In the commercial world specs are flexible, but it has to work.
If you've ever worked in that atmosphere you'll understand why every defense project costs a trillion dollars. There is absolutely no willingness to make tradeoffs as the design progresses and you find out what's practical and necessary and what's not. I'm not talking about meeting difficult requirements if they serve a purpose (that's what you're paid for) but being unwilling to compromise on any spec that somebody at the beginning of the project pulled out of their posterior and obviously doesn't need to be so stringent. An elephant is a mouse built to government specifications. Ok, you can get such things changed, but it requires 10 hours from program managers for every hour of engineering. Conversely, don't even think about offering a feature or capability that will be useful and easy to implement but is not in the spec. They'll just start writing additional specs to define it and screw you by insisting you meet those.
As you might imagine, I'm very happy to be back in the commercial world.
It's wouldn't necessarily not bring it down either. Seagulls can bring down an airliner (only survivable if you have a pilot good enough to bring it down safely in the Hudson). You can always cite cases of people falling out of airplanes without parachutes and surviving, or an F-15 landing with one wing sheared off with one wing sheared off, but for the sake of safety you might want to avoid those things.
Good to know. Unfortunately the European Court of Human Rights has no jurisdiction in the US.
Whatever happened to the days when the US and the UK could rightfully pride themselves on having stronger individual rights than those closet non-constitutional monarchies on the continent?
The problem here isn't so much with the collection of DNA, but the retention.
Agreed, but note how that isn't addressed by the court, the press or the laws. As noted by posters above, even fingerprints are not usually deleted even if charges are dropped because a school bus full on nuns says "we saw the whole thing and he didn't do it". The best approach would be not only to delete the info for anyone who has charges dropped or is acquitted, but to change who analyzes the DNA. Because there have been cases of incompetent or corrupt police crime labs ("we know he's guilty so just say it matches"), the Innocence Project has suggested that DNA matching be done by N independent and accredited labs, with which lab a sample is sent to chosen at random. The samples should also be submitted and tracked in such a way that it doesn't indicate who or where the sample is from. Lastly, the labs should undergo random checks by having already known samples sent to them in exactly the same way as "real" samples are.
For once the UK leads the USA in the long, slow slide to a police state.
IIRC in the UK there have also been questions about when the samples/information could be kept, at first for those who were acquitted or had charges dropped, and later for those who weren't even arrested but submitted DNA (as "requested" by the police) to exonerate themselves of any suspicion. The latter is sometimes done in blanket sweeps (we have reason to believe the perpetrator is a male living in Greater London is but a bit of an exaggeration).
Slashdot Mirror
Agile is designed to operate in chaos, and bring structure to it.
The problem is when you start with chaos. At best Agile is a band-aid.
This is why you should keep the pure programmers to the higher level functions and let the HW engineers write the hardware interface. They still need to write debug code to test the hardware (Unless you enjoy sending prototype hardware between SW and HW engineers just to find out if the problem is a SW or HW bug.) so they might just as well add the extra glue to make the hardware abstraction. This way the HW bugs are discovered early in the prototype stage and the right person is given an opportunity to do a proper fix before it leaves his desk.
You still want the SW-guys to be involved in specifying the interface to the HW access functions, otherwise you will get an extra level of bloat when they try to adapt their code around the access functions.
An excellent approach. It forces the hardware guys to think about the software. It's also good for a team if at least some of the software people have a good understanding of how to write low level code to talk to the hardware. Even if they don't do most of it, it helps them to specify reasonable interfaces.
No, you send e-mail about it immediately. If you get blocked 10 minutes after the day's stand-up has ended, you're doing to wait 23+ hours to bring it up? Seriously?
Yes, seriously. That's what good Agile drones are supposed to do. No talking about things outside of class, and no talking unless the whole class is there, lest someone feel left out. Leads/managers can pretend they know what's going on by attending the 15 minute roll call. The more I hear Agilistas justify it, the more Agile sounds like a good way to run a kindergarten class. If people can't manage themselves a week at a time, you really really need better people.
The daily Party meeting sounds like a way to attempt to herd people who are too inexperienced to know when to report/consult/discuss something, or to compensate for a culture that otherwise discourages that, or too irresponsible to be allowed to work for more than a day by themselves, or who are not good team players, because good team players will report/consult/discuss whenever it's desirable. To deal with the fact that it's never perfect, and some greater formality and general sharing is required, a weekly meeting should be more than good enough. If it's not, find better managers and/or programmers.
Daily is better as it is closer to real time.
And hourly would be even closer. Perhaps people should learn, and be allowed to, talk with each other outside of approved Party meetings.
From your slashdot id, use of language, and over reliance on the wisdom of others I would say you are new to the game.
If I'd known that one was required to obtain a Slashdot ID before starting in this business, I would have done so immediately. "Use of language" is too vague to address. I'm also puzzled where I showed an "over reliance on the wisdom of others", though I confess I don't think I invented everything myself.
As for new to the game, definitely newer than Grace Hopper was. So far I only have about 30 years. Long enough to have seen many fads and buzzwords come and go. "Agile" is just another buzzword to sell books and for "gurus" to suck up consulting and speaking fees. To the extent that it contains ideas that are useful in certain situations, there is nothing new about it. Software technology changes but most "new" program management practices are at best old wine in new bottles.
Oh, us poor beleaguered software engineers, everything gets dumped on us! Hardware screws up and we have to fix it.
Gimme a break. I see it from both sides. Yes I have an advantage of sorts when I get to design both the hardware and the software because I'm eating my own dog food (not so bad if you're a good cook), but if hardware gets designed without consultation with software then you're working in a disfunctional organization, period. Believe me, the hardware people in your organization have plenty of complaints about software too.
BTW, that "hardware gets fixed in software" line especially doesn't cut it when designing analog/RF or even digital that just runs too fast for software intervention.
You've fallen into the trap of using their terminology. As soon as 'the problem' is defined in terms of 'upfront design', you've already lost half the ideological battle.
'The problem' (with methodology) is that people want to avoid the difficult work of thinking hard about the business/customer's problem and coming up with solutions that meet all their needs. But there isn't a substitute for thinking hard about the problem and almost certainly never will be.
We're getting buzzwordy here. To me "thinking hard about the business/customer's problem and coming up with solutions that meet all their needs" is the first and most important part of the upfront design, though I could care less what you call it.
That's the flexibility needed in the real world, which is what overly rigid advocates of the Big Plan don't get. Real projects always have with a certain unavoidable level of changes and chaos. Agile is designed to be pure chaos.
http://www.fastcompany.com/28121/they-write-right-stuff
This is my evidence that "proper software engineering" *can* work. The fact that most businesses (and their customers) are willing to save money by accepting less from their software is not the fault of software engineering. We could and did build buildings much faster than we do today, if you are willing to make more mistakes and pay more in human lives. If established industries and their customers began demanding software at that higher standard and were willing to pay for it like it was real engineering, then maybe it would happen more often.
Impressive stuff, and not unique to the space shuttle. Fly-by-wire systems are the same way. You're talking DO-178B Level A stuff. It works, and it's very very expensive. If it was only 10x the cost of normal software development I'd be amazed. I agree that way too much software is poorly planned and implemented crap, and part of the reason is that nobody wants realistic cost estimates or to make the difficult decisions about what it's supposed to do up-front. But what you're talking about is aerospace quality. You couldn't afford a car or even a dishwasher made to those standards.
Until the thing is built or the software is shipped there are many options and care should be taken that artificial administrative constraints don't remove too many of them.
Exactly, and as someone who does both hardware and software I can tell you that that's better understood by Whoever Controls The Great Spec in hardware than in software. Hardware is understood to have physical constraints, so not every change is seen as the result of a screw-up. It's a mentality. I'll also admit that there is a tendency to get sloppy in software specs because it is easier to make changes. Hardware, with the need to order materials, have things fabbed, tape out a chip, whatever, imposes a certain discipline that's lacking when you know you can change the source code at anytime. Being both, I'm not saying this is because hardware engineers are virtuous and software engineers are sloppy, but because engineers are human (at least some of them).
If you're not doing Xtreme Agile, you're not doing Agile right.
That is correct comrade. Failure is always caused by insufficient ideological zeal and purity.
So you're saying if a fellow team member is doing something in a way you think could be done better, you just stay silent? If a team member is planning to do something that you think isn't actually necessary because of something you're doing, you just stay silent? If a team member is struggling with a problem you have the skill set to help out with, you just stay silent?
It's true, for Agile to work, you need to have a team.
There is a difference between a team and trying to create a hive mind. If everybody has to talk about this crap every day, I fall asleep. We have a 0.5-1 hour meeting every week where everyone talks about what they've done, will do, what problems they face, and how they think they can fix it. That seems about right. It isn't necessary for everyone to go into excruciating detail about everything they do. Good people generally know when they've got things under control by themselves and when they have a big worries or need multiple people working on an issue. The daily confessionals seem like something out of 1984. In addition to the weeklies we have either impromptu or sometimes official meetings whenthings come up. We're also trying a new technology called "email".
"Proper software engineering" does work, its called "Systems Engineering", is well established and successfully used for large-scale mission-critical projects in almost every industry outside of IT - which seems to be blind to anything invented outside of IT.
Systems engineering has its own professional accreditation organization: http://www.incose.org/practice/whatissystemseng.aspx
True, but the problem is when the Great Plan becomes too rigid. It's necessary to change it as warranted. This actually seems to be easier in hardware, when you say "sure I could get that last 1dB of dynamic range you wanted, but it'll double the price and power consumption", or "yes we could put that function in module X as planned, but it'd be half the cost and complexity if we move it to module Y". Good systems engineers know this. Of course the headache of being a systems engineer is fighting with the implementers - half the changes or spec relaxations they ask for make sense, and the other half are about laziness or incompetence.
I do hardware and software in a 50/50 split, so I see both sides. Hardware is more easily understood to have physical constraints, but every thing in software that isn't exactly as planned is seen as a failure of the implementers.
So you've worked at Google? Beta is the goal.
"Proper software engineering" doesn't work.
You're right, but you're going to the other extreme. The problem with all methodologies, or processes, or whatever today's buzzword is, is that too many people want to practice them in their purest form. Excessive zeal in using any one approach is the enemy of getting things done.
On a sufficiently large project, some kind of upfront design is necessary. Spending too much time on it or going into too much detail is a waste though. Once you start to implement things, you'll see what was overlooked or why some things won't work as planned. If you insist on spinning back every little change to a monstrously detailed Master Design Document, you'll move at a snail's pace. As much as I hate the buzzword "design patterns", some pattern is highly desirable. Don't get bent out of shape though when someone has a good reason for occasionally breaking that pattern or, as you say, you'll wind up with 500 SLOC's to add 2+2 in the approved manner.
Lastly, I agree that there is no substitute for good engineers who actually talk to and work with each other. Also don't require that every 2 bit decision they make amongst themselves has to be cleared, or even communicated, to the highest levels. If you don't trust those people to make intelligent decisions (including about when things do have to be passed up) then you've either got the wrong people or a micromanagement fetish. Without good people you'll never get anything decent done, but with good people you still need some kind of organization.
The problem the article refers to about an upfront design being ironclad promises is tough. Some customers will work with you, and others will get their lawyers and "systems" people to waste your time complaining about every discrepancy, without regard to how important it is. Admittedly bad vendors will try and screw their customers with "that doesn't matter" to excuse every screw-up and bit of laziness. For that reason I much prefer working on in-house projects, where "sure we could do exactly what we planned" gets balanced with the cost and other tradeoffs.
The worst example of those problems is defense projects. As someone I used to work with said: In defense everything has to meet spec, but it doesn't have to work. In the commercial world specs are flexible, but it has to work.
If you've ever worked in that atmosphere you'll understand why every defense project costs a trillion dollars. There is absolutely no willingness to make tradeoffs as the design progresses and you find out what's practical and necessary and what's not. I'm not talking about meeting difficult requirements if they serve a purpose (that's what you're paid for) but being unwilling to compromise on any spec that somebody at the beginning of the project pulled out of their posterior and obviously doesn't need to be so stringent. An elephant is a mouse built to government specifications. Ok, you can get such things changed, but it requires 10 hours from program managers for every hour of engineering. Conversely, don't even think about offering a feature or capability that will be useful and easy to implement but is not in the spec. They'll just start writing additional specs to define it and screw you by insisting you meet those.
As you might imagine, I'm very happy to be back in the commercial world.
not an automatic "bring down" though.
It's wouldn't necessarily not bring it down either. Seagulls can bring down an airliner (only survivable if you have a pilot good enough to bring it down safely in the Hudson). You can always cite cases of people falling out of airplanes without parachutes and surviving, or an F-15 landing with one wing sheared off with one wing sheared off, but for the sake of safety you might want to avoid those things.
"OK, where or how do I get an Emacs phone?"
"You expect it to write itself?"
Actually... kinda... I mean if any piece of software's about to hit the singularity, I'd bet on Emacs!
Arrgh, read the mailing list - that feature won't be available until V26! (though there is a reason it's written in an AI language).
OK, where or how do I get an Emacs phone?
You expect it to write itself? Brush up on your Lisp.
EMACS was an OS before the web browser was even invented.
Managing this data in an appropriate and accountable fashion is officially Not Rocket Science.
Yes, but the fact that something can be done doesn't mean it is or will be done.
If you want to think for yourself, you should avoid any "isms" including, but not limited to, libertarianism, liberalism and conservatism.
That solves the problem of selective abuse by making it a universal abuse.
Good to know. Unfortunately the European Court of Human Rights has no jurisdiction in the US.
Whatever happened to the days when the US and the UK could rightfully pride themselves on having stronger individual rights than those closet non-constitutional monarchies on the continent?
The problem here isn't so much with the collection of DNA, but the retention.
Agreed, but note how that isn't addressed by the court, the press or the laws. As noted by posters above, even fingerprints are not usually deleted even if charges are dropped because a school bus full on nuns says "we saw the whole thing and he didn't do it". The best approach would be not only to delete the info for anyone who has charges dropped or is acquitted, but to change who analyzes the DNA. Because there have been cases of incompetent or corrupt police crime labs ("we know he's guilty so just say it matches"), the Innocence Project has suggested that DNA matching be done by N independent and accredited labs, with which lab a sample is sent to chosen at random. The samples should also be submitted and tracked in such a way that it doesn't indicate who or where the sample is from. Lastly, the labs should undergo random checks by having already known samples sent to them in exactly the same way as "real" samples are.
The hidden bomb shelter 10 feet underground ... That's protected by secrecy.
Security through obscurity is no security at all (especially if it's only 10 feet down - I know people with basements deeper than that.
For once the UK leads the USA in the long, slow slide to a police state.
IIRC in the UK there have also been questions about when the samples/information could be kept, at first for those who were acquitted or had charges dropped, and later for those who weren't even arrested but submitted DNA (as "requested" by the police) to exonerate themselves of any suspicion. The latter is sometimes done in blanket sweeps (we have reason to believe the perpetrator is a male living in Greater London is but a bit of an exaggeration).