Nice to know that I have so much of your attention, but that wasn't me. I'm not afraid to sign my criticisms of Freenet. BTW, you never did get back to me regarding my Freenet FIQ like you said you would. Guess you got "too busy" eh?
Frankly, "high latencies and frequent failures" are why such an idea is impractical, regardless of whether or not the theoretical problems can be solved (and i argue that they already have been solved).
Hm. So we have a set of "theoretical" problems, for which it's doubtful that solutions exist. Except that you say they've already been solved...and apparently they're not just theoretical either. Truly, you have a dizzying intellect.
Local disk space is far, far cheaper and more robust than network storage!
Cheaper, yes. More robust? For what value of "robust"? Are we talking about data that only exists in one place, or in multiple places? Which one's more resistant to the type of failure that takes out a whole site? Please provide a definition by which something that exists only on your machine (whose mere existence is only known locally) is more robust than something that exists in multiple places.
How long will it take to transmit a few dozen gig via DSL?
Irrelevant. In any but the most stupidly designed distributed data stores, most data would be served out of a local cache under most conditions. In many, the next step would be to serve it out of another geographically-local machine over a fast LAN connection. Just because you personally can't think of a distributed-storage architecture any better than traversing the globe for every datum doesn't mean that better architectures don't exist.
there is no reason to not use a user-level process to manage the data exchange
Really? Ever try to do mmap-style I/O over Napster? How about plain old open/read/write over Gnutella? Byte-range locking within a Freenet file? Hmmm. If you want to talk about solved problems, how about ideas like VFS layers and network-protocol abstractions? To provide generalized, transparent access to data, on a par semantically with the sort of access that you get with a local filesystem, your "user-level process" isn't going to cut it. Not by a long shot. That's like going back to the days when every application needed its own library just to get keyboard input or draw stuff on the screen. This kind of thing belongs, at least partially, inside the operating system so that all applications can use all equivalent protocols without special linkage; see my file-sharing manifesto for a fuller explanation.
This is two days in a row now that Slashdot has posted articles on the great new idea of distributed operating systems that CS theorists solved and have largely ignored for the last ten years. Besides Amoeba, there was the Connection Machine, VMS clusters, and others.
...none of which were designed to tolerate the high latencies and frequent failures that a truly Internet-scale OS would face. Legion and similar projects are much nearer the mark, but this is still nowhere near being the sort of "solved problem" you claim it is.
As happens too often, this proposal concentrates entirely too much on distributed computation, and pretty much ignores the problem of distributed storage. They're quite different problems, each requiring its own solution, even though it's intuitively obvious that any true "Internet Scale Operating System" would have to deal with both.
If you're interested in this "other half of the problem" here are some links:
Farsite (Microsoft; focus on many nodes, not long distances, but still relevant)
There are many more. The bibliographies for the above will mention many earlier systems, while a quick Google search for these project names will show more recent ones.
Those blocks would, in general, still be on your local hard drive...
Doesn't this kind of defeat the whole purpose of the scheme?
Not at all. The fact that they're present on your hard drive does not preclude their being in other places as well - in case your hard drive fails, or someone else nearer to one of those other places wants the data, etc.
So it only/maybe/ adds another 30s to Word's startup time
Incorrect. You're assuming that data cannot be in more than once place at a time, or cannot safely be so; either way, the assumption is Just Plain Wrong.
they're arguing that they are going to reduce the risk of "geographically localized faults" by distributing files on computers that are geographically localized
The contradiction is all in your mind, Sparky. Their target is neither a single machine room nor a global network. It's a campus like that of a university or large company (such as Microsoft itself), or a single metropolitan area. Machines in such a scenario might well be "interconnected by a high-bandwidth, low-latency, switched network" with "no significant geographical differences" and yet be immune to the single-meteor-strike scenario (unless the meteor happens to be enough to wipe out a whole metro area).
Imagine if the OS was storing my Word application files on another machine (using that great file coalescing scheme). Now it takes an extra 30s to a minute to start
You must have been asleep during the part about local caching. Those blocks would, in general, still be on your local hard drive and would be accessed at the same speed as they ever were. Sure, they'd be on a different part of the disk, and there'd be a tiny bit of extra metadata to check, but the performance impact would be negligible. This is a shared filesystem and, compared to sharing via protocols such as NFS or CIFS (which many people are happy with as a primary access mechanism), you'd still be way ahead of the game.
Maybe Farsite doesn't do what you want it to do, but that doesn't make it "ridiculous" or "laughable" as you claim. What do you do that's less ridiculous or laughable than that? Maybe you shouldn't be so quick to slag other people's projects just because you don't understand what it is that they're doing.
Microsoft keeps building on this idea that people will have unlimited bandwidth...Microsoft is on the wrong track
I once brought up this very same issue myself, only to find that it's already addressed in the Farsite FAQ:
We assume that the machines are interconnected by a high-bandwidth, low-latency, switched network. Also, at least for our initial version, we are assuming no significant geographical differences among machines.
In other words, it's not being designed for geographic-scale distribution. I have mixed feelings about that. On the one hand, it makes it less "interesting" to me than it could be, although there's still great research value even if it is limited to a fairly particular environment. On the other hand, if they were targeting geographically distributed environments they'd be competing more directly with me. I guess I shouldn't complain.;-)
BTW, the FAQ also anticipates several other questions that you and others here have raised. I won't say RTFM, but...well...no, I won't.
The idea of my files being stored on some Iowa farmhands computer does not sit well with me... ...The only real security is hardware security, which is to say, my files on my machine, your files on your machine
The first statement above makes perfect sense if you consider the second as axiomatic. However, the people working on these types of systems don't accept that axiom. Instead, they believe that cryptography-based security is just as strong as physical security...the odds that someone will factor a couple of hundred-digit numbers (or accomplish some equally difficult mathematical feat) are no higher than that they'll break into your home/office and steal your hardware. If they're right then there should be no problem with storing your files on some Iowa farmhand's computer (so long as you also have other replicas elsewhere for availability purposes), because Iowa Farmboy still can't access or modify your data without the right keys.
That's a big "if" you say. Well, yes it is. But if you want to make an argument that hardware security is the only real security, you'll need to show that cryptographically based systems aren't as secure as skilled and experienced implementors of such systems seem to think. Good luck.
Surely there will be major scalability problems with something like this, a la Gnutella
That's why it's research. I've met and talked to Bill Bolosky (Farsite project lead); he's very clueful wrt scalability in general, and well aware of the problems that networks like Gnutella (an unusually naive protocol, BTW) have run into. However, like the folks working on OceanStore or CFS or many other projects, the Farsite folks have a fairly formidable arsenal of innovative techniques they can apply to the problem. The details are still being worked out, of course, because that's what research is all about, but the people working in this area do seem to be making real progress toward solutions that could scale to such levels.
Some finer points in design; I see some stuff like this a lot as well:
function bob( varlist ) { $var = $joe + 12345; return $var; }
That's only a "fine point of design" to a 15-year-old. No, scratch that; it's not design at all. Any decent or even semi-decent compiler or interpreter should be able to make that particular optimization all by itself. A real fine point of design is whether to use events or threads, update or invalidate, distance vector or shortest path, this class hierarchy or that class hierarchy, this module layering or that module layering...stuff that can't be automated or even delegated to an inexperienced programmer.
It would be nice if the QA staff, who may have software programming skills, would be allowed to be developers as well (e.g. all the rights of a developer but QA is their main focus). ... in effect, have two programming teams.
Dream much? Ever hear of specialization? You're right that QA tends to get the short end of the stick in a lot of ways. QA engineers should have some programming experience, should attend (some) development meetings, should have more authority wrt the disposition of bugs...but they should not be checking in production code. Good QA is hard work, requiring its own specialized set of knowledge and skills. Any QA engineer who's making (and, one would hope, unit testing) their own changes to the production code is not going QA, and QA needs to get done. Hire another developer or extend the schedule, but don't take good QA engineers away from the necessary task that they do best to have them do someone else's job.
Inefficient code probably has other flaws, and if you're spending an unexpected amount of time in a single procedure it deserves a careful look.
Inefficient code probably has other flaws only because all code probably has flaws. Making code robust takes effort, and making code fase takes effort. The combined effort of trying to do both at once overwhelms many programmers, with the entirely predictable result that as the code improves on one axis it tends to get worse on the other. In other words, optimization is a good way to introduce bugs. That's not to say you should never optimize, but if your goal is to spend a month doing nothing but fix bugs then you shouldn't be anywhere near a profiler.
Then compare the number of open() and close(), the number of malloc() and free().
Check out Engler et al's meta-level compilation project as a way to automate this process. These tools were run on real code that had already been examined by thousands of skilled developers - the Linux kernel. The tools still found hundreds of real and potentially serious bugs. If it can be that effective for the Linux kernel, it can probably be used for just about any other project.
What exactly did you feel supported your claim in the presentation? I know there was a bit on people who do FS at their work but I was under the impression that it was _part_ of their work, not replacing their work.
Look at page 37 again. 46% said that they don't participate in free software at work. Another 34% said they did so with permission, mostly as part of their core job. That's the easy answer for people who want to waffle on the issue. The 19% who specifically said that they participated in free software at work without permission are the ones that interest me. Considering that most people communicate with their bosses, and most bosses are either sympathetic or indifferent to free software, these 19% are probably not the people you're thinking of. They're not the people counting on the "X% downtime is normal and it doesn't matter how you spend it" theory; they're deliberately hiding their free-software activity. And 19% is just the number that admitted it.
If (at least) one in five free-software programmers is willing not just to bend but actually to break rules and agreements like that, I think it's worth making a fuss over. Their behavior reflects on all free-software programmers and advocates, including the (maybe) 81% who are doing nothing wrong. The "good guys" owe it to themselves to create some distance between the altruism and idealism of free software vs. the selfishness and dishonesty of the time-stealers.
I have to disagree that individual who work on FS projects are likely to be less productive than those who don't.
That's not what I meant to say, and I'll try to clarify. You seemed to seemed to be stating a syllogism of the following form:
Premise 1: Employers expect and tolerate a certain amount of time at work being spent on non-work activities.
Premise 2: Time spent working on free software comes (at least partly) out of "goof-off" time rather than work time. Conclusion: People who work on free software at work do not have more total downtime.
I was not trying to prove that either the premises or the conclusions are false. What I was trying to show was that one does not necessarily follow from the other by describing a counterexample in which the premises were true but the conclusion was still false - which does not preclude other examples where all three are true, or where the conclusion is true even without the premises being true. Does that make things clearer?
I don't actually believe that free-software programmers are less productive or have more downtime than other programmers in general or on average. However, I do believe - and the survey seems to back this up - that a substantial percentage of free-software programmers do "steal time" from their employers by working on free software at work, without permission, to an extent or in a way that is actually detrimental to their employers' legitimate interests. This segment of the free-software community is IMO too large to be ignored or dismissed as irrelevant, and their actions cannot be justified by reference to other free-software programmers at other companies who have a better grip on ethics. People who feel that they're the targets of hyper-moral rants from people like RMS, ESR, or Bruce Perens deeply resent those same people's failure to address (or even acknowledge) this huge blot on free software's moral scorecard, and rightly so. Free-software advocacy will always be tainted with hypocrisy until its leaders take a firm stance regarding time theft.
Sorry. Got a little carried away there. I hope you see my point, though.
FS would likely seem to cut into the goof-time and not the productive time
OK, so programmer A has 30% downtime (from the employer's perspective) and spends all of it goofing off. Programmer B has 40% downtime, which could also be spent goofing off, but FS work cuts into the goof-off time so that it's only 20%. Less time goofing off, sure, but still a net loss for the employer. To the employer, working on non-work-related FS at work is exactly the same as goofing off. I still find it hard to believe that work time spent on FS is always balanced by an exactly equal or greater reduction in goofing off.
if most FS programmers have 11+ years experience, they have learned by know how to manage their time
The dark side of learning how to manage time is learning what you can get away with, but being able to get away with something isn't the same as it being right. In one sense, as long as a programmer is meeting deadlines and the employer feels they're getting good value for their money, everything's OK. That's not the same as saying that it involves zero cost to the employer. Besides the "lost time" there are issues of resource consumption, reductions in stability from running unapproved development version of software, risks of intellectual-property contamination, etc. These costs might not appear as separate line items on a budget, but they do contribute (negatively) to the bottom line. The expense of developing that "free" software is all too often borne by people who don't even know they're paying for it.
OTOH, most programmers are on much less explicit contracts. While that doesn't mean that they're free to work on OSS, at least mostly the contract doesn's include statements like "All IP rights are assigned to the employer"
I don't have any hard numbers - I'd love to find some - but my experience has been quite different. At every job I've worked at for the last several years, such IP assignment has been a standard part of the contract for all developers. You do raise a good point, though, that non-developers might not be subject to the same terms. On the third hand, the survey seemed to show that your hypothetical code-writing admin is a rarity.
Their supervisor knows only that they're keeping the servers up and runnning, not that the code they create to do their job is released as F/OS software.
OK, fair enough. Question: does it really make a difference that the employer gets to use free software created "on the sly" like this? Is it the writing, or the giving away, that's prohibited by the employee agreement? If some third party came and took the code, they'd be in big trouble. Why is it different because someone on the inside - who was not authorized to do so - gave it away?
If the biggest compliant of an FS programmer is that he spends some time at the office hacking away at a bug or scribbling some code on a notepad while the other goobers are talking about what they saw on Survivor, then that's a damn good thing
You're assuming that the FS programmer is doing their FS work instead of playing chess online or talking about the Patriots winning the Super Bowl. Is that a reasonable assumption? I'm not saying it's wrong, but I don't think we should accept uncritically the notion that FS programmers don't indulge in pure goof-off time like (or often with) their non-FS colleagues, in addition to the work time they use for their FS projects. Given the intense familiarity that many FS folks seem to have with every political current and gadget and video game and anime flick and window manager theme and filesharing program, such an assumption seems suspect at best. FS programmers do have non-FS interests, and there's no reason to suppose they don't indulge them at work like everyone else does.
I appreciate the fact that downtime at work is a fact of life for programmers. I have nearly 600 posts right here to demonstrate that appreciation, plus plenty of other time "wasted" in other ways. Maybe those FS programmers truly are "making something out of nothing" and their FS activities represent zero cost to their employers. Maybe doing FS at work is actually good, because it helps develop skills. I'd just rather know than assume.
On page 7, the authors make a distinction between three groups: "leadership", "virtual teams", and "user developers". Their selection methods seem to be skewed toward identifying many of the first group, somewhat less of the second, and relatively few of the third. I wonder - I really wonder - how their findings wrt motivation, experience level, or licit/illicit use of work time might be different if they'd managed to capture a more balanced cross-section of the three categories. Heck, it would help even to have an estimate in hand of the relative numbers of people in each category. At the very least, BCG should have asked in the survey which category the respondent felt best represented their own role in their open-source project(s).
I found the most interesting numbers to be on Page 37. There, 19% of respondents admitted that they were stealing time from their employers to write open-source software. Would anyone like to bet on what fraction of the 46% who answered "do not participate at work" were telling the truth, the whole truth, and nothing but the truth? What fraction of the "part of core job" respondents were in the "not known by supervisor" category at some point in their careers?
There are a lot of people who work on open-source software in their spare time. There are quite a few who get paid to do it. Bless them all. However, these numbers seem to indicate that at least 19% and probably much more than a third are regularly working on open-source projects while they're being paid to do something else. Maybe it's time to question whether the equation "open source == moral high ground" has any validity.
First is that routing in an ad-hoc network of that scale can be very difficult. People are working on it (see books by Charles Perkins or C-K Toh) but it's sorta not there yet.
The second problem, which exacerbates the first, is that battery power will likely continue to be an issue. The reason this matters is that it can make routing even more of a challenge, especially when nodes keep dropping out to conserve battery power. There are also issues with trying to run expensive algorithms - e.g. crypto - on slow power-constrained devices.
If you allow at least some of the devices in your system to be stationary (and therefore mains-powered) things become a lot more interesting. They key is not so much the wired/wireless nature of the network, but rather the number of nodes - more nodes generally means more opportunities to obscure who's sending and who's receiving what - and how the high-level protocols they're using above TCP/IP.
Stopping all new development for a month to fix bugs seems like a pretty serious step. It's much more than anyone involved in the Linux 2.4 development process ever did. And yet, people are still bitching about how it's not enough, security needs to be designed in, and miscellaneous other whines. OK, wise guys. Given the state of the real world with all of its history and constraints, what should a Microsoft that truly and sincerely had started to care about security do? Will anything satisfy you, short of the following press release signed in the blood of every Microsoft employee?
Here's all of our code, under GPL. You guys are geniuses, we were wrong, we're getting out of this business so we can do penance by wearing hair shirts and devoting our lives to clearing minefields without equipment or pay.
Dream on. Microsoft may suck, but they're making quite reasonable progress toward respectability. If everyone else made as serious an effort as they seem to be making, we'd be a lot better off even in free-software-land than we are today. Don't piss on them for trying to improve.
The fact of the matter is that most people do not want to give their credit card to fifty million different websites just to get some silly software.
The real fact of the matter is that shareware authors aren't set up to handle credit cards themselves, and distribute their software through outfits like DigiBuy or RegNow that handle all that stuff for them much like eBay acts as a broker for millions of their sellers. Your "difficulty of payment" argument is pure BS.
Many shareware authors absolutely refuse to take the time to offer support too.
The majority do offer support, and many will go the extra mile for users even though they can only devote "hobby time" to their projects. Your "lack of support" argument is BS for shareware just as much as it's BS for free software, and it's insulting a lot of decent people besides.
Software gets cracked not generally because of a public demand for it to be cracked, but rather because it presents a challenge for a potential cracker.
...who should find a hobby that doesn't involve taking revenue from someone else who was willing to take a risk by releasing something as shareware. Cracking erodes the entrepreneurial spirit, and crackers should be considered equivalent to leeches by anyone who believes in capitalism.
But how much does your purchase really help that process along? Product revenue is not really much of a factor right now; any company in this space is still getting the vast majority of their funds on a research/speculative basis. The only "point" to selling a product at all right now is:
To prove that all the manufacturing kinks have been ironed out, the product can pass whatever certification it needs, etc.
To get attention, which might attract either more research-stage funding or (much less likely) the attention of some company who will buy several thousand units based on some unique need (which will get even more attention, and so on.
My point is that individual purchases don't really help much with either of those. If you're serious about promoting a technology, there are better ways than to become part of a trivial revenue stream that probably costs the company more in infrastructure than they actually get out of it. Write your congressmen, donate to advocacy groups or relevant research labs, go to work for one of these companies, put banners on your website...any of these probably do more to promote the technology than actually buying anything. When the products reach the point that they offer a compelling value proposition compared to existing products based on earlier technologies, the dynamics will change, but buying what is still basically an inferior product (whatever promise it represents for the future) is IMO not very effective.
This is not meant as a flame or criticism. If you still feel that there's some value - even if it's just a philosophical point - in actually buying one of these, more power to you. Heh. I'm just trying to point out some reasons why it might not be any more than a gesture.
A lot of people don't seem to've noticed that this unit won't exactly be convenient to carry around. Their 25W prototype is 120x160x170mm (5"x6.5"x7") and 2.8kg (5lb)! That's less than a large desktop-equivalent laptop but almost double the weight of some lightweight models. I don't know how many road warriors will really want to triple their carry weight and pay extra money for a few extra hours of runtime. It will probably seem much more convenient and cost-effective to get one of those LiPoly external batteries or something.
That would indeed be an interesting comparison. Unfortunately, I'm not sure if the Intel compiler preserves enough gcc-specific behavior for that, since so much of Linux (especially the kernel) is written so that it doesn't even work properly with different versions of gcc. Of course, I'm absolutely certain that the code in Windows depends on an even bigger bucket full of language extensions, but then nobody claims that Microsoft is the champion of standards and compatibility so at least they're not being hypocritical about it.
Slashdot Mirror
Nice to know that I have so much of your attention, but that wasn't me. I'm not afraid to sign my criticisms of Freenet. BTW, you never did get back to me regarding my Freenet FIQ like you said you would. Guess you got "too busy" eh?
Hm. So we have a set of "theoretical" problems, for which it's doubtful that solutions exist. Except that you say they've already been solved...and apparently they're not just theoretical either. Truly, you have a dizzying intellect.
Cheaper, yes. More robust? For what value of "robust"? Are we talking about data that only exists in one place, or in multiple places? Which one's more resistant to the type of failure that takes out a whole site? Please provide a definition by which something that exists only on your machine (whose mere existence is only known locally) is more robust than something that exists in multiple places.
Irrelevant. In any but the most stupidly designed distributed data stores, most data would be served out of a local cache under most conditions. In many, the next step would be to serve it out of another geographically-local machine over a fast LAN connection. Just because you personally can't think of a distributed-storage architecture any better than traversing the globe for every datum doesn't mean that better architectures don't exist.
Really? Ever try to do mmap-style I/O over Napster? How about plain old open/read/write over Gnutella? Byte-range locking within a Freenet file? Hmmm. If you want to talk about solved problems, how about ideas like VFS layers and network-protocol abstractions? To provide generalized, transparent access to data, on a par semantically with the sort of access that you get with a local filesystem, your "user-level process" isn't going to cut it. Not by a long shot. That's like going back to the days when every application needed its own library just to get keyboard input or draw stuff on the screen. This kind of thing belongs, at least partially, inside the operating system so that all applications can use all equivalent protocols without special linkage; see my file-sharing manifesto for a fuller explanation.
...none of which were designed to tolerate the high latencies and frequent failures that a truly Internet-scale OS would face. Legion and similar projects are much nearer the mark, but this is still nowhere near being the sort of "solved problem" you claim it is.
As happens too often, this proposal concentrates entirely too much on distributed computation, and pretty much ignores the problem of distributed storage. They're quite different problems, each requiring its own solution, even though it's intuitively obvious that any true "Internet Scale Operating System" would have to deal with both.
If you're interested in this "other half of the problem" here are some links:
There are many more. The bibliographies for the above will mention many earlier systems, while a quick Google search for these project names will show more recent ones.
Not at all. The fact that they're present on your hard drive does not preclude their being in other places as well - in case your hard drive fails, or someone else nearer to one of those other places wants the data, etc.
Incorrect. You're assuming that data cannot be in more than once place at a time, or cannot safely be so; either way, the assumption is Just Plain Wrong.
The contradiction is all in your mind, Sparky. Their target is neither a single machine room nor a global network. It's a campus like that of a university or large company (such as Microsoft itself), or a single metropolitan area. Machines in such a scenario might well be "interconnected by a high-bandwidth, low-latency, switched network" with "no significant geographical differences" and yet be immune to the single-meteor-strike scenario (unless the meteor happens to be enough to wipe out a whole metro area).
You must have been asleep during the part about local caching. Those blocks would, in general, still be on your local hard drive and would be accessed at the same speed as they ever were. Sure, they'd be on a different part of the disk, and there'd be a tiny bit of extra metadata to check, but the performance impact would be negligible. This is a shared filesystem and, compared to sharing via protocols such as NFS or CIFS (which many people are happy with as a primary access mechanism), you'd still be way ahead of the game.
Maybe Farsite doesn't do what you want it to do, but that doesn't make it "ridiculous" or "laughable" as you claim. What do you do that's less ridiculous or laughable than that? Maybe you shouldn't be so quick to slag other people's projects just because you don't understand what it is that they're doing.
I once brought up this very same issue myself, only to find that it's already addressed in the Farsite FAQ:
In other words, it's not being designed for geographic-scale distribution. I have mixed feelings about that. On the one hand, it makes it less "interesting" to me than it could be, although there's still great research value even if it is limited to a fairly particular environment. On the other hand, if they were targeting geographically distributed environments they'd be competing more directly with me. I guess I shouldn't complain. ;-)
BTW, the FAQ also anticipates several other questions that you and others here have raised. I won't say RTFM, but...well...no, I won't.
The first statement above makes perfect sense if you consider the second as axiomatic. However, the people working on these types of systems don't accept that axiom. Instead, they believe that cryptography-based security is just as strong as physical security...the odds that someone will factor a couple of hundred-digit numbers (or accomplish some equally difficult mathematical feat) are no higher than that they'll break into your home/office and steal your hardware. If they're right then there should be no problem with storing your files on some Iowa farmhand's computer (so long as you also have other replicas elsewhere for availability purposes), because Iowa Farmboy still can't access or modify your data without the right keys.
That's a big "if" you say. Well, yes it is. But if you want to make an argument that hardware security is the only real security, you'll need to show that cryptographically based systems aren't as secure as skilled and experienced implementors of such systems seem to think. Good luck.
That's why it's research. I've met and talked to Bill Bolosky (Farsite project lead); he's very clueful wrt scalability in general, and well aware of the problems that networks like Gnutella (an unusually naive protocol, BTW) have run into. However, like the folks working on OceanStore or CFS or many other projects, the Farsite folks have a fairly formidable arsenal of innovative techniques they can apply to the problem. The details are still being worked out, of course, because that's what research is all about, but the people working in this area do seem to be making real progress toward solutions that could scale to such levels.
That's only a "fine point of design" to a 15-year-old. No, scratch that; it's not design at all. Any decent or even semi-decent compiler or interpreter should be able to make that particular optimization all by itself. A real fine point of design is whether to use events or threads, update or invalidate, distance vector or shortest path, this class hierarchy or that class hierarchy, this module layering or that module layering...stuff that can't be automated or even delegated to an inexperienced programmer.
Dream much? Ever hear of specialization? You're right that QA tends to get the short end of the stick in a lot of ways. QA engineers should have some programming experience, should attend (some) development meetings, should have more authority wrt the disposition of bugs...but they should not be checking in production code. Good QA is hard work, requiring its own specialized set of knowledge and skills. Any QA engineer who's making (and, one would hope, unit testing) their own changes to the production code is not going QA, and QA needs to get done. Hire another developer or extend the schedule, but don't take good QA engineers away from the necessary task that they do best to have them do someone else's job.
A good list, but...
Inefficient code probably has other flaws only because all code probably has flaws. Making code robust takes effort, and making code fase takes effort. The combined effort of trying to do both at once overwhelms many programmers, with the entirely predictable result that as the code improves on one axis it tends to get worse on the other. In other words, optimization is a good way to introduce bugs. That's not to say you should never optimize, but if your goal is to spend a month doing nothing but fix bugs then you shouldn't be anywhere near a profiler.
Check out Engler et al's meta-level compilation project as a way to automate this process. These tools were run on real code that had already been examined by thousands of skilled developers - the Linux kernel. The tools still found hundreds of real and potentially serious bugs. If it can be that effective for the Linux kernel, it can probably be used for just about any other project.
Look at page 37 again. 46% said that they don't participate in free software at work. Another 34% said they did so with permission, mostly as part of their core job. That's the easy answer for people who want to waffle on the issue. The 19% who specifically said that they participated in free software at work without permission are the ones that interest me. Considering that most people communicate with their bosses, and most bosses are either sympathetic or indifferent to free software, these 19% are probably not the people you're thinking of. They're not the people counting on the "X% downtime is normal and it doesn't matter how you spend it" theory; they're deliberately hiding their free-software activity. And 19% is just the number that admitted it.
If (at least) one in five free-software programmers is willing not just to bend but actually to break rules and agreements like that, I think it's worth making a fuss over. Their behavior reflects on all free-software programmers and advocates, including the (maybe) 81% who are doing nothing wrong. The "good guys" owe it to themselves to create some distance between the altruism and idealism of free software vs. the selfishness and dishonesty of the time-stealers.
That's not what I meant to say, and I'll try to clarify. You seemed to seemed to be stating a syllogism of the following form:
I was not trying to prove that either the premises or the conclusions are false. What I was trying to show was that one does not necessarily follow from the other by describing a counterexample in which the premises were true but the conclusion was still false - which does not preclude other examples where all three are true, or where the conclusion is true even without the premises being true. Does that make things clearer?
I don't actually believe that free-software programmers are less productive or have more downtime than other programmers in general or on average. However, I do believe - and the survey seems to back this up - that a substantial percentage of free-software programmers do "steal time" from their employers by working on free software at work, without permission, to an extent or in a way that is actually detrimental to their employers' legitimate interests. This segment of the free-software community is IMO too large to be ignored or dismissed as irrelevant, and their actions cannot be justified by reference to other free-software programmers at other companies who have a better grip on ethics. People who feel that they're the targets of hyper-moral rants from people like RMS, ESR, or Bruce Perens deeply resent those same people's failure to address (or even acknowledge) this huge blot on free software's moral scorecard, and rightly so. Free-software advocacy will always be tainted with hypocrisy until its leaders take a firm stance regarding time theft.
Sorry. Got a little carried away there. I hope you see my point, though.
OK, so programmer A has 30% downtime (from the employer's perspective) and spends all of it goofing off. Programmer B has 40% downtime, which could also be spent goofing off, but FS work cuts into the goof-off time so that it's only 20%. Less time goofing off, sure, but still a net loss for the employer. To the employer, working on non-work-related FS at work is exactly the same as goofing off. I still find it hard to believe that work time spent on FS is always balanced by an exactly equal or greater reduction in goofing off.
The dark side of learning how to manage time is learning what you can get away with, but being able to get away with something isn't the same as it being right. In one sense, as long as a programmer is meeting deadlines and the employer feels they're getting good value for their money, everything's OK. That's not the same as saying that it involves zero cost to the employer. Besides the "lost time" there are issues of resource consumption, reductions in stability from running unapproved development version of software, risks of intellectual-property contamination, etc. These costs might not appear as separate line items on a budget, but they do contribute (negatively) to the bottom line. The expense of developing that "free" software is all too often borne by people who don't even know they're paying for it.
I don't have any hard numbers - I'd love to find some - but my experience has been quite different. At every job I've worked at for the last several years, such IP assignment has been a standard part of the contract for all developers. You do raise a good point, though, that non-developers might not be subject to the same terms. On the third hand, the survey seemed to show that your hypothetical code-writing admin is a rarity.
OK, fair enough. Question: does it really make a difference that the employer gets to use free software created "on the sly" like this? Is it the writing, or the giving away, that's prohibited by the employee agreement? If some third party came and took the code, they'd be in big trouble. Why is it different because someone on the inside - who was not authorized to do so - gave it away?
You're assuming that the FS programmer is doing their FS work instead of playing chess online or talking about the Patriots winning the Super Bowl. Is that a reasonable assumption? I'm not saying it's wrong, but I don't think we should accept uncritically the notion that FS programmers don't indulge in pure goof-off time like (or often with) their non-FS colleagues, in addition to the work time they use for their FS projects. Given the intense familiarity that many FS folks seem to have with every political current and gadget and video game and anime flick and window manager theme and filesharing program, such an assumption seems suspect at best. FS programmers do have non-FS interests, and there's no reason to suppose they don't indulge them at work like everyone else does.
I appreciate the fact that downtime at work is a fact of life for programmers. I have nearly 600 posts right here to demonstrate that appreciation, plus plenty of other time "wasted" in other ways. Maybe those FS programmers truly are "making something out of nothing" and their FS activities represent zero cost to their employers. Maybe doing FS at work is actually good, because it helps develop skills. I'd just rather know than assume.
On page 7, the authors make a distinction between three groups: "leadership", "virtual teams", and "user developers". Their selection methods seem to be skewed toward identifying many of the first group, somewhat less of the second, and relatively few of the third. I wonder - I really wonder - how their findings wrt motivation, experience level, or licit/illicit use of work time might be different if they'd managed to capture a more balanced cross-section of the three categories. Heck, it would help even to have an estimate in hand of the relative numbers of people in each category. At the very least, BCG should have asked in the survey which category the respondent felt best represented their own role in their open-source project(s).
I found the most interesting numbers to be on Page 37. There, 19% of respondents admitted that they were stealing time from their employers to write open-source software. Would anyone like to bet on what fraction of the 46% who answered "do not participate at work" were telling the truth, the whole truth, and nothing but the truth? What fraction of the "part of core job" respondents were in the "not known by supervisor" category at some point in their careers?
There are a lot of people who work on open-source software in their spare time. There are quite a few who get paid to do it. Bless them all. However, these numbers seem to indicate that at least 19% and probably much more than a third are regularly working on open-source projects while they're being paid to do something else. Maybe it's time to question whether the equation "open source == moral high ground" has any validity.
First is that routing in an ad-hoc network of that scale can be very difficult. People are working on it (see books by Charles Perkins or C-K Toh) but it's sorta not there yet.
The second problem, which exacerbates the first, is that battery power will likely continue to be an issue. The reason this matters is that it can make routing even more of a challenge, especially when nodes keep dropping out to conserve battery power. There are also issues with trying to run expensive algorithms - e.g. crypto - on slow power-constrained devices.
If you allow at least some of the devices in your system to be stationary (and therefore mains-powered) things become a lot more interesting. They key is not so much the wired/wireless nature of the network, but rather the number of nodes - more nodes generally means more opportunities to obscure who's sending and who's receiving what - and how the high-level protocols they're using above TCP/IP.
Stopping all new development for a month to fix bugs seems like a pretty serious step. It's much more than anyone involved in the Linux 2.4 development process ever did. And yet, people are still bitching about how it's not enough, security needs to be designed in, and miscellaneous other whines. OK, wise guys. Given the state of the real world with all of its history and constraints, what should a Microsoft that truly and sincerely had started to care about security do? Will anything satisfy you, short of the following press release signed in the blood of every Microsoft employee?
Dream on. Microsoft may suck, but they're making quite reasonable progress toward respectability. If everyone else made as serious an effort as they seem to be making, we'd be a lot better off even in free-software-land than we are today. Don't piss on them for trying to improve.
The real fact of the matter is that shareware authors aren't set up to handle credit cards themselves, and distribute their software through outfits like DigiBuy or RegNow that handle all that stuff for them much like eBay acts as a broker for millions of their sellers. Your "difficulty of payment" argument is pure BS.
The majority do offer support, and many will go the extra mile for users even though they can only devote "hobby time" to their projects. Your "lack of support" argument is BS for shareware just as much as it's BS for free software, and it's insulting a lot of decent people besides.
...who should find a hobby that doesn't involve taking revenue from someone else who was willing to take a risk by releasing something as shareware. Cracking erodes the entrepreneurial spirit, and crackers should be considered equivalent to leeches by anyone who believes in capitalism.
But how much does your purchase really help that process along? Product revenue is not really much of a factor right now; any company in this space is still getting the vast majority of their funds on a research/speculative basis. The only "point" to selling a product at all right now is:
My point is that individual purchases don't really help much with either of those. If you're serious about promoting a technology, there are better ways than to become part of a trivial revenue stream that probably costs the company more in infrastructure than they actually get out of it. Write your congressmen, donate to advocacy groups or relevant research labs, go to work for one of these companies, put banners on your website...any of these probably do more to promote the technology than actually buying anything. When the products reach the point that they offer a compelling value proposition compared to existing products based on earlier technologies, the dynamics will change, but buying what is still basically an inferior product (whatever promise it represents for the future) is IMO not very effective.
This is not meant as a flame or criticism. If you still feel that there's some value - even if it's just a philosophical point - in actually buying one of these, more power to you. Heh. I'm just trying to point out some reasons why it might not be any more than a gesture.
A lot of people don't seem to've noticed that this unit won't exactly be convenient to carry around. Their 25W prototype is 120x160x170mm (5"x6.5"x7") and 2.8kg (5lb)! That's less than a large desktop-equivalent laptop but almost double the weight of some lightweight models. I don't know how many road warriors will really want to triple their carry weight and pay extra money for a few extra hours of runtime. It will probably seem much more convenient and cost-effective to get one of those LiPoly external batteries or something.
That would indeed be an interesting comparison. Unfortunately, I'm not sure if the Intel compiler preserves enough gcc-specific behavior for that, since so much of Linux (especially the kernel) is written so that it doesn't even work properly with different versions of gcc. Of course, I'm absolutely certain that the code in Windows depends on an even bigger bucket full of language extensions, but then nobody claims that Microsoft is the champion of standards and compatibility so at least they're not being hypocritical about it.