back away from the distinction again and again. It's not programming vs not-programming, and it's not about building stuff in meatspace vs. building stuff in memory.
It's about public vs. private effects and risks. People who build stuff for backyards = not engineers. People who build stuff for the public square = engineers. People who build a bridge over their own stream themselves = not engineers. People who build a bridge over a public byway for heavy public use = engineers.
The same ought to be true in codingland. If you are building critical public infrastructure or software that many people must use in order to participate in society, then you should be licensed and bonded and held to higher standards. If you are building software that is for personal, private use, then this is not the case.
Note that I am talking about instance of execution, not instance of code. For example, MS Windows would not be "engineering" code because each instance of execution is private, while the code inside, say, a traffic lights system, aircraft control system, or automobile is "engineering" code because these instances of execution happen in public, with public effects. (If MS Windows was to be used to run, say, a battleship, then it ought to be a separate "engineered" version held to higher standards of scrutiny, professionalism, and liability.)
We don't certify and license people as engineers because they might touch wood and steel we think wood and steel are special or important, we certify and license people as engineers whose single instances of wood and steel construction will each touch (and possibly put at risk) many lives.
He's trying to say that if people want powerful, flexible networking, they'll choose an 80% safe OS that enables this easily over a 90% safe OS that imposes lots of overhead costs to make it possible; that people will choose a 60% secure OS that runs their processing jobs in 3 hours over an 85% secure OS that runs their processing jobs in 6 hours.
He's pointing out that people like security well enough, but they want to get stuff DONE even more, and that most people will take the calculated risk to be less secure if it makes them more productive at lower costs. That if there is a less secure but more productive option, up to some arbitrary point (that is different in each case, but that can be inferred by the movement of markets and communities as a whole), they'll choose the more productive option.
And that there is no point in saying "then all of us that produce these things must get together and make highly secure, if less capable stuff, so that all choices are equally highly secure!" because as soon as that happens, a garage coder somewhere is going to have a project on github that says "I got tired of waiting for jobs to finish, so I wrote my own from scratch. It's totally insecure, but damned if it doesn't finish the job in half the time!" and that people will immediately flock to it.
In other words, his goals for Linux aren't for Linux to be the most secure OS on the planet, but to be one of the most useful and used ones.
We are talking about securing tools. But the point is that tools do things. We want tools to help us to accomplish the things that the tools do.
A perfectly safe hammer is entirely possible. Make it out of flame-resistant, soft, synthetic materials and fill it with something equally soft. Shape it more like a ball than like a stick, so no-one can accidentally stick it in their mouth and suffocate.
Of course, now you have something that can't be used to pound in nails—but it's entirely the safest hammer on the planet.
Will anyone buy it or use it? Of course not. And they'll still need something with which to pound in nails. That's Linus' point.
This is not an easy "but." Most cases generate aggregate billing across all facets of the case at a rate of five or six figures monthly, and it is already easy to make the claim that the finance limits in every case are the most direct limits on just outcomes.
Who is going to pay the significant costs? And at the expense of what other part of the case that might have had them instead? There are many victims and defendants that would argue that the first thing that ought to be paid for is increased time for the prosecutor or defense attorney in a case to pay attention to the case, since in most instances the parties feel as though the attention of their legal representatives is being catastrophically rationed.
And what about other tests or forms of forensic expertise? Is it better to pay for two tests or, if funds exist, to pay for one of the other forms of corroborating expertise and testing that are otherwise unavailable due to cost constraints?
Access to truth and law are significantly cost-constrained on both sides. Adding one additional confirming test does little to change this, and just as importantly, it's not clear—given this constraint—just how even this one more test will be paid for without drawing resources from some other dimension of a given case that could be equally important in a system in which it's impossible to predict just which dimensions of a case will be definitive until case post-mortems and debriefings are being carried out.
The civil justice system is just as big a mess, and for the same reason as TFA implies. Strangely absent from both systems are the victims and the accused, the petitioners and the respondents, the actual citizens that the system purportedly serves.
Instead, both parties touch the system only at its very edge. Their actual cases happen more or less without them, winding their way through an impossibly large-scale system full of paid actors whose jobs, practically speaking, at the level of everyday experience, are not to think about the parties in the case, but to perform the same particularistic tasks day after day in massive volumes before handing things off to other paid actors in a massive division of labor.
It's a kind of assembly line or factory for legal activity and paperwork production. The complete details of any single case, civil or criminal, are not known by anyone within the system—even the judges, commissioners, and magistrates that hear them—even though the actual parties to the case know their own stories inside and out. There is no facility or room within the system for its paid actors to actually get to know a case, through either party's eyes. Instead, each professional focuses only on the tiny fragment of each case that they are responsible for before handing it off to the next professional.
There is essentially no oversight for any part of the system, and even if there was, plausible deniability is huge, since each professional knows and interacts with only the tiniest part of each case, yet most legal statues offer recourse only if poor or unprofessional practice are more likely than not to have actually altered the final outcomes of this division of labor involving many months, dozens or hundreds of specialists, and a significant degree of uncertainty due to the vagaries of interaction and logistics.
It is a forest-for-trees problem to the Nth power, but it's difficult to see any way to address it; to be just, the law needs to be well-documented, clear and explicit, and to have nuance and detail. This necessarily makes it large and complex. That implies the need for professionals that have been trained in it. But a professional that dedicates their life to law must be able to make a living. Most individuals cannot afford to pay an entire salary to a legal professional, much less the many that must work on a given case due to the complexity of the law, and thus, they cannot expect these professionals to dedicate themselves to a single case. Instead, the costs of the professionals' salaries must be shared amongst literally many thousands of victims, accused, petitioners, and respondents, meaning that the professionals must limit their consumption of case details to just those in which they specialize, or face mountains of information with which they can't possibly cope and the consumption of which would impact their ability to do the job in which they specialize.
As a result, for the average citizen, bringing a case or participating in a case is like playing a giant, almost comically huge game of Plinko. The case enters at one end of the machine and knocks about between pegs endlessly and seemingly at random, well out of their reach, for what seems like ages, while they stand by, breathless and helpless. At the end, the case exits somewhere, with some sort of decision, but the relationship between its final disposition and its initial circumstances are completely unpredictable and due to the nature of the machine, and it's difficult to argue that any part of the game machine is "broken" most of the time—a peg in the machine has to be severely affected (i.e. missing, malformed, completely bent) for such a claim to be viable. Minor variances throughout may influence outcomes for a very long time without being detectable, even under scrutiny. And for the most part, there is no budget, much less any avenue even for the funding and organization of a program of scrutiny.
The institutions themselves are benefitting in revenue terms through bookstore sales, which also benefit publishers significantly in a kind of win-win.
I made a post about my years as a professor below and about being fired for allowing students to work from used textbooks.
What I didn't post below was that in the early '00s, before I was a professor, I worked in a well-known academic publisher of textbooks and journals in the Los Angeles area. I was over a department / topic area and one of the things that we did to stay ahead of revenue neutrality in our publications was ensure that they were "updated" every year. In many cases, preparation for the new "edition" entailed hiring two independent contractors: one freelance photo editor to replace all the images, and another freelance academic (often at a total cost of $1k-$2k tops) to re-do the exercises, tweak a few chapter titles, and perhaps reorder some chapters.
This was a strategy to enable "stale" books (read: books with large presences in used channels) to be revenue positive again (new edition = now more books in used channels, meaning a rash of new sales for 2-4 semesters). The initial investment in the text had in many cases happened years ago; subsequent annual investments were often in the low four figures or even less.
Of course, this high-margin model also enabled us to do deals with universities and their bookstores. Because of the low overhead for many refreshed titles, we could offer favorable terms to them for their revenue generation, often demanding minimum buys or various kinds of exclusivity in exchange for better revenue terms.
It's a kind of wealth transfer from the taxpayers (as student loans), through students, into the pockets of publishers and institutional administrations.
I'll preface this by saying that I was not tenured faculty. But I was adjunct faculty with a thriving career outside of the university and seven years as a part-time faculty member.
This happened to me at local State U (I'm in a flyover state) and ended my years as a professor. I was a top-rated instructor in the department by both student evaluations and faculty observations, advising graduate students, experienced, and had been there a long time teaching courses that I developed and that were well-received.
New leadership came in at the divisional level, and I was called in to a meeting with my chair one day. I was told I could no longer do what I had been doing for at least half a decade: assigning a textbook that was several editions old (there were no substantive changes in the newer editions, just replaced photos) and instructing students on the syllabus to pick the books up for literally pennies on Amazon.com, Alibris, eBay, or other online venues.
Instead, I had to assign the latest issue of the textbook and do it only through the university bookstore, at a cost of >$150.00 in one class, >$200.00 in another (compared to an average of $4.00 plus shipping most semesters for the online used versions). I had it listed as my first assignment on each syllabus—buy a used textbook online and submit proof of purchase (to be sure the students actually did get ahold of the textbooks).
I refused. I said I would provide both options—I'd order the textbook through the university bookstore and provide that as an option to students that preferred to buy new, through the bookstore, but would also allow both current and old editions to be used in my classes for students that wanted to rely on used books. I was threatened again. New only, bookstore only.
I refused. I was fired.
That semester (in 2014) was the last time I set foot on a college campus as a professor, after nearly a decade in the classroom every semester. Again, I wasn't tenured—but it left a significant hole in curriculum and advising. They were more interested in ensuring that students contributed to revenue and partnerships through bookstore purchases than they were in actually enabling students to learn in a cost-effective way.
As an employee, I can just as easily say that when I've worked at jobs at which I was compensated well, knew I could be rewarded even more, and certainly didn't want to lose the position no matter what, I worked my tail off and held myself to very high standards.
When I knew that I could easily get another job at the same level of compensation, and that no matter how hard I worked it was unlikely to be rewarded with more pay, I put more of my time and energy into other avenues, like improving my skill sets and networking outside the company, so that I could transition elsewhere—as well as into simple quality of life stuff, since I knew that work wasn't going to enhance my quality of life all that much.
So you're saying employees work less well = pay them less well.
I'm saying pay them more = they'll work more and better.
Chicken and egg. Which comes first? The rewards or the pay? You make one argument. I'll make another.
Why are employees not as great as they could be? I'll go out on a limb and say it's because you're not motivating them to be as great as they could be, either through great leadership (which is rare), a great environment (slightly more common), or great compensation.
And if you get great leadership, a great environment, *and* great compensation, I'm betting you'll somehow magically find that you have the best employees on the planet, who would do almost anything for the company—and do it at a very high level.
I am also in my 40s and have a huge library of books (including a roomful of books on shelves from my Ph.D. years). And at this point, I can't *stand* paper books. They're heavy, have slow page turns, are not searchable, can only be carried in small numbers, are difficult to use (no changeable font, low contrast, drop it and you've lost your page), take FOREVER to find (Not at the bookstore? And let's face it, what's at the bookstore any longer? Then you'll have to wait days for the book to arrive in the mail, no impulse buying/reading), use up space in your house, and so on.
I am basically ebooks only these days. I buy and read probably 3-6 ebooks a week. If it's not available electronically? I've probably bought four paper books over the past year, if that. I have to really, really want it to put up with paper and the inconveniences of buying/reading paper.
not just for the question of how a contributor should be treated, but also for the question of how a leader should act.
If a leader is able to get world-beating results by being an asshole, then so be it. That leader has beaten the world, and I am not going to quibble with success. If a leader is an asshole and subpar output is the result, then by all means, tell them to treat their team differently.
Team dynamics are a complicated thing. You just don't fuck with a winning team. If they are using four letter words all the time and sacrificing live chickens at midnight, but the results are running circles around everyone else, I for one do not want them to stop, even if it would save a chicken's life.
At the same time, if they are doing all of these things and the results are uneven or poor, then by all means, change the behavior.
In this case, I'd say that the results of Linux kernel development speak for themselves. And if you just don't belong in the culture, then go somewhere else. If the culture starts to be counterproductive, give the world a great, big "I told you so!" and collect your profits on the book deal. But otherwise, to expect people to fuck up a successful operation for your feelings, for manners, or for high-minded ethics concerns is just bad juju. It's not lawyering or doctoring, ethical concerns are not front and center. It's software. The goal is that it works and works well, and in fact that's the highest ethical aspiration *of* software, given the many critical ways in which it gets used in today's world.
The value to the users is first. The comfort of the developers is second. If the culture and development process are working well, get the hell out of the way if you don't like them. As this person has done. So—problem solved.
The first thing I wanted after installing El Capitan was information on how to disable rootless mode, not advice about the soundness of this idea. Thankfully, I found an informative post or two by searching Google. Any advice would not have been the information I was looking for.
And for those that are interested, yes, there is actually a.conf file that controls the rootless mode protections. I forget the path, but if you Google, you'll find it. The catch of course is that you have to disable rootless mode in order to edit it, and each time you want to edit it, which means multiple reboots for each edit.
Given the fact that I use software from across the 'net on my Mac, much of it not Apple developer signed and some of it development oriented, I figured I'd likely encounter problems along the way by trying to edit the.conf file and would have to keep banging on and editing it over time. And I've been using Macs and Mac OS for years already without rootless mode, so I don't feel too catastrophically ba about not having it now.
Problem (1): I found out very quickly that root has been neutered; you can't make any changes to "system" files (in this case, meaning files that were included in the OS distribution, including things like the Mail.app folder or binaries, etc.). You get a message about not having permission, despite being root, and without any extended attributes being set on the files. Turns out that El Capitan uses a new "rootless" model in which root is no longer root and many parts of the system are off limits to any human user. Solution: Boot into recovery mode, start a terminal, and enter the command "csrutil disable" then reboot. You'll get root back and will be able to change files again.
Problem (2): Parts of Office 2011 didn't work at all—just beach balled upon startup. I tried to figure this out for a while but didn't see anyone else talking about solutions online, so I installed Office Mac 2016 (since I'm already paying for Office 365 anyway so that I can use it on my tablet and phone). I've been using the Office Mac 2016 applications all day (Outlook, Word, and Excel for work) heavily, without any trouble, so as a data sample of one I can say that in my case, 2016 is definitely a better bet on El Capitan than 2011, since Word and Outlook 2011 didn't work at all.
I guess I tie the idea of "malware" to two concepts:
1) Mal, as in harmful to the user. 2) Ware, as in software.
To me, "malware" as a concept is basically about end users. It is software that is installed by endusers that does something contrary to what they expect, possibly without their knowledge, that is harmful to them. Malware is inherently deceptive, and the method of its deception is posing as something else. It is directed from bad actors toward strangers that these actors wish to exploit. It is a numbers game, a volume game.
This was not installed by end users, it did not pose as something else, and the harm was directed at an organization by individuals within the organization. It was not distributed widely, but was a single instance. I'd call this a "hack" or a "sabotage" or an "embezzlement" of some kind before I'd call it malware. Maybe a new term is needed.
But it seems a big jump from the widespread distribution of a Windows wizard to millions of hapless end users all the way to the willful and direct modification of company equipment by employees for gain.
All of the following hardware from Apple has been absolutely groundbreaking/pathbreaking. When it came to market, there was nothing else like it:
- The original Mac 128k - The Apple Newton - The iMac - The iPod - The iPhone - The iPad
Complaining that the Mac couldn't be expanded is like complaining that you can't rebore the cylinders on your Tesla to get more horsepower. These were products that changed users' understandings of the product space in question.
As a young geek, I cut my teeth on multiple computing systems. Three were old 8-bit systems: a C64, a TRS-80 CoCo2, and an Apple II. One was a Mac 128k. They were not even the same kinds of products. To call them all simply "computers" is ridiculous.
The same thing goes for:
Newton vs. other embedded "tablets" of the era (Fujitsu, IBM, GRiD, and others) iMac vs. white-box PC iPod vs. previous MP3 players or digital MiniDisc players iPhone vs. previous smartphones like Treo iPad vs. Windows CE "Handheld PC Pro" tablets
I can remember when everyone was making fun of the iPad for not including a stylus, a CF card slot, or a removable battery. And some geeks here continue to try to pretend that it was only a matter of advertising and image that caused consumers to gobble iPads up, and eventually, other companies (the entire Android field, for example) to essentially throw away any previous work and design to iPad specs and form factor.
But go ahead. Try it. Get someone a Viewsonic or Fujitsu tablet from 2009 and then hand them an iPad 1 and ask them which they prefer. Consumers aren't as stupid as people here make them out to be. They care about their hard-earned dollars just like everyone else. And they had no problem deciding that stylus-based resistive tablets with two hour batteries that were an inch and a half thick and two pounds (to accomodate replaceable batteries and removable storage) and that ran Windows... were not something they wanted to spend their money on.
Everyone here is quick to call it all "bullshit." Yet how many here own and use a tablet and/or a smartphone with a multitouch display and a purpose-specific operating system with a touch-oriented user interface? A lot, I'd bet, including many who mock such things as they peck out their mockings on an on-screen keyboard (which they also mock). Who brought these things from the halls of CERN to consumer electronics, not thinking that it was an impossiblility?
that there is a difference between "marketing," "advertising," and "sales."
Marketing is tremendously important *at the stage of product design.* Marketing, when done profitably, means *understanding your market* (i.e. users) and what they need (which may or may not be what they think they want) and then ensuring that your engineers get wind of that need and design to it.
Good marketing is an integral part of good tech, and happens well before any advertising takes place. The best products may or may not sell themselves (I think there's a good argument to be made that people need to actually know about a product, and it needs to be available in channels, before they are able to realize that it exists and buy it), but the only way to *get* to the "best products" for a large audience is to have a very good marketing division helping engineering to understand just what "best" means for a large, diverse userbase.
Yes, Apple has been very good at marketing over the last two decades. This skill is inseparable from their ability to design, and the fact of their having designed and taken to mass production, rather good hardware that is in high demand.
becuase they're geeks and (understandably, self-servingly) want to point how how central and important geeks are to, say, computing and technology hardware and software development, design, and production.
But it is one of the rarest geniuses on earth to be able to conduct a group of people to produce to their maximum potential, to be able to somehow lead talent to actually produce what the talent is capable of as a group and to do things that everyone else wants to do, but everyone else also falls short of time and time again.
The founding of Apple was really far less miraculous than the turnaround, when Jobs was able to get a huge bureaucracy to start making really high-quality, completely realized products without the significant compromises that everyone else took for granted. I'm typing this on a Macbook Pro right now. For many years I used Thinkpads. There is a difference in the aesthetic, as is so often pointed out, but it's a difference that in the aesthetic of functionalism that has to be realized through design, logistics, manufacturing, etc. involving teams of many very smart people. The Macbook Pro isn't perfect, but it's a far superior machine to the Thinkpads I used to use, not because it's faster or has more features but because it has fewer flaws and compromises; it represents something far closer to a fully realized idea and goal.
The same thing goes for smartphones and tablets. I used to carry around Treos in the early 2000s. I used them heavily. They were my go-to tools. I wrote a book on a Treo, no kidding, riding on the subway every morning, that's still generating me about $20 in royalties a year (big money, heh). But I used them. The same thing for my Windows CE tablets, first a Vadem Clio and later a ViewSonic something-or-other. But they were exercises in taken-for-granted compromises. They were "as good as it gets," it takes a big company to design and make such things, and the end products, though flawed, were the best that could be accomplished. They were "hard problems" and "best-case solutions" as products. They worked well.
Or so everyone thought.
And then? iPhone. And iPad. And they set an entirely new bar and benchmark for their respective industries. The previous products were obsolete in a moment and everyone has struggled to catch up. Tim Cook has not been able to replicate this precisely because he does not have the particular genius that Steve Jobs had. That's not to say that other people inside Apple don't also have genuis of many varieties. Half of the people on Slashdot (okay, not half, but some) are probably geniuses in their own right, in algorithms, or some area of hardware engineering, or whatever.
That doesn't take away from the fact that Steve Jobs was a rare genius in management and leadership. He was the opposite of the pointy-haired boss. We make fun of the pointy-haired boss precisely because we realize that it is the norm. Jobs was not the clueless leader; he was the leader that always somehow managed to get it right and squeeze more great, historic, memorable, and compromise-free stuff out of the geniuses at his company, by far, than the vast majority of other leaders—even the highly regarded, very well paid ones—are able to ever come close to getting out of the geniuses at their own companies.
That's not nothing. And given the multiplier affect of getting the best out of many geniuses, it's quite a lot.
I didn't know that, though based on what I do know about the jpeg format, it makes a kind of sense that this would be possible. Thanks for posting this, great nugget of information!
what consumers had access to by walking into a retail computer dealership (there were many independent white box makers at the time) and saying "give me your best."
You're probably right about me underestimating the graphics, though it's hard to remember back that far. I'm thinking 800x600 was much more common. If you could get 1024x768, it was usually interlaced (i.e. "auto-headache") and rare if I remember correctly to be able to get with 24-bit color—S3's first 16-bit capable chips didn't come out until late-1991, if I remember correctly, though I could be off.
SCSI was possible, but almost unheard of as stock, you either had to buy an add-on card and deal with driver/compatibility questions or one of the ESDISCSI bridge boards or similar. Same thing with ethernet, token, or any other dedicated networking hardware and stack. Most systems shipped with a dial-up "faxmodem" at the time, and users were stuck using Winsock on Windows 3.1. It was nontrivial to get it working. Most of the time, there was no real "networking" or "networking" support in the delivered hardware/software platform; faxmodems were largely used for dumb point-to-point connections using dial-up terminal emulator software.
And in the PC space, the higher-end you went, the less you were able to actually use the hardware for anything typical. Unless you were a corporate buyer, you bought your base platform as a whitebox, then added specialized hardware matched with specialized software in a kind of 1:1 correspondence—if you needed to perform task X, you'd buy hardware Y and software Z, and they'd essentially be useful only for task X, or maybe for task X1, X2, and X3, but certainly not much else—the same is even true for memory itself. Don't forget this is pre-Windows95, when most everyone was using Win16 on DOS. We can discuss OS/2, etc., but that again starts to get into the realm of purpose-specific and exotic computing in the PC space. There were, as I understand, a few verrry exotic 486 multiprocessors produced, but I've never even heard of a manufacturer and make/model for these—only the rumor that it was possible—so I doubt they ever made it into sales channels of any kind. My suspicion (correct me if I'm wrong) was that they were engineered for particular clients and particular roles by just one or two orgnaizations, and delivered in very small quantities; I'm not aware of any PC software in 1992 timeframe that was even multiprocessor-aware, or any standard to which it could have been coded. The Pentium processor wasn't introduced until '93 and the Pentium Pro with GTL+ and SMP capabilities didn't arrive until 1995. Even in 1995, most everything was either Win16 or 8- or 16-bit code backward compatible to the PC/XT or earlier, and would remain that way until around the Win98 era.
The UNIX platforms were standardized around SCSI, ethernet, big memory access, high-resolution graphics, and multiprocessing and presented an integrated environment in which a regular developer with a readily available compiler could take advantage of it all without particularly unusual or exotic (for that space) tactics.
The 386 box that I installed Linux on my first time around was 4MB (4x1MB 30-pin SIMMs). 4MB! I mean, holy god, that's tiny. It seemed sooooo big compared to the 640kb of 8-bit PCs, and yet it's basically the same order of magnitude. Not even enough to load a single JPG snapshot from a camera phone these days.
For more than just a couple of us here, I suspect, there was a time when "Sparc," "UNIX," "graphics," "Internet," and "science" were all nearly synonymous terms.
Simpler times. Boy did that hardware last and last and last in comparison to the hardware of today.
Well, I suppose it can finally no longer be said that the Sparcstation 10 I keep here just for old times' sake can still run "current Linux distributions." But it's still fun to pull it out for people, show them hundreds of megabytes of RAM, 1152x900 24-bit graphics, gigabytes of storage, multiple ethernet channels, and multiple processors, running Firefox happily, and tell them it dates to 1992, when high-end PCs were shipping with mayyybe 16-32GB RAM, a single 486 processor, 640x480x16 graphics, a few dozen megabytes of storage, and no networking.
It helps people to get a handle on how it was possible to develop the internet and do so much of the science that came out of that period—and why even though I don't know every latest hot language, the late '80s/early '90s computer science program that I went to (entirely UNIX-based, all homework done using the CLI, vi, and gcc, emphasis on theory, classic data structures, and variously networked/parallelized environments, with labs of Sparc and 88k hardware all on a massive campus network) seems to have prepared me for today's real-world needs better than the programs they went to, with lots of Dell boxes running Windows-based Java IDEs.
here are my answers. Spreadsheets are used in several cases:
1) When you have a small-to-medium-sized dataset (100m data points) and want to do a particular set of calculations or draw a particular set of conclusions from it just once or twice—so that the time invested in writing code in R or something similar is less than the time needed just to bung a few formulas into a spreadsheet and get your results. Once you get into analyses or processes that will be repeated many times, it makes more sense to write code.
2) Similar case, when you need to work with essentially tabular database data, but the operations you're performing (basic filtering, extracting records based on one or two criteria, just handing data from one person to the next) are either so simple or will be repeated so rarely that a MySQL database is overkill and just emailing a file back and forth is easier.
3) When you are working with data as a part of a team, and certain members of the team that are specialists in some areas related to the data, or (for example) members of the team that are doing your data collections, aren't particularly computationally expert. Spreadsheets are hard for laymen, but it's doable—a dozen or two hours of training and people can get a general, flexible grasp of spreadsheets and formulae. It takes a lot longer for someone to become basically proficient with R, MATLAB, MySQL, Python, etc., and you really want those specialists to just be able to do what they do to or with the data, rather than focusing their time and effort on learning computational tools. Spreadsheets are general purpose and have a relatively shallow learning curve relative to lots of other technologies, but they enable fairly sophisticated computation to take place—if inefficiently at times. They're like a lowest-common-denominator of data science.
We use Spreadsheets all the time in what we do, mostly as a transient form. The "heavy hitting" and "production" data takes place largely in MySQL and R, but there are constant temporary/intermediate moments in which data is dumped out as a CSV, touches a bunch of hands that are really not MySQL or R capable, and then is returned in updated form to where in normally lives.
Slashdot Mirror
back away from the distinction again and again. It's not programming vs not-programming, and it's not about building stuff in meatspace vs. building stuff in memory.
It's about public vs. private effects and risks. People who build stuff for backyards = not engineers. People who build stuff for the public square = engineers. People who build a bridge over their own stream themselves = not engineers. People who build a bridge over a public byway for heavy public use = engineers.
The same ought to be true in codingland. If you are building critical public infrastructure or software that many people must use in order to participate in society, then you should be licensed and bonded and held to higher standards. If you are building software that is for personal, private use, then this is not the case.
Note that I am talking about instance of execution, not instance of code. For example, MS Windows would not be "engineering" code because each instance of execution is private, while the code inside, say, a traffic lights system, aircraft control system, or automobile is "engineering" code because these instances of execution happen in public, with public effects. (If MS Windows was to be used to run, say, a battleship, then it ought to be a separate "engineered" version held to higher standards of scrutiny, professionalism, and liability.)
We don't certify and license people as engineers because they might touch wood and steel we think wood and steel are special or important, we certify and license people as engineers whose single instances of wood and steel construction will each touch (and possibly put at risk) many lives.
He's trying to say that if people want powerful, flexible networking, they'll choose an 80% safe OS that enables this easily over a 90% safe OS that imposes lots of overhead costs to make it possible; that people will choose a 60% secure OS that runs their processing jobs in 3 hours over an 85% secure OS that runs their processing jobs in 6 hours.
He's pointing out that people like security well enough, but they want to get stuff DONE even more, and that most people will take the calculated risk to be less secure if it makes them more productive at lower costs. That if there is a less secure but more productive option, up to some arbitrary point (that is different in each case, but that can be inferred by the movement of markets and communities as a whole), they'll choose the more productive option.
And that there is no point in saying "then all of us that produce these things must get together and make highly secure, if less capable stuff, so that all choices are equally highly secure!" because as soon as that happens, a garage coder somewhere is going to have a project on github that says "I got tired of waiting for jobs to finish, so I wrote my own from scratch. It's totally insecure, but damned if it doesn't finish the job in half the time!" and that people will immediately flock to it.
In other words, his goals for Linux aren't for Linux to be the most secure OS on the planet, but to be one of the most useful and used ones.
We are talking about securing tools. But the point is that tools do things. We want tools to help us to accomplish the things that the tools do.
A perfectly safe hammer is entirely possible. Make it out of flame-resistant, soft, synthetic materials and fill it with something equally soft. Shape it more like a ball than like a stick, so no-one can accidentally stick it in their mouth and suffocate.
Of course, now you have something that can't be used to pound in nails—but it's entirely the safest hammer on the planet.
Will anyone buy it or use it? Of course not. And they'll still need something with which to pound in nails. That's Linus' point.
This is not an easy "but." Most cases generate aggregate billing across all facets of the case at a rate of five or six figures monthly, and it is already easy to make the claim that the finance limits in every case are the most direct limits on just outcomes.
Who is going to pay the significant costs? And at the expense of what other part of the case that might have had them instead? There are many victims and defendants that would argue that the first thing that ought to be paid for is increased time for the prosecutor or defense attorney in a case to pay attention to the case, since in most instances the parties feel as though the attention of their legal representatives is being catastrophically rationed.
And what about other tests or forms of forensic expertise? Is it better to pay for two tests or, if funds exist, to pay for one of the other forms of corroborating expertise and testing that are otherwise unavailable due to cost constraints?
Access to truth and law are significantly cost-constrained on both sides. Adding one additional confirming test does little to change this, and just as importantly, it's not clear—given this constraint—just how even this one more test will be paid for without drawing resources from some other dimension of a given case that could be equally important in a system in which it's impossible to predict just which dimensions of a case will be definitive until case post-mortems and debriefings are being carried out.
The civil justice system is just as big a mess, and for the same reason as TFA implies. Strangely absent from both systems are the victims and the accused, the petitioners and the respondents, the actual citizens that the system purportedly serves.
Instead, both parties touch the system only at its very edge. Their actual cases happen more or less without them, winding their way through an impossibly large-scale system full of paid actors whose jobs, practically speaking, at the level of everyday experience, are not to think about the parties in the case, but to perform the same particularistic tasks day after day in massive volumes before handing things off to other paid actors in a massive division of labor.
It's a kind of assembly line or factory for legal activity and paperwork production. The complete details of any single case, civil or criminal, are not known by anyone within the system—even the judges, commissioners, and magistrates that hear them—even though the actual parties to the case know their own stories inside and out. There is no facility or room within the system for its paid actors to actually get to know a case, through either party's eyes. Instead, each professional focuses only on the tiny fragment of each case that they are responsible for before handing it off to the next professional.
There is essentially no oversight for any part of the system, and even if there was, plausible deniability is huge, since each professional knows and interacts with only the tiniest part of each case, yet most legal statues offer recourse only if poor or unprofessional practice are more likely than not to have actually altered the final outcomes of this division of labor involving many months, dozens or hundreds of specialists, and a significant degree of uncertainty due to the vagaries of interaction and logistics.
It is a forest-for-trees problem to the Nth power, but it's difficult to see any way to address it; to be just, the law needs to be well-documented, clear and explicit, and to have nuance and detail. This necessarily makes it large and complex. That implies the need for professionals that have been trained in it. But a professional that dedicates their life to law must be able to make a living. Most individuals cannot afford to pay an entire salary to a legal professional, much less the many that must work on a given case due to the complexity of the law, and thus, they cannot expect these professionals to dedicate themselves to a single case. Instead, the costs of the professionals' salaries must be shared amongst literally many thousands of victims, accused, petitioners, and respondents, meaning that the professionals must limit their consumption of case details to just those in which they specialize, or face mountains of information with which they can't possibly cope and the consumption of which would impact their ability to do the job in which they specialize.
As a result, for the average citizen, bringing a case or participating in a case is like playing a giant, almost comically huge game of Plinko. The case enters at one end of the machine and knocks about between pegs endlessly and seemingly at random, well out of their reach, for what seems like ages, while they stand by, breathless and helpless. At the end, the case exits somewhere, with some sort of decision, but the relationship between its final disposition and its initial circumstances are completely unpredictable and due to the nature of the machine, and it's difficult to argue that any part of the game machine is "broken" most of the time—a peg in the machine has to be severely affected (i.e. missing, malformed, completely bent) for such a claim to be viable. Minor variances throughout may influence outcomes for a very long time without being detectable, even under scrutiny. And for the most part, there is no budget, much less any avenue even for the funding and organization of a program of scrutiny.
The institutions themselves are benefitting in revenue terms through bookstore sales, which also benefit publishers significantly in a kind of win-win.
I made a post about my years as a professor below and about being fired for allowing students to work from used textbooks.
What I didn't post below was that in the early '00s, before I was a professor, I worked in a well-known academic publisher of textbooks and journals in the Los Angeles area. I was over a department / topic area and one of the things that we did to stay ahead of revenue neutrality in our publications was ensure that they were "updated" every year. In many cases, preparation for the new "edition" entailed hiring two independent contractors: one freelance photo editor to replace all the images, and another freelance academic (often at a total cost of $1k-$2k tops) to re-do the exercises, tweak a few chapter titles, and perhaps reorder some chapters.
This was a strategy to enable "stale" books (read: books with large presences in used channels) to be revenue positive again (new edition = now more books in used channels, meaning a rash of new sales for 2-4 semesters). The initial investment in the text had in many cases happened years ago; subsequent annual investments were often in the low four figures or even less.
Of course, this high-margin model also enabled us to do deals with universities and their bookstores. Because of the low overhead for many refreshed titles, we could offer favorable terms to them for their revenue generation, often demanding minimum buys or various kinds of exclusivity in exchange for better revenue terms.
It's a kind of wealth transfer from the taxpayers (as student loans), through students, into the pockets of publishers and institutional administrations.
I'll preface this by saying that I was not tenured faculty. But I was adjunct faculty with a thriving career outside of the university and seven years as a part-time faculty member.
This happened to me at local State U (I'm in a flyover state) and ended my years as a professor. I was a top-rated instructor in the department by both student evaluations and faculty observations, advising graduate students, experienced, and had been there a long time teaching courses that I developed and that were well-received.
New leadership came in at the divisional level, and I was called in to a meeting with my chair one day. I was told I could no longer do what I had been doing for at least half a decade: assigning a textbook that was several editions old (there were no substantive changes in the newer editions, just replaced photos) and instructing students on the syllabus to pick the books up for literally pennies on Amazon.com, Alibris, eBay, or other online venues.
Instead, I had to assign the latest issue of the textbook and do it only through the university bookstore, at a cost of >$150.00 in one class, >$200.00 in another (compared to an average of $4.00 plus shipping most semesters for the online used versions). I had it listed as my first assignment on each syllabus—buy a used textbook online and submit proof of purchase (to be sure the students actually did get ahold of the textbooks).
I refused. I said I would provide both options—I'd order the textbook through the university bookstore and provide that as an option to students that preferred to buy new, through the bookstore, but would also allow both current and old editions to be used in my classes for students that wanted to rely on used books. I was threatened again. New only, bookstore only.
I refused. I was fired.
That semester (in 2014) was the last time I set foot on a college campus as a professor, after nearly a decade in the classroom every semester. Again, I wasn't tenured—but it left a significant hole in curriculum and advising. They were more interested in ensuring that students contributed to revenue and partnerships through bookstore purchases than they were in actually enabling students to learn in a cost-effective way.
As an employee, I can just as easily say that when I've worked at jobs at which I was compensated well, knew I could be rewarded even more, and certainly didn't want to lose the position no matter what, I worked my tail off and held myself to very high standards.
When I knew that I could easily get another job at the same level of compensation, and that no matter how hard I worked it was unlikely to be rewarded with more pay, I put more of my time and energy into other avenues, like improving my skill sets and networking outside the company, so that I could transition elsewhere—as well as into simple quality of life stuff, since I knew that work wasn't going to enhance my quality of life all that much.
So you're saying employees work less well = pay them less well.
I'm saying pay them more = they'll work more and better.
Chicken and egg. Which comes first? The rewards or the pay? You make one argument. I'll make another.
Why are employees not as great as they could be? I'll go out on a limb and say it's because you're not motivating them to be as great as they could be, either through great leadership (which is rare), a great environment (slightly more common), or great compensation.
And if you get great leadership, a great environment, *and* great compensation, I'm betting you'll somehow magically find that you have the best employees on the planet, who would do almost anything for the company—and do it at a very high level.
I am also in my 40s and have a huge library of books (including a roomful of books on shelves from my Ph.D. years). And at this point, I can't *stand* paper books. They're heavy, have slow page turns, are not searchable, can only be carried in small numbers, are difficult to use (no changeable font, low contrast, drop it and you've lost your page), take FOREVER to find (Not at the bookstore? And let's face it, what's at the bookstore any longer? Then you'll have to wait days for the book to arrive in the mail, no impulse buying/reading), use up space in your house, and so on.
I am basically ebooks only these days. I buy and read probably 3-6 ebooks a week. If it's not available electronically? I've probably bought four paper books over the past year, if that. I have to really, really want it to put up with paper and the inconveniences of buying/reading paper.
not just for the question of how a contributor should be treated, but also for the question of how a leader should act.
If a leader is able to get world-beating results by being an asshole, then so be it. That leader has beaten the world, and I am not going to quibble with success. If a leader is an asshole and subpar output is the result, then by all means, tell them to treat their team differently.
Team dynamics are a complicated thing. You just don't fuck with a winning team. If they are using four letter words all the time and sacrificing live chickens at midnight, but the results are running circles around everyone else, I for one do not want them to stop, even if it would save a chicken's life.
At the same time, if they are doing all of these things and the results are uneven or poor, then by all means, change the behavior.
In this case, I'd say that the results of Linux kernel development speak for themselves. And if you just don't belong in the culture, then go somewhere else. If the culture starts to be counterproductive, give the world a great, big "I told you so!" and collect your profits on the book deal. But otherwise, to expect people to fuck up a successful operation for your feelings, for manners, or for high-minded ethics concerns is just bad juju. It's not lawyering or doctoring, ethical concerns are not front and center. It's software. The goal is that it works and works well, and in fact that's the highest ethical aspiration *of* software, given the many critical ways in which it gets used in today's world.
The value to the users is first. The comfort of the developers is second. If the culture and development process are working well, get the hell out of the way if you don't like them. As this person has done. So—problem solved.
The first thing I wanted after installing El Capitan was information on how to disable rootless mode, not advice about the soundness of this idea. Thankfully, I found an informative post or two by searching Google. Any advice would not have been the information I was looking for.
And for those that are interested, yes, there is actually a .conf file that controls the rootless mode protections. I forget the path, but if you Google, you'll find it. The catch of course is that you have to disable rootless mode in order to edit it, and each time you want to edit it, which means multiple reboots for each edit.
Given the fact that I use software from across the 'net on my Mac, much of it not Apple developer signed and some of it development oriented, I figured I'd likely encounter problems along the way by trying to edit the .conf file and would have to keep banging on and editing it over time. And I've been using Macs and Mac OS for years already without rootless mode, so I don't feel too catastrophically ba about not having it now.
Because I keep my Applications folder lean and uncluttered and I never use Mail.app, as one example, so I want it gone from sight.
My prerogative as the ower of the system, I'd say.
Just upgraded to El Capitan last night.
Problem (1): I found out very quickly that root has been neutered; you can't make any changes to "system" files (in this case, meaning files that were included in the OS distribution, including things like the Mail.app folder or binaries, etc.). You get a message about not having permission, despite being root, and without any extended attributes being set on the files. Turns out that El Capitan uses a new "rootless" model in which root is no longer root and many parts of the system are off limits to any human user. Solution: Boot into recovery mode, start a terminal, and enter the command "csrutil disable" then reboot. You'll get root back and will be able to change files again.
Problem (2): Parts of Office 2011 didn't work at all—just beach balled upon startup. I tried to figure this out for a while but didn't see anyone else talking about solutions online, so I installed Office Mac 2016 (since I'm already paying for Office 365 anyway so that I can use it on my tablet and phone). I've been using the Office Mac 2016 applications all day (Outlook, Word, and Excel for work) heavily, without any trouble, so as a data sample of one I can say that in my case, 2016 is definitely a better bet on El Capitan than 2011, since Word and Outlook 2011 didn't work at all.
I guess I tie the idea of "malware" to two concepts:
1) Mal, as in harmful to the user.
2) Ware, as in software.
To me, "malware" as a concept is basically about end users. It is software that is installed by endusers that does something contrary to what they expect, possibly without their knowledge, that is harmful to them. Malware is inherently deceptive, and the method of its deception is posing as something else. It is directed from bad actors toward strangers that these actors wish to exploit. It is a numbers game, a volume game.
This was not installed by end users, it did not pose as something else, and the harm was directed at an organization by individuals within the organization. It was not distributed widely, but was a single instance. I'd call this a "hack" or a "sabotage" or an "embezzlement" of some kind before I'd call it malware. Maybe a new term is needed.
But it seems a big jump from the widespread distribution of a Windows wizard to millions of hapless end users all the way to the willful and direct modification of company equipment by employees for gain.
Just sayin.
Or at least, stretches the definition of "malware" to the breaking point.
All of the following hardware from Apple has been absolutely groundbreaking/pathbreaking. When it came to market, there was nothing else like it:
- The original Mac 128k
- The Apple Newton
- The iMac
- The iPod
- The iPhone
- The iPad
Complaining that the Mac couldn't be expanded is like complaining that you can't rebore the cylinders on your Tesla to get more horsepower. These were products that changed users' understandings of the product space in question.
As a young geek, I cut my teeth on multiple computing systems. Three were old 8-bit systems: a C64, a TRS-80 CoCo2, and an Apple II. One was a Mac 128k. They were not even the same kinds of products. To call them all simply "computers" is ridiculous.
The same thing goes for:
Newton vs. other embedded "tablets" of the era (Fujitsu, IBM, GRiD, and others)
iMac vs. white-box PC
iPod vs. previous MP3 players or digital MiniDisc players
iPhone vs. previous smartphones like Treo
iPad vs. Windows CE "Handheld PC Pro" tablets
I can remember when everyone was making fun of the iPad for not including a stylus, a CF card slot, or a removable battery. And some geeks here continue to try to pretend that it was only a matter of advertising and image that caused consumers to gobble iPads up, and eventually, other companies (the entire Android field, for example) to essentially throw away any previous work and design to iPad specs and form factor.
But go ahead. Try it. Get someone a Viewsonic or Fujitsu tablet from 2009 and then hand them an iPad 1 and ask them which they prefer. Consumers aren't as stupid as people here make them out to be. They care about their hard-earned dollars just like everyone else. And they had no problem deciding that stylus-based resistive tablets with two hour batteries that were an inch and a half thick and two pounds (to accomodate replaceable batteries and removable storage) and that ran Windows... were not something they wanted to spend their money on.
Everyone here is quick to call it all "bullshit." Yet how many here own and use a tablet and/or a smartphone with a multitouch display and a purpose-specific operating system with a touch-oriented user interface? A lot, I'd bet, including many who mock such things as they peck out their mockings on an on-screen keyboard (which they also mock). Who brought these things from the halls of CERN to consumer electronics, not thinking that it was an impossiblility?
Apple under Jobs. Mock away.
that there is a difference between "marketing," "advertising," and "sales."
Marketing is tremendously important *at the stage of product design.* Marketing, when done profitably, means *understanding your market* (i.e. users) and what they need (which may or may not be what they think they want) and then ensuring that your engineers get wind of that need and design to it.
Good marketing is an integral part of good tech, and happens well before any advertising takes place. The best products may or may not sell themselves (I think there's a good argument to be made that people need to actually know about a product, and it needs to be available in channels, before they are able to realize that it exists and buy it), but the only way to *get* to the "best products" for a large audience is to have a very good marketing division helping engineering to understand just what "best" means for a large, diverse userbase.
Yes, Apple has been very good at marketing over the last two decades. This skill is inseparable from their ability to design, and the fact of their having designed and taken to mass production, rather good hardware that is in high demand.
becuase they're geeks and (understandably, self-servingly) want to point how how central and important geeks are to, say, computing and technology hardware and software development, design, and production.
But it is one of the rarest geniuses on earth to be able to conduct a group of people to produce to their maximum potential, to be able to somehow lead talent to actually produce what the talent is capable of as a group and to do things that everyone else wants to do, but everyone else also falls short of time and time again.
The founding of Apple was really far less miraculous than the turnaround, when Jobs was able to get a huge bureaucracy to start making really high-quality, completely realized products without the significant compromises that everyone else took for granted. I'm typing this on a Macbook Pro right now. For many years I used Thinkpads. There is a difference in the aesthetic, as is so often pointed out, but it's a difference that in the aesthetic of functionalism that has to be realized through design, logistics, manufacturing, etc. involving teams of many very smart people. The Macbook Pro isn't perfect, but it's a far superior machine to the Thinkpads I used to use, not because it's faster or has more features but because it has fewer flaws and compromises; it represents something far closer to a fully realized idea and goal.
The same thing goes for smartphones and tablets. I used to carry around Treos in the early 2000s. I used them heavily. They were my go-to tools. I wrote a book on a Treo, no kidding, riding on the subway every morning, that's still generating me about $20 in royalties a year (big money, heh). But I used them. The same thing for my Windows CE tablets, first a Vadem Clio and later a ViewSonic something-or-other. But they were exercises in taken-for-granted compromises. They were "as good as it gets," it takes a big company to design and make such things, and the end products, though flawed, were the best that could be accomplished. They were "hard problems" and "best-case solutions" as products. They worked well.
Or so everyone thought.
And then? iPhone. And iPad. And they set an entirely new bar and benchmark for their respective industries. The previous products were obsolete in a moment and everyone has struggled to catch up. Tim Cook has not been able to replicate this precisely because he does not have the particular genius that Steve Jobs had. That's not to say that other people inside Apple don't also have genuis of many varieties. Half of the people on Slashdot (okay, not half, but some) are probably geniuses in their own right, in algorithms, or some area of hardware engineering, or whatever.
That doesn't take away from the fact that Steve Jobs was a rare genius in management and leadership. He was the opposite of the pointy-haired boss. We make fun of the pointy-haired boss precisely because we realize that it is the norm. Jobs was not the clueless leader; he was the leader that always somehow managed to get it right and squeeze more great, historic, memorable, and compromise-free stuff out of the geniuses at his company, by far, than the vast majority of other leaders—even the highly regarded, very well paid ones—are able to ever come close to getting out of the geniuses at their own companies.
That's not nothing. And given the multiplier affect of getting the best out of many geniuses, it's quite a lot.
I didn't know that, though based on what I do know about the jpeg format, it makes a kind of sense that this would be possible. Thanks for posting this, great nugget of information!
what consumers had access to by walking into a retail computer dealership (there were many independent white box makers at the time) and saying "give me your best."
You're probably right about me underestimating the graphics, though it's hard to remember back that far. I'm thinking 800x600 was much more common. If you could get 1024x768, it was usually interlaced (i.e. "auto-headache") and rare if I remember correctly to be able to get with 24-bit color—S3's first 16-bit capable chips didn't come out until late-1991, if I remember correctly, though I could be off.
SCSI was possible, but almost unheard of as stock, you either had to buy an add-on card and deal with driver/compatibility questions or one of the ESDISCSI bridge boards or similar. Same thing with ethernet, token, or any other dedicated networking hardware and stack. Most systems shipped with a dial-up "faxmodem" at the time, and users were stuck using Winsock on Windows 3.1. It was nontrivial to get it working. Most of the time, there was no real "networking" or "networking" support in the delivered hardware/software platform; faxmodems were largely used for dumb point-to-point connections using dial-up terminal emulator software.
And in the PC space, the higher-end you went, the less you were able to actually use the hardware for anything typical. Unless you were a corporate buyer, you bought your base platform as a whitebox, then added specialized hardware matched with specialized software in a kind of 1:1 correspondence—if you needed to perform task X, you'd buy hardware Y and software Z, and they'd essentially be useful only for task X, or maybe for task X1, X2, and X3, but certainly not much else—the same is even true for memory itself. Don't forget this is pre-Windows95, when most everyone was using Win16 on DOS. We can discuss OS/2, etc., but that again starts to get into the realm of purpose-specific and exotic computing in the PC space. There were, as I understand, a few verrry exotic 486 multiprocessors produced, but I've never even heard of a manufacturer and make/model for these—only the rumor that it was possible—so I doubt they ever made it into sales channels of any kind. My suspicion (correct me if I'm wrong) was that they were engineered for particular clients and particular roles by just one or two orgnaizations, and delivered in very small quantities; I'm not aware of any PC software in 1992 timeframe that was even multiprocessor-aware, or any standard to which it could have been coded. The Pentium processor wasn't introduced until '93 and the Pentium Pro with GTL+ and SMP capabilities didn't arrive until 1995. Even in 1995, most everything was either Win16 or 8- or 16-bit code backward compatible to the PC/XT or earlier, and would remain that way until around the Win98 era.
The UNIX platforms were standardized around SCSI, ethernet, big memory access, high-resolution graphics, and multiprocessing and presented an integrated environment in which a regular developer with a readily available compiler could take advantage of it all without particularly unusual or exotic (for that space) tactics.
The 386 box that I installed Linux on my first time around was 4MB (4x1MB 30-pin SIMMs). 4MB! I mean, holy god, that's tiny. It seemed sooooo big compared to the 640kb of 8-bit PCs, and yet it's basically the same order of magnitude. Not even enough to load a single JPG snapshot from a camera phone these days.
used that abbreviation that it just doesn't roll off the fingers any longer.
For more than just a couple of us here, I suspect, there was a time when "Sparc," "UNIX," "graphics," "Internet," and "science" were all nearly synonymous terms.
Simpler times. Boy did that hardware last and last and last in comparison to the hardware of today.
Well, I suppose it can finally no longer be said that the Sparcstation 10 I keep here just for old times' sake can still run "current Linux distributions." But it's still fun to pull it out for people, show them hundreds of megabytes of RAM, 1152x900 24-bit graphics, gigabytes of storage, multiple ethernet channels, and multiple processors, running Firefox happily, and tell them it dates to 1992, when high-end PCs were shipping with mayyybe 16-32GB RAM, a single 486 processor, 640x480x16 graphics, a few dozen megabytes of storage, and no networking.
It helps people to get a handle on how it was possible to develop the internet and do so much of the science that came out of that period—and why even though I don't know every latest hot language, the late '80s/early '90s computer science program that I went to (entirely UNIX-based, all homework done using the CLI, vi, and gcc, emphasis on theory, classic data structures, and variously networked/parallelized environments, with labs of Sparc and 88k hardware all on a massive campus network) seems to have prepared me for today's real-world needs better than the programs they went to, with lots of Dell boxes running Windows-based Java IDEs.
100m data points *or less*
here are my answers. Spreadsheets are used in several cases:
1) When you have a small-to-medium-sized dataset (100m data points) and want to do a particular set of calculations or draw a particular set of conclusions from it just once or twice—so that the time invested in writing code in R or something similar is less than the time needed just to bung a few formulas into a spreadsheet and get your results. Once you get into analyses or processes that will be repeated many times, it makes more sense to write code.
2) Similar case, when you need to work with essentially tabular database data, but the operations you're performing (basic filtering, extracting records based on one or two criteria, just handing data from one person to the next) are either so simple or will be repeated so rarely that a MySQL database is overkill and just emailing a file back and forth is easier.
3) When you are working with data as a part of a team, and certain members of the team that are specialists in some areas related to the data, or (for example) members of the team that are doing your data collections, aren't particularly computationally expert. Spreadsheets are hard for laymen, but it's doable—a dozen or two hours of training and people can get a general, flexible grasp of spreadsheets and formulae. It takes a lot longer for someone to become basically proficient with R, MATLAB, MySQL, Python, etc., and you really want those specialists to just be able to do what they do to or with the data, rather than focusing their time and effort on learning computational tools. Spreadsheets are general purpose and have a relatively shallow learning curve relative to lots of other technologies, but they enable fairly sophisticated computation to take place—if inefficiently at times. They're like a lowest-common-denominator of data science.
We use Spreadsheets all the time in what we do, mostly as a transient form. The "heavy hitting" and "production" data takes place largely in MySQL and R, but there are constant temporary/intermediate moments in which data is dumped out as a CSV, touches a bunch of hands that are really not MySQL or R capable, and then is returned in updated form to where in normally lives.