From the manpage, mount_ntfs(8) has significant limitations:
There is limited writing ability. Limitations: file must be nonresident
and must not contain any sparces (uninitialized areas); compressed files
are also not supported.
I can't seem to find a straight definition of "nonresident files" in the context of NTFS, but my best guess from glancing over google results is that "resident" files are ones which have their contents in a small block embedded in the inode itself. That'd be an optimization to minimize internal fragmentation on small files. If I'm right, a Windows-produced NTFS is likely to have a lot of these files. Not sure if OS X can't write to them at all, or if it will just make them non-resident when it does so.
For example, say that there's an issue where 95% of the poll participants agree. In order to present a balanced view containing the opposing side, a new journalist may take the majority opinion and a minority opinion. When presented as opposing sides it may give the impression that people are evenly divided.
I hear you, but I think the even bigger issue is that journalists often just hand the microphone to the two parties (frequently metaphorically; sometimes literally), without making any attempt to evaluate if what is said is true. There's certainly a place in this world for opposing viewpoints based on value judgements...but when both parties have access to essentially the same information, there shouldn't be such huge disagreements over basic facts. Often one party is mistaken or lying. It's the journalist's job to point that out (whether it's legislated or not), and many are failing.
To see exactly what I mean, watch The Daily Show. Jon Stewart, in spite of being a "fake" journalist, does a much better job of this than most. He'll frequently show a news clip where a politician will make a simple objective statement like "I have never said 'X'", then show an older news clip where the politician said 'X'. How often do you see that on Fox News, or even CNN? Journalists who do not do this are not presenting the facts in an "honest, equal, and balanced manner".
I don't think any legislation toward this goal will be successful. It's a lot more subtle than "give 50% of the airtime to viewpoint A and 50% of the airtime to viewpoint B", or even "give 95% of the airtime to the viewpoint 95% of the people have, and 5% of the airtime to the viewpoint 5% of people have".
It's also not one-size-fits-all: I expect journalists to find things obviously wrong when politicians speak, but global warming is far more subtle and beyond the understanding of the average journalist. They're not going to say "CNN was unable to reproduce expert A's Monte Carlo supercomputer simulation in our own laboratory, so expert A is lying"...there they have to scale back to the more realistic approach of checking if expert A's credentials are good, where his ideas were published, what sort of peer review his ideas have gotten, what the results of that peer review were, and who is paying for his research. It'd be nice if everyone were skilled enough to evaluate the ideas on their own merit, but that is just not to be. I don't understand global warming research, and I certainly don't expert a journalist to.
I pay my ISP for my bandwidth, it shouldn't matter to whom I connect.
You didn't read my first post, did you?
If that's what you want, that's great for you. I, on the other hand, would be willing to pay extra for QoS. Yes, I would like my connections to be treated differently based on who I connect to - I want my VOIP traffic to be prioritized over my BitTorrent traffic and go through shorter queues, as far through my ISP's network as technically possible. Preferably for both upstream (where I can just set the TOS/DSCP header to specify the class) and downstream (in which that header can't be trusted - there's just the addresses, protocols, and ports). Please don't pass overbearing laws that forbid that.
I might also buy some discounted "when available" bandwidth. So please don't overbearing pass laws against prioritizing one customer's traffic over another's, either.
There are technical challenges in doing what I want, so I'm not sure what exact form it would take. (QoS equipment is common, but it's not like you can just have an arbitrarily long ruleset for each customer on every edge router in your network.) I think it's premature to be passing laws about it, especially ones written by lawmakers who know nothing of technology.
Net Neutrality is *not* about preventing people from optimizing the Internet!
It *is* completely about preventing abuse of monopolistic power by telco companies.
I think it's both. The goal is to prevent abuse, and it's a good goal. But I don't really know what the specifics of the legislation would be, and it'd be easy to throw the baby (service-based QoS) out with the bathwater. See my other post for details.
I don't understand his point in describing Akamai:
Is he saying that Akamai would be violating this network neutrality legislation? If so, how?
Is he saying "websites push data faster by paying Akamai to host it, so they should pay the end user's ISP for preferred bandwidth, too?" I missed a step...
His other example (actually using packet classification and QoS based on the type of service) makes a bit more sense. I haven't really been watching the network neutrality debate, but I have wondered how this is supposed to mesh in.
Currently, I classify my own uplink to make my ssh connections still go fast while I'm running BitTorrent. (Side note: for those with cheap DSL modems that don't support doing their own QoS, I highly recommend these Linux patches. Without it, tbf shaping doesn't work because its bandwidth numbers are all wrong.) QoS is extremely effective - you can have short, usually-empty queues for interactive packets (about 20 ms in my case) while still having longer full queues (more like 500 ms) necessary for full bandwidth utilization on bulk traffic.
My ADSL upstream is my bottleneck. Network neutrality doesn't come into play, because it's under my control and only used by me.
However, it's not hard to imagine a small heavily-used ISP where their connection is the bottleneck. Maybe some customers are willing to get deprioritized at peak times in favor of cheaper service. This is much like the physical world, where people use 5-7 day shipping instead of next-day air for cheaper rates. Do existing network neutrality proposals forbid this? If they do, I think they're going too far.
If laws do allow this, it gets more complicated. Let's imagine for a second that I'm paying for 1 Mbps of interactive (short queues, prioritized, drops unlikely) bandwidth on select packets, 5 Mbps of bulk bandwidth on non-select packets or the leftovers, and everything above that is dropped. In the upstream case, it's pretty easy - I can set the TOS/DSCP field to distinguish between the two.
In the downstream case, how does classification work through their network? Apparently the telephone companies have been making noises like they're going to use preferred bandwidth if it came from an IP owned by someone who paid them money. Seems ridiculous. Let's say I'm a third-party VOIP provider. My customer has to pay their ISP extra for preferred bandwidth, I have to pay my own ISP (possibly extra for preferred bandwidth), and I have to pay my customer's ISP for preferred bandwidth, too? My customer's ISP is doing pretty well on this deal, but I'm not. I don't have much negotiating position - my customer has probably already chosen an ISP, I want to get stuff to them quickly, the customer's ISP will tell me "our way or the high way". It seems anti-competitive.
The ideal thing economically would be for the customer to control the downstream classification, too. But technically, that sounds pretty difficult - there'd have to be some way for them to say "SIP packets from Vonage to me should use my 'preferred' bandwidth if available" and have those classifiers propogate through the ISP's entire network. The ISP would have to give them some sort of a web interface or something for them to do that (a lot of integration work), and I'm trying to imagine the average consumer using it successfully...nope, can't do it.
I worked with a lot of embedded development people (I'm also one of them) and most people I met are extremely good programmers, that can not only code well but they can hand-optimize their software without making a mess out of it. After all, we work with 30-200 MIPS processors that sometimes need to execute data-processing tasks that would bring a multitasked P4 to its knees. We know A LOT about optimization and we're far better coders than any basement rat you can find.
Sounds like the product I'm working on (using an obscure 400 MHz VLIW processor), and my team is filled with developers much better and more experienced than me. But your group is not the sort I'm talking about - it's a far cry from a network card's drivers.
I'd say NASA's software practices are prudent and entirely appropriate. I would much rather fly on their code than yours, no offense.
You're missing my point. They're appropriate for the shuttle group, where lives depend on the code. Nowhere else. The Mars Rover code is not made to this standard, and consumer code could never be. It's impossible. Anyone who points to this as an example of how we should all be writing code is simply wrong.
As someone who has depended on NASA flight software, I'd rather sacrifice features for bug free code. That's a basic difference between consumer software and mission critical software.
Agreed. If my life were depending on it, I'd want code written in this way, too. The rest of the time, I want my shiny OpenGL-accelerated windows zipping around the screen, and I realize that writing "QUALITY" in giant, bold, all-caps letters at the top of the priority list (above "not wildly exceeding our meager budget" and "write mandatory features") would make that impossible. Line-for-line, this most expensive code ever written by at least an order of magnitude.
No other group at NASA writes flight software like this, because Shuttle is the only man rated launch vehicle. Orion will be similar (and it's software is being written by the same people). Other flight software at NASA is not this extreme, but there is a NASA software development standard for all flight software and it's still pretty rigid compared to consumer software.
I wrote: Give me our full-featured, buggy software over nothing any day.
An AC replied: Strange, I thought all those computers on board the Shuttle, ISS etc. were actually doing something other than an idle loop.
The point is that they wrote 420,000 lines of code in 30 years and an estimated 7,800 man-years. That's 14,000 lines per year, or 54 lines per man-year. Considering that I can single-handedly outperform their entire team of 260 people, those had better be 420,000 glorious lines of code. Furthermore, if all software were written that way, the software you know today wouldn't just be bug-free - it wouldn't exist at all.
As far as I know the On-Board Shuttle Software Group has a track record of 3 (in words: 'three') software bugs in installed operating code within 30 years of writing code.
I was much more impressed by that number before the story about avoiding having a shuttle in orbit at New Year's because the software can't handle it. That's been known for years and they haven't dared fix it. Is that counted as one of the three? No? Then they've fixed only three bugs in the last 30 years, and they have more than that, unless you think a serious misdesign is not a bug. If I confused the presence of bugs with having fixes for them, didn't consider a serious misdesign to be a bug, and had barely added a real feature in 30 years (at current head count, 7,800 man-years), I too could claim some ridiculously low bug count.
It also seems to me that the shuttle group's software situation is totally irrelevant to anyone but the shuttle group. Look at this part of the article you mentioned:
Take the upgrade of the software to permit the shuttle to navigate with Global Positioning Satellites, a change that involves just 1.5% of the program, or 6,366 lines of code. The specs for that one change run 2,500 pages, a volume thicker than a phone book. The specs for the current program fill 30 volumes and run 40,000 pages.
That sort of rigidity makes their methodology totally useless for software outside NASA. I occasionally hear people talk about how the Shuttle Group does software right, but for non-life critical systems, the cure is worse than the disease. Give me our full-featured, buggy software over nothing any day. There's got to be a better way.
I suspect it's also useless to the other groups in NASA. Do you actually know that the Mars Rover software was written in this manner?
Please mod the parent down as "flamebait" or "troll". Anne Thwacks either has shockingly low reading comprehension or is deliberately misrepresenting the article [*]:
There is ABSOLUTELY NO REASON why a FOSS driver cannot install the firmware. This is NOT the problem. There MAY be a problem with distribution rights, or with documenting how to load the firmware, but these are NOT what TFA described.
The article says:
The first challenge for operating system developers is obtaining the right to distribute the firmware file, which some manufacturers will not allow without significant restriction.
Anne Thwacks says:
While one might like to have the spec for writing one's own GPL firmware, and I dont see prob;lems with that, I do see a problem with expecting $100,000 worth of firmware development for free, when the hardware can be replicat4ed for a $10, and the combination normally sells for $100. Ie there are products on the market where the majority of the value is in the firmware. and Yes, it does sometimes take more than three man-years of $100/day consultants to write firmware for a product with a predicted lifetime of 8-months. (Graphics card, anyone?)
First of all, you're missing the initial cost of the hardware. If the marginal cost (materials) is $10/card, there's also the cost of the hardware design, the manufacturing plant, etc. Make this same error for software and you'll think it's all basically free, since the marginal cost is just bandwidth for downloads or some pressed CDs and inked cardboard.
Secondly, there is not $100,000 of firmware for $10 hardware, even considering only the hardware's marginal cost. You're comparing graphics card firmware/driver prices (atypically high) with network card materials prices (much lower; a high-end GPU has more transistors than a CPU, and a network card...doesn't). You may think your network card firmware is worth $100,000, but it's not. Hardware people think their software is valuable because they see other people selling three man-years of software for huge amounts. Here's what they miss: those other people are good at writing software, while hardware people and poorly supervised contractors are horrible at it. In those three man-years, they'll produce code that's bad in every way you can imagine - filled with magic numbers, race conditions, deadlocks, spaghetti code, massive duplication, inefficiencies, and bizarre workarounds for bugs both in the hardware and the firmware. It's entirely free of comments or documentation. It's not even indented right. It's a miracle it ever works. Be thankful this software does not require sophisticated algorithms, or it would be entirely hopeless. It's nearly worthless - it certainly isn't well-written enough to be useful on a competitor's chipset. But for some reason, hardware people think their software is the secret sauce, so they're afraid to let anyone even distribute the binary, much less see their awful code.
Thirdly, if you'd READ THE FINE ARTICLE (properly), you'd see that most people would be happy with just distribution rights for the binary. For now, anyway. I'm sure there will come a time when people will want more of the system to be Free - so they can fix the dreadful bugs long after the manufacturer's stupid contractors have all been fired - but we have to pick our battles. We don't even have Free drivers (or specifications, with which to write our own) everywhere, so it's too soon to be insisting on having the same a level down. Get the code running on the CPU straightened out first, then the other stuff can come...
[*] Probably the former. "Never attribute to malice that which is adequately explained by incompetence."
Oh, please. Does Kerry's "I voted for the war before I voted against it" really belong in that list? Keep in mind that if "inarticulate moments" are in the same league as bribery, voter suppression, and manslaughter (the other items on your list), Bush is a much worse president than I ever realized... a few examples.
Please, politicians by default are dishonest, not just Republicans. So just remember who's dog food you are eating when shilling for one side or the other.
You're as much in denial as those who claim there's no corruption at all, and your beliefs are just as harmful. There's been political corruption in all parties throughout history, but not all politicians are corrupt, and not all parties are equal. Voters have recently realized that they can exert control by voting out the more corrupt party. Maybe in 20 years the Republican party will reform and the Democratic party will regress. Until then, I'm voting for Democrats.
Though I would argue that your complaint about mailbox corruption during a network outage is only true with soft mounts. Not that hard mounts make it a good idea though.:)
Hard mounts wouldn't help all cases. What if the outage is longer than soft fail time * retries? And is the file lock retained over soft mount failure? And what happens if either machine goes down during this time? Or if the client process crashes? These are all failures which can occur on even a single machine, and ones which a well-thought-out file format can solve, much like the write-ahead log does for ACID databases like Oracle or PostgreSQL. But the traditional appended messages is not such a format, and NFS makes the failures likely. Bad combination...
In any case, good luck with your mail setup. I sympathize...
Wait. UID support requires that I write my inbox spool files out in mbx format, right? I can't do that because I'm NFS exporting the mail spool to a bunch of other 'nix clients along with IMAP. Which is why I'm still using traditional sendmail appended spool files. Is this what you were talking about?
Yeah, that would do it. I don't use wuimapd, so I'm not familiar with mbx format, but the traditional appended message format is horrible:
Inefficient - not only is there no standard place to put uidvalidity and uid information (and other clients couldn't be trusted to maintain the invariants if you came up with a nonstandard one), but there are no persistent indexes at all. I guess you could keep at least the byte offsets of each message cached externally with validity keyed by the mtime of the mail file, but it can't keep it locked for the whole session or no mail would be delivered. A single external access (whether by sendmail delivering a new message or by one of these NFS-based clients) would force it to recompute. It would just know that the file changed; it doesn't have any way of knowing if the only change was appending a new message.
Unreliable - The only way in Unix to replace the middle of a file with a different-length segment (as when marking a message read or moving it between folders) is to write out the whole new file on the same filesystem then rename() it into place. Your over-NFS clients probably don't even have permission to create files anywhere on the same filesystem, so instead they just overwrite all data later in the file and ftruncate() to the correct new size. If there's a network outage while that operation is underway, the file will be corrupted. At best, there will be a chunk that's skipped or written twice. At worst, it will be a total jumble of old and new blocks - it can't put a fdatasync() barrier between each write because it'd be way too slow.
I use Cyrus IMAPd. It's a different approach - all the mail on the system is owned by user cyrus. It has its own quota system. Email is stored in basically maildir format (one file per message), but with extra Berkeley DB stuff in each directory (uidvalidity/uid, full-text search indexes, imap flags, etc.). The SMTP daemon (Postfix in my case) hands the mail off to it for delivery. You only access mail through IMAP.
It works well for me. My worst problem is that Mail.app will occasionally go into this slow "synchronization" procedure where it retrieves the flags of every message to see if they've changed - that's the situation RFC 4551 should fix.
I'd suggest getting rid of the NFS export. It's holding you back, and just about every client out there (even many text-based ones like Pine and mutt) speak IMAP. Even ones which don't support the UID stuff (like really old mutt versions) should be faster, since they can at least retrieve a single message later in the session without causing it to iterate through everything before. Not corrupting the mailbox should be a nice plus, too.
Remember, every IMAP connection performs a linear traversal of that spool file to extract new mail headers. That's the real problem, and it can't be solved with cheap commodity hardware.
But it can be solved with cheap commodity software. This is why IMAP has had UIDs in the spec since at least RFC 2060. The only case I'm aware of in which the entire mailbox still must be traversed (flag synchronization) will be solved when RFC 4551 is implemented on both the client and server side.
If you're thinking of something else, could you please give details?
Many people are stuck with user-hostile Exchange accounts due to fear of litigation. Companies impose rules like deleting all mail older than 30 days and not allowing the users to backup their email.
I think the lawyers are just a convenient scapegoat. As I understand it, they got mad at corporations for saying "we don't have that on our server" when asked for something old but damaging but the user being able to produce it from a.pst file when it's helpful. So the lawyers said you'd better always produce everything you have.
Consequently, companies that don't want to deal with digging through.pst files (can't blame them) disallowed user backups. But here's where the problem lies - they didn't increase the mailbox sizes at the same time. They kept the draconian limits, either because of technical limitations (eased by this new Exchange version) or because they have skeletons in their server closet and don't want anyone to see them...
We have an ~100mb limit so that *users do not use mailboxes to store vast quantities of data*. If you have 2gb of data, it should be on a shared server!
I'ts not just attachments - my personal email account holds over 500 megabytes of mostly text. (To be precise, the vast majority of emails are pure text; don't have numbers handy, but I think the majority of storage is taken up by text. Large emails or ones with Office attachments are rejected.) It also holds another couple gigs of spam/viruses I keep around for Bayesian filter training purposes.
Ideally in a corporate environment people put stuff (even text) on a wiki or into a repository, but it's good to be able to go back to old emails when something falls through the cracks. Why delete anything when I have the space?
Personally I would like to see a system that kept attachments only for a week and then stripped messages to text only - those could be kept forever as a useful archive. But 8 copies of different and non config controlled bid spec documents? That's only going to cost you money and lots and lots of pain.
I would prefer people didn't send those as email attachments at all; the alternative I discussed in a recent thread is superior. The only case it doesn't help with is revising documents with someone outside the company. Don't have a really good answer to that one, but email certainly sucks...
Am I the only one who wishes that the laptops with the built-in iSight had a way to manually close the shutter, like the standalone iSight?
First, there is an obvious green light by the camera that comes on when it's active. I hope that it's hardwired so that a firmware update can't just flash that functionality away. So you can at least know something's up when the light comes on, though you can't necessarily prevent it in advance without add-on security. (Something seems wrong about taping over my shiny laptop, though Zen and the Art of Motorcycle Maintenance says it's okay to fix the motorcycle with the cheap metal from a pop can.)
But no, you're not the only person who wishes that. I'd like to go further. I wish there were a similar mechanism (at least the light, maybe a manual switch) for the microphone. I'm not sure why people are so worried about people looking at them but don't even think of the possibility that somone could be listening to them. Almost all laptops (not just new Apple ones) come with a built-in microphone, and it's only a matter of time until people start using viruses/worms/trojans to spy on conversations.
Still...if we can't trust our computers not to spy on us in the physical world, that's a pretty sad statement for eCommerce. At some point you have to type the credit card number into the computer...better make the software trustworthy...
Dead serious: Before any new law may be passed, the legal code shall be reviewed in it's entirety and thoroughly checked for existing laws serving the same purpose. If any such law shall exist, the proposed law may not be passed. If multiple laws serving the same purpose are found, they shall be reconciled into one non-self-contradictory law with the eldest law taking precedence. Not only will Congress be too preoccupied by this to do any more damage, but eventually the legal code will become understandable again.
Rationalizing regulation: In ancient Iceland, the people would gather together in an assembly, the Althing, once each year to hear their corpus of law recited from memory by a professional lawspeaker. If a law was forgotten during the hours-long proclamation and no Icelander objected then it lost its force, limiting the number of rules that could be pronounced before the speaker dropped from exhaustion. Thus, only rules that concerned the people and advanced the public good could remain "on the books".
I'm inclined to agree with you; our laws are the legal equivalent of spaghetti code. They're poorly crafted - too permissive in places, too restrictive in others, too complicated altogether. When most citizens break laws during the course of a normal day, something's wrong with the laws.
According to TFA, 'MySQL does very good job even on the busiest sites,' while for PostgreSQL 'Random disconnects, core dumps and memory leaks are usual.' This flies in the face of my own experience and testing results I have seen.
Agreed; that statement is a big fat lie.
Under heavy load, PostgreSQL has a habit of slowing to a crawl, while MySQL just dies. How many web pages have you seen where the entire text was a PHP error saying it was unable to connect to the MySQL server
As much as I hate MySQL, I can't blame it for this one. I'd say that under heavy load, PostgreSQL keeps on going fine and MySQL slows to a crawl, but both still operate.
PostgreSQL will have the same problem when misconfigured in the same way. It's pretty simple to avoid: add up the maximum sizes of all pools that connect to the database. Make the databases's maximum allowed connections at least that number + however many administrative connections you might open. If it's a huge number, you'll probably want to reduce the pool sizes for performance. (And reduce it a lot further for MySQL than for PostgreSQL, because MySQL does not scale. It does not perform under high concurrency, and additional processors don't help.)
If it's possible you'll be accepting many connections quickly under heavy load, up the listen queue size to that number as well. (Not sure if this is tunable in the config file - I've only heard of this problem in practice on a MS SQL server with something like 20 nodes that synchronized with NTP and started doing database stuff in a crontab. I think they just did listen(fd, 5) so that didn't work out.)
Worked for me. Before I got a GPS, I was always lost. Now that I have a GPS, I find that I am able to learn whole areas pretty easily.
I hear you (and will probably buy and use a GPS similarly myself), but your idea of "knowing a whole area" would probably be different if you went through "The Knowledge" - the intensive three-year (on average) study of every little alleyway, landmark, and traffic-related event in the core of London.
It will (eventually) be used as a replacement for study. When they start just dumping GPS-enabled cabbies out there instead of putting them through that first, I very much doubt that they'll ever learn the area as well. I don't think its value as a learning aid over the years will ever add up to the value of the intense study it replaces. And maybe that's not so terrible, if it does the job well enough without requiring the memorized knowledge.
There seems to be an assumption that people won't learn what they don't "have to" learn (I've heard this argument against PDAs too). But maybe people just learn what they're repeatedly exposed to, or things with emotional connections. Technology may or may not interfere with that. It's not a question I would guess at the answer without some evidence either way.
I generalized from similar observations:
Now that I have a cell phone with a good phonebook, I no longer memorize phone numbers. (I remember phone numbers I called 10 years ago, but I don't remember phone numbers I now call all the time. There's no need.)
Now that cashiers have cash registers, they no longer do basic arithmetic. (Sadly, most don't even remember how to do the arithmetic. They were all instructed in elementary school, but it didn't stick...)
Now that cashiers have bar code scanners, they no longer remember prices. (But they do remember the typed codes for fruit and vegetables.)
Now that I have Eclipse toolhints, I no longer remember Java library functions' argument orders.
...
In general, it seems that when it's more convenient and about as effective to use a machine as to do something by hand, people will no longer take the effort to do it themselves. And memorization (of prices, phone numbers, street names, anything) is way harder for people than for computers.
If the software works well enough that cabbies can reliably enter an address and find the street, why should cabbies be made to remember all the street names? And if it works so well that it can reliably pick an optimal route (including traffic, construction, etc.) why should they even remember how to get anywhere?
In fact, I predict they'll start depending on it before it's reliable. The test will go away, and for better or for worse, there will be a lot more cabbies out there, and they won't be able to get around very well when the computer acts up, just like a lot of businesses now can barely sell anything when their cash registers act up. It will be a pain to get to certain streets because the database is wrong, and cabbies will unknowingly avoid certain more optimal turns/intersections because the software can't navigate through them.
Do taxi drivers' brains expand to provide more memory, or do people with poor memory just forget to become taxi drivers?
A huge problem with any of these correlation studies is determining, accurately, which way the cause->effect relationship runs.
A good question, but RTFA:
Dr Maguire said: "We are now looking at the brains of taxi-drivers before they start training, and at those of retired cabbies to see whether that area of the brain gets smaller when it is not used."
Hopefully they'll actually follow the pre-training drivers through all the way through training so they don't compare future wash-outs with present successful cabbies rather than future successful cabbies with present successful cabbies. If so, it should go a long way toward answering your question.
The ultimate would be to compare the same population of cabbies vs. bus drivers (control group) through their entire careers. Obviously that'd be a long-term study, and it will become impossible when "the Knowledge" is obsoleted by GPS mapping software. (I say "when" rather than "if". It will happen sooner or later.)
Slashdot Mirror
What's Moore, they broke the last law by fabrication, too...
I can't seem to find a straight definition of "nonresident files" in the context of NTFS, but my best guess from glancing over google results is that "resident" files are ones which have their contents in a small block embedded in the inode itself. That'd be an optimization to minimize internal fragmentation on small files. If I'm right, a Windows-produced NTFS is likely to have a lot of these files. Not sure if OS X can't write to them at all, or if it will just make them non-resident when it does so.
I hear you, but I think the even bigger issue is that journalists often just hand the microphone to the two parties (frequently metaphorically; sometimes literally), without making any attempt to evaluate if what is said is true. There's certainly a place in this world for opposing viewpoints based on value judgements...but when both parties have access to essentially the same information, there shouldn't be such huge disagreements over basic facts. Often one party is mistaken or lying. It's the journalist's job to point that out (whether it's legislated or not), and many are failing.
To see exactly what I mean, watch The Daily Show. Jon Stewart, in spite of being a "fake" journalist, does a much better job of this than most. He'll frequently show a news clip where a politician will make a simple objective statement like "I have never said 'X'", then show an older news clip where the politician said 'X'. How often do you see that on Fox News, or even CNN? Journalists who do not do this are not presenting the facts in an "honest, equal, and balanced manner".
I don't think any legislation toward this goal will be successful. It's a lot more subtle than "give 50% of the airtime to viewpoint A and 50% of the airtime to viewpoint B", or even "give 95% of the airtime to the viewpoint 95% of the people have, and 5% of the airtime to the viewpoint 5% of people have".
It's also not one-size-fits-all: I expect journalists to find things obviously wrong when politicians speak, but global warming is far more subtle and beyond the understanding of the average journalist. They're not going to say "CNN was unable to reproduce expert A's Monte Carlo supercomputer simulation in our own laboratory, so expert A is lying"...there they have to scale back to the more realistic approach of checking if expert A's credentials are good, where his ideas were published, what sort of peer review his ideas have gotten, what the results of that peer review were, and who is paying for his research. It'd be nice if everyone were skilled enough to evaluate the ideas on their own merit, but that is just not to be. I don't understand global warming research, and I certainly don't expert a journalist to.
You didn't read my first post, did you?
If that's what you want, that's great for you. I, on the other hand, would be willing to pay extra for QoS. Yes, I would like my connections to be treated differently based on who I connect to - I want my VOIP traffic to be prioritized over my BitTorrent traffic and go through shorter queues, as far through my ISP's network as technically possible. Preferably for both upstream (where I can just set the TOS/DSCP header to specify the class) and downstream (in which that header can't be trusted - there's just the addresses, protocols, and ports). Please don't pass overbearing laws that forbid that.
I might also buy some discounted "when available" bandwidth. So please don't overbearing pass laws against prioritizing one customer's traffic over another's, either.
There are technical challenges in doing what I want, so I'm not sure what exact form it would take. (QoS equipment is common, but it's not like you can just have an arbitrarily long ruleset for each customer on every edge router in your network.) I think it's premature to be passing laws about it, especially ones written by lawmakers who know nothing of technology.
I think it's both. The goal is to prevent abuse, and it's a good goal. But I don't really know what the specifics of the legislation would be, and it'd be easy to throw the baby (service-based QoS) out with the bathwater. See my other post for details.
His other example (actually using packet classification and QoS based on the type of service) makes a bit more sense. I haven't really been watching the network neutrality debate, but I have wondered how this is supposed to mesh in.
Currently, I classify my own uplink to make my ssh connections still go fast while I'm running BitTorrent. (Side note: for those with cheap DSL modems that don't support doing their own QoS, I highly recommend these Linux patches. Without it, tbf shaping doesn't work because its bandwidth numbers are all wrong.) QoS is extremely effective - you can have short, usually-empty queues for interactive packets (about 20 ms in my case) while still having longer full queues (more like 500 ms) necessary for full bandwidth utilization on bulk traffic.
My ADSL upstream is my bottleneck. Network neutrality doesn't come into play, because it's under my control and only used by me.
However, it's not hard to imagine a small heavily-used ISP where their connection is the bottleneck. Maybe some customers are willing to get deprioritized at peak times in favor of cheaper service. This is much like the physical world, where people use 5-7 day shipping instead of next-day air for cheaper rates. Do existing network neutrality proposals forbid this? If they do, I think they're going too far.
If laws do allow this, it gets more complicated. Let's imagine for a second that I'm paying for 1 Mbps of interactive (short queues, prioritized, drops unlikely) bandwidth on select packets, 5 Mbps of bulk bandwidth on non-select packets or the leftovers, and everything above that is dropped. In the upstream case, it's pretty easy - I can set the TOS/DSCP field to distinguish between the two.
In the downstream case, how does classification work through their network? Apparently the telephone companies have been making noises like they're going to use preferred bandwidth if it came from an IP owned by someone who paid them money. Seems ridiculous. Let's say I'm a third-party VOIP provider. My customer has to pay their ISP extra for preferred bandwidth, I have to pay my own ISP (possibly extra for preferred bandwidth), and I have to pay my customer's ISP for preferred bandwidth, too? My customer's ISP is doing pretty well on this deal, but I'm not. I don't have much negotiating position - my customer has probably already chosen an ISP, I want to get stuff to them quickly, the customer's ISP will tell me "our way or the high way". It seems anti-competitive.
The ideal thing economically would be for the customer to control the downstream classification, too. But technically, that sounds pretty difficult - there'd have to be some way for them to say "SIP packets from Vonage to me should use my 'preferred' bandwidth if available" and have those classifiers propogate through the ISP's entire network. The ISP would have to give them some sort of a web interface or something for them to do that (a lot of integration work), and I'm trying to imagine the average consumer using it successfully...nope, can't do it.
Sounds like the product I'm working on (using an obscure 400 MHz VLIW processor), and my team is filled with developers much better and more experienced than me. But your group is not the sort I'm talking about - it's a far cry from a network card's drivers.
You're missing my point. They're appropriate for the shuttle group, where lives depend on the code. Nowhere else. The Mars Rover code is not made to this standard, and consumer code could never be. It's impossible. Anyone who points to this as an example of how we should all be writing code is simply wrong.
Agreed. If my life were depending on it, I'd want code written in this way, too. The rest of the time, I want my shiny OpenGL-accelerated windows zipping around the screen, and I realize that writing "QUALITY" in giant, bold, all-caps letters at the top of the priority list (above "not wildly exceeding our meager budget" and "write mandatory features") would make that impossible. Line-for-line, this most expensive code ever written by at least an order of magnitude.
Interesting. Thanks for the information.
An AC replied: Strange, I thought all those computers on board the Shuttle, ISS etc. were actually doing something other than an idle loop.
The point is that they wrote 420,000 lines of code in 30 years and an estimated 7,800 man-years. That's 14,000 lines per year, or 54 lines per man-year. Considering that I can single-handedly outperform their entire team of 260 people, those had better be 420,000 glorious lines of code. Furthermore, if all software were written that way, the software you know today wouldn't just be bug-free - it wouldn't exist at all.
I was much more impressed by that number before the story about avoiding having a shuttle in orbit at New Year's because the software can't handle it. That's been known for years and they haven't dared fix it. Is that counted as one of the three? No? Then they've fixed only three bugs in the last 30 years, and they have more than that, unless you think a serious misdesign is not a bug. If I confused the presence of bugs with having fixes for them, didn't consider a serious misdesign to be a bug, and had barely added a real feature in 30 years (at current head count, 7,800 man-years), I too could claim some ridiculously low bug count.
It also seems to me that the shuttle group's software situation is totally irrelevant to anyone but the shuttle group. Look at this part of the article you mentioned:
That sort of rigidity makes their methodology totally useless for software outside NASA. I occasionally hear people talk about how the Shuttle Group does software right, but for non-life critical systems, the cure is worse than the disease. Give me our full-featured, buggy software over nothing any day. There's got to be a better way.
I suspect it's also useless to the other groups in NASA. Do you actually know that the Mars Rover software was written in this manner?
Please mod the parent down as "flamebait" or "troll". Anne Thwacks either has shockingly low reading comprehension or is deliberately misrepresenting the article [*]:
The article says:
Anne Thwacks says:
First of all, you're missing the initial cost of the hardware. If the marginal cost (materials) is $10/card, there's also the cost of the hardware design, the manufacturing plant, etc. Make this same error for software and you'll think it's all basically free, since the marginal cost is just bandwidth for downloads or some pressed CDs and inked cardboard.
Secondly, there is not $100,000 of firmware for $10 hardware, even considering only the hardware's marginal cost. You're comparing graphics card firmware/driver prices (atypically high) with network card materials prices (much lower; a high-end GPU has more transistors than a CPU, and a network card...doesn't). You may think your network card firmware is worth $100,000, but it's not. Hardware people think their software is valuable because they see other people selling three man-years of software for huge amounts. Here's what they miss: those other people are good at writing software, while hardware people and poorly supervised contractors are horrible at it. In those three man-years, they'll produce code that's bad in every way you can imagine - filled with magic numbers, race conditions, deadlocks, spaghetti code, massive duplication, inefficiencies, and bizarre workarounds for bugs both in the hardware and the firmware. It's entirely free of comments or documentation. It's not even indented right. It's a miracle it ever works. Be thankful this software does not require sophisticated algorithms, or it would be entirely hopeless. It's nearly worthless - it certainly isn't well-written enough to be useful on a competitor's chipset. But for some reason, hardware people think their software is the secret sauce, so they're afraid to let anyone even distribute the binary, much less see their awful code.
Thirdly, if you'd READ THE FINE ARTICLE (properly), you'd see that most people would be happy with just distribution rights for the binary. For now, anyway. I'm sure there will come a time when people will want more of the system to be Free - so they can fix the dreadful bugs long after the manufacturer's stupid contractors have all been fired - but we have to pick our battles. We don't even have Free drivers (or specifications, with which to write our own) everywhere, so it's too soon to be insisting on having the same a level down. Get the code running on the CPU straightened out first, then the other stuff can come...
[*] Probably the former. "Never attribute to malice that which is adequately explained by incompetence."
Oh, please. Does Kerry's "I voted for the war before I voted against it" really belong in that list? Keep in mind that if "inarticulate moments" are in the same league as bribery, voter suppression, and manslaughter (the other items on your list), Bush is a much worse president than I ever realized... a few examples.
You're as much in denial as those who claim there's no corruption at all, and your beliefs are just as harmful. There's been political corruption in all parties throughout history, but not all politicians are corrupt, and not all parties are equal. Voters have recently realized that they can exert control by voting out the more corrupt party. Maybe in 20 years the Republican party will reform and the Democratic party will regress. Until then, I'm voting for Democrats.
Hard mounts wouldn't help all cases. What if the outage is longer than soft fail time * retries? And is the file lock retained over soft mount failure? And what happens if either machine goes down during this time? Or if the client process crashes? These are all failures which can occur on even a single machine, and ones which a well-thought-out file format can solve, much like the write-ahead log does for ACID databases like Oracle or PostgreSQL. But the traditional appended messages is not such a format, and NFS makes the failures likely. Bad combination...
In any case, good luck with your mail setup. I sympathize...
Yeah, that would do it. I don't use wuimapd, so I'm not familiar with mbx format, but the traditional appended message format is horrible:
I use Cyrus IMAPd. It's a different approach - all the mail on the system is owned by user cyrus. It has its own quota system. Email is stored in basically maildir format (one file per message), but with extra Berkeley DB stuff in each directory (uidvalidity/uid, full-text search indexes, imap flags, etc.). The SMTP daemon (Postfix in my case) hands the mail off to it for delivery. You only access mail through IMAP.
It works well for me. My worst problem is that Mail.app will occasionally go into this slow "synchronization" procedure where it retrieves the flags of every message to see if they've changed - that's the situation RFC 4551 should fix.
I'd suggest getting rid of the NFS export. It's holding you back, and just about every client out there (even many text-based ones like Pine and mutt) speak IMAP. Even ones which don't support the UID stuff (like really old mutt versions) should be faster, since they can at least retrieve a single message later in the session without causing it to iterate through everything before. Not corrupting the mailbox should be a nice plus, too.
But it can be solved with cheap commodity software. This is why IMAP has had UIDs in the spec since at least RFC 2060. The only case I'm aware of in which the entire mailbox still must be traversed (flag synchronization) will be solved when RFC 4551 is implemented on both the client and server side.
If you're thinking of something else, could you please give details?
I think the lawyers are just a convenient scapegoat. As I understand it, they got mad at corporations for saying "we don't have that on our server" when asked for something old but damaging but the user being able to produce it from a .pst file when it's helpful. So the lawyers said you'd better always produce everything you have.
Consequently, companies that don't want to deal with digging through .pst files (can't blame them) disallowed user backups. But here's where the problem lies - they didn't increase the mailbox sizes at the same time. They kept the draconian limits, either because of technical limitations (eased by this new Exchange version) or because they have skeletons in their server closet and don't want anyone to see them...
I'ts not just attachments - my personal email account holds over 500 megabytes of mostly text. (To be precise, the vast majority of emails are pure text; don't have numbers handy, but I think the majority of storage is taken up by text. Large emails or ones with Office attachments are rejected.) It also holds another couple gigs of spam/viruses I keep around for Bayesian filter training purposes.
Ideally in a corporate environment people put stuff (even text) on a wiki or into a repository, but it's good to be able to go back to old emails when something falls through the cracks. Why delete anything when I have the space?
I would prefer people didn't send those as email attachments at all; the alternative I discussed in a recent thread is superior. The only case it doesn't help with is revising documents with someone outside the company. Don't have a really good answer to that one, but email certainly sucks...
First, there is an obvious green light by the camera that comes on when it's active. I hope that it's hardwired so that a firmware update can't just flash that functionality away. So you can at least know something's up when the light comes on, though you can't necessarily prevent it in advance without add-on security. (Something seems wrong about taping over my shiny laptop, though Zen and the Art of Motorcycle Maintenance says it's okay to fix the motorcycle with the cheap metal from a pop can.)
But no, you're not the only person who wishes that. I'd like to go further. I wish there were a similar mechanism (at least the light, maybe a manual switch) for the microphone. I'm not sure why people are so worried about people looking at them but don't even think of the possibility that somone could be listening to them. Almost all laptops (not just new Apple ones) come with a built-in microphone, and it's only a matter of time until people start using viruses/worms/trojans to spy on conversations.
Still...if we can't trust our computers not to spy on us in the physical world, that's a pretty sad statement for eCommerce. At some point you have to type the credit card number into the computer...better make the software trustworthy...
Have you heard of a lawspeaker? Here's a good article:
I'm inclined to agree with you; our laws are the legal equivalent of spaghetti code. They're poorly crafted - too permissive in places, too restrictive in others, too complicated altogether. When most citizens break laws during the course of a normal day, something's wrong with the laws.
Agreed; that statement is a big fat lie.
As much as I hate MySQL, I can't blame it for this one. I'd say that under heavy load, PostgreSQL keeps on going fine and MySQL slows to a crawl, but both still operate.
PostgreSQL will have the same problem when misconfigured in the same way. It's pretty simple to avoid: add up the maximum sizes of all pools that connect to the database. Make the databases's maximum allowed connections at least that number + however many administrative connections you might open. If it's a huge number, you'll probably want to reduce the pool sizes for performance. (And reduce it a lot further for MySQL than for PostgreSQL, because MySQL does not scale. It does not perform under high concurrency, and additional processors don't help.)
If it's possible you'll be accepting many connections quickly under heavy load, up the listen queue size to that number as well. (Not sure if this is tunable in the config file - I've only heard of this problem in practice on a MS SQL server with something like 20 nodes that synchronized with NTP and started doing database stuff in a crontab. I think they just did listen(fd, 5) so that didn't work out.)
I hear you (and will probably buy and use a GPS similarly myself), but your idea of "knowing a whole area" would probably be different if you went through "The Knowledge" - the intensive three-year (on average) study of every little alleyway, landmark, and traffic-related event in the core of London.
It will (eventually) be used as a replacement for study. When they start just dumping GPS-enabled cabbies out there instead of putting them through that first, I very much doubt that they'll ever learn the area as well. I don't think its value as a learning aid over the years will ever add up to the value of the intense study it replaces. And maybe that's not so terrible, if it does the job well enough without requiring the memorized knowledge.
I generalized from similar observations:
In general, it seems that when it's more convenient and about as effective to use a machine as to do something by hand, people will no longer take the effort to do it themselves. And memorization (of prices, phone numbers, street names, anything) is way harder for people than for computers.
If the software works well enough that cabbies can reliably enter an address and find the street, why should cabbies be made to remember all the street names? And if it works so well that it can reliably pick an optimal route (including traffic, construction, etc.) why should they even remember how to get anywhere?
In fact, I predict they'll start depending on it before it's reliable. The test will go away, and for better or for worse, there will be a lot more cabbies out there, and they won't be able to get around very well when the computer acts up, just like a lot of businesses now can barely sell anything when their cash registers act up. It will be a pain to get to certain streets because the database is wrong, and cabbies will unknowingly avoid certain more optimal turns/intersections because the software can't navigate through them.
A good question, but RTFA:
Hopefully they'll actually follow the pre-training drivers through all the way through training so they don't compare future wash-outs with present successful cabbies rather than future successful cabbies with present successful cabbies. If so, it should go a long way toward answering your question.
The ultimate would be to compare the same population of cabbies vs. bus drivers (control group) through their entire careers. Obviously that'd be a long-term study, and it will become impossible when "the Knowledge" is obsoleted by GPS mapping software. (I say "when" rather than "if". It will happen sooner or later.)