Slashdot Mirror
Looking Back At Microsoft's Rocky History In Storage Tech
nk497 writes "Following the demise of Windows Home Server's Drive Extender, Jon Honeyball looks back on Microsoft's long, long list of storage disasters, from the dodgy DriveSpace to the Cairo Object File System, and on to the debacle that was WinFS."
I would have to include NTFS alternate data streams as well. It sounded like a good idea, but in practice it just left huge security holes.
IMHO, Microsoft worst offense in storage is drive letters, which provide no information about either the type and structure of the underlying disks or the data they contain, and have caused untold headaches from applications (and the OS itself) being reliant on paths that are arbitrarily assigned, subject to change, and often out of the user's control.
Admittedly, Microsoft didn't invent the system, but the fact that drive letters still exist in 2011 is entirely their fault.
How can I believe you when you tell me what I don't want to hear?
Because Windows Server has Active Directory and Group Policies. and Linux doesn't. Thats what sells Windows Server 2000/2003/2008. When there was a proposal to incorporate OpenLDAP auto confguration policy into KDE - it was rejected. That is why Linux lost the war for the Enterprise desktop.
It's not just closed source, but closed standard. Microsoft keeps the specification officially secret (Though I believe you can see if it you agree to an agreemet saying you won't disclose or actually impliment it). That linux can use NTFS is a tribute to many hours of dedicated reverse-engineering and various tidbits of information that escaped until a full picture could be assembled,
What's wrong with NTFS?
Many, many things. The MFT (Master File Table) which contains all your file system info... may also contain actual files if they are small enough to fit into the remaining space (i.e. disk block sizes matter). Why is this a problem? file wiping software usually won't go near the MFT since you twiddle the wrong bits in the MFT and the entire file system is likely to go poof.
The fact that NTFS doesn't happen to conform to a particular naive model of "how file systems work", namely "all file data is stored in blocks separate from file headers", might be an annoyance to developers of third-party file wiping software, but I wouldn't consider that to be something "wrong" with it. If Microsoft don't supply a file-wiping API that allows user-mode code to request that all the data in a file be wiped, regardless of how that data might happen to be stored for particular files on a particular version of a particular file system, one might consider that to be something "wrong" with Windows, but that's another matter.
Oh the MFT is also basically not documented properly, I actually had an MS employee that owed me a favor try to find the docs on it and he couldn't.
That's not a problem with NTFS, it's a problem with Microsoft's software development practices (although it's probably not the only piece of software for which the documentation of the data structures lags behind the current data structures).
NTFS still doesn't have shared cluster filesystem capability. This has a bunch of flow-on effects, which basically means that Windows Server clusters are actually "Failover Clusters". The key part of that being the "Fail".
Really basic services like the file shares are impossible to make truly highly available using Windows, because neither NTFS nor SMB support transparent fail-over of open files. There isn't even a way of doing a clean administrative cluster fail-over, such as a drain-stop. The only option is forcibly closing all open files, potentially corrupting user data, and forcing users to click through dirty error messages that their PCs may or may not recover from.
I've tried things like Polyserve, which is a third-party filesystem that has proper cluster support, but it's still hamstrung by SMB. What's doubly ridiculous is that Microsoft basically re-engineered SMB for Vista, and called it "SMB2", but it still can't do clean fail-over!
Similarly, SQL Server can't do proper failover of cluster nodes, nor can it do proper active-active database clusters that share a single database file, because of the limitations of the underlying filesystem. It can no active-active clustering for read-only files, but that's only rarely useful.
Even within Microsoft, workarounds had to be found to make some of their key products somewhat resilient. Both SQL Server and Exchange now use software mirroring for cleaner failover. Ignoring the cost of having to purchase twice as much disk, mirroring has other issues too, like becoming bottle-necked by the network speed, or limiting the features that can be used. For example, if your application performs queries across two databases in a single query, then you can't use Mirroring, because there's no way to specify that the two databases should fail over in a group.
VMware has become a multi-billion dollar company in a few short years because a single non-clustered Windows Server on a VMware cluster is more robust than a cluster of Windows Servers!
"Enterprise Edition" my ass.
Would you mind explaining carefully and precisely why you think that OS X's filesystem (and others) aren't prone to fragmentation? It's true that many filesystems incorporate techniques to reduce the likelihood and effect of fragmentation, but it still happens, and it's still possible to optimise the position of data on rotating media - as any good defragmenter will do.
Filesystems which claim not to suffer from defragmentation concern me more because people end up not noticing the decrease in performance over time. For a machine not in 24/7 operation, a scheduled defrag run is always a good idea; otherwise, slowly doing the same during less busy moments should be mandatory.
How about getting the directory separator wrong? This has indirectly led to a generation of TV and radio presenters having to say "forward slash" when reading out URLs...
In a survey of 100 programmers, 111111 thought that duck-typing was a good idea.
I have used LVM2 now for two years with my various notebooks and netbooks. They had various crashes and power downs but I never loosed one bit of data. My small home server is using LVM2 as well with my 3 USB hard disks, serves videos and music to my home.
With my notebooks and netbooks I can grow or shrink my root or home partition and with my server I can just plug in another USB hard disk and grow my partition. No fuss not complicated at all and works all the time.
All that for free, just download Fedora, Debian or Ubuntu and install it in 10 minutes. If you want, setup a FTP server, apache server or what ever you like. Or you get what you pay for with Windows for 100$ or more.
http://www.mueller-public.de - My site http://www.anr-institute.com/ - Advanced Natural Research Institute
drive extender's "features" that Microsoft couldn't get to work right include stuff that has been standard in the Unix world for over a decade
"Drive extender" in WHS is, essentially, RAID that can be hot-extended - you plug a new hard drive into your server, and your storage (which shows up as a single disk) grows immediately. I don't know about the Unix world, but I certainly haven't seen any Linux distro that was configured that way. Can you name some names?
Please name a linux based solution, apart from 100% proprietary Unraid, which allow for me to do what drive extender does. I'm serious. I refuse to install WHS, and thus far the closest I can find is going Unraid, which feels dirty to me, or nexentastor.
What drive extender does, in a nutshell:
-all of your hard drives show up as one big storage pool.
-100% of disparate drive sizes can be used (excluding copies/parity obviously). So if you have 3 old 1tb drives, 2 old 1.5tb drives, and 1 2tb drive you'll have 8gb of storage
-configurable redundancy such that any single disk failure, no matter the size, all files are still available
-if two drives fail, you only lose the files that were on those two drives, not the entire array
-take any one drive out of the array, plug it into ANY windows vista or higher PC (new NTFS version), have access to all the files that were stored in that drive.
-add a drive, get that much more storage (excluding copies/parity obviously)
ZFS comes DAMNED close, but you cannot grow the number of disks in a raidz array, you have to add an entire extra array (meaning 3+ disks) to the pool. You also lose the entire array if 2 (or 3 with raidz2, or 4 with raidz3) disks die, and cannot have direct file access just by plugging in 1 disk of the array, but that honestly doesn't bother me that much.
Oh, and ZFS isn't on linux except through fuse.
There is no advantage to "control of the Windows System Registry" except for windows users, so your logic appears a bit circular. You do know that there are Linux LDAP servers, right? Because so often discussing technology with microsofties is like trying to talk about good food with McD's addicts. They often have no frame of reference with which to discuss these things.
Caveat Utilitor
Surely that sort of thing has been done in UNIX using .rc files and other . configuration files in the user's home directory since the mid-70s, or does KDE ignore those?
Nobody is saying you can't do that stuff on *NIX but its hard to do that on stand alone machines. When you are talking about shared machines or terminals where everything can be handled with NIS and home directories reside on an NFS share used by all hosts the facilities to manage user experience exist.
As soon as you start having laptops and desktops running all around the office you can't manage the settings the user is talking about anymore. Yes you can do it at deployment time. Sure you could write init scripts to go fetch and overwrite/update rc files and stuff but you'd have to do all of it yourself and it would be a security nightmare to try and get correct without putting a lot of resources into it.
GPOs make it really easy change all the CSRs home pages to the new customer service portal, and set all the sales reps wall paper to the latest product sheet instead of their embarrassing personal photos any time its needed. It also makes it possible to do things like yes your screen saver is going to turn on and the desktop will be locked after 15min, no exceptions. Sometimes that sort of thing gets required for PayCardIndustry rules and the like, and those things change every now and then.
Got a way for me to change your screen saver settings on every Ubuntu box in the company? Yes I know I can run a sed script to go into each home directory and alter the config file for whatever desktop environment is being used, I still have to find away to do it to every box.
Trust me I have been doing this for some years and this is one place where Windows gets it right, so right in fact that it in some ways justifies the use of Windows even though its otherwise a really inferior platform.
Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
Sweet, so LVM provides redundancy? Or I still need to use software raid for that, which reqires all disks to be the same size in order to get full usage out of them?
And if I create an LVM array, and 1 disk dies, I no longer lose everything, the filesystem is easily mounutable and I only lose the files that were on that disk and weren't redundant (which you've also assured me LVM handles for you?)
And who mentioned drive letters? You're telling me that every unix will combine all of my storage devices into one pool, as opposed to having to mount them discretely in mount points? So if I have 5 disks, by default all of my files, regardless of location in the filesystem, will get nicely distributed across said disks? That's great to know as well! Last time I checked (about 2 seconds ago, from the ubuntu box I'm posting this from) you have to choose a mount point for any volume (logical or physical), and it only provides storage to that section of the filesystem. If my /var/log is full, and I just throw in another 1tb disk, /var/log does not get access to that new storage.
I asked a serious question. I really am interested in a set of technologies that have the same capabilities as unraid (which is linux based but NOT open nor free) and drive extender. LVM and software raid are in no way comparable.
And how many technologies fail on FOSS systems. I just got a patch last year that allowed my wifi to connect in under 5 minutes. While Mac and Windows did it very fast for year. (2002 from my experience). Sure we like to see Microsoft fail because it is a huge company and has a strong foothold in our technology, and during the 90's it seemed to the media it could do no wrong. But looking at it's failures and saying Microsoft is all bad while I zealiously promote an other product ignoring it deficiencies. Is really a stupid idea that doesn't help either side.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
Yes, well it works very well in the *nix world -- the NT failures you cite do not affect us at all.
Huh? No-one can seriously brag about the *registry*, it's the Achilles' Heel of windows! This can only mean you've never seriously used any other OS.
MS has indoctrinated you well, but you don't appear to know enough about *nix to really participate in this discussion. Of course, it can easily be argued that I do not know enough about windows to participate in it, so perhaps we're even :)
Caveat Utilitor
Honestly.... this argument is stupid, Group Policy arose because on Windows everything is a COM object with an ACL and it was neigh impossible to manage to provide even a modicum of security without some sort of system policy at a high level. Linux of course doesn't need this because it operates in a fundamentally different manner where everything is a file and the file system permissions (group based) determine if a is executable or not. Thus the Linux kernel doesn't need to know what specific COM+ handler needs to be loaded, but rather if a file is a supported executable format or not, and what to do from there. Both systems have fundamental advantages, Linux is deceptively simple leading to a power on the command line that is daunting for many users. Whereas Windows can be easy worked with to extend using COM and the registry (The registry was never designed to hold most of the crap that people shove in there... it was designed to be a central repository of information for COM objects).
If anything this model shows MS's lack of foresight into the importance of networking and their focus on the single standalone box.
Any and all content posted above may be ignored, considered irrelevant, or otherwise dismissed.
Stop awfulizing it -- scripting this is elementary in *.nix, and if you can't do it you're not a competent *.nix admin, period full stop.
Caveat Utilitor
The article does a fairly thorough job of roasting MS over their lack of internal coordination, outlining how one wing starts to work on a new technology and other departments that need to get on board "wanted nothing to do with it'. In any well-managed company, a department that refuses to get on board with a new technology gets hell rained down on them from above until they fall into line.
Take Apple's "spotlight" meta search feature for example. Imagine the team working on the AddressBook app "wanted nothing to do with it"? There'd be hell to pay, and either team managers would change their tune or get replaced. In a large project like an operating system, lack of cooperation simply cannot be tolerated. But it seems that MS is just so large at this point that it doesn't have the power to guarantee their different projects cooperate fully with each other.
I have read from time to time that there was this sort of internal battle going on at MS, where different projects worked in isolation and there was infighting, but I'd never really seen the effects of these issues before. It's interesting to see the result. This appears to be an upper management or communications problem. Whoever is above the Outlook team needs to be asking that team manager "so how's integration with drive extender going?" If they get foot-dragging and complaining and brush-offs, that manager needs to be dragged into the director's office for some "re-education" on cohesive development. If the director isn't asking these questions, THEY need to be replaced. Something of this sort is isn't working properly at MS.
Its like a construction project. You've got all these separate units coming in, doing electrical, plumbing, structural, heating, floors. The general contractor has to make sure these people work together. Refusing to cooperate with one of the other groups simply cannot be tolerated, and it's the GC's responsibility to make sure everything works smoothly. Problems between groups need to be brought to the GC, and the GC needs to settle them immediately. Otherwise the finished building has serious problems. You can't just turn over the house to the owner and say "Oh by the way we removed the heating from the bathroom. The plumbers wouldn't route the pipes around where the heating ducts needed to go. You don't REALLY need heat in such a small room anyway." But that's the sort of thing that MS is pulling from time to time.
I think MS is just taking the cowardly way out. "We can't control our own internal development processes well enough to get this feature integrated properly in with the rest of our technology, so we're just canceling it." The article states simply that companies like Dropbox and DataRobotics (makers of Drobo) that have only one core technology are forced to "get it right", because dropping it simply isn't an option. MS seems to think they have the option to just drop any feature at any time on a whim if it's not going well, instead of going to the additional effort of kicking some butts and making it work. It's not like its an impossible task. This is doable. They just lack the necessary internal management to pull it off consistently.
Bottom line: At MS, with any new project, unless all the key players decide to get on board, the project is doomed.
In other words, the Outlook team manager should not be capable of tanking Drive Extender. But it is, and it did. And THAT is a serious internal management problem that MS has demonstrated over and over. Something's gotta change.
I work for the Department of Redundancy Department.
The logic is fine, the terminology is poor.
Group Policy provides a mechanism and comprehensive set of capabilities for central configuration management of Windows machines. "Control of the Registry" is merely an irrelevant implementation semantic.
Central configuration management is - amazingly, given its age - functionality significantly lacking from Linux/UNIX. IME this is largely due to traditional Linux/UNIX admins taking great pride in how awesome they are at continually reinventing the wheel (and castigating those who try to minimise it - the common complaint being it would make things "too easy").