Slashdot Mirror
Going Deep Inside Vista's Kernel Architecture
bariswheel wrote to mention an episode of 'Going Deep' on Channel 9 which takes a hard look at the architecture of Windows Vista. From the post: "Rob Short is the corporate vice president in charge of the team that architects the foundation of Windows Vista. This is a fascinating conversation with the kernel architecture team. It's our Christmas present to all of the Niners out there who've stuck with us day after day. This is a very candid interview." Topics discussed include the history of the Windows Registry, and the security/reliability of Microsoft's upcoming operating system.
My favorite is: "do you ever wish the registry had never been developed?"
But that was the worst porn video I've ever seen. There wasn't even any nudity, but considering how these people looked (think your local linux user group visits The Gap), that was probably for the best. My rating? Totally Limp.
In the words of Calvin, verbing weirds language.
...good old ini files are much more easy to use (i.e. copy around, fiddle and the like)
That will also make applications easier to port. Something Microsoft doesn't want. Registry is a good lock-in tool for Microsoft.
"You can't be anal retentive if you don't have an anus"
You can retain someone else's. I have several on a string around my neck. They look like calamari.
AC: Only on slashdot... could the sentence "My hovercraft is full of eels." be moderated "+4, Insightful
http://download.microsoft.com/download/8/1/c/81cdb 151-0aae-4f50-ab44-654b5f7ae0db/kernel_windows_vis ta_2005.wmv
Now I'm only half way through the video, but holy minimizer Batman, is that all they're doing?
So they discovered software dependencies and configuration management, error handling in the kernel, and reversed one of their previous errors - putting device drivers inside the kernel.
I'm no OS guru (I'm just an applications guy), but shouldn't they have thrown the whole mess in the garbage and started over? They're referring to the Vista kernel as "NT"!! It's freakin NT!
NT's karma has waned (especially this week). God help us - we'll be stuck with MS security holes forever.
- The Kessel run is for nerf herders. I can circumnavigate the entire Central Finite Curve in a lot less than 12 parse
Hmm... Well, assuming you have the source and are ready to start porting code, it's just about changing the behavior of a number of well documented API calls. You can make a library out of it with your own preferred behavior to make the code reusable. Actually, I'd be surprised if someone hadn't already done so and posted it somewhere on the web.
It's hardly a lock-in method when it's both documented methods and it's easy to find out what happens -- the Windows registry is hardly rocket science, but more like a tree of settings that can have a few different data types.
Beware: In C++, your friends can see your privates!
I haven't read this anywhere yet but I did some testing today and found that Windows Vista is vulnerable to the nasty WMF dealio. I am wondering what else Microsoft is importing into Windows Vista? hmmmm
Well, it was precisely this sort of hype that kept Windows 3.1 at the forefront while an actual 32-bit operating system that would run existing Windows applications (better than Windows itself) actually existed. Microsoft, through various "computer" magazines (which were nothing more than MS shills), painted a beautiful picture of Chicago, through artists renderings and feature lists for features that didn't even exist. Of course, when Windows 95 finally arrived, it was a bug-ridden piece of crap, but the marketing onslaught and MS's corrupt ways of dealing with PC manufacturers destroyed OS/2. People actually willingly went for one of the most unstable operating systems that MS ever produced.
The world's burning. Moped Jesus spotted on I50. Details at 11.
Narrator: Alright, so we're here for "Going Deep." We have the corporate vice president and some of his architects and they're going to talk about the Vista Kernel so, hello. Can you introduce yourselves.
Rob Short: Yeah. I'm Rob Short, and I wrote the Kernel and Architecture team for Windows. The Kernel team obviously is the core piece of a system: schedules processes and finds devices, things like that.
The Architecture Team is something that I wanted to talk a little bit about, because about two years ago, we realized that we were in a little bit more trouble in terms of our ability to predict the impact of changes and to make broad, cross-group changes to Windows, and what we decided to do was have a core group of experts that would help the teams and work right across all of windows to really help figure out the impact of changes and make sure things were happening the way we'd like to see them happen, and I have some of the people with me here today. This is just a few of the people on that team. We've about six people full-time, and we have a much broader team of about thirty architects working the different groups, and they all participate as part of our architecture team but they belong to the different teams
Narrator: Okay.
Rob Short: And the idea is to really improve our engineering process and improve our quality of our engineering and be able to predict the outcome of changes that we make.
Narrator: Okay.
Rob Short: I've been in Windows for basically ever, I've been in Windows for about fifteen years. I worked on a couple of other things in between, so I left and came back again but mostly I've been working on where the hardware meets the software.
Narrator: Excellent!
Rob Short: And I'd like to introduce my next partner in crime.
Narrator: (laughs)
Rich Neves: My name's Rich Neves. I've been working here almost three years. I work on the architecture team as Rob just described, and what my responsibility or role these days is is figuring out how to police the dependency between different pieces of the systems so that we can figure out how to compose the system in a more efficient way. By efficient, I mean in a way that isolates developers from the damage they can do to other developers. So basically, Microsoft's a very innovative company, and there's people working on amazing technologies in almost every nook and cranny, particularly in Windows. The challenge we face is delivering that innovation, and what our hope is that we can make innovation itself the bottleneck, instead of delivering innovation, which has been the problem in the past, and to do that, what we're trying to do is isolate pieces of the system from each other, so that developers can know that they can work in a particular area of the system, innovating a technology, without adversely impacting larger parts of the system, that as Rob said we can't predict they're going to be impacted, and in a way that would actually jeopardize our agility in getting those features out that result from that innovation.
So specifically what we've been doing is taking every binary in the system and assigning it a layer number, which is a rank in a directed acyclic graph. There's about 5,500 binaries in the system. And what we've been doing is getting transparency now into every dependency that developers add to any of those binaries, so that we can understand what's going on. And what's falling out of that is not necessarily just the isolation I described, but also, issues. We call them, sort of, conventional wisdom ... controversies. For example, people might be thinking, well, I want to combine a whole bunch of DLL's into one DLL for perf. Well, it turns out that that's a
DIR SIR
GREETINGS TO YOU GOOD SIR, PRAISE GOD. MY NAME IS ABDUL-MUQADDIM, A CIVIL SERVANT IN LAGOS, AND GREAT GRAND-NEPHEW OF EXILED MICROSOFT VP ROB SHORT. BEFORE MY GREAT UNCLE'S EXILE, HE DEPOSITED $20,000,000 (TWENTY MILLION US DOLLARS) IN AN ALGERIAN BANK ACCOUNT. UPON HIS EXILE, HIS ACCOUNT WAS FROZEN AND TURNED OVER TO THE GOVERNMENT OF ALGERIA. MY FRIEND IN THE ALGERIAN GOVERNMENT WAS ABLE TO SECURE ACCESS TO THIS ACCOUNT, BUT WE NEED A MOST TRUSTWORTHY THIRD PARTY ACCOUNT INTO WHICH WE CAN TRANSFER THE FUNDS.
I AM WRITING TO YOU ON BEHALF OF MY UNCLE REGARDING THIS MOST PRIVATE AND PERSONAL MATTER. FOR YOUR COOPERATION AND ABSOLUTE CONFIDENTIALITY, WE OFFER YOU 40% (EIGHT MILLION US DOLLARS) OF THE FUNDS UPON RECEIPT OF THE TRANSACTIONS.
IN ORDER TO BEGIN OUR TRANSACTION, GOOD SIR, I HUMBLY REQUEST THAT YOU SEND $50,000 (FIFTY THOUSAND US DOLLARS) TO THE BELOW ADDRESS, SO THAT I MAY OPEN A FOREIGN ACCOUNT ON YOUR BEHALF. IN ADDITION, I REQUEST THAT YOU WATCH THE FOLLOWING VIDEO, IN ORDER TO KNOW OF MY GREAT UNCLE. YOUR IMMEDIATE RESPONSE WILL BE HIGHLY APPRECIATED.
THANK YOU, YOUR HUMBLE SERVANT
ABDUL-MUQADDIM
Fix whats there!
A long term plan for fixing the underlying architecture problems is as important as maintaining the current release... otherwise you're just turd polishing (which is more expensive to Redmond & the end users in the long run). System Architects and QA are almost apples and oranges too.
Not flame, genuine curiosity from a 20 year old IT major. Why do people just accept this... Businesses would never accept this kind of qualty from, for example, partners, suppliers, and so on, so why do they "just take" this seeminly QC-lacking products from redmond with glee?
I really don't think there are that many people drinking the MS kool aid. People have been switching to Apple desktops and *nix servers fairly steadily, but you're not going to see an overnight change because the cost of migration is so high
I mean for home users, it boils down to a Wintel system or an Apple... if you're buying a new system its an easy choice IMHO, but what does an unhappy windows user do if they have nice x86 hardware? What do you really expect non-tech-savy users to do when presented with the options of (a) selling their current sytem at a loss and buy new hardware, (b) really making an effort educate themselves for the purpose of switching to an OS with little-to-no commercial apps/games/tech support, mediocre media playback, and a clunky UI (no, I'm not hating on Linux. Fantastic workstation/server, craptacular home desktop) or (c) just accept it & hit the reset button/ bust out the system recovery disk every now and then until it's time for a new box (or a stable release comes out).
For buisnesses, migrating workstations/servers is only possible if the application support is present, and you have the cost of re-training. Porting any custom C#/ASP/MSSQL/etc to cross-platform solutions is time consuming and software developers are expensive, ditto with *nix sysadmins. Not to mention the fact that any good Windows should be able to eliminate (or at least mitigate) the threat of said security flaws.
If you already paid for WinXP, why the hell should you have to pay AGAIN for the "security" that was supposed to be there...and in 2k, NT4, yadda yadda yadda?
Well I'm not exactly a MS fan, but I don't think its quite so sinister. Old versions (even pirated versions) are entitled to security patches for a few years, which is pretty reasonable. To expect lifelong upgrades for free is asking a bit much though. I mean, I expect Honda to issue recalls on any safety issues on my Accord, but don't angry when they won't retrofit it with a hybrid engine.
Narrator: Fantastic. So can you talk a little bit about what's new in the Vista kernel? So we go from XP; now we're going to Vista. So what are some of the new components?
Rob Short: A term I like to use is probably kind of politically incorrect on TV is, some of the work we do is kind of like sewers, but if we do this work incredibly well, the stuff is essential, but nobody knows that it's there.
Narrator: Yes.
Rob Short: So, if things go bad, obviously you know about it.
Narrator: Certainly.
Rob Short: Most of the work that I've been focused on for the last several years has been improving the experience where the hardware meets the software. Things like power management. We have a team of people looking at power management and working to improve how the system behaves, say a laptop for example.
If you have a laptop, how fast does it turn on, turn off, how good is the battery life? What's the experience when you dock or undock? And we've done a huge amount of work on that. We've redesigned the algorithm for hibernation so that we do a better job of figuring out which pages are already on the disk so you don't have to send more of the pages back to the disk. We've changed the way the power management interfaces to the drivers so that we have a better feel for understanding if we can just shut this thing off, right now. Today, in the older system, in XP, we actually query the driver, say, "Hey, would you, like, mind if we turn off the power?" A lot of times, people haven't coded up the driver correctly. Mostly the drivers don't care, where some really do. A disk driver, it really matters if you, you know, turn the power off in the middle of a transfer. But a lot of other things, you don't care. Mouse, it doesn't really matter that much. You know, you can go across the extreme. So we've done a bunch of work in that area.
We obviously do a lot of work in performance. One example is we had problems with heap fragmentation, and we've redesigned some of the heap algorithms so we can deal much better with much more random requests. We can deal with those and do a better job with defragmenting the heaps. So those are the types of things.
Several people--Darryl works specifically on the multimedia, and understanding how we do a better job of not having glitches in multimedia, but that also goes right through the full length of the system. It's not just buried in the kernel.
We've improved the inter-process procedure call. We have a new sort of fast, lightweight procedure call inside, in the core parts of the system. We ... stop me.
Narrator: (Laughing) He has a whole list! A cheat list!
Rob Short: There's an awful lot going on. One area where we actually make a lot of changes over time that I feel really good looking back is in the memory management area. If you think about the early NT systems, Bill Gates used to beat us up, and say, "How come you don't run in four megabytes?" And when you look at that today, and think, we're running regularly in four gigabytes today, and we have the systems in the lab that run with a terabyte of memory, the algorithms that worked back then, and the priorities back then are completely different than they are today. So we've put in work in Vista for improving the NUMA support, which is Non-Uniform Memory Access when you have a multi-processor where some of the memory is closer to some processors than to others, so we do a better job of doing the allocation, making sure that they're allocating memory that's on the CPU, near the CPU that you're running on, and then you try to run the process on the CPU where the memory actually is so you don't get cache thrashing.
Narrator: Interesting.
Rob Short: We've done some stuff for the graphics. The graphics processors today are more powerful than the CPU'
...I'll put on the asbestos underwear for this post...
;-)
In 1990 at Microsoft there were several requirements that drove the registry. The number of third party applications and application writers was growing very fast. Making this worse, a new object system was on the horizon which could dramatically increase the number of independently-authored "components" that needed to be registered. There was a need to store state in a segregated manner so that apps wouldn't stomp on other app's information. Also there was a "new" notion of remote manageability for the objects, so the access method should be easily remotable early in the boot process. Also the OS needed a place to store lots of very small data items.
It would have been best to use the file system, but the file system at that time was FAT which could not store small data items efficiently. The registry was the first API common between Windows 3 and OS/2 (and also NT), which was a goal at the time. Of course it quickly went out of control, since there was no rational security or ownership model. The registry was kept very very simple in order to maximize the likelihood that the next file system (either the object file system or NTFS) would be able to implement it, including in the NT kernel (which had a very simple API model). It was also the first API from Microsoft that had unused parameters for future features, such as context ids for security, query features, and other stuff. Unfortunately much of that didn't work as planned since very few applications paid attention to the requirement to set them to 0L!
I didn't expect it to be so massively overused, nor for it to survive beyond Windows 3.x. It was supposed to be superceded by an object file system (that was designed and implemented several times, but never released.)
There's a good story behind the registry, though: I designed the registry while on a bachelor party for a friend, mostly on a car ride between San Diego and Las Vegas, and faxed in the design from Las Vegas the morning after the party to the responsible program manager. Which might explain much about the design...