Slashdot Mirror
Going Deep Inside Vista's Kernel Architecture
bariswheel wrote to mention an episode of 'Going Deep' on Channel 9 which takes a hard look at the architecture of Windows Vista. From the post: "Rob Short is the corporate vice president in charge of the team that architects the foundation of Windows Vista. This is a fascinating conversation with the kernel architecture team. It's our Christmas present to all of the Niners out there who've stuck with us day after day. This is a very candid interview." Topics discussed include the history of the Windows Registry, and the security/reliability of Microsoft's upcoming operating system.
My favorite is: "do you ever wish the registry had never been developed?"
Because I'm only interested if it was BALLS DEEP.
But that was the worst porn video I've ever seen. There wasn't even any nudity, but considering how these people looked (think your local linux user group visits The Gap), that was probably for the best. My rating? Totally Limp.
...good old ini files are much more easy to use (i.e. copy around, fiddle and the like)
There is no word in the English language that can't be verbed.
OK, am I the only one who has grown weary of the "oh well, another month, another insain exploit" state of mind in which windows users and admins seem to be willing to accept? Why do people just accept this, I understand a few bugs, and maybe a SINGLE large scale outbreak in something as commonplace as Windows, but this crap is just outright crazy now-a-days.
Businesses would never accept this kind of qualty from, for example, partners, suppliers, and so on, so why do they "just take" this seeminly QC-lacking products from redmond with glee?
if (defaultBrowser != MSIE || defaultMediaPlayer != WiMP || defaultMailClient != LookOut || defaultGUI != FisherPrice)
{
alert(Microsoft)
}
Heh, my "confirm you're not a script" is "issues." Not surprising.
In the words of Calvin, verbing weirds language.
Why do I get the feeling this is the programmer's equivalent of that scene in the teen slasher movies where the girl is going into the dark basement, unarmed and with nothing but a flickering candle for light?
And the brethren went away edified.
"You can't be anal retentive if you don't have an anus"
You can retain someone else's. I have several on a string around my neck. They look like calamari.
Click on link + server not responding + hosted on a microsoft server +MS publicity = slashdotted
#!/bin/bash
login root
chmod 775 universe://
Oh no, something from just over a week ago! Trash it people, its obviously of no use.
"I use a Mac because I'm just better than you are."
AC: Only on slashdot... could the sentence "My hovercraft is full of eels." be moderated "+4, Insightful
http://download.microsoft.com/download/8/1/c/81cdb 151-0aae-4f50-ab44-654b5f7ae0db/kernel_windows_vis ta_2005.wmv
Now I'm only half way through the video, but holy minimizer Batman, is that all they're doing?
So they discovered software dependencies and configuration management, error handling in the kernel, and reversed one of their previous errors - putting device drivers inside the kernel.
I'm no OS guru (I'm just an applications guy), but shouldn't they have thrown the whole mess in the garbage and started over? They're referring to the Vista kernel as "NT"!! It's freakin NT!
NT's karma has waned (especially this week). God help us - we'll be stuck with MS security holes forever.
- The Kessel run is for nerf herders. I can circumnavigate the entire Central Finite Curve in a lot less than 12 parse
It must be nice to have mainstream consumers for your main customers, rather than IT pros. You can sell 'em anything, and they'll never know it's crap, because they don't keep up with the industry
That's why I always skip all these "new Windows release" articles - they're pap. Usually just alot of mouth breathing over widgets and rather pedestrian implementations of mundane technology. Boring, and not very informative. Keeps alot of boring writers in jobs, though. Microsoft is like a 5 year jobs program for "IT Professional" writers that otherwise don't know their ass from their hat.
It would be "viri", not "virii", except that "virus" was a mass term in Latin. Also, aside from "ox"/"oxen", there are no other such plurals of nouns ending in "ox" (no "foxen").
I haven't read this anywhere yet but I did some testing today and found that Windows Vista is vulnerable to the nasty WMF dealio. I am wondering what else Microsoft is importing into Windows Vista? hmmmm
It's almost as if this MSDN interview of an MS executive on future MS technology is somehow MS related.
Well, it was precisely this sort of hype that kept Windows 3.1 at the forefront while an actual 32-bit operating system that would run existing Windows applications (better than Windows itself) actually existed. Microsoft, through various "computer" magazines (which were nothing more than MS shills), painted a beautiful picture of Chicago, through artists renderings and feature lists for features that didn't even exist. Of course, when Windows 95 finally arrived, it was a bug-ridden piece of crap, but the marketing onslaught and MS's corrupt ways of dealing with PC manufacturers destroyed OS/2. People actually willingly went for one of the most unstable operating systems that MS ever produced.
The world's burning. Moped Jesus spotted on I50. Details at 11.
The answer to one question will determine whther Vista is really an improvement in security for Windows.
Is the current test version of Vista susceptible to the .wmf exploit that is currently making the rounds on the internet?
Sorry, but what's an "american accent"?
I thought an accent was any difference in the way someone speaks compared to american english. If it sounds like american english, it's not an accent.
I'm from canada myself, but what I'm saying still applies, doesn't it ay?
# Erik
For example one of the interviews with the vista audio engine guys they talk about how Mac OSX has been a long way in front and how they are inspired by great compeditors.
They have an OS X box on the wall
And if you look at the MS Office user interface work, you can't claim that isn't innovative work
Finally if you actually watched the linked video you'd see they actually talk in depth about the flaws in the windows architecture and how they are trying to move forwards.
Narrator: Alright, so we're here for "Going Deep." We have the corporate vice president and some of his architects and they're going to talk about the Vista Kernel so, hello. Can you introduce yourselves.
Rob Short: Yeah. I'm Rob Short, and I wrote the Kernel and Architecture team for Windows. The Kernel team obviously is the core piece of a system: schedules processes and finds devices, things like that.
The Architecture Team is something that I wanted to talk a little bit about, because about two years ago, we realized that we were in a little bit more trouble in terms of our ability to predict the impact of changes and to make broad, cross-group changes to Windows, and what we decided to do was have a core group of experts that would help the teams and work right across all of windows to really help figure out the impact of changes and make sure things were happening the way we'd like to see them happen, and I have some of the people with me here today. This is just a few of the people on that team. We've about six people full-time, and we have a much broader team of about thirty architects working the different groups, and they all participate as part of our architecture team but they belong to the different teams
Narrator: Okay.
Rob Short: And the idea is to really improve our engineering process and improve our quality of our engineering and be able to predict the outcome of changes that we make.
Narrator: Okay.
Rob Short: I've been in Windows for basically ever, I've been in Windows for about fifteen years. I worked on a couple of other things in between, so I left and came back again but mostly I've been working on where the hardware meets the software.
Narrator: Excellent!
Rob Short: And I'd like to introduce my next partner in crime.
Narrator: (laughs)
Rich Neves: My name's Rich Neves. I've been working here almost three years. I work on the architecture team as Rob just described, and what my responsibility or role these days is is figuring out how to police the dependency between different pieces of the systems so that we can figure out how to compose the system in a more efficient way. By efficient, I mean in a way that isolates developers from the damage they can do to other developers. So basically, Microsoft's a very innovative company, and there's people working on amazing technologies in almost every nook and cranny, particularly in Windows. The challenge we face is delivering that innovation, and what our hope is that we can make innovation itself the bottleneck, instead of delivering innovation, which has been the problem in the past, and to do that, what we're trying to do is isolate pieces of the system from each other, so that developers can know that they can work in a particular area of the system, innovating a technology, without adversely impacting larger parts of the system, that as Rob said we can't predict they're going to be impacted, and in a way that would actually jeopardize our agility in getting those features out that result from that innovation.
So specifically what we've been doing is taking every binary in the system and assigning it a layer number, which is a rank in a directed acyclic graph. There's about 5,500 binaries in the system. And what we've been doing is getting transparency now into every dependency that developers add to any of those binaries, so that we can understand what's going on. And what's falling out of that is not necessarily just the isolation I described, but also, issues. We call them, sort of, conventional wisdom ... controversies. For example, people might be thinking, well, I want to combine a whole bunch of DLL's into one DLL for perf. Well, it turns out that that's a
So they're more or less admitting "essentially ... windows is one big binary..." Woah! Low level libraries and frameworks depending on stuff that's higher level, "in the past we've relied on... lockstep... development process..." and "we're now looking at dependencies in the 6 digits range..." Man, these guys are giving one hell of a bashing to the Microsoft codebase.
:-)
One guy starts talking about modularity and inserting features and plugins into essential services... and I thought objC. But before that another one gets all hot (I chuckled, this guy is a True Nerd, he really likes fiddling with code... congrats) about semicoop multitask where an app renices itself to 100% resource hog tier for a limited time slot (nice try, but what when all the silly apps do the same trick?), but before that there's a talk about usermode ukernel services... I thought about when I used to renice X11R6 to get better performance (when the graph card module was part of the X process).
I think Bill needs to pull out of tech and sell Microsoft to Apple. These techs are good guys, all they need is a solid process and some decent vision.
Jobs, are you reading this? Watch this video, it'll make you feel good!
e
Mi domando chi à il mandante di tutte le cazzate che faccio - Altan
The OS/2 heritage is far more complicated. There are similarities, but the kernel is quite unlike what you found in OS/2 2.x, while NT at some point could have been OS/2 3.x. It's almost as dissimilar to OS/2 as it is to Win 3.1. It was a new kernel that was supposed to be able to run both Win 3.1 and OS/2 user mode apps, so the kernel provides services suitable for that purpose. The OS/2 support was of course never fully developed, but HPFS was supported until a few years ago and NTFS also shares some ideas with it, while not in the actual disk layout.
If your DX(2?)/66 didn't perform well with NT, I would think about memory rather than CPU. Just the fact that NT is all-UNICODE in the kernel, means that every single string is longer than in, for example. OS/2 and 9x. If all you have is 4 or 8 MB, that alone can be quite significant (especially when you're running Win16 and ANSI Win32 apps and every string needs copying and conversion before really being used in the APIs).
First of all, the video is unviewable even with Microsoft Media Player on Mac, but you can find a whitepaper describing the kernel changes here. Keep in mind that all of this is basically Microsoft advertising for developers; it's not taking a "hard look" at the kernel architecture, it's the kernel developers portraying their work in the best light.
What's interesting is how little innovation there actually is. They seem to be struggling with the complexity of the system and its dependencies (5500 components)--similar to the problems Linus is having, but multiplied many times over by greater complexity of the NT system architecture. Most of their actual improvements seem to be cleanups and performance enhancements.
My impression is that the Vista kernel and system libraries are still playing catch-up with Linux in terms of modularity, performance, and functionality.
DIR SIR
GREETINGS TO YOU GOOD SIR, PRAISE GOD. MY NAME IS ABDUL-MUQADDIM, A CIVIL SERVANT IN LAGOS, AND GREAT GRAND-NEPHEW OF EXILED MICROSOFT VP ROB SHORT. BEFORE MY GREAT UNCLE'S EXILE, HE DEPOSITED $20,000,000 (TWENTY MILLION US DOLLARS) IN AN ALGERIAN BANK ACCOUNT. UPON HIS EXILE, HIS ACCOUNT WAS FROZEN AND TURNED OVER TO THE GOVERNMENT OF ALGERIA. MY FRIEND IN THE ALGERIAN GOVERNMENT WAS ABLE TO SECURE ACCESS TO THIS ACCOUNT, BUT WE NEED A MOST TRUSTWORTHY THIRD PARTY ACCOUNT INTO WHICH WE CAN TRANSFER THE FUNDS.
I AM WRITING TO YOU ON BEHALF OF MY UNCLE REGARDING THIS MOST PRIVATE AND PERSONAL MATTER. FOR YOUR COOPERATION AND ABSOLUTE CONFIDENTIALITY, WE OFFER YOU 40% (EIGHT MILLION US DOLLARS) OF THE FUNDS UPON RECEIPT OF THE TRANSACTIONS.
IN ORDER TO BEGIN OUR TRANSACTION, GOOD SIR, I HUMBLY REQUEST THAT YOU SEND $50,000 (FIFTY THOUSAND US DOLLARS) TO THE BELOW ADDRESS, SO THAT I MAY OPEN A FOREIGN ACCOUNT ON YOUR BEHALF. IN ADDITION, I REQUEST THAT YOU WATCH THE FOLLOWING VIDEO, IN ORDER TO KNOW OF MY GREAT UNCLE. YOUR IMMEDIATE RESPONSE WILL BE HIGHLY APPRECIATED.
THANK YOU, YOUR HUMBLE SERVANT
ABDUL-MUQADDIM
Narrator: Fantastic. So can you talk a little bit about what's new in the Vista kernel? So we go from XP; now we're going to Vista. So what are some of the new components?
Rob Short: A term I like to use is probably kind of politically incorrect on TV is, some of the work we do is kind of like sewers, but if we do this work incredibly well, the stuff is essential, but nobody knows that it's there.
Narrator: Yes.
Rob Short: So, if things go bad, obviously you know about it.
Narrator: Certainly.
Rob Short: Most of the work that I've been focused on for the last several years has been improving the experience where the hardware meets the software. Things like power management. We have a team of people looking at power management and working to improve how the system behaves, say a laptop for example.
If you have a laptop, how fast does it turn on, turn off, how good is the battery life? What's the experience when you dock or undock? And we've done a huge amount of work on that. We've redesigned the algorithm for hibernation so that we do a better job of figuring out which pages are already on the disk so you don't have to send more of the pages back to the disk. We've changed the way the power management interfaces to the drivers so that we have a better feel for understanding if we can just shut this thing off, right now. Today, in the older system, in XP, we actually query the driver, say, "Hey, would you, like, mind if we turn off the power?" A lot of times, people haven't coded up the driver correctly. Mostly the drivers don't care, where some really do. A disk driver, it really matters if you, you know, turn the power off in the middle of a transfer. But a lot of other things, you don't care. Mouse, it doesn't really matter that much. You know, you can go across the extreme. So we've done a bunch of work in that area.
We obviously do a lot of work in performance. One example is we had problems with heap fragmentation, and we've redesigned some of the heap algorithms so we can deal much better with much more random requests. We can deal with those and do a better job with defragmenting the heaps. So those are the types of things.
Several people--Darryl works specifically on the multimedia, and understanding how we do a better job of not having glitches in multimedia, but that also goes right through the full length of the system. It's not just buried in the kernel.
We've improved the inter-process procedure call. We have a new sort of fast, lightweight procedure call inside, in the core parts of the system. We ... stop me.
Narrator: (Laughing) He has a whole list! A cheat list!
Rob Short: There's an awful lot going on. One area where we actually make a lot of changes over time that I feel really good looking back is in the memory management area. If you think about the early NT systems, Bill Gates used to beat us up, and say, "How come you don't run in four megabytes?" And when you look at that today, and think, we're running regularly in four gigabytes today, and we have the systems in the lab that run with a terabyte of memory, the algorithms that worked back then, and the priorities back then are completely different than they are today. So we've put in work in Vista for improving the NUMA support, which is Non-Uniform Memory Access when you have a multi-processor where some of the memory is closer to some processors than to others, so we do a better job of doing the allocation, making sure that they're allocating memory that's on the CPU, near the CPU that you're running on, and then you try to run the process on the CPU where the memory actually is so you don't get cache thrashing.
Narrator: Interesting.
Rob Short: We've done some stuff for the graphics. The graphics processors today are more powerful than the CPU'
hate to say it, especially since I never downloaded the betas to this "groundbreaking" software, but isn't Vista simply another iteration of OS/2 / NT?
Not to be rude, but you need to learn a bit about OSes and OS Architecture... Especially the NT Kernel and Architecture, as it somewhat unique.
NT is the underlying technology that was designed to be the low level OS. Win32 (Windows) runs in a subsystem on top of NT. The NT architecture will be around for many many years, as it was designed to be very extensible and grow to support OSes for many more years to come.
NT is the actual OS technology, Windows and the majority of the changes of Vista are in the Win32 subsystem or truly a new subsystem that is evolved from the Win32 system, as there is a new API, Graphics Model, etc.
You see, NT doesn't even have to be Windows, it also run *nix subsystems and DOS subsystems and it even use to have a OS/2 subsystem, and they all ran side by side - being equal. (Win32 got a bit of preference as it was the base Window Manager for the other subsystems. And it has more of a role for managing NT that runs underneath it.)
Even today you can download a full blown *nix subsystem and install it on any NT based OS, like Win2k, WinXP, Win 2003, Vista, etc. It will run on top of NT just like Windows does and provide you with a full *nix OS with no emulation or vitualization and yet take advantage of the NT Kernel.
As for great new OSes, 10years from now, even a full Virtual Reality based OS that has no reference to Windows itself could be released by Microsoft and still use NT technology to run the higher level new OS.
...I'll put on the asbestos underwear for this post...
;-)
In 1990 at Microsoft there were several requirements that drove the registry. The number of third party applications and application writers was growing very fast. Making this worse, a new object system was on the horizon which could dramatically increase the number of independently-authored "components" that needed to be registered. There was a need to store state in a segregated manner so that apps wouldn't stomp on other app's information. Also there was a "new" notion of remote manageability for the objects, so the access method should be easily remotable early in the boot process. Also the OS needed a place to store lots of very small data items.
It would have been best to use the file system, but the file system at that time was FAT which could not store small data items efficiently. The registry was the first API common between Windows 3 and OS/2 (and also NT), which was a goal at the time. Of course it quickly went out of control, since there was no rational security or ownership model. The registry was kept very very simple in order to maximize the likelihood that the next file system (either the object file system or NTFS) would be able to implement it, including in the NT kernel (which had a very simple API model). It was also the first API from Microsoft that had unused parameters for future features, such as context ids for security, query features, and other stuff. Unfortunately much of that didn't work as planned since very few applications paid attention to the requirement to set them to 0L!
I didn't expect it to be so massively overused, nor for it to survive beyond Windows 3.x. It was supposed to be superceded by an object file system (that was designed and implemented several times, but never released.)
There's a good story behind the registry, though: I designed the registry while on a bachelor party for a friend, mostly on a car ride between San Diego and Las Vegas, and faxed in the design from Las Vegas the morning after the party to the responsible program manager. Which might explain much about the design...