Going Deep Inside Vista's Kernel Architecture
bariswheel wrote to mention an episode of 'Going Deep' on Channel 9 which takes a hard look at the architecture of Windows Vista. From the post: "Rob Short is the corporate vice president in charge of the team that architects the foundation of Windows Vista. This is a fascinating conversation with the kernel architecture team. It's our Christmas present to all of the Niners out there who've stuck with us day after day. This is a very candid interview." Topics discussed include the history of the Windows Registry, and the security/reliability of Microsoft's upcoming operating system.
Can someone post a transcript please?
My favorite is: "do you ever wish the registry had never been developed?"
Because I'm only interested if it was BALLS DEEP.
But that was the worst porn video I've ever seen. There wasn't even any nudity, but considering how these people looked (think your local linux user group visits The Gap), that was probably for the best. My rating? Totally Limp.
...good old ini files are much more easy to use (i.e. copy around, fiddle and the like)
Only when used as a plural noun.
There is no word in the English language that can't be verbed.
OK, am I the only one who has grown weary of the "oh well, another month, another insain exploit" state of mind in which windows users and admins seem to be willing to accept? Why do people just accept this, I understand a few bugs, and maybe a SINGLE large scale outbreak in something as commonplace as Windows, but this crap is just outright crazy now-a-days.
Businesses would never accept this kind of qualty from, for example, partners, suppliers, and so on, so why do they "just take" this seeminly QC-lacking products from redmond with glee?
if (defaultBrowser != MSIE || defaultMediaPlayer != WiMP || defaultMailClient != LookOut || defaultGUI != FisherPrice)
{
alert(Microsoft)
}
Heh, my "confirm you're not a script" is "issues." Not surprising.
In the words of Calvin, verbing weirds language.
Why do I get the feeling this is the programmer's equivalent of that scene in the teen slasher movies where the girl is going into the dark basement, unarmed and with nothing but a flickering candle for light?
And the brethren went away edified.
"You can't be anal retentive if you don't have an anus"
You can retain someone else's. I have several on a string around my neck. They look like calamari.
Click on link + server not responding + hosted on a microsoft server +MS publicity = slashdotted
#!/bin/bash
login root
chmod 775 universe://
Oh no, something from just over a week ago! Trash it people, its obviously of no use.
"I use a Mac because I'm just better than you are."
Microsoft has been releasing a lot of Vista video "interviews" and tech intros lately. If you believed what they're trying to sell you, you would easily think that the Microsoft Vista teams are developing ground-breaking new technology for the benefit of us all.
However, any remotely circumspect look at them will reveal that they're carefully choreographed attempts to show microsoft as a powerhouse with new ideas behind every corner... i.e., "Ohh look, here's Joe, the guy responsible for all this, right behind the camera...". What's more, they're basically doing what they've always done, stealing other peoples' technology and claiming is as their own, in the process. One of these videos, for instance, is all about microsoft's new printing architecture, which is basically just a rip-off of postscript. Microsoft is finally catching up, and yet they tell their customers that they're doing new stuff.
It must be nice to have mainstream consumers for your main customers, rather than IT pros. You can sell 'em anything, and they'll never know it's crap, because they don't keep up with the industry.
http://download.microsoft.com/download/8/1/c/81cdb 151-0aae-4f50-ab44-654b5f7ae0db/kernel_windows_vis ta_2005.wmv
Now I'm only half way through the video, but holy minimizer Batman, is that all they're doing?
So they discovered software dependencies and configuration management, error handling in the kernel, and reversed one of their previous errors - putting device drivers inside the kernel.
I'm no OS guru (I'm just an applications guy), but shouldn't they have thrown the whole mess in the garbage and started over? They're referring to the Vista kernel as "NT"!! It's freakin NT!
NT's karma has waned (especially this week). God help us - we'll be stuck with MS security holes forever.
- The Kessel run is for nerf herders. I can circumnavigate the entire Central Finite Curve in a lot less than 12 parse
It would be "viri", not "virii", except that "virus" was a mass term in Latin. Also, aside from "ox"/"oxen", there are no other such plurals of nouns ending in "ox" (no "foxen").
I haven't read this anywhere yet but I did some testing today and found that Windows Vista is vulnerable to the nasty WMF dealio. I am wondering what else Microsoft is importing into Windows Vista? hmmmm
It's almost as if this MSDN interview of an MS executive on future MS technology is somehow MS related.
I can't believe that we /.'ed Microsoft!
. torrent
I just posted the torrent, enjoy:
http://64.226.48.88/kernel_windows_vista_2005.wmv
Good security is based upon reality and common sense. Common sense is a function of having common knowledge.
The answer to one question will determine whther Vista is really an improvement in security for Windows.
Is the current test version of Vista susceptible to the .wmf exploit that is currently making the rounds on the internet?
Fri, Dec 23, 2005 6:16 PM
Over a week old? It should have been duped by now.
"Kittens give Morbo gas!"
Sorry, but what's an "american accent"?
I thought an accent was any difference in the way someone speaks compared to american english. If it sounds like american english, it's not an accent.
I'm from canada myself, but what I'm saying still applies, doesn't it ay?
# Erik
I could barely hear the guy and the other architects were nudging him a little about being so quiet. I wonder why;)?
recent news (oxy moron? Isn't all news recent?)
"old news" would be an oxymoron.
"recent news" is redundant.
If the tool will "gimmee" enough, I could care less whether it was created by Apple or Microsoft or Walmart. Merit trumps all.
Even ethics and the law?
Narrator: Alright, so we're here for "Going Deep." We have the corporate vice president and some of his architects and they're going to talk about the Vista Kernel so, hello. Can you introduce yourselves.
Rob Short: Yeah. I'm Rob Short, and I wrote the Kernel and Architecture team for Windows. The Kernel team obviously is the core piece of a system: schedules processes and finds devices, things like that.
The Architecture Team is something that I wanted to talk a little bit about, because about two years ago, we realized that we were in a little bit more trouble in terms of our ability to predict the impact of changes and to make broad, cross-group changes to Windows, and what we decided to do was have a core group of experts that would help the teams and work right across all of windows to really help figure out the impact of changes and make sure things were happening the way we'd like to see them happen, and I have some of the people with me here today. This is just a few of the people on that team. We've about six people full-time, and we have a much broader team of about thirty architects working the different groups, and they all participate as part of our architecture team but they belong to the different teams
Narrator: Okay.
Rob Short: And the idea is to really improve our engineering process and improve our quality of our engineering and be able to predict the outcome of changes that we make.
Narrator: Okay.
Rob Short: I've been in Windows for basically ever, I've been in Windows for about fifteen years. I worked on a couple of other things in between, so I left and came back again but mostly I've been working on where the hardware meets the software.
Narrator: Excellent!
Rob Short: And I'd like to introduce my next partner in crime.
Narrator: (laughs)
Rich Neves: My name's Rich Neves. I've been working here almost three years. I work on the architecture team as Rob just described, and what my responsibility or role these days is is figuring out how to police the dependency between different pieces of the systems so that we can figure out how to compose the system in a more efficient way. By efficient, I mean in a way that isolates developers from the damage they can do to other developers. So basically, Microsoft's a very innovative company, and there's people working on amazing technologies in almost every nook and cranny, particularly in Windows. The challenge we face is delivering that innovation, and what our hope is that we can make innovation itself the bottleneck, instead of delivering innovation, which has been the problem in the past, and to do that, what we're trying to do is isolate pieces of the system from each other, so that developers can know that they can work in a particular area of the system, innovating a technology, without adversely impacting larger parts of the system, that as Rob said we can't predict they're going to be impacted, and in a way that would actually jeopardize our agility in getting those features out that result from that innovation.
So specifically what we've been doing is taking every binary in the system and assigning it a layer number, which is a rank in a directed acyclic graph. There's about 5,500 binaries in the system. And what we've been doing is getting transparency now into every dependency that developers add to any of those binaries, so that we can understand what's going on. And what's falling out of that is not necessarily just the isolation I described, but also, issues. We call them, sort of, conventional wisdom ... controversies. For example, people might be thinking, well, I want to combine a whole bunch of DLL's into one DLL for perf. Well, it turns out that that's a
There are plenty of guidelines on how to deal with trademark dispute lawsuits, what are you talking about?
(supposed to be funny....)
Most cases of this are resolved long before the programs are installed.
Laws are horrible moral guides, moral guides make even worse laws.
So they're more or less admitting "essentially ... windows is one big binary..." Woah! Low level libraries and frameworks depending on stuff that's higher level, "in the past we've relied on... lockstep... development process..." and "we're now looking at dependencies in the 6 digits range..." Man, these guys are giving one hell of a bashing to the Microsoft codebase.
:-)
One guy starts talking about modularity and inserting features and plugins into essential services... and I thought objC. But before that another one gets all hot (I chuckled, this guy is a True Nerd, he really likes fiddling with code... congrats) about semicoop multitask where an app renices itself to 100% resource hog tier for a limited time slot (nice try, but what when all the silly apps do the same trick?), but before that there's a talk about usermode ukernel services... I thought about when I used to renice X11R6 to get better performance (when the graph card module was part of the X process).
I think Bill needs to pull out of tech and sell Microsoft to Apple. These techs are good guys, all they need is a solid process and some decent vision.
Jobs, are you reading this? Watch this video, it'll make you feel good!
e
Mi domando chi à il mandante di tutte le cazzate che faccio - Altan
However, once I got my beta of XP (NT 5.1) I was sorely dissapointed when the ntoskrnl.exe and other nt*.exe and nt*.dll files (I forget exactly what they are named.) had similar architecture and functions to the same ntoskrnl.exe files in NT 3.1, which I recall running like a dog on my DX/66 (particularly compared to OS/2 2.0 which ran great).
The Kai's Semi-Updated Website Thingy
Doesn't this guy that's talking about media glitching remind you of Howard Dean? He's way too excited about program priorities. He's seriously hitting 80dbs from time to time. Sure would hate to work with that fool on something truely exciting.
Why would you trust a testimonial when choosing hosting?
First of all, the video is unviewable even with Microsoft Media Player on Mac, but you can find a whitepaper describing the kernel changes here. Keep in mind that all of this is basically Microsoft advertising for developers; it's not taking a "hard look" at the kernel architecture, it's the kernel developers portraying their work in the best light.
What's interesting is how little innovation there actually is. They seem to be struggling with the complexity of the system and its dependencies (5500 components)--similar to the problems Linus is having, but multiplied many times over by greater complexity of the NT system architecture. Most of their actual improvements seem to be cleanups and performance enhancements.
My impression is that the Vista kernel and system libraries are still playing catch-up with Linux in terms of modularity, performance, and functionality.
DIR SIR
GREETINGS TO YOU GOOD SIR, PRAISE GOD. MY NAME IS ABDUL-MUQADDIM, A CIVIL SERVANT IN LAGOS, AND GREAT GRAND-NEPHEW OF EXILED MICROSOFT VP ROB SHORT. BEFORE MY GREAT UNCLE'S EXILE, HE DEPOSITED $20,000,000 (TWENTY MILLION US DOLLARS) IN AN ALGERIAN BANK ACCOUNT. UPON HIS EXILE, HIS ACCOUNT WAS FROZEN AND TURNED OVER TO THE GOVERNMENT OF ALGERIA. MY FRIEND IN THE ALGERIAN GOVERNMENT WAS ABLE TO SECURE ACCESS TO THIS ACCOUNT, BUT WE NEED A MOST TRUSTWORTHY THIRD PARTY ACCOUNT INTO WHICH WE CAN TRANSFER THE FUNDS.
I AM WRITING TO YOU ON BEHALF OF MY UNCLE REGARDING THIS MOST PRIVATE AND PERSONAL MATTER. FOR YOUR COOPERATION AND ABSOLUTE CONFIDENTIALITY, WE OFFER YOU 40% (EIGHT MILLION US DOLLARS) OF THE FUNDS UPON RECEIPT OF THE TRANSACTIONS.
IN ORDER TO BEGIN OUR TRANSACTION, GOOD SIR, I HUMBLY REQUEST THAT YOU SEND $50,000 (FIFTY THOUSAND US DOLLARS) TO THE BELOW ADDRESS, SO THAT I MAY OPEN A FOREIGN ACCOUNT ON YOUR BEHALF. IN ADDITION, I REQUEST THAT YOU WATCH THE FOLLOWING VIDEO, IN ORDER TO KNOW OF MY GREAT UNCLE. YOUR IMMEDIATE RESPONSE WILL BE HIGHLY APPRECIATED.
THANK YOU, YOUR HUMBLE SERVANT
ABDUL-MUQADDIM
Given evidence from the era of Microsoft hacking with Win 3.11 to make sure that it broke Windows compatibility, OS/2s demise was only partly that IBM couldn't market eternal life in 1993.
We ran it too, used it to multitask DOS programs, run Win3.1 apps more stabily than Win3.1 did, and to run native apps that needed the 32-bit address space. It was great to be able to recompile our VAX apps with Watcom Fortran, run them (and get a speed-boost over the VAX), and still be able to use the computer for other apps. Other research groups had it powering their Mass-Spectrometers, and similar fussy hardware.
More importantly, we never had a problem with frequent crashes. We bought good memory and standard hardware, and made sure that we had 8-16 meg, which seemed to be the sweet spot. It just ran. I didn't leave it behind until NT 4 had a service pack or two behind it, and I'd acquired a PowerIndigo2 with the Cray-derived Fortran compiler at work, pretty much eliminating why I was still running OS/2.
We're still paying for the mistake of not adopting it, as many of the security problems in Windows stem from single-user, insecure, Windows95 getting released and established first, rather than VMS|OS/2 derived NT.
the more accurate the calculations became, the more the concepts tended to vanish into thin air. R. S. Mulliken
I wrote him an email: " Zonk, Don't take this the wrong way, but I submitted the kernel architecture article to slashdot just yesterday...I see you've posted the link that I was going to post....it's funny that this is on slashdot and it doesn't have my name on it....I also see thay you're a slashdot editor...hmm.... Is that how slashdot works? You guys take submissions from people like us and put your name on it? I didn't know slashdot worked that way... I'm bariswheel on slashdot... -baris "
Insinct is stronger than Upbringing - Irish Proverb
Very good account of Postscript.
In response to the poster above that sees Microsoft as ripping off Postscript, they have no idea what Microsoft is doing and how it is different than Postscript.
Everyone that thinks MS is ripping anyone off needs to just go to msdn.microsoft.com and read up on what Microsoft is actually doing before slamming it with a generalization. (
Even what I say below, don't take my word for it, take 10min and go look at it. Even if MS is your enemy, it is better to know what they are doing, especially if they are doing something that might be unique or at least innovative one particular field.)
Sure there are similarities as MS new technologies can be used to render things on screen, to a printer, or store it in a document.
One thing that is different is MS is using a XML based format that will allow Windows and applications to pass this information internally from screen to printer to clipboard to document. (Although this isn't a giant leap, but will be handy for a more streamlined protocol internally within the GUI as well as doing remote operations.)
MS's technologies are like the next generation of what Postscript was in the 80s.
It is font independant, has more advanced rendering concepts built in, from blending and transparencies and other normal graphical application types of display that are a bit more advanced than Postscript and what you would normally find in CorelDraw or Illustrator and Photoshop to AutoCad.
It also fully handles 3D dimentional space, animation, control and message handling as well as other forms of media like video, ink, audio and is extensible beyond current media concepts of today.
What probably would be the giant leap is that it even inherently handles modeling and things like collision detection in a 3D space with support for user control and interaction, which is kind of cool for a presentation technology. (Envision how nicely this will adapt to printer technologies that blur displays and printed output - i.e. digital ink)
This is a bit way beyond what Postscript does, although what Postscript does, it does well and shouldn't be dismissed.
But don't say MS is ripping of Postscript, any more than Postscript was ripping off the first Vector drawing formats that predated it.
MS said for years that Unix is so old. Now Windows is becoming more and more like Unix. What a bunch of idiots these guys are that took them so long to realize that their architecture is flawed and that Unix's architecure is superior.
I think it was Cutler or someone from his team in 1991 that made a comment along these lines, but it wasn't about the age of UNIX, it was the inherent problems in the architecture of UNIX and its limitations.
And if you know anything about NT and its architecture, you will surely realize that not only is there a great deal of difference from UNIX by design, but the direction Microsoft is evolving NT has very little relevance to anything in the UNIX world.
UNIX zealots should flame you as well as NT proponents.
If Microsoft wanted UNIX, they had XENIX and Cutler had full control to make NT a full UNIX implementation/evolution. However the NT team did not want the UNIX limitations, and they were from the UNIX world themselves.
UNIX is great in many ways, but by definition, when you adhere to a base operational specification, you are limiting yourself, no matter how good it is.
NT doesn't have these rules, and whether people like it or not, it doesn't have to adhere to anything but what they want it to do or believe works well, so it by definition it will never have these imposed limitations.
Narrator: Fantastic. So can you talk a little bit about what's new in the Vista kernel? So we go from XP; now we're going to Vista. So what are some of the new components?
Rob Short: A term I like to use is probably kind of politically incorrect on TV is, some of the work we do is kind of like sewers, but if we do this work incredibly well, the stuff is essential, but nobody knows that it's there.
Narrator: Yes.
Rob Short: So, if things go bad, obviously you know about it.
Narrator: Certainly.
Rob Short: Most of the work that I've been focused on for the last several years has been improving the experience where the hardware meets the software. Things like power management. We have a team of people looking at power management and working to improve how the system behaves, say a laptop for example.
If you have a laptop, how fast does it turn on, turn off, how good is the battery life? What's the experience when you dock or undock? And we've done a huge amount of work on that. We've redesigned the algorithm for hibernation so that we do a better job of figuring out which pages are already on the disk so you don't have to send more of the pages back to the disk. We've changed the way the power management interfaces to the drivers so that we have a better feel for understanding if we can just shut this thing off, right now. Today, in the older system, in XP, we actually query the driver, say, "Hey, would you, like, mind if we turn off the power?" A lot of times, people haven't coded up the driver correctly. Mostly the drivers don't care, where some really do. A disk driver, it really matters if you, you know, turn the power off in the middle of a transfer. But a lot of other things, you don't care. Mouse, it doesn't really matter that much. You know, you can go across the extreme. So we've done a bunch of work in that area.
We obviously do a lot of work in performance. One example is we had problems with heap fragmentation, and we've redesigned some of the heap algorithms so we can deal much better with much more random requests. We can deal with those and do a better job with defragmenting the heaps. So those are the types of things.
Several people--Darryl works specifically on the multimedia, and understanding how we do a better job of not having glitches in multimedia, but that also goes right through the full length of the system. It's not just buried in the kernel.
We've improved the inter-process procedure call. We have a new sort of fast, lightweight procedure call inside, in the core parts of the system. We ... stop me.
Narrator: (Laughing) He has a whole list! A cheat list!
Rob Short: There's an awful lot going on. One area where we actually make a lot of changes over time that I feel really good looking back is in the memory management area. If you think about the early NT systems, Bill Gates used to beat us up, and say, "How come you don't run in four megabytes?" And when you look at that today, and think, we're running regularly in four gigabytes today, and we have the systems in the lab that run with a terabyte of memory, the algorithms that worked back then, and the priorities back then are completely different than they are today. So we've put in work in Vista for improving the NUMA support, which is Non-Uniform Memory Access when you have a multi-processor where some of the memory is closer to some processors than to others, so we do a better job of doing the allocation, making sure that they're allocating memory that's on the CPU, near the CPU that you're running on, and then you try to run the process on the CPU where the memory actually is so you don't get cache thrashing.
Narrator: Interesting.
Rob Short: We've done some stuff for the graphics. The graphics processors today are more powerful than the CPU'
Part of the WMF handling is in Win32K in current Windows versions, so it is in ring 0
WMF handling has been in the API of the OS since Win32 was designed. (i.e. it has always been able to inherently draw a WMF to any surface.)
However, this is not Ring 0, not even Ring 0 if you consider the Win32 Kernel as Ring 0, and in NT(XP,Win2k,2003), the Win32 Kernel is far from Ring 0 being in its own subsystem sitting above NT itself.
Just clearing up what you were saying in your post, trying not to nit pick too much...
The thing that irritates me about the tone of Microsoft is that they still live in this world where they spin everything they do as amazing and its just not any more. Computers aren't "amazing" anymore. Operating systems and things like Windows does, even if new, don't have the same impact as the basic innovations of GUI displays did in 1992. The amazing stuff that is happening is, um, usually at Google.
This is my sig.
...I'll put on the asbestos underwear for this post...
;-)
In 1990 at Microsoft there were several requirements that drove the registry. The number of third party applications and application writers was growing very fast. Making this worse, a new object system was on the horizon which could dramatically increase the number of independently-authored "components" that needed to be registered. There was a need to store state in a segregated manner so that apps wouldn't stomp on other app's information. Also there was a "new" notion of remote manageability for the objects, so the access method should be easily remotable early in the boot process. Also the OS needed a place to store lots of very small data items.
It would have been best to use the file system, but the file system at that time was FAT which could not store small data items efficiently. The registry was the first API common between Windows 3 and OS/2 (and also NT), which was a goal at the time. Of course it quickly went out of control, since there was no rational security or ownership model. The registry was kept very very simple in order to maximize the likelihood that the next file system (either the object file system or NTFS) would be able to implement it, including in the NT kernel (which had a very simple API model). It was also the first API from Microsoft that had unused parameters for future features, such as context ids for security, query features, and other stuff. Unfortunately much of that didn't work as planned since very few applications paid attention to the requirement to set them to 0L!
I didn't expect it to be so massively overused, nor for it to survive beyond Windows 3.x. It was supposed to be superceded by an object file system (that was designed and implemented several times, but never released.)
There's a good story behind the registry, though: I designed the registry while on a bachelor party for a friend, mostly on a car ride between San Diego and Las Vegas, and faxed in the design from Las Vegas the morning after the party to the responsible program manager. Which might explain much about the design...