Mark Russinovich on Windows Kernel Security

An anonymous reader writes to mention that in the final part of his three part series, Mark Russinovich wraps up his look at changes made in the Windows Vista Kernel by exploring advancements in reliability, recovery, and security. "Applications written for Windows Vista can, with very little effort, gain automatic error recovery capabilities by using the new transactional support in NTFS and the registry with the Kernel Transaction Manager. When an application wants to make a number of related changes, it can either create a Distributed Transaction Coordinator (DTC) transaction and a KTM transaction handle, or create a KTM handle directly and associate the modifications of the files and registry keys with the transaction. If all the changes succeed, the application commits the transaction and the changes are applied, but at any time up to that point the application can roll back the transaction and the changes are then discarded."

Comments

Not dupe, but almost

[vivaoporto]

Although this is technically not a dupe, it is almost, as the above linked article is the Part 3 and the other submitted and discussed article was the Part 1, isn't it kinda repetitive? What now, someone post a multipart article and we will get one story here on front page for each part?

On topic now, I don't like the way Russinovich is blowing Vista's horn now. I liked him more when he was more critical and analytical on what could be improved, instead of what has already been done.

Re:Not dupe, but almost

[Bonker]

MS made a good move in hiring Russovitch. We can hope that he has more positive influence over kernel changes to XP and Vista than MS has bad influence over things that he does and does not get to say and software (sysinterals) he does or does not get to release.

Re:Not dupe, but almost

[suv4x4]

On topic now, I don't like the way Russinovich is blowing Vista's horn now. I liked him more when he was more critical and analytical on what could be improved, instead of what has already been done.

He's working at Microsoft now, you do realize that "what could be improved" he's actually doing right now, so he can call it "already been done".

You don't really prefer pointless critique with no improvement, do you.

Re:Not dupe, but almost

[LordSnooty]

Yeah, GP is a rather odd comment. Maybe what he said should be done has been done, and now he can talk about how it's implemented. He's a clever guy, I'm sure MS didn't employ him just so they could get their hands on Process Explorer. I guess this factoid was lost on 'first post'

Re:Not dupe, but almost

[vivaoporto]

He's working at Microsoft now, you do realize that "what could be improved" he's actually doing right now

You don't really prefer pointless critique with no improvement, do you.

Point conceded.

Re:Not dupe, but almost

[dedazo]

On topic now, I don't like the way Russinovich is blowing Vista's horn now. I liked him more when he was more critical and analytical on what could be improved, instead of what has already been done.

Fresh blood always brings fresh perspectives. Mark has been on the outside looking in for long enough that he probably has a fairly asymetrical view of the NT kernel to that of the core MS developers, and in software development (especially at that level) that tends to be extremely useful.

Re:Not dupe, but almost

[EllynGeek]

Sysinternals was one the very few companies with the expertise to dissect and analyze Windowses truthfully. It's no accident that after the Sony rootkit fiasco they were quietly acquired by microsoft. Just one more assimilation.

Re:Not dupe, but almost

[JackMeyhoff]

He should be the kernel code pope that was talked about :)

Re:Not dupe, but almost

[lundqvist]

Sysinternals created some really useful stuff, I agree the guy may be making improvements but its MS marketing thats giving him the script now. Vista wont ever be really secure until MS totally rework their approach to OS design and its default install. The problem is its mass market mostly consists of users with less than average technical know-how and thats why so many compromised systems are out there. If it was secure by default then viruses and malware wouldnt be the huge problem they are now although the tech support overhead would be higher.

Re:Not dupe, but almost

[acidosmosis]

>>I don't like the way Russinovich is blowing Vista's horn now. I liked him more when he was more critical and analytical on what could be improved, instead of what has already been done.

Are you sure that you do not mean that you don't want to hear anything that does not shed negative light on a Microsoft product?

Message to Microsoft

[bonefry]

Just leave my applications alone !

doesn't belong in the kernel

[nanosquid]

There is little reason to put these kinds of transactional services into the kernel: they don't involve security or user permissions and they must be efficiently implementable in user code anyway (otherwise, most databases wouldn't work well on NT). So, I'd classify this as "kernel bloat".

Re:doesn't belong in the kernel

[Bacon Bits]

Yet PS3 portability was *absolutely vital*.

Re:doesn't belong in the kernel

[Lally Singh]

They also involve atomic I/O to multiple systems simultaneously. Userland can't do this. Databases work on one system, their own data files, and have full control over these files.

Userland apps don't have that kind of control over the registry. Hell they may not be sure to have that kind of control over the files they're manipulating.

Besides, I'd rather have this code once in a DLL than 10 times in 10 different apps. That's real bloat.

Re:doesn't belong in the kernel

[Anonymous Coward]

Because, unlike a DB's transactional system, Vista's is fully pluggable meaning that anything can build volatile transaction management into their application or driver and automagickally gain the benefits of a single logging mechanism and full support for distributed transactions.

And yes, this is a really big deal. Other than maybe OS/400 I know of no system where file system operations can be done atomically with database operations. I can poll data, write a file and update that data all in one atomic operation. Either the data has been exported and marked as such, or not. No in-betweens. No incomplete files. No complete files despite a failure to mark the data in the database. No attempt to manually compensate for errors. It's all there, one operation, atomic and automatically so. This is absolutely massive for my world where financial and business transactions are moved constantly in such a fashion.

Yes, Microsoft does innovate sometimes. This is one of those occasions.

Re:doesn't belong in the kernel

[Anonymous Coward]

The majority of the framework is implemented in userland. See the Distributed Transaction Coordinator service.

Re:doesn't belong in the kernel

[Anonymous Coward]

Think outside your small box. These features can provide significantly more than current user-mode transaction management that databases (or OLTP monitors) provide today.

For instance, how does a user-mode controlled file-system transactions rollback changes on reboot? Consider what happens when the change involves system files. NTFS handles this cleanly in Vista but any attempt to do this with a user-mode service would fail because the system files would already be in-use by the time the service started. Further, any transactions in-doubt would remain in-doubt until the service started (if you could get that far).

Similarly, what about settings in the registry? Consider if the transaction spanned installing a driver and updating its settings. What happens if the system powers down midway through that update? Without kernel-level transactions that are available at boot-time you cannot easily recover before inconsistent settings are used with the driver.

Beyond the boot-level support; another exciting aspect about the KTM is that this is a transaction-manager that provides transactions that can be shared across multiple processes. Try that with a user-mode transaction manager like XA or DTC and see your per transaction performance. By managing the transaction in the kernel, the 2PC performance is significantly improved.

In the end, NTFS and the registry live in the kernel -- so their tramsaction manager also has to live in the kernel. That is just the way it works.

Another exciting aspect covered is the ability to coordinate DTC transactions with KTM transactions. Meaning, you can coordinate your SQL Server (or Oracle) updates with your file-system updates. This is cool! No longer do you have to worry about finding orphan-files in workflow applications or other applications that manage both files and relational data.

Lastly, they are talking about full ACID properties with NTFS's transactions. Think about it -- you could update every file on your web server, in place, while its wild with activity. All your changes are isolated from the users until you say "commit" in your application.

Re:doesn't belong in the kernel

[Bacon Bits]

#ifdef is a pre-processor instruction. It's not C code at all.

Re:doesn't belong in the kernel

[EvanED]

Here's another top... the preprocessor is part of C.

Not that you should be using ifdef to separate platform-specific code anyway... it should be put into another file.

Re:doesn't belong in the kernel

[Bacon Bits]

It's not really part of C, though. It's a common program that parses C code for errors and also listens to instructions means specifically for it. It's run as part of the compiler, but the compiler is not C either. C is a language, not a program.

It's like saying that PHP *is* HTML.

Re:doesn't belong in the kernel

[TheRaven64]

The ISO C specification includes the preprocessor functionality. The original K&R spec did too.

I have not seen a copy of the HTML specification that included any mention of PHP.

Re:doesn't belong in the kernel

[EvanED]

C is a language, not a program.

Yes, I realize that.

And the preprocessor directives are part of that language. Or perhaps you missed sections 5.1.1 and 6.10 of the description of that language? (Assuming the numbering stayed the same from draft to final.)

Re:doesn't belong in the kernel

[nigelo]

> Yes, Microsoft does innovate sometimes. This is one of those occasions.

Well, DEC VMS had this capability decades ago, so is it really innovation?

http://h71000.www7.hp.com/commercial/decdtm/index. html

Re:doesn't belong in the kernel

[Bacon Bits]

Yes, I'm aware it's standardized in the ANSI spec. Nevertheless, I still don't consider it true C code. Preprocessor code is metacoding. Similarly, I don't consider !doctype to be true HTML, and so on. Metacode is extremely useful, and in the case of C it was decided that it is so useful that standardizing it was required. But metadata and metacoding is a secondary language used to facilitate the requirements of the primary language.

If you want to stomp your feet and pout, go ahead.

Still. Not. C.

Re:doesn't belong in the kernel

[swillden]

They also involve atomic I/O to multiple systems simultaneously. Userland can't do this.

Of course it can. As long as the OS provides atomic I/O operations on individual files, the logic for implementing two-phase commit is the same in userland or in kernel space. For that matter, atomic I/O can be implemented entirely in userspace, as long as the OS provides a mechanism for userspace to find out when the data has actually been written. It's more convenient to put single-file atomic I/O in the kernel, though (or at least in the file system).

Besides, I'd rather have this code once in a DLL than 10 times in 10 different apps.

That's not an argument for putting it in the kernel.

Re:doesn't belong in the kernel

[Anonymous Coward]

Yes, it is innovation. The feature you linked to in VMS is a distributed transaction coordinator. Windows has had this component (called DTC) for years.

The actual innovation is making a Kernel Transaction Manager, along with a resource manager for the filesystem. The KTM means that transactions can be inherited from parent process to child or joined by a cooperating process. Having a transactional filesystem means that all file operations can be all-or-nothing. Power goes out during a major update? No problem, because when you boot up again the filesystem will be recovered and it will be like the update was never started. And the beauty of having the resource manager in the filesystem is that you can combine the filesystem transactions with any other ones in the system (via the DTC). In fact, they could even make it so that you can combine transactions across systems so that moving files from one server to another would be atomic.

dom

Re:doesn't belong in the kernel

[dgatwood]

Of course it can be done in user space. If a user space app can't do it, neither can the kernel. And it isn't atomic I/O. There's no such thing as atomic I/O. I/O operations are reordered, split, combined, etc. by everything from the OS to the controller to the hard drive, and for network volumes, it is even worse. There's no practical way to guarantee atomicity, so you have two choices: have filesystems (including remote filesystems) with rollback capabilities (which still don't completely guarantee anything) or design a file structure that achieves the same thing (which still doesn't guarantee anything). The former is nice for a lot of reasons (e.g. so that every developer doesn't have to reinvent the wheel), but isn't essential by any means. It also would greatly increase the complexity of the VFS layer and filesystems written for it, so if that is the only purpose for doing transactions, it makes a LOT more sense to implement them in a user space library instead of in the kernel.

If your sole purpose is to be able to do multi-file rollbacks, user-space transactional support is as easy as designing your file format and/or layout around it. There are two easy ways to do this: files with built-in history and using swappable directories.

Files with built-in history:

For the initial modification pass, modify each file by appending what amounts to a diff footer. If an error occurs, you can undo all of the changes by truncating the files prior to the latest diff footer. Once these modifications are complete, you no longer need to worry about rolling anything back (except for cleaning up temp files if something fails in the second pass) because the data is safely on disk. (Note: this does require that the kernel and all devices and/or network disks reliably flush data to disk upon request. Don't get me started on buggy ATA drives.)

In the second (optional) pass, you coalesce the diff into a new copy of the file and swap the compressed version in place of the original file. This is generally an atomic operation in most operating systems. If anything occurs during the second stage, it is a recoverable failure, so there is no need to roll anything back. Heck, Microsoft's file formats pretty much do this anyway. (Notice the 500 megabyte single page MS Word document that occurs when you make lots of changes and always "save" rather than "save as".)

Swappable directories

The easiest way (tm) to handle system configuration files in an atomic fashion is to modify config files in the same way you would perform a firmware update: you have an active configuration directory and an inactive configuration directory. You read the active one, make changes, and write to the inactive one. Then you trip a magic switch (tm) that says that the previously inactive directory is now active, and vice versa. Assuming you don't have out-of-order writes going on (which the kernel can't really guarantee any better than user space, sadly), this is a very easy, effective way to perform an atomic commit. And if you have an "exchange in place" operation in which the data for two files or directories in the same directory are swapped in a single atomic operation, that's a really lightweight way to implement an atomic commit/rollback mechanism without most of the complexity.

Considering how easy this is to deal with in user space, the only legitimate reason I can think of to do it in the kernel is so that you can take it out of application control entirely (e.g. to make it easier to sandbox an untrusted application). Otherwise, it makes a lot more sense to do this in a library. Now if it had snapshotting where you could roll back the entire filesystem to arbitrary points in time, that might be interesting (for different reasons)... but basic transactional support in a filesystem is much less so, IMHO, unless your purpose is to be able to sandbox an application. If so, then all this other stuff basically comes for free. In that context, doing this in the filesystem layer makes sense. However, if that is not their purpose for doing this in Vista, then kernel bloat definitely strikes me as an accurate depiction.

Just my $0.02.

Re:doesn't belong in the kernel

[nanosquid]

Of course, you can do it in user space. You could do that in user space already in UNIX V7, and it's been used for anything from atomic system upgrades to maintaining gigabyte mail spool directories without data loss (something that Exchange doesn't seem to be able to achieve to this day). You really don't need a transaction manager in the kernel for that; any one of a number of existing kernel operations is sufficient.

Re:doesn't belong in the kernel

[Herby Sagues]

It IS atomic IO. IO commands can be reordered as much as you want, but commands can be tagged for flush or immediate execution, and if a device doesn't comply with those, it is either badly writen or it has incorporated enough safeguards so it doesn't matter. For all practical purposes it is atomic IO. In addition, how do you expect a user mode component to be able to implement transactional services for kernel components? This is not only about begin trans /end trans for copying files. It is about bringing transactiona capability to every operation in an application and in the kernel. Yes, it is not yet exploited by every single kernel process, and probably will never be, but it is already being used by kernel code to simplify error handling and to provide better guarantees of consistency. Making it user mode would only increase state transitions with no real benefit to anyone.

Re:doesn't belong in the kernel

[dgatwood]

IO commands can be reordered as much as you want, but commands can be tagged for flush or immediate execution, and if a device doesn't comply with those, it is either badly writen or it has incorporated enough safeguards so it doesn't matter.

I'm laughing. No, really. A lot of devices lie in one way or another. It's not just a few badly written devices. You'd be shocked. My favorite are the drives that completely lie and say that the cache has been flushed when it really hasn't. Most of those drives, you can issue a second flush request and it will block, but there are some that always instantly return success for a flush request. It's not even all that uncommon, ESPECIALLY with external drives because of bridge firmware bugs.

I've heard some true horror stories about the number of problems companies find when qualifying off-the-shelf ATA drives for server use. If you heard them, you would cry. Truly, all you can assume is that the operating system has made a best attempt at a guarantee. Anything more than that is just kidding yourself... and you can get that same level of guarantee just as easily from user space (at least in UN*X) by executing two sync system calls in a row.

In addition, how do you expect a user mode component to be able to implement transactional services for kernel components?

WHAT!?!?!?!?!?! Why in you-know-where would anything in the OS kernel need filesystem transaction support? Stuff in the kernel shouldn't even be READING files, much less writing them! Yes, I'm serious. The kernel should provide only the most critical, basic system-wide OS services. Anything that requires such heavy lifting is NOT providing such a critical, low-level service, and thus, does NOT belong in the kernel. It's that sort of thinking that has led to such abominations as khttpd....

I'm going to go hide now. I'm very scared....

Re:doesn't belong in the kernel

[sych]

Another exciting aspect covered is the ability to coordinate DTC transactions with KTM transactions. Meaning, you can coordinate your SQL Server (or Oracle) updates with your file-system updates. This is cool! No longer do you have to worry about finding orphan-files in workflow applications or other applications that manage both files and relational data.

If this is a problem for MS SQL Server, then it is poorly designed; UNIX RDBMs do not have this problem.

I think he was talking about this:

Step 1. Program writes a file to a directory.
Step 2. Program updates database to say "file X written to location Y"

I think he's saying single transaction can surround both of those actions... so, if the DB update or insert fails inside the transaction, then the file write or move or whatever will also be rolled-back.

Re:doesn't belong in the kernel

[nanosquid]

This issue crops up in just about every major software system: database servers, web servers, mail server, multimedia servers, package install managers, etc. UNIX has been used for these kinds of applications for decades. Do you seriously think UNIX systems would be lacking a general purpose kernel transaction manager if it was useful for solving this issue? It's not rocket science to write one, and if it's not there, it's a good bet that it's not needed.

The functionality you want can be encapsulated neatly in user code, using standard UNIX file system semantics, with the same guarantees and behavior as if you had a kernel transaction manager.

Re:doesn't belong in the kernel

[Lally Singh]

Atomic I/O generally (I'm from the posix side of things, so maybe there are different definitions on win32) means that the entire operation's done by the kernel at once. E.g. I write 16 bytes at offset 0 to this file, and another app's writing 16 bytes at offset 0 to this file. When we're both done writing with a single write() call, the file's got 16 bytes from one of us, not some from one and some from another. No amount of userspace I/O calls will guarantee this (outside some godawful locking, which essentially forces the same thing, only with more context switches and a lot more userland complexity).

In terms of filesystem changes, all the files are created (possibly overwriting other files) or, if one fails, they all undo. Same for registry changes. Them all running together to completion or not at all sounds great. I definitely want registry changes to be atomic, b/c there's a good chance other apps are mucking around with it too. Also, I want those changes to be consistent -- with an atomic change that includes both the registry and a filesystem change, I can make sure I don't change a file with the registry pointed to the old value, or visa versa. In userland I could roll back, but I couldn't stop that time period when it's inconsistent.

Re:doesn't belong in the kernel

[EvanED]

As long as the OS provides atomic I/O operations on individual files, the logic for implementing two-phase commit is the same in userland or in kernel space.

Really? How?

You CAN'T do it without kernel support. You can do it for cooperating programs through auxilary files and such, but not in general. If you have a program create two files, even if you can create each file atomically, you have to make two system calls which means you could be preempted between them and have another program see only one file.

I'm not *totally* convinced of the real world benefits of this, but it's something that *can't* be done in userspace.

Re:doesn't belong in the kernel

[EvanED]

Give me a program that will create two files in a directory while guaranteeing that no program can see only one of them.

Re:doesn't belong in the kernel

[maxwell demon]

Stuff in the kernel shouldn't even be READING files, much less writing them!

Well, I'd expect the kernel's read system call to read files, and the kernel's write system call to write them. :-)

Re:doesn't belong in the kernel

[maxwell demon]

Simple:
Step 1: Add commands creating those files at the beginning of the boot sequence (directly after mounting the file systems, before starting anything else).
Step 2: Reboot.
Step 3: Remove the commands from the boot sequence.
SCNR :-)

Re:doesn't belong in the kernel

[nanosquid]

Give me a program that will create two files in a directory while guaranteeing that no program can see only one of them.

mkdir newdir; touch newdir/file1; touch newdir/file2; mv newdir dir

On UNIX, dir/file1 and dir/file2 both appear at once, atomically.

This is only one of a large number of design possibilities for atomic updates/changes on UNIX. People tend to pick designs that offer robustness not only against interruptions, but also against other kinds of corruptions, which is probably why UNIX systems tend to be more robust than Windows.

Re:doesn't belong in the kernel

[dgatwood]

Of course, this doesn't completely solve consistency for the registry. Atomicity does not guarantee consistency with shared data structures. It just guarantees that everything (right or wrong) happens at once. Only exclusive locking can guarantee consistency.

For example, say that you have two values, A and B, and two processes, 1 and 2. Both of these are counters, both set to zero. Process 1 intends to increment both counters, so it reads the contents of both counters. Process 2 increments counter A in an atomic operation. Process 1 now writes changes to both counters in a single atomic operation. The resulting values are 1 and 1, but if you want true consistency, the values should have been 2 and 1. The problem is that, while the writes were atomic, the reads were not part of the same atomic operation. You can't prevent that unless you prevent process 2 from modifying the data after process 1 has read the data and before it has written the data. This requires exclusive locking. While exclusivity can be used to guarantee atomicity, atomicity does not inherently guarantee exclusivity. More accurately, the presence of atomic operations is only one of the requirements for exclusivity. The best you can really do without locking is to order your modifications from least to most volatile and hope for the best.

Further, atomic commits do not even fully solve the problem of the registry and files being inconsistent with each other in a real-world environment. Processes do not read the registry and files in a single atomic operation. Processes 1 and 2, registry entry E, file F. Process 1 wants to read the file. It asks the registry. Process 2 commits a change that simultaneously changes the registry and changes the name of file F to F1. Process 1 tries to read file F (at the old location) and fails. The data on disk is consistent---the registry entry points to F1---but from the perspective of process 1, the data was not consistent.

The -only- way to guarantee absolute consistency is to not allow any process to read any portion of the data that is being modified until after all modifications are complete, which means locking. Otherwise, there is still a period if inconsistency, and thus, atomicity does not guarantee consistency. If you have to lock to guarantee consistency anyway, you might as well do it all in user space.

Re:doesn't belong in the kernel

[swillden]

Upon further thought, I stand corrected. There are lots of cases you can implement without kernel support, either with cooperating programs or with symbolic links, but a fully general implementation has to be in the file system.

Re:doesn't belong in the kernel

[EvanED]

mkdir newdir; touch newdir/file1; touch newdir/file2; mv newdir dir

If dir/ already exists, that will move newdir/ INTO dir/.

Perhaps I should have said that you need to create two files in an existing directory.

(I'm not sure about the use cases of this, if they are actually helpful in any real world programs, but that's not what I'm arguing.)

Re:doesn't belong in the kernel

[nanosquid]

If dir/ already exists, that will move newdir/ INTO dir/.

Of course it will. So what? The code I wrote illustrated how your requirement can be satisfied. In fact, this is not the way real UNIX programs usually do atomic operations (there are far better ways).

(I'm not sure about the use cases of this, if they are actually helpful in any real world programs, but that's not what I'm arguing.)

Yes, there are many kinds of operations you cannot make atomic with the UNIX file system primitives. My point is that you don't actually need those. Therefore, adding the bloat and complexity of a general purpose transaction manager to the kernel simply is not necessary.

Kernel enhancements? Are you sure?

[Applekid]

According to the bottom of just one function of the KTM reference:

"Requires Ktmw32.dll."

Why would a kernel add-on require a .dll, exactly? Did I miss some Windows fundamental about it's kernel? And if it's not really a result of a kernel enhancement, is this yet another potentially useful technology specificly excluded from earlier versions of Windows entirely for business purposes instead of technological limitations?

Re:Kernel enhancements? Are you sure?

[Cyberax]

Because this DLL is just an interface to kernel features.

Windows NT was initially designed to use single kernel for different subsytems (OS/2 subsystem, POSIX subsystems, etc.). Subsystems are implemented as dynamic modules talking with the kernel through LPC (Local Procedure Call, see http://en.wikipedia.org/wiki/Local_Procedure_Call ). So in this case ktmw32.dll just wraps LPC calls in a nice API. That's actually a rather good design.

Re:Kernel enhancements? Are you sure?

[Keeper]

The dll performs the mapping from the "nice" function call you see in MSDN to the system call(s) required to perform the desired operation.

Re:Kernel enhancements? Are you sure?

[EvanED]

Windows NT was initially designed to use single kernel for different subsytems (OS/2 subsystem, POSIX subsystems, etc.)

Not just initially designed, it DOES use a single kernel for different subsystems. You can't get the OS/2 one any more, but the POSIX subsystem morphed into (part of) the Services for Unix which has become the Subsystem for Unix-based applications.

On 32-bit Windows, 16-bit Windows applications are handled by the "Windows on Windows" subsystem. On 64-bit Windows, 32-bit Windows applications are also handled by a "Windows on Windows" subsystem, though a different one than WOW16.

Re:Kernel enhancements? Are you sure?

[anImortal]

Its a small DLL that provides you with Win32 APIs to call kernel (Nt) functions.

What is the registry in Vista?

[mosel-saar-ruwer]

For years, the "Registry" was some weird mish-mash of binary files, many of which represented Jet databases.

Has Jet been completed abandoned in Vista?

If so, did they switch to the slimmed down SQLServer [that was supposed to be part of WinFS]?

Re:What is the registry in Vista?

[allanw]

Nope, the registry is still a bunch of binary files, located in the same folder.

Re:What is the registry in Vista?

[Foolhardy]

The registry is a single root hierarchical database with registry hive files mounted at the second level (below \REGISTRY\MACHINE and \REGISTRY\USER for the computer's config and user config, respectively). The registry engine is implemented in kernel mode as an executive subsystem (inside ntoskrnl.exe), where it is known as the Configuration Manager. Registry hives use a transaction journal (like many filesystems do) to avoid corruption during a power failure or crash. Standard system hives are located in %SYSTEMROOT%\System32\Config and include SAM for local user accounts, SECURITY for various secrets held by the computer, SYSTEM for core system configuration early during boot, and SOFTWARE that stores all other config associated with the computer in the registry. Every user profile has its own registry hive for user-specific configuration. Everything above is still the same in Vista as it was in NT 3.1.

There are two database engines that have been known as Microsoft "Jet", known as Jet Red and Jet Blue. Jet Red is also known as the Access database engine. It is a fairly featureful SQL database. Jet Blue is now officially the Extensible Storage Engine (ESE), and has been a system component since Windows 2000, backing WMI data, Active Directory, Exchange, and others. It is an ISAM database and is optimized for large sparse tables and also supports a transaction journal. Both are 100% user-mode and were not a part of the initial release of Windows NT. Microsoft has said that Jet Red is depreciated, and that future versions of the Access database engine will be integrated with Access and not have a public interface. Jet Blue's interface is well documented and will continue to see use for some time to come. Both being user-mode, dependent on Win32 and the wrong type of database (relational vs hierarchical), the Jet engines would not be suitable replacements for the registry.

SQL Server is a high-end SQL database engine. It was rumored that WinFS would use SQL Server Express and that Microsoft eventually plans to move some of the services that use Jet Blue to SQL Server (such as Active Directory). In any case, SQL Server is an even less possible replacement for the registry.

Microsoft has not gotten rid of the Registry in Vista. In fact, the new boot manager uses a registry hive to store boot configuration, replacing the old boot.ini.

Re:What is the registry in Vista?

[textureglitch]

Microsoft has not gotten rid of the Registry in Vista. In fact, the new boot manager uses a registry hive to store boot configuration, replacing the old boot.ini.

Oh wonderful. Does this mean that WHEN Windows crashes and refuses to boot up again I'll have absolutely no way of editing the boot configuration with a rescue disc?

Re:What is the registry in Vista?

[Foolhardy]

Like I said, you can mount any registry hive on a running Windows system, including the %SYSTEMDRIVE%\Boot\Bcd hive. bcdedit.exe does exactly that. A windows-based rescue disc can just as easily mount the hive for editing. It will make editing boot config from a different operating system's rescue disc a lot harder, though. More about bcdedit.exe and the BCD hive

To mount a registry hive manually from regedit, select either HKEY_LOCAL_MACHINE key or HKEY_USERS and select File->Load Hive. Note that you can also save and restore any key as a hive by selecting "Registry Hive Files" as the file type during export or import. See also reg.exe's load, unload, save and restore functions.

Hmmm... Upgrade to Ultimate Vista perhaps?

[pegr]

"If all the changes succeed, the application commits the transaction and the changes are applied, but at any time up to that point the application can roll back the transaction and the changes are then discarded."
 
What, was my credit card declined for my upgrade to Vista Ultimate Edition?

Not exactly "error recovery"

[Ancient_Hacker]

Recovery? The Windows Registry is the exact opposite of recoverable:

• "Regedit" has no Undo command. It applies changes immediately.
• The Registry file is in a proprietary and undocumented format.
• One scrozzled byte in the Registry can make it completely broken and make the system unbootable.

Not exactly most people's idea of robust and recoverable.

Re:Not exactly "error recovery"

[bmajik]

The registry can be backed up and restored, in whole or in subsections.

The format of the registry is largely irrelevant, but it is described to some extent in the "Inside Windows xxx" series of books (which Russinovich co-authored)

Which specific byte would you "scrozzle" in the registry to render a machine unbootable? How and why would you do it?

Extra credit:
Which files under /boot, /etc, /sbin, and so on would you be willing to stake your career on being "safe to corrupt by 1 byte" and still guarantee a bootable system?

There are things to like or dislike about either a registry based approach (opaque data storage with a defined interface) vs a flat text based approach (clear data storage with an undefined interface). I don't think you make a compelling "anti-registry" argument with the points you list, however.

Re:Not exactly "error recovery"

[DeifieD]

It does have an undo command, just not Edit - Undo. Welcome to a post Windows95 world.

If you screwed up the registry you could always make them bootable again, you just had to screw around at the command line. Linux does not make leaps and bounds over this fix. X not booting? Time to go screw around at the command line again.

Re:Not exactly "error recovery"

[Anonymous Coward]

I am surprised that no one mentioned the registry should be a CVS revisioning system. Being able to roll back/audit and have full control on what gets to read/write registry should help on security.

Re:Not exactly "error recovery"

[Doctor Crumb]

You are confusing a windowing system (X11) with an OS (Linux). While you may have to "screw around on the command line" to get X working again, everything else will continue to work just fine (filesystem, webserver, internet, etc), all of which you can use either from a virtual console or a remote connection. If explorer.exe won't start, how exactly do you fix that without sitting down with a recovery CD?

Re:Not exactly "error recovery"

[fabs64]

Difference being you're not supposed to modify things in /boot and /sbin for all your settings, and /etc is text and therefore much harder to screw up. (you could put an EOF as the first byte in the file, but the system will still probably at least give you a "file x is empty" error message).

What you've said is correct, the gp's gripe is really about using a binary configuration file.. a fairly stupid decision but that is only my opinion.

My argument for flat text (or xml or whatever) over binary, is the same as every argument about the two for the past 5 years, binary is too system specific, reading/writing it is almost always more painful (faster though), reverse-engineering the format is always a much bigger job.

The "defined interfaces" argument doesn't really wash, you can define an interface just as easily for a text file as you can for binary, however with text someone may be able to edit it by hand or write their own interface, with binary this is much more difficult.

I think what it really comes down to is this; If you decided to write a new OS today from scratch and wanted to have a central configuration database (a good idea, as shown by /etc), would YOU come up with the windows registry?

Re:Not exactly "error recovery"

[Zwaxy]

Viruses which immediately kill their hosts don't survive very long. Modern viruses mostly want cause as little disruption to their host as possible in order to avoid detection while they carry out their owner's bidding.

Re:Not exactly "error recovery"

[Penguinisto]

What is that magic, dangerous byte? If it ever existed, wouldn't be trivially easy to write a virus and scrozzle it? Where are these viruses?

You can't pump V1@grA spam or DDoS packets out of a dead machine, and malware writers are definitely in it for the cash nowadays. (IIRC, a large percentage of malware specifically hides in the registry...)

/P

Re:Not exactly "error recovery"

[Talchas]

/boot - kernels/initrds I'm not using (which should be tossed out). configs. I don't know what would happen if an insignificant byte in grub.conf changed, but I generally wouldn't bet on being safe. /sbin - oddly enough, lots. fdisk/cfdisk. fsck + friends would probably boot somewhat. e2image, e2label, ctrlaltdel,shutdown,mkfs*,installkernel. /etc - much of it, just to boot. Boot and have everything work, far less of it - but at least I could usually boot and I would usually know what was broken.

But yes, I understand your point. However - a broken byte in an unbacked up (yeah a bad idea) registry that causes complete failure is far far harder to find than one in /etc, as the one in /etc would generally be in a single plain-text file, and you'd often be able to guess which file from error messages. /sbin and /boot are binary files that don't contain custom configuration data - if something breaks, you can easily replace it without thinking hard. If you lost all of /etc or the registry, you would have to reconfigure your system, which may be far more work.

Re:Not exactly "error recovery"

[EvanED]

The registry is no more vulnerable than any number of Unix config files. An extra newline in /etc/password will cause the same problem.

Even if you accept that, it's still easier to diagnose and recover from the extra newline.

Re:Not exactly "error recovery"

[Beryllium Sphere(tm)]

I've been known to put my /etc into cvs.

Re:Not exactly "error recovery"

[CliffSpradlin]

Just to be clear, there are some standard unix programs that store binary configuration information in /etc, such as tripwire. Either way, you can quite definitely completely fuck up your system by editing the wrong files in /etc. /etc is NOT a central configuration database by any means, and it certainly is not one with a standardized interface. This is not to say that the registry is any better or worse. (Note, I hate the registry, but it's largely in part due to the below reason)

What it comes down to is it's not the configuration system provided at fault as much as how the system is used by applications and system libraries. Any configuration paradigm can be abused.

Re:Not exactly "error recovery"

[drsmithy]

Difference being you're not supposed to modify things in /boot and /sbin for all your settings, and /etc is text and therefore much harder to screw up. (you could put an EOF as the first byte in the file, but the system will still probably at least give you a "file x is empty" error message).

Users aren't supposed to modify the Registry, either.

What you've said is correct, the gp's gripe is really about using a binary configuration file.. a fairly stupid decision but that is only my opinion.

It was designed in the late '80s (well, arguably the late '70s) when a 20Mhz 386 with 4MB of RAM was "bleeding edge". Text-parsing is expensive.

I think what it really comes down to is this; If you decided to write a new OS today from scratch and wanted to have a central configuration database (a good idea, as shown by /etc), would YOU come up with the windows registry?

Windows NT wasn't designed today, it was designed in the late 80s.

Re:Not exactly "error recovery"

[fabs64]

How is the interface to /etc not standard..? Last I checked we have the navigation of filesystems pretty down-pat.
Obviously what an app puts into its configuration files is undefined, but I'm pretty sure you can put Strings in the registry as well?
The registry is essentially a (bad) re-implementation of a filesystem, it's no more standardised (less so really) than just USING the filesystem.

Re:Not exactly "error recovery"

[manastungare]

either a registry based approach (opaque data storage with a defined interface) vs a flat text based approach (clear data storage with an undefined interface).

The GP's point was not entirely about automatic recovery from a broken structure; it was about human-led recovery. In a binary dump, even with defined interfaces, a single byte can render the entire structure non-human-readable. With a plain text file, a human can look at it and try to diagnose the problem -- simply by looking.

Re:Not exactly "error recovery"

[fabs64]

So about when did /etc become the de-facto standard in Unix for configuation?
"It was designed a long time ago" is not a valid argument for how something is today.
"Users" aren't supposed to modify the registry, but it's still where apps store settings, what app is storing settings in /sbin or /boot?

Re:Not exactly "error recovery"

[drsmithy]

However - a broken byte in an unbacked up (yeah a bad idea) registry [...]

The Registry is automatically backed up at the completion of a successful system boot. This has been true since at least Windows 2000, and probably longer.

Re:Not exactly "error recovery"

[newt0311]

I am being picky here but oh well. Most UNIX systems and at least Linux if nothing else have gotten rid of the EOF character in favor of just recording the file size seperately. Tossing a ^D or multiple ^Ds in a file doesn't have an effect on reading it though the parser used may make a difference.

Re:Not exactly "error recovery"

[drsmithy]

So about when did /etc become the de-facto standard in Unix for configuation?

No idea, but since /etc and the Registry are worlds apart in function, purpose, usage and capabilities, it's hardly relevant.

"It was designed a long time ago" is not a valid argument for how something is today.

It most certainly is. You can't just remake something as big as a modern operating system from the ground up every time the latest cool fad comes around.

(And I'm sure you'd be happy to defend the train wreck that is /etc with some line about how long it's been around, so that's just the way it is.)

"Users" aren't supposed to modify the registry, but it's still where apps store settings, what app is storing settings in /sbin or /boot?

How is it relevant ? The complaint was that twiddling the right bit in the Registry could render the whole system unbootable. The point is that the same thing can happen on any system.

Re:Not exactly "error recovery"

[ScrewMaster]

How is it relevant ? The complaint was that twiddling the right bit in the Registry could render the whole system unbootable. The point is that the same thing can happen on any system.

Not to dispute your conclusion, but one relevant aspect is that it seems to happen one hell of a lot more often in registry-based systems. For example, my development system at work is XP-based, and for no apparent reason my Recycle Bin disappeared. Off the desktop, My Computer, just gone. No idea where it went. I first tried Microsoft's "Recycle Bin Recovery" procedure to re-enter the appropriate keys: no dice, the corruption was deeper. Fortunately found a registry tweaks site that had a reg file that restored all the required keys. I was lucky that time, other times I've been less so.

As a developer, I do use the registry for storing some settings, but only non-critical ones. I don't like the idea of putting my customer's configuration data (and in the software I work on, there's a lot of it) into the registry. Too risky, and there's no easy way for a customer to back anything up. Personally, I've never really seen the light when it comes to application data storage using the registry. I've been burned too many times.

Re:Not exactly "error recovery"

[bmajik]

There's no file to look at. The computer didn't boot.

If you've booted the computer off of some other media and are attempting to repair the configuration info, you're using a special tool to do it (like a text editor for /etc files, or a registry editor for the registry)

Also, I've not seen evidence that a single byte defect renders the registry unreadable or the computer unbootable. I don't dispute that this is possible, but I'd be curious to know if this is a practical issue or a theoretical complaint. For instance, there is redundancy in the registry (last known good being a different spot than current control set, for example).

Re:Not exactly "error recovery"

[bmajik]

Are you seriously asking?

what is the comment convention?
What is the whitespace convention?
Does this version of this flavor use spaces or tabs in /etc/fstab (or is it /etc/vfstab on today's OS?)
What are the file locking semantics?
Which set(s) of files are logically related to the same peice of functionality?
What character encodings are supported?
What _line termination_ is supported?

nfs export your /etc directory r/w some time. Let someone edit your /etc/fstab in TextEdit on a Mac. Make sure they cut-and-paste from a sample they saw on a website, or that they got via an email. Are you willing to bet your uptime on that working correctly? Even if you control the email or webpage they're pasting from.. are you _sure_ it would work? That there'd be no conversion problems?

Have you ever done a *nix upgrade on a machine? What happens to most of the files in /etc? The answer is NOT "your changes are seamlessly rolled forward"

Re:Not exactly "error recovery"

[larry bagina]

I've been known to put my /penis into vagina.

Re:Not exactly "error recovery"

[bmajik]

Care to explain that? The "list of OSes" information that i think you're talking about is in the BCD store, which i dont think is part of the Vista registry. You muck with that via bcdedit, and the bcdstore is specifiable as an argument.

Re:Not exactly "error recovery"

[daverabbitz]

The GP's point stands. The registry is no more vulnerable than any number of Unix config files. An extra newline in /etc/password will cause the same problem. Firstly, there is no such file /etc/password, it is /etc/passwd since K&R didn't like typing and skimped on letters.

Secondly, I just tested this and it is completely untrue. extra newlines have absolutely no affect on the functioning of /etc/passwd in a linux-pam system.

Thirdly, transparent human readable and logically laid-out config files in /etc are much easier to diagnose and fix than an arbitrary (and slow) binary registry where the typical entry is something like "0x67386768787ab7aab87b78cd87687" and the key is "GRPAccountMaskFoo" (made up). Even when you take into account that files in /etc don't have any uniform format.

Sadly, some unix developers are oblivious to this simple truth, and are intent at re-creating the windows registry and COM *glares at Gnome developers*.

Re:Not exactly "error recovery"

[TrancePhreak]

Clearly trolling.. Since the days of at least Windows 98 there were built in mechanisms to backup and restore the registry. Windows XP even does this on its own when needed.

Re:Not exactly "error recovery"

[weicco]

If explorer.exe doesn't start, user gets (at least in XP) blue screen (not that BSOD, but just a blue screen) in front of him. Then user can press CTRL+ALT+DEL, execute Task Manager and use that to start other applications. You can test this by terminating all explorer.exe processes with Task manager. But you must be quick, since XP will try to automatically start shell process if it sees it has been terminated. Btw. there's a registry entry which tells which shell should be started: HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\WinLogon Shell=Explorer.exe

Maybe you are confusing windowing system to NT kernel :)

Re:Not exactly "error recovery"

[Talchas]

Huh, to where? I recently had to fix a computer which wouldn't boot because part of the registry was corrupted, and I couldn't find any backup other than in a System Restore point. Maybe I just didn't know where it was.

Re:Not exactly "error recovery"

[Keeper]

System restore is the backup mechanism. If you've got system restore enabled, it manages creation of restore points (which include registry backups) and deletion of old restore points on a regular basis. If your registry is corrupted, it is supposed to fallback to a version of the registry contained in a previously generated restore point. If none of the restore points have a valid snapshot of the registry (ex: you've disabled system restore or your drive is horked) you're SOL.

Re:Not exactly "error recovery"

[weicco]

But tell me, how do you add extra new-line to registry? It's much more easier to mess up text files with text editor than mess up registry with registry editor. And trust me, I've done both :)

Re:Not exactly "error recovery"

[Penguinisto]

nfs export your /etc directory r/w some time

Dunno about him, but... I'm crazy, not stupid.

/P

Re:Not exactly "error recovery"

[Bu11etmagnet]

Extra credit: Which files under /boot, /etc, /sbin, and so on would you be willing to stake your career on being "safe to corrupt by 1 byte" and still guarantee a bootable system?

/etc/motd ?

Re:Not exactly "error recovery"

[Penguinisto]

True, but it doesn't always work as advertised, especially in teh case of a half-munged registry. Say a user screws something up, thinks "reboot!", and does so... after it reboots, the registry is still hosed, and if it can get even barely to a running state (not necessarily one in which you can do anything...) Then the bad registry now gets copied over the backup, and now the once-good registry is now gone. I think (don't know for sure) that was one of the reasons and rationales for putting in the System Restore stuff when XP and Win2k3 came out - so that at least there's something to fall back on.

/P

Re:Not exactly "error recovery"

[Foolhardy]

The BCD store is implemented as a registry hive, kept in either EFI or \Boot\Bcd. It's not mounted most of the time though; the boot loader mounts it long enough to read it and bcdedit mounts it just long enough to make an edit.

On the topic of one byte corruption of binary databases, what about the filesystem database? How many FSes could break boot with a single corrupted byte, e.g. in a filename record?

Re:Not exactly "error recovery"

[Talchas]

OK, then, that makes sense. Fallback mechanism must have screwed up or something, as copying all the old registry data from a recent restore point worked just fine.

Re:Not exactly "error recovery"

[Ancient_Hacker]

>The registry can be backed up and restored, in whole or in subsections.

... not when it's scrozzled so the ReadRegistry API crashes to the BSOD. I'm not typing thru my hat-- I've had this happen more than once.

>Which files under /boot, /etc, /sbin, and so on would you be willing to stake your career on being "safe to corrupt by 1 byte" and still guarantee a bootable system?

Apples and Oranges. I can edit the text files and cp the binary files. Can't do either with the Registry.

>The format of the registry is largely irrelevant, but it is described to some extent in the "Inside Windows xxx"

Well, when my Registry got scrozzled I looked all over the net for source code that could read a Registry and didnt find anything. So the descriptions out there are likely less than usable.

Re:Not exactly "error recovery"

[DeifieD]

What I meant by that was: It's easy to screw up Windows and go fix it if you know what you are doing.

It's easy to screw up Linux configs and go fix those too, whether it be webserver, internet, whatever. Albeit probably much easier.
  (Didn't need a windows CD pre NT4 or 2K, but that's kinda reaching).

Just some screwing around at the command prompt. I see your point on usability of the entire system though.

'Protected Processes' and PC games

[JanusFury]

A fairly common trend these days in PC games (mostly multiplayer ones) is the use of a kernel-mode windows driver (effectively a rootkit in most cases) to 'protect' the game from hacking. Many eastern (korean, taiwanese, etc.) game development companies opt to use this mechanism to secure their games instead of writing secure client and server code - for example, GunBound, Maple Story, Ragnarok Online, Rakion, etc... pretty much any MMO you see an ad for these days that isn't from a US or European studio uses this stuff for security. The basic mechanism it uses is that it hooks all the low level operations you can do on your system (file access, process access, etc.) and prevents you from touching anything related to the game. The end result is that you can't even so much as end-task a misbehaving game 'protected' by this driver.

With the huge amount of popularity this approach seems to have (I personally suspect it's a result of some very, very aggressive marketing on the part of the driver's developers), I wouldn't be suprised to see many games start demanding that users run them on Windows Vista, so that the 'protected process' mechanism can be used to fully 'protect' the games from users' interference. While you'd at least be able to end-task them, I can't say I see this as an improvement. It's saddening that many companies believe the solution to security is a series of hacks, workarounds, and black boxes - the only real solution is careful, methodical design and engineering. It seems very likely to me that within a few years, many PC games will refuse to run on anything except a Vista system with nothing but signed drivers loaded, and that's saddening. I dislike the notion that I am denied even basic rights to investigate what an application is doing on my machine simply for the sake of 'security', when it's trivial to set up a second machine to inspect and modify a game's network packets and cheat all I want.

Re:'Protected Processes' and PC games

[mpapet]

the solution to security is a series of hacks, workarounds, and black boxes

Outside of some very specific gov't spooks contract work, no one is willing to adopt the other because it's more expensive and requires more thought and risk than the average PHB can handle.

Case in point: The number of cashless casino gaming systems that are centrally managed. The average casino manager understands their casino systems are a single, massive, point of failure. Just wait till they go to cashless floors and someone engineers a jackpot for themselves.

Security programming is hard. Really hard. The developers at these companies may _want_ to do the right thing, but they don't because of the complexity and limited resources.

The big-digital-bank-robbery just hasn't happened yet.

Re:'Protected Processes' and PC games

[nuzak]

> Just wait till they go to cashless floors and someone engineers a jackpot for themselves.

That would be a neat trick to do with the end-user UI of a slot machine. Physical security is a pretty important first step.

> The big-digital-bank-robbery just hasn't happened yet.

Actually, when you look at identity theft and fraud, you can see it's happening every single day. Think breadth and not depth.

Re:'Protected Processes' and PC games

[admactanium]

> Just wait till they go to cashless floors and someone engineers a jackpot for themselves. That would be a neat trick to do with the end-user UI of a slot machine. Physical security is a pretty important first step.

unless i'm misunderstanding you, slot machines already work this way. you can play a slot machine with cash, or you can use a bar-coded receipt from other slot machines. i've sat down and played a few slot machines with no cash and, even after winning some money, stood up and walked away with my winnings without any additional cash. at some point i did have to put cash into the system (the first slot machine) but you can interact with slot machines, execute transactions (plays) and be paid all without cash.

Re:'Protected Processes' and PC games

[mpapet]

It's clearly not going to happen on the floor in front of a slot machine.

The backend running those games is on a network.

What are the chances they've got a gateway that goes to the interweb?

What are the chances they are relying on Windows for their security?

The people running the casino are pretty busy with their day-to-day stuff, not infosec.

Re:'Protected Processes' and PC games

[Anonymous Coward]

The basic mechanism it uses is that it hooks all the low level operations you can do on your system (file access, process access, etc.) and prevents you from touching anything related to the game. The end result is that you can't even so much as end-task a misbehaving game 'protected' by this driver.

This is not true, and it's also not what a rootkit is. These games use rootkits to hide files and drivers from the Windows API, which you can do yourself just by creating a share that starts with '$' or a registry key with a NULL in it. The game is not monitoring your every action, it is simply hiding itself from your interference (and potential reverse engineering).

It goes around and comes around though - the most commonly installed rootkit is Daemon Tools, which uses rootkits to hide from games. :-)

Re:'Protected Processes' and PC games

[Joe The Dragon]

I don't think that you will see chips go a way even more so in poker games.

Re:Hmmm... Upgrade to Ultimate Vista perhaps?

[anImortal]

Just like any database -- from any vendor. If you start running out of system resources, your transaction will likely roll-back.

I've seen that before...

[harry666t]

...Somewhere... ...Yeah, I know where!

So, they reinvented the wheel once again? It seems to be: every database more complex than a flat file processed by a pair of simple perl scripts has support for transactions like this. So they invented nothing, just applied an old patch to new code.

Re:I've seen that before...

[EvanED]

It seems to be: every database more complex than a flat file processed by a pair of simple perl scripts has support for transactions like this.

Yes, DBs do. But traditionally file systems don't. The only other system that provides this that I know of that isn't a full-fledged database is VMS.

Re:I've seen that before...

[textureglitch]

Yeah, they reinvented the wheel again, but they still haven't figured out how to make it round.

i will explain what he is saying...

[DeadDarwin]

when the NTFS files access the GHY it extends a random signal to the DFT which emulates the chip switch Architecture (CSA). Hard drives can be extruded and raised to the eye level, the apt facing the sun and look for errors at the kernel module. Then the stubs in the IIOP cloud extends its virginity toward the distributed computing components. thats how the Eifel tower was made. Hope I cleared your doubts.

Sysinternals Utilities

[NearlyHeadless]

I just noticed today that Russinovich's utilities are available in a single-file download: http://www.microsoft.com/technet/sysinternals/Util ities/SysinternalsSuite.mspx

Re:Sysinternals Utilities

[PetoskeyGuy]

I have a spider of sysinternals.com and all file downloads in a 24MB Zip file from the day it was announced Mark was hired by Microsoft. Not sure if anyone is interested in it. I don't have any where to host the files but I can try to email them to someone via Gmail if anyone can host it or wants a copy.

Re:Sysinternals Utilities

[user24]

you won't be able to gmail exe files, even inside a zip unless you rename the extension.

slap it on mytempdir.com or rapidshare.de and then anyone can grab it.

Re:Sysinternals Utilities

[Reziac]

Well, I'd be interested, in case there's anything I missed in my own smash-and-grab from that fateful day. (Not very organized here, I'm afraid)

Mod Parent (Farther) Up

[jthill]

This is cross-facility transaction management: registry and filesystem updates combined into a single transaction. The example in TFA that an entire install can be atomic: multiple filesystems, registry, everything appears complete and as requested, all at once, or it never happened.

It's extensible, if TFA is to be believed at all, and the facility works. It's actually there and in use, rather than an it'll be there someday and won't it be wizzo promise, so I'm in "trust-but-verify" mode. It'll be interesting to see if it's actually extensible by coders excluded from the Blessed Realm.

Whether it belongs in the kernel or not is all but irrelevant: so what if it could be implemented as a userland service? Where they choose to put their code is up to them. They wanna play micro-kernel, Giga-kernel, or kernel-a-la-carte, that's up to them; the only question is whether the result is as reliable as they want us to believe.

If it is, it will make building absolutely-bulletproof applications a whole hell of a lot easier. I know something about that. Being able to say ~`if (!quickcheck()) die(fromhere());`~ without leaving a mess means, just for starters, that you don't have to concoct a file format for complex data; you can just use the filesystem, and that choice won't complicate your life. Big win. Big.

Re:Mod Parent (Farther) Up

[nanosquid]

The example in TFA that an entire install can be atomic: multiple filesystems, registry, everything appears complete and as requested, all at once, or it never happened.

UNIX has been able to do atomic installs/updates without a general purpose kernel transaction manager for decades; you simply don't need it for that. More importantly, a kernel transaction manager simply doesn't solve the problem because partial/inconsistent installs can exist for many other reasons besides interruptions; assuming that this case doesn't happen because you're using a transaction manager is a surefire way of making the system less robust.

Whether it belongs in the kernel or not is all but irrelevant: so what if it could be implemented as a userland service? Where they choose to put their code is up to them.

Well, it's "irrelevant" only if you don't care about things like performance, bug count, or reliability.

If it is, it will make building absolutely-bulletproof applications a whole hell of a lot easier. I know something about that. Being able to say ~`if (!quickcheck()) die(fromhere());`~ without leaving a mess means, just for starters, that you don't have to concoct a file format for complex data; you can just use the filesystem, and that choice won't complicate your life. Big win. Big.

If you write code like that, I hope you're actually just a CS undergraduate with an overactive imagination and people let you nowhere near writing applications that need to be "bullet-proof".

Re:Mod Parent (Farther) Up

[jthill]

So "UNIX" has a non-general-purpose or non-kernel transaction manager, and has for decades. Seeing as how UNIX has been implemented at just about every conceivable point on the spectrum from microkernel to OneBigBlob, we can take that as a general assertion that you can find some kind of transaction manager out there somewhere for some variant of UNIX. Good. So what? Now Vista's going to have one too. Good for them.

I'm already on record here as saying that VM boundaries aren't the only way to maintain module independence, isolation, abstraction, coherence, whatever the POV-du-decade is on good design. You've got it or you don't. <shrug> I still don't see how their choice of toolset, whatever it is, is relevant here.

And what, pray tell, does your recovery code for ~can't-happen~ errors do if it isn't to spill its guts and get out without adding to whatever mess it encountered? Are you saying you *don't* put cheap sanity checks at strategic locations? That you don't try to recover when they fail? That recovery should leave detritus? That you've never anticipated a situation that needs human intervention to fix? That I should have put detailed recovery-routine code in a /. post? That a pre-existing rollback facility *won't* greatly simplify bailout code? I'm having a really hard time making sense of that criticism.

Oh, well.

This is...

[Organic Brain Damage]

Windows Kernel. This is Windows Kernel on ACID. Any questions?

Re:This is...

[textureglitch]

That expression is just sooooo wrong. Have you ever been on acid? It's only wild and colorfull and amazing and explosive and out there for YOU, not for everyone else. You end up examining the pattern of your floor tiles for 8 hours straight and to you it's the most fascinating thing you've ever seen in your entire life, but to everyone else you become the most boring person in the world. ...come to think of it, your analogy was fairly accurate after all.

Re:learn from biology

[quux4]

OK, good points about users turning off security protections. But hardly unique to Vista; you can do that in any OS. It's the price of popularity.

Transactional File Support?

[endianx]

I really think that transactional file support is cool. I try and make sure any software I design works on multiple operating systems though. Now I know code that specifically uses DTS won't work on other operating systems. However, I write in .NET, which means the specifics of how a feature works doesn't matter. For example, I know Mono implements the FileSystemWatcher class using a completely different mechanism than Microsoft does.

Anyway, what I am asking is if anybody knows if the Linux Kernel and/or popular file systems have support for this. How about Mac?

Re:Transactional File Support?

[EvanED]

Anyway, what I am asking is if anybody knows if the Linux Kernel and/or popular file systems have support for this. How about Mac?

AFAICT, this is a feature unique to NTFS/Vista and maybe VMS. (The latter wouldn't surprise me; NT is a descendant of VMS designwise.)

Re:Transactional File Support?

[Assimilated Member]

This is unique to NTFS. It is also the only implementation of transactional file systems that can coordinate cleanly with a database. For VMS, you were thinking of RMS Journaling. HP still keeps the manuals on-line. As you can see, RMS/J with recovery unit journaling is "close". Combine RMS with DEC ACMS (a TP monitor) and DEC RDB (a relational DB) and you are very very close to what Transactional NTFS provides "but not quite there". Further, RMS/Journaling was said to be a dog -- and it was very expensive. When you start looking closely, you see many many differences between NTFS and any other attempt. For instance, it work works closely with the Volume Snapshot Service to provide transactionally consistent backups; transactional support for memory-mapped files; and enhancements to the mini-filter framework.

Can we please drop file systems in favor or RDBMS?

[master_p]

Instead of putting more and more RDBMS features in file systems, why don't we drop file systems entirely and use RDBMS instead? RDBMS already provide all the required mechanisms for information management (transactions, security, duplication, distribution, strong typing, queries, caching etc), and the concepts of file/directory/hard-soft link are outdated and create more problems than what they solve, in the end.

So what is this thing?

[mosel-saar-ruwer]

The registry engine is implemented in kernel mode as an executive subsystem (inside ntoskrnl.exe), where it is known as the Configuration Manager. Registry hives use a transaction journal (like many filesystems do) to avoid corruption during a power failure or crash...

So you're saying that the engine which drives "the Configuration Manager" is neither Jet Red, nor Jet Blue, nor SQLServer Express?

So what is it? YAMIHDE [Yet Another Microsoft In-House Database Engine]?

Everything above is still the same in Vista as it was in NT 3.1.

I could have sworn that I read a few years ago that they were ditching the existing registry engine, and were going with a new engine for Longhorn/Vista.

So did that initiave prove to be YALFTEUOTCRF [Yet Another Longhorn Feature That Ended Up On The Cutting Room Floor]?

Re:So what is this thing?

[Foolhardy]

So what is it? YAMIHDE [Yet Another Microsoft In-House Database Engine]?

Yes, and a fairly old one. The NT registry format was created at the same time as NTFS. Here are some pages about it. There are at least two versions of the format-- the newest one was introduced in XP. XP also made loading registry hives more efficient, and allows much larger hives to be created and loaded.

The Jets, the registry, NTFS, FoxPro (which they bought), and SQL Server are all the Microsoft multi-purpose binary database engines that I can think of. Jet Blue, the registry and NTFS are the only ones that the OS uses for itself today.

I could have sworn that I read a few years ago that they were ditching the existing registry engine, and were going with a new engine for Longhorn/Vista.

There was something about Cairo being a directory, registry, filesystem, etc. (and everything else under the sun). This blogger remembers it too.

I know that some Microsoft teams have been using the registry less, e.g. IIS6 now uses a new XML database for config instead, but not all of MS's many developers are moving in the same direction. The registry is required very early in the boot process to determine which drivers are necessary to load to access the boot volume and filesystem. Unless that changes, there would be little reason to replace the entire registry as it is with something else.

Re: DEC DTM

[BBandCMKRNL]

The actual innovation is making a Kernel Transaction Manager, along with a resource manager for the filesystem. The KTM means that transactions can be inherited from parent process to child or joined by a cooperating process. Having a transactional filesystem means that all file operations can be all-or-nothing.

Yes. DEC DTM does this with the file system (RMS). I've been away from DEC and VMS for too long to remember if it handles the cooperating process part. Perhaps someone can refresh my memory.

A file system is still needed

[gr8dude]

How can that be possible? The DBMS itself sits on top of a file system; you can hide this detail from the user using a layer of abstraction, but the file system is still there.

YES!!!!

[StarKruzr]

HAHAHAHAHAHAHAHAHAHAHAHA

I knew you wouldn't be able to resist. It never ever fails. I'm convinced I could troll you on a Russian forum about opera and you would STILL find it and HAVE to say something.

Sure, I'm a nut

[StarKruzr]

You managed to find ONE POST about you in ONE PLACE on the Internet. I put out the bait and you chomped onto it like a hungry piranha.

I think to be "stalking" someone you have to actively track them down, Alec. This was about as passive as it gets.

And the best part? I know that you will keep this up as long as I do because you can never, EVER give up having the last word. It's hilarious. You are the most reliable infinite-supply lulzcow on any of the internets.

I can't decide what I love more -- your bombastic language or the way you pretend to be a third party in every post you write about yourself.

This just keeps getting better.

[StarKruzr]

I simply adore your hierarchy of human beings that is based on what kind of and how much software they've developed. Wonderful.

Are you STILL clinging to the TPU forums? I suppose there are enough passive little sycophants there to jerk you off that you just can't resist hanging out there, can you?

Not that it matters, but I happen to be a graduate student in computer engineering, and expect two papers I've submitted to be published within the next two months. What do YOU know about low-power reconfigurable computing using nanotechnology, "Iron Man?"

Also, in point of fact (it's personal note time!), I actually just broke up with my girlfriend tonight. It just wasn't working out -- she's 4 hours away from me driving and in the end, despite how attracted we were to each other, we just didn't have that much in common. Sad, but it happens.

Nevertheless, I suspect that over the next couple of weeks I'll still get about 4X more tail than you do, since you must be a true social maladjust in person.

roofles

[StarKruzr]

You are so incredibly predictable, but now you're getting boring, using the same old tropes you always have -- responding to graduate education with "boy," claiming hundreds of conquests with your 3" dong, and pointing again and again to your silly Delphi software as evidence of your uebermanhood.

Can't you get some new material, you poor, twisted fool? Come on. Keep me entertained over here!

You can't write.

[StarKruzr]

Did you pass your grammar classes in elementary school, APK?

WHY do you keep pretending to be someone else?

And why DOES MySpace suck so much?

Dear Mr. Kowalski,

[StarKruzr]

Your "writing style" is inimical. You couldn't hide it behind a mountain range. It is obvious that no one has "agreed" with you except for your own AC sockpuppets.

I like picking fights with you because I know you cannot ever let anything go. I knew it would be utterly impossible for you to see something posted about you somewhere without your massive ego needing to reply to it with your usual string of self-important bombast. Do you actually presume that I have anything personally invested in calling you a narcissistic twit with an inflated sense of his own internet celebrity? You are a sad little flea, Alec, and worth my time only so long as I derive enjoyment from wasting yours.

Also, regarding my being a "critic" -- the funny thing is that NOT "any idiot" can do that. I don't even consider myself a "critic," but that's beyond the point -- to be a critic of anything, a serious critic, what is required is a broad comparative knowledge of the subject, not having created something in the subject themselves. It is not often, for example, that a film critic is an accomplished filmmaker. They are two different skillsets.

I also note that you're not commenting logged in. Why's that, pray tell?

"Everyone knows"

[StarKruzr]

Right. Everyone on Slashdot. Because everyone on Slashdot is reading this thread, right? Everyone is very, very concerned with the fact that poor APK is getting trolled.

You don't have a single original thought in your head, APK. It's really quite amazing. It's like your brain is a finite state machine, going from "show me what you've done" to "you're just a student" to "you don't have a degree in psychology" and back to "show me what you've done" again.

You live such a sad little life. And I am so, so enjoying wasting your time.

Figures you'd be a conservative bonehead, too

[StarKruzr]

Now lesbians are "mental aberrations and genetic errors?" Intolerance AND stupidity in one package! Man, what a bargain you are.

You've officially crossed the Rubicon, chuckles

[StarKruzr]

Your ranting actually caused Slashdot to cut the post to a "Read more..." link.

Oh, now you're Russinovich's BUDDY?

[StarKruzr]

That's rich.

Remember, APK -- all you have to do is stop replying to my posts and this is over. I won't hunt you down and stalk you the way you did to me.