What, exactly, does the registry have to do with spyware and viruses? If you're saying that the large size permits hiding by obsurity, a file system structure to store the data would be the same. If you're saying that it provides some kind of security hole, please enlighten us as to how. If anything, giving each key its own security descriptor makes it MORE secure than a flatter version implemented in a filesystem.
Why have this whole extra registry implementation and toolset when it's almost completely redundant with the filesystem and all the familiar tools that operate on the filesystem?
That's a good question.
My SOFTWARE registry hive has 99865 keys and 143228 values in it. User has 18154 keys and 106945 values. Added to SYSTEM, my registry contains a total of 131181 keys, and 291410 values. 131k keys is more than 10x larger than the 11584 files in my \WINNT directory. My main disk has 351k files in 20k directories; moving the registry structure there would mean six and a half times the number of directories and a 83% increse in files, assuming one file per value. I'd characterize my computer as one a typical developer would have.
It's not completely unreasonable, though: NTFS already stores files smaller than about 900 bytes directly in the MFT file record (space that would otherwise be wasted) and NTFS's extended attributes (or even file streams) would provide a nice implementation of key values, but there's still the issue of 6x directories for structure. Although they have the same basic structure, the scale of the registry and filesystem are each quite different. The registry sucks for large entries: the entire SOFTWARE hive is only 26MB, but then again filesystems traditionally suck at tiny files and directories. I suppose Microsoft could've added a special directory flag that optomizes for tiny directories. I suppose the hierarchy in the registry could be flattened/simplified to require less keys, but that would give up security granularity (but I'm sure some could give), and would require a very major overhaul at this point.
The registry hasn't changed much since it was first introduced in its modern form in NT 3.1, so let's look at how things were back then. I've got a VM of NT 3.51 with Office 97 and VB 5 on it. In SOFTWARE, SYSTEM, and USER, there are 12673 keys and 18851 values. Compared to 254 directories and 4k files, moving the registry to the filesystem would've meant 50x the directories and more than 4x the files (or extended attributes, assuming you could get apps to use 'em).
There's a reason that different types of databases exist. Filesystems are general purpose DBs that have direct access and handle large files well. SQLish table oriented databases are used for lots of things that flat files directly on the filesystem suck for. The registry is a specialized database for the high structure low leaf data that comprises config files. Windows is hardly the only software product that uses a special binary config database; see the 'registries' that Mozilla or Eclipse use for examples. Could they have been implemented directly in the file system? Sure... but would it've been the best way?
Sometimes you don't want the complexity of being able to have different settings for different users, but do want to be able to take your settings with you wherever you copy the app.
That's a good point. Still, you don't want a central config system of any type in this case; an/etc directory wouldn't be any better. Besides, an app that donesn't store its settings per-user isn't multiuser. What is it supposed to do when multiple people try to run it at the same time on the same computer? You may not be using it that way, but someone may need it on a terminal server.
Er, you mean with File->Export and File->Import in the registry editor? If the machine with the settings isn't booted, use File->Load Hive to mount first so you can use export. I've transferred lots of apps this way.
I personally think the Windows Registry is the software implementation of the saying "putting all eggs in one basket".
Kinda like how putting all your filesystem metadata in one database is the software implementation of the saying "putting all eggs in one basket"? The registry is a binary hierarchical database optomized for config (small entries), broken into mountable volumes (registry hives), journalled and with hot backup functions. A filesystem is a binary hierarchical database optomized for large files, often journalled and with hot backup functions. If you want backups, see ntbackup: the "System State" option includes all the machine's hives. System restore also makes backups of the system hives.
Basically, they said that the registry was too successful for its own good; so many apps use it that it has almost become a general purpose database, which it was never intended to be. They say that one of the biggest problems was that there were never any comprehensive standards published on how it was to be used, so devs did whatever they wanted, which caused chaos that contributed to it becoming a mess.
The registry first existed for registering OLE document types in about Windows 3.0. At the time, it had almost no structure and responsibility. The first release of Windows NT (NT 3.1) made the registry much more important: everything that used to be stored in.ini files moved into the registry, along with all the driver and system config for a full OS. There were even compatibility shims created to redirect access to the old.inis into the appropriate registry keys. A couple of old.inis were kept, but only to placiate old apps. Notably, the entire NT shell used the registry exclusively, unlike DOS/Windows: for example, the HKLM\SOFTWARE\Classes tree originated in NT 3.1. NT also expanded the registry concept into multiple hives all mounted in a common hierarchy: each hive file is like a mounted filesystem. The machine's hives are in system32\config: a SYSTEM hive that the bootloader loads that drivers need to get the system started, a SOFTWARE hive that contains all of HKLM\SOFTWARE, and the SECURITY and SAM hives that store the accounts database and the machine's private keys. Each user also gets their own profile hive. Originally, the user profile consisted of only the user's hive, but NT4 made it a full directory for storing documents and other things. Before Windows 2000, a domain controller stored the entire user database in the registry, which led to some serious scalablity problems. I suppose they could've used a heavier database (like 2000 does), but at the time i'm sure it seemed like good code reuse.
Meanwhile, the Windows 9x/ME series still uses.ini files (like win.ini and system.ini) for much of the old 16bit core code, but uses the registry for the code they imported from NT, like the shell. COM (introduced in Win95) also uses the registry heavily for GUID registrations and such. Many of MS's own products countinued to use the registry for all their settings, but some of the newer versions are starting to move away. IIS 6 has a XML configuration "metabase", with the old registry entries kept only for compatibility..NET is threatening to use XML for more config, as is Vista. It seems to me that Exchange has also switched to an XML type thing recently.
Personally, I don't see what the big problems with the registry are. The registry is a hierarchial database system provided by the OS designed to store configuration information. The only differences between it and a/etc type config directory is that
The storage is managed by another database on top of the filesystem, instead of the filesystem db directly.
The seperation of config entries is handled by the db in the registry, whereas each app comes up with its own format in/etc.
The registry has several documented functions for hot backup and restoration, and has always been journalled (like the fs metadata). A lot of apps abuse the registry, but I think they'd do the same thing with their config files on another OS.
Far be it for me to diminish your story, but I did an F3 search for all my SID files (20327 of em, more files than any other music type) in the entire music directory, and it took about 15 seconds. Now that it's cached it takes about 3 seconds for successive searches on those directories. Most of the CPU time is spent in Explorer doing file name matching. dir \music\*.sid/s is similar. This is on an Athlon XP 2800, 1GB RAM, nVidia chipset, a 100GB NTFS volume and Windows Server 2003 sp1. Are you sure your filesystem isn't corrupted? What FS are you using, btw? You may need to defrag the MFT. When you say it hangs, do you mean the entire OS or just Explorer? Also, Explorer (shell) type searches will also search the index of ZIP files, so if you have a large or damaged ZIP file, Explorer might be choking on it. If you want the shell to quit doing that, unregister the zip folder support library: regsvr32/u %windir%\system32\zipfldr.dll
I'm not ready to pass judgement on GUI vs command line in general, but in this case on Windows, I'd hit WindowsKey+F to open a search box, put *.mp3 in the find description, ALT+L for location, type \MyFiles, enter, wait for the results, hit CTRL+A to select all, CTRL+X to cut, hit ALT+A to focus the address bar, stick \MyMP3s in there, hit enter, SHIFT+TAB to refocus the files pane, and CTRL+V to paste.
There are several steps, but fewer overall keypresses. (35 vs 49) One major disadvantage is that I'd have to wait for the search to complete before I could copy anything, whereas you can enter the entire command at once.
You can use NtCreateProcess to create processes that aren't Win32 clients, so have no GUI overhead. You can even use it to get a real copy on write fork (this is how SFU does it). The catch is that the new process can't use Win32 services: it can't link to kernel32, user32, gdi32, advapi32 or any other libraries that depend on them, since Win32 is required to use any of them. You have to use the functions that ntdll exports directly for IO, IPC, syncronization, etc. All the primitive functionality the kernel provides to user processes are there, but some things like sockets can be a pain, since you have to interface with TDI directly. Most of the standard Win32 programming doesn't apply for such a process.
Microsoft doesn't document the native API, so you have to find third party docs. The Windows NT/2000 Native API Reference by Gary Nebbett (ISBN:1578701996) is an excelent reference. Reactos is an open source clone of Windows NT, and it documents and implements almost all of the same internal functions. Sysinternals also has a short native example program. The books Inside Windows 2000 and Undocumented Windows NT are also quite useful.
An easier way might be to just write SFU apps. SFU processes aren't Win32 clients (SFU is its own subsystem) and it provides a UNIXy programming environemnt.
CreateProcess is not a syscall; it's a Win32 function exported by a Win32 client library, kernel32.dll. The internal API handled by the kernel is the native API. All of the syscalls start with Nt* and are exported by ntdll. The related syscall is NtCreateProcess, a function that creates a process object, loads ntdll.dll, maps an image section (for the program binary) and nothing else. It does not have a STARTUPINFO parameter.
CreateProcess from Win32 does call NtCreateProcess, but it also does a lot of other things, like create a startup thread and report the new process to the Win32 subsystem server (csrss), where the GUI parameters are used. The GUI is a core part of Win32, but Win32 is not a core part of the kernel. Almost all Windows software depends on Win32 (and so the GUI), but the kernel is happy to without win32: see the recovery console or first stage setup program. Full kernel, no Win32.
Before NT4, the Win32 subsystem was implemented entirely in user mode (mainly winsrv.dll hosted by csrss.exe for the server, kernel32.dll, user32.dll, gdi32.dll, advapi32.dll for client libraries used in programs that depend on Win32). Since NT4, much of winsrv's code got moved into kernel mode in win32k.sys. The kernel mode part of Win32 is all contained in win32k.sys; it's not part of the kernel, it just runs in kernel mode.
Economically speaking, no deaths are without consequenses. If it's preventable, then it can be calculated how much the solution would cost and how many deaths it would prevent. Those "non-dead" people earn incomes and pay taxes. If those expected taxes are greater than the proposed solution, then we have a winner. Of course, not all decisions are made based on pure economics. Many people are simply willing to pay higher taxes in favor of more safety, just because we like not having to go to our loved one's funerals.
Great post. I agree that people's lives should be worth more than what can be calculated, at least for the reason that we can't calculate the total value of a person, even in gross income. People are too complicated for that. People are a critical resource in today's world, and there should definitely be money spent on their safety.
Once there is a budget for saving lives, the next question is how can it be spent to maximize the amount of lives saved/dollar. Since terrorism is so low on the causes of death, and it's so expensive and difficult to fight, I can't imagine a program of heavy counter-terrorism getting a very good return: not compared to medical research or sanitary infrastructure or even safer car designs. There should be more research on just how effective various government programs that are designed to make people safer, as far as cost per person saved/helped.
I know it's hard to put things in terms of how many people weren't killed because a certain program prevented it, but that really depends on the individual program: some have easy to measure results and some don't. We should be spending most of the budget on programs that are known to work. Lack of data isn't a reason to put more trust in something; only actual results are. I'm not seeing any real information about how many lives are being saved by counter-terrorism programs either domestic or abroad, by invading Iraq, or by invasive laws like the Patriot Act. I mean, that's the supposed reason for all these things; to make us safer, right? There are real results from new and improved medical treatments, car designs and many other programs. These successful programs are losing the funding that could be saving lives at a higher (and much more predictable) rate to programs that cater to fear.
Yes, it's a design flaw. It's not insurmountable, but does make certain practices unsafe. I don't think that it's unreasonable for programs that are trusted with elevated privileges to be aware of this and protect themselves accordingly. Those programs already have a long list of things they have to be careful about, like not leaving unnecessary ports open, care to avoid buffer overruns, not using shared memory for IPC (unless you're really careful), etc. Not opening windows on the interactive desktop (like it is by default for services) isn't an excessive requirement.
Actually, using a program like jobprc, an you can separate processes with different privileges with jobs and desktops if you want. The extended Software Restriction Policies levels can also do the same thing automatically, on a hash, path or certificate basis.
Wait. Did I just say message passing? Why, that would imply that anyone can pass messages to the background process. In fact, thanks to design flaws I don't want to get into right now, it is possible to any program, running under that user, to create a window handle and pass it in using a certain type of message, and get the privs of that program. Aka, superuser. This cannot be fixed any and have Windows remain backwards compatible.
The only way to guard against this is to make sure that all programs that might run under a different user never make a window. Ever. Even Microsoft has been guilty of doing this once in a while, although they have cleaned up their act and a default install of XP has no such windows.
Windows are always created in the context of a desktop object. Desktop objects have been available since NT 3.51. Every GUI thread has exactly one associated desktop object, and it can only create and interact with windows on that desktop. Since desktop objects have a security descriptor, the calling process must have the proper permission to open and interact with a desktop. Privileged services can open all the windows they want, as long as the desktops they do so in are not accessible from unprivileged users. By default, the service control manager automatically creates a non-interactive window station (the container for desktops) for every service that tries to create windows, inaccessible to unprivileged users. Some ignorant developers are ignoring the stated best practices and are accessing the user's desktop from privileged process. This isn't Microsoft's fault; they can't control what other people do with their computers.
Another way to protect against shatter attacks is to create the untrusted processes in a job object that denies access to window handles outside the job. The untrusted processes can access each other's windows, but can't access the window handles of processes that aren't in the job. No destination = no malicious messages.
When the term [multi-user] was developed, it meant the capability of the OS/System to serve multiple actual people simultaneously.
Um, I can serve as many users as I want if they connect using SSH or X-Windows from any version of Windows NT, and they will each have their own environment, secure from the others. The backend support for using the Win32 GUI (desktop, window station objects) has existed since NT 3.51 (although it required Citrix Winframe for frontend and protocol), and Microsoft Terminal Server (no third-party components required) has been available since NT4 TSE. The Win32 GUI is not required for a multiuser system; in fact, many UNIX and IBM iSeries systems are considered fully multiuser using only text mode connections.
The most important part about being multiuser is the security that protects users from each other when simultaneously executing code on the same machine. NT has supported this since its first release.
Create a shortcut to the "Add/Remove Programs" control panel applet. Make sure you do this as an unprivileged user. Right-click and click on the "RunAs"... oh wait, it's not there. Hmm... Seems like you have to login with an Administrative account to "Add/Remove" programs from your system. That is an important, yet missing feature of the "RunAs" command. (I thought you knew Windows.)
You can run the control panel (and so add/remove) as a different user, although it does take some extra work. This is a known issue and it does suck. Normally, when you launch Explorer (control panel is part of explorer), it just sends a signal to the existing explorer process to open a new window and exits. This behavior dates back to Windows 95 on 4MB of memory when creating a new process would have wasted memory. I think this behavior should have gone away a while ago, but in the meantime launch Internet Explorer as a different user and browse to the control panel from there (type in C: in the address bar, enable folder view and open the control panel on the left)
I believe that "RunAs" only works for the initial application being called, it's not capable of spawning off sub-processes under the elevated privileges the application inherits from the initial "RunAs" activation.
No, in Win32, child processes always inherit the security context (token) of the parent process unless the parent has AssignPrimaryToken privilege (only SYSTEM has it by default) or the child will be created with a restricted version of the parent's token. See CreateProcessAsUser.
On modern UNIX servers/workstations, very few processes that could be whacked and overtaken are running as root. The best practices security models demand that dummy accounts be created for processes that accept external input so that if one of those is compromised, at best, the low or unskilled cracker will end up with a useless sandbox that won't even play nice.
Microsoft's best practices since NT 3.51 stipulate that privileged services should never, ever access the interactive desktop for just this reason. This doesn't stop ignorant third party application developers from ignoring that. You're supposed to create a client program that runs in the context of the logged-on user that communicates with the sever using IPC like a named pipe. There's even a flag you can set to disable all interactive services. It's not set by default because it would break too many crappy third party programs.
No, but IE does have some components that have access beyond the level required for basic User Space interaction. These components are the weakness inherent in IE and why, even if running IE as an unprivil
Windows NT 3.1 was released in 1993. It had ACLs the same as NT 4.0. That page is wrong. See SetKernelObjectSecurity, a function that sets a security descriptor for a kernel object, including ACL, supported since NT 3.1.
RPC on Windows is a set of runtime libraries, compiler support (MIDL), and a server process (rpcss) for coordinating connections and hosting libraries, used to abstract function calls so that they can be used across process and possibly machine boundaries. Since every process runs in a different address space, the application interface for making a remote function call is the same if the call destination is just in another process or on another machine, so Microsoft uses the same system for both local and remote out-of-process function calls, a system called RPC.
Every RPC interface has a set of allowed transports. One of these transports, NCALRPC, is a wrapper over LPC ports, a primitive IPC method that the OS provides that uses shared memory and only works on the local machine. If the only allowed transport is NCALRPC, then the interface can only be connected to from the local machine, even though it's still called RPC and still uses most of the common RPC components. This way, while the RPC system is used in local calls for the convenient marshaling support, support for connections to remote computers is disabled.
Whats wrong if one is? Genetically modifying crops, can result in a new breed of crops, which can be harmful, or bad for human health. You never know, what you end up with.
Killing diversity is no good. Mutations, do help in diversifying, but they do so over a very long period of time. And killing the existing bio-diversity in the hope that it will replenish itself in the future is just outright stupid.
In nature, new species are created all the time. Some are harmful to humans. These mutations are out of human control, so no one knows what they will end up like. Extinction is just as natrual, and it too happens all the time, even without human intervention. Life on this planet is in a constant state of change, which involves creation of new unkowns and destruction of old things.
The natural mutations are what we don't have any control over, they are the ones that are most unpredictible; new breeds of every species are created naturally on a much larger scale than genetic engineering and are more likely to be harmful to humans because no one has control over them.
Creating new variants increases diversity. The alternative is to have all domestic crops of a certain type (e.g. wheat) be of the same species, without the oppourtunity to modify it into vairiants that are better adapted to each target growing environment and target market.
The people you deem as "anti-science leftists" (many of whom are extremely conservative or libertarian) are often very pro-science. They take a stand against what may very well be considered unjustifiable use of scientific knowledge. We're talking about taking a stand against genetically modified crops, animal testing, and so forth.
Does this mean you are against genetically modified crops?
In any case, I think that well engineered genetically modified crops are a good thing. Modified crops can reduce usage of pesticides by making them more resistent to pests and having them create their own natural repellants. Modified crops can be more productive, more nutritious and exist in climates that wouldn't support them naturally. See the Golden Rice Project, a genetically modified form of rice that produces vitamin A to ameliorate deficiencies in developing countries.
Genetic modification of plants and animals by humans has been happening for thousands of years through controlled breeding. The method of breeding is quite inefficient since a parent passes on many genes, only one of which may be the desired one. Genetic modification is simply a more direct and precise way to do the same thing: create and propogate mutations in plants and animals that are useful to humans.
There has been fear that these "frankenfoods" will "infect" established wild variants of the same species. If they do, so what? Mutations happen all the time, and the entire purpose of sexual reproduction (including polliation) is to spread mutations that might be beneficial. This has been happening since the beginning of life on the planet; what's so special about man-made mutations, except that we can have plants and animals do what we want more effectively?
That sounds like the same as NT's event pairs, used to implement Quick LPC. An event pair consists of a high and a low event. The server thread waits on the high event and the client thread waits on the low event. Only one event can be signalled at one time, and two software interrupts are provided to toggle the event pair's state: interrupt 0x2C calls the function KiSetLowWaitHighThread() and interrupt 0x2B calls the function KiSetHighWaitLowThread(). When one of these is called and another thread (in another process) is waiting on the other event, the kernel schedules the other thread immediately, continuing the same timeslice of the calling thread. Transfer of data is done through a shared memory section mapped to both processes. Quick LPC was introduced in NT 3.51 to make the out-of-process GUI server faster. Apparently Microsoft didn't think it was fast enough, so they moved much of the GUI server into kernel mode in NT4.
For more information, scroll down to Quick LPC in Local Procedure Call from Undocumented Windows NT.
First of all, the Windows processes created in this example are Win32 processes, which have a lot more baggage than the posixy processes that FreeBSD, Linux and Windows's Posix subsystem use. I'd like to see added to that list the number of cycles to create a native Windows process and a SFU process (they're going to be a lot shorter), and also a WINE process under Linux.
Some of the things that Win32 processes do that SFU and native processes don't:
The Application Compatibility Database, a user mode service has to be contacted to see if the new program needs to have any compatibility shims added. Half of the compatibility that XP has comes from modifying programs as they start, or giving them special treatment. This stage alone causes so much overhead that Windows Server 2003 has a special group policy that lets you turn it off to make starting processes faster.
The Software Restriction Policies database, a set of registry entries that have allow/deny rules for starting processes based on hash, filename or certificate. To make any actual comparisons, the entire binary has to be hashed and checked for certificates before the program even starts.
Registering with the Win32 subsystem server (csrss). This involves several out-of-process function calls.
Load the current locale, including NLS files.
If enabled, contact the Themes service.
Except for talking to the Themes service, all those steps are done for every new Win32 process, even if it doesn't have a GUI.
I'm currently running 2003 SP1 with 1GB of memory, of which only 10MB is unused. I have 56 processes, 551 threads, 32K handles, a commit charge of 752MB (207MB of which is in standby) and 135MB file cache.
Since there is only 10MB unused, I don't consider this poor memory utilization. When I upgraded my laptop running XP from 256MB to 512MB I noticed quite an improvement, especially in Eclipse which likes to commit 80-100MB. On top of Mozilla (another 60-80MB at least), Explorer (20MB) and services, everything didn't fit into memory and I had noticeable paging between applications until I upgraded.
I understand the basic concept of standby pages, but didn't you just make my argument for me?
In a way, but you were implying that Windows's memory management was stupid because it liked to write pages to disk too agressively in order to provide free memory that wasn't being used. I explained the rationale behind the behavior: it actually speeds things up during heavy loading. Writing standby pages only happens when the machine is idle.
Slashdot Mirror
What, exactly, does the registry have to do with spyware and viruses? If you're saying that the large size permits hiding by obsurity, a file system structure to store the data would be the same. If you're saying that it provides some kind of security hole, please enlighten us as to how. If anything, giving each key its own security descriptor makes it MORE secure than a flatter version implemented in a filesystem.
My SOFTWARE registry hive has 99865 keys and 143228 values in it. User has 18154 keys and 106945 values. Added to SYSTEM, my registry contains a total of 131181 keys, and 291410 values. 131k keys is more than 10x larger than the 11584 files in my \WINNT directory. My main disk has 351k files in 20k directories; moving the registry structure there would mean six and a half times the number of directories and a 83% increse in files, assuming one file per value. I'd characterize my computer as one a typical developer would have.
It's not completely unreasonable, though: NTFS already stores files smaller than about 900 bytes directly in the MFT file record (space that would otherwise be wasted) and NTFS's extended attributes (or even file streams) would provide a nice implementation of key values, but there's still the issue of 6x directories for structure. Although they have the same basic structure, the scale of the registry and filesystem are each quite different. The registry sucks for large entries: the entire SOFTWARE hive is only 26MB, but then again filesystems traditionally suck at tiny files and directories. I suppose Microsoft could've added a special directory flag that optomizes for tiny directories. I suppose the hierarchy in the registry could be flattened/simplified to require less keys, but that would give up security granularity (but I'm sure some could give), and would require a very major overhaul at this point.
The registry hasn't changed much since it was first introduced in its modern form in NT 3.1, so let's look at how things were back then. I've got a VM of NT 3.51 with Office 97 and VB 5 on it. In SOFTWARE, SYSTEM, and USER, there are 12673 keys and 18851 values. Compared to 254 directories and 4k files, moving the registry to the filesystem would've meant 50x the directories and more than 4x the files (or extended attributes, assuming you could get apps to use 'em).
There's a reason that different types of databases exist. Filesystems are general purpose DBs that have direct access and handle large files well. SQLish table oriented databases are used for lots of things that flat files directly on the filesystem suck for. The registry is a specialized database for the high structure low leaf data that comprises config files. Windows is hardly the only software product that uses a special binary config database; see the 'registries' that Mozilla or Eclipse use for examples. Could they have been implemented directly in the file system? Sure... but would it've been the best way?
Er, you mean with File->Export and File->Import in the registry editor? If the machine with the settings isn't booted, use File->Load Hive to mount first so you can use export. I've transferred lots of apps this way.
The registry first existed for registering OLE document types in about Windows 3.0. At the time, it had almost no structure and responsibility. The first release of Windows NT (NT 3.1) made the registry much more important: everything that used to be stored in
Meanwhile, the Windows 9x/ME series still uses
Personally, I don't see what the big problems with the registry are. The registry is a hierarchial database system provided by the OS designed to store configuration information. The only differences between it and a
- The storage is managed by another database on top of the filesystem, instead of the filesystem db directly.
- The seperation of config entries is handled by the db in the registry, whereas each app comes up with its own format in
/etc.
The registry has several documented functions for hot backup and restoration, and has always been journalled (like the fs metadata). A lot of apps abuse the registry, but I think they'd do the same thing with their config files on another OS.Hey, whatever works.
Far be it for me to diminish your story, but I did an F3 search for all my SID files (20327 of em, more files than any other music type) in the entire music directory, and it took about 15 seconds. Now that it's cached it takes about 3 seconds for successive searches on those directories. Most of the CPU time is spent in Explorer doing file name matching. dir \music\*.sid /s is similar. This is on an Athlon XP 2800, 1GB RAM, nVidia chipset, a 100GB NTFS volume and Windows Server 2003 sp1. Are you sure your filesystem isn't corrupted? What FS are you using, btw? You may need to defrag the MFT. When you say it hangs, do you mean the entire OS or just Explorer? Also, Explorer (shell) type searches will also search the index of ZIP files, so if you have a large or damaged ZIP file, Explorer might be choking on it. If you want the shell to quit doing that, unregister the zip folder support library: /u %windir%\system32\zipfldr.dll
regsvr32
I'm not ready to pass judgement on GUI vs command line in general, but in this case on Windows, I'd hit WindowsKey+F to open a search box, put *.mp3 in the find description, ALT+L for location, type \MyFiles, enter, wait for the results, hit CTRL+A to select all, CTRL+X to cut, hit ALT+A to focus the address bar, stick \MyMP3s in there, hit enter, SHIFT+TAB to refocus the files pane, and CTRL+V to paste.
There are several steps, but fewer overall keypresses. (35 vs 49) One major disadvantage is that I'd have to wait for the search to complete before I could copy anything, whereas you can enter the entire command at once.
You can use NtCreateProcess to create processes that aren't Win32 clients, so have no GUI overhead. You can even use it to get a real copy on write fork (this is how SFU does it). The catch is that the new process can't use Win32 services: it can't link to kernel32, user32, gdi32, advapi32 or any other libraries that depend on them, since Win32 is required to use any of them. You have to use the functions that ntdll exports directly for IO, IPC, syncronization, etc. All the primitive functionality the kernel provides to user processes are there, but some things like sockets can be a pain, since you have to interface with TDI directly. Most of the standard Win32 programming doesn't apply for such a process.
Microsoft doesn't document the native API, so you have to find third party docs.
The Windows NT/2000 Native API Reference by Gary Nebbett (ISBN:1578701996) is an excelent reference.
Reactos is an open source clone of Windows NT, and it documents and implements almost all of the same internal functions.
Sysinternals also has a short native example program.
The books Inside Windows 2000 and Undocumented Windows NT are also quite useful.
An easier way might be to just write SFU apps. SFU processes aren't Win32 clients (SFU is its own subsystem) and it provides a UNIXy programming environemnt.
CreateProcess is not a syscall; it's a Win32 function exported by a Win32 client library, kernel32.dll. The internal API handled by the kernel is the native API. All of the syscalls start with Nt* and are exported by ntdll. The related syscall is NtCreateProcess, a function that creates a process object, loads ntdll.dll, maps an image section (for the program binary) and nothing else. It does not have a STARTUPINFO parameter.
CreateProcess from Win32 does call NtCreateProcess, but it also does a lot of other things, like create a startup thread and report the new process to the Win32 subsystem server (csrss), where the GUI parameters are used. The GUI is a core part of Win32, but Win32 is not a core part of the kernel. Almost all Windows software depends on Win32 (and so the GUI), but the kernel is happy to without win32: see the recovery console or first stage setup program. Full kernel, no Win32.
Before NT4, the Win32 subsystem was implemented entirely in user mode (mainly winsrv.dll hosted by csrss.exe for the server, kernel32.dll, user32.dll, gdi32.dll, advapi32.dll for client libraries used in programs that depend on Win32). Since NT4, much of winsrv's code got moved into kernel mode in win32k.sys. The kernel mode part of Win32 is all contained in win32k.sys; it's not part of the kernel, it just runs in kernel mode.
Me too.
Once there is a budget for saving lives, the next question is how can it be spent to maximize the amount of lives saved/dollar. Since terrorism is so low on the causes of death, and it's so expensive and difficult to fight, I can't imagine a program of heavy counter-terrorism getting a very good return: not compared to medical research or sanitary infrastructure or even safer car designs. There should be more research on just how effective various government programs that are designed to make people safer, as far as cost per person saved/helped.
I know it's hard to put things in terms of how many people weren't killed because a certain program prevented it, but that really depends on the individual program: some have easy to measure results and some don't. We should be spending most of the budget on programs that are known to work. Lack of data isn't a reason to put more trust in something; only actual results are. I'm not seeing any real information about how many lives are being saved by counter-terrorism programs either domestic or abroad, by invading Iraq, or by invasive laws like the Patriot Act. I mean, that's the supposed reason for all these things; to make us safer, right? There are real results from new and improved medical treatments, car designs and many other programs. These successful programs are losing the funding that could be saving lives at a higher (and much more predictable) rate to programs that cater to fear.
Yes, it's a design flaw. It's not insurmountable, but does make certain practices unsafe. I don't think that it's unreasonable for programs that are trusted with elevated privileges to be aware of this and protect themselves accordingly. Those programs already have a long list of things they have to be careful about, like not leaving unnecessary ports open, care to avoid buffer overruns, not using shared memory for IPC (unless you're really careful), etc. Not opening windows on the interactive desktop (like it is by default for services) isn't an excessive requirement.
Actually, using a program like jobprc, an you can separate processes with different privileges with jobs and desktops if you want. The extended Software Restriction Policies levels can also do the same thing automatically, on a hash, path or certificate basis.
Another way to protect against shatter attacks is to create the untrusted processes in a job object that denies access to window handles outside the job. The untrusted processes can access each other's windows, but can't access the window handles of processes that aren't in the job. No destination = no malicious messages.
Um, I can serve as many users as I want if they connect using SSH or X-Windows from any version of Windows NT, and they will each have their own environment, secure from the others. The backend support for using the Win32 GUI (desktop, window station objects) has existed since NT 3.51 (although it required Citrix Winframe for frontend and protocol), and Microsoft Terminal Server (no third-party components required) has been available since NT4 TSE. The Win32 GUI is not required for a multiuser system; in fact, many UNIX and IBM iSeries systems are considered fully multiuser using only text mode connections.
The most important part about being multiuser is the security that protects users from each other when simultaneously executing code on the same machine. NT has supported this since its first release.
You can run the control panel (and so add/remove) as a different user, although it does take some extra work. This is a known issue and it does suck. Normally, when you launch Explorer (control panel is part of explorer), it just sends a signal to the existing explorer process to open a new window and exits. This behavior dates back to Windows 95 on 4MB of memory when creating a new process would have wasted memory. I think this behavior should have gone away a while ago, but in the meantime launch Internet Explorer as a different user and browse to the control panel from there (type in C: in the address bar, enable folder view and open the control panel on the left)
No, in Win32, child processes always inherit the security context (token) of the parent process unless the parent has AssignPrimaryToken privilege (only SYSTEM has it by default) or the child will be created with a restricted version of the parent's token. See CreateProcessAsUser.
Microsoft's best practices since NT 3.51 stipulate that privileged services should never, ever access the interactive desktop for just this reason. This doesn't stop ignorant third party application developers from ignoring that. You're supposed to create a client program that runs in the context of the logged-on user that communicates with the sever using IPC like a named pipe. There's even a flag you can set to disable all interactive services. It's not set by default because it would break too many crappy third party programs.
Windows NT 3.1 was released in 1993. It had ACLs the same as NT 4.0. That page is wrong. See SetKernelObjectSecurity, a function that sets a security descriptor for a kernel object, including ACL, supported since NT 3.1.
RPC on Windows is a set of runtime libraries, compiler support (MIDL), and a server process (rpcss) for coordinating connections and hosting libraries, used to abstract function calls so that they can be used across process and possibly machine boundaries. Since every process runs in a different address space, the application interface for making a remote function call is the same if the call destination is just in another process or on another machine, so Microsoft uses the same system for both local and remote out-of-process function calls, a system called RPC.
Every RPC interface has a set of allowed transports. One of these transports, NCALRPC, is a wrapper over LPC ports, a primitive IPC method that the OS provides that uses shared memory and only works on the local machine. If the only allowed transport is NCALRPC, then the interface can only be connected to from the local machine, even though it's still called RPC and still uses most of the common RPC components. This way, while the RPC system is used in local calls for the convenient marshaling support, support for connections to remote computers is disabled.
The natural mutations are what we don't have any control over, they are the ones that are most unpredictible; new breeds of every species are created naturally on a much larger scale than genetic engineering and are more likely to be harmful to humans because no one has control over them.
Creating new variants increases diversity. The alternative is to have all domestic crops of a certain type (e.g. wheat) be of the same species, without the oppourtunity to modify it into vairiants that are better adapted to each target growing environment and target market.
In any case, I think that well engineered genetically modified crops are a good thing. Modified crops can reduce usage of pesticides by making them more resistent to pests and having them create their own natural repellants. Modified crops can be more productive, more nutritious and exist in climates that wouldn't support them naturally. See the Golden Rice Project, a genetically modified form of rice that produces vitamin A to ameliorate deficiencies in developing countries.
Genetic modification of plants and animals by humans has been happening for thousands of years through controlled breeding. The method of breeding is quite inefficient since a parent passes on many genes, only one of which may be the desired one. Genetic modification is simply a more direct and precise way to do the same thing: create and propogate mutations in plants and animals that are useful to humans.
There has been fear that these "frankenfoods" will "infect" established wild variants of the same species. If they do, so what? Mutations happen all the time, and the entire purpose of sexual reproduction (including polliation) is to spread mutations that might be beneficial. This has been happening since the beginning of life on the planet; what's so special about man-made mutations, except that we can have plants and animals do what we want more effectively?
That sounds like the same as NT's event pairs, used to implement Quick LPC. An event pair consists of a high and a low event. The server thread waits on the high event and the client thread waits on the low event. Only one event can be signalled at one time, and two software interrupts are provided to toggle the event pair's state: interrupt 0x2C calls the function KiSetLowWaitHighThread() and interrupt 0x2B calls the function KiSetHighWaitLowThread(). When one of these is called and another thread (in another process) is waiting on the other event, the kernel schedules the other thread immediately, continuing the same timeslice of the calling thread. Transfer of data is done through a shared memory section mapped to both processes. Quick LPC was introduced in NT 3.51 to make the out-of-process GUI server faster. Apparently Microsoft didn't think it was fast enough, so they moved much of the GUI server into kernel mode in NT4.
For more information, scroll down to Quick LPC in Local Procedure Call from Undocumented Windows NT.
Some of the things that Win32 processes do that SFU and native processes don't:
- The Application Compatibility Database, a user mode service has to be contacted to see if the new program needs to have any compatibility shims added. Half of the compatibility that XP has comes from modifying programs as they start, or giving them special treatment. This stage alone causes so much overhead that Windows Server 2003 has a special group policy that lets you turn it off to make starting processes faster.
- The Software Restriction Policies database, a set of registry entries that have allow/deny rules for starting processes based on hash, filename or certificate. To make any actual comparisons, the entire binary has to be hashed and checked for certificates before the program even starts.
- Registering with the Win32 subsystem server (csrss). This involves several out-of-process function calls.
- Load the current locale, including NLS files.
- If enabled, contact the Themes service.
Except for talking to the Themes service, all those steps are done for every new Win32 process, even if it doesn't have a GUI.I'm currently running 2003 SP1 with 1GB of memory, of which only 10MB is unused. I have 56 processes, 551 threads, 32K handles, a commit charge of 752MB (207MB of which is in standby) and 135MB file cache.
Since there is only 10MB unused, I don't consider this poor memory utilization. When I upgraded my laptop running XP from 256MB to 512MB I noticed quite an improvement, especially in Eclipse which likes to commit 80-100MB. On top of Mozilla (another 60-80MB at least), Explorer (20MB) and services, everything didn't fit into memory and I had noticeable paging between applications until I upgraded.