Slashdot Mirror

The idea of IRQLs came from VMS.

http://www.osronline.com/showT...

OK, gather the responses from this forum and show to him.

IRQL is derived on the hardware interrupt level (the interrupt controller
register). Historically this is the PDP/VAX-11 feature, and thus a VMS feature,
though things are going back - in modern x64 CPUs, you have CR8 register as APIC
TPR, so, once again the IRQL register is embedded to the CPU.

But it is too convinient to also implement "preemptivity suspend" as an IRQL
raise. After all, ISRs run with preemptivity suspended.

Though perhaps the name IRQL was invented for NT

https://blogs.msdn.microsoft.c...

The people who built DEC's VMS operating system also helped design the processors that DEC used, and many of them came to Microsoft and designed Windows NT, which was the basis for modern versions of Windows, including Windows XP and Windows 7. These guys wanted a way to disable (very quickly) just some of the interrupts in the system. They considered it useful to hold off interrupts from some sources while servicing interrupts from other sources.

They also realized that, just as you must acquire locks in the same order everywhere in your code to avoid deadlocks, you must also service interrupts with the same relative priority every time. It doesn't work if the clock interrupts are sometimes more important than the IDE controller's interrupts and sometimes they aren't.

Interrupts are frequently called "Interrupt ReQuests" and the priority of a specific IRQ is its Level. These letters, all run together, are IRQL.

So if you lay out all the interrupt sources in the system and create a priority for each one, or sometimes a priority for each group, you can start to do interesting things.

Consider a spinlock. Spinlocks (at least in the traditional sense) are implemented by having a processor spin in a tight loop trying to atomically modify a variable. The cache coherency hardware guarantees that only one processor can do that at a time, so lock acquisition goes only to the processor that succeeds. Other processors keep spinning until they succeed.

The processor that "owns" the lock needs to release the lock as soon as possible, as the other (waiting) processors are burning up processor time waiting to acquire the lock. So you really don't want to interrupt that processor and schedule some other thread for execution, causing all the waiters to spin until the owning thread is rescheduled.

In this situation, some operating systems encourage the owner of the spinlock to disable all interrupts so that the code can't be interrupted. (Note, too, that interrupts really need to be disabled before trying to acquire the lock, or the thread might be interrupted between acquiring the lock and disabling interrupts.)

The designers of VMS and NT decided that they didn't want to disable all interrupts just because some code somewhere acquired a spinlock. Some things shouldn't wait. TLB flushes, are a good example. So if only some interrupts are disabled while a spinlock is held, then you can still briefly interrupt the code that owns the lock for much more important tasks. Perhaps even more importantly, you can interrupt the processors which are spinning, waiting to acquire a spinlock for these important tasks, causing them to do something useful instead of just spinning.

Note that this means that every spinlock has an associated IRQL, and you have to use that IRQL consistently, or the machine will deadlock. In NT, by default, every spinlock has the same IRQL, called DISPATCH_LEVEL. DISPATCH_LEVEL means, essentially, that the interrupts which can cause a thread to stop running are disabled. (More about that later.)

Interestingly you had

Why don't you examine the dmp file and find out exactly why it crashed? You can do it online here: http://www.osronline.com/page.cfm?name=analyze, Or user this tool to examine it yourself: http://www.nirsoft.net/utils/blue_screen_view.html.

But more to the point, I didn't see much about what might be NEW with this file system, only what's OLD and being discarded.

Mind you, some basic feature cleanup never hurt anyone. But if that's the case, why not NTFS2 instead of a marketing buzzword?

The article hints in various areas that they are restricted by maintaining a high level of compatibility. ReFS is merely a transitional FS from NTFS, and as an unfortunate result it carries some of that burden with being compliant to a high compatibility standard. Part of me thinks this may be the "Vista" of file systems (much of what caused Vista to be awful was due to extreme efforts to maintain compatibility), but I will be critically optimistic about it, given the changes it advertises.

On a side note, the original MSDN blog confuses me on a couple things, namely their statement on deduplication. While they say the ReFS itself does not natively incorporate deduplication, but - like NTFS - will permit 3rd-party support of it, why is it that people have found new FSCTL ops for it in the Win8 header files? Maybe they dropped it? Curious.

FYI here is the story:
http://www.osronline.com/showThread.cfm?link=21604

It doesn't have drive letters internally - they're just symbolic links into the Object Manager's namespace

http://www.osronline.com/article.cfm?article=381

So the "C:" seen by Win32 code is really a link to \Device\HardiskVolume1

And a native application doesn't have the restrictions on filenames.

NT based OSs have a VMS like kernel with a Win64, Win32 and originally Win16/Dos, Posix and character mode OS/2 subsystems on top.

A hanging driver or non-responsive hardware is different than a crashing driver. If code crashes while in kernel mode, bad things happen.

That's true. Actually Windows has a strange history with display drivers. Initially NT based OSs (NT 3.1 and NT 3.5) were supposed to run almost all of the display driver in user mode with a small kernel mode component called a video miniport that could access hardware.

Of course that was too slow in practice so everyone mapped the hardware registers into user mode memory. E.g.

http://www.osronline.com/ddkx/graphics/vidintro_5d7r.htm

The display driver has direct access to I/O-mapped and memory-mapped video registers. This access allows a display driver to achieve high performance. For example, the driver might need to access video hardware registers to send line-drawing commands at high throughput.

Similarly, for graphics cards, such as the S3, many of the innermost loops in the graphics engine code require reads and writes of several video controller ports (for example, text output in graphics mode, bit block transfers, and line drawing). Rather than requiring the display driver to send an IOCTL to the miniport driver for each request, the display driver is permitted to access the video hardware directly.

But user mode that can access DMA registers can bring down the system.

NT 4.0 moved everything to kernel mode for performance with the memorably bogus justification that "if you crash the display driver the system is useless anyway". Then again if everyone just mapped both the framebuffer and the hardware registers to user mode memory anyway, the kernel mode part wasn't really doing anything.

XP was the same but added time outs for threads stuck in the driver. Vista moved most of the code back to user mode with a small KMD.

The architecture is clever than the NT 3.x video miniport though - see here

http://download.microsoft.com/download/5/b/9/5b97017b-e28a-4bae-ba48-174cf47d23cd/pri103_wh06.ppt

It all sounds quite sensible the way they describe it - most of the code is in user mode. It packs commands into packets and calls kernel mode code whose sole purpose is to add the packets into a DMA list. Graphics hardware then DMAs the commands and executes them. In the later versions of WDDM the display memory is virtualised. So theoretically you can stop buggy user mode code using the DMA engine to clobber memory it shouldn't be able to access.

Mind you I'm pretty sure that architectural changes like this are the reason for Vista's reputation for being a resource hog. OTOH Windows 7 seems OK, and it still has much the same architecture.

Windows Filtering Platform:

http://www.microsoft.com/whdc/device/network/wfp.mspx

This is NOT the same as it was done in Windows 2000/XP/Server 2003, as I stated here earlier... &, here is the "issue" I have with it, in a single sentence there:

----

"Because all the applications and services use the same filtering engine, it is easier to determine whether other applications or services exist that perform the same function."

&

"You have a fine level of access control to the TCP/IP packet processing path. This control differs from the filter and firewall hook methods that are supported in Windows XP and Windows Server 2003"

----

That? That seems to say that the Windows VISTA/Server 2008/Windows 7 protective measures of IPSec, &/or Windows Firewall are DRIVEN by the same engine, & thus, that represents a SINGLE POINT OF POSSIBLE FAILURE (and only a single point to attack, for an interloper)...

The "older method" of using IPFLTDRV.SYS (PORT FILTERING), IPSEC.SYS (IP Security Policies), IPNAT.SYS (Windows Software Firewall) had 3 diff. spots/levels/layers it worked in (yes, they did not "sync" automatically, which is GOOD in respect to making it harder for intruders to attack just a SINGLE POINT as WPF seems to say it allows)...

Get it now, ComputersHack?

APK

P.S.=> ADDED INFORMATION/PROOF:

http://www.osronline.com/ShowThread.cfm?link=120152

Re: IpFilterDriver firewall hook in Windows Vista

"The filter hook extension interface is not supported on Vista. Vista
provides the Windows Filtering Platform (WFP) which replaces this hook as
well as the firewall hook - it is well documented in the WDK. As for why the
driver is still loaded, I am not sure."

(What he is NOT sure of, is WHY IPFLTDRV.SYS is still loading in VISTA... same thoughts here, also!)

apk

http://www.osronline.com/showthread.cfm?link=97522

In short, there's NO WAY to disable driver checks now without resorting to test mode.

General public SHOULD be able to install unsigned drivers. It's not your right to tell them what NOT to do. Anyway, inability to install drivers is certainly a limitation compared to WXP.

If you disagree, then please explain how freedom is slavery and ignorance is strength.

In an NT based OS there is a rule that you can't touch paged memory at a raised IRQL. This is because is you're running at a raised IRQL you must have acquired a spinlock and there is a risk of a deadlock if the kernel tries to page with spinlocks already held.

DRIVER_IRQL_NOT_LESS_OR_EQUAL means that a driver has tried to touch code it marked as paged at a raised IRQL. The less (than) or equal level is PASSIVE_LEVEL.

It means that the guy that wrote the driver didn't know what he was doing and didn't run driver verifier which has some code to make sure that paged code is never present at a raised IRQL. Or maybe you POS homebuilt machine can't fetch instructions from memory properly.

http://www.osronline.com/showthread.cfm?link=130328

Install WinDBG and open the .dmp file. Type analyze -v and look at the stack trace. If there is always the same non Microsoft driver in the trace update it or disable the device and see if the problem goes away. If you get this and other errors randomly with no obvious guilty driver I'd try to swap the Ram (or stop overclocking).

And actually if you're not lazy, BSODs and WinDBG are good learning tools.

But that link is completely disengenuous. Lots of other OSs have stable API (source level) between drivers and the kernel. Unix type OSs have a relatively simple driver API, so it's not very hard to keep it stable. And in fact every processor has an ABI document describing how the parameters in his "Binary Kernel Interface" section should be set, so it's not hard too get a stable binary interface too.

Even in the Windows world which has a much more complex driver API, it's not too hard to write a driver to support a couple of OS revisions (E.g. Win2k, XP and Vista) and distribute it as a binary, I know because I've done it. Other driver classes are even better - SCSI miniports can run from Win98 and Me,to NT,2k,XP and Vista. They can also run in NTLDR's weird pre OS environment and on Risc NT workstations or 64 bit Vista boxes. You need to build a binary per architecture, but that's probably for the best.

Sure it's extra work for the kernel maintainer to keep interfaces stable, but Microsoft do it because they want OEMs to write Windows drivers for their hardware.

Greg doesn't want to do this because he wants to make things so hard for NVidia and all the other binary blob companies that they decide to GPL and release the source code to their drivers. And he wants to be able to refactor things without having to discuss it with anyone else. That document doesn't mention all that though, he just tries to raise a bunch of technically hard sounding problems, announce they are unsolvable and leave it at that.

NTVDM really does depend on the CPU being able to run in V86 and DPMI compatible modes. That's why the non x86 versions of NT couldn't run DOS or Win16 apps (but would have been able to if NTVDM was a full emulator, although the Alpha port did have a 486 emulator using unique Alpha features). Ntvdm.exe is a container for a Win16 or DOS environment instance, but it depends on the kernel NtVdmControl function to switch to V86 or 16-bit protected modes and setup traps as needed. Since NT doesn't allow user apps to access hardware as a matter of principle (unlike Win9x), NTVDM does indeed emulate some hardware components by trapping interrupt, IO and DMA requests.

I'm sure that Microsoft could write a full emulator if they really wanted to, ala DosBox, but I think they'd rather the whole thing just went away. When the VDM system was created for NT 3.1, full emulation would have been far too slow. The architecture has not changed since.

See also:
DOS from the ReactOS project
Writing a VDD
VDMSound
NTVDM Compatibility Drivers

I don't know why the Windows equivalent of fork() is slower than the Unix fork(). Perhaps it is a historical thing. Unix programs often use fork() - shell scripts use it all the time (this is one reason why a Python or Perl script is often faster). I'm just guessing now, but perhaps Unix fork() is efficient because it is frequently used and has therefore been optimised in various ways (e.g. memory is only copied if there is a write on Linux).

So with FUSE ported, Windows users can also enjoy in-filesystem versioning, seamless ssh integration, RAR files as folders and so on.

Why is this such a great goal, when FS developers have been trying to meet the basic features of NTFS already...

NTFS already does journalling, has file versioning (far beyond what any *nix FS does), encyrption, compression, and with Win32, zip and rar integration.

The trick in writing a FS for Windows isn't so much a NT issue, but how Win32 see the FS and what it expects to be there. This can best be demonstrated with the Unix subsystem on Windows, or how NFS is handled.

BTW, this is kind of a baited post to see how well people really do understand NTFS and also what they are trying to accomplish.

For developers interested there are some good resources and help on writing FS for NT, like at: http://www.osronline.com/cf.cfm?PageURL=showlists. CFM?list=NTFSD

Take Care...

Check here:
http://www.osronline.com/article.cfm?article=476

Check out http://www.osronline.com/ . They have some similar utilities and are the place to go for windows device driver questions and debug. They are the folks that finally fixed much of the DDK documenation . I still have the mugs they gave away for finding doc errors.
By the way, I highly recommend their classes. I have taken a bunch of them and I am pretty sure that these folks know windows internals better than any other organization...maybe even MS.

"The problem with windows security is primarily one of legacy support."

Noncense, backward compatibility should not break security. Windows was sold as suitable for secure use in a networked environment. It was even given C2 security certification. The problem is the WinNT memory management unit running under the x86 processor. Something that was first tackled under Linux with Exec Shield. The Windows version called NX can be bypassed as otherwise JIT bytecode won't work.

"inter-processes communication was flawed lacking any authentication method, kernel / userland seperation was virtually nonexistant,"

Wait a minute WinNT was touted as being more secure because of it's use of operating modes. Ring 0 had full access while user apps were restricted to Ring 3, the highest restriction. At least that was the theory.

"these issues persisted right up till XP when microsoft started to take security seriously with SP2."

Er, They still persist. See here, much of this code is included in Windows Server 2003 and will be included in Longhorn

"Microsoft just like the rest of us is new to the whole OS design thing."

When Microsoft hired on the Digital VAX/VMS team they had an oppurtunity to design a secure OS. Most of the defects in the OS can be traced to managment decisions to favor features over security. Embedding Internet Explorer in the OS was one such decision.

"What needs to be done is .. implement a version of windows that incorporates everything we've learned over the last 20 years or so"

If by "We" you mean Microsoft, "We" haven't learned anything since 1988, 18 years ago. Why wait, why not upgrade to SuSE, all the eye candy of Vista without the security vulnerabilities.

I see a lot of this kind of revisionist history on the Internet and in the media. Is there a whole department that does nothing all day but pollute the athmosphere with self serving distortions such as this. How anyone say this with a straight face is beyond me.

'the security kernel of the Windows NT server software was written before the Internet,
and the Windows Server 2003 software was written
before buffer overflows became a frequent target of recent attacks'

David Aucsmith, Security Architect, Microsoft.

Here's where you saw that:
http://www.osronline.com/article.cfm?article=461

If you're feeling paranoid enough, how about some other questions that same developer has asked around other forums. Try piecing something together.

Messing with CDROM drivers is scummy enough, but could they be messing with network drivers too?

A pass-through NDIS driver would make a grat tool for spying on, oh, say, p2p traffic?

I was actually thinking about this a few days ago. There's lots of work out there getting linux to run windows drivers, but I haven't seen much work on writing windows drivers for posix (*nix, whatever) stuff.

A while ago I downloaded the Windows DDK from Microsoft for something, but I didn't end up using it, uninstalled it and now I can't find the download. Unfortunately, it doesn't seem avail. for free from Microsoft's site anymore either (Microsoft WHDC DDK page). I have work to do, but this page seems like it might be of some help: OSROnline.com... maybe.

Anyways, the idea still stands, why aren't there win32 branches of open source file system drivers? Of course, I know squat about writing drivers, especially filesystem drivers, so there may be a damn good reason why not. But figured I'd throw it out anywho.