I looked at some demo footage of the game and it seemed like time travel really is just like literally adding another dimension to an RTS game. Where in a normal RTS you can be attacked in the West, East, North, South (and potentially on different levels, if the game has land and air units, in Achron you can also be being attacked at a physical location that's also "in the past" and "in the future". You can go to the past and future like you'd go to different places on a map.
To make it sane, the player exists in "meta time", a kind of overall time that ties together all the different positions in game time. The difference from a spacial dimension is:
a) the further away from the current moment you want to operate, the more time energy you use up. I think you can observe any time period for free, it's just if you want to send information or objects through time that it gets expensive. b) effects take a while to propagate - stuff causally resulting from a battle in the past takes a certain amount of meta time (player time) to propagate to the present game time. Sounds weird but think of it like this: if your opponent goes back in time and blows up *all your stuff* you will not see anything change in the present have until the "time wave" propagates the results of these events forward to the present. At that point all your units are going to disappear. But in the meantime you have a (limited) window in which to go back with some units and "fix" the past.
This *sounds* complicated but it really is just like an extra dimension of movement with some odd properties added. It makes a lot more sense if you watch the videos, once it clicks, it clicks.
And yes, you can do grandfather paradoxes and travel to the future. Have fun!
Right! That's interesting to know but also slightly disturbing:-) I'm surprised that MS don't have a more structured way of doing this stuff. But given that it does rather sound like they're limiting the use of potentially flaky configurations to people who know what they're doing, which doesn't actually sound too crazy!
It'd still be nice to have a secret legitimate switch on other versions but I'd guess they do value the revenue.
I should really read the article and see if it mentions any of this, I've just been enjoying discussing the beauties of x86 design too much:-)
No matter how much RAM you have, ONE 32-bit application will NOT use more than 4GB (2+2 or 3+1 if/3GB is used) even on 64-bit OS.
Not disagreeing with anything you said but thought I'd just note for interest's sake that something approximating 4+0 is also a possibility for user address spaces, even with 32-bit tasks.
On a 32-bit system that implies having a separate page table tree for the kernel (i.e. not mapped into the user process pagetables at all) and switching back and forth on each entry to the kernel. The userspace process then gets 4G to play with (well, probably a few pages less than that, as there'll need to be a little page table switching stub somewhere in there, with kernel-level protections on its memory). I know there were patches for Linux to do this but switching page tables all the time is going to really hurt performance. This is really a 4+4 split since the kernel has an entirely separate address space.
On 64-bit platforms, it's possible to have the kernel mapped into the address space all the time (as you usually would) but put it about the 4G boundary so that 32-bit tasks running in compatibility mode just cannot see it. This is really a 4G+ split but again from the the user process PoV the kernel isn't taking up any space. I heard an AMD chap say at a presentation that you could use this trick to get legacy 32-bit apps a bit more usable memory. Somebody hint in another thread on here that Apple do something like this. I wouldn't know, I don't get to play with 64-bit anything;-)
Anyhow, not very relevant really but thought I'd chip in.
What if the buggy drivers assume that all physical memory addressable using 32-bits? Many PCI cards can't DMA above 4G. If the drivers are not well tested on >4GB systems then they could barf on this sort of thing. It's not a PAE problem, it's just a large memory problem.
Windows ships both PAE and non-PAE kernels, just like most Linux distros. The bootloader selects which binary to run at boot time.
Ah, cool, thanks! FWIW I did once ask a kernel hacker if the paravirt_ops stuff in Linux (which allows you to hook pagetable updates, etc) could be used to make the same Linux kernel binary work with PAE / non-PAE and he reckoned it'd still require a fair bit of work even with paravirt_ops.
Shipping two 32-bit kernels hardly seems that much of a sacrifice, given 64-bit is the future and will eventually fully take over.
If you use PAE mode, the motherboard can potentially remap that "hidden" RAM to a higher physical address, so that you can use it.
The issue here is that the driver for that re-mapped RAM must support 64-bit pointers (well, technically it must support 36-bit pointers, but practically speaking 64-bit addresses are used and the high 28 bits are ignored). Many legacy Windows drivers (and some legacy Linux drivers) don't support this, and will crash when a pointer overwrites the memory following its 32-bit expected width.Driver crashes are a Bad Thing, so to avoid the danger, client versions of Windows use 32-bit pointers even in PAE mode. This means the (not-even-quite) 4GB limit is still in force, but means you can use legacy drivers. Server editions of Windows permit you to use the full address width (it's a different HAL, I believe)on the assumption that any driver you use on a server will be well-coded and adhere to the guidelines that allow PAE to work.
Interesting.
Are there any specific guarantees about server hardware? e.g. a compatibility list or a server-specific driver signing program? Or do you trust that your system vendor puts well-supported hardware in the box?
A "nice" thing about the Linux approach is that you don't end up wasting pointer space since your driver will be compiled for the correct memory mode when it's built. I can't imagine the wastage is very significant for most drivers, though. And the memory mapping API and DMA API should ensure that high memory are Done Right whatever the memory model but only if they're understood and used correctly by the driver authors. I can imagine that there are drivers doing more or less naughty things that will violate these APIs in interesting ways, though, as you say.
Development-wise, an interesting trick the Linux kernel has up its sleeve is the "sparse" tool Linus wrote, which amongst other things can handle annotations on pointer types to detect cases where the kernel deferences a pointer to userspace, which you usually "get away with" but no always. I understand it's found a fair few real bugs in the kernel so far.
PAE enlarges the page table entries to 64-bit (IIRC). On a 32-bit PAE system you can stuff 36 bits of physical address in there. To compensate for that, the page table has one more level, making it 3 levels high instead of the 2 that non-PAE 32-bit x86 uses. The root node of the page table is 4096 bytes, the size of an x86 page, however it only contains 4 entries. This is because the userspace code running on it still only gets a 4GB virtual address space.
So basically it's a slightly weird-looking extension of the original x86 page table format that can handle larger physical addresses. You end up being able to have >4GB physical memory but you can't have more than 4GB mapped in any process (or the kernel) at any given time. On Linux this usually means 3G address space per process, with the kernel having 1GB. The extra bits in the page table entries also allow support for the NX (no execute) extension.
It sounds like you think PAE could, in principle, enable 32-bit apps to support more than 4GB address spaces? Maybe I'm misunderstanding. For avoidance of confusion, that is not what PAE does; if you're using 32-bit + PAE you're *always* stuck with 4GB address space per-process no matter what (and 1-2GB will usually be taken up by your kernel). All PAE gives you is the ability to use more physical memory across the whole system. If you need one app to use lots of memory, as you rightly pointed out, you need 64-bit.
I saw another post on here suggesting that server versions of Windows have an API for giving an app access to >4GB memory on 32-bit but that is going to be some kind of hack (e.g. switching between multiple address spaces for one task or using effectively a big ramdisk like you mentioned), not true >4GB virtual memory.
I don't know exactly about the Apple technique but I do remember seeing a presentation by an AMD chap who was describing how you could use 64-bit to increase the address space available to 32-bit apps. In his example, though, that was "merely" enabling apps to use a *full* 4GB for application data whereas on a 32-bit system 1-2GB of the address space is occupied by the kernel. This sounds like the same kind of thing you're referring to. I'd guess it would be tricky to do this without upsetting some poorly or strangely coded apps somewhere along the line. It still leaves you stuck with 32-bit pointers, so really big apps are still ruled out:-(
There were patches developed for Linux to allow a usable 4GB address space (well, probably *slightly* less!) for applications on 32-bit platforms. That meant using a separate pagetable for the kernel and the userspace task and switching back and forth for each syscall. That's going to be expense so you wouldn't have wanted to run those patches unless you *really really* couldn't manage without.
For Linux, 64-bit would be my preference. But my only 64-bit machine is limited to 32-bit non-PAE for "technical reasons" at the moment (meaning the kernel I need to run needs that setup).
I was just wondering whether Windows could support PAE and non-PAE with the same kernel binary - like Solaris and unlike Linux.
The/3GB switch intrigues me - surely there must have been some sensible reason why processes were limited to 2GB (presumably with the kernel taking 2GB). Do you lose anything by applying the switch?
Intel has had 32-bit CPUs with PAE support since before 64-bit CPUs existed. AFAIK they specced PAE and AMD just implemented it. I think it might actually date back to the Pentium Pro, though I've not checked that's true for sure.
However, one source of trouble is that even years after they introduced it not *all* Intel CPUs actually support it. I've generally had the impression that most do, however I suspect they would have disabled it in their budget (and possibly in their mobile) lines. I had a Celeron M laptop that didn't support PAE but as it's both mobile *and* budget I never figured out who to blame;-) I guess that's not entirely unreasonable as those chips probably wouldn't live in a motherboard that could address any more memory. These days it'd be a bit more annoying to lack PAE, even on small memory systems, since the NX bit depends on PAE being enabled.
I actually thought that all 64-bit x86 CPUs were backwards-compatible with PAE, however what you say seems to contradict this. I wonder if in the case of the hackintosh it's possible that it's a difference in chipset support or MacOS behaviour, rather than a CPU problem?
As you say, though, at the end of the day the approach that allows you to make the most of your memory is to have a decent mobo and a 64-bit CPU (of whatever make). That way you can use all physical memory and processes are not limited to a 4GB address space.
If you want the icing on the cake, either get PCI devices that can DMA to all your available physical memory (many won't be able to) or get a system with an IOMMU (and an OS that supports it) to compensate for those devices that can only access limited areas of physical RAM.
I'm not sure what feature you have in mind (although I can think of some possibilities) but PAE doesn't work like a paging file. It also doesn't require you to move data around in memory so that the CPU can access it - PAE is a hardware feature and gives you the ability to address up to 64GB (if your chipset allows) of physical memory. You only have 32-bit virtual memory addresses, i.e. pointers. But there are 36-bit physical addresses at the hardware level. Because you still have a 4GB virtual address space, so you can't address the full 64GB *at once*. But the kernel can update the pagetables to map different portions of the physical RAM any time it wants, so you don't have to move the actual data about.
The 4GB virtual address space limits any given process (and the kernel) to a maximum of 4GB mapped memory at any given time but you can still use all the physical memory by running different processes in different parts of it, so it can still be useful as long as you didn't want to just run one *huge* process. If you're running scientific or database workloads you might *really* need a larger program address space, in which case you really do want 64-bit.
The main example of where you actually do need to copy stuff from low to high memory (and vice versa) is for supporting DMA to devices that can't address high memory. The CPU can always directly address any memory by creating a pagetable mapping appropriately. To compensate for a DMA device that can't access a given high memory page, the OS creates a temporary mapping, copies data somewhere the device can DMA it, lets the device do its thing, then copies the data back . That *is* ugly but 64-bit does not solve it either if your devices have this kind of limitation. An IOMMU can translate from the device's DMA addresses to real physical addresses, making it possible for even limited devices to DMA to any part of memory. That's the "proper" fix, short of having all devices support DMA to all of memory. IOMMUs are starting to appear more and more on x86 hardware, so hopefully this issue will start to resolve itself.
Anyhow, with respect to the current article, Windows *can* do PAE to support systems with >4GB physical memory, you're just not able to run single huge processes because there's still a limited virtual address space. But despite its PAE support, it deliberately limits you to 4G physical memory, unless you buy a more expensive license. This isn't that unusual a practice but it's still always a bit upsetting to realise that the OS is being intentionally broken for commercial reasons.
Mmmm - there was a guy at AMD (I think) working on (ab)using the GART to remap DMA so that virtual machines could drive DMA-capable hardware without knowing about the real memory layout on the host. He was doing this for Xen (which I also do work with, sometimes) but I don't know what came of it.
General purpose IOMMUs are good for all kinds of tasty stuff, so it would be nice to have them more widely available - it seems to be coming.
Yeah, if you have a device that can't DMA above 4GB (as you say, this is common) in the case where you're DMA-ing to memory about 4GB the kernel often has to allocate "bounce buffers" in lower memory, DMA to them, then copy using the CPU.
That's the case for 64-bit systems, too. But with a 64-bit address space available, the kernel can probably keep mappings to all memory so that the copies are faster (I assume it does but I don't work with 64-bit and have never checked). On 32-bit, the kernel would have to create a temporary mapping each time a copy was needed, which would slow things down even more.
There seem to be a surprising number of systems out there that have >4GB of memory but are not 64-bit capable though - lots of people bought lots of cheap RAM for their 32-bit servers.
I hadn't realised that the video RAM was in those regions. I guess it makes sense, though, it has to be somewhere:-) That's going to be a bit hit in physical address space right away.
However, AIUI the other PCI MMIO regions can also waste more space than you'd expect since (depending on the machine) the BIOS sometimes sets them up in a sucky way and wastes way more space than it needs to. Which sucks.
But the article seems to be about a licensing-based restriction on allowed physical memory, depending on your version of Windows. So even if Windows and your hardware supports PAE, it may not let you use >4GB RAM. (nb. PAE is potentially useful even if you don't have more than 4GB RAM, since it gets you the NX bit).
So is that the reason that Linux has the same limit?
Unless I'm misunderstanding you and the summary (not RTFAed yet - I know, I'm bad) Linux doesn't necessarily have the same limitation. The summary seems to imply that only 4GB of *physical* memory are being supported on 32-bit systems by some versions of Windows. All OSes running on 32-bit x86 are restricted to a 4GB virtual address space, therefore neither Linux nor Windows can offer more than 4GB of directly addressable memory *per process*. But many 32-bit x86 CPUs can address more than 4GB of physical memory using PAE mode.
The base requirement for using PAE is that the OS supports it - Windows can use it, Linux can too. PAE gets you the ability to address physical memory addresses over 4GB but only if the motherboard has enough address lines to actually relay these requests to the memory. Even if you can't / don't have more than 4GB of RAM installed, it may still be worth it because PAE also lets you use NX (no execute) on 32-bit. PAE on 32-bit doesn't make the process address space any larger, so any given process is still limited to 32-bit pointers.
32-bit Linux has to be compiled with or without PAE support, so your distro should make sure it installs a the right kernel version to correctly support your hardware. Some OSes (e.g. Solaris) can switch between PAE and non-PAE at boot time, so they only need one kernel image to support both modes. I don't know how Windows handles this.
A further benefit of PAE, that I've left till last because it's really somewhat gross: the memory mapped IO regions used by PCI devices also take up part of your physical address space. This can result in a "memory hole" where your *real* RAM is overshadowed by an area of memory-mapped IO regions - the real RAM is there but you can't access it. As a result, if you put 4GB of RAM in your system and use a non-PAE kernel (only 4GB of physical address space) then you are *guaranteed* not to be able to use all that RAM because you will *definitely* have hidden some of it "behind" the memory mapped IO regions. If you use PAE mode, the motherboard can potentially remap that "hidden" RAM to a higher physical address, so that you can use it. Not all motherboards support this remapping, even if they *do* support 4GB worth of DIMMs - which is rubbish because they're claiming to support memory that the hardware will never let you actually address. It pays to be wary of this kind of craziness - my mobo has this limitation, which I discovered after buying 4GB of DIMMs (so I stuck one in another PC, which kept me happy in the end).
Uh, sorry to the parent poster, I've rambled a bit off the point I was originally making and probably just told you a load of stuff you already knew!
When I went away to university in 2000, I intended to use my digital wristwatch as an alarm clock. My Dad insisted that I should have something more substantial to wake me up - as a result I inherited his old bedside clock from... the 80s, perhaps? Ancient, huge, unsubtle - one of its most noticeable features was a glowing digital display that was clearly not LED based. I had rather assumed it was some sort of fluorescent display but I've never seen another one like it. Looking at this clock the display is quite obviously the same stuff. So my ancient alarm clock has now become cool?! Awesome! Only trouble is that it broke a couple of years ago, so I don't get to ride the retro wave:-(
A shame, since the continuous, vicious sawtooth soundwave it produced instead of a "beep beep" could be heard from other floors in the building and would continue without a break for upwards of 30 minutes if you didn't wake up and hit the thing. For years that had me bolt fully awake in an instant with my fight-or-flight reflexes fully primed, wondering what the awful noise was and ready to sprint to lectures. They don't make 'em like they used to.
I read "Scope Creep" as meaning creepy people, looking at you with telescopes. Having read your post I see what you mean. But your meaning and my original understanding are not mutually exclusive in this instance:-)
Actually I think TFA also quoted a mention from Carmack that they're only using GL 2.x stuff - which makes the claim seem even more odd.
I took it to mean that they're making use of driver features that are badly debugged / optimized. Still, I'd have thought a port with the option to turn down detail if your drivers can't keep up would be acceptable to many people who want to run commercial games on Linux...
There's also an irony in here: the massive corporations of Hollywood (and Big Media) in general seems very happy to - with a straight face - cast "OMG Corporations!" as the soulless villains. Of course, it's rarely a *media* company that's being evil, it's the industrial-military complex. Whilst I don't entirely disagree with this, I do wonder if they really *get* that they are also large, faceless corporations who seek to control assets to enhance their own profit? I understand their point but it does seem a bit hypocritical.
"and that the Linux port would have to use the binary blob graphics driver in order to work"
From TFA, it seems that Carmack believes it would be hard to get the necessary performance without using the NVidia drivers. It's somewhat surprising to me if it wouldn't be possible to get it running acceptably on anything else, even if the game does use a lot of advanced features - but if Carmack says so!
Yea it's really hard to imagine that an up-to-date, specialised driver for specialised hardware straight from the manufacturer would be faster than a 3rd party driver.
That's not what I'm saying - I'm happy to assume that the manufacturer's binary blob drivers are *faster*. What I'm surprised by is the suggestion that no other drivers are fast *enough*. I'm quite surprised by the assertion that only the NVidia binary driver is fast enough to be worth running the game on at all. Can't I lose a little performance and use a different driver? What about ATI's binary drivers, they're made by the hardware manufacturer. What if I turn down the detail settings in the game - sure that defeats the point of the cutting edge graphics engine a bit but maybe if the game is fun *to play* it'd be worth it anyhow!
This argument sounds like a cop-out to me. They don't want to port to Linux and try to blame some sort of belief that blobs are evil. I wouldn't even want a 3rd party driver for something as performance-critical as the GPU. This whole "we don't port because then you would have to use something that's readily available for free" sounds bogus to me.
Now I definitely agree with that - it doesn't sound like a particularly convincing technical argument, more like a bit of an excuse. It's fine if they don't want to do the port but it would be nice to have more insight as to the actual reason.
The licensing on the various OSS projects allows you to use them to create closed-source products. It's one of the Freedoms you're being granted - if the developers had considered fighting closed source code to be more important than that freedom (and other related ones) then they'd have licensed the tools differently. They could have chosen a license that says "If you use this then you must donate other stuff to the open source community for free" but the developers of the code didn't, since that was not the conditions *they* chose for *their* code.
We're typically all pleased to hear Linux is being used as a platform for rendering the latest awesome movie's CGI sequences but it's probably running custom closed code. We're all pleased when a major company switches to Firefox or Open Office but they're probably also using copyright law and patents to control various of their products. I don't see how this is different, OSS code being used to produce stuff is generally a good thing no matter what license the product is released under.
"and that the Linux port would have to use the binary blob graphics driver in order to work"
From TFA, it seems that Carmack believes it would be hard to get the necessary performance without using the NVidia drivers. It's somewhat surprising to me if it wouldn't be possible to get it running acceptably on anything else, even if the game does use a lot of advanced features - but if Carmack says so!
However I'm not so keen on his assertion that if you're using the binary drivers you might as well run the code under Windows. I guess this probably *does* make sense for most people, since there are relatively few people who don't have a Windows license available somewhere. However, it would be *nicer* not to have to reboot into Windows for a specific app even if that were unnecessary.
Unfortunately I saw a fair few quite negative reactions in the linked thread and I expect we'll see others here. Carmack has not ruled out a port for sure. But even if he does, that's not exactly evil or a betrayal of open source or anything else negative. Many gamers here will have benefited in some way from the GPLed code he's released to the OSS community in the past at some point, pretty much all gamers will have benefited from his position as a developer pushing the games industry forwards. He's not done anything *bad* here, he's just not necessarily doing something we'd hoped for.
Hopefully the Rage code will - one day - be GPLed and get ported to Linux. I think that's a fair way down the road at this point, though.
Slashdot Mirror
I looked at some demo footage of the game and it seemed like time travel really is just like literally adding another dimension to an RTS game. Where in a normal RTS you can be attacked in the West, East, North, South (and potentially on different levels, if the game has land and air units, in Achron you can also be being attacked at a physical location that's also "in the past" and "in the future". You can go to the past and future like you'd go to different places on a map.
To make it sane, the player exists in "meta time", a kind of overall time that ties together all the different positions in game time. The difference from a spacial dimension is:
a) the further away from the current moment you want to operate, the more time energy you use up. I think you can observe any time period for free, it's just if you want to send information or objects through time that it gets expensive.
b) effects take a while to propagate - stuff causally resulting from a battle in the past takes a certain amount of meta time (player time) to propagate to the present game time. Sounds weird but think of it like this: if your opponent goes back in time and blows up *all your stuff* you will not see anything change in the present have until the "time wave" propagates the results of these events forward to the present. At that point all your units are going to disappear. But in the meantime you have a (limited) window in which to go back with some units and "fix" the past.
This *sounds* complicated but it really is just like an extra dimension of movement with some odd properties added. It makes a lot more sense if you watch the videos, once it clicks, it clicks.
And yes, you can do grandfather paradoxes and travel to the future. Have fun!
Right! That's interesting to know but also slightly disturbing :-) I'm surprised that MS don't have a more structured way of doing this stuff. But given that it does rather sound like they're limiting the use of potentially flaky configurations to people who know what they're doing, which doesn't actually sound too crazy!
It'd still be nice to have a secret legitimate switch on other versions but I'd guess they do value the revenue.
I should really read the article and see if it mentions any of this, I've just been enjoying discussing the beauties of x86 design too much :-)
No matter how much RAM you have, ONE 32-bit application will NOT use more than 4GB (2+2 or 3+1 if /3GB is used) even on 64-bit OS.
Not disagreeing with anything you said but thought I'd just note for interest's sake that something approximating 4+0 is also a possibility for user address spaces, even with 32-bit tasks.
On a 32-bit system that implies having a separate page table tree for the kernel (i.e. not mapped into the user process pagetables at all) and switching back and forth on each entry to the kernel. The userspace process then gets 4G to play with (well, probably a few pages less than that, as there'll need to be a little page table switching stub somewhere in there, with kernel-level protections on its memory). I know there were patches for Linux to do this but switching page tables all the time is going to really hurt performance. This is really a 4+4 split since the kernel has an entirely separate address space.
On 64-bit platforms, it's possible to have the kernel mapped into the address space all the time (as you usually would) but put it about the 4G boundary so that 32-bit tasks running in compatibility mode just cannot see it. This is really a 4G+ split but again from the the user process PoV the kernel isn't taking up any space. I heard an AMD chap say at a presentation that you could use this trick to get legacy 32-bit apps a bit more usable memory. Somebody hint in another thread on here that Apple do something like this. I wouldn't know, I don't get to play with 64-bit anything ;-)
Anyhow, not very relevant really but thought I'd chip in.
What if the buggy drivers assume that all physical memory addressable using 32-bits? Many PCI cards can't DMA above 4G. If the drivers are not well tested on >4GB systems then they could barf on this sort of thing. It's not a PAE problem, it's just a large memory problem.
Good post. Two quick points, though:
Windows ships both PAE and non-PAE kernels, just like most Linux distros. The bootloader selects which binary to run at boot time.
Ah, cool, thanks! FWIW I did once ask a kernel hacker if the paravirt_ops stuff in Linux (which allows you to hook pagetable updates, etc) could be used to make the same Linux kernel binary work with PAE / non-PAE and he reckoned it'd still require a fair bit of work even with paravirt_ops.
Shipping two 32-bit kernels hardly seems that much of a sacrifice, given 64-bit is the future and will eventually fully take over.
The issue here is that the driver for that re-mapped RAM must support 64-bit pointers (well, technically it must support 36-bit pointers, but practically speaking 64-bit addresses are used and the high 28 bits are ignored). Many legacy Windows drivers (and some legacy Linux drivers) don't support this, and will crash when a pointer overwrites the memory following its 32-bit expected width.Driver crashes are a Bad Thing, so to avoid the danger, client versions of Windows use 32-bit pointers even in PAE mode. This means the (not-even-quite) 4GB limit is still in force, but means you can use legacy drivers. Server editions of Windows permit you to use the full address width (it's a different HAL, I believe)on the assumption that any driver you use on a server will be well-coded and adhere to the guidelines that allow PAE to work.
Interesting.
Are there any specific guarantees about server hardware? e.g. a compatibility list or a server-specific driver signing program? Or do you trust that your system vendor puts well-supported hardware in the box?
A "nice" thing about the Linux approach is that you don't end up wasting pointer space since your driver will be compiled for the correct memory mode when it's built. I can't imagine the wastage is very significant for most drivers, though. And the memory mapping API and DMA API should ensure that high memory are Done Right whatever the memory model but only if they're understood and used correctly by the driver authors. I can imagine that there are drivers doing more or less naughty things that will violate these APIs in interesting ways, though, as you say.
Development-wise, an interesting trick the Linux kernel has up its sleeve is the "sparse" tool Linus wrote, which amongst other things can handle annotations on pointer types to detect cases where the kernel deferences a pointer to userspace, which you usually "get away with" but no always. I understand it's found a fair few real bugs in the kernel so far.
PAE enlarges the page table entries to 64-bit (IIRC). On a 32-bit PAE system you can stuff 36 bits of physical address in there. To compensate for that, the page table has one more level, making it 3 levels high instead of the 2 that non-PAE 32-bit x86 uses. The root node of the page table is 4096 bytes, the size of an x86 page, however it only contains 4 entries. This is because the userspace code running on it still only gets a 4GB virtual address space.
So basically it's a slightly weird-looking extension of the original x86 page table format that can handle larger physical addresses. You end up being able to have >4GB physical memory but you can't have more than 4GB mapped in any process (or the kernel) at any given time. On Linux this usually means 3G address space per process, with the kernel having 1GB. The extra bits in the page table entries also allow support for the NX (no execute) extension.
What's EMS? ;-)
It sounds like you think PAE could, in principle, enable 32-bit apps to support more than 4GB address spaces? Maybe I'm misunderstanding. For avoidance of confusion, that is not what PAE does; if you're using 32-bit + PAE you're *always* stuck with 4GB address space per-process no matter what (and 1-2GB will usually be taken up by your kernel). All PAE gives you is the ability to use more physical memory across the whole system. If you need one app to use lots of memory, as you rightly pointed out, you need 64-bit.
I saw another post on here suggesting that server versions of Windows have an API for giving an app access to >4GB memory on 32-bit but that is going to be some kind of hack (e.g. switching between multiple address spaces for one task or using effectively a big ramdisk like you mentioned), not true >4GB virtual memory.
I don't know exactly about the Apple technique but I do remember seeing a presentation by an AMD chap who was describing how you could use 64-bit to increase the address space available to 32-bit apps. In his example, though, that was "merely" enabling apps to use a *full* 4GB for application data whereas on a 32-bit system 1-2GB of the address space is occupied by the kernel. This sounds like the same kind of thing you're referring to. I'd guess it would be tricky to do this without upsetting some poorly or strangely coded apps somewhere along the line. It still leaves you stuck with 32-bit pointers, so really big apps are still ruled out :-(
There were patches developed for Linux to allow a usable 4GB address space (well, probably *slightly* less!) for applications on 32-bit platforms. That meant using a separate pagetable for the kernel and the userspace task and switching back and forth for each syscall. That's going to be expense so you wouldn't have wanted to run those patches unless you *really really* couldn't manage without.
For Linux, 64-bit would be my preference. But my only 64-bit machine is limited to 32-bit non-PAE for "technical reasons" at the moment (meaning the kernel I need to run needs that setup).
I was just wondering whether Windows could support PAE and non-PAE with the same kernel binary - like Solaris and unlike Linux.
The /3GB switch intrigues me - surely there must have been some sensible reason why processes were limited to 2GB (presumably with the kernel taking 2GB). Do you lose anything by applying the switch?
Intel has had 32-bit CPUs with PAE support since before 64-bit CPUs existed. AFAIK they specced PAE and AMD just implemented it. I think it might actually date back to the Pentium Pro, though I've not checked that's true for sure.
However, one source of trouble is that even years after they introduced it not *all* Intel CPUs actually support it. I've generally had the impression that most do, however I suspect they would have disabled it in their budget (and possibly in their mobile) lines. I had a Celeron M laptop that didn't support PAE but as it's both mobile *and* budget I never figured out who to blame ;-) I guess that's not entirely unreasonable as those chips probably wouldn't live in a motherboard that could address any more memory. These days it'd be a bit more annoying to lack PAE, even on small memory systems, since the NX bit depends on PAE being enabled.
I actually thought that all 64-bit x86 CPUs were backwards-compatible with PAE, however what you say seems to contradict this. I wonder if in the case of the hackintosh it's possible that it's a difference in chipset support or MacOS behaviour, rather than a CPU problem?
As you say, though, at the end of the day the approach that allows you to make the most of your memory is to have a decent mobo and a 64-bit CPU (of whatever make). That way you can use all physical memory and processes are not limited to a 4GB address space.
If you want the icing on the cake, either get PCI devices that can DMA to all your available physical memory (many won't be able to) or get a system with an IOMMU (and an OS that supports it) to compensate for those devices that can only access limited areas of physical RAM.
I'm not sure what feature you have in mind (although I can think of some possibilities) but PAE doesn't work like a paging file. It also doesn't require you to move data around in memory so that the CPU can access it - PAE is a hardware feature and gives you the ability to address up to 64GB (if your chipset allows) of physical memory. You only have 32-bit virtual memory addresses, i.e. pointers. But there are 36-bit physical addresses at the hardware level. Because you still have a 4GB virtual address space, so you can't address the full 64GB *at once*. But the kernel can update the pagetables to map different portions of the physical RAM any time it wants, so you don't have to move the actual data about.
The 4GB virtual address space limits any given process (and the kernel) to a maximum of 4GB mapped memory at any given time but you can still use all the physical memory by running different processes in different parts of it, so it can still be useful as long as you didn't want to just run one *huge* process. If you're running scientific or database workloads you might *really* need a larger program address space, in which case you really do want 64-bit.
The main example of where you actually do need to copy stuff from low to high memory (and vice versa) is for supporting DMA to devices that can't address high memory. The CPU can always directly address any memory by creating a pagetable mapping appropriately. To compensate for a DMA device that can't access a given high memory page, the OS creates a temporary mapping, copies data somewhere the device can DMA it, lets the device do its thing, then copies the data back . That *is* ugly but 64-bit does not solve it either if your devices have this kind of limitation. An IOMMU can translate from the device's DMA addresses to real physical addresses, making it possible for even limited devices to DMA to any part of memory. That's the "proper" fix, short of having all devices support DMA to all of memory. IOMMUs are starting to appear more and more on x86 hardware, so hopefully this issue will start to resolve itself.
Anyhow, with respect to the current article, Windows *can* do PAE to support systems with >4GB physical memory, you're just not able to run single huge processes because there's still a limited virtual address space. But despite its PAE support, it deliberately limits you to 4G physical memory, unless you buy a more expensive license. This isn't that unusual a practice but it's still always a bit upsetting to realise that the OS is being intentionally broken for commercial reasons.
Mmmm - there was a guy at AMD (I think) working on (ab)using the GART to remap DMA so that virtual machines could drive DMA-capable hardware without knowing about the real memory layout on the host. He was doing this for Xen (which I also do work with, sometimes) but I don't know what came of it.
General purpose IOMMUs are good for all kinds of tasty stuff, so it would be nice to have them more widely available - it seems to be coming.
Often would have done - but this time ;-) I thought of it roughly in the middle of the rest of my ramble. Maybe I'm just slow today!
Yeah, if you have a device that can't DMA above 4GB (as you say, this is common) in the case where you're DMA-ing to memory about 4GB the kernel often has to allocate "bounce buffers" in lower memory, DMA to them, then copy using the CPU.
That's the case for 64-bit systems, too. But with a 64-bit address space available, the kernel can probably keep mappings to all memory so that the copies are faster (I assume it does but I don't work with 64-bit and have never checked). On 32-bit, the kernel would have to create a temporary mapping each time a copy was needed, which would slow things down even more.
There seem to be a surprising number of systems out there that have >4GB of memory but are not 64-bit capable though - lots of people bought lots of cheap RAM for their 32-bit servers.
I hadn't realised that the video RAM was in those regions. I guess it makes sense, though, it has to be somewhere :-) That's going to be a bit hit in physical address space right away.
However, AIUI the other PCI MMIO regions can also waste more space than you'd expect since (depending on the machine) the BIOS sometimes sets them up in a sucky way and wastes way more space than it needs to. Which sucks.
But the article seems to be about a licensing-based restriction on allowed physical memory, depending on your version of Windows. So even if Windows and your hardware supports PAE, it may not let you use >4GB RAM. (nb. PAE is potentially useful even if you don't have more than 4GB RAM, since it gets you the NX bit).
So is that the reason that Linux has the same limit?
Unless I'm misunderstanding you and the summary (not RTFAed yet - I know, I'm bad) Linux doesn't necessarily have the same limitation. The summary seems to imply that only 4GB of *physical* memory are being supported on 32-bit systems by some versions of Windows. All OSes running on 32-bit x86 are restricted to a 4GB virtual address space, therefore neither Linux nor Windows can offer more than 4GB of directly addressable memory *per process*. But many 32-bit x86 CPUs can address more than 4GB of physical memory using PAE mode.
The base requirement for using PAE is that the OS supports it - Windows can use it, Linux can too. PAE gets you the ability to address physical memory addresses over 4GB but only if the motherboard has enough address lines to actually relay these requests to the memory. Even if you can't / don't have more than 4GB of RAM installed, it may still be worth it because PAE also lets you use NX (no execute) on 32-bit. PAE on 32-bit doesn't make the process address space any larger, so any given process is still limited to 32-bit pointers.
32-bit Linux has to be compiled with or without PAE support, so your distro should make sure it installs a the right kernel version to correctly support your hardware. Some OSes (e.g. Solaris) can switch between PAE and non-PAE at boot time, so they only need one kernel image to support both modes. I don't know how Windows handles this.
A further benefit of PAE, that I've left till last because it's really somewhat gross: the memory mapped IO regions used by PCI devices also take up part of your physical address space. This can result in a "memory hole" where your *real* RAM is overshadowed by an area of memory-mapped IO regions - the real RAM is there but you can't access it. As a result, if you put 4GB of RAM in your system and use a non-PAE kernel (only 4GB of physical address space) then you are *guaranteed* not to be able to use all that RAM because you will *definitely* have hidden some of it "behind" the memory mapped IO regions. If you use PAE mode, the motherboard can potentially remap that "hidden" RAM to a higher physical address, so that you can use it. Not all motherboards support this remapping, even if they *do* support 4GB worth of DIMMs - which is rubbish because they're claiming to support memory that the hardware will never let you actually address. It pays to be wary of this kind of craziness - my mobo has this limitation, which I discovered after buying 4GB of DIMMs (so I stuck one in another PC, which kept me happy in the end).
Uh, sorry to the parent poster, I've rambled a bit off the point I was originally making and probably just told you a load of stuff you already knew!
Neater to add
kernel.randomly_discharge_battery = 0
to your /etc/sysctl.conf
When I went away to university in 2000, I intended to use my digital wristwatch as an alarm clock. My Dad insisted that I should have something more substantial to wake me up - as a result I inherited his old bedside clock from ... the 80s, perhaps? Ancient, huge, unsubtle - one of its most noticeable features was a glowing digital display that was clearly not LED based. I had rather assumed it was some sort of fluorescent display but I've never seen another one like it. Looking at this clock the display is quite obviously the same stuff. So my ancient alarm clock has now become cool?! Awesome! Only trouble is that it broke a couple of years ago, so I don't get to ride the retro wave :-(
A shame, since the continuous, vicious sawtooth soundwave it produced instead of a "beep beep" could be heard from other floors in the building and would continue without a break for upwards of 30 minutes if you didn't wake up and hit the thing. For years that had me bolt fully awake in an instant with my fight-or-flight reflexes fully primed, wondering what the awful noise was and ready to sprint to lectures. They don't make 'em like they used to.
I feel old now!
I read "Scope Creep" as meaning creepy people, looking at you with telescopes. Having read your post I see what you mean. But your meaning and my original understanding are not mutually exclusive in this instance :-)
Actually I think TFA also quoted a mention from Carmack that they're only using GL 2.x stuff - which makes the claim seem even more odd.
I took it to mean that they're making use of driver features that are badly debugged / optimized. Still, I'd have thought a port with the option to turn down detail if your drivers can't keep up would be acceptable to many people who want to run commercial games on Linux ...
There's also an irony in here: the massive corporations of Hollywood (and Big Media) in general seems very happy to - with a straight face - cast "OMG Corporations!" as the soulless villains. Of course, it's rarely a *media* company that's being evil, it's the industrial-military complex. Whilst I don't entirely disagree with this, I do wonder if they really *get* that they are also large, faceless corporations who seek to control assets to enhance their own profit? I understand their point but it does seem a bit hypocritical.
"and that the Linux port would have to use the binary blob graphics driver in order to work"
From TFA, it seems that Carmack believes it would be hard to get the necessary performance without using the NVidia drivers. It's somewhat surprising to me if it wouldn't be possible to get it running acceptably on anything else, even if the game does use a lot of advanced features - but if Carmack says so!
Yea it's really hard to imagine that an up-to-date, specialised driver for specialised hardware straight from the manufacturer would be faster than a 3rd party driver.
That's not what I'm saying - I'm happy to assume that the manufacturer's binary blob drivers are *faster*. What I'm surprised by is the suggestion that no other drivers are fast *enough*. I'm quite surprised by the assertion that only the NVidia binary driver is fast enough to be worth running the game on at all. Can't I lose a little performance and use a different driver? What about ATI's binary drivers, they're made by the hardware manufacturer. What if I turn down the detail settings in the game - sure that defeats the point of the cutting edge graphics engine a bit but maybe if the game is fun *to play* it'd be worth it anyhow!
This argument sounds like a cop-out to me. They don't want to port to Linux and try to blame some sort of belief that blobs are evil. I wouldn't even want a 3rd party driver for something as performance-critical as the GPU. This whole "we don't port because then you would have to use something that's readily available for free" sounds bogus to me.
Now I definitely agree with that - it doesn't sound like a particularly convincing technical argument, more like a bit of an excuse. It's fine if they don't want to do the port but it would be nice to have more insight as to the actual reason.
Well, that's fine, isn't it?
The licensing on the various OSS projects allows you to use them to create closed-source products. It's one of the Freedoms you're being granted - if the developers had considered fighting closed source code to be more important than that freedom (and other related ones) then they'd have licensed the tools differently. They could have chosen a license that says "If you use this then you must donate other stuff to the open source community for free" but the developers of the code didn't, since that was not the conditions *they* chose for *their* code.
We're typically all pleased to hear Linux is being used as a platform for rendering the latest awesome movie's CGI sequences but it's probably running custom closed code. We're all pleased when a major company switches to Firefox or Open Office but they're probably also using copyright law and patents to control various of their products. I don't see how this is different, OSS code being used to produce stuff is generally a good thing no matter what license the product is released under.
"and that the Linux port would have to use the binary blob graphics driver in order to work"
From TFA, it seems that Carmack believes it would be hard to get the necessary performance without using the NVidia drivers. It's somewhat surprising to me if it wouldn't be possible to get it running acceptably on anything else, even if the game does use a lot of advanced features - but if Carmack says so!
However I'm not so keen on his assertion that if you're using the binary drivers you might as well run the code under Windows. I guess this probably *does* make sense for most people, since there are relatively few people who don't have a Windows license available somewhere. However, it would be *nicer* not to have to reboot into Windows for a specific app even if that were unnecessary.
Unfortunately I saw a fair few quite negative reactions in the linked thread and I expect we'll see others here. Carmack has not ruled out a port for sure. But even if he does, that's not exactly evil or a betrayal of open source or anything else negative. Many gamers here will have benefited in some way from the GPLed code he's released to the OSS community in the past at some point, pretty much all gamers will have benefited from his position as a developer pushing the games industry forwards. He's not done anything *bad* here, he's just not necessarily doing something we'd hoped for.
Hopefully the Rage code will - one day - be GPLed and get ported to Linux. I think that's a fair way down the road at this point, though.
Is that where X.org came from? ;-)