Slashdot Mirror
SuSE and Siemens Release Linux Memory Extension
hussar noted that SuSE and Siemens have developed a memory extension that will allow Linux to use up to 4GB of memory. Linus has reportedly approved its inclusion in kernel 2.3.15. The strangest part is excite has taken to posting about Linux Kernel Patches. Pretty crazy stuff.
Not on my desktop sadly, but the latest UNIX server we bought here has 2GB of memory and that didn't cause too much excitement.
That kind of memory is pretty common place - and Linux needs to support it.
Has anyone played with SGI's journaled file system for Linux? I am glad to see this major shortcoming being looked at.
Thinking of Maud you forget everything else. -- hack v1.0.3 [] Who was that Maud person anyway? -- nethack v3.1.0
Hello,
:) In the right crowds, I'm sure that's an important feature. :)
/just/ the right circumstances.
:)
Yes, it was key to supporting IRIX.
What's important to note and understand is that everyone, and I mean EVERYONE, has a differnet idea of what is and isn't a good feature. And believe me, I get a fair amount of feedback on my articles and I rarely can please everyone. There's always the person that just isn't happy until the PC internal speaker is supported, or until the kernel can do ISA PnP (it can now), or until it can do some other completely bizarre task that seems trivial to us at first, but them you realize how critical this can be under
That's what makes Linux great: we have such a selection of great things that we have to please everyone a little; but we'll never please everyone entirely.
As for me, this 4G thing seems pretty silly.
Joe
Part of the FreeBSD kernel are actually under GNU...
Most certainly not!
Berlin-- http://www.berlin-consortium.org
DNA just wants to be free...
HP-UX 11, being a 64-bit OS, breaks the 4 Gig limit, but IIRC the actual limit varies a bit between "patch levels". They're moving it upwards all the time, I seem to recall.
--
Linux user since early January 1992.
wow that's a user of my irc client! ;)
just a typo...
The issue may be one of whether the extra 4 physical-address lines are actually connected to the world off the chip; that might not be the case on all P6-series processors, even though the processor on the die might contain all the hardware necessary to put non-zero values on those lines.
Yes, NT (unless you turn on the right boot flag, which might work even in non-enterprise editions) has 2GB of the virtual address space of the machine available to user-mode code and 2GB for kernel-mode code, just as had been the case for Linux all along.
However, I'm not sure that limited you to 2GB of physical RAM on NT, as NT might not require that all of physical memory be mapped into the kernel's address space.
Linux has, for a long time, supported "unlimited user space" in the sense that user-mode code can map stuff into and out of its address space, so that the no-more-than-4GB window you get on a processor with 32-bit linear addresses (e.g., an x86 processor) can be moved around a more-than-4GB set of data. The patch doesn't change that.
However, that's true of most OSes that run on x86 processors, these days.
You don't need to use segmentation to get at >4GB of physical memory.
In fact, segmentation doesn't even help, given that the x86 MMU maps 48-bit segmented addresses to 32-bit linear addresses before running said addresses through the page table and translating them to 32-bit or 36-bit physical addresses.
However, for any single process to access more than 4GB, it does have to do something that amounts to bank switching, i.e. map stuff into and out of its no-more-than-4GB linear address space as necessary; the same applies to kernel-mode code and the kernel-mode portion of the address space.
NT 4.0, or NT 5.0^H^H^H^H^H^HW2K? I don't think NT 4.0 supports more than 4GB of physical memory, but W2K, at least in the DataCenter Server edition, will.
See, for example, this note on "Address Windows Extensions and Windows 2000 DataCenter Server", and this press-release-like document which indicates that this is new in W2K and not in NT 4.0, but also seems to imply that memory above 4GB will be available to the page pool (I'd seen stuff that gave me the impression that it would be wired-down memory that had to be specially mapped into a process's address space, but perhaps that's not the case).
No, it doesn't. It just has a 32-bit base linear address, a 20-bit length (which can either be in units of bytes or 4K pages), and a bunch of other flags. A 48-bit far address gets translated to a 32-bit linear address, and that is what gets translated to a 32-bit or 36-bit physical address via the page table.
See (this page that has a link to the PDF document) Intel Architecture Software Developers Manual Volume 3: System Programming for the full story.
thanks for the update - I think. Would be better to hear that it was working great and I should switch to it now. Not having a decent journaled FS is a major downside to Linux in my opinion. Who wants to put a lot of disk on a server and then have it take 3 weeks (ok a slight exaggeration) to fsck. Not me.
Actually I do have a couple of servers with a small amount of disk on them (16GB each as I recall) and the main problem is how slow NFS is.
still kernal level appletalk support is nice. Is all swings and roundabouts.
Thinking of Maud you forget everything else. -- hack v1.0.3 [] Who was that Maud person anyway? -- nethack v3.1.0
Windows98 supports more than a gig.
God, I like FreeBSD (maybe even a bit more than I like Linux, in fact), and I use it in both my gateway and my laptop, but looking at some of its supporters, maybe they should rename it FreeBSE and be done with it...
I really expected better from the user base than a bad case of Linux Envy...
"Be nice, veer left, and never stop thinking" Iain Banks - Walking On Glass
yep. for all the fears people had (and FUD that got spread) about the commercialization of Linux, here we see the Open Source (and specifically the GPL) magic working just right. Large companies like Siemens suddently find it (financially) worthwhile to contribute to an open project. this is a huge ball that got rolling; it's not just a bunch of hackers competing with MS (and Solaris and ...), now it's large companies and their resources, too. and to think that some MS drone was quoted saying that "the linux hype has peaked" just a few days ago... boy he's in for a surprise :)
Yes, P6's have 36-bit physical memory addressing; this does not mean that virtual (linear) addresses are 36 bits. (Some other 32-bit processors, e.g. 32-bit SPARC processors or processor modules with the SPARC Reference MMU, also supported more than 32 bits of physical address. Heck, PDP-11s with MMUs often supported either 18 or 22 bits of physical address, even though they only supported 16 - or 17, sort of, if you count split I&D space - bits of virtual address.)
Dynix, on Sequent's servers, might - they support much more than 4GB of physical memory, and if the NUMA part of "NUMA-Q" just means that accessing "other people's" memory is slower, not that you have to play bus-mapping games to get at it, they may use the 36-bit features to do that - although I thought I saw something indicating that some of their machines supported 128GB, which requires 37 bits.
what's the point of giving 2GB to the OS? I know the NT kernel is obsese compared to Linux, but still... that sounds like some serious overkill to me, and using almost half of your machine's memory as page cache (in Linux terms, I'm sure NT has an equivalent that's calle dsomething else) doesn't strike me as a very balanced use of a large box.
on a completely unrelated point, I wonder if this new 4GB support is a compile-time option, or if it can be somehow enabled at runtime without a performance hit when it's disabled. that would be neat, but it does sound a bit improbable for a change like this.
yeah it does. it has to involve mapping memory zones on demand instead of keeping the same lot of things mapped at the same time. the good old equation that 32bit = 64Gb doesn't mean that you can put 4Gb of RAM in your memory and use it: you need virtual addresses for all kinds of pages that are either paged out or mapped to disk and set to load on demand. and Linux so far has been mapping kernel memory while userlevel code runs too, with just the permissions changed, b/c it's cheaper to do this than to remap it for every syscall and unmap it on the way back. I have no idea how much of this scheme this new patch changes.
heh, so much for the "code freeze in 2 weeks" thing :) oh well we've known that Linux code freezes are quite fluid after all, and that's probably a good thing. good software comes out when it's ready, not when someone decides on a deadline.
woohoo! moderated up! i know, i know, pointless comment, moderate me down i dare ya. i'd like to point out however (before you do) that each app gets 2GB and not all apps sharing 2GB in userspace like my previous otherwise wonderful post *cough* seems to imply. somebody else posted somewhere that the PIII has 36bit addressing letting it potentially use 4 terabytes of RAM. who wants to help me write the linux kernel patch for that :)
Satan, oscillate my metallic sonatas.
We need 3rd party support by manufacturers for products that they sell to make linux more profesisonal
Shine on, you crazy diamond.
...and Andrea works for SuSE, you ... (ok, I don't wanna get downgraded for this psoting)
--
Michael Hasenstein
http://www.csn.tu-chemnitz.de/~mha/
My best example of this was when I saw a rehash of Pravenich's 2.4 kernel thing on a tech news site - they'd duplicated his mistakes, and were brainwashing the masses with the stuff! ;)
The GNU thing is certainly untrue.
And *when did I ever say bsd has no drivers*.
-- dieman - Scott Dier
That would be funnier if what you said about MS were true and didn't contradict.
Office 2000 can now run on Linux with 4GB.
W2K only supports 128M?
None since w2k is no longer supported on non-intel architecture (Compaq recently stopped NT development on Alpha)
core dumped.
I'm interested if anybody knows what are the limits of Win 9x/NT, BEOS, *BSD, MacOS,...
Linux rox!!!
A big company does something, they send out a press release. Nothing new there, it is just that it is about Linux.
Now, that someone has come up with a bigmem patch that Linus will live with, THAT is news!
This is a big deal for some users. A real shocker will be if someone comes up with a patch to use the 36 bit addressing on the P6 cpu, for up to 64GB ram on Intel machines.
Plato seems wrong to me today
i'd guess it doesn't affect it at all. ia64 has plenty of address space, so there's probably no need to play remapping tricks there.
Its not unusual really. It fits into excite's usual MO.
:)
For the non-journalists/editors in the crowd, I'll point out that this story was A) from a newswire service (check the DATELINE--ITS IN ALL CAPS LIKE THIS) and B) the story was obviously a press release (witness the "About SuSE" section of the article -- an obvious shameless plug for the company putting out the press release).
Excite's news section consists almost entirely of newswires because its cheaper than writing your own stories and they generally don't require too much editing (because they are written by professionals who know what newspaper editors like to see in an article in terms of structure, content, syntax etc.)
FWIW, I used to design/edit/publish several newspapers for non-profit veterans groups like the AMVETS and PVA...at least until I got a "real" job in the IT field.
My journal has hot
I find it somewhat telling that the article didn't mention Andrea Arcangeli and Gerhard Wichert, workin at SuSe and Siemens respectively and wrote this patch pretty much dual-handedly. I suppose with the corpratization of linux, the companies are more important now than the actual people who make linux what it is. At the very least, a link to Andrea's archived message on l-k would give credit where it is due.
I vaguely remember reading that Solaris now supports it, although finding detailed technical information on Sun's shiny new Marketing-Driven(TM) Web site looks as if it'd demand more patience than I have.
It's probably a question of whether any "commodity" x86 machines support it; if not, then Solaris for Intel, and UnixWare, may not support it either.
Not in the case of P6-core machines; segmentation turns a 48-bit segmented address into a 32-bit linear address. They're using paging, instead, i.e. the page table entries, in one of the 36-bit-physical-address modes, generate more than 32 bits of physical address from 32 bits of (linear) virtual address.
Any one process would have to map stuff into and out of its address space to use more than 4GB (or 4GB minus what kernel-mode code takes) of physical memory, but
Kernel-mode code could also map stuff into and out of its part of the address space.
This is great work. Just think what will happen when the SGI big memory project is ready. Check out http://oss.sgi.com/projects/bigmem/. Wow. Two big memory solutions. I just don't know which to choose. Oh. Hang on. I don't have four *fricking* gigabytes of RAM....
- The hardware vendor gets to show off their hardware.
- The hardware vendors gains some purely positive publicity and goodwill from the community
- Everyone else benefits from the contribution.
Plus, a significant contribution can be merely the seed for further developments, since anyone in the world can read the patch and contribute their own. It's a win-win situation all around.Well ... it depends what you do. Where I work (in the EDA industry), most of our boxes are 2Gb, and we have a few 4Gb boxes, and that's not enough... We still swap like crazy for some jobs.
I unfortunately can't find more than 4Gb on UltraSparcs (our main platform) without going to a large server. 8(
However, it appears that most of the 64 bit architectures (Alpha and UltraSPARC) don't use 64-bit addressing. They use something more like 39-bit or 42-bit -- something in that general neighborhood.
:-)
39-bit = 549755813888 = 512 gigabytes.
42-bit = 4398046511104 = 4 terabytes.
45-bit = 35184372088832 = 32 terabytes.
You only need 4 terabytes of memory if you're altavista or fedex and you want to keep your entire database in memory at once. Otherwise a couple-hundred gigabytes should do you fine.
(I wish I could remember the specific figure...)
I believe the 36 bit addressing only allows you to use memory beyond 32 bits for paging; buffers, cache, etc. Logical addressing (program addresses) is still limited to 2GB or 3GB or whatever. So if you put 64G on your quad Xeon factory heater (tm), programs still max out at 4G. But you get plenty of disk buffers....
--
Infuriate left and right
On a dual xeon with 2GB, I was able to run vmware sessions of 95, 98, and NT all at the same time under linux. Besides having the machine taking up four IP addresses, it all ran very fast.
Unfortunately, my boss did not like the idea of running MSSQL server that way, and the ServerRAID would not work (IBM has since release drivers, however.)
Just a quick note to everybody. It is not about being able to use 4GB of physical memory. It is to enable process to use more than 2GB of memory. Traditional Linux memory model to is split the lower half for kernel memory and upper half for user memory. To check if the pointer is pointing to a kernel memory you just need to check its MSB.
test %eax,0x8000000
je user_mem_label
I think they have worked on 3GB prior to this. SOrry, been a while since I checked the kernel lists.
Anyhow, this is only of practical use to database developers. maybe some but not many. In any case, you might as well use a 64-bit architecture.
Hasdi
...from Siemens, not a media story.
Has anyone played with SGI's journaled file system for Linux? Argh.. I submitted this the other day, but apparently Warcraft 3 was more interesting.. (You call it sour grapes, I'll cal it wine...) 'ext3' (ext2 fs w/journaling) has been officially released as a testable beta. also, reiserfs is close (like, days) away from releasing a journaling patch. check out www.devlinux.org for more info. to paraphrase Hans Reiser, "XFS (on linux) is more of a well-funded press release than a well funded software prject." -- blue
i browse at -1 because they're funnier than you are.
IIRC, a swap partition or swap file can only be about 128 MB under linux, with a maximum of 16 swap spaces, leading to a total of 2GB of swap space. Since we can now use 4GB, how are we supposed to allocate enough swap space (I prefer 2 times the physical memory).
Has this annoying restriction of 128MB been removed, or can we use more swap spaces (lets see, 4 GB divided by 128 MB gives way too much swap spaces to be practical), or are we not supposed to use virtual memory any more ?
If the restriction of 128 MB per swap space still exists, is there anybody working on removing this so Linux can become practical for modern computers? If the restriction is removed, is it possible to create a single 8GB swap space ?
Well, from expearance, win 98 wont be stable with 1 gig. I once put 512Meg in a Win 98 system (we had 4 128M DIMMS at the shop, and I just wanted to see). Strange things started to happen, it blue screened quite a bit. It wasn't prity. I would say that you could stick 4 Gigs in a 98 machine, but it wouldent stay running for more then 5 minutes.
640Mb should be enough for anybody!
So what was the difficulty in allowing Linux to use 4G? I understand their was some issue with flushing certain buffers so was this a x86 only issue?
How is it that FreeBSD handles more RAM? Do they merely take a performance hit or do they have better kernel architecture?
Marriage is the "pseudo-ethics" that cloaks the messy truth of sexuality in the raiment of propriety -- it's "Don't Ask,
Umm. Only the Xeon CPUs have the 36-bit address bus (versus 32-bit for the standard PIIs - and probably the standard PIIIs also) allowing addressing up to 64 GB of RAM. Also, using those top 4 address bits takes special OS support, and AFAIK Linux doesn't support that as of yet, unless these Siemens/SuSE kernel mods also allow for that.
Sam: "That was needlessly cryptic."
Max: "I'd be peeing my pants if I wore any!"
Well, don't discount hardware (in)stability. Very often, pc motherboards deal with (lots of) ram, especially in lots of slots, quite poorly.
What type of motherboard were you using?
Try the same eact setup with linux, you'll probably have problems there too.
Code freeze? They should more call it a "congeal" rather than a freeze...
What am I missing.. I remember ages ago a friend of mine put Linux on 2 4 gig quad processor Xeon boxes which at the time needed kernel patches to have it recognize the rest of the mem but they got it working.. We only run solaris on here (w/ only a gig) but don't have this problem on intel boxes.. but (i'm at home now) but this is what my xconfig 2.0.38 says :: under "General Setup" subsection "Memory Configuration" There are three memory configurations available the standard allows use of just under a gig of ram w/ 3 gigs of virt space per process , the enterprise uses 2 gigs of mem but limits process space to 2 gigs.. the custom option allows you to specify the split, subject to kernel constraints.. Linux/x86 can use up to ~3.4 gigs of physical memory... Anyway it sounded vaguely like the NT kernspace userspace splitting you could set.. I know, no one is reading this anymore :(
Where do you get 64 Gb from 32 bits of address space? It'd be more like 32 Gb (4 GBytes).
Sam: "That was needlessly cryptic."
Max: "I'd be peeing my pants if I wore any!"
Ditto.
I'm surprised to see this sort of thing again. I can remember the first time I saw this kind of stuff, on the old Atari 130xe: 128 Kbytes through 'bank switching'...
chris
San Francisco values: compassion, tolerance, respect, intelligence
Could somebody explain to me how (or perhaps why) some of the top-of-the-line Intel-based servers claim to support even more than 4 Gigs of ram? The page for the new IBM Netfinity 8500R, for example claims that it supports "Up to 16GB ECC SDRAM". How is that even possible, since I thought 32-bit architecture made 4 Gigs the max addressable limit, period. Am I wrong? Are they simply selling a capacity nobody can use? This has confused me ever since I first heard about it.
Thanks.
5) Office 2000 memory requirements now supported by Linux, making port much easier.
4) Enhances sales potential of Windows 2000 -- WINE now able to run W2K under Linux.
3) Yet another fun Linux feature to deny and obfuscate.
2) Can complain before tech-unclued journalists about Linux's memory requirements -- 4G compared to W2K's 128M.
1) Now that Linux supports 4G of RAM, it will be competition on the everyday Joe's desktop, thus making MS-DOJ trial irrelevant.
(Darn Excite, slashdotted again.)
We want endless gardens of data, where the bits can flower, flourish and reproduce. -- Andy Mueller-Maguhn
since linux can't run with more then 960 megs....
"Subtle mind control? Why do all these HTML buttons say 'Submit' ?"
ReadThe ReflectionEngine, a cyberpunk style n
geez, you guys are as bad as the Mac people. Now the linux kernel can do this to. We all know FreeBSD can access 4gb of memory. Who cares?
"Subtle mind control? Why do all these HTML buttons say 'Submit' ?"
ReadThe ReflectionEngine, a cyberpunk style n
Supports or demands? :)
I don't subscribe to RMS's GNUtopian vision.
That means your heavy-duty stand-alone applications require you to purchase a very expensive product, which is tuned exclusively for network server and database work. You'll get an expensive set of server apps and licenses "for free" which you'll never use, and wait several weeks after the initial release for each and every service pack (they're always delayed for the high-end NT versions).
If you're interested in spending this kind of money, run Linux or *BSD on hardware that's designed to have 64Gb of memory and 12 CPUs (eg UltraSparc or Alpha) and don't worry about stupid kludges from Intel or any other 32-bit vendor. When even Intel are telling you that IA32 is a dead-end, it's time to get off.
In fact, if you've really got this type of job to do, your priority is probably scalability and performance, in which case a Proprietary Unix on it's own native hardware is going to look much more attractive. Want to buy some E450s?
Nick.
95 a limit of 64 or 128? uh NO.
2GB system, 2GB user...same with 98.
I've run 95 on 256 - so it definitely isn't anywhere near as low as you think.
I've used a gig. Nothing special. Just never swap, and buffers up the wazooo. How much ram does ol' 98 support - I know 95 had a 64 or 128 MB limit. And I'll bet 98 can't do a fat gig. But then again, WTFDIK (what the F do I know...)
http://www.bombcar.com It's where it is at.
Fellowship 9/11
The site where I work has a 4-processor HP/UX machine with 4GB of RAM, running an Oracle backend for one of the webservers. It's woefully inadequate during peak times, and there are plans to upgrade it to something more robust in a few months.
AFAIK, from reading www.kt.opensrc.org, there is already a patch for this.
" Andrea Arcangeli posted a patch and announced that he and Gerhard Wichert had co-developed the patch to allow nearly 4 gigs of memory on 32-bit systems. A big discussion followed."
Located here
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
In short, way too much to be talking about ram for now.
My Freakin Blog
I'm just wondering if it involves more than changing a lot of `int's to `unsigned int's.
My Freakin Blog