Conquest FS: "The Disk Is Dead"

andfarm writes "A few days ago, I sat in at a presentation of a what seems to be a new file system concept: Conquest. Apparently they've developed a FS that stores all the metadata and a lot of the small files in battery-backed RAM. (No, not flash-RAM. That'd be stupid.) According to benchmarks, it's almost as fast as ramfs. Impressive." The page linked above is actually more of a summary page - there's some good .ps research reports in there.

Comments

well and good

[iamweezman]

this is great. We all have seen this coming, but how is the industry going to take this and implement it. My bet is it won't. The only way that it will take hold is if you can find some small application that will take and apply it.

Re:well and good

[robslimo]

I've predicted and eagerly anticipated the demise (by replacement) of spinning media (magnetic and optical) for 10 or more years now... I've predicted it will happen, not when.

As this new filesystem implicitly admits, the price/MB is still so much dramatically lower for HDD's than solid state memory, it will still take quite a will for this replacement to happen.

I disagree that some small killer app must come along to make this happen. Yes, solid state media is coming down in cost and increasing in density, but both need to change by 2 or 3 orders of magnitude before the HDD is dead. What we're waiting for here is the classis convergence of technology and its applications... the apps won't some until the technology can support it and the tech is driven by our demand for it. Expect another 10 years at least.

Re:well and good

[CrosbieSmith]

It will happen when the price difference between solid-state devices and magnetic storage gets narrower. That's not happening.

This was also pointed on Saturday's Slashdot Story

A mere $US5,000 would be something of a price sensation by the standards of current large capacity SSDs, whose prices aren't dropping nearly as quickly as are those of magnetic media.

Re:well and good

[robslimo]

True, and that narrowing will have occurred by the time the cost/density ratio of SSM has improved by 2 or 3 orders of magnitude.

A couple of reasons I see the death of the HDD to be not-to-imminent:

(1) Those damned HDD makers keep pulling new physics out of their as^H^H hats and keep pushing the storage densities to rediculous new levels.

(2) the solid state memory of the future ainta gonna be Flash as we know it now (with slow and limit write cycles) and it also will not be battery-backed RAM (unless we go write it all back to disk for 'permanent' storage at some point). I bet on some variation on today's Flash without its limitations, but the tech has got some ground to make before this all happens.

My other long-term prediction has been that CRTs (vacuum tube, for pete's sake!) will be replaced with LCD or similar tech and we're getting really close.

Re:well and good

[_anomaly_]

I've predicted and eagerly anticipated the demise (by replacement) of spinning media (magnetic and optical) for 10 or more years now... I've predicted it will happen, not when.

well, i've been "predicting" that the end of life as we know it will come, not when though.

Re:well and good

[blixel]

well, i've been "predicting" that the end of life as we know it will come, not when though

Ahahahahah... well said! Excellent point.

Re:well and good

[iabervon]

One important thing to realize about storage is that people's storage needs for some types of files grow over time, but storage needs for other things do not grow significantly. For example, if you separate attachments and filter spam, you can now buy an SD card which will store all of the email you will get in the next few years; when that runs out, you'll be able to buy a card which will store the rest of the email you will ever receive. There are similar effects for all of the text you'll ever write.

Furthermore, there are a number of important directories on any system whose total size won't double in the next ten years, because they add one more file of about the same size for each program you install, and they already have ten years of stuff.

In the cases where you do have exponential growth of storage use, the structure of the stored data is extremely simple; you have directories with huge files which are read sequentially and have a flat structure.

I see a real opportunity for a system when you have one gig of solid-state storage for your structured data and HDDs (note that you can now add a new HDD without any trouble, because it's only data storage, not a filesystem) for the bulk data.

Re:well and good

[cruff]

They've been saying the same thing about the demise of tape for large storage applications (we have over a petabyte at work). Tape is still going very strongly.

Re:well and good

[Anonymous Coward]

It's all about the OLEDs! Check out Kodak, they are leading the research field.

Re:well and good

[DoraLives]

I see a real opportunity for a system when you have one gig of solid-state storage for your structured data

It will be OS-on-a-chip (and a good OS at that), it will go for about twenty bucks a pop down at WalMart or CompUSA and Bill Gates will die of an apoplectic fit when it hits the streets. Hackers will figure out ways to diddle it, but corporations and average users will upgrade by merely dropping another sawbuck on the counter and plugging the damned thing in when they get back to their machine(s). Computers will come with these things preinstalled, so there'll be no bitching about not having an OS with any given machine. High-end weirdness will, as ever, continue to drive a niche market, but everybody else will regard it about the same as they regard their pair of pliers; just another tool. Ho hum.

Re:well and good

[Hard_Code]

So basically, as the cost of intermediate levels of storage comes down, you can manifest the different levels of persistence in that storage. Certainly not all data is created equally, and things like temporary file systems, and "real" RAM swap can be kicked to an intermediate cache before hitting disk. In fact, IIRC, hard disks already have some memory on board for internal use...it just isn't visible or available for general use to the operating system or rest of the computer (I could be wrong here). I would never want to have to rely on a battery for persistent storage, but perhaps advancing technology is allowing us to make these choices in gradation of persistence.

Re:well and good

[Anonymous Coward]

umm we've been there awhile on the lcd thing. Around 40% of new computers we sell are sold with lcd's, wake up and smell the pixels.

Re:well and good

[GreyPoopon]

Around 40% of new computers we sell are sold with lcd's, wake up and smell the pixels.

I think you're just confirming what the parent said ... we're real close to replacing the CRT in most uses with LCD.

Re:well and good

[Milik]

If you consider this new media it will require completely new device that will be build from scratch allowing for the battery and permanent memory. Also you have to worry about reliable memory and batteries, not only disks. That will reduce safety of the data. Also such disks will not be used in RAID arrays. I see this technology making it to very small market of embedded devices and computers but not to every computer. Once we can fit 60-80 GB of RAM inside of the HDD sized box than we can talk about new media, and death of HDD. This is a major step on the way but I don't' believe that we will ever implement this technology

Re:well and good

[Daniel Phillips]

I've predicted and eagerly anticipated the demise (by replacement) of spinning media (magnetic and optical) for 10 or more years now... I've predicted it will happen, not when.

You may have to keep predicting for some time yet. So far, nobody has managed to come up with a solid-state approach that gets anywhere close to the cost of spinning media, and though solid state gets cheaper over time, spinning media does too.

For the most part, posters to this thread missed the point of this effort. The authors observed that some relatively small portion of filesystem data - the metadata - accounts for a disproportionate amount of the IO traffic. So put just that part in battery-backed ram, and get better performance. Hopefully, the increased performance will outweigh the cost of the extra RAM.

The fly in the ointment is that, in the case where there's a small amount of metadata compared to file data, the cost of transferring the metadata isn't that much. But when there's a lot of metadata, it won't all fit in NVRAM. Oops, it's not as big a gain as you'd first think.

It's surprising how well Ext2 does compared to RAMFS and ConquestFS in the author's benchmarks.

Re:well and good

[Slime-dogg]

Yeah, and you could also stick the image of your OS immediately after booting onto that thing. Than you could have a near-instantaneous boot speed, something which would be very nice. A throw-back to DOS 5.0.

Re:well and good

[ePhil_One]

As this new filesystem implicitly admits, the price/MB is still so much dramatically lower for HDD's than solid state memory, it will still take quite a will for this replacement to happen.

The change is going to have less to do with the $/MB of "spinning media" and more to do with the the cost of Solid State media. If Western Digital introduces a 1 TB drive next year for $150, most folks still won't have a need for it. If Solid State can hit a target such as $500 for 9GB, it would likely start a landslide of conversions. The OS, your website, and a decent amount of logs could go on this to improve reliability, remote logging could log everything out to a central server with a few TB of storage just in case.

This isn't about replacing desktop p0rn stashes or Mega sized databases (though Megasized DB's feed of high porformace disks, not cheap desktop disks). Its about ultra fast random i/o, reliability, etc.

Re:well and good

[jandrese]

I remember doing this back when I worked for SGI. You could have the filesystem log anywhere you wanted it, even on a seperate device. If you put the log on a RAM Disk, you would be doing pretty much exactly what the article suggested.

The problem is that it only helps in certain instances, where you are continually accessing the same files over and over. Because these systems were running big databases, that wasn't really the case. What it DID do was smooth out the access times (latencies) significantly.

Re:well and good

[moonbender]

There are similar effects for all of the text you'll ever write.

HAH! Not if Unicode and it's successors have anything to say in it! ;)

Re:well and good

[Paradise Pete]

I've predicted and eagerly anticipated the demise (by replacement) of spinning media (magnetic and optical) for 10 or more years now... I've predicted it will happen, not when.

Well yeah. Let's take a few more wild stabs: CRTs will go away, the voice interface will become prevalent, wireless networking will be ubiquitous. And oh yeah, flying cars.

Re:well and good

[ces]

we're real close to replacing the CRT in most uses with LCD

Most high end users who are concerned about image quality are still buying CRTs. If you have to do color matching, CAD/CAM, or are gaming you probably still want a CRT.

The price differential between CRT and LCD monitors is still enough that most larger businesses are still only buying CRTs for most of their users. Sure the executives and receptionists are getting LCDs but everyone else gets cheap 17" CRT monitors.

Re:well and good

[Anonymous Coward]

pushing the storage densities to rediculous new levels.

Here's a handy tip: When writing the word ridiculous, think of ridicule and it'll be easy to spell.

Re:well and good

[Suidae]

you can have my CRT when you can give me a 19+ inch 2Megapixel LCD for the same price.

My code does not word wrap at 80 characters.

Re:well and good

[Lethyos]

I disagree that some small killer app must come along to make this happen.

I disagree with your disagreement. Microsoft products made us rethink how disk space was used. Now look at how much of it we have!

Killer application for storage: an operating system that's larger than any of the apps you run on it!

Re:well and good

[j3110]

Disks will never die. Most of everyone's drives are filled with multimedia. Disks, of some form, will always be the preferred format of this kind of information. All the functionality you could ask for is inherent in the media. You can seek and play. That's all you need, and you really only need to play one multimedia file at a time. With compression, I could get out really old 2MB/s drives and be more than happy if it would hold enough data.

Augmenting disk storage is where it's at. Putting the OS on Flash is such a great idea. Programs and the OS should be stored on some non-mechanical fast media, not neccisarily the same media. Flash will work for programs too. Program data should be stored on Compact Flash or some small, non-mechanical, portable media so that you can carry around your data in case you need it elsewhere.

Multimedia, on the other hand, will probably always be stored on some horrible media.

The idea of storing meta data on battery backed ram sounds a bit dumb to me though. I can't see how this would be any better than more system ram. If it is cheaper ram, then why not just build a better disk controller and have smart cache on it. If they have a more efficient way of using memory to access a disk, then it would seem that all they have done is make a better cacheing algorithm.

Re:well and good

[GreyPoopon]

Most high end users who are concerned about image quality are still buying CRTs.

Can I assume that by "image quality", you mean color-trueness within images? I don't see resolution as driver in the high-end market. There are high-resolution LCD screens available for that. But color is definitely an issue.

...CAD/CAM...

Enlighten me. What's the issue there?

gaming

I think we had an article in the last week or so about this. Apparently, some of the new LCDs with faster refresh are winning over some gamers.

The price differential between CRT and LCD monitors is still enough that most larger businesses are still only buying CRTs for most of their users.

Although I agree with you, I'm not sure WHY this is true. At risk of giving the pointy-haired boss new ideas, I'd like to observe that by replacing monitors with LCDs, you can cut down on the total space per employee and squeeze in a few extra cubes. That may offset the cost enough to make it worth it -- especially if the alternative is to rent more space. :-)

Re:well and good

[GreyPoopon]

you can have my CRT when you can give me a 19+ inch 2Megapixel LCD for the same price.

Yeah, I'm very attached to my 22" Diamondtron, too.

Re:well and good

[jedidiah]

It is highly unlikely that this person gives a d*mn about some "form over function" pedantic weenie such as yourself.

In a large enough forum, someone is always going to find some reason to ridicule you. This is quite independent of your spelling proficiency.

Re:well and good

[kalidasa]

Unicode itself tops out at 21.1 bits per character; the UTFs top out at I think 6 8-bit bytes for astral plane characters in UTF-8. A sixfold increase ain't likely to be a hardship to anyone. And since there's pretty much enough room in Unicode for all the writing systems we're likely to discover on this planet, I don't think we'll have to worry about successors to Unicode for a long, long, long time.

Re:well and good

[WatertonMan]

I think this is exaggerated. The actual source code isn't where most of my disk space goes. It is for (to me) unnecessary templates, clip art, and graphics. Even looking at my disk usage the majority of it is disappearing due to files and not applications. (I was rather surprised when I had to clean things up) The advent of DV and MP3's really changed things. While Office is, indeed, bloated, a lot of that bloat is in graphics and not code. But it can't hold a candle to all my MPEG-2 stores and MP3s.

Re:well and good

[Anonymous Coward]

Wow. You're cool. I predict that the sun will eventually expend all its energy. That means I'm psychic. It -will- happen!

Personally, I anticipate the demise of idiots like yourself.

Re:well and good

[machine of god]

My other long-term prediction has been that CRTs (vacuum tube, for pete's sake!)

Hey man, crt's aren't all that bad. Where else are you going to get a good flyback transformer without ending up on some gov't list. Or for free for that matter.

I dare you to tell my my paranoia isn't warranted. (Cause i'll just claim you're one of "them")

Re:well and good

[LotusNailo]

Rofl and I love morons who post anonymously to rip on innocent people with minor comments that really have nothing to do with the initial arguments. Your comments are not needed, so keep them to yourself! Or at least post with your name so we can smash your karma rating ;)

Re:well and good

[ces]

Can I assume that by "image quality", you mean color-trueness within images?

Yes color matching is the big issue.

Enlighten me. What's the issue there?

At the very high-end CRTs are still cheaper and still have better performance. Some places all of the engineers doing CAD work have something like the Sony W900 on their desk. A 24" Sony GDM-FW900 is $1,800 and does 2304x1440, a 23" Sony SDM-P232W/B is $2,600 and does 1920x1200. Also it is only fairly recently that large (19 inches and up) LCD displays have come anywhere near CRTs in cost or resolution. This is also a factor with graphic design as well.

Apparently, some of the new LCDs with faster refresh are winning over some gamers.

Still most FPS gamers buy CRTs and most that have a decent display won't be replacing it any time soon.

Although I agree with you, I'm not sure WHY this is true. At risk of giving the pointy-haired boss new ideas, I'd like to observe that by replacing monitors with LCDs, you can cut down on the total space per employee and squeeze in a few extra cubes. That may offset the cost enough to make it worth it -- especially if the alternative is to rent more space.

Don't forget reduced HVAC costs, CRTs pump out a lot of heat compared to LCDs.

In most organizations the IT people are responible for ordering computers, while facilities handles things like space allocation, squeezing in more cubes, and paying the HVAC bills. They don't necessarily talk to each other. Given the layoffs and downturns most companies I know of have a surplus of space right now so there really isn't a good reason to spend a bunch of money upgrading everyone to an LCD.

Most businesses buy the absolute cheapest monitors they can get away with for a majority of their users. Large monitors or LCDs are considered a status symbol much as laptops are. Unless you can give a valid business reason why you need something other than the standard you aren't going to get it unless you are high enough on the pecking order.

Some places may be buying LCDs for new systems but LCDs still cost slightly more and there is the status symbol problem as well.

Re:well and good

[Directrix1]

Not as long as you have that hulk of an ATX tower under each employees desk. Not gonna be saving much power all that soon.

Re:well and good

[Directrix1]

Excuse me, I meant to say: Not gonna be saving much space all that soon.

Re:well and good

[Anonymous Coward]

In a large enough forum, someone is always going to find some reason to ridicule you. This is quite independent of your spelling proficiency.

I think you read too much into it. I was merely suggesting that thinking of the way "ridicule" is pronounced helps in remembering it's spelled Ri. I was not suggesting he be ridiculed. Had he written "definate," for instance, I might have suggested remembering that definite contains the word finite. That's all. Nothing more.

Re:well and good

[iabervon]

I'm holding out for the brick fileserver (has more battery life than you'll survive, rechargable inductively if you insist; all the storage you'll ever use for non-bulk content; exports the filesystem wirelessly with encryption; watertight, rustproof, silent; has one button, used to tell it when the owner connects the first time). Then you put your graphics card and DVD drive in your flat panel display, run your OS and software on the GPU, and donate your computer to charity. Or maybe you run your OS and software on your smart card.

Re:well and good

[Anonymous Coward]

Actually, I wanted to kicked myself severely when I re-read my post. 'rediculous' as a spelling has been a /. peeve of mine since the first time I saw it. I can usually take pride in my spelling of words like to/too, than/then, etc. But I keep screwing those up too.

Preview, preview, preview....

-robSlimo

Re:well and good

[Pharmboy]

The price differential between CRT and LCD monitors is still enough that most larger businesses are still only buying CRTs for most of their users. Sure the executives and receptionists are getting LCDs but everyone else gets cheap 17" CRT monitors.

We just did some calculating and figured we could use 15" LCDs instead of 17" CRTs and lose a little in size, and cost a little more up front, but save enough in less than 2 years to pay for the cost difference, in ELECTRICITY. Our graphics stations will likely stay with 19 to 21" CRTs but we will be phasing in LCDs in large part because the TCO is good, the space consumed is better, the employee moral benefits is great, thus the overall value is exceptional.

Re:well and good

[DataPortWizard]

It is not a small application that will drive the migration to electronic storage media, but rather large disk intensive applications that make many small disk requests (such as a large database). The advantage of electronic media will be the reduced access time that allows more I/O operations per second (IOPS). Currently, a fast SCSI RAID array, with an access time of 5ms, can push about 120 IOPS. What will drive the move to solid state storage is not the cost per megabyte, but the relative cost per transaction. Twelve years ago I patented three technologies (hardware, drivers and file system) that have an access time of about 6us and theoretically put IOPS up to well over 100,000 on today's pentium class machines. Tomorrow's machines will be even faster as it is machine front-side bus speeds that determine throughput using my device. (For example, a 64-bit, 400Mhz motherboard provides the potential for reaching over 512,000 IOPS.) Only recently, the cost of memory has come down enough to make the move to solid state storage viable based upon that proposition. Funding is a big problem these days, but I expect to be in business building these machines within a few years. BTW: I call the device a DataPort.

Yea yea...

[Anonymous Coward]

Something magical has been ready to replace my filesystem for years now. Everything is a revolution in the way computers work, AND NOTHING EVER ACTUALLY COMES OUT.

When was the last "next big thing" that changed the way *all* your machines work? 32 bit maybe?

Drawback

[ifreakshow]

One quick draw back I see in this system is on a computer where you have more small files than available RAM space. How does the system decide which small files to keep on the regular disk and which ones to keep in RAM?

Re:Drawback

[oconnorcjo]

How does the system decide which small files to keep on the regular disk and which ones to keep in RAM?

That is an algorithm issue but the worst that happens as far as I can see is that when the ram borks a little, it just means the throuput goes down while it writes more stuff to disk. I highly doubt that will be a big issue since algorithm's for determining priority [they could almost be plucked from a "VM" HOWTO] are abundent and if they are not all perfect- many are good enouph.

Re:Drawback

[AaronMB]

Since the metadata is stored in RAM, it'll have access to the atime,ctime and mtime variables quickly. Thus, it'd be pretty easy to do an LRU scheme to dump rarely-used files to disk periodically.
-Aaron

Re:Drawback

[ottffssent]

LRU eviction is somewhat costly, but highly effective. Pseudo-LRU can be much cheaper and nearly as effective. The replacement policy is not hard - it is a well-researched problem in cache design.

What I find telling is that such a system has to be implemented at all. It seems clear to me that the operating system's filesystem, in conjunction with the VM, should implement this automatically. In Linux, this is true - large portions of the filesystem get cached if you have gobs of RAM lying around. Why certain more commonly-used OSes do the exact opposite is beyond me.

From my perspective, the right way to handle this is obvious. RAM is there to be used. Just as we have multiprogramming to make more efficient use of CPU and disk resources, we should be making the best possible use of available RAM. Letting it sit idle on the odd chance the user will suddenly need hundreds of meg of RAM out of nowhere is rediculous. From the perspective of the CPU, RAM is dog slow, but from the perspective of the disk, it's blazing fast. ANYTHING that can be done to shift the burden from magnetic storage to RAM should be done. Magnetic storage excels in one area and one area only: cheap permanent storage of vast amounts of data. RAM should be used to cache oft-used data. Why is this not painfully obvious to anyone designing an operating system?

Re:Drawback

[lib]

One easy way to implement exact LRU in software.... (doubly) linked list.

Each item points to its node in the linked list. On each access to an item move its node to the head of the list. When you need to evict, kick out the tail element. Alot of software cache simulators use this for exact LRU

This way you can maintain LRU by only changing a few pointers in a nice generic manner.

Also, since this would be fully associative with ALOT of associativity, the things that get evicted are the files that you nearly NEVER use.

Re:Drawback

[Anonymous Coward]

Why is this not painfully obvious to anyone designing an operating system? It is painfully obvious to those folks, and it has been for years. That's right, Mr. Insightful, you've summarized a well-known and decades-old idea from operating systems research and the 16 year old hax0r monkeys here have modded you up for your originality. Hey, why go and learn about what's been done before when you can invent it yourself instead and think you're clever for it? Gotta love slashdot.

Re:Drawback

[Anonymous Coward]

Why is this not painfully obvious to anyone designing an operating system?

It always has been.... to the Amiga. From the good ol' days, all the way to the upcoming AmigaOS 4.0 which continues to actually handle memory properly (and minimally). :)

Re:Drawback

[Anonymous Coward]

Just because it's an idea which is already known doesn't mean it's not +5 Interesting.

Re:Drawback

[Have Blue]

If you have far more RAM than you are using for your processes, you could have saved some money. Also remember that quite often you are using a good bit more RAM than you have, due to virtual memory.

It would be difficult to use RAM to cache disk writes, since dirty areas would have to be flushed when a process wanted more memory.

Re:Drawback

[DarienJax]

From what I understand, it doesn't handle this at all. Small files are simply determined by a size -- currently 1MB. If the file is smaller than that, it goes in RAM; bigger, it goes on disk. A size and amount of RAM need to be chosen such that all small files will fit in RAM. One of the things that was emphasized at the talk was that the on-disk filesystem code is greatly simplified because they don't have to worry about small files. They can use large chunk sizes, they don't have to worry so much about fragmentation, etc, specifically because they don't need to deal with small files on disk. Having to write small files to disk, or using an algorithm to determine un-used small files were asked about at the talk, and the presentor basically said "Right now, we don't want to deal with that because of the simplified code for only large files on disk"

Re:Drawback

[Anonymous Coward]

What's next on the curriculum, professor? Please, explain how to make a program run in a "loop" using a "test", a "branch", and something called an "index variable"!

Re:Drawback

[Gromer]

I think I was at the same talk as the poster.

In point of fact, Conquest does not use LRU. Conquest uses a very simple rule- files larger than a threshold are stored on disk, and files smaller than a threshold are stored in RAM. The threshold is currently a compiled-in constant (1 MB), but plans are for it eventually to be dynamic.

The advantage of this approach is that it eliminates the many layers of indirection needed to implement LRU-type caching, which is one reason Conquest consistently outperforms FS's based on LRU cacheing.

Re:Drawback

[ratboy666]

The problem is file system meta-data. Competition is with journaling fs's! Meta-data must be updated, and if it is to disc, the operation is slow. It is not clear to me at all that RAM caching as you describe is an answer. At some point the data must be put to permanent storage. And this is where the bottleneck comes in. Unless
you want all your work to be eaten on a power failure.

Re:Drawback

[Ed Avis]

You could also eliminate LRU-caching overhead another way. Keep the files on disk in size order, with smaller files towards the start of the disk. Then instead of having a complex LRU block cache, just cache the bottom N megabytes of disk space in memory (and if you have battery-backed memory available, do long-term write caching as well as read caching). The trouble is how do you shuffle files around when their sizes change...

Re:Drawback

[Anonymous Coward]

What I find telling is that such a system has to be implemented at all. It seems clear to me that the operating system's filesystem, in conjunction with the VM, should implement this automatically.

Well, a point I haven't seen anyone mention is resiliance to failures.

Sure you could cache all the metadata and write all the data, but if the power fails you're SOL. Likewise on a kernel panic.

Think about it. Running sync should put the data to hardware, and so should finishing a database transaction. Caching doesn't help here.

OTOH if you have battery-backed ram (with a full battery), sync can throw all the data to spinning disk and the metadata onto battery-ram, and you're ready for a power failure. Faster and just about as reliable, and transparent to the OS.

You can fake it with a file server and a UPS, and set the server up to cache writes very aggresively, but to throw everything on disk when the UPS signals that shit is happenning. Of course then you're limited by LAN bandwith, and you have to keep that server simple to avoid any other program crashing it.

Anyway, just wanted to point out that out

Re:Drawback

[Fulcrum of Evil]

If you have far more RAM than you are using for your processes, you could have saved some money. Also remember that quite often you are using a good bit more RAM than you have, due to virtual memory.

Not true. If you knock your processor down a notch and buy gobs of RAM (which you can), then you can get better performance for the same money.

It would be difficult to use RAM to cache disk writes, since dirty areas would have to be flushed when a process wanted more memory.

Well, yeah. Linux already does this, though I don't know if they even cache writes (much). In point of fact, linux caches a hell of a lot, and it isn't all easy. In practice, it works out rather well.

Re:Drawback

[Zaak]

In point of fact, Conquest does not use LRU. Conquest uses a very simple rule- files larger than a threshold are stored on disk, and files smaller than a threshold are stored in RAM.

That doesn't answer the question of what to do when you have more small files than will fit in your RAM. Obviously, if you have infinite RAM you don't need a disk at all, so what is the system's behavior when it runs out? It seems to me that it either uses LRU (or a similar scheme), or its performance degrades to that of a regular file system.

TTFN

Re:Drawback

[Anonymous Coward]

You dont need to flame the guy. Some people who read this dont program, others aren't familiar with LRU.

Re:Drawback

[be-fan]

It would be difficult to use RAM to cache disk writes, since dirty areas would have to be flushed when a process wanted more memory.
>>>>>>>>>>
Um, it already does this. Linux will merrily cache pretty much everything in the filesystem until something actually needs the memory. If I'm sitting there just downloading something in the background, and reading something on the net, a big several hundred meg download will lead to very little disk activity, with the occasional write being done to update the filesystem journal as the filesize increases. Then, if I start up my compiler, the system's memory demand will suddenly increase, and I'll hear the drive writing out that entire file in one go.

Re:Drawback

[Anonymous Coward]

I've seen the talk, and the answer is that there is no solution to that problem. The system simply stops working.

Re:Drawback

[Beliskner]

You can fake it with a file server and a UPS, and set the server up to cache writes very aggresively, but to throw everything on disk when the UPS signals that shit is happenning. Of course then you're limited by LAN bandwith, and you have to keep that server simple to avoid any other program crashing it.

You mean like this 1 TB RAMServer with Battery Backup?

wow, if it only could be cost effective...

[Falconpro10k]

wow... thats all i have to say, something like this could make waiting over a min or two to boot totally obselete... sort of like a "turn on" welcome to your OS of choise type of thing... i also tons of other possibilities such as high end graphics work and maybe even phasing out the disk as we know it 100%.... all solid state.. the possibilities are endless

Re:wow, if it only could be cost effective...

[Blaine Hilton]

Like loading a device with a hardened linux OS that is only stored on flash ROM.

Go calculate something.

Re:wow, if it only could be cost effective...

[Anonymous Coward]

Or, like a good spelling and grammar checking device. (not aimed at you, Hilton, but the one to whom you replied)

Re:wow, if it only could be cost effective...

[NanoGator]

"wow... thats all i have to say, something like this could make waiting over a min or two to boot totally obselete... sort of like a "turn on" welcome to your OS of choise type of thing.."

Kinda like Windows CE!

Re:wow, if it only could be cost effective...

[Ari Rahikkala]

Um... I don't know so much about Conquest so take this with a grain of salt... but I'd expect that it takes more time to mount a block device using the Conquest FS than some "other" filesystem because you don't have all those small files and metadata in RAM yet. Meaning you have to do quite a lot of reads (or a couple of big reads, or a lot of small reads that are executed as a couple of big reads anyway if you're using a good IO scheduler) to set it up. This means that it could actually slow down the boot-up of an OS.

Re:wow, if it only could be cost effective...

[Anonvmous Coward]

"wow... thats all i have to say, something like this could make waiting over a min or two to boot totally obselete... sort of like a "turn on" welcome to your OS of choise type of thing.."

"Kinda like Windows CE!"

Flamebait? What he described is exactly what my PocketPC, and before it, my Palm Pilot does.

Re:wow, if it only could be cost effective...

[Anonymous Coward]

This is slashdot. The comment was pro microsoft. Logic dictates it could only be flamebait, and *you* are a terrorist.

Who are they kidding?

[asdfasdfasdfasdf]

The idea of RAM as storage is great and all, but can we work towards the elimination of STORAGE as RAM before we get to RAM as storage?

I mean, why *DO* we still have pagefiles?

A MS Gripe: I seriously don't understand why I can't turn it off completely. With multiple GB of RAM dirt cheap, writing to a disk pagefile slows my system down-- It has to!

Re:Who are they kidding?

[DigitalGlass]

what version of windows are you running? I have had no problem with turning off the pagefile in 2000 and xp, my machines have 1gb in them and they cranked when i disabled the pagefile.

should be in control panel - system - advanced - performance --- look in there for something to set the page file to 0 or to disable it.

Re:Who are they kidding?

[pla]

Heh... I've "Ask Slashdot"'ted this one myself (rejected, of course).

Personally, I agree with you 100%. Get rid if the idea of "pagefiles" and "swap partitions" completely, and enjoy the performance boost resulting from the "loss".

As for the idea at hand (to make this at least vaguely on-topic)...

How does this differ from a large RAM disk cache, with slightly tweaked heuristics for keeping something in cache (size, rather than most-recently-read)?

Re:Who are they kidding?

[Neck_of_the_Woods]

Correct me if I am wrong, but the only time that it starts to write to your page file is the moment that the amount of data passes the amount of RAM you have. At which point it starts writing information from the ass end of your RAM to the pagefile. While this is not as fast as your RAM data pull it is faster than finding it on the disk, and parsing it back into usable info. It knows where it is already and drops it back to the front of your RAM storage.

This is just my limited view of what I kind of figured happened. Please correct me if I am wrong but if you turned off your page file you would be slowing yourself down a bit.

Re:Who are they kidding?

[hurtta]

Who says that you need to have pagefile? (Or is that A MS Gripe refering some certain Operating System?)

Re:Who are they kidding?

[gl4ss]

with some versions of windows it will swap to pagefile even before the ram fills up (recent too), sometimes putting there stuff that will be needed very shortly too, theres some programs that try to battle this though(i had success with one with win2k back when i still had only 128mb, supposedly it prohibited windows from swapping some constantly needed system resources to disk, and worked mostly this way, i don't have a lot of faith in 'memory managing' programs that will just do a malloc(some_big_number_of_bytes) every now and then and free that straight after supposedly resulting in most useless stuff getting thrown into the swap and leaving the ram free for what you're going to run next)

Re:Who are they kidding?

[Anonymous Coward]

2000 and XP have problems with certain applications when the pagefile is disabled.

Re:Who are they kidding?

[Eneff]

Very simple... slow ram isn't what you want in your system memory, but it's cheap enough to justify putting a smaller database on.

It's being continually backed up to hard disk, but in the background.

Re:Who are they kidding?

[pclminion]

I mean, why *DO* we still have pagefiles?

Well, a couple of reasons. Most important, the "pagefile" is there to protect against a hard out-of-memory condition. Modern operating systems are in the habit of overcommitting memory, which means they grant allocation requests even if the available RAM can't fulfill them. The idea is that an app will never actually be using all those pages simultaneously. If things go wrong and all that extra memory is actually needed, the system starts kicking pages to disk to satisfy the cascade of page faults. This means the system will become slow and unresponsive, but it will keep running. But say you didn't have anywhere to swap to. The system can't map a page when a process faults on it, and the process gets killed. But which process gets killed? After all, is it the process's fault if the OS decided to overcommit system memory? The swap space serves as a buffer so a real administrator with human intelligence can come in and kill off the right processes to get the system back in shape.

Swap is also important because not all data can just be reloaded from the filesystem on demand. Working data built in a process's memory is dynamic and can't just be "reloaded." If there's no swap, that means this memory must be locked in RAM, even if the process in question has been sleeping for days! We all know the benefits of disk caching on performance. Process data pages are higher priority than cache pages. Thus if old, inactive data pages are wasting space in RAM, those are pages that could have been used to provide a larger disk cache.

You basically always want swap.

Re:Who are they kidding?

[hamsterboy]

Swap files are still very useful, and to discount them as useless is foolish.

Example: the memory pagefile. The OS needs to know which program or user owns each block of memory. Say your OS supports up to 4GB of RAM, and you only have 512 MB. If you can't offload some of this pagefile to disk, you're using RAM to store ownership data about the 3.5GB of RAM that doesn't exist.

This becomes even more important with 64-bit address spaces; your memory pagefile becomes larger than 1GB. Without the ability to page to disk, you're using all your RAM to store data about your RAM.

Hamster

Re:Who are they kidding?

[Anonymous Coward]

i can turn it of that way, but i get a warning everytime i start windows telling me to turn it on. do you know how to get rid of that?

Re:Who are they kidding?

[Anonymous Coward]

This wouldn't be a problem if Microsoft's (and well Apple's just to name a few) paging/swapfile subsystem wasn't so inefficient.

On modern mainframes it's almost an art form.

Re:Who are they kidding?

[ceswiedler]

First, pagefiles (or swap partitions) are usually of fixed size. So their existance doesn't necessarily mean they're being used.

Second, RAM is used for disk cache as well as application memory space. 'Unused' RAM is really being used to cache your slow hard drive, which is a good thing.

Third, stuff is paged in and out when necessary to free up space for other stuff. It's swapped at the memory page level (4k) based on when data is used. So if you leave your system running for a while, the stuff that gets written to disk is stuff that hasn't been used.

The result is that the OS uses the physical RAM in a very efficient way. Your pagefile speeds up your system (assuming it's well written, and both NT and Linux do very well in this regard) and you really don't want to get rid of it.

Re:Who are they kidding?

[Surak]

So long as data + applications physical RAM you won't have a problem. I often have to deal with files that are several hundred megabytes in size and grow once in RAM. Your setup wouldn't work for at me.

That's why we still have swapfiles.

Re:Who are they kidding?

[Anonymous Coward]

Does not quite work like that, you can set your pagefile size to 0, but Windows 2000 wil create a small pagefile about 20 MB and still use it !

Re:Who are they kidding?

[Anonymous Coward]

in 2k, nope... you'll also notice that the pagefile cannot be disabled in windows 2000(unless there's a reg hack), you can set the pagefile to 0, but windows will dynamically create a pagefile at boot regardless.

it just isn't in the usual location, so you think it's gone... do a search for large files, say over 10,000 KB and you'll find it. I'd need access to a 2k machine to tell you exactly where it puts it, but I'm fairly sure it's either in %systemroot% or %systemroot%\system32.

Re:Who are they kidding?

[theLOUDroom]

I mean, why *DO* we still have pagefiles?

Because if we didn't our computers would crash when we ran out of RAM. Seriouly, a lot of programs don't cope well with being told: there's no more RAM left on the system, you can't create that object.

Swap is a good thing. Right now my PC has 512MB of RAM this is more than I need for just about anything I do. Right now my RAM is 31% free and my 2GB swap space is 78% free. Why the hell should so much be in the swap space when I have free RAM? Because that information is old and rarely accessed. Say I write a program that just creates a 100MB data object and then sits there for days waiting for something to happen. Eventually all that data will end up being stored in swap space. By keeping it in swap I have lots of RAM free for any programs that need memory space which they'll actually use all the time.

The problem isn't as simple as: "swap space is slow so it must be bad." Swap being slow costs you speed when you're using it, but it also gives you speed, by letting you use your RAM for things like disk caching. You have to look at the cost vs the benfit of having swap, not just the cost.

Re:Who are they kidding?

[TopShelf]

Correct me if I'm wrong, but I believe Windows, be default, allows the size of the page file to fluctuate as needed. One of the performance tips I saw recently at MaximumPC was to set this to a constant to avoid unnecessary HD work...

Re:Who are they kidding?

[deblau]

but can we work towards the elimination of STORAGE as RAM before we get to RAM as storage?

I mean, why *DO* we still have pagefiles?

You've obviously never done any real computing. There are some computational problems that are still too big to do entirely in RAM, hence pagefiles.

Actually, Beowulf clusters are spot on topic here. Ever wonder why the concept of a Beowulf cluster was such a breakthrough? (Hint: have you ever tried to crunch 100GB of data from a heat transfer problem?) Beowulfs allowed the huge amount of necessary RAM to be farmed out to lots of boxes, driving down costs, among other advantages.

Re:Who are they kidding?

[Anonymous Coward]

Hahaha, thats priceless. Windows doesn't so much as ignore your settings, it sees them, decides it doesn't like them and does what it wants anyway. But the designers were at least smart enough to try and hide this little feature from the user.

Re:Who are they kidding?

[Anonymous Coward]

You've obviously never done any real computing...

You obviously have no social skills...

Re:Who are they kidding?

[dlakelan]

When the power goes out you don't lose anything, even if it goes out for several weeks.

plus, rather than basing its heuristics on recently used data, it is actually designing around having RAM to store files (as opposed to RAM to store disk blocks)

This guarantees that entire files are in RAM and are rapidly available rather than just a few blocks of many different large files.

Plus it stores the metadata in RAM, so the filesystem on disk is much simpler (and thus more robust). If it's smart, it backs up the RAM data onto disk, but only during quiescent periods.

I want it. If I could get 512 MB of battery backed RAM on a PCI card and a robust version of this filesystem in Linux I'd switch rather quickly.

Re:Who are they kidding?

[shepd]

>A MS Gripe: I seriously don't understand why I can't turn it off completely. With multiple GB of RAM dirt cheap, writing to a disk pagefile slows my system down-- It has to!

This has been an option since Windows 3.0 (possibly earlier, I've never experienced anything before that).

In windows XP, the swap can be disabled as such:

Start->Control Panel->System->Advanced->Performance (Settings)->Advanced->Virtual Memory (Change)->No Paging File->Set (etc, etc)

Unlike linux, windows apps normally throw a nice error when they're out of memory, rather than "Segmentation Fault" (Uggggh...)

Re:Who are they kidding?

[TheRaven64]

A MS Gripe: I seriously don't understand why I can't turn it off completely. With multiple GB of RAM dirt cheap, writing to a disk pagefile slows my system down-- It has to!

This has to do with the way in which Windows NT handles disk cache and paging. Windows treats your swap file as your main RAM, and your RAM as a disk cache, so it can unify the swapping and caching alogrithms. If you disable swap, then it will act as if you have no RAM (with the exception of the unpagable RAM reserved for the kernel). This can be particularly irritating if you have a lot of disk throughput, since your RAM will be paged in preference to some of the disk cache, resulting in hundreds of page faults when you context switch to an application. To see how badly your system is affected by this enable the PD Delta column in the task manager and watch it when you switch to an app you haven't used for a while.

I believe that this is no longer the case with XP, but it is with 2000.

Re:Who are they kidding?

[Anonymous Coward]

hehe, if you think that's bad, read down to the MCSE that has decided a ramdrive is the best place to put his pagefile....

Re:Who are they kidding?

[xenocide2]

It gets a little bit tricky, since disk writes take so long, and consequently, page writes take a while. What you can do is actually find a segment of RAM thats pretty old and hasn't been written to recently, and copy it to disk ahead of time. The overhead is minimal if you're not limited on disk throughput. It helps keep disk channel utilizations up, while reducing peak utilization and latency. I don't claim that this is what windows does, but it has been tried...

Re:Who are they kidding?

[Anonymous Coward]

But why does windows start up swap automatically instead of waiting until or unless it's needed like linux does. This is just another example of piss poor windows design.

Re:Who are they kidding?

[johnburton]

No. It speeds it up. If nothing else it means that the memory can be used a disk cache to speed up the slowest part of your system instead of being wasted storing program code that may never be used at all.

Re:Who are they kidding?

[ZenShadow]

If people would design software that doesn't do things the most inefficient way possible, maybe that wouldn't be the case....

It's amazing how much you can do with 1MB or so of memory. It really is. But you wouldn't know it by the way today's corporate code monkey uses it.

Re:Who are they kidding?

[caluml]

Because if we didn't our computers would crash when we ran out of RAM.

Why is that then?
Assuming the OS has enough ram for itself, the other apps should act as though limits are artificially enforced by ulimit. Try ulimit -H -m 1 (if I remember rightly)

Re:Who are they kidding?

[GiMP]

Older versions of Windows did have that as default, I don't know about recent versions as I've no experience with them.

It is a very well-known performance boaster to manually specify a fixed amount of swap for Windows.

Re:Who are they kidding?

[Anonymous Coward]

Most important, the "pagefile" is there to protect against a hard out-of-memory condition. Modern operating systems are in the habit of overcommitting memory, which means they grant allocation requests even if the available RAM can't fulfill them.

Nope. Pagefile is there to enable system managed physical memory purging in sense of helping application developers avoid writing such garbage collectors themselves over and over again. Mentally there isn't much difference if you have 1Gb RAM + 1Gb swap comapred to 2Gb RAM. It gives you roughly 2Gb data storage. Maybe in the first case the RAM alone can't satisfy request for 1.5Gb block allocation, but if on such any of these configurations I try to request 2.5Gb block allocation...

The system can't map a page when a process faults on it, and the process gets killed. But which process gets killed? After all, is it the process's fault if the OS decided to overcommit system memory? The swap space serves as a buffer so a real administrator with human intelligence can come in and kill off the right processes to get the system back in shape.

... and for this reason alone the system actually preallocates swap space at the time of allocation request. Under *NIX systems this is usually bogus, as they mostly swap on fixed-size devices, however on Windows NT, you end up with pagefile that is constantly growing and shrinking (let's leave it to others to decide whether this is any good or bad). Pagin-in and page-out process can occur only on the pages that have associated swap page. When it comes to memory management it simply means that this is paging mumbo-jumbo is just implementation detail that modern CPUs provide infrastructure for. The simple consequence of this is that system might truly allocate more RAM that it has, and thus compensate with swap media, but it will never overallocate combined RAM + swap storage. If I try to allocate 2.5gb in above example, I will simply trigger some exception ultimately resulting in not having the request honored.

Swap is also important because not all data can just be reloaded from the filesystem on demand. Working data built in a process's memory is dynamic and can't just be "reloaded." If there's no swap, that means this memory must be locked in RAM, even if the process in question has been sleeping for days!

Actually it can be. VM subsystems when looked upon from purely application point of view don't do much than negate need to write your own memory conservation routines. Conceptualy you could completely reimplement this particular functionality (but not the disk I/O, scheduling, and couple of other nifty things that VM subsystem does as an extra) in any way you want. In C++ you could probably use smart-pointer class that would dump data block to swap when not needed any more and read it back when accessed properly. The only benefit of the system doing it is easing the lives of programmers. It isn't much different when it comes to the code pages. Implementation is only a little different, but basically you can have certain benefits of VM memory management without actually having any VM, but I digress... Check for details at the RT front. As far as the memory management goes, some applications actually implement it by themselves by directly using system facilites for locking, unlocking, and paging, but not system provided algorithms to decide when to do what. Operating system VM is jack of all trades as it has to accomodate for different (sometimes conflicting) demands users and applications have on the system. Sometimes the applications simply has to force it's own will on this subjects, as it intimately "knows" which is optimal way to manage its own memory.

Overall I claim that if you have 1Gb RAM and you know that combined allocations of all the applications hardly ever reach over 700MB (as it is case with my workstation for example), you can disable swap management and spare a couple of cycles each second that would gome in some smart algo. deciding whether to swap pa

Re:Who are they kidding?

[Penguin Follower]

Correct me if I am wrong, but the only time that it starts to write to your page file is the moment that the amount of data passes the amount of RAM you have.

That would be the logical conclusion... However, from my experience on Windows 2000 that is not what I have seen. I have 512MB in this system. I highly doubt that with a web broswer, email client, and a couple instant messangers I am using all 512MB, yet Win2k insists on using approx 768mb swap. For what?! I have this machine set up as a dual boot, and besides Win2k, it also runs Red Hat 9. I have had evolution, mozilla (with several tabs open), gaim, licq, a couple konsoles, and usually quanta is open as well. (and I use KDE as my desktop). With that said... Linux doesn't even TOUCH the swap partition (if I'm running KsCD to play an audio disc, there is some usage of swap to buffer playback, but very little used still). And most of the physical ram is being used to cache the disk (instead of the other way around). I keep the memory info open so I can watch it now and then and it just amazes me how little swap my machine uses under linux, yet Windows with that many apps open is using 3/4 a gig for swap! I know there was a big deal about the VM in the kernel during the early 2.4.x days, but as far as I can tell, that's long been fixed, and puts windows to shame now.

Re:Who are they kidding?

[operagost]

Only an extremely underutilized system does not need a page file. You can fill up 1 GB of RAM easier than you think. All it takes is opening a very high resolution image file- the file must fit into RAM in uncompressed form. Even systems that generally have enough RAM still need pagefiles because someday, they won't- and it's better than crashing.

Pagefiles are also useful for reserving disk space for memory dumps. Unless your system never crashes, you'll need that, too.

Re:Who are they kidding?

[ckaminski]

I'd really like to know how NT can do paging, when there's no pagefile? NT (2000|XP) will always attempt to keep the biggest disk cache that it possibly can, even when swapping is turned on, and yes, other programs can suffer for it. However this buffer is always resized depending on system utilization and NEVER goes below a certain size (2MB? 8, I forget the exact number).

Re:Who are they kidding?

[operagost]

No modern OS works like this. They start paging when a process has been idle for X amount of time. Some (like OS/2- and I did hate the feature) page things into RAM during the boot. Paging during idle time is a good thing because it gets the job done quietly and leaves extra RAM free instead of thrashing the hard disk while you're waiting for a process to execute.

Re:Who are they kidding?

[be-fan]

There is a difference between "should" and "does." In theory, we'd all check for "malloc()" returning zero (or "operator new" throwing an exception) but we don't. The VM system allows programs to pretty much ignore memory limits, and like it or not, a lot of programs (maybe most programs) don't rigorously check to see if allocations actually succeed.

Re:Who are they kidding?

[hfranz]

In my experience at least Windows NT and 2000 swap out pages of applications when they get minimized.
Why Microsoft mixes the functionality of paging algorithms and window management is beyond me though.

The next boost will be

[scorp1us]

Execute in Place (EIP)- currently, your system will copy the program to RAM. Here, you'd copy everything from volatile ram to Non-volatile ram - a rather wasteful operation don't you think?

This is not just for exe's but for datafiles as well...

Re:The next boost will be

[jetmarc]

> Execute in Place (EIP)

Ha! No problem. Just make the file system "sector size" equal to the MMU page
size, and you can dynamically map the executable file into the machine address
space.

Re:The next boost will be

[hurtta]

Execute in Place (EIP)- currently, your system will copy the program to RAM. Here, you'd copy everything from volatile ram to Non-volatile ram - a rather wasteful operation don't you think?

And then "RAM" on execution is replaced with just one more cache (after all programs are not executed from RAM directly, but instead there is usually two different speed cache (ie faster ram) between CPU (or on CPU) and main memory.

Re:The next boost will be

[hey]

Doesn't Linux almost do this?
It nmap()s executables before running them.

Re:The next boost will be

[scorp1us]

I never said it was hard. Getting changes to made to the kernel is the hard part though, particularly if you are proprietary...

For exes, you have to setup a stack and heap for the program, so there's still a _little_ work to be done, but it would rule. You could boot Windows in under 10 minutes this way!

Re:The next boost will be

[scorp1us]

Yes, Linux is very close to this, but it can be done better still if the filesystem is known to be of this sort.

Re:The next boost will be

[ceswiedler]

No modern operating system "copies" the entire executable into memory when it's run. It's paged in from disk as necessary. If it's already in the cache, then it basically is run from RAM anyway.

The concept of a paging VM system is so elegant that when it's done right, it makes most other optimizations (such as your EIP suggestion) unnecessary.

Re:The next boost will be

[Anonymous Coward]

TRS-80 model 100 of 1980s vintage already does this. Software stored in its RAMFS are executed in place. SMC works too.

Albeit 32K of RAM is kinda limited and all the disk space is mmapped (which it must be)...

Re:The next boost will be

[Anonymous Coward]

Heh, this was called XIP (same acronym expansion) when I worked on PCMCIA cards.

The big problem was that the PCMCIA interface appears at a fixed address in the system, and position on the disk changes the location again, so any executable code files which were not PIC (position-independent code) had to be run through a relocator which wrote the appropriate absolute addresses into the executable's code segments. Needless to say, this was not a portable way of storing executable files.

Thankfully, these days we have MMUs to let us pretend every executable is loaded at offset $00000000 in the memory map. Makes life a lot easier...

Re:The next boost will be

[Anonymous Coward]

Sorry but this is the most ridiculous thing I've read in a long time. It gives insight into wth people at microsoft are thinking when they come up with their program designs and that's bad.

Every executable is loaded to memory and THEN paged, it's a double hit when something is paged, paging optimizes NOTHING. Paging slows a system down, the last thing you want is a system that pages everything like certain MS os's. On a system with half a gig of ram paging should be damn near non-existant in 90% of cases. When executed programs ARE copied to memory, this is the point at which they should not be copied to a disk swap but kept in memory, no paging unless the earth has becomed scorched and full of dreadful famine.

It's not neccesarily loaded into memory in one piece, though it ususally is since a sequential access is faster, and some of those pieces in so called modern OS's are paged before the others are loaded, but most assuredly those pieces are copied to memory before written back to disk in the swap.

Re:The next boost will be

[ZenShadow]

That would be mmap(). nmap is a command line network toy.

--ZS

Re:The next boost will be

[b0r1s]

The inventer of Beowulf is currently working on something called "Processor in memory", the idea being that you embed a number of smaller, slower processors within the memory to speed up the smaller, easier calculations, and send the slower, longer calculations to the main processor.

For instance, if you were searching through a huge 1000000x1000000 matrix for a single entry to hash, you don't want to have to move each and every entry to the processor to decide whether it's the right one: offload the searching work to the processor in memory, and then once the right entry is found, send that to the main processor.

It's a rather novel idea, but it seems that it'll take a few years before anyone even tries to implement it.

Re:The next boost will be

[ceswiedler]

Are you sure? I don't know much about NT's paging system directly, but rather by way of Linux and other 'modern OS' designs. As I understand it, an executable file's segments (code,data,bss) are simply mapped into the process's address space at load time, and only when the process accesses a page will it be faulted in.

The whole point of a paging system with unified disk-cache buffers / memory pages is to access memory completely virtually, and let page faulting and flushing read/write to disk as necessary. And in the vast majority of cases (of course there are always exceptions) that 'as necessary' clause is pretty accurate, and speeds up your system.

Re:The next boost will be

[Anonymous Coward]

It nmap()s executables before running them.
Can you imagine proper file nmap on 64 bit machines where you can address all the disk space? This is a heaven for developers...

Re:The next boost will be

[hey]

Thanks for the correction.
My oops.

Re:The next boost will be

[Wiz]

What? You mean like this?

Sounds exactly like you are describing!

Re:The next boost will be

[ckaminski]

The last thing I want is for my OS to get a little too amorous with my filesystems... mmap() is perfect as far as I'm concerned... What kinda hell could be wrought if Linux, for example, was allowed to just bring in random pages from say, a ReiserFS partition?

Re:The next boost will be

[Suidae]

Interesting idea, and probably not that hard to integrate into modern OO programming languages. The code for the search operation could be built into a class for handling the data. Different kinds of data objects could override the search code if necessary.

Re:The next boost will be

[b0r1s]

It is exactly what I was describing.

Apparently my wild-ass-guess on the time until implementation was a few years off.

Thanks for the link.

Re:The next boost will be

[lostchicken]

This is similar to the way mainframes work. Every device has pre and post data processors to prep data for the main CPUs. Sometimes, the data never even goes through the CPU, that's why, even without fast CPUs, S/390s are still really fast for I/O stuff.

Re:The next boost will be

[Anonymous Coward]

Every Unix does this...in fact other systems did it before Linux, and arguably more cleanly (since they had filesystem block sizes bigger than the size of a memory page, which allowed the buffer containing the page to naturally be shared with the mapped instance).

Dead? Hardly...

[KC7GR]

This is 'stepping-stone' technology, along the same lines as hybrid gasoline/electric vehicles. They're still depending on hard drives for mass data storage. It's just the executables, libraries, and other application-type goodies that they're sticking into RAM.

You can do exactly the same thing by sticking an operating program into any sort of non-volatile storage (EPROM, EEPROM, memory card, whatever), and including a hard drive in the same device if need be. The new filesystem they're describing simply shifts more of the load to the silicon side instead of the electromechanical realm.

In short; The Disk is far from dead. This is just a first step in that direction.

Re:Dead? Hardly...

[oconnorcjo]

In short; The Disk is far from dead. This is just a first step in that direction.

As far as I am aware, mainframe and other highend systems had hard drives like this since even before the eighties. It is just becoming standard in low-end HD's. Now I can buy a HD with several meg buffer for about 3(?) years. The only thing new in this article is a filesystem that (possibly) helps the hardware/software utilize this buffer better.

Re:Dead? Hardly...

[MisterFancypants]

This is also a very old idea. I (like many others) used to do the same thing with a RAM Disk on my Amiga.

Re:Dead? Hardly...

[pingbak]

Actually, you're incorrect. It's not the specific types of files that get stuck into RAM (that's a side effect.) It's small files under 1M that get stuck in RAM. Large files pretty much stay out on disk.

BTW: The paper's author sits next to me in my office here at UCLA. So I might know a thing or two about this research.

Old news.

[Nathan Ramella]

These guys have already done it..

http://www.superssd.com/products/tera-ramsan/

Up to a terabyte even.

-n

Re:Old news.

[dildatron]

you are correct. many disk array vendors have been doing this for years. they have huge amounts of ram cache that is backed up by largish batteries, with redundancy of course. This is really nothing new. Even a low or mid-end disk array that scales perhaps up to around 10 terabytes has this involved.

Re:Old news.

[cube_mudd]

Actually, what they've done is novel.

If you read the article, the point of Conquest is to remove the filesystem complexities that pander to disks. Giant RAM based storage arrays do nothing to simplify and streamline the filesystem code in your kernel.

Re:Old news.

[Duck_Taffy]

That RAM-SAN is remarkably inefficient, compared to HDDs. For a comparable hard drive-based SAN, you'd think the manufacturer was insane if they said it required 5,000 watts of electricity to operate. I know it's fast, but I don't want a dedicated 60-amp circuit just for a single storage device. And I can hardly imagine the heat it produces.

Re:Old news.

[Anonymous Coward]

it required 5,000 watts of electricity to operate. I know it's fast, but I don't want a dedicated 60-amp circuit just for a single storage device.

You're 30 amp, 240 volt for your dryer could deliver 5kW.

But if you spent that much for 10 TB of SDRAM, you probably would want to install a dedicated circuit!

Re:Old news.

[Anonymous Coward]

very old news. see this google groups thread

Re:Old news.

[Bytenik]

I don't believe that there is "a comparable hard drive-based SAN". If there is, I want to know about it!

They are different types of devices and have different power requirements. If you have the type of application that requires RAM-SAN levels of IO performance, getting the power for it is unlikely to be your biggest expense.

We met with a rep for the RAM-SAN last month, and will probably be getting a trial unit in a couple of months.

Like you said though, the heat must be incredible. Break out the marshmallows.

Re:speaking of ramfs...

[Anonymous Coward]

From the Documentation folder, file ramdisk.txt:

ramdisk_size=N
==============

This parameter tells the RAM disk driver to set up RAM disks of N k size. The
default is 4096 (4 MB).

Yeah wutever

[Apparition-X]

God is dead. (Neitzche) Tape is dead. (Innumerable pundits) Disk is dead. (Conquest FS) Yet somehow, they all seem to be alive and kicking as of right now. I wouldnt be throwing a wake just yet for any of 'em.

Re:Yeah wutever

[skahshah]

You forgot *BSD!

Re:Yeah wutever

[gsfprez]

don't forget the most popular one of all time... Apple is dead. (everyone, starting with that stupid ass Dvorak)

unfortunately, i find any of these calls of "teachnology death" a waste of time... i'm working at a frmr TRW (now NG) location - and my cow-orker just brought in a floppy disk to my computer because they can't seem to get us network access to the printer.

nothing is dead - its all just where you're at.

Re:Yeah wutever

[Anonymous Coward]

Nietzsche is dead.
--God
(Lameness filter lameness.)

Re:Yeah wutever

[slasher999]

You all forget the most popular one - OS/2 is dead.

Re:Yeah wutever

[akorvemaker]

Some things are so obvious they don't need to be said

**ducks**

(Sorry, you walked right into that one :-)

Re:Yeah wutever

[jdgeorge]

God is dead. (Neitzche) Tape is dead. (Innumerable pundits) Disk is dead. (Conquest FS)
Also:
OS/2 is dead. (Almost everybody)

The same thing is true for all of the above:
They are only as alive or dead as the people who depend on them.

Re:Yeah wutever

[Big_Monkey_Bird]

"Print is Dead" - Egon Spengler

Re:Yeah wutever

[pudge_lightyear]

I think the correct way to state this is that they are only perceived as alive or dead based on the number of people who depend on them.

I took this as somewhat of a wise-crack towards religion... if not... sorry, I misunderstood. God is a being that exists outside of our need or usage of Him. One may perceive Him as being alive based on our apparent need or lack thereof, but that's an assuming belief on that perceivers part. Ancient Greek religion had the idea that if you stopped following a god, that god may cease to exist, but that's not the case with the God that Neitzche thought himself to be referring to.

In the same way, tape and disk are ideas that represent physical devices. Obviously, just because we don't need or use them, doesn't mean that the device or idea ceases to exist...

OS/2 the same way. The thousands/millions of disks still exist regardless of whether people use them. It is no more (physically) dead or alive based on it's usage... it is just perceived to be.

In short... just because we declare something dead, doesn't mean it is.

Re:Yeah wutever

[samhalliday]

Neitzche is dead. (God)

Re:Yeah wutever

[Mike1024]

BSD, Apple, Decent rock music and 'BSD is Dying' are also dying.

Or so I'm told.

Michael

Re:Yeah wutever

[NDSalerno]

God is dead. (Neitzche)

Actually, Hegel originally wrote that line. However, society seems to attribute this to Nietzsche because he followed up on the idea and proclaimed it louder than any of the other atheist philosophers.

Re:Yeah wutever

[Andy_R]

God is dead...I wouldnt be throwing a wake just yet...

Yeah, Easter would be a dumb time to do that :-)

Re:Yeah wutever

[Anonymous Coward]

Iraq is dead. (Courtesy of 'God')

Re:Yeah wutever

[moonbender]

That's tragic. Whoever this "Neitzche" fellow was, my condolences to his friends and family.

(I just searched Google for "Neitzche". 2,020 results, nearly all of them misspellings of the philosopher's name. Sad.)

Re:Yeah wutever

[Drakonian]

The already beleagured disk community was hit with a crippling bombshell today... etc etc... Truly and American icon.

Re:Yeah wutever

[ces]

OS/2 is dead?

Quick, someone tell the thousands of ATMs and voicemail systems out there that are still running it.

OS/2 never died. It just dropped out of the consumer OS market.

Re:Yeah wutever

[eyeye]

I think that is spelled "Oestrus" ;-)

Re:Yeah wutever

[samhalliday]

Neitzche is but a translation of his name into my native language, er, or something...

Nietzche is God -- (dead)

Re:Yeah wutever

[mt2mb4me]

I hope you don't mind me making this my sig in the future, i'll give you the credit

Re:Yeah wutever

[pyrrho]

well, yeah, except for God.

Re:Yeah wutever

[pyrrho]

>Ancient Greek religion had the idea that if you stopped following a god, that god may cease to exist, but that's not the case with the God that Neitzche thought himself to be referring to.

why not?

Re:Yeah wutever

[pudge_lightyear]

I'm not sure that this is worth keeping alive... but I'm a sucker...

What if I said... pyrrho is dead? Does that make you dead? Surely not. pyrrho (which I'm guessing is not your real name) was established by more than my mere words. You exist because of parents, because of others experiences with you, etc.

Neitzche has been called genious, insane, filled with hate, etc. etc. etc. Regardless of his qualifications, he was a man that truly hated both the Christian faith and the Christian religion ("...It has been almost the terror of terrors and out of that terror the contrary type has been willed, cultivated and attained: the domestic animal, the herd animal, the sick brute-man--the Christian." - Neitzche), and since the Christian faith did not establish the existance of God, rather the godhood (if that is a word) of Jesus, his distrust certainly did not limit itself to Christianity, but to, I imagine, the Abramic religions, and for that matter, religion in general.

Anyway, whether genius or insane, the man truly had phsycological, theological and emotional issues with Christianity. So... it's his word against the thousands of years of writings, experiences, etc. that have established both the God of the Abramic religions and specifically the God of Christianity.

All I'm saying is that I would hardly trust him, you, or any other philosopher, scholar, doctor, computer programmer, garbage man, or other, to declare whether God is dead or not, while His existance in both the past, present, and future (however real you claim it to be) cannot be sufficiently debunked.

Could be accomplished with a "preferential cache?"

[mcgroarty]

I wonder if a kernel could realize many of the same performance benefits with current filesystems by identifying directory inodes and small file inodes and lowering the probability of those falling out when it's time to free pages.

Re:Linux is dying

[Anonymous Coward]

Wow, that was a hilarious response...I just don't know how in the HELL I'm ever going to pick myself up off the floor and stop laughing...

I mean, hey, the amount of originality and wit just astounds me...and they were SO creatively applied...

how does he do it????

Who are you kidding?

[amembrane]

What do you mean you can't turn it off? I haven't had a pagefile since I hit 1GB of RAM in my desktop. The Windows XP option (System Properties, Advanced, Performance Options, Advanced, Change) is even called "No paging file".

Re:Who are you kidding?

[asdfasdfasdfasdf]

Thanks for the info. That must be new with XP. In 2000 (still much more stable) you can't go below 2 MB, and it still insists on using it, even though it's so small. Perhaps there are registry hacks to do it, thanks again for the info!

Re:Who are you kidding?

[Anonymous Coward]

Some applications, a few games pop into mind, require virtual memory regardless of how much memory you have.

Re:Who are you kidding?

[Anonymous Coward]

You could do it in WinNT and W2K also.

Re:Who are you kidding?

[Dot.Com.CEO]

It's not new with xp, it's been there since windows 95. Of course running non-NT based Windows systems without a pagefile was not advisable (too many memory leaks from badly programmed apps and no way for the OS to resolve them).

Also, please stop with that myth that 2000 is more stable than XP. If you don't like the eye-candy, fair enough, but 2000 crashes as much as XP. No more, no less. And on a properly configured system they crash as much as Linux, that is to say, they don't.

Re:Who are you kidding?

[amembrane]

Anytime, it has to be one of the most noticable performance increases. It is too bad that 2000 has a minimum of 2MB. One of the love / hate things about Microsoft is that they do tend to eventually add all the features you want, but there is usually no chance of getting them in any but the latest and greatest version.

Too expensive

[xRelisH]

good idea, but it doesn't seem to have any use other than for high end servers. The per GB price for RAM is high compared to Hard Drives.
Perhaps when it's cheaper it may be more feasible for home users.

Re:Too expensive

[Anonymous Coward]

Actually, I think it would have less usefulness on high end servers. Such servers would rarely power down, and metadata and file blocks are already cached, because these servers have gobs of high speed RAM. You wouldn't gain anything by throwing in a battery-backed NVRAM.

where....?

[bryam]

Where are the code Luck?

How is this any different from ....

[binaryDigit]

a very aggressive caching algo? I mean other than the battery backed part. You should be able to attain similar performance benefits using a purely software solution assuming your app doesn't do a lot of "important" writing to "small" files (where Conquest would do it all in RAM and still be able to persist it). But things like dll's,exes and whatnot don't change.

I guess I can understand the benefits (as minor as they may be relative to price), but the thing that bothers me the most is why does it take 4 years and NSF funds to come up with something that seems so obvious?

And one major problem would be getting over the fact that if the machine craters, you can't just yank the drive and have everything there, though I assume they have some way to "flush" the ram (can't read the .ps files to check).

Re:How is this any different from ....

[Ranger Rick]

Seems to me that this method is different in that the storage has it's own memory, so you wouldn't be maxing out bandwidth on the memory being used for the rest of the system. Otherwise your programs would be half as slow at accessing memory, and the other half of that bandwidth would be taken by accessing storage (obviously depending on the app).

Re:How is this any different from ....

[platos_beard]

I'd go further than that. This is like having a caching algorithm that determines what to keep in RAM based entirely on file size and whether the data is FS metadata. That seems unlikely to be the best way to use whatever RAM you have available for caching.

Re:How is this any different from ....

[pingbak]

Even with an aggressive caching algo, you can't flush memory to disk at the disk's bandwidth because you have metadata to update (inode arrays being one, directory blocks being another.)

Conquest doesn't impose much of a FS on the disk, so it doesn't have a lot of metadata to update. Therefore, it flushes to disk at close to the controller's bandwidth. It also doesn't have all of the other complexities that come with the disk-oriented parts of the OS such as caching, after all RAM is already the cache.

Re:How is this any different from ....

[Pheersome]

I guess I can understand the benefits (as minor as they may be relative to price), but the thing that bothers me the most is why does it take 4 years and NSF funds to come up with something that seems so obvious?

First, a wee bit of background. The research is the thesis project of An-I Wang, a grad at UCLA. One of his advisors is Geoff Kuenning, who teaches at my school, Harvey Mudd College. I went to Geoff's presentation here.

If you'd seen the performance statistics, you wouldn't be calling the benefits "minor." It stomps all over XFS, ext2, and ReiserFS in pretty much every test, and as opposed to ramfs actually has the capability to store more than what fits in RAM.

Also, a big part of what they did was matching the FS behavior to typical usage patterns. All small files are in RAM because they are the most frequently used. Most large files are read mostly-sequentially: in the example of MP3s, tags are read from the last few dozen bytes, then the player application reads the rest of the file straight from the beginning to the end. Also, most large files (media being the main example) are written once and rarely or never modified.

So, my point is, Conquest is designed to perform optimally for a desktop user. That was a nontrivial task, and while it seems an obvious idea in retrospect, it seems plain to me that there was in fact a substantial amount of research that went into this design.

RTFA

[Anonymous Coward]

Sorroy, they are still using the HD.

The RAM only holds the meta-data and small files.

Cost

[Sophrosyne]

1. As of October 2001, Conquest can be used effectively for a hardware cost of below $200.

For a whooping 512MB's no doubt.

Re:Cost

[kwerle]

...cost of below $200.

For a whooping 512MB's no doubt.

Dunno where you buy your RAM, but CNET is willing to sell me Kingston memory (512MB 133 MHZ DIMM) for less than $90 (one place says $65, but I don't believe them).

Time for you to find a new RAM supplier.

Re:Cost

[00_NOP]

Still means 40Gb is going to be extrememly expensive - and that RAID 1 array of 20 200 Gb disks that drives your business, well how much is tat going to cost to replace?

Don't think the disk is dead just yet.

Re:Cost

[kwerle]

Don't think the disk is dead just yet.

Nor do I - I was just suggesting the the RAM cost the previous poster suggested was somewhat inflated.

Re:Cost

[Anonymous Coward]

ram cost was extremely inflated. I HAVE purchased kingston 512mb modules for $62/ea including shipping from a retailer in the past month.

Obvious solution

[palad1]

Put your swap file on a Ramdisk.

Palad1, MCSE

Re:Obvious solution

[rikkards]

You must be an MCSE (I am too but I don't flaunt it). Especially since Microsoft recommends approximately twice your RAM (really 1.5-3X the amount of RAM) see here for the scoop

Re:Obvious solution

[Anonymous Coward]

Especially since Microsoft recommends approximately twice your RAM

MS suggests a lot of things, their old rule was 1x + 12M...

regardless, it doesn't really matter what the suggestion is, since you should really do your own needs analysis based on the role of the machine.

that said, lol @ pagefile on a ramdisk... that sounds like a gentoo solution to me. 8)

not that NT-XP's aggressive use of the paging file doesn't cause people to do stuff like that.

can someone please

[Anonymous Coward]

explain to me the difference between what these guys are suggesting and what WD is doing by putting 8 mB of cache in their drives?

if their method really provides a speed increase over expanded cache, can't their method be moved into the hardware of the drive somehow?

Well

[Vej]

That's nice, but it's not much for industrial grade when you need a solid state disk that can last in the field without having replaceable parts all the time that could go wrong.

MCSE, eh?

[Anonymous Coward]

Does that come in handy when you're working the register at Burger King?

Re:MCSE, eh?

[RedAlgaron]

you're just jealous because you haven't gotten past your senior finals at highschool

Re:MCSE, eh?

[Anonymous Coward]

hopefully, the answers to his senior finals aren't available verbatim on like 500 different websites, so when he does complete his tests he'll actually have accomplished something.

What if the battery fails?

[GrimReality]

...battery backed RAM...

Pardon my ignorance, but what happens if the battery fails? Of course, this is highly unlikely, but just a scenario.

In a conventional disk the data would remain even if power is switched off, but a RAM would lose the data (or get corrupted or cannot be sure if the data is exactly the same).

Thank you.
GrimReality
2003-04-21 15:51:18 UTC (2003-04-21 11:51:18 EDT)

Re:What if the battery fails?

[Anonymous Coward]

It's not highly unlikely. Batteries fail all the time, especially rechargables. More than any other problem as the 'tech guy' in our office I see computers that cant tell time/save bios settings, because the little 2032 clock batteries have failed. And on machines 2 or 3 years old, as well.

Re:What if the battery fails?

[sporty]

Use it like write back cache.. or write through. Your choice. Less chance of loss, eh?

Re:What if the battery fails?

[GrimReality]

Use it like write back cache.. or write through. Your choice. Less chance of loss, eh?

But this does not seem to be behaving like a cache, rather according the description, the metadata and small files are stored only on the volatile RAM. In this case write-back/write-through may not be applicable.

Maybe, I misunderstood it, but the description said that the disk is for storing large files only!

Furthermore, doesn't hard-drives already have some form of cache. So this has to be a different system.

Thank you.
GrimReality
2003-04-21 16:59:33 UTC (2003-04-21 12:59:33 EDT)

Re:What if the battery fails?

[darkroot]

Yeah. I agree.
I am saying this because we don't know exactly when the battery will fail and whether we will know it when it does.
Even if we use rechargeable batteries which automatically gets recharged whenever the system is turned on, the battery keeps losing its capacity.
And it is VERY annoying when you were working on your paper and suddenly the battery fails.

Page files considered good

[Merlin42]

Even with tons of RAM pagefiles are a GOOD thing and if used properly (Even MS uses them properly these days) they speed up the system on the whole.I have done a *little* OS development so I may not be an expert but I do have an idea what I 'm talking about.

The swapfile is where the OS puts things it hasn't used in a while. On windows this would probably include things such as the portions of IE that are now part of the OS and you are forced to have loaded even if you are not using the box for web browsing. Having placed these items in the page file frees up room for things that are currently usefull such as IO buffers/cache (disk and/or net) that can dramatically increase speed by storing things such as recently used executables, meta-information .... wait this sounds familiar ;)

That being said I think the technology discussed in this article is a bit too single minded. I think adding an extra level in the storage heirarchy between main ram and non-volitile HD is probably a good thing. My idea is to add a HUGE pile of PC100 or similar ram into a system and have this RAM accessed in a NUMA style which is becoming very popular. The nintendo GameCube uses a form of this aproach, there are two types of RAM with a smaller-faster section and a larger-slower section.

The problem with my idea is that the price difference b/w cheap-slow RAM and fast-expensive RAM is not enough to make it worth the extra complexity currently. But, I would guess that if someone took the effort to design/build cheap slow RAM they could find a niche market for a system accelerator device ... but then again I could just be not well enough informed (a little knowledge is dangerous ;) and rambling like an idiot.

Re:Page files considered good

[orthancstone]

That would lead to a lot of ram banks on the motherboard. I wonder how stability would be. (Unless mobo manufacturers started implementing the extra RAM on the boards) Still, the potential for caching to disk would be there anyway so I'm not sure that idea actually solves anything except further delaying the inevitable. Good thoughts though.

filesystem

[supergeektux]

what tells you when your battery are dead? all of your small files being wiped? also if you really hate someone you can pull the battery and next time they power cycle where did all their files go!

Re:filesystem

[Anonymous Coward]

About the same as if someone removed your hard drive. Hopefully the battery will be behind a screwed down cover, if not a lock.

Remember, beware of programmers carrying screw drivers.

Solaris does this... not sure about linux.

[pr0ntab]

Solaris is very finnicky about ever writing out the pages in working RAM into swap. In particular, it fills free RAM with cached inodes, directories, and small files until the free memory ceiling hits a watermark, then it merely starts running the page reclaimer. The page reclaimer is the only way certain files and directories get written back to disk unless some option is turned on in the UFS layer, or the file is opened O_SYNC, AFAIK.

I imagine linux does this too, I remember reading how the vfs layer and page buffer layer are tied at the hip, and the same sort of thing happens.

Re:Solaris does this... not sure about linux.

[mcgroarty]

Are you sure Solaris selects small files and directory data over chunks out of long files? That's the key thing here.

ok, so the FS is faster...

[illumina us]

but what about drive seek times. the storage is still on magnetic media and the seektimes aren't really that good, not when you comepare it to oh let's say PC100 RAM. they really need solid-state storage but make it so it's usuable without external power supplies.

Umm..

[Anonymous Coward]

My raid controller basically does this already..

It's an old IBM 3H 64 bit PCI model with 32MB of ram and battery backup.. newer 4H models support more ram.. but how is this any different?

The most used and smallest files stay in the cache.. the rest are called when needed.. and if god forbid the power fails, and the ups fails.. the card has a battery backup to write out the final changes once the drives come back online.

Re:Umm..

[TheSync]

Is you RAID controller caching files or caching sectors?

Re:Umm..

[Mannerism]

I believe the researchers' point was that existing filesystems were (understandably) not designed with a RAM component in mind. Therefore, caching controllers and similar solutions are inefficient compared to Conquest, which assumes the presence of the RAM and is optimized appropriately.

sorry about repeating battery part

[supergeektux]

i was writeing it as if noone had wrote a comment on that, when i posed it i noticed someone beat me to it

right....

[hawkbug]

And when the battery goes dead, bye-bye data. Stuff like this scares me. I'm concerned enough about losing all my data on a current hard disk that can exist without power - if I had to keep my machine pumped full of electricity all the time, I'd be even more paranoid about losing stuff at random.

Re:right....

[Rary]

"And when the battery goes dead, bye-bye data."

And when the hard drive crashes, bye-bye data.

Whether using "old-fashioned" hard drives or "newfangled" solid-state storage, the lesson remains: always backup your data.

Reliability

[InodoroPereyra]

There are several goals in a next generation filesystem implementation. Low cost and speed are important, and conquest goes in the right direction. But how reliable is persistent RAM for storage ? Hard drives go belly up every once in a while, and we would all like to see some sort of affordable solid state storage come up and replace HDD. Persistent RAM seems to be a step in the opposit direction, or Am I wrong ?

Re:Reliability

[TClevenger]

A decent system would maintain an image of the RAM on the HDD. In case of battery failure, replace the battery, boot the system up, and it should rebuild the RAMdisk from the hard disk--just like rebuilding a drive in a RAID.

wow

[hpavc]

battery backed ramdisk, just like my TI99 had.

i guess this might be neat now that computers dont have extra 'cards' of memory. but when that was the way to expand your computer it was quite easy to have this method of storage.

Probably the same way...

[Mr. Underbridge]

...your system decides what memory to put in cache, what to put in main memory, and what to swap out (for the RAM-challenged).

Still, worst-case scenario for any file is the status-quo that we're dealing with for *all* files, so no matter what, this would be nice.

Nonvolatile cache does similar things

[Anonymous Coward]

Nice to have a filesys like this, but a cache living on nonvolatile storage that is much faster than the host is probably easier to get folks to adopt. That notion can be used with cache on fast disk being used with backing store on some much slower technology too. Only changes to the metadata about what is in cache need be synched across a cluster as I recall (designed one c. 1995 but never built it). The direct execution part is hard to do in a pure cache system, but on the other hand >1 layer of such caching can be done. The cache has to do some of the jobs a filesystem will do, to manage what is on its store, but because it is nonvolatile, many boundary conditions during cluster state transitions become easy to handle, and as a practical matter it gets much easier to get people to adopt a cache system which is keeping their familiar filesystem and all the utilities which know about it.

Full paper in HTML

[monk]

For those who are not PS worthy.
The paper
Looks like a great server side file system. This is finally a step away from this whole "file" madness. All storage and IO should be memory mapped, and all execution should be in place. Anything else is just silly.

Re:Full paper in HTML

[Jeff DeMaagd]

All storage and IO should be memory mapped, and all execution should be in place. Anything else is just silly.

I'm not sure what you mean by this. I would think that this system would merely adjust the distribution of RAM is in a computer, the only difference being that some of it has power backup for the memory.

Dead?

[Sandman1971]

Dead? I don't think so. Get back to me when they start using Non-volatile RAM and the price per byte is equal or less than the price for harddrives. Until then, the HD is going to be alive and kicking.

One thing I've always wondered though. Why not release an OS on an EPROM? It would make boot time and OS operations extremely fast. I'm still surprised to this day that this isn't mainstream. Ahhh, the good ol' days of Commodore when you OS was instantly on when you turned on the PC.....

Re:Dead?

[kitty tape]

Actually, the full name of the talk was "The Disk is Dead, Long Live the Disk". Obviously, the idea of eliminating the disk was meant to be taken with a grain of salt.

Re:Dead?

[GigsVT]

Why not release an OS on an EPROM?

With security patches every couple days for all modern OSs, you'd be pushing the write limits. :)

Re:Dead?

[pjrc]

Why not release an OS on an EPROM?

1. Hard to update the code
2. More expensive than cdrom or internet download
3. Slow execution (unless copied to fast memory at startup, as the bios usually is). EPROMs are very slow devices. Eg, fast ones are 55 ns and 16 bits wide... memory bandwidth of 36 Mbyte/sec. That's 10X slower than SDRAM. Slower 70, 90 and 120 ns, 8 bit wide EPROM and Flash chips are the most cost effective.
4. Would not boot faster. OS boot time is dominated by hardware detection and initialization, and system-level (Sys5 init) startup. Loading the kernel image from the drive is an insignificant portion of the startup time.

In the good 'ole days of the C-64 and Apple ][, you booted quite instantly into a basic intepreter... but if you wanted to boot into an "operating system", well, that took a LOT longer. As I recall, the C64 disk drive was extreemly slow.

...goes great with 64-bit && cheap RAM

[ceswiedler]

Though I don't think it's a useful general-purpose concept to have a RAM-only FS, I'm hoping that fast RAM will catch up to magnetic disks in size. A standard FS/VM will end up caching everything if the RAM is available. I seem to recall that ext3 on Linux, if given the RAM for cache, is faster than many ramfs/tmpfs implementations. Plan9 completely removes the concept of a permanent filesystem versus temporary memory. Everything is mapped in memory, and everything is saved to disk (eventually). It's a neat concept, and it happens to go very well with 64-bit pointers and cheap RAM.

I'm hoping that hardware people will realize that we need huge amounts of fast memory...whether or not we think we need it. We're stuck in a "why would I need more RAM than the applications I run need?" kind of mindset. I think that the sudden freedom 64-bit pointers will provide to software developers will result in a paradigm shift in how memory (both permanent and temporary) is used. Though like all paradigm shifts, it's difficult to predict ahead of time exactly what the change will be like...

Re:...goes great with 64-bit && cheap RAM

[neurojab]

the iSeries has had the "single-level store" concept a la Plan 9 in working operation since 1985. Just thought I'd mention it since tens of thousands of people use it everyday and don't even realize the advanced technology under the hood.

Same as "what if the hard drive crashes?"

[Ars-Fartsica]

Battery failure in this case is the as the case of hard drive failure in the nonvolatile model. You either have backups on another media or you are screwed.

Note that hard drive failures are still common and likely to be much more common than a battery failure, as it would be trivial to implement a scheme through which batter recharding would be automatic while the computer was plugged in. The battery would only be directly employed when the system was unplugged or the power was out. Even in that case it would be also trivial to implement a continuous/live backup system to a nonvolatile media like a hard disk, which by that point would be ridiculously cheap.

Re:Same as "what if the hard drive crashes?"

[Slime-dogg]

Yeah, I'd also like to point out that your CMOS has a battery. I've never had the case where my CMOS battery failed, but it could happen. If it happens, then I'll probably have to be running all default settings with my BIOS and stuff, with no boot password.

Having a battery in the system is nothing new, and I don't really see it as a problem.

huh

[cybercuzco]

Great, until yoru battery back up dies and you lose your entire fs.

HFS

[tsetem]

This really should be implemented via an HFS approach. ie: The commonly used files are placed & kept in memory, while less-frequently used files are kept on another medium (Disk or Tape).

This way, the 20% of your disk that is used 95% of the time is kept in memory forever. While the other 80% is placed on disk. With HFS, as soon as the system saw that you needed the file, it would automatically pull it off of the hard-drive, and stick it in memory.

Similar techiniques are used for tape & hard-drives. ie: The Hard-Drive has a bazillion files, but only 20% are used. If you need a file, the system automatically pulls it off of the tape for you. It's like a tape archival system, but it's integrated into the OS, rather than requiring an external application to be run.

Now if HFS could be implemented & documented on Linux...

Re:Linux is dying

[Anonymous Coward]

mean, hey, the amount of originality and wit just astounds me...and they were SCO creatively applied...

Unanswered Questions

[Salamander]

While this is great for some environments, it will remain a research toy until several real-world problems and limitations are addressed. Several people have already brought up the issue of having more small files than will fit into the BB-RAM. Another issue is portability. With a traditional filesystem, if a whole machine dies you can slap the disk into another one (of the same type). With Conquest, you have to transplant the BB-RAM as well. How many slots do you think a machine has for BB-RAM, vs. how many disks can you attach? At the very least you'd need to coordinate use of the BB-RAM across filesystems, plus a way to flush/restore one filesystem's portion to actual disk.

There are many more issues like these, which would need to be addressed before a Conquest-like approach is really viable in the real world. One of more of those issues might turn out to be a show-stopper. It's interesting research, but don't expect it to replace traditional filesystems any time soon.

Just like BSD

[nonmaskable]

First BSD, and now disks - when will the madness stop?

Re:speaking of ramfs...

[Anonymous Coward]

use tmpfs instead

No RAM cards?

[yerricde]

i guess this might be neat now that computers dont have extra 'cards' of memory.

Then what are DDR DIMMs?

Re:No RAM cards?

[hpavc]

my assumption is that when you add these simm/dimms to the motherboard (or to a video/raid card) that they are being added to a contigious pool of memory (or they are starting one if there isnt any there before hand)

when you have card slot (isa or some homebrew bus extension) and you have a memory on that card you dont have contigious memory. this memory also tended to have the option to be battery backed up since a new device driver needed to be written for the os to access it anyways.

Simple optimization...

[Anonymous Coward]

Ok, if you *are* going to have them (and they can be good)
Make a seperate fat partition (sectors as big as possible), which you use *only* for the page file, make the max and min sizes the same... and it zips along.

Re:This is Blatant Karma Whoring

[Anonymous Coward]

Yes you do. Usually 2 points. Or at least, you used too.

You'll need a new motherboard anyway

[yerricde]

How many slots do you think a machine has for BB-RAM, vs. how many disks can you attach?

If both the RAM drive and the rotating-magnetic-media drive use the same type of connection to the motherboard (e.g. Serial ATA), you have nothing to worry about except battery failure. Otherwise, you'll need a new motherboard for battery-backed RAM anyway.

Re:You'll need a new motherboard anyway

[Salamander]

If the RAM drive is using the same connection as a rotating-media drive, you're really dealing with an SSD and you're subject to the bottlenecks that Conquest supposedly avoids. Some filesystems already allow you to store the journal on a separate disk from the target filesystem, and it's a pretty common trick already to put that journal on an SSD.

I have an app. for this today.

[OwnerOfWhinyCat]

I'm sure some very savvy person out there has discovered how to netboot all versions of Windows, but whatever it takes, I haven't found it.

If I could put one giga-byte stick of ram ($124 from pricewatch) onto a DIMM -> IDE drive board (say $100), then workstations could netboot, download the OS of choice, and run off the local ram disk. They could store their important data on net drives, and (as various Windows versions often need) they could reboot at tremendous speeds. This would eliminate hard drive failures outside the computer room, and would provide an easy solve for many virus problems. I wouldn't even need the Conquest method for dividing up the data, as I would manually divide the big data onto the netdrives and the OS onto the machine.

With a Customer Care staff of 100 the amount of disk-swapping and disk cleaning that goes on is a serious chore that would just go away. Well worth the small extra investment. This would also make it easier to switch people over to Linux. "If you want to try my latest Linux desktop, just boot in "Linux (test) mode", and if you don't like it, reboot in Windows mode."

Hybrid storage devices

[swb]

I've always thought it'd be great to have a hybrid storage device, coupling RAM, hard disk and tape or optical into a single cartridge. The device would then manage the migration of data between the three mediums transparently based upon access.

Rather than being filesystem dependent, I'd have the device not know or care about filesystem, just logical disk sectors. Those that were accessed frequently would stay on the higher speed medium and those that weren't, the less frequent. Large files that were only partially read wouldn't penalize the computer for being on tape or penalize the high speed storage for unaccessed chunks taking their space.

Unfortunately its probably too complex to actually implement, and disk storage capacities have grown so fast to quickly that it seems like disk is the way to go, its applying disk to the servers that need it intelligently that's the bigger challenge (iscsi, fiber channel, EMC, etc).

Re:Hybrid storage devices

[GigsVT]

Well, hard disks have a lot of RAM on board these days, 8 megs is getting to be pretty much standard on ATA now.

As far as optical, the various movie companies have stifled any optical disks with the capacity to mean anything.

But to take your concept to something that we can do right now, in software.... (hint to any industrious open source devs!):

Imagine this. A hybrid RAID kind of system involving one high speed disk like a 10K SCSI drive, and several slow (5400rpm) but high capacity ATA drives. The 10K SCSI drive acts like an intelligent cache for the slower drives.

With a system like this, you get all the latency/transaction time benefits of 10K SCSI, with the capacity of the huge 5400rpm ATA drives that are out these days, all at a fraction of the cost.

This could all be done below the file system level, just as software RAID is done today.

Or it could be done in hardware. Imagine if the platters in a hard disk didsn't spin at the same speeds. What if instead there was one high speed platter, and several very low speed, very high areal density platters. A similar system could be developed.

Re:Hybrid storage devices

[swb]

That's a great idea, and its probably something that could be easily integrated into a RAID controller so you could stuff a 12-disk cabinet with 8 high capacity disks and 4 high speed disks and let the controller sort it out.

Although I still like the idea of integrating some kind of ultacapacity tape medium into the mix as well at the far end of the storage cycle for maximum storage. I think in a lot of storage situations, 10-30% of files are just not accessed all that often.

Re:Hybrid storage devices

[GigsVT]

That's true, but really, tapes don't have much on high end ATA these days. The biggest tapes only run about the same size as the biggest ATA drives, and the tapes require a $2500-$3000 tape drive to work. Even if it did make sense price wise, do you really want to wait 10 minutes to get that file you don't use very often? Or 20 minutes if someone else requests another file that was on the same tape before you?

To me, tape has really lost it's edge. It's not much more reliable than an ATA drive (in my experience with DLT robots and DAT), it's sequential, slow, and it costs a hell of a lot more than an ATA drive.

Only situation I see is if you have massive archives that you very rarely need data from (so you can leverage the higher price of the tape drives vs. the lower cost per MB of the tape itslf). This would be the case in something like bank transaction record keeping and applications like that.

Re:Hybrid storage devices

[swb]

I've been talking about augmenting our backup system (five LTO Ultrium drives, need to add about two more if we up storage on another system) with a couple of systems stuffed with ATA RAID-5.

And then management decided they wanted the tapes used once and kept forever, so I'm stuck doing shit on tape for the rest of my life...

Ramdrive

[DarkBlackFox]

My plan-

Once 64 bit procsesing becomes mainstream, and price per gigabyte of memory better (say, 16 gigs DDR 3200), store the OS on a small (~5 gig) hard drive partition, and transfer the entire thing to a 5 gig ramdrive on startup. Using serial ATA that shouldn't take too long, and the OS will run at dramatically increased speeds, especially if the swap is housed in the ramdrive as well. On shutdown, transfer the contents of the ramdrive back to the hard drive. With the massive RAM support 64 bit processing promises, I'll wager some incredible things are possible for those willing to experiment with technologies like this. Perhaps that's where the technology in this article is heading, although far less volatile/risky as my approach.

Re:Ramdrive

[Anonymous Coward]

especially if the swap is housed in the ramdrive as well.

Ok, now i lost you ...

Not a new concept or idea at all

[Status Quo]

Way back when I was growing up, we bought an Apple IIgs. After a while, my dad went for the memory upgrade with a battery back-up option. Remeber, this is about 12 years ago. It was nice because you didn't need to wait for the system to boot anywhere near as long.

Additionally, laptops take a similar concept and save the system memory image to hard drive and just read that in order to make your boot time a little shorter when you are away from the machine and it powers down.

ram storage

[larry bagina]

Back in the 1980s, Applied Engineering produced an Apple II card called "RamKeeper", IIRC. It was a memory card backed up with a battery, so you had a permanent RAM disk, which standard software could read/write from as if it were a normal device.

Nietzsche is dead (God)

[RelliK]

Now that seems to be true...

Has anyone statistics on RAM/HD prices?

[mseeger]

Hi,

does anyone has a statistic on HD and RAM prices throughout the last years?

I only have a feeling, that the last years RAM prices have fallen quicker than HD prices. This will naturally lead towards such developments as mentioned.

I think it would be very interesting to study the technical developments in the light of price developments. My bet is, that most inventions are not caused by bright minds but the need for them. For most technical breakthroughs, the mind is not cause but catalysator ;-).

CU, Martin

Re:Has anyone statistics on RAM/HD prices?

[Slime-dogg]

It's also important to notice the concentration of data associated with HD's vs. RAM. There are no vendors out there that are peddling 256 MB HDDs, or even 1 GB HDDs. You just can't find them anymore, and the ones you do find are likely to be cheaper than their RAM equivalents will ever be.

RAM is costlier on a per MB basis than HDDs could ever be, and every pricing chart that takes this into account will show how this will probably never change. Heh. I'd love to pick up a 320 GB stick of RAM for ~$300... I wouldn't be able to use it all any time soon, but it would still be cool.

I can't wait till MRAM is finally figured out. That stuff is the wave of the future.

Re:Has anyone statistics on RAM/HD prices?

[mseeger]

RAM is costlier on a per MB basis than HDDs could ever be, and every pricing chart that takes this into account will show how this will probably never change.

Correct: But if RAM started out to be 1000 times as costly as an HD an now is only 30 times as costly, this will influence the design of operating systems drastically.

This shell be no justification of memory consuming monsters (do read me Billy), but if you can speed up a system by using two extra megs of RAM, this may be an appropiate choice ;-).

Bye, Martin

P.S. I love operating systems which use memory economically. But this is admiration for a form of art ;-).

That's what I said about Linux . . .

[Idou]

"something like this could make waiting over a min or two to [re]boot totally obselete..."

But my company's IS dept. is still very content on making me reboot whenever I try to do something "luxurious," like having two applications open at the same time.

I will post my sig as soon as I finish rebooting.

Re:That's what I said about Linux . . .

[Karl Cocknozzle]

But my company's IS dept. is still very content on making me reboot whenever I try to do something "luxurious," like having two applications open at the same time.

Perhaps you need to upgrade your legacy IS people. Under NT/2k/XP you shouldn't need to re-boot except when windows asks you nicely to. (IMO it asks much too often, but that's a seperate argument...)

Perhaps some of your "legacy" IS people are still in the "re-boot" habit from the days of 9x/ME when re-boot really did solve 80% of our "weirdo-helpdesk tickets."

PDAs

[enos]

Just another piece to make me think that desktop computers should become more like overpowered PDAs and not the other way around.

The question is...

[carlmenezes]

Will the battery backed RAM reset when the computer does? Because if it does, this does not become practical for everyday machines that crash once in a while - I don't want to lose part of my filesystem because of a crash.

And if it doesn't reset, that's good, but then would it not also introduce security concerns? For example, if we're talking small files, wouldn't the internet cache qualify? What about /etc/passwd files? Wouldn't it a simple matter to read this off the RAM?

Re:The question is...

[CableModemSniper]

And if it doesn't reset, that's good, but then would it not also introduce security concerns? For example, if we're talking small files, wouldn't the internet cache qualify? What about /etc/passwd files? Wouldn't it a simple matter to read this off the RAM?

is it any less simple to read it off a traditional HDD?

How soon we forget?

[Anonymous Coward]

I proposed a similar idea a 5 years ago.
It even got posted to Slashdot, I just can't
find the old link. Neither the posted idea or
my own proposal are new.

http://www.redwoodsoft.com/~dnelson/tram/

dru
(sorry for the anonymous coward, I don't have an acct)

Might work on IDE

[mnmn]

IDE drives have large caches. I suppose if the control to the caches could be programmable, it could be used by a driver to achieve this on a regular PC, but then, we'd lose the cache speedup. Better still, move part of the Conquest FS functionality to the south bridge of the chipset, in the IDE wirings, using part of the RAM.

Re:Might work on IDE

[pingbak]

No, you've missed the point. The Conquest FS is part of the OS, not the drive. Putting more RAM into the drive is the wrong thing to do, because you're running at the drive interface's bus speed (33, 66, .. Mhz) instead of at the CPU-RAM bus speed (100, 200+ FSB speed.)

I'd like to make a prediction

[Anonymous Coward]

Linux will fully replace windows as the prefered desktop operating system- I'm not predicting when this will happen.

I'm the one who gave the talk

[one-egg]

Well, since I'm the person who gave the talk referenced in the original post, I suppose I ought to clear up a few misconceptions for folks. I'm not going to address every objection that's been raised, because most of them have been well addressed in our papers. I'll just highlight the most common misunderstandings.

First, the full title of the talk was "The Disk is Dead! Long Live the Disk!" We make no claim that disk manufacturers are going to go out of business tomorrow; history suggests that the technology will survive for at least a decade, and probably more than two. Talk titles are intended to generate attendance, not to summarize important research results in 8 words.

Second, the most common objection to the work boils down to "just use the cache". This point has been raised repeatedly on Slashdot over the past few years. However, if you read our papers or attend one of my colloquium talks (UCSC, May 22nd -- plug), you'll learn that LRU caching is inferior for a number of reasons. We were surprised by that result, but it's true. Putting a fake disk behind an IDE or SCSI interface is even worse, since that cripples bandwidth and flexibility.

Third, for people worried about battery failures, the only question of interest is the MTBF of the system as a whole. All systems fail, which is why we keep backups and double-check them. If your disk failed every 3 days, you couldn't get work done, but there was a time when we dealt with a failure every few months. Conquest's MTBF hasn't yet been analyzed rigorously, but I believe it to be more than 10,000 hours, which is good enough to make it usable.

Finally, I have chosen not to put my talk slides on the Web, at least not for the moment. But you're welcome to mail me with questions: geoff@cs.hmc.edu. It might take me a few days to answer, so be patient.

Re:I'm the one who gave the talk

[andfarm]

Hah! I was the one who stuck around after the talk to berate MP3 for silly metadata setup. Hope you don't mind the hordes of visitors too much.

Talk about not reading the linked page

[fizbin]

This is already a feature of Conquest: (from the linked page, bold emphasis my own doing)

Conquest uses memory to store all metadata, small files (currently based on a size threshold), executables, and shared libraries, leaving only the content of large files on disk. All accesses to in-core data and metadata incur no data duplication or disk-related overhead, and executions are in-place. For the large-file-only disk storage, we use a larger access granularity to reduce the seek-time overhead. Because most accesses to large files are sequential, we can relax many historical disk design constraints, such as complex layout heuristics intended to reduce fragmentation or average seek times.

Re:swap on ramdrive

[Anonymous Coward]

Swap space is only used when the system is nearly out of ohysical RAM. It's a hack that trades speed for capacity (RAM is fast, but hard drives hold much more). Putting swap space on a ramdrive would be pointless, since all you would be doing is moving data from one area of the (already crowded) RAM to another.

Unfortunately . . .

[Idou]

for a company about to face MAJOR global workforce reductions, my "upgrade" would be software only, and 2k literally CRAWLS on this harware specs (some of my coworkers were dumb enough to "upgrade").

I really wish they would get off their high-horses, realize they are encapable of meeting ALL their user's needs, and let the more capable users support themselves, using the tools of their choice. Perhaps employees will be required to support themselves when "telecommuting" becomes more wide-spread and having a central IS authority will become impractical (those unable to support themselves will either be stuck at a central office or stuck without a job).

So, basically, with the new MS licensing, our IS dept. as blown its budget on software licenses that it can't effectively use, because it doesn't have enough $ left over to upgrade hardware specs. Meanwhile, the same hardware runs far better on a Knoppix live cd which is FREE.

On the bright side, maybe they will be able to afford better hardware after they lay some of us off.

And using MS products is good for the economy, how?

Re:Could be accomplished with a "preferential cach

[jafuser]

Why not just load the whole filesystem into virtual memory? =)

Netapp filer NVRAM???

[lazardo]

Network Appliance OS 'Data OnTap' has been storing FS metadata/data in battery backed NVRAM for many years. Disclosure/SEC: former employee, individual and/or family members may hold NTAP stock.

Re:Could be accomplished with a "preferential cach

[mcgroarty]

Why not just load the whole filesystem into virtual memory? =)

If you're springing for the DIMMs, then you're my new best friend!

It's dead, Jim!

[Anonymous Coward]

You knew I had to.

We've been doing this for YEARS.

[Anonymous Coward]

My company, and companies like it have been running filesystems out of battery backed RAM for years. This is hardly new or interesting. Yes we run 533Mhz DDR off battery. scary huh?

silly

[g4dget]

What needs to be stored in battery backed RAM is not "small files", it is "frequently accessed data". And for that, you don't need a new file system, you merely need to add a persistent cache to the existing file system.

You have two ways of doing that: either put the logic for that sort of caching into the disk driver, or put battery backed RAM into the disk drive or controller itself. I think both have been explored in the past.

obligatory quote

[Sepper]

"God is dead" --Nietzsche "Nietzsche is dead" --God

Re:obligatory quote

[pyrrho]

where did God say that? cave wall perhaps?

Synergy

[Liquor]

It seems like this would be an ideal complement to a system that was built with mram - non-volatile (modernised 'core') memory since no battery backup would be required. But I'm sure you already had that in mind. :)

MULTICS & Drive/Controller Cache

[Anonymous Coward]

1. This is different from a big cache . . . how?

2. Instead of creating a "whole new file system concept", how about putting cache on the IDE cable, or the controller, or wherever? Back in the old days when a controller was a separate card, that would have been the obvious approach - and it would improve any/all drives attached.

3. MULTICS's philosophy, all the way back in 1965, envisioned all "segments", whether executable or data, as existing in virtual space backed by however many levels of whatever size/type physical memory you wanted. There are no "files" distinct from "buffers" distinct from "memory", no "i/o" operations; one simply accesses a named segment at the desired displacement, and leaves it to the paging mechanism to get the required data to a place where one can operate upon it. When it's not in current use, it's rolled out (if dirty) to backing store as needed. See http://www.multicians.org, particularly http://www.multicians.org/general.html#tag13 1.3.1 and 1.3.2 which will lead you to http://www.multicians.org/mgs.html#segment. Of course, the size of a segment seems ludicrously small to us today, but one meg sounded like a lot back in 1965.

I'm with ya buddy

[Anonymous Coward]

Myself, I've been predicting that the Sun will go nova... someday. When will the rest of you listen?

Yawn; Where's the source?

[Anonymous Coward]

These techniques along with many others for file systems can be read about add nauseum from scientific papers, thesis, etc found doing a google search. If you want me interested then let me poke at the source, else nevermind.

not just cost/meg

[jonhuang]

I was actually _at_ the presentation and the speaker was aware of this. He pointed out that the adapation of new storage media was not driven just by an absolute cost/meg measure, but more of an absolute point; specifically, when the cost of the storage shrinks below the cost of text on paper.

As an example, he pointed out that despite being insanely cheap per gig, tape media is rarely used except for what can be considered really big, really slow files (backup images).

Prediction:
It'll not be cheapest, but cheap enough.

Last note: Conquest is a transition model to fill the gap until solid state takes over completely. The breakpoint for small files will be hit much sooner than the breakpoint for everything.

"Yeah wutever"

[pyrrho]

...'cause this is my United States of Whatever!

This begs a question...

[El Camino SS]

When will we get to the point where storage is so cheap and fast that we no longer need RAM?

Anyone have any ideas?

Re:This begs a question...

[makapuf]

DUH ! now.

If you want 15 years ago speed.

Or never if you want to have what you have with the latest static, electrical device with a spinning, mechanical device.

10 years ago

[Anonymous Coward]

On the Apple ][ GS I had, Applied Engineering made a batter backed up device called the "Ramkeeper". I'm sure there were things out before then too. It's not really new technology.

OH MY GOD!

[Anonymous Coward]

These guys have discovered what NetApp has been doing for over 5 years!!!!!

In the Interim ... Holographic Storage

[Ex-MislTech]

http://www.aprilisinc.com/holographic_storage.htm

Excerpt:

Holography makes use of the full thickness of the recording material, providing data densities proportional to media thickness.
This makes possible capacities of more than 1,000 GB on a CD disk format. By comparison, DVD technology provides only 9 GB on a double-sided disk.

Peace...
Ex-MislTech

What if I said... pyrrho is dead?

[pyrrho]

You probably mean the original pyrrho (ancient skeptic philosopher)! But no, I get you point. Certainly, there is an existence there. But with God and gods it's not so clear that they do exist other than in the mind of man. N. seemed to think that if God isn't really metaphysical as advertised, God still exists, and even acts, through man, by virtue of being an idea of man... a wide spread idea.

He was not, in fact, against Abramic religions in general, certainly not judaism, which he thought was a life affirming religion. He judged religions not on their accuracy, for they are all inventions of man according to he and I, but on their utility and form. To believe in spirits to promote creativity and life is a good thing, even if an amount of self deciet happens to be involved (he pointed out the same issues with mathematics, but mathematics is still quite useful and has truth in it, in spite of the errors/approximations of the truth). Judaism he thought was a strong religion, a healthy one, and that Christianity was more or less it's plot and revenge against their oppressors.

>So... it's his word against the thousands of years of writings, experiences, etc.

no, it can't be... as you said, it's not a matter of whose word to take, it's a matter of fact. Pyrrho either exists or does not exist, and although most people do not know he exists (in my case) or existed (in Pyrrho of Ellis' case), he does and he did. Nietzsche could be the lone advocate of his position, and may still be right. It's not a democracy unless N. is right and it's a human idea. Then you could say, "well, we still see the idea widespread in the wild, so how can God be dead if that's what God is?" That would be a good point... N.'s answer was that Buddha's shadow too was said to be shown on a cave wall for 1000 years. It's a bit thick in poetry and semantics at that point.

In the end there is no verifying if "God is dead" or not, and that is not the point of the statement. The real point is to open up the can of worms of what constitutes the real material of God, where does God reside, assenting that God might be imaginary but still exist, and all these related issues.