Domain: llnl.gov
Stories and comments across the archive that link to llnl.gov.
Comments · 548
-
Re:Already the most powerful UV laser at UR
As a technician on the Omega Laser I guess have a bit of an inside track on what's going on around the LLE.
First you must make a distinction between most powerful(energy/time) laser and most energetic(energy per pulse) laser, this is a distinction not made in the article. The Omega laser is currently the most energetic ultraviolet(frequency tripled Neodymium:Glass) laser in the world now at ~25 Kilojoules per pulse, very soon to be eclipsed by the preliminary first light of the National Ignition Facility. However each "shot" on the system, as they are called, is only a couple hundred picoseconds to a couple nanoseconds long (depending on the shot pulse shape) making it's peak power around a maximum of about 60 Terawatts. This is not the most powerful laser in the world. The Rutherford Appelton laboratory in England has a "Petawatt" system they just commissioned which is capable of at least hundreds of Terawatts of power albeit only with a couple hundred joules of energy per shot.
It is interesting to note that the mechanism the Petawatt upgrade at the LLE will use to achieve it's million billion watts of power in a pulse time of a few picoseconds to hundreds of femtoseconds is called Optical Parametric Chirped Pulse Amplification(OPCPA) and was invented right at THE UofR in the late 1980's!! Chirped Pulse Amplification lasers are the only means to get to petawatt intensities and they are interesting because they are the first technology to allow nuclear reactions to be directly caused by intense light radiation(ie. no implosion/ heating stage as in ICF). This is really interesting because in addition to the spark plug type inertial confinement fusion catalyzing experiments that are planned, the intensity fluences allowed by petawatt lasers approaches (possibly >10^21 watts/sq. inch) what is necessary to do an experiment called "sparking the vacuum" whereby enough energy is placed in a small enough volume of space in a short enough period of time to cause a spontaneous transformation of energy directly into particles(via E=Mc^2). Neat eh? -
Re:petawatt may sound good ...
It may be a lot of power, but it's not a lot of energy. The way they currently make really high-power lasers is to make really, really short pulses. Power = Work / Time. One watt is one joule for one second. To raise the power, you make make Work larger, or you can make Time shorter.
In this case, the pulse width is under half a picosecond. If you divide 1 joule by a tiny number, like 1e-12, then the Power side of that equation gets very large, like 1e12 (a whole terawatt!).
So, while the power rating seems huge (over 1000 times the electrical generating capacity of the entire US), the generators can keep that rate going for a long time, while the laser can't. The actual energy in the laser pulse was only about 600 joules -- say, ten seconds of light from a 60-watt bulb. No need to worry about great piles of coal ash or tons of nuclear waste.
Here is a much more detailed story.
-
Re:Wasn't nmap the tool of controversy from SGI?Thanks I am going to download a copy now!
It still pisses me off today that clueless SGI managers view security through obscurity as a means to an end. Irix today is knows to be one of the least secure versions of Unix out of the box right besied SCO openserver. Hmm how did that happen? Judging by how SGI treated security in the past including this incident shows how Irix got the way it did. Here is sgi's opinion on it.Non biased info is here.
Anyway he should named it something different. A clueless person HR looking at a firing request seeing the words "satan" and "hacker" together certainly cost him his job.
As far as I know its still only a scanning tool like nmap does not actually carry attacks to me knowledge.
Looking at the docs it seems that Satan is cool in terms of you can hide your scanning tracks easier then standard tools like nmap. This is great for a counter cracker attack when I am hacked.
Now lets fire all the system administrators who use this dangerous tool called nmap.
-
Re:How does it know?
Here's some more info:
Press release -
Re:A kid playing with a handgun
Oh really? I thought the main concerns were over the increased leukemia rates near nuclear installations and the long-term storage of hazadous materials.
Increased leukemia rates near nuclear installations? Doesn't work that way. Nuclear plants emit almost no radiation. In fact, the radiation emitted by nuclear power plants is dwarfed by what is emitted from coal power plants.
The volume of all nuclear waste ever produced in the US would fit into a high school gym. Safe installations like Yucca Mountain leave no chance for leakage of waste. Anyway, PU is not that toxic compared to many substances. Take a look at this. -
check out national labsYou can look for internships at other national labs (doing more than just nuclear engineering).
Check AWU about the possibilities at these facilities.
Also, check these:
And there are other other national labs that I did not mention.
-
Re:You needn't worry about that...In which scientific report did you read that stated several kilograms of plutonium in an RTG could kill thousands of people??? Before replying (if you reply) check out this site [nasa.gov]. Here's a direct quote on its safety design:
Look, you are satisfied with the safety of a NASA design based on a NASA and DOE press release. Come on, doesn't that strike you as silly?
What I'm saying is: neither of us is qualified to determine whether these designs are safe. And given NASA's recent track record, I simply do not trust them to make the call by themselves--they have screwed up too much. I'm glad that Greenpeace and lots of other organizations scrutinize, pressure, and protest: it will hopefully get NASA to take more precautions than they seem to be capable of by themselves.
As for the general question of plutonium toxicity, don't take my word for it, look at the CDC site. Or, even take the plutonium-friendly LLNL report, which, in whatever way you look at it, ultimately does argue that there can be around a thousand deaths per kilogram of plutonium released over, say, Munich. But it, like you, assumes that if the additional risk is small compared to other risks, the additional deaths just don't count.
-
Re:self sustaining?Take a look at the National Ignition Facility at Lawrence Livermore National Laboratory.
Basically, it's a stadium sized facility to research and test inertial-confinement fusion (using lasers to produce extreme heat and pressure).
The way they've designed the system, they can produce a total of 1.8 MegaJoule in laser power that all starts from piddly lasers in the nano-joule range.
So, for about $5's worth of electricity and lots of huge slabs of perfect laser glass and crystal, they can more than break even. Theoretically, NIF will generate about 500 TeraWatts of laser power. Sounds promising.
-Cyc
-
FTP _MUCH_ faster than HTTP
It is possible to get approximately 80% of the theoretical maximum throughput of your pipe using a single FTP connection, whereas HTTP can hope for around 60% max for a single connection. The only thing faster than an FTP-based protocol (tftp, pftp) is a raw socket, and they rarely get better then 90%. Most schemes like pftp (parallel ftp, see this paper) are implemented to get as close to theoretical maximum throughput by having multiple data connections transfer the file. Of course you'll see the difference in performance more for large file transfers. The previous comment about HTTP being OK for small files is right on the mark...you will hardly notice a 20% gain when the transfers are only taking a few seconds.
-
Re:Uhhhhhh .....
Does anybody know of anyone that actually needs these abr.'s???
As an employee of Lawrence Livermore National Laboratory, I can honestly answer "yes." You hear it all the time when people talk about the I/O requirements of various simulations. For example, check out this web page about scalable I/O.
As for ftp, be my guest. Dunno if there's anything you'd find interesting there, though. -
Re:Uhhhhhh .....
Does anybody know of anyone that actually needs these abr.'s???
As an employee of Lawrence Livermore National Laboratory, I can honestly answer "yes." You hear it all the time when people talk about the I/O requirements of various simulations. For example, check out this web page about scalable I/O.
As for ftp, be my guest. Dunno if there's anything you'd find interesting there, though. -
Re:Uhhhhhh .....
Does anybody know of anyone that actually needs these abr.'s???
As an employee of Lawrence Livermore National Laboratory, I can honestly answer "yes." You hear it all the time when people talk about the I/O requirements of various simulations. For example, check out this web page about scalable I/O.
As for ftp, be my guest. Dunno if there's anything you'd find interesting there, though. -
Lawrence Livermore National LabsLivermore (SF Bay Area), California.
LLNL offers tours Tuesdays and Thursdays, including:Tour stops include a trip to the National Ignition Facility, the National Atmospheric Release Advisory Center (NARAC), the Biology and Biotechnology building, ASCI White, the country's largest and most powerful supercomputer, and the Discovery Center.
There are some restrictions and rules about visiting, read the linked page for details. -
Easier than....
Well, it's probably easier than building this at home: The National Ignition Facility -
what how why tutorial
You said the manager was tech savvy - bit of an oxymoron, but Check here for a what/how/why tutorial on pthreads.
-
Right...I don't even know what pthreads are, but I can answer this one...
http://www.google.com/search?q=pthreads shows, for starters :
-
Why ?
Why would a manager need to know this ?
Secondly, if he wants to know it, and already understands fork/exec approache, it should be very easy for you developers to exlain what threads are, and the various synchronization primitives provided. Thats basically all he needs to know, the pthreads api in detail shouldn't concern him unless he wants to program.
What he might be concerned about is portability, remember pthreads are not fully supported on many platforms, and also there are many surprises on other platforms.
this chapter should be all he needs to know + portability issues. -
Re:I send you this post to have your advice
Links man, come on! This is Slashdot, you can't just make wild assertions without backing them up with facts.
You're new around here, aren't you?
Since you asked, though, the first thing to realise that you may well have written a program in a high-level language which can way outperform the naive C equivalent yourself. Ever used yacc (or bison)? It's a high-level language compiled to C, after all. I defy you to write a naive (the "naive" part is important) LALR(1) parser in C yourself which outperforms those generated by yacc and related tools.
Slightly less facetious examples:
- C++ beats C
- The functional language Sisal beats High Performance Fortran on numerical simulations. (Note: This is an older paper; Sisal has changed a bit since then.)
Still, this is not the real benefit of high-level languages. The point is that in many situations, you gain a lot without paying too much. There's always C and assembly language below a high-level language if you find you really need it. If you don't, or for those part of an application where you don't, you get rapid development, higher robustness and better maintainability, and that's where you win.
-
Re:built one these in high school
-
Re:Molecular computers may benefit from this...
Synchrotrons are used for x ray crystalography. they can produce X-ray photons at a wide range of frequencies and you can carefully select the photons you want using an x-ray monochromator.
The X-rays will not tell you anything about the nuclei of the molecules you are looking at, as the photons go through the electrons in the crystalised protein they will make an interference pattern, and from that you can calculate the shape of the electron cloud around the molecule.
Note that this gives you no infomation on the quantum state of the nuclei, which is what this quantum computer needs to know.
Nuclear Magnetic Resonance molecular analyisis works in a similar way to Magnetic Resonance Imaging, just on a smaller scale.
for more information click here -
Re:Solaris is slowly dying
You are an idiot. The LLNL MCR system is running linux. Beats the heck out of most high end processing environments. Free UNIXes work great. There is sometimes a link between cost and ability - but not always.
-
Re:question : OSS/free project in this space
-
Re:Almost but not quite
-
Whoa that's cool
image from first link
I love those giant black racks, even if it's not the fastest cluster in the world the Space Odyssey nostalgia is still there.
"My God, it's full of stars!"
-Matt -
Re:I've seen it
You mean like these? It's not a classified project, dude.
-
Re:Interesting Approach on Network
More details -- probably more than you want
:) -- here:
http://www.llnl.gov/linux/mcr/. -
Typo in article
You'd think the EETimes would catch something like this:
nearly the same performance as the ASCII White system
No, it's ASCI White. Accelerated Strategic Computing Initiative, not the text format. -
Mildly Shocked no one has put these up
A bit of Karma whoring here, wish I'd gotten online sooner so that people would see this much earlier:
TheHigh Energy Laser Systems Test Facility (so-called HELSTF). Let's see if Tom's webserver can survive this...This is the laser test facility for the army and navy at White Sands Missile Range. They've got the world's most powerful laser (MIRACL: Mid Infrared Advanced Chemical Laser) there.
Being developed for them, by Livermore by the same guys that are doing the National Ignition Facility is a solid state laser. It works.
Also at HELSTF, and the first functional laser weapon, is Tactical High Energy Laser (aka THEL, and I hate that URL, btw...)
Search TRW for more stuff on lasers as well as Lockmart and Boeing, of course.
-
Re:Liquid N colder than air.. wow!
On other news, U.S. nuclear scientists discovered that ASCI White simulates nuclear explosions much faster than a guy wielding a slide rule...
-
Re:Very Nice if it works
I work in a laser lab, were the laser we work with (an Argon Ion) puts out a maximum 15 watts of power (of multiple wavelengths of visible light) in a ~5mm diameter beam.
Now you can imagine what 100,000 watts will do:)
No , but I've see what 1 000 000 000 000 000 Watts can do. -
LLNL simulation of CO2 ocean sequestrationLast month, Lawrence Livermore National Laboratory scientists released a study modeling the effectiveness of direct injection of carbon dioxide into different oceans at different depths. They've proposed injection into the ocean as a way to slow the accumulation of CO2 in the atmosphere.
Injections were simulated at 800 meters, 1500 meters and 3000 meters for 100 hypothetical years near the Bay of Biscay, New York City, Rio de Janeiro, San Francisco, Tokyo, Jakarta and Bombay.
The models showed that injection at 3000 meters is quite effective at sequestering carbon from the atmosphere for several centuries while injections at shallower depths are less effective. (Not too surprising.) In general, injections into the Pacific Ocean (San Francisco and Tokyo) were more effective than injection at the same depth in the Atlantic Ocean (New York City, Rio de Janeiro and the Bay of Biscay).
The full press release is available here.
-
This maybe a little off topic but
according to this article http://www.llnl.gov/str/Durham.html they did try to create Hydrates with other gasses. Namely with carbon dioxide. Wouldn't this method be a cost effective way of removing the "green house" gasses from the athmosphere? Could someone with chemist background provide some more info plz.?
-
It's not Frozen MethaneThey are talking about Methane Hydrate. There is no way methane can be frozen solid at the range of temperatures and pressures found in the ocean floor. It will be above it's critical temperature.
More information can be found under methane hydrate in google or:
among other. It's really an interesting compound and future power source.
-
Shaped charges project plasma jet, not molten jetLars's link is a very interesting one.
A lot of respondents here have said that a shaped charge projects a jet of molten copper. Years ago, when I used to subscribe to sci.military, I made that mistake. Many of the correspondents there didn't hesitate to quickly set me straight, and explain that the shaped charge projects a plasma jet.
Here is an article from Lawrence Livermore Labs with some excellent pictures of the jets in action.
Here is another article.
And here are some animations.
1 meg avi
770K avi
10 meg aviThis newspaper article gets the scale wrong. It says the jet travels at around 1000 miles per hour, ie not much more than the speed of sound, whereas the Lawrence Livermore article I linked to above says the jet travels at 10,000 kilometers per second. Michael Smith, the telegraph's defence correspondent, was off by a factor of just 57,000,000.
-
88 percent?
-
Yes
Well, I know I'm in the minority here, but as an employee of Lawrence Livermore National Laboratory, whose security rules come from the U.S. Department of Energy I can say that all of our janitors have a background check.
-
Re:I hope not
Also, plutonium is only an alpha emmiter. This form of radiation canot penetrate a sheet of paper or your dead epidermis. So a chunk of plutonium sitting right next to you would have no effect at all. It is only dangerous if you inhale it, or, to a lesser extent, ingest it.
Here is an excellent Lawrence Livermore Nat'l Lab paper on plutonium toxicity. Pay special attention to the section on plutonium in drinking water. It shows how plutonium would simply settle out of the water, producing almost no health risk, even in large quantities. -
Re:Unfortunately...
Here is an Lawrence Livermore report on plutonium toxicity. Plutonium oxide actually is not that toxic when ingested. It passes out of your system fairly harmlessly in small quantities. In larger quantities (~5 grams) it still probably won't kill you, but only increase the risk of cancer somewhat. Plutonium is only really deadly if inhaled.
-
ubc!
UBC: An Efficient Unified I/O and
Memory Caching Subsystem for NetBSDChuck Silvers
The NetBSD Project
chuq@chuq.com, http://www.netbsd.org/
Abstract
This paper introduces UBC ("Unified Buffer Cache"), a design for unifying the filesystem and virtual memory caches of file data, thereby providing increased system performance. In this paper we discuss both the traditional BSD caching interfaces and new UBC interfaces, concentrating on the design decisions that were made as the design progressed. We also discuss the designs used by other operating systems to solve the same problems that UBC solves, with emphasis on the practical implications of the differences between these designs. This project is still in progress, and once completed will be part of a future release of NetBSD.
IntroductionModern operating systems allow filesystem data to be accessed using two mechanisms: memory mapping, and I/O system calls such as read() and write(). In traditional UNIX-like operating systems, memory mapping requests are handled by the virtual memory subsystem while I/O calls are handled by the I/O subsystem. Traditionally these two subsystems were developed separately and were not tightly integrated. For example, in the NetBSD operating system[1], the VM subsystem ("UVM"[2]) and I/O subsystem each have their own data caching mechanisms that operate semi-independently of each other. This lack of integration leads to inefficient overall system performance and a lack of flexibility. To achieve good performance it is important for the virtual memory and I/O subsystems to be highly integrated. This integration is the function of UBC.
BackgroundIn order to understand the improvements made in UBC, it is important to first understand how things work without UBC. First, some terms:
- "buffer cache":
A pool of memory allocated during system startup which is dedicated to caching filesystem data and is managed by special-purpose routines.
The memory is organized into "buffers," which are variable-sized chunks of file data that are mapped to kernel virtual addresses as long as they retain their identity.
- "page cache":
The portion of available system memory which is used for cached file data and is managed by the VM system.
The amount of memory used by the page cache can vary from nearly 0% to nearly 100% of the physical memory that isn't locked into some other use.
- "vnode":
The kernel abstraction which represents a file.
Most manipulation of vnodes and their associated data are performed via "VOPs" (short for "vnode operations").
The major interfaces for accessing file data are:
- read() and write():
The read() system call reads data from disk into the kernel's cache if necessary, then copies data from the kernel's cached copy to the application's address space. The write() system call moves data the opposite direction, copying from the application's address space into the kernel's cache and eventually writing the data from the cache to disk. These interfaces can be implemented using either the buffer cache or the page cache to store the data in the kernel.
- mmap():
The mmap() system call gives the application direct memory-mapped access to the kernel's page cache data. File data is read into the page cache lazily as processes attempt to access the mappings created with mmap() and generate page faults.
In NetBSD without UBC, read() and write() are implemented using the buffer cache. The read() system call reads file data into a buffer cache buffer and then copies it to the application. The mmap() system call, however, has to use the page cache to store its data since the buffer cache memory is not managed by the VM system and thus not cannot be mapped into an application address space. Therefore the file data in the buffer cache is copied into page cache pages, which are then used to satisfy page faults on the application mappings. To write modified data in page cache pages back to disk, the new version is copied back to the buffer cache and from there is written to disk. Figure1 shows the flow of data between the disk and the application with a traditional buffer cache.
This double-caching of data is a major source of inefficiency. Having two copies of file data means that twice as much memory is used, which reduces the amount of memory available for applications. Copying the data back and forth between the buffer cache and the page cache wastes CPU cycles, clobbers CPU caches and is generally bad for performance. Having two copies of the data also allows the possibility that the two copies will become inconsistent, which can lead to application problems which are difficult to debug.
The use of the buffer cache for large amounts of data is generally bad, since the static sizing of the buffer cache means that the buffer cache is often either too small (resulting in excessive cache misses), or too large (resulting in too little memory left for other uses).
The buffer cache also has the limitation that cached data must always be mapped into kernel virtual space, which puts an additional artificial limit on the amount of data which can be cached since modern hardware can easily have more RAM than kernel virtual address space.
To solve these problems, many operating systems have changed their usage of the page cache and the buffer cache. Each system has its own variation, so we will describe UBC first and then some other popular operating systems.
So what is UBC anyway?UBC is a new subsystem which solves the problems with the two-cache model. In the UBC model, we store file data in the page cache for both read()/write() and mmap() accesses. File data is read directly into the page cache without going through the buffer cache by creating two new VOPs which return page cache pages with the desired data, calling into the device driver to read the data from disk if necessary. Since page cache pages aren't always mapped, we created a new mechanism for providing temporary mappings of page cache pages, which is used by read() and write() while copying the file data to the application's address space. Figure2 shows the changed data flow with UBC.
Figure 1: NetBSD before UBC.
Figure 2: NetBSD with UBC.
UBC introduces these new interfaces:
- VOP_GETPAGES(), VOP_PUTPAGES()
These new VOPs are provided by the filesystems to allow the VM system to request ranges of pages to be read into memory from disk or written from memory back to disk. VOP_GETPAGES() must allocate pages from the VM system for data which is not already cached and then initiate device I/O operations to read all the disk blocks which contain the data for those pages. VOP_PUTPAGES() must initiate device I/Os to write dirty pages back to disk.
- ubc_alloc(), ubc_release()
These functions allocate and free temporary mappings of page cache file data. These are the page cache equivalents of the buffer cache functions getblk() and brelse()[3]. These temporary mappings are not wired, but they are cached to speed repeated access to the same file. The selection of which virtual addresses to use for these temporary mappings is important on hardware which has a virtually-addressed CPU data cache, so the addresses are carefully chosen to be correctly aligned with the preferred addresses for user file mappings, so that both kinds of mappings can be present at the same time without creating coherency problems in the CPU cache. It is still possible for applications to create unaligned file mappings, but if the application lets the operating system choose the mapping address then all mappings will always be aligned.
- ubc_pager
This is a UVM pager which handles page faults on the mappings created by ubc_alloc(). (A UVM pager is an abstraction which embodies knowledge of page-fault resolution and other VM data management. See the UVM paper[2] for more information on pagers.) Since its only purpose is to handle those page faults, the only action performed by ubc_pager is to call the new VOP_GETPAGES() operation to get pages as needed to resolve the faults.
In addition to these new interfaces, several changes were made to the existing UVM design to fix problems which were glossed over in the original design.
Previously in UVM, vnodes and uvm_objects were not interchangeable, and in fact several fields were duplicated and maintained separately in each. These duplicate fields were combined. At this time there's still a bit of extra initialization the first time a struct vnode is used as a struct uvm_object, but that will be removed eventually.
Previously UVM only supported 32-bit offsets into uvm_objects, which meant that data could only be stored in the page cache for the first 4 GB of a file. This wasn't much of a problem before since the number of programs which wanted to access file offsets past 4 GB via mmap() was small, but now that read() and write() also use the page cache interfaces to access data, we had to support 64-bit uvm_object offsets in order to continue to allow any access to file offsets past 4 GB.
What do other operating systems do?
The problems addressed by UBC have been around for a long time, ever since memory-mapped access to files was first introduced in the late 1980's. Most UNIX-like operating systems have addressed this issue one way or another, but there are considerable differences in how they go about it.
The first operating system to address these problems was SunOS[4,5], and UBC is largely modeled after this design. The main differences in the design of the SunOS cache and UBC result from the differences between the SunOS VM system and UVM. Since UVM's pager abstraction and SunOS's segment-driver abstraction are similar, this didn't change the design much at all.
When work on UBC first began over two years ago, the other design that we examined was that of FreeBSD[6], which had also already dealt with this problem. The model in FreeBSD was to keep the same buffer cache interfaces to access file data, but to use page cache pages as the memory for a buffer's data rather than memory from a separate pool. The result is that the same physical page is accessed to retrieve a given range of the file regardless of whether the access is made via the buffer cache interface or the page cache interface. This had the advantage that filesystems did not need to be changed in order to take benefit from the changes. However, the glue to do the translation between the interfaces was just as complicated as the glue in the SunOS design and failed to address certain deadlock problems (such as an application calling write() with a buffer which was a memory mapping of the same file being written to), so we chose the SunOS approach over this one.
The approach taken by Linux[7] (as of kernel version 2.3.44, the latest version at the time this paper was written) is actually fairly similar to the SunOS design also. File data is stored only in the page cache. Temporary mappings of page cache pages to support read() and write() usually aren't needed since Linux usually maps all of physical memory into the kernel's virtual address space all the time. One interesting twist that Linux adds is that the device block numbers where a page is stored on disk are cached with the page in the form of a list of buffer_head structures. When a modified page is to be written back to disk, the I/O requests can be sent to the device driver right away, without needing to read any indirect blocks to determine where the page's data should be written.
The last of the operating systems we examined, HP-UX, takes a completely different stance on the issue of how to cache filesystem data. HP-UX continues to store file data in both the buffer cache and the page cache, though it does avoid the extra of copying of data that is present in pre-UBC NetBSD by reading data from disk directly into the page cache. The reasoning behind this is apparently that most files are only accessed by either read()/write() or mmap(), but not both, so as long as both mechanisms perform well individually, there's no need to redesign HP-UX just to fix the coherency issue. There is some attempt made to avoid incoherency between the two caches, but locking constraints prevent this from being completely effective.
There are other operating systems which have implemented a unified cache (eg. Compaq's Tru64 UNIX and IBM's AIX), but we were unable to find information on the design of these operating systems for comparison.
Performance
Since UBC is unfortunately not yet finished, a detailed performance analysis would be premature. However, we have made some simple comparisons just to see where we stand. The hardware used for this test was a 333MHz Pentium II with 64MB of RAM and a 12GB IDE disk. The operations performed were a series of "dd" commands designed to expose the behaviour of sequential reads and writes. We create a 1GB file (which is much larger than the physical memory available for caching), then overwrite this file to see the speed at which the data modifications caused by the write() are flushed to disk without the overhead of allocating blocks to the file. Then we read back the entire file to get an idea of how fast the filesystem can get data from the disk. Finally, we read the first 50MB of the file (which should fit entirely in physical memory) several times to determine the speed of access to cached data. See Table 1 for the results of these tests.
Table 1: UBC performance comparison. Experiment Run Time (seconds) Input Output Size NetBSD NetBSD FreeBSD Linux 1.4.2 with UBC 3.4 2.2.12-20smp raw device /dev/null 1GB 72.8 72.7 279.3 254.6 /dev/zero new file 1GB 83.8 193.0 194.3 163.9 /dev/zero overwrite file 1GB 79.4 186.6 192.2 167.3 non-resident file /dev/null 1GB 72.7 86.7 279.3 254.5 non-resident file /dev/null 50MB 3.6 4.3 13.7 12.8 resident file /dev/null 50MB 3.6 0.8 4.1 11.5 repeat above /dev/null 50MB 3.6 0.8 0.7 4.5 repeat above /dev/null 50MB 3.6 0.8 0.7 0.8 repeat above /dev/null 50MB 3.6 0.8 0.7 0.8
The great disparity in the results of the first four tests on the three non-UBC operating systems is due to differences in performance of their IDE disk drivers. All of the operating systems tested except NetBSD with UBC do sequential buffered reads from a large file at the same speed as reads from the raw device, so all we can really say from this is that the other caching designs don't add any noticable overhead. For reads, the UBC system is not yet running at device speed, so there's still room for improvement. Further analysis is required to determine the cause of the slowdown.
UBC obviously needs much improvement in the area of write performance. This is partly due to UVM not being very efficient about flushing modified pages when memory is low and partly because the filesystem code currently doesn't trigger any asynchronous writes to disk during a big sequence of writes, so the writes to disk are all started by the inefficient UVM code. We've been concentrating on read performance so far, so this poor write performance is not surprising.
The interesting part of this test series is the set of tests where we read the same 50MB file five times. This clearly shows the benefit of the increased memory available for caching in the UBC system over NetBSD without UBC. In NetBSD 1.4.2, all five reads occured at the speed of the device, whereas in all the other systems the reads were completed at memory speed after several runs. We have no explanation for why FreeBSD and Linux didn't complete the second 50MB read at memory speed, or why Linux didn't complete even the third read at memory speed.
Conclusion
In this paper we introduced UBC, a improved design for filesystem and virtual memory caching in NetBSD. This design includes many improvements over the previous design used in NetBSD by:
- Eliminating double caching of file data in the kernel (and the possibility of cache incoherency that goes with it) when the same file is accessed via different interfaces.
- Allowing more flexibility in how physical memory is used, which can greatly improve performance for applications whose data fits in physical memory.
Availability
This work will be part of a future release of NetBSD once it is completed. Until then, the source code is available in the "chs-ubc2" branch in the NetBSD CVS tree, accessible via anonymous CVS. See http://www.netbsd.org/Sites/net.html for details.
This being a work-in-progress, there is naturally much more work to do! Planned work includes:
- Integration of UBC into the NetBSD development source tree and performance improvement. The pagedaemon needs to be enhanced to deal with the much larger amount of page-cache data which will be dirty.
- Elimination of the data copying in read() and write() via UVM page loanout when possible. This could be done without UBC too, but with UBC it will be zero-copy instead of one-copy (from buffer cache to page cache).
- Elimination of the need to map pages to do I/O to them by adding a page list to struct buf and adding glue in bus_dma to map pages temporarily for hardware that actually needs that.
- Adding support for "XIP" (eXecute In Place). This will allow zero-copy access to filesystem images stored in flash roms or other memory-mapped storage devices.
- Adding support for cache coherency in layered filesystems. (The current UBC design does not address caching in layered filesystems.)
Acknowledgments
We would like to thank everyone who helped review drafts of this paper. Special thanks to Chuck Cranor!
Bibliography
1 The NetBSD Project.
The NetBSD Operating System.
See http://www.netbsd.org/ for more information.2 C. Cranor and G. Parulkar.
The UVM Virtual Memory System.
In Proceedings of the 1999 USENIX Technical Conference, June 1999.3 Marice J. Bach.
The Design of the UNIX Operating System.
Prentice Hall, February 1987.4 J. Moran, R. Gingell and W. Shannon.
Virtual Memory Architecture in SunOS.
In Proceedings of USENIX Summer Conference, pages 81-94. USENIX, June 1987.5 J. Moran.
SunOS Virtual Memory Implementation.
In Proceedings of the Spring 1988 European UNIX Users Group Conference, April 1988.6 The FreeBSD Project.
The FreeBSD Operating System.
See http://www.freebsd.org/ for more information.7 L. Torvalds, et al.
The Linux Operating System.
See http://www.linux.org/ for more information.About this document
... UBC: An Efficient Unified I/O and
Memory Caching Subsystem for NetBSDThis document was generated using the LaTeX2HTML translator Version 98.1p1 release (March 2nd, 1998)
Copyright 1993, 1994, 1995, 1996, 1997, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
The command line arguments were:
latex2html -split 0 -no_navigation ubc_usenix.The translation was initiated by Chuck Cranor on 2000-04-25
Chuck Cranor
2000-04-25 - "buffer cache":
-
Re:OS'es for the supercomputers...
Here is a link for ASCI Red @ Sandia National Labs.
From the article:
The system uses two operating systems to make the computer both familiar to the user (UNIX) and non-intrusive for the scalable application (Cougar). And it makes use of Commercial Commodity Off The Shelf (CCOTS) technology to maintain affordability.
Hmm, I see one familiar OS in there... -
Re:why so keen on earth-sized?
Yes. And the Carbon, Iron, Silicon, and any other heavier element would be left to form a crunchy center. The hydrogen would eventualy float to a certain point where the buoyancy in the atmosphere would be equal to the gravitational pull of the planet.
Also, it's been hypothesized that any Hydrogen at the center would be under such immense pressure it would change into a metallic state.
B -
Re:Wont die
This is very interesting.
However: I am just curious. What exactly is it they *do* with that big laser in the NIF project? Do they just shoot it at various household objects, watch them blow up, and giggle? Are they using it to etch images onto the moon, like in that Batman episode? Is this like a supercomputer, where various groups have things that they need done that would involve a big laser, and they rent time on the laser?
Seriously, though-- the web page doesn't give a lot of help, but it seems to indicate that they are using the laser somehow to create a containment field in which to keep plasma which they are trying to coax into sustaining fusion reactions for, um, research into fusion energy. Is this last guess the accurate one? Or are there other uses for the biggest laser in the world as well?
I am just curious. Thank you. -
Such a shame...
Just another example of the short-sighted nature of the Bush administration. After all, LLNL are already working on things like Anthrax Detection and other Bioterrorist agents.
I doubt that anything organized under Tom Ridge is going to be nearly as effective in bringing advances in the realms of fighting terrorists, let alone helping to make the advances that LLNL have brought to all of science. -
Such a shame...
Just another example of the short-sighted nature of the Bush administration. After all, LLNL are already working on things like Anthrax Detection and other Bioterrorist agents.
I doubt that anything organized under Tom Ridge is going to be nearly as effective in bringing advances in the realms of fighting terrorists, let alone helping to make the advances that LLNL have brought to all of science. -
Late to the Party
As someone who has spec'd out and designed industrial grade battery banks for electric utility substations, I would not suggest doing anything this person has done in a household that doesn't have these things:
- Proper Ventilation (Like those hoods you see in Chem Class)
- Proper Fire Safety Equipment (Chemical Fire Extinguisher as well as conventional water)
- Proper Acid Retention and/or Mitigation (Chemical type Acid neutraliser) and a way to keep all of that acid from eating up your carpet)
Also, the household shouldn't have any of these things:
- Small Children
- Smokers (People who smoke)
- Pets
- Wives
- Intelligent Human Beings
Oh, let's not forget Safety Glasses and a Hard hat while you sit at your computer... dufus should have gotten a regular old APC.
-
Re:A very simple question:
-
Totally different technologies
The trend seems to be toward simulating 3D with high-resolution flat screens, though.
These are completely different technologies. The first is an "actual" 3D display. The voxels have a true location in 3D space, for instance. People can view it from any angle with no equipment.
The second appears to be just a large screen. People wear shutter or polarized glasses to send different images to the left and right eyes.
While the second techology is great, especially for high-resolution display to a single person, it really is annoying when used with multiple people with different locations in space.
Since there is only one set (left and right) images on the flat screen, only one viewpoint can be chosen. If a group of people is sufficiently far from the screen, or sufficiently close together in the room, it's fine. But if you let the people wander around the room, you start getting perspective problems that really make collaborative viewing troublesome.
I have a feeling that we will be seeing voxel-based visualization like the one mentioned in this post more and more often. It's just more natural to use.
As someone who is in the field of high-resolution scientific visualization (that's me on the left), I certainly hope that technology will move in this direction. -
One area I am amazed they don't touch on...
...is the use of orbital lasers for antiaircraft work (unless I just missed it). Reason being that if you can force an enemy airforce below 30k ft (9k m) you have an immense advantage in an airwar. (Below 30k feet the laser's beam tends to have too much in the way of atmospheric problems).
Another thing that startled me is that they are talking about using HF (hydrogen flouride) lasers. While definitely cheaper than DF (deuterium flouride), their atmospheric propogation sucks raw eggs.
Additionally, no mention of solid state lasers like the one Lawrence Livermore National Labs is developing for HELSTF is made. FEL's are, but they're slower going work than the SSL's seem to be. -
One area I am amazed they don't touch on...
...is the use of orbital lasers for antiaircraft work (unless I just missed it). Reason being that if you can force an enemy airforce below 30k ft (9k m) you have an immense advantage in an airwar. (Below 30k feet the laser's beam tends to have too much in the way of atmospheric problems).
Another thing that startled me is that they are talking about using HF (hydrogen flouride) lasers. While definitely cheaper than DF (deuterium flouride), their atmospheric propogation sucks raw eggs.
Additionally, no mention of solid state lasers like the one Lawrence Livermore National Labs is developing for HELSTF is made. FEL's are, but they're slower going work than the SSL's seem to be. -
Mirror
Just in case the site gets slashdotted, here's a mirror.