Slashdot Mirror

P.S. the Intel 600P NVMe drive is also horrid, don't buy it.

http://apollo.backplane.com/DF...

-Matt

According to the Linux kernel, Intel NVMe devices have the block stack stick to certain alignments for performance reasons. Now quoting the above article: "All tests were done on a DragonFlyBSD". I doubt Intel did the same enabling there as they did for Linux.

Smoke. Total and complete nonsense. Why would I want to buy their over-priced octane junk verses a Samsung 951* or 960* NVMe drive? Far more storage for around $115-$130, 1.4 GBytes/sec consistent read performance, decent write performance, and decent durability.

P.S. the Intel 600P NVMe drive is also horrid, don't buy it.

http://apollo.backplane.com/DF...

-Matt

The main bottleneck for a modern cpu are main memory accesses. What is amazing is that all of the prediction and the huge (192+) number of uOPS that can be on the deck at once is able to absorb enough of the massive latencies main memory accesses cause to bring the actual average IPC back towards roughly ~1.4. And this is with the cache misses causing *only* around ~6 GBytes/sec worth of main memory accesses per socket (with a maximum main memory bandwidth of around 50 GBytes/sec per socket, if I remember right).

Without all of that stuff, a single cache miss can impose several hundred clock cycles of latency and destroy average IPC throughput.

So, for example, here is a 16 core / 32 thread - dual socket E5-2620v4 @ 2.1 GHz system doing a bunch of parallel compiles, using Intel's PCM infrastructure to measure what the cpu threads are actually doing:

http://apollo.backplane.com/DF...

Remember, 32 hyperthreads here so two hyperthreads per core. Actual physical core IPC (shown at the bottom) is roughly 1.39. At 2.1-2.4 GHz this system is retiring a total of 55 billion instructions per second.

In this particular case, being mostly integer math, the bottleneck is almost entirely memory-related. It doesn't take much to stall out a core. If I were running FP-intensive programs instead it would more likely be bottlenecked in the FP unit and not so much on main memory. Also note the temperature... barely ~40C with a standard copper heatsink and fan. Different workloads will cause different levels of cpu and memory loading.

-Matt

If we're talking about bulk builds, for any language, there is going to be a huge amount of locality of reference that matches well against caches. shared text RO, lots of shared files RO, stack use is localized (RW), process data is relatively localized (RW), and file writeouts are independent. Plus any decent scheduler will recognize the batch-like nature of the compile jobs and use relatively large switch ticks. For a bulk build the scheduler doesn't have to be very smart, it just needs to avoid moving processes around between cpus excessively so and be somewhat HW cache aware.

Data and stack will be different, but one nice thing about bulk builds is that there is a huge amount of sharing of the text (code) space. Here's an example of a bulk build relatively early in its cycle (so the C++ compiles aren't eating 1GB each like they do later in the cycle when the larger packages are being built):

http://apollo.backplane.com/DF...

Notice that nothing is blocked on storage accesses. The processes are either in a pure run state or are waiting for a child process to exit.

I've never come close to maxing out the memory BW on an Intel system, at least not with bulk builds. I have maxed out the memory BW on opteron systems but even there one still gets an incremental improvement with more cores.

The real bottleneck for something like the above is not the scheduler or the pegged cpus. The real bottleneck is the operating system which is having to deal with hundreds of fork/exec/run/exit sequences per second and often more than a million VM faults per second (across the whole system)... almost all on shared resources BTW, so it isn't an easy nut for the kernel to crack (think of what it means to the kernel to fork/exec/run/exit something like /bin/sh hundreds of times per second across many cpus all at the same time).

Another big issue for the kernel, for concurrent compiles, is the massive number of shared namecache resources which are getting hit all at once, particularly negative cache hits for files which don't exist (think about compiler include path searches).

These issues tend to trump basic memory BW issues. Memory bandwidth can become an issue, but it will mainly be with jobs which are more memory-centric (access memory more and do less processing / execute fewer instructions per memory access due to the nature of the job). Bulk compiles do not fit into that category.

-Matt

Yes it's the cool Amiga stuff Matt Dillon. I still remember getting DICE on two Amiga Shoper coverdisks and the ever useful FMS (thanks Matt!). It's interesting to note that Dave Jones (who later tweaked FMS) is a Red Hat Linux kernel hacker.

Matt has posted a very in-depth PDF whitepaper describing the Hammer filesystem. A very interesting read!

The other problem is simply the stock market itself. Getting 10% a year out of it might be possible over the long haul, but volatility in the time-frame of a decade could give you anywhere from -20% to +20% a year, or worse depending on what you are invested in. Plus if you have to take money out regularly then you have to take profits and you wind up paying a big chunk of those profits in taxes to the government and, depending on your income level, you can even push yourself into a different tax bracket. It isn't cut-and-dry.

On the flip side, it might be possible to take a tax credit for the money spent on the solar system, and if you can manage to spend the money up front it does in fact improve the value of your home and also significantly improves your monthly cash flow. Some people tend to burn their cash reserves regardless of what they think they've saved and burning it on something more worthwhile, like a solar system instead of a vacation, would definitely be an improvement. If you see the solar as a long-term investment then those improvements can, in fact, be more beneficial to you.

In any case, a standard California home does not need a 4KW system. 2KW will do just fine. I have a 2.5KW system and a fairly large house and if I didn't have 12 computers running 24x7 my electric bill would be nearly zero.

You get the most payback by cutting away the top tier electric rate. You hit diminishing returns if you cut away the entire electric bill. A 2KW system costs a lot less then a 4KW system. The best price point for a consumer inverter such as a Sunny Boy is 2.5KW.

I strongly recommend that anyone getting a solar system get it professionally installed. A solar panel system with a high voltage DC inverter setup (~400VDC, typically one or two strings hooked in series), grid-tie (no battery), requires zero maintainance.

Another thing people should consider, even before considering a PV system, is to get a solar water heating system. These don't use PV panels but instead convert sunlight directly into heat (pipes and glass basically). The efficiency is very good and the cost is far lower then a PV system, and will chop off a good chunk of the gas bill from your water heater.

My Solar System

-Matt

The real question here is how will these panels stack up to current poly panels with regards to their life span? All solar panels degrade over time - that is, produce less power as they get older. Rule of thumb for a poly panel is around 25 years. While there are many types of panels only a few are actually in mass production and have the required life spans. If you are looking to install solar now, polycrystalline panels are what you want to get.

1.5 to 2 KW worth of panels is enough to run a typical house unless you have a machine room. Even if you use more power then your panels can produce, it's actually all to the good because it means the panels are recovering the highest-tier electricity costs for you, dropping you down to a lower tier with your utility company.

You don't want batteries unless you are off-grid, and most people will be on-grid. There are many grid-tie solutions available and costs have come down considerably over the years. Batteries are of course essential if you are off-grid but knowing the many hackers here I'm sure many of you would like to be able to disconnect from the utility completely, survive blackouts, and so forth... but generally speaking, the batteries and equipment required to do that adds a lot to the cost of the system and involve considerably more maintenance and worry.

A straight grid-tie system is completely maintenance free. I literally have not had to touch my system since the day it was installed. I just pop into the garage and stare at the cumulative power display every so often :-)

http://apollo.backplane.com/Solar/

-Matt

Solar water heating is very inexpensive and environmentally friendly (because no solar cells are actually needed, just something to soak up the sun's heat and a heat exchanger). You generally want to get a closed system heat exchanger, with a separate fluid loop, and not actually loop the water heater's water through the solar unit.

Battery backup is *NOT* inexpensive, nor is it environmentally friendly. Only lead-acid batteries have the kind of capacity required and they need maintainance and space and have relatively short lifespans (5-10 years typically). They require a separate charging system and a transfer switch. In short... if you have a good connection to the utility, putting together a battery system is not worth the cost.

The cheapest most environmentally sensitive solar electric system are standard solar panels and a direct grid-tie inverter. Not the shingles or any of the other experimental junk... they just don't have the life span or the efficiency. Zero maintainance, very long life. This is what I have on my roof.

In terms of (almost) zeroing out your electricity bill with net-meetering... well, it is fairly inexpensive if you have a newer home with energey efficient appliances. My system is somewhat bigger then a standard home needs, 2.5KW, and I can't zero out my electricity bill because I have a machine room. Note however that no solar system can even come close to the electricity requirements of a home Air Conditioner. If you need air conditioning you will never be able to zero-out your electricity bill with a standard 'home' solar electric system.

Solar Cell Manufacturing has gotten a lot better over the years. The environmental cost for manufacturing a panel is something like 6 months now vs the 30 year+ lifespan of the panel. Direct grid-tie inverters take up very little space and require no maintainance whatsoever. Generally you want to use a high voltage inverter, where the solar panels are linked in series instead of in parallel. Such inverters are a lot less bulky then LV systems (and the wiring is a lot less bulky too because it is high-voltage and low-current instead of low-voltage and high-current). My recommendation is a Sunny Boy direct-tie inverter. Never use an inverter which requires a fan.

Some states, in particularly California, have extremely good rebate programs. The Federal tax credit is crap.

Neighbors of mine have tried the shingles, and have tried flexible solar mats on their roofs, with terrible results.

http://apollo.backplane.com/Solar/

You may want to look at backplane.

Here are some more examples of flash shots. My parent's maintain a wonderful garden and they were on the Secret Garden Tour in Berkeley this year. I took a lot of shots of flowers. Unlike the NextFest shots, which were in a dark warehouse, nearly all of the garden shots were in bright sunlight. Proper use of the flash filled in the shadows and narrowed the contrast range, producing some incredible flower shots.

I should have saved some of the test shots I took with and without the flash but I didn't. Suffice it to say that most of these shots would not look anywhere near as good as they look without the flash to provide shadow fill.

Secret Garden Tour Shots

(1) Get a good bounce flash, e.g. like the Canon 420 EX for Canon EOS cameras.

(2) Get a diffusor ($0.01 worth of milky plastic, usually $5-$10 retail). For most shots either bounce the flash off the ceiling or use the diffusor. Never use a direct-pointing flash unless you have no choice (e.g. shots from a distance).

(3) Learn how to properly use Tv, Av, and Manual modes with the flash to properly fill the image. I generally either use Av with the flash sync fixed at 1/200, or Manual mode to control how much of the shot is from natural light and how much is from the flash (on the Canon the flash exposure is automatic when operating in manual mode though for obvious reasons you have to be more careful about its exposure range capabilities).

(4) The proper use of a flash for fill is even more important in bright sunlight due to the huge contrast between shadow and sunlight (especially on faces). I almost universally use the flash with the diffusor for daylight shots.

And that's pretty much it. Most people don't use flashes properly, but it doesn't take much exposure :-) to at least double and maybe even triple the number of good shots you take in a day. As usual, I just happen to have some great examples:

The BalloonHat guy at NextFest

Flower shots from my folks Garden

All of these pictures were taken with my Canon-EOS10D, 420EX flash (used mainly for shadow fill), and Sigma 20mm 1:1.8 EX DG prime lens. The shots were taken hand-held in AP mode using F4.0-F16 depending on the conditions. This particular lens produces ultra sharp results at F4.0-F13 or so. The 10D (and 300D) use a 6 MPix low-noise CMOS sensor and you can see it in the above shots.

Insofar as all the discussion goes, from my point of view it all comes down to three things: Lens Quality, Sensor Quality, and Dynamic Range (of the exposure). SLR's like the 10D have gotten good enough that I don't use film any more. The lens quality is there (being an SLR and taking the same lens as the film EOS's), sensor quality is there, and while dynamic range still needs another 2-4 bits of resolution for my comfort it's still good enough for 99% of the shots I take. Film is dead, digital rendition at 11!

And I tend to agree with the few other obviously experienced comments (verses the bozo comments from people that don't know jack about taking photographs). You first need to know how to take a picture before you can take a good one. Then comes lens and sensor noise. A lens hood is important, and a good flash (articulated for bounce shots and also be sure to have a diffusor handy) is very important (even when you don't think you need it). For example, most of those flower shots I took were with flash+diffusor, even though it was a bright sunny day outside. The flash was used primarily to fill in some of the shadow (one way to correct for limited dynamic range but it also makes the shots look a lot better).

-Matt

In terms of batteries... you only ever use batteries if you are off-grid or if the grid is really unreliable. If you are tied to the grid you do not usually use batteries... the Solar system goes through an inverter and powers the house, and any excess is fed back to the grid (running your meter backwards). The grid acts as the 'battery' in this case. If you are not producing enough to power your house, the remainder is fed to you from the grid. Grid-tie systems without batteries are the *simplest* and *cheapest* type of PV system you can buy, but you are still talking about $15-$20K for the system you see above. Systems with batteries cost a lot more (add another $5-$10K at least) plus you have maintainance requirements (Batteries wear out), and you need an ATS (Automatic Transfer Switch) which itself is expensive. On the otherhand, systems without batteries are nearly 100% maintinance free. Without batteries means that if the grid goes down, you go down too. Most people live in areas where the grid is reliable enough that there's no point doing battery storage. Also keep in mind that battery systems have much higher losses then grid-tie systems because you have a loss charging the battery and another one pulling energy out of the battery on top of the inverter losses.

Typical home AC systems eat 3kW while larger home systems eat 5kW (for homes, not apartments). Lets say you had an AC unit that eats 3kW while operating. A 2.5KW grid-tie system producing 16KwH/day would be able to run such a unit for 5 hours. As you can see, the PV system itself would not be able to power the AC unit alone, it would definitely need help from the grid, but if you only ran the AC for 5 hours the PV system would run your meter backwards the rest of the time and make up for it.

Five hours is not usually enough running time to really be able to cool a house unless you live in dessert conditions where it gets cold at night, in which case you really need to cool the house down at night so it stays cool enough so you don't have to turn on the AC until the afternoon (12-5p.m.)

So, generally speaking, trying to run an AC system with a PV (solar power) system is a bad investment. You could try running a smaller AC system but the sun generates something like a kilowatt of heat per square meter and it will easily overpower a small AC system if you do not have good insulation. Note, in particular, that if you do not have good wall insulation the sun is likely to overpower your AC during the afternoon when the sun is hitting the side of the house instead of the roof.

You would be far wiser to invest in passive technologies such as improved insulation and infra-red reflective shading. If water is cheap (or even if it isn't), a swamp cooler (rooftop evaporator) is often a great investment... it's cheap and it provides some cooling at a far lower cost then AC in electricity use. I've heard people mention GeoThermal, and it does work, but if GeoThermal is not put in when the house is actually built it can't take advantage of an under-the-house installation. Getting enough suds out o

Even medium format is taking a beating. Something like the EOS-10D (which I have) may not quite have the color depth or contrast to beat medium format, but the Canon sensor has virtually no noise and plenty sufficient resolution to take probably 70%+ of medium format's market share, and the convenience is two-orders of magnitude greater then anything you will ever get with any sort of film-based system. Hell, it's even more convenient to take multiple shots and stitch them together for poster-sized HR output then it is to use a medium format camera. I take thousands of shots now where I used to only take hundreds with my film SLR.

The new generation of photoprinters, like the i960, is probably an even worse indicator for Kodak. While it is still less expensive to send your digital photos off to a professional shop for printing the fact of the matter is that high quality consumer printing technology is now widely available and that spells doom for traditional companies like Kodak which depend on marketshare and volume to make their money. Photoprinting is now a market that cannot be 'cornered'.

In anycase, I wrote up a review of my 10D which is available:

Canon10D review

As soon as I saw Matt Dillon's name associated with BSD, I got really excited. I mean, who knew that the ol' gunslinger Marshal Matt Dillon became a programmer?

Then, I realized I must have been thinking about the wrong Matt Dillon, but I still thought it was weird that the guy from There's Something About Mary became involved in a BSD project.

Finally I remembered the other Matt Dillon who developed the DICE C compiler for the Amiga back in the good old days.

Two options I haven't seen anyone mention yet are PostgreSQL eRServer 1.0+ (see PostgreSQL news item "PostgreSQL now has working, tested, scalable replication!" from August 28, 2003 or a lengthier press posting "PostgreSQL, Inc. Releases Open Source Replication Version") and Backplane.

eRServer has been in development for over two years, is used in production settings and is released under a BSD license (as with PostgreSQL). It uses a single master/multiple slave asynchronous replication scheme. There are cautions in the release that replication may be difficult to setup.

Backplane seems to be particularly well-suited to clustering data quickly across a WAN. A quote may explain it better:

The Backplane Open Source Database is a replicated, transactional, fault-tolerant relational database core. Currently supported on Linux and FreeBSD, Backplane is designed to run on a large number of small servers rather than a small number of large servers. With Backplane, it is possible to spread the database nodes widely, allowing database operations to work efficiently over WAN latencies while maintaining full transactional coherency across the entire replication group.

Backplane's native quorum-based replication makes it easy to increase the database capacity (by simply adding a new node), re-synch a crashed server, or take down multiple nodes for maintenance (such as an entire co-location facility) - all without affecting the database availability.

I haven't used either yet, but you may wish to give them a look.

Matt Dillon of FreeBSD fame (no, not the actor) has a new startup called Backplane. They are creating a replicated, distributed SQL database and it's open-source. It's not PostgreSQL, but it sounds like an interesting technology.

Matt Dillon of FreeBSD fame (no, not the actor) has a new startup called Backplane. They are creating a replicated, distributed SQL database and it's open-source. It's not PostgreSQL, but it sounds like an interesting technology.

Matt Dillon (of FreeBSD fame, not the actor :-) has a new database startup company called Backplane . They are creating a high-performance, distributed database which should be massively scalable (using many small servers ala Google's web farm). It's open source, too.

Matt Dillon (of FreeBSD fame, not the actor :-) has a new database startup company called Backplane . They are creating a high-performance, distributed database which should be massively scalable (using many small servers ala Google's web farm). It's open source, too.

That said, the only time I use film any more is in extreme low light situations and even then I usually don't bother if I can get the equivalent with a long-exposure on my G2.

My Canon G2 Tests

-Matt

So here is a good example of why open source is having a hard time:

"An OpenBSD presentation slide"

Note the lack of pointless animations to drive home the point that there is a lack of content. Note the lack of gradiant/textured background to "enhance" the slide in some way as to somehow looking "professionally done with Power Point" (cute font is a good start though). Notice the lack of bullets to point out that you didn't have much to say in the first place, and just have a few points with bullets.

Most importantly (from the slide), "Recently improved":

tcron/popen.c; md5(1); altqd parser; hash/rmd160.c, etc..

Any chance of a mortal person understanding even remotely what that means? That looks suspicously like content.

-Matt

Well have a look to what a FreeBSD developer says about Linux VM. Very clean, but not designed to survive under high load. Face it, the FreeBSD people focus on performance, and FreeBSD might well be faster than Linux for high load.
Now the same couldn't be necessary be said for NetBSD and OpenBSD.