No, actually, very little energy is lost in transmission. Nationally its something like 6% in the U.S. currently.
We need very high voltage transmission lines (e.g. 750Kv and higher) with more capacity if we want to link grids together and be able to get electricity all the way across the country with reasonably low losses, but that certainly is not needed when moving electricity across a few states... and that is all that is really needed when talking about linking up renewable sources of energy.
Not that it wouldn't be nice to have new high-capacity cross-country lines. I'd love to see it happen. But not having it isn't a show-stopper.
Not exactly. In both India and China, coal is producing so much pollution that people's life spans are significantly affected. Basically they tried to build a western-style full-on system and almost choked to death. Coal is still widely used, but these countries recognize the environmental disaster that you get when you use coal without pollution controls. And China, even with some pollution controls, is hitting the limit as well.
In the U.S. coal's demise is mostly driven by the price of natural gas and the recognition of its real cost, not just environmentally, but also its real cost in terms of deferred requirements that politicians have let the coal companies get away with for decades. Now with the coal companies going bankrupt the states that formulated those lax rules are paying the price. That is adding nails to coal's coffin.
Just as with Nuclear, decomimissioning cleanup costs revealed when the coal mines and plants are actually shut-down are running into the billions of dollars... costs that the coal companies conveniently did not have to take into account in the years they were operating. Kinda like spent nuclear fuel, it just builds up over time when the rules allow you to ignore it.
Much of India doesn't even *have* 24/7 power, so Solar power is actually a pretty damn good fit. Solar is definitely cheaper in this situation. Minimal battery (just needed to stabilize the load and handle occasional occlusions from clouds), the panels, and the inverter and you are done. Night-time LED lighting can be battery powered.
Big deal in India which has virtually no reliable national power infrastructure.
Its not actually that bad, period. All Intel consumer cpus since Haswell have roughly the same cpu performance. Only power consumption and GPU performance have steadily improved since then. So frankly it isn't much of an upgrade unless your old Macbook is going dead on you from battery drain.
If you still have an old Macbook with a HDD in it, then an easy mini-upgrade is to replace that HDD with a SSD. Poof, it will feel like new even with relatively limited ram. I did the 'ol switch-r-roo on my wife's old 5-year-old MacBook Pro, mostly for me, but the thing worked so well after the change my wife decided to keep it. Oops.
It's almost certainly Intel's fault. Some of their SSDs do not follow the SATA spec properly on reset which can cause the initial probe to fail with a timeout. If you probe a second time it will succeed. I actually had to add a second probe to DragonFlyBSD's AHCI driver to work around the problem. It doesn't seem to be related to startup time, even with a long delay I'll see first-probe failures on Intel SSDs in various boxes.
Strangely enough the failures occur with Intel AHCI chipsets + Intel SSDs, but do not occur with AMD AHCI chipsets + Intel SSDs.
Just one of those things. Intel firmware generally suck across all of their chips. They write specs to make their hardware designs easier and don't really give a shit how much it complicates the drivers people have to write for the hardware. That said, their SSD firmware, once you can probe the SSD, seems to work just fine.
Calculate the cost of the replacement cycle too and suddenly SSDs look a lot cheaper. It's just that most people can't think beyond the end of their noses, so if the up-front cost looks expensive they stop right there.
I bought my last HDDs last year. Two 4TB 'archival' drives for backups. My existing pile of new 1TB or 2TB HDDs (I have around a dozen 3.5" and half a dozen 2.5" left) will be dribbled out as needed, but won't be buying any new HDDs from now on. In fact, I couldn't even foist off some of my extra 2TB 3.5" HDDs onto friends last weekend. They weren't interested! Bryce happily took one of my original 40G Intel SSDs (2.5") but had no interest whatsoever in a brand spanking new 2TB WDC (3.5").
Last year I scrapped the 3.5" form factor (I made two exceptions for the two backup drives). All new systems have only 2.5" hot swap slots now. And until recently I had a growing pile of 2.5" HDDs to maintain those systems.
But now my pile of 2.5" SSDs continues to grow while my pile of 3.5" and 2.5" HDDs has begun to shrink. The strange thing about my pile of 2.5" SSDs... I haven't had to throw any SSDs away since I started it! That pile began with the old 40G Intel 2.5" SSDs that really started the industry ramp. All of those originals that I had slowly replaced with higher-capacity SSDs are still in the pile and still in perfect working order! And I actually use them, they are perfect for small test systems.
So its a bit of a strange situation. HDDs would die or get read errors and I'd wipe and throw them away. I never recycled old HDDs very much (they become unreliable when you mix cold and hot cycling or shelve them). But SSDs are a completely different matter. You can mix cold and hot cycling just fine, they basically don't go bad unless they fail outright (at a much lower rate than any HDD) or unless they are worn out from write wear (which is quite hard to do if you don't do stupid things with them). If its just an unrecoverable block due to chance, a reformat fixes the problem and the continue soldiering on (which hasn't happened to me yet but that's what I'll do).
So my SSD pile continues to grow and the capacities continue to cycle up. The pile saw its first 1TB SSDs last year. At this juncture if I have to replace my archival HDDs I might use some of the spares from my HDD pile, but after that I'll happily spend the money to throw in a bunch of SSDs for the same storage because they'll last a whole lot longer.
That standard is for a flash cell that is at its wear-limit (basically at its end-of-life). A brand new flash cell that has only been written a few times has 10x the shelf life.
Those numbers are also quite misleading, because in addition to the above, a powered SSD will rewrite cells when their data becomes weak. So data retention for a powered SSD is going to be a very long time. Since SSD flash cell life is based on write activity, if you don't wear it out from writing your SSD to its limit and you leave it powered most of the time, it will retain your data and probably last a very long time (well in excess of 10 years, probably in excess of 30 years, possibly even longer).
SSDs don't eat very much power, a data warehouse would be workable.
HDDs on the other-hand wear out whether they are powered or not. Corrosion, stress (even if not doing anything), lubricant issues, and any number of other factors. A HDD on a shelf isn't going to last even 5 years in any reliable sense. You might be able to recover the data from the magnetic media, but the expense would become stratospheric if you have more than one drive to recover.
No, not necessarily. The problem isn't so much the cpu cores, those will be mostly backwards-compatible. The real problem are all the other discrete PCI devices. If Microsoft does not provide updated drivers for their older OS releases, those older releases have no chance of working on newer hardware.
For example, Intel's Skylake chipset has I219 gigabit ethernet now (uprev from I218 which itself was an uprev from I217). The chance of older ethernet drivers working with newer chips is zero. In the case of the I219, the flash mapping and access mechanics changed drastically.
The integrated GPU is another good example. Skylake is up to Gen9. The chances that Gen8 code will work with it are zero.
One can go down the list. The only chipsets which are generic enough for older drivers to work are going to be the USB and AHCI chipsets. Everything else? Forget it.
But I don't know why people are complaining so much. The same can be said for BSD and Linux distros. An older BSD or Linux release is not going to work on newer systems. Most people don't care since they just update to the latest. While it is possible to backport the drivers to older OS releases, not very many people have the skill required so for all intents and purposes you need to run newer OpenSource OS's on these newer chips too.
The problem with projections is that they rarely predict how things will actually develop. What happens in reality is that society slowly recognizes that a problem is present. Often too slowly (and probably too late when it come to climate change), but never-the-less it eventually gets recognized, and society shifts and adjusts.
Nobody seems to understand just how huge the energy economy is in the developed world, just to support our current life-style. The numbers you quoting, despite being probably wrong (very wrong most likely)... still, the numbers you are quoting are *nothing* compared to the energy infrastructure that drives the U.S. economy today. On a relative measure, if we can have what we have now, we can certainly achieve anything you've mentioned above.
At some point in the last 10 years this whole 'thorium reactor' movement cropped up, and frankly its hard debunk the utter stupidity of the model because very few people talking about it are bonafide scientists who know what they are talking about. Me included, on this issue. But on the other-hand I'm probably one of the few people who actually knows how to use a geiger counter and has had conversations with scientists standing in front piles of lead shielding crap I wouldn't want to touch with my bare hands.
When one of those guys tells me that thorium is a disaster due to its secondary byproducts, I believe him.
At least, not totally correct. Memory bus non-volatile storage such as Intel's X-Point stuff still requires significant cache management by the operating system. Why? Because it doesn't have nearly enough durability to just be mapped as general purpose memory. A DRAM cell goes through trillions of cycles in its live time. Something like X-Point might be 1000x more durable than standard flash, but it is still 6 orders of magnitude LESS durable than DRAM. So you can't just let user programs write to it however they like.
Secondly, in terms of data-center machines becoming obsolete. Also not correct. SSDs make a fine bridge between traditional HDD or networked storage and something like X-Point. For two reasons: First, all data center machines have multiple SATA busses running at 6GBits. Gang them all together and you have a few gigabytes/sec worth of standard storage bandwidth. Secondly, you can pop nVME flash (PCI-E based flash controllers) into a server and each one has in excess of 1 GByte/sec of bandwidth (and usually much more).
Third, in terms of memory management, paging to/from SSD or nVME 'swap' space, or using it as a front-end cache for slower remote storage or spinny disks, already provides servers with a fresh new life that means they won't be obsolete for years to come.
And finally there is the cost. These specialized memory-bus non-volatile memories are going to be expensive. VERY expensive. To the point where existing configurations still have a pretty solid niche to play in. Not all workloads are storage-intensive and these new memory-bus non-volatile memories don't have the density to be able to replace the storage required for large databases (or anywhere near it).
So, the article is basically a bit too pie-in-the-sky and ignores a lot of issues.
LLVM/Clang builds the DragonFly world and kernel but does not yet build the boot loader. It can be brought in via dports. So it isn't 100% yet but very close. When it does get to 100% it will become one of our two officially supported compilers. Those are currently gcc-4.7 and gcc-5.2.1.
Wayland support isn't really up to us, but there is wayland support in XOrg that I think works for programs desiring to use that API. Don't quote me on it though.
If we're talking about bulk builds, for any language, there is going to be a huge amount of locality of reference that matches well against caches. shared text RO, lots of shared files RO, stack use is localized (RW), process data is relatively localized (RW), and file writeouts are independent. Plus any decent scheduler will recognize the batch-like nature of the compile jobs and use relatively large switch ticks. For a bulk build the scheduler doesn't have to be very smart, it just needs to avoid moving processes around between cpus excessively so and be somewhat HW cache aware.
Data and stack will be different, but one nice thing about bulk builds is that there is a huge amount of sharing of the text (code) space. Here's an example of a bulk build relatively early in its cycle (so the C++ compiles aren't eating 1GB each like they do later in the cycle when the larger packages are being built):
Notice that nothing is blocked on storage accesses. The processes are either in a pure run state or are waiting for a child process to exit.
I've never come close to maxing out the memory BW on an Intel system, at least not with bulk builds. I have maxed out the memory BW on opteron systems but even there one still gets an incremental improvement with more cores.
The real bottleneck for something like the above is not the scheduler or the pegged cpus. The real bottleneck is the operating system which is having to deal with hundreds of fork/exec/run/exit sequences per second and often more than a million VM faults per second (across the whole system)... almost all on shared resources BTW, so it isn't an easy nut for the kernel to crack (think of what it means to the kernel to fork/exec/run/exit something like/bin/sh hundreds of times per second across many cpus all at the same time).
Another big issue for the kernel, for concurrent compiles, is the massive number of shared namecache resources which are getting hit all at once, particularly negative cache hits for files which don't exist (think about compiler include path searches).
These issues tend to trump basic memory BW issues. Memory bandwidth can become an issue, but it will mainly be with jobs which are more memory-centric (access memory more and do less processing / execute fewer instructions per memory access due to the nature of the job). Bulk compiles do not fit into that category.
Urm. And you've investigated this and found that your drive is pegged because? Of What? Or you haven't investigated this and you have no idea why your drive is pegged. I'll take a guess... you are running out of memory and the disk activity you see is heavy paging.
Let me rephrase... we do bulk builds with pourdriere of 20,000 applications. It takes a bit less than two days. We set the parallelism to roughly 2x the number of cpu threads available. There are usually several hundred processes active in various states at any given moment. The cpu load is pegged. Disk activity is zero for most of the time.
If I do something less strenuous, like a buildworld or buildkernel, almost the same result. Cpu is mostly pegged, disk activity is zero for the roughly 30 minutes the buildworld takes. However, smaller builds such as a buildworld or buildkernel, or a linux kernel build, regardless of the -j concurrency you specify, will certainly have bottlenecks in the build subsystem that have nothing to do with the cpu. A little work on the Makefiles will solve that problem. In our case there are always two or three ridiculously huge source files in the GCC build that the Make has to wait for before it can proceed with the link pass. Similarly with a kernel build there is a make depend step at the beginning which is not parallelized and the final link at the end which cannot be parallelized which actually take most of the time. Compiling the sources in the middle finishes in a flash.
But your problem sounds a bit different... kinda sounds like you are running yourself out of memory. Parallel builds can run machines out of memory if the dev specifies more concurrency than his memory can handle. For example, when building packages there are many C++ source files which #include the kitchen sink and wind up with process run sizes north of 1GB. If someone only has 8GB of ram and tries a -j 8 build under those circumstances, that person will run out of memory and start to page heavily.
So its a good idea to look at the footprint of the individual processes you are trying to parallelize, too.
Memory is cheap these days. Buy more. Even those tiny little BRIX one can get these days can hold 32G of ram. For a decent concurrent build on a decent cpu you want 8GB minimum, 16GB is better, or more.
Hyperthreading on intel gives about a +30 to +50% performance improvement. So each core winds up being about 1.3 to 1.5 times the performance with two threads verses 1.0 with one. Quite significant. It depends on the type of load, of course.
The main reason for the improvement is of course due to one thread being able to make good use of execution units while the other thread is stalled on something (like memory or TLB, significant integer shifts, or dependent Integer or FPU multiply and divide operations).
Actually, parallel builds barely touch the storage subsystem. Everything is basically cached in ram and writes to files wind up being aggregated into relatively small bursts. So the drives are generally almost entirely idle the whole time.
It's almost a pure-cpu exercise and also does a pretty good job testing concurrency within the kernel due to the fork/exec/run/exit load (particularly for Makefile-based builds which use/bin/sh a lot). I've seen fork/exec rates in excess of 5000 forks/sec during poudriere runs, for example.
Indeed, though one would have to examine the NUC/BRIX specs carefully. They are being driven (typically) by a mobile chipset GPU which will have some limitations.
In fact, one could probably stuff them without any storage at all, just ram, and netboot the suckers from a single PC. I have a couple of BRIX (basically the same as a NUC) for GPU testing with 16GB of ram in each and they netboot just fine.
Maintainance -> basically none.
Expandability -> unlimited w/virtually no setup/work required.
Performance -> highly distributed and powerful.
Wiring -> highly localized, only the ethernet cables and power leave the monitor space (WIFI is available on these devices, but I would recommend hardwired ethernet and you might not be able to netboot over WIFI).
I'd use a NUC form factor with one mounted on the back of each monitor (or mounted on the back of every other monitor since it has two outputs). Basically no maintenance, easy to expand, and the off-the-shelf solution means easy to upgrade later. Will never fail if a small SSD is used, and has an ethernet hard port and plenty of resources (including 8-32GB of ram). Most monitors already have the necessary mounts.
Forking a large project is a tough, many-years job, it will need a lot more than just a few patches that weren't accepted to make it fly and it will need dedicated developers. But I think it's possible and I wish him luck.
There is a conceivable advantage to doing this. With some care, the forked linux kernel could be stabilized (something Linux really needs at the current juncture, frankly) and provide a goal for the FreeBSD linux emulation layer to go after, resulting in significant synergies between Linux and FreeBSD. Ultimately it might be possible to merge the device framework and solve the major problem that all kernel projects have of device-driver chasing by allowing developer resources to become more concentrated. That would be a difficult, but worthy goal.
Actually, I think its because many of the comments disparage the reporters writing the articles. Usually for good cause... the quality of most news articles these days is pretty horrible. But news organizations don't like to be told that they are idiots.
But there are certainly also lots of instances where the commenters start fighting among themselves... usually it devolves down into politics or religion. People with very strong views often come up against the hard, harsh wall of reality and the result is typically fireworks.
Firefox has lots of memory leaks, particularly if you run javascript-heavy sites or flash-emulated javascript.
You need to kill and restart your firefox if it is eating 21GB. It will return to eating ~1-2 GB, but then start building up again over time. I usually have to completely close my firefox browsers at least once a week.
Honestly, these days if it has two memory slots I stuff it with 16GB of ram. If it has four, then 32GB of ram. Simple as that. Hell, I just put together a 'gaming box' for the son of a friend of mine a few weeks ago and thought 16GB would be enough (4x 4GB). I didn't even follow my own rule because I was being cost conscious. The first thing he did with it? Run minecraft with a visibility setting that ate up all 16GB of ram.
Even more important than ram, stuffing a SSD into the box is what really makes everything more responsive. And even if it has to do a bit of paging it's hardly noticeable when its paging to/from a SSD. And if you do both, the box will stay relevant for a very long time, probably 10 years.
Most time consuming bug - The AMD cpu stack corruption bug. Errata 721. It took me a year to track it down. Half that period I thought it was a software bug in the kernel, for a month I thought it was memory corruption in gcc. And most of the rest of the time was spent trying to reproduce it reliably and examine the cores from gcc to characterize the bug. Somewhere in there I realized it was a cpu bug. It took a while to reduce the cases enough to be able to reproduce the bug within 60 seconds. And the last week was putting the whole thing together into a bootable USB stick image to send to AMD so they could boot up the test environment and reproduce the bug themselves.
Bug that was the most fun - The 6522 I/O chip was a wonderful multi-feature chip with a lot of capability. There was a hardware timer bug which could jam the timer interrupt if it timed out at just the wrong time.
My general advice: Add assertions for complex pre-conditions instead of assuming that said complex pre-conditions are always properly in place. The more non-stupid assertions you have in your code, the earlier you detect the bug and the easier it is to fix.
Yup, they sure do. Not only is HTML5 video in ads happening a lot more these days, some sites insert the ads in-line with the article making it difficult for adblock software to distinguish them from graphs and other things that are part of the article.
I've got adblock installed in chrome, but not firefox yet. For some reason some sites think I'm on a chromebook when I use chrome, instead of DragonFly, which I find hilarious. Adblock in firefox is next.
No flash for ages. Last thing I would ever do. HTML5 or nothing, baby! I complain to sites like Pandora that still have flash requirements for certain browsers, but not for others.
Slashdot Mirror
No, actually, very little energy is lost in transmission. Nationally its something like 6% in the U.S. currently.
We need very high voltage transmission lines (e.g. 750Kv and higher) with more capacity if we want to link grids together and be able to get electricity all the way across the country with reasonably low losses, but that certainly is not needed when moving electricity across a few states... and that is all that is really needed when talking about linking up renewable sources of energy.
Not that it wouldn't be nice to have new high-capacity cross-country lines. I'd love to see it happen. But not having it isn't a show-stopper.
-Matt
Not exactly. In both India and China, coal is producing so much pollution that people's life spans are significantly affected. Basically they tried to build a western-style full-on system and almost choked to death. Coal is still widely used, but these countries recognize the environmental disaster that you get when you use coal without pollution controls. And China, even with some pollution controls, is hitting the limit as well.
In the U.S. coal's demise is mostly driven by the price of natural gas and the recognition of its real cost, not just environmentally, but also its real cost in terms of deferred requirements that politicians have let the coal companies get away with for decades. Now with the coal companies going bankrupt the states that formulated those lax rules are paying the price. That is adding nails to coal's coffin.
Just as with Nuclear, decomimissioning cleanup costs revealed when the coal mines and plants are actually shut-down are running into the billions of dollars... costs that the coal companies conveniently did not have to take into account in the years they were operating. Kinda like spent nuclear fuel, it just builds up over time when the rules allow you to ignore it.
-Matt
Much of India doesn't even *have* 24/7 power, so Solar power is actually a pretty damn good fit. Solar is definitely cheaper in this situation. Minimal battery (just needed to stabilize the load and handle occasional occlusions from clouds), the panels, and the inverter and you are done. Night-time LED lighting can be battery powered.
Big deal in India which has virtually no reliable national power infrastructure.
-Matt
Its not actually that bad, period. All Intel consumer cpus since Haswell have roughly the same cpu performance. Only power consumption and GPU performance have steadily improved since then. So frankly it isn't much of an upgrade unless your old Macbook is going dead on you from battery drain.
If you still have an old Macbook with a HDD in it, then an easy mini-upgrade is to replace that HDD with a SSD. Poof, it will feel like new even with relatively limited ram. I did the 'ol switch-r-roo on my wife's old 5-year-old MacBook Pro, mostly for me, but the thing worked so well after the change my wife decided to keep it. Oops.
-Matt
It's almost certainly Intel's fault. Some of their SSDs do not follow the SATA spec properly on reset which can cause the initial probe to fail with a timeout. If you probe a second time it will succeed. I actually had to add a second probe to DragonFlyBSD's AHCI driver to work around the problem. It doesn't seem to be related to startup time, even with a long delay I'll see first-probe failures on Intel SSDs in various boxes.
Strangely enough the failures occur with Intel AHCI chipsets + Intel SSDs, but do not occur with AMD AHCI chipsets + Intel SSDs.
Just one of those things. Intel firmware generally suck across all of their chips. They write specs to make their hardware designs easier and don't really give a shit how much it complicates the drivers people have to write for the hardware. That said, their SSD firmware, once you can probe the SSD, seems to work just fine.
-Matt
Calculate the cost of the replacement cycle too and suddenly SSDs look a lot cheaper. It's just that most people can't think beyond the end of their noses, so if the up-front cost looks expensive they stop right there.
I bought my last HDDs last year. Two 4TB 'archival' drives for backups. My existing pile of new 1TB or 2TB HDDs (I have around a dozen 3.5" and half a dozen 2.5" left) will be dribbled out as needed, but won't be buying any new HDDs from now on. In fact, I couldn't even foist off some of my extra 2TB 3.5" HDDs onto friends last weekend. They weren't interested! Bryce happily took one of my original 40G Intel SSDs (2.5") but had no interest whatsoever in a brand spanking new 2TB WDC (3.5").
Last year I scrapped the 3.5" form factor (I made two exceptions for the two backup drives). All new systems have only 2.5" hot swap slots now. And until recently I had a growing pile of 2.5" HDDs to maintain those systems.
But now my pile of 2.5" SSDs continues to grow while my pile of 3.5" and 2.5" HDDs has begun to shrink. The strange thing about my pile of 2.5" SSDs... I haven't had to throw any SSDs away since I started it! That pile began with the old 40G Intel 2.5" SSDs that really started the industry ramp. All of those originals that I had slowly replaced with higher-capacity SSDs are still in the pile and still in perfect working order! And I actually use them, they are perfect for small test systems.
So its a bit of a strange situation. HDDs would die or get read errors and I'd wipe and throw them away. I never recycled old HDDs very much (they become unreliable when you mix cold and hot cycling or shelve them). But SSDs are a completely different matter. You can mix cold and hot cycling just fine, they basically don't go bad unless they fail outright (at a much lower rate than any HDD) or unless they are worn out from write wear (which is quite hard to do if you don't do stupid things with them). If its just an unrecoverable block due to chance, a reformat fixes the problem and the continue soldiering on (which hasn't happened to me yet but that's what I'll do).
So my SSD pile continues to grow and the capacities continue to cycle up. The pile saw its first 1TB SSDs last year. At this juncture if I have to replace my archival HDDs I might use some of the spares from my HDD pile, but after that I'll happily spend the money to throw in a bunch of SSDs for the same storage because they'll last a whole lot longer.
-Matt
That standard is for a flash cell that is at its wear-limit (basically at its end-of-life). A brand new flash cell that has only been written a few times has 10x the shelf life.
Those numbers are also quite misleading, because in addition to the above, a powered SSD will rewrite cells when their data becomes weak. So data retention for a powered SSD is going to be a very long time. Since SSD flash cell life is based on write activity, if you don't wear it out from writing your SSD to its limit and you leave it powered most of the time, it will retain your data and probably last a very long time (well in excess of 10 years, probably in excess of 30 years, possibly even longer).
SSDs don't eat very much power, a data warehouse would be workable.
HDDs on the other-hand wear out whether they are powered or not. Corrosion, stress (even if not doing anything), lubricant issues, and any number of other factors. A HDD on a shelf isn't going to last even 5 years in any reliable sense. You might be able to recover the data from the magnetic media, but the expense would become stratospheric if you have more than one drive to recover.
-Matt
No, not necessarily. The problem isn't so much the cpu cores, those will be mostly backwards-compatible. The real problem are all the other discrete PCI devices. If Microsoft does not provide updated drivers for their older OS releases, those older releases have no chance of working on newer hardware.
For example, Intel's Skylake chipset has I219 gigabit ethernet now (uprev from I218 which itself was an uprev from I217). The chance of older ethernet drivers working with newer chips is zero. In the case of the I219, the flash mapping and access mechanics changed drastically.
The integrated GPU is another good example. Skylake is up to Gen9. The chances that Gen8 code will work with it are zero.
One can go down the list. The only chipsets which are generic enough for older drivers to work are going to be the USB and AHCI chipsets. Everything else? Forget it.
But I don't know why people are complaining so much. The same can be said for BSD and Linux distros. An older BSD or Linux release is not going to work on newer systems. Most people don't care since they just update to the latest. While it is possible to backport the drivers to older OS releases, not very many people have the skill required so for all intents and purposes you need to run newer OpenSource OS's on these newer chips too.
-Matt
The problem with projections is that they rarely predict how things will actually develop. What happens in reality is that society slowly recognizes that a problem is present. Often too slowly (and probably too late when it come to climate change), but never-the-less it eventually gets recognized, and society shifts and adjusts.
Nobody seems to understand just how huge the energy economy is in the developed world, just to support our current life-style. The numbers you quoting, despite being probably wrong (very wrong most likely)... still, the numbers you are quoting are *nothing* compared to the energy infrastructure that drives the U.S. economy today. On a relative measure, if we can have what we have now, we can certainly achieve anything you've mentioned above.
At some point in the last 10 years this whole 'thorium reactor' movement cropped up, and frankly its hard debunk the utter stupidity of the model because very few people talking about it are bonafide scientists who know what they are talking about. Me included, on this issue. But on the other-hand I'm probably one of the few people who actually knows how to use a geiger counter and has had conversations with scientists standing in front piles of lead shielding crap I wouldn't want to touch with my bare hands.
When one of those guys tells me that thorium is a disaster due to its secondary byproducts, I believe him.
-Matt
At least, not totally correct. Memory bus non-volatile storage such as Intel's X-Point stuff still requires significant cache management by the operating system. Why? Because it doesn't have nearly enough durability to just be mapped as general purpose memory. A DRAM cell goes through trillions of cycles in its live time. Something like X-Point might be 1000x more durable than standard flash, but it is still 6 orders of magnitude LESS durable than DRAM. So you can't just let user programs write to it however they like.
Secondly, in terms of data-center machines becoming obsolete. Also not correct. SSDs make a fine bridge between traditional HDD or networked storage and something like X-Point. For two reasons: First, all data center machines have multiple SATA busses running at 6GBits. Gang them all together and you have a few gigabytes/sec worth of standard storage bandwidth. Secondly, you can pop nVME flash (PCI-E based flash controllers) into a server and each one has in excess of 1 GByte/sec of bandwidth (and usually much more).
Third, in terms of memory management, paging to/from SSD or nVME 'swap' space, or using it as a front-end cache for slower remote storage or spinny disks, already provides servers with a fresh new life that means they won't be obsolete for years to come.
And finally there is the cost. These specialized memory-bus non-volatile memories are going to be expensive. VERY expensive. To the point where existing configurations still have a pretty solid niche to play in. Not all workloads are storage-intensive and these new memory-bus non-volatile memories don't have the density to be able to replace the storage required for large databases (or anywhere near it).
So, the article is basically a bit too pie-in-the-sky and ignores a lot of issues.
-Matt
LLVM/Clang builds the DragonFly world and kernel but does not yet build the boot loader. It can be brought in via dports. So it isn't 100% yet but very close. When it does get to 100% it will become one of our two officially supported compilers. Those are currently gcc-4.7 and gcc-5.2.1.
Wayland support isn't really up to us, but there is wayland support in XOrg that I think works for programs desiring to use that API. Don't quote me on it though.
-Matt
If we're talking about bulk builds, for any language, there is going to be a huge amount of locality of reference that matches well against caches. shared text RO, lots of shared files RO, stack use is localized (RW), process data is relatively localized (RW), and file writeouts are independent. Plus any decent scheduler will recognize the batch-like nature of the compile jobs and use relatively large switch ticks. For a bulk build the scheduler doesn't have to be very smart, it just needs to avoid moving processes around between cpus excessively so and be somewhat HW cache aware.
Data and stack will be different, but one nice thing about bulk builds is that there is a huge amount of sharing of the text (code) space. Here's an example of a bulk build relatively early in its cycle (so the C++ compiles aren't eating 1GB each like they do later in the cycle when the larger packages are being built):
http://apollo.backplane.com/DF...
Notice that nothing is blocked on storage accesses. The processes are either in a pure run state or are waiting for a child process to exit.
I've never come close to maxing out the memory BW on an Intel system, at least not with bulk builds. I have maxed out the memory BW on opteron systems but even there one still gets an incremental improvement with more cores.
The real bottleneck for something like the above is not the scheduler or the pegged cpus. The real bottleneck is the operating system which is having to deal with hundreds of fork/exec/run/exit sequences per second and often more than a million VM faults per second (across the whole system)... almost all on shared resources BTW, so it isn't an easy nut for the kernel to crack (think of what it means to the kernel to fork/exec/run/exit something like /bin/sh hundreds of times per second across many cpus all at the same time).
Another big issue for the kernel, for concurrent compiles, is the massive number of shared namecache resources which are getting hit all at once, particularly negative cache hits for files which don't exist (think about compiler include path searches).
These issues tend to trump basic memory BW issues. Memory bandwidth can become an issue, but it will mainly be with jobs which are more memory-centric (access memory more and do less processing / execute fewer instructions per memory access due to the nature of the job). Bulk compiles do not fit into that category.
-Matt
Urm. And you've investigated this and found that your drive is pegged because? Of What? Or you haven't investigated this and you have no idea why your drive is pegged. I'll take a guess... you are running out of memory and the disk activity you see is heavy paging.
Let me rephrase... we do bulk builds with pourdriere of 20,000 applications. It takes a bit less than two days. We set the parallelism to roughly 2x the number of cpu threads available. There are usually several hundred processes active in various states at any given moment. The cpu load is pegged. Disk activity is zero for most of the time.
If I do something less strenuous, like a buildworld or buildkernel, almost the same result. Cpu is mostly pegged, disk activity is zero for the roughly 30 minutes the buildworld takes. However, smaller builds such as a buildworld or buildkernel, or a linux kernel build, regardless of the -j concurrency you specify, will certainly have bottlenecks in the build subsystem that have nothing to do with the cpu. A little work on the Makefiles will solve that problem. In our case there are always two or three ridiculously huge source files in the GCC build that the Make has to wait for before it can proceed with the link pass. Similarly with a kernel build there is a make depend step at the beginning which is not parallelized and the final link at the end which cannot be parallelized which actually take most of the time. Compiling the sources in the middle finishes in a flash.
But your problem sounds a bit different... kinda sounds like you are running yourself out of memory. Parallel builds can run machines out of memory if the dev specifies more concurrency than his memory can handle. For example, when building packages there are many C++ source files which #include the kitchen sink and wind up with process run sizes north of 1GB. If someone only has 8GB of ram and tries a -j 8 build under those circumstances, that person will run out of memory and start to page heavily.
So its a good idea to look at the footprint of the individual processes you are trying to parallelize, too.
Memory is cheap these days. Buy more. Even those tiny little BRIX one can get these days can hold 32G of ram. For a decent concurrent build on a decent cpu you want 8GB minimum, 16GB is better, or more.
-Matt
Hyperthreading on intel gives about a +30 to +50% performance improvement. So each core winds up being about 1.3 to 1.5 times the performance with two threads verses 1.0 with one. Quite significant. It depends on the type of load, of course.
The main reason for the improvement is of course due to one thread being able to make good use of execution units while the other thread is stalled on something (like memory or TLB, significant integer shifts, or dependent Integer or FPU multiply and divide operations).
-Matt
Actually, parallel builds barely touch the storage subsystem. Everything is basically cached in ram and writes to files wind up being aggregated into relatively small bursts. So the drives are generally almost entirely idle the whole time.
It's almost a pure-cpu exercise and also does a pretty good job testing concurrency within the kernel due to the fork/exec/run/exit load (particularly for Makefile-based builds which use /bin/sh a lot). I've seen fork/exec rates in excess of 5000 forks/sec during poudriere runs, for example.
-Matt
Indeed, though one would have to examine the NUC/BRIX specs carefully. They are being driven (typically) by a mobile chipset GPU which will have some limitations.
In fact, one could probably stuff them without any storage at all, just ram, and netboot the suckers from a single PC. I have a couple of BRIX (basically the same as a NUC) for GPU testing with 16GB of ram in each and they netboot just fine.
Maintainance -> basically none.
Expandability -> unlimited w/virtually no setup/work required.
Performance -> highly distributed and powerful.
Wiring -> highly localized, only the ethernet cables and power leave the monitor space (WIFI is available on these devices, but I would recommend hardwired ethernet and you might not be able to netboot over WIFI).
-Matt
I'd use a NUC form factor with one mounted on the back of each monitor (or mounted on the back of every other monitor since it has two outputs). Basically no maintenance, easy to expand, and the off-the-shelf solution means easy to upgrade later. Will never fail if a small SSD is used, and has an ethernet hard port and plenty of resources (including 8-32GB of ram). Most monitors already have the necessary mounts.
-Matt
Forking a large project is a tough, many-years job, it will need a lot more than just a few patches that weren't accepted to make it fly and it will need dedicated developers. But I think it's possible and I wish him luck.
There is a conceivable advantage to doing this. With some care, the forked linux kernel could be stabilized (something Linux really needs at the current juncture, frankly) and provide a goal for the FreeBSD linux emulation layer to go after, resulting in significant synergies between Linux and FreeBSD. Ultimately it might be possible to merge the device framework and solve the major problem that all kernel projects have of device-driver chasing by allowing developer resources to become more concentrated. That would be a difficult, but worthy goal.
-Matt
Why ? At normal viewing distance I can't see the pixels on my 28" 4K monitors.
-Matt
(1) Buy/build a super-yacht big enough to live on as your home.
(2) Travel the world, taking your home with you.
Requires 'only' a few hundred million to really make it work.
-Matt
Actually, I think its because many of the comments disparage the reporters writing the articles. Usually for good cause... the quality of most news articles these days is pretty horrible. But news organizations don't like to be told that they are idiots.
But there are certainly also lots of instances where the commenters start fighting among themselves... usually it devolves down into politics or religion. People with very strong views often come up against the hard, harsh wall of reality and the result is typically fireworks.
-Matt
Firefox has lots of memory leaks, particularly if you run javascript-heavy sites or flash-emulated javascript.
You need to kill and restart your firefox if it is eating 21GB. It will return to eating ~1-2 GB, but then start building up again over time. I usually have to completely close my firefox browsers at least once a week.
-Matt
Honestly, these days if it has two memory slots I stuff it with 16GB of ram. If it has four, then 32GB of ram. Simple as that. Hell, I just put together a 'gaming box' for the son of a friend of mine a few weeks ago and thought 16GB would be enough (4x 4GB). I didn't even follow my own rule because I was being cost conscious. The first thing he did with it? Run minecraft with a visibility setting that ate up all 16GB of ram.
Even more important than ram, stuffing a SSD into the box is what really makes everything more responsive. And even if it has to do a bit of paging it's hardly noticeable when its paging to/from a SSD. And if you do both, the box will stay relevant for a very long time, probably 10 years.
But more to the point, why not?
-Matt
Most time consuming bug - The AMD cpu stack corruption bug. Errata 721. It took me a year to track it down. Half that period I thought it was a software bug in the kernel, for a month I thought it was memory corruption in gcc. And most of the rest of the time was spent trying to reproduce it reliably and examine the cores from gcc to characterize the bug. Somewhere in there I realized it was a cpu bug. It took a while to reduce the cases enough to be able to reproduce the bug within 60 seconds. And the last week was putting the whole thing together into a bootable USB stick image to send to AMD so they could boot up the test environment and reproduce the bug themselves.
Bug that was the most fun - The 6522 I/O chip was a wonderful multi-feature chip with a lot of capability. There was a hardware timer bug which could jam the timer interrupt if it timed out at just the wrong time.
My general advice: Add assertions for complex pre-conditions instead of assuming that said complex pre-conditions are always properly in place. The more non-stupid assertions you have in your code, the earlier you detect the bug and the easier it is to fix.
-Matt
Yup, they sure do. Not only is HTML5 video in ads happening a lot more these days, some sites insert the ads in-line with the article making it difficult for adblock software to distinguish them from graphs and other things that are part of the article.
I've got adblock installed in chrome, but not firefox yet. For some reason some sites think I'm on a chromebook when I use chrome, instead of DragonFly, which I find hilarious. Adblock in firefox is next.
No flash for ages. Last thing I would ever do. HTML5 or nothing, baby! I complain to sites like Pandora that still have flash requirements for certain browsers, but not for others.
-Matt