Slashdot Mirror

There has been an implementation of row hammer testing in MemTest86 V6.0 for over 6 months now as well.
Disclaimer: I'm one of the MemTest86 developers.

From website:

Version 6.0.0 13/Feb/2015
New Features
* New "Hammer Test" for detecting disturbance errors

Feb 2015 is more than 6 months ago [within error: +/- 600%] Do you write software within the same tolerance?

It is worth noting that the row hammer issue isn't new. It as been known about for some time. Including this old Slashdot post
http://hardware.slashdot.org/s...

There has been an implementation of row hammer testing in MemTest86 V6.0 for over 6 months now as well. MemTest86 implements just the single sided hammer, whereas Google used a double sided hammer.
http://www.memtest86.com/
While the double hammer might produce more RAM errors, this pattern of memory accesses isn't very likely to occur in real life software. So is of limited use as a RAM reliability test.

What is new in this report is the fact that they manipulated the RAM bit flips to turn them into an exploit. Something that was previously speculated on but considered too hard to implement.

What they didn't show however is any results from desktop machines. All their testing was on laptops. In fact they state, "We also tested some desktop machines, but did not see any bit flips on those". So the problem isn't as grave as it might at first appear. They speculate that ECC RAM blocks the bit flips and this has also been the experience with MemTest86, most (but not all) of the flips are single bit flips, which ECC would correct.

Disclaimer: I'm one of the MemTest86 developers.

You don't deal with it daily in Windows 7. You deal with it daily on your computer with the bad RAM in it. Treat yourself to this.

Always test new RAM before relying on it, even when you're using ECC memory. It doesn't take long, it's free, and saves you a lot of time later. Memtest86 is an excellent free tool for testing memory. Just burn it onto a CD, boot up with it, and let it run for a new minutes.

There are two of them:

http://www.memtest86.com/
http://www.memtest.org/

Or use both?

That recordable media levy is the main reason that Canada doesn't have a DMCA. And the price is virtually non-existent.

That still doesn't justify it any better to my way of thinking. I have to help fund RIAA if I want to burn a copy of memtest86 or my favorite Linux distro? What's wrong with that picture?

I hafta wonder if the bit flipped due to a bad RAM stick?

We use MD5 checksums throughout the system, for example, to prevent, detect, and recover from corruption that can occur during receipt, storage, and retrieval of customers' objects. However, we didn't have the same protection in place to detect whether this particular internal state information had been corrupted.

Nothing specific about *what* caused the bit to flip.

This comes to mind only because bad RAM on a new server at work caused installation of a stock Perl module to throw excessive errors during the XS compile phase - the same package installed without error on an identical machine 20 minutes earlier. Took over an hour before we realized it was probably hardware. Memtest86 quickly turned up the problem.

Would hashes and the like protect against RAM suddenly going south? Wouldn't any piece of data that passes through main memory be vulnerable to corruption? Makes me wonder why ECC memory isn't being used much anymore... we have various flavors of RAID to protect slow memory from corruption, but not many machines I see have ECC anymore.

One should be more concerned as to why your files are becoming corrupted.

I'd say its a safe bet that the files from apple.com are in perfect condition.

Which means it either became corrupted in transit to, or on arrival to your machine.

Which leads the question, is your memory defective
run memtest86 to check your memory.
http://www.memtest86.com/

Check if your Harddrives have SMART and are reporting anything. A disk checker would also be a good idea.

The other idea that springs to mind is if your behind some proxy with the above problems, although i doubt anyone would want to proxy a 1.5gig file.

Fact is, if files are being corrupted on your disk, its just a matter of time before something more important is hit by corruption.

I'd check your RAM. http://www.memtest86.com/ or use any Ubuntu disk.

Yep, memory problems have about as many causes as yaks do fleas. From the memtest troubleshooting section:

http://www.memtest86.com/tech.html#trouble

Please be aware that not all errors reported by Memtest86 are due to bad memory. The test implicitly tests the CPU, L1 and L2 caches as well as the motherboard. It is impossible for the test to determine what causes the failure to occur. However, most failures will be due to a problem with memory module. When it is not, the only option is to replace parts until the failure is corrected.

Once a memory error has been detected, determining the failing SIMM/DIMM module is not a clear cut procedure. With the large number of motherboard vendors and possible combinations of memory slots it would be difficult if not impossible to assemble complete information about how a particular error would map to a failing memory module. However, there are steps that may be taken to determine the failing module. Here are four techniques that you may wish to use: ...

Sometimes memory errors show up due to component incompatibility. A memory module may work fine in one system and not in another. This is not uncommon and is a source of confusion. In these situations the components are not necessarily bad but have marginal conditions that when combined with other components will cause errors.

There have been numerous reports of errors with only tests 5 and 8 on Athlon systems. Often the memory works in a different system or the vendor insists that it is good. In these cases the memory is not necessarily bad but is not able to operate reliably at Athlon speeds. Sometimes more conservative memory timings on the motherboard will correct these errors. In other cases the only option is to replace the memory with better quality, higher speed memory. Don't buy cheap memory and expect it to work with an Athlon! On occasion test 5/8 errors will occur even with name brand memory and a quality motherboard. These errors are legitimate and should be corrected.

I am often asked about the reliability of errors reported by Mestest86. In the vast majority of cases errors reported by the test are valid. There are some systems that cause Memtest86 to be confused about the size of memory and it will try to test non-existent memory. This will cause a large number of consecutive addresses to be reported as bad and generally there will be many bits in error. If you have a relatively small number of failing addresses and only one or two bits in error you can be certain that the errors are valid. Also intermittent errors are without exception valid. Frequently memory vendors question if Memtest86 supports their particular memory type or a chipset. Memtest86 is designed to work with all memory types and all chipsets. Only support for ECC requires knowledge of the chipset.

If you suspect it (or you can't pin down a specific problem) all you have to do is run http://www.memtest86.com/ [memtest86.com]. It will run on any computer that has a floppy or CD drive. All you gotta do is boot from the memtest disc, and it will scan it completely automatically.

Ummm, if you bothered to read the post you are replying to, he said that the bad ram would pass memtest86, which made it difficult to find the cause, since many people (including me) trust the results of memtest86.

Are you kidding?

If you suspect it (or you can't pin down a specific problem) all you have to do is run http://www.memtest86.com/. It will run on any computer that has a floppy or CD drive. All you gotta do is boot from the memtest disc, and it will scan it completely automatically. If I remember correctly, it will even tell you (in plain English) if something is wrong right at the end. It takes all of about 2 minutes to run and you have a definitive answer as to whether the problem is your RAM or not. If that is hard, I don't know how you're going to diagnose something like a bad driver.

In OS memory testing is mostly useless in my experience.

1) You have to boot up your system to use it. Much of the time I've seen bad RAM, your system won't boot as the OS uses too much of said bad RAM.

2) If your system has had a virus and/or the OS is corrupted, you're not really isolating the problem as you're still testing the OS + hardware.

I've found Memtest 86 to be a better solution since (1) uses its own OS (freeDos, very small memory footprint, so it WILL boot) (2) doesn't rely on the system having on OS so it can be used with system corruption/viruses/with a hard drive (if you're building a system) and (3) is free (can download/use on as many systems as you own without needing to buy an OS license to check you memory)

Why is the Vista tool so good again? (Am I missing something?)

Here's what I have in my CD case, in approximate order of how regularly use them...

Memtest86--because the RAM in the cheap PCs I come across sucks. Some of the other tool CDs have this one as well, I like to get the latest one regularly here. Good for stress testing, and even handy for figuring out things like whether the RAM is running correctly in dual-channel mode.

SystemRescueCD--I particularly like the partition editor and imaging utilities. Been weaning myself off Partition Magic/Drive Image even for Windows work with these two.

Ubuntu live CD and DVD. The CD works in more systems, the DVD version is a completely usable system with a lot of stuff in it. What most impresses me about the Ubuntu live disc is that I can download packages over the network and install them, even thing that run as services, from the live environment. I actually got PostgreSQL installed and some database tests completed, all without a single Postgres file on the media.

Knoppix--Some days, your first choice in Linux live CDs just doesn't work on a random machine; that's why I still carry around this one as a backup.

Bart PE--A bit of a pain to build the first time, but very handy for fixing Windows machines.

Offline NT Password & Registry Editor--this one has been less useful lately, as I've been running into NTFS partitions it really doesn't want to write to. My fallback position is to use this to generate a new SAM file, then copy it over with a BartPE disc.

RedHat Enterprise 3 and 4 CDs. While not technically live CDs, you can do a lot with booting into this environment, and I deal with enough people running RedHat versions that they're worth carrying around. I still keep one of the older versions around so I have something running the 2.4 kernel to tests against; occasionally I'll run into some old hardware that 2.6 pukes on, while 2.4 still works great.

Detecting faulty hardware is done by the BIOS, not by system crashes

There are many good approaches for testing memory. However, many tests simply throw some patterns at memory without much thought or knowledge of memory architecture or how errors can best be detected. This works fine for hard memory failures but does little to find intermittent errors. BIOS based memory tests are useless for finding intermittent memory errors.

http://www.monarchcomputer.com/Merchant2/merchant. mv
http://www.ibexpc.com/linuxsystems.html
http://gnupc.com/
http://www.sunsetsystems.com/
http://h10018.www1.hp.com/wwsolutions/linux/produc ts/clients/workstationcert.html

"You must be on crack."
Must be. After all, I'm responding to your nut job comment.

"The BIOS on all Dell systems made in the past 4-5 years, maybe longer, can now be updated from Linux. You don't even need to reboot into DOS. Did others put that kind of effort in supporting customers?"
I have news for you. HP has been doing it for few years. I believe IBM as well. Matter of fact, every Linux embedded device manufactures has been.

"That crack must be really good."
Must be. Your rampant ignorance is starting to make me dazed.

"Dell has in my opinion been doing a decent job, given the constraints."
Huh? What constratins are you talking about? Oh you mean, like outsource call centers to India? If Dell doesn't want to support Linux, that's fine. Good for them. But if Dell is going to support Linux, at least do a half way decent job of doing it. It has been less than half ass device support and inadaquate engineers filling up Dell Linux department since 2001. For instance, getting Dell's onboard SATA chipset with kernel 2.4.x was pulling teeth. I was fraustrated up to my eye balls with their engineering staffs and ended up emailing the motherboard manufacture for support which emailed me within 10 minutes with module tarball attachment.

"Everything in my 600m is supported."
I'm happy for you. I really am.

"On my PowerEdge running Fedora, I can even tell which DIMM bank has been causing parity errors."
Yeah, that's great. So can you with memtest86+. http://www.memtest86.com/ I think, it's been around for several years now. It seems, you get amuzed a lot. Install memtest86 rpm package from Fedora distro CD and run memtest-setup from Fedora and grub will give you an option to boot you right into memtest86+.

"Their engineers have been more helpful than they were required to."
Really? Did they fix your car or something? Because they are supposed to resolve issues with every items they sell.

Alright, I don't mean to bash Dell fanboys out there, but this bs has to stop. Having low expectation from vendors and manufactures for supporting Linux makes you applaude for what they are "supposed" to do, then go right ahead. Just don't bitch about same type of support for Windows installed systems.

Also, have you tested your memory with memtest86?

Have you checked your RAM? Bad RAM could easily cause massive instability in both OSes. There's no reason why either Linux or Windows should crash so quickly. Try Memtest86 or its successor/fork Memtest86+. A quick test is to copy a large file and then md5sum both versions. If the MD5s are different, you may have bad RAM.

Whenever we buy new RAM, mostly as part of new PCs, we run Memtest86. It's easy to do, it takes a while so do it overnight. There's so much that can go wrong with RAM, even with "good" RAM: it might not work together with the board, the SPD-timings might be off, whatever. Every once in a while we find some RAM that doesn't work for us and return it to the shop. We never had any problems at all to get it exchanged.

For hardware-sellers it's probably more expensive if they have to factor in a certain return-rate (and the overhead for that) so they will look to it that the RAM they buy is ok. That way market forces will work for the benefits of all of us: untested RAM will, in the end, be more expensive than tested RAM. It's much easier and cheaper to do RAMtesting factoryside than having it returned by millions of customers.

Of course that doesn't work if you buy your PC in a supermarket, but even for cheap PCs it's better to configure them yourself than buying crap. That way you can specify exactly where to save money and if anything breaks you get it fixed much quicker.

I've run into my share of bad memory in the past. That's why all new memory I get runs through extensive run through memtest86 http://www.memtest86.com/ before moving to a production system.

However, it will be interesting to see where this memory ends up in other applications that could cause a whole lot more problems than my pr0n server rebooting at random..

Here is the obligitory Memtest86 post. It's a great program, and chances are that you might already have a copy on your Linux install CD depending on the distro. There are even kernel patches that allow you to avoid the bad bits if they are isolated enough.

Before I found memtest my computers were VERY unpredicable.

Usually when you experience many random crashes, or seemingly random results from a program, there's usually a problem with your system memory (RAM).

Try using Memtest86 to diagnose your system. It may be nothing, bad luck, or some other component of your system misbehaving, but it's usually bad memory.

like Memtest86. I'm wondering if Hl2 isn't just stressing out some flakey RAM.

memtest86 is free and in beer and speech, and is operating system independent. You just write the binary on to a floppy disk and boot the machine off of it.

I've cured several machines of mysterious problems by identifying bad RAM with memtest86. It was always cheap, unbranded RAM that was the problem. I get all my RAM from Crucial nowadays and I never have a problem with it. (I am not affiliated with Crucial or memtest86).

Personally I use memtest86 http://www.memtest86.com/ on all the tests, I found 5,7,10 very good at spotting my problems. You can burn and image on a cd or put it in lilo. When I got my latest PC, I had lots of problems which I traced back to the memory not working at DDR400 in dual channel mode. Another program is prime http://www.mersenne.org/prime.htm in a torture test mode (but in windows). I noted some else suggested GCC the problem with that is that it does not test ALL memory, so will find some bit problems but many bus related ones.

If you have a Dual channel system and have memory problems try clocking back one speed, it made my memory rock solid.

James

There are some good, free memory testing utilities out there like memtest86. They probably still have an ISO image on their website that can be used to create a bootable CD-ROM with the utility configured to start automatically.

I suggest you run Memtest86 and see if you have any errors in your system RAM. I just spent a bunch of time trying to install Ubuntu on a new computer, and finally ran Memtest86 and discovered that my brand-new RAM had an error.

You can run Memtest86 from the Ubuntu "live CD". Or you can download a special Memtest86 CD image from the Memtest86 site:

http://www.memtest86.com/

steveha

Yeah, I know this is off-topic but I just had to respond. I have 256 megs of PC133 SDRAM with some bad areas. I found the bad areas with Memtest86. Then I allocated them using MmAllocateContiguousMemorySpecifyCache in a driver that runs at boot. I found a nice example driver that I just had to modify slighty. The example I used seems to be the NT example at the bottom here.

Tip: To test memory, use Memtest86

The problem with the benchmark:
They were running Windows XP.
Any pre-emptive OS will give you terribly innaccurate results for low-level hardware testing.

If you want real results, don't run tests from an operating system--run them from the bootloader (with interupts disabled). Myself, I would probably modify something like Memtest86, and have it time the tests. This would reduce problems due to chipset differences and chance. Any test that can't be exactly repeated with the same results is marketing fodder.

The reviewers otherwise appear to have done a good job. The two test rigs used very similar hardware. They even thought to disable unneccessary peripherials in the bios. They probably even ran each of their tests after a fresh Windows install.

This doesn't mean that their results are wrong, in fact their results are about what I would expect (after having poked around the internet looking for technical information on DDR1 and DDR2). Their presentation is a bit misleading: They test the DDR1 memory with tighter timings than the DDR2. Since these timings reflect how much the memory can get done in a clock, is it suprising that DDR2 requiring 4 clocks for an operation running at 667MHz is equal to DDR1 requiring 2.5 clocks for an operation running at 500 MHz? (For the math impaired this means that with their setup the DDR2 can do 166 million ops per second, and the DDR1 can do 200 million ops per second). I suspect that the reason they tested this way is that the DDR2 chips don't quite have the timing of their DDR1 counterparts yet. Even so, if you claim to be comparing two different technologies, wouldn't you at least run one test with them set up equally?

Whenever I buy a new computer or new memory, I always run it through a memory tester I boot from a CD. It's good to run one of these for at least 24 hours. If you find any memory problems, you might want to return the product.

It's also fun to benchmark it after the tests are done. I used SiSoftware Sandra BenchmarkHQ.

One thing to look out for though is BIOS programs that load into memory addresses that will effect the test. I kept getting errors until I disabled my onboard IDE RAID controller, and then the memory checked out fine. I guess the controller was loading something into the memory and the test would read it back after the controller changed it.

You mean they're not just using this?

Why don't you just get memtest86? It runs on the bare metal without an OS.

Funny. I would have just run MemTest from a floppy (or pxe boot at work) instead of wiping the whole OS. Then again, I have a dual boot machine and use linux 95% of the time, anyway.

the java jvm can lock up hard. makes recovery quite interesting.

also java and .net are "successfull" b/c of general investment from big companes. there's lots of marketing dollars selling products and articles about these platforms. the PHB's read the PHB magazines, and those mags have articles re java and .net. Do those mags have articles on D? then it's not a competition.

Java is successfull because:

• it offers a nice abstraction for system specific issues which is only seldomly leaky.
• you can dive into an unknown project, select a random source file and understand it. You may have problems getting the big picture, but the code itself is there - there are no suprises like operator overloading, defines etc. All you need to know about the class is in it (and it's superclass and interfaces)
• its complex enough to do some magic in it, but the idiot next cubicle can't run totally amok and wreck the whole system.

I'd still run a memory check anyway, just to double check there's not an issue with it. It's virtually the first thing i check when i get random lockups/crashes on any system.

you can grab a memtest86 bootable iso (only a couple of meg) which is simplest, or you can stick it in lilo or on a dos floppy.

Another thing which is always worth double checking is making sure dma is enabled (hdparm /dev/hda)

Anyway, if you're happy using icewm, kde and gnome are both probably overkill anyway. If you want a bit more candy though, you could try out xfce, it's a gtk powered DE that's not as chunky as gnome.

Other than that, I'll have to leave you to the tender mercies of the debian mailing lists ;)

Its really the damnedest thing, runs great, soon as he steps within ten feet of it, it goes stupid.

Maybe it doesn't like the magnet in his pocket.

Seriously though, I've seen quite a few kernel panics, on the machine I'm using at this moment. Every last one of them was a direct or indirect result of bad RAM. I'll take this opportunity to point out a wonderful tool called Memtest86. It diagnosed the problem in minutes.

Based upon what you describe it sounds like either your video drivers are killing you or you have some bad memory. Given that both Diablo II & VS are crashing so erraticly I would put my money on the memory.

You can test the memory with memtest86. If it is the video drivers though unfortunately there isn't much you can do (as you said you already have the most recent drivers).

But given that you've seen one machine work GREAT for you (and I would imagine with Diablo II, or do you only play that at work? :) ), and you have one machine that's flaking out, how can you determine it to be the operating system? That's clearly not what's different between the two machines!

There are many tools which come with today's GNU/Linux distributions that can be used to test hardware. Knoppix and LiveCD distros are useful for their portability. Full-blown distros can be even more useful due to the sheer breadth of tools that they contain. GNU/Linux drivers and tools can usually be made to output large volumes of information on what they're doing and what they've found. Being open source, they can be more fully understood. Tools can be combined to perform all sorts of tasks and tests.

I have used The Linux Hardware Stability Guide at IBM DeveloperWorks (Part 1 | Part 2) to test and tweak a number of systems. I have found the best stress testers to be a looped kernel compilation, cpuburn and Memtest86. Mprime is good as well.

I suggest you make a Knoppix CD with LTP installed. With a little configuration, that will take care of all of all your tests for the memory, disk, IO, and CPU. You might want to install America's Army or something to test the video subsystem.

If you put a little effort into it, you'll have a test suite as good as, and likely better than, anything you could pay money for. If you want to buy something, you can make a donation to the LTP and Knoppix projects.

There are also simpler tools, like Memtest86. I find this tool to be invaluable when I try to salvage old hardwar. I can't begin to tell you how much time it's saved me that I would have spent aimlessly swapping components around.

Going back on topic, let me share what I've learned about troubleshooting hardware issues. First, let me stress that it's usually not a hardware issue. Drivers, resource conflicts, and buggy BIOS/Firmware code is usually the issue. When it comes to "diagnostic" software for real hardware problems, there's little you can do. For memory, there's MemTest-86. This program hasn't failed to spot bad memory for me yet. For hard drives, go to the manufacturer's web site to get a bootable diagnostic floppy. Usually the quick tests are all that's needed, but it could pay off to do the extended test if the quick one says everything's OK. Even if the extended test comes back fine, don't rule out the drive. The problem could be intermittent. For the rest of the hardware, simply swap components around until you narrow it down. Take out that video card and toss in a spare one that you've had for several years and know works. For the CPU, toss in any other CPU that matches the original's architecture (Just a different speed rating is OK). I think you get the idea on the (Swapping out) method. Also, never underestimate the crap a poor power supply unit (PSU) can throw at you! Stability issues could be coming from an unreliable current. Of course, temperature problems could also cause these same stability issues, but that's much easier to monitor. When you get to a point where you're truly stumped, some manufacturers have forums that you can go to for help. This is especially true in the homebuilt market with Abit, VIA, AMD, etc.

Good luck and happy hunting!

-Lucas

memtest86 gives you a bootable floppy that will scan *all* RAM in a system, and turns up the most obscure memory errors. Some errors are not consistent, they only appear in transitions from one bit pattern to another, for example. Or adjacent bit cells may bias the bit in question.

That's all it does, but it's good. And it's free. One other point, systems with mismatched parts (designed for different bus speeds or timings), and overclocked systems, may generate memory errors. Since I started using memtest86, I've stopped overclocking, as every single overclocked system I've checked has shown errors under memtest86!

free memory tester here

The only effective hardware tests I've used in the ten years I've been supporting all kinds of hardware and software have been SpinRite and Memtest86. Between these two, I can check for the most insidious and hard-to-detect hardware problems; i.e. flakey hard drives and RAM. A cheap $20 POST card is highly useful for dead machines. You don't need all the extra features the Microscope card gives you unless you are designing motherboards or doing some other such serious work. No software will replace your own experience and ability to know where a problem is forming based on the specific failure of the machine. All the rest of the so-called diagnostic software is more or less useless from a practical perspective, aside from testing serial ports with loopback plugs and printing cute certification reports for anal customers. This is detective work. You have to suss out the exact problem based on clues left by the failure of the system. Learn how the hardware works, and it's easier than you think.

The best tools for checking memory are memtest86 and the follow-up memtest86+.

Both of these are free to download and use. I usually leave them running for roughly 24hrs for a reasonable level of confidence. You should also burn-in the other major components too but memory is the best place to start.

While it's not an "all-in-one" diag software, MemTest86 seems to find problems with memory very easily.

When ever there is a problem with a PC I might be working on, if the problem seems sporatic, or I can't find the simple solution, I turn to MemTest86. It's free, it has one job, and it does it well.

THE memory tester for PC computers:
Memtest86

memtest86 is free (GPL, no restrictions for use). So that handles memory. Works great, but can be time consuming.

Okay, so that's just one thing, but I'm sure you'll get buried with suggestions for other piecemeal items.

CPU Burn