Slashdot Mirror
Recovering Secret HD Space
An anonymous reader writes "Just browsing hardocp.com and noticed a link to this article.
'The Inquirer has posted a method of getting massive amounts of hard drive space from your current drive. Supposedly by following the steps outlined, they have gotten 150GB from an 80GB EIDE drive, 510GB from a 200GB SATA drive and so on.' Could this be true? I'm not about to try with my hard drive." Needless to say, this might be a time to avoid the bleeding edge. (See Jeff Garzik's warning in the letters page linked from the Register article.)
Sorry, but this is complete bullshit.
Did aureal density technology increase to 200GB/platter overnight? No.
Please refer to this thread on StorageReview.com for more information.
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra
I'm a Ghost developer.
This is just a method of corrupting your partition table so the same disk sectors appear more than once. If you try this, don't ask Symantec for help afterwards.
So either the whole thing is a hoax, or, more likely, the OS is looking at a damaged drive (damaged partition table, at least) and seeing the same partition in multiple ways. Try to write on that shiny new partition and you'll be overwriting data on the old one. Guaranteed.
Some drives are known to short stroke their platters. This raises the more serious problem of this idiocy... The problem is modern drives store important information on those hidden inner areas of their platters (firmware, disk information, reallocated bad sectors), who knows what you could be overwriting whenever you use that space. Put something down in the wrong place and the drive will never start again or corrupt data at certain sectors. It's a lottery ticket everytime you write data in that partition. That's not what I call useable capacity.
Don't believe me? Go ahead and try it. You'll lose all those Buffy episodes you've downloaded on KaZaA, and instead you'll have to spank it to the Portman pictures your mom doesn't know you have stashed under your bed.
The theory of relativity doesn't work right in Arkansas.
Then what kind of disks did you use? I did that to literally hundreds of disks more than 10 years ago, and they still work perfectly today; I've used some in the past week.
New news forum for Canadians - CanadaSpeaks
'A representative for large hard drive distributor Bell Micro said: "This is NOT undocumented and we have done this in the past to load an image of the original installation of the software. When the client corrupted the o/s we had a boot floppy thatopened the unseen partition and copied it to the active or seen partition. It is a not a new feature or discovery. We use it ourselves without any qualms' Which, having worked for a PC sales company, I can confirm is true. And certainly, while earlier models had partitions you could wipe with partition software, later PC builds had this hidden space. But the space was 1GB at most - there's no way there was the kind of 40GB plus hidden space the article claims.
I think posting in the "letters" linked article sums it up pretty well:
About the "recover unused space on your drive" article:
Working for a data-recovery company I know a thing or two about harddisks....
One is that if the vendors would be able to double the capacity for just about nothing, they would.
All this probably does is to create an invailid partition table which ends up having:
|*** new partition ***|
|*** old partition ***|
overlapping partitions. So writing either partition will corrupt the other. It probably so happens that whatever situation people tried it, it just so happened that the (quick) format of the "new" partition didn't corrupt the other partition to make it unbootable.
And the 200G -> 510Gb "upgrade" probably has ended up with three overlapping partitions....
Roger
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
I might note that it is the inquirer, not the register. Some editors might take offense ;)
In 1994 I bought a box of 720K single-density floppies by TDK. After discovering that making this extra hole could double the disk capacity, I crudely bashed the holes in them with the end of scissors.
These floppies were used almost daily for 3 years. (no hard disks available at that time). They were reformatted countless times.
Not single one of them ever failed. About a year ago, when failed to reformat and make a boot disk from several fresh-brought floppies I digged up one of them, reformatted again and succeeded in making a reliable boot disk.
Quality of todays media just makes me cry.
Taken from the afore-mentioned StorageReview thread:
Find it here
Mod down!!
but in case you are not:
HD are sold in GB with GB "defined" as 1,000,000,000 bytes, which is ~7.4% less than a real GB (2^30 bytes). After formatting, (depending on your FS) a extra few percent goes away for your file table, sector marker, directory structure, etc. so in real GB (in units of 2^30 bytes), it'll be a lot less than 160, or whatever your "bought" size.
Don't expect to recover those.
RAM is sold with truthful advertising. 128MB = 128*2^20 bytes, which is like 134,217,728 bytes - despite the 134, it's still 128MB.
My life in the land of the rising sun.
Back in the day of MFM and RLL controllers, the hard drive controller did much of what the drive electronics and firmware do in modern hard drives, that's why you could have MFM or RLL controllers. Hard drives still use RLL encoding today.
Before Ghost was bought out by Symantec it was a NZ company so there are probably still NZers working for them.
CHECK IT OUT before you rape your hd
Disks of today have no direct mapping from head, cylinder and track number to physical location on the platter. Rather there is an internal table of the mapping with room for remapping potential weak sectors to unused space. When the head signal is getting close to be inconclusive the just read sector is written at a spare sector, the mapping table is updated, and the old one is marked as bad.
If this article had show how to manipulate the disk so a number of the spare sectors could be used for enlarging the disk it would have been interesting...
:-) = I am happy
:^) = I am happy with my big nose
C:\> = I am happy with my OS
I used to do a lot of data recovery... lemme tell you whats happening here.
:)
Remember the "Good old days" where hard drive sizes were sub 540mb - We addressed hard drives using C/H/S size (Cylinder/Heads/Sectors) - It was common to scandisk and start seeing bad blocks (sectors) on your hard drive...
When we broke the 540mb 'barrier' we quit using C/H/S mappings and started using LBA mode, Logical Block Addressing. What this effectively did was take control of the physical drive access, data storage and retrieval, away from the operating system. This was because the OS/Bios would only recognize a maximum of 512 Cylinders.
Quick facts about hard drives:
1) There are *ALWAYS* defects on the hard drive surface. There is no such thing as a flawless platter.
2) As hard drive sizes have increased, all the innovations have taken place in your head.
Yes, there have been minor changes in the platter structure. As rotational speeds increased, sector sizes decreased, and operating temperatures increased, manufacturers had to move away from aluminum platters as they would shrink/grow too much as the drive reached operating temp. So they moved to glass. -- The surface of the drive has always been coated using the same exact ionization process.
However, the read/write head is where all the innovations have taken place. Because the size of the bits are getting smaller and smaller, a surface defect that previously would only wipe out a single bit would now wipe out an entire sector. For this reason, drive manufacturers allocate plenty of extra space on the drive to move data from failing areas of the drive (which is happening all the time). This drive maintenance happens independant of the operating system on the PC. It is an operation of the hard drive firmware. IT IS AUTOMATIC.
After drive manufacture, there is an initial low-level format of the drive (platter) where the drive establishes its sector boundaries. This is when it maps out the defective areas of the drive and stores it in the eeprom. As the drive operates and sectors fail, the drive automatically moves the data to a different area of the drive. These areas where the data is moved to are typically adjacent to the defective area. Space allocated to compensate for defects can be as much as 100% of the original drive space.
If the drive didn't maintain itself, then you'd see TONS of surface defects whenever you run scandisk, even on a brand new drive.
Think about it, when is the last time you ran a scankdisk and had it come back with surface errors. It doesn't happen anymore.
Anyhow... What these guys did was use a utility that creates a quick and dirty MBR(Master Boot Record) that likely archives the legitamate MBR within the 8mb partition while it does its business. These bozo's have essentially wiped out the MBR (READ: Defect Map) and formatted the full capacity of the entire disk.
Sure, you can install an OS, even run it, but as the hard drive tries to manage itself... well... I've explained enough here, be it suffice to say that you're fsck3d.
This isn't like Intel that creates a single chip and labels it 3 different speeds (The pentium 75/90/100 comes to mind) where you can overclock it...
Good security is based upon reality and common sense. Common sense is a function of having common knowledge.
The only saving grace of this article it that even the most intelligent person would have trouble following the Computurs-Fer-Nascar-Dads style instructions. From the article:
...
It has worked completely fine with no loss before and it has also lost the data on the drive before. (so it obviously WILL 'lost' your data)
Do not try to delete both partitions on the drive so you can create one large partition. This will not work. (this is because they are overlapping and you won't see 'extra' space if you delete the overlap)
You have to leave the two partitions separate in order to use them. Windows disk management will have erroneous data (again alluding to the error in reporting space)
in that it will say drive size = manus stated drive size and then available size will equal ALL the available space with recovered partitions included.
I can tell your intelligence by your signature.
..The 120GB hard drive you purchased may have been physically identical to a 250GB hard drive, but simply it only passed qualification at 120GB.
This is possible and is regularly used by HDD manufacturers (if you bothered to read the article)
Intel does the same thing with processors. A 3.0Ghz processor may be sold as 2.4Ghz, simply because it didn't pass qualification at 3.0Ghz but did at a lower clock speed.
all hard drives reserve a certain amount of free space to use for reallocation of bad sectors. These "spare sectors" are free space on your drive... completely unused until your hard drive starts finding problems on the physical media.
The IBM Thinkpad (R-series atleast) has 4 Gb of hidden diskspace that you can enable for ordinary usage in BIOS.
It sounds fairly little, but on a 20 Gb drive that's 20%
Usually there is some kind of backup-image there, but it isnt really necessary (especially for us Linux people).
GAAH! MY PRINTER IS ON FIRE!!! PUT IT OUT! PUT IT OUT!
But yeah more then doubling the HD capacity sounds fishy and there are plenty of letters to the inquirer article explaining how and why it ain't true.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
CLV is constant linear velocity and is what the first generation CD players used. That meant the data passed under the head at a constant speed, 150kbytes/second. The further out on the disc the slower the disc turned as each turn had more data than close-in.
Once the speeds went up the manufacturers moved to CAV or constant angular velocity where the disc spins at a predetermined speed and the data comes in at different rates depending on the head position over the disc. What really happens is there's a table of different CAVs stored in the drive's firmware depending on the absolute position on the disc. Close into the hub the disc spins faster, further out it spins slower. If there are a lot of errors it will slow down to try and read the data better. On a 48x drive there might be as many as 12 different CAV speeds available to the firmware.
Company A gets the business of people who are willing to shell out $200 for a 200 GB HDD. Company A does not get the business who have a budget of less than $200 for their HDD purchase.
Company B get the business of people who are willing to shell out $200 for 200 GB HDD and the business of people who have a smaller budget.
Company A buys company B. The new Company AB sells both 150GB and 200GB drives, so they get money from everybody.
Except, of course, that Company AB is in competition with Company C, which makes a real 150GB drive which costs less to produce than company AB's "150GB" drive because it's not really a 200GB drive with modified firmware. Company C sells their 150GB drive for less, and starts driving company AB's margins down; Company C can keep doing this because their costs are lower.
$x='S24;r)>63/* h@<5+oZ)32"5cz';$me='phroggy'x$];
$x=~y+ -xz+\0-Tx+;print$_^chop$me for split'',$x;
Notice how they say an unpatched version of ghost is required:
Ghost 2003 Build 2003.775 (Be sure not to allow patching of this software)
That's because the patched version fixes A BUG that allowed the "ever expanding miracle".
I can't possibly see how this would work. They're reporting a (more than?) 2x size increase on the largest harddrive they alledgedly did this trick on.
If it works at all, all it really accomplishes is trick windows into thinking the partition really is bigger than it is. There's NO WAY it could get any bigger in reality, since drive capacity is based on the number of sectors the drive reports to the computer, and that is a fixed, hard-coded number that can't be changed by Norton Ghost or any other utility. If you try to address sector maxcapacity+1, you'll just get an error message back from the drive, it won't actually do anything.
This is just a case of someone making sh** up in order to appear on the front page of hardware websites... A bit like participating in a 'reality show' on TV.
You're joking right?
On the subject of the Inquirer article.
The 200JB, or BB or whatever is clearly impossible. There is no hidden space on them to recover at all, let alone 310GB! I can't imagine what kind of idiocy provoked someone to believe that was even possible. Western Digital doesn't make drives with more than 3 platters! The 200GB Western Digitals are only available with 80GB/platters. They only have 5 heads. It's therfore impossible to recover any capacity from them at all (5*40GB=200GB).
Some of the other drives are known to short stroke their platters. This raises the more serious problem of this idiocy... The problem is modern drives store important information on those hidden inner areas of their platters (firmware, disk information, reallocated bad sectors), who knows what you could be overwriting whenever you use that space. Put something down in the wrong place and the drive will never start again or corrupt data at certain sectors. It's a lottery ticket everytime you write data in that partition. That's not what I call useable capacity.
Also, if this was working properly, the 80GB deskstar would yield:
either 90GB (+10GB) if it was a 180GXP (three heads on 60GB platters)
or 80GB (+0GB) if it was a 7K250 (2 heads on 80GB platters)
Anyone with most basic knowledge of hard drives should know that most of the numbers up there are simply impossible, not to mention simply ridiculous.
It's not that there aren't hard drives which are short stroked and sold at a capacity below that available for access in theory, but that something is clearly wrong with this method in that it is simply inventing space that physically can't be there. Perhaps hard drive manufacturers are shortstroking disks to the point that they are formatted with the capacity of drives with fewer platters or heads, but this could never justify the failure of this method on the 200GB Western Digital drive. This drive is a known quantity. No matter what, even if they got a disk that was a shortstroked 6 head drive (which would make no sense), the maximum capacity is 250GB, not 510GB. You would need 7 platters to get that capacity with todays technology!
Not quite true. This only affects how much the super user has reserved. You'll see that df reports the same for the 'size' column. The Avail column goes up simply because it reports with respect to what a normal user can write. System files owned by root could still be created when no space was available for user owned files. -m is not for file system repair, it is so that no user can make the system unusable for root. Don't set to zero. Even if your private workstation, if something goes awry and consumes your disk space as a user, your system can still log, can still write system tmp files and do that sort of thing allowing the user to fix the situation or else a super user to still log in, work with the system, and rectify the situation while still having persistant storage to work with.
XML is like violence. If it doesn't solve the problem, use more.
Yeah, its amazing.... I changed the partition table without updating the vfat table and put an ext2 filesystem in the second partition.
The vfat partition stayed the same and the ext2 partition was non-zero size... woah....
Its just pesky random file corruption on both partitions you have to worry about...
In all seriousness:
*THIS IS VERY VERY VERY DANGEROUS* DO NOT DO THIS *PERIOD*. It may give neat apperances at first, and both filesystems may appear fundamentally functional, but it will *CORRUPT DATA* when the first partition is populated enough to creep into the partition overlay.
XML is like violence. If it doesn't solve the problem, use more.
The Host Protected Area is space on your hard drive that your bios, your operating system or even your applications can be set aside for certain management information. I take it that some backup programs (ab)use it to "hide" compressed boot images on hard drives. I wouldn't be very surprised if companies like Dell or IBM stole some of your hard disk so you can restore a windows installation.The "Host Protected Area" has nothing at all to do with the drive-internal handling of bad sectors or other drive-interal.Drive-internal information as well as sectors used for replacing sectors gone bad are not accessible through the ATAPI commandset for accessing the HPA.
The ANSI T13 Standard Document for ATAPI-6 (current) are overprized at $18.00 but you can download a draft of upcoming ATAPI-7 from the T13 working group's site at http://www.t13.org. There you will find in Section 4.9 of the document: "A reserved area for data storage outside the normal operating system file system is required for several specialized applications". Systems may wish to store configuration data or save memory to the device in a location that the operating system cannot change. The optional Host Protected Area feature set allows a portion of the device to be reserved for such an area when the device is initially configured. A device that implements the Host Protected Area feature set shall implement the following minimum set of commands:"
READ NATIVE MAX ADDRESS
SET MAX ADDRESS ... ...
I take it that READ NATIVE MAX ADDRESS tells you how many sectors of user addressable space have been configured on the drive and SET MAX ADDRESS lets you adjust that.
The way I see it there may be a lot of preinstalled hard drives out there with a compressed windows installation images on them "hidden" in the HPA. Maybe a new version of hdparm will allow linux users to reclaim that dead space.
but, just becuase the FAT table says that the partition is (x) size does not mean once you get past the true phyical limitation of the hard drive does not mean the whole house of cards is not going to come down.
I too could use Norton Disk Edit to make the FAT table say lots of other intresting things...
Like I had a 300 gig drive on a 20 gig.
It's called a currupt FAT table.
"Select the file system type you prefer and format with quick format" This should be your first clue, this only rewrites the fs table (TOC).
It sounds to me like this is simply a case of ghost screwing up the geometry settings in the partition table, and then ofcourse there is yet another Windoze bug to exploit it - sorry, I mean get hosed by it...
This sounds sort of like something I used to do for automatic installation way back when, use 'dd' to dump then entire contents of "hdX" to some file
# dd if=/dev/hdN of=/tmp/dump
then dump the contents of that file to another HD that is the same size or bigger.
# dd if=/tmp/dump of=/dev/hdN+1
The result is that everything will work just fine, and running fdisk (on Linux) will show an uncorrupt partition table, BUT that geometry (obtained via BIOS) shows a much bigger drive, but DO NOT save the resulting table (w) - fdisk will rewrite it and then hose everything up! Pretty much just the opposite of this method....
I followed the directions to the letter. I ended up with a 1GB drive! (On a supposedly 540MB drive. In the end, FDISK claimed 965 MB.) I filled up the first partition (with mp3s, naturally.) I then started filling up the second partition...
Surprise, surprise. It crashed halfway through copying the mp3s. Reboot? BZZZT! Windows 98 crashed a quarter of the way through loading. Starting up from a DOS disk, and my directory structure is all frooed up on the C partition. Filenames with random ASCII characters in them, inaccessible directories, all sorts of data corruption goodness. The D partition had correct names, though. (So my second batch of mp3s was probably fine.)
(Or, more specifically, do not try this on a hard drive you want to keep, or with data you want to keep.)
Another non-functioning site was "uncertainty.microsoft.com."
The purpose of that site was not known.
ATI is a perfect example I think. Ya'll remember the various mods to convert their otherwise identical top-of-the-line video card into their top-of-the-line 3D rendering graphics pro card? Sometimes the designs are basically identical for good reason. Cost savings comes to mind. They simply use software and/or a few well-placed jumpers to differentiate between the two.
I used to use the program the parent speaks of, and it really did work. The format tool let you adjust the number of tracks and sectors on a floppy, with the 1.72 Meg combination working well but anything beyond that not working right. The space gains were quite real, back when my hard drive was a mere 40 megs I used this to offload things and make room. It used a small TSR program (i.e., a memory-resident driver) which had to be loaded, or you would get errors trying to read the disks.
I wonder how many slashdotters (including me) hooked their MFM hard drive up to an RLL controller to get that extra 50% out of it?
Now that's kickin' it old school.
60MB out of an ST-251, baybee!
Chris Owens
San Carlos, CA
As someone who QA'ed Ghost 2003 for Symantec, I agree with you. The VPSGHBOOT stands for Virtual Partition Symantec Ghost Boot. Notice the word Virtual.
The bits actually reside in a contiguous sector file in the root of the primary partition. This file may be 8-100MB. If your disk is too fragmented, Ghost cannot create it.
The real reason for this stunt file is to eliminate the need for a boot floppy to launch Ghost (a PC-DOS 7 program compiled with DJGPP)
Hey, Mom! Is it beer, yet?
I believe they did something like this with the 486's as well. I believe that their manufacturing process started getting so good that they weren't turning out enough 20 and 25MHz chips - just lots of 33's and up. However, people weren't willing to pay for the higher-end processors. So, Intel segmented the market by selling the same product at two different prices - albeit rebranded in the one case.
In that case they probably weren't afraid of Cyrix/AMD so much as maybe the Mac - I don't think that Intel had a whole lot of competition on the 486.
Once a fab process has had the kinks worked out, they chips undergo much less thorough speed binning. Intel often uses dies near center of the wafer(where focus is more exact) for higher speeds and dies nearer the edge of the wafer for lower speeds. It's a lot simpler than testing every processor at every speed.
the entire chip is "scale-free" which means it is designed to work at a variety of speeds and tolerances.
HOWEVER! The manufacturing process is much more of a crap shoot. You have to grow this perfect layer of silicon in the shape of a disc (usually it's cut from a cylinder), and grind it to be incredibly smooth. It has to be perfect. Then you expose it to one chemical, then light which reacts with it, then you expose it to another chemical to leave behind something where the light hit. And you do this over, and over again to deposit layers of different dopants to the chip to build it's structure.
Except if the tiniest bit of dust, or particle gets in the way, that whole chip is ruined. And you can't make it in a vacuum, so you have to have filtered air. But even then, you can't filter perfectly, so you have some loss.
And even then, the wafer is not guaranteed to be 100% flat all over to within a nanometer (whereas the chip components themselves are only 130-90nm these days) so there is going to be some chips whose parts are better lined up or formed more evenly than others, overall.
So you make about 200 or so on a wafer, then cut them apart and test them, to see which ones work, and how well they do.
It's the manufacturing that makes the cost-competetive tradeoffs...
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
I once had an compact flash hard reader, that for whatever reason, couldn't properly access the partition table of the CF cards. I was the greatest thing though!! Those crazy CF card companies were hiding Gigabytes of space from me. Here were my results: 32 Mb -> 60 Gb 64 Mb -> 40 Gb 128 Mb -> 90 Gb And best of all, I have one very special CF card: 256 Mb -> 1.2 Tb. Yes, I acutally had this happen, right there in the logical disk manager under Windows XP, the disk showed up as 1.2 Terabytes. It was great hearing SimpleTech's support guy: You what!!?? A 1.2 Terabyte CF Card? He said I should hang on to it... I did. Later on, I got a Zaurus, and just for kicks popped in the CF card. A few commands later, I had rebuilt the partition, and was back in business. Bottom line: Busted partition tables != extra space.
MFM and GCR, respectively.
--