Some Windows Apps Make GRUB 2 Unbootable

KwahAG writes "Colin Watson, one of the Ubuntu developers, published in his blog information about Windows applications making GRUB 2 unbootable. Users of dual-boot Windows/Linux installations may face the problem, which boils down to particular Windows applications (Colin does not name them, but users point at least to HP ProtectTools, PC Angel, Adobe Flexnet) blindly overwriting hard disk content between the MBR and the first partition destroying information already stored there, in this particular case — the 'core image' of GRUB 2 (GRand Unified Bootloader) making the system unbootable."

Comments

I thought nothing was supposed to be there

[guruevi]

... and that's the reason why BIOS 'virus protection' blocks access to that portion of the hard drive. Too bad that DRM breaks everything once again and too bad the mainstream of users isn't affected by it.

Re:I thought nothing was supposed to be there

[mysidia]

Nothing is supposed to be there except the user-installed system boot code, boot data, and hard drive parameters.

Third party software certainly has no business messing with Sector 0 or the boot blocks unless it gets explicit permission, advises users of the risks in messing with the boot block, prompts the user to back anything up that's there right now, and writes its bits only to the portion of the boot block that is provided for its required purpose.

It may detect bootloaders, and update their configuration, if the user accepts that, but bootloader configuration is generally stored on the boot volume not the boot block

Re:I thought nothing was supposed to be there

[FuckingNickName]

The "boot block" is precisely one sector right at the start of the fixed disk, with some space being taken up by the primary partition table, signature, etc. The problem is not Grub (and certain Windows software) writing to this area, but writing to unpartitioned space elsewhere on the drive.

This is as wrong as looking at some filesystem, discovering that certain free blocks are unlikely to be allocated, and then using that space for storage.

HP ProtectTools

[Nemyst]

Protecting your laptop from open source commies. And maybe viruses.

Re:HP ProtectTools

[Joebert]

I'll just leave this here.
http://h30434.www3.hp.com/t5/Operating-systems-and-software/New-laptop-harddrive-non-OEM-Vista-disk/m-p/314927

WTF is the "embedding area"?!

[Anonymous Coward]

WTF is this "embedding area?" It sound like GRUB is misusing the disk geometry to find unused space and then getting upset that other programs do that too.

Googling for "embedding area" find that it's a term that GRUB 2 made up and that it's not really a part of anything. In fact, apparently this space doesn't even exist under EFI systems, and that this "embedding area" is an artifact from DOS.

So, basically, GRUB is misusing the disk to store information in a place it has no right to be touching, and then getting upset that other people make the same mistake. Genius.

Re:WTF is the "embedding area"?!

[Spazmania]

It makes sense for a bootloader to place data and code outside of partitioned space. It makes more sense to place the code inside a partition, even if it's a one-track partition dedicated to the bootloader. If they collided with components of Windows' bootloader or FreeBSD's bootloader, or some pre-boot hard disk encryption software I'd have little sympathy for them.

On the other hand, user-level apps storing data on the hard disk outside of partitioned space is very bad mojo. They should not be doing that. Ever. Period.

Re:WTF is the "embedding area"?!

[FuckingNickName]

Bingo. It is absolutely wrong to put data outside of partitioned space, and it is insane to blame something else for your own bug. Indeed, one security measure when installing a new system might be to zero out all unpartitioned space and then make sure nothing is ever written to it - Grub makes this impossible.

Grub should use an existing partition to store all the bits which don't fit inside the MBR, following the lead of EFI system partitions if necessary but supporting various common filesystems otherwise. Instead they use an atrocious hack to try to make things look neat.

Re:WTF is the "embedding area"?!

[alexhs]

It's also called "GRUB with blocklists"

You can find more here,
and in my other post

Re:WTF is the "embedding area"?!

[0123456]

It makes sense for a bootloader to place data and code outside of partitioned space. It makes more sense to place the code inside a partition, even if it's a one-track partition dedicated to the bootloader.

It would, if you could actually get more than four partitions on a hard drive with the 90+% of BIOSes which can't boot properly from a GPT drive.

My new laptop came with _THREE_ recovery partitions and a Windows partition, so I had to delete one of the recovery partitions to be able to install Linux at all... where would I get another partition for Grub to run from without deleting all the recovery data?

So the big problem is that we're still stuck with shitty MS-DOS disk formats from the 1980s.

Re:WTF is the "embedding area"?!

[MBCook]

If it's wrong to put data outside of partitioned space, what are these user spaces apps doing writing there? I can see a pretty good case for boot loaders doing this (the comment below about the 4 partition limit is one). Why is a copyright/licensing program writing there (which is what Flexnet seems to be)?

What's to prevent one of these programs from overwriting the data another makes? How would you like it if every time you ran NewSuperGameWithDRM, Photoshop lost it's license and forced you to phone home to reconfirm it?

Re:WTF is the "embedding area"?!

[Drinking Bleach]

What's wrong with FAT? Pretty much everything from space inefficiency to lack of permissions to filename restrictions to reliability issues, especially after unclean shutdowns.

Also GRUB's core.img already only supports one filesystem -- whichever one is /boot (which can be FAT, GRUB can do that as well), filesystem modules for other types if necessary are loaded directly from said /boot partition.

Re:WTF is the "embedding area"?!

[vadim_t]

I wasn't imagining that I was forced to manually type out a list of blocks occupied by the file. But I was concerned by exactly what you say. Dear God, why do it like that?

It's the same thing LILO did, which is why most people use Grub now.

And the reason why is because the MBR is tiny, and has no room for code that reads say, ext4.

The MBR did the boot menu and loaded the boot sector from any given partition. That boot sector would do as you say. You don't need to "bet" - it's pretty much what I said :-).

The MBR has no menu. The basic stuff is "find active partition, load first sector, jump to it". With Grub it's more like "load code from embedding area, run it". Which contains enough to read things like ext4 to load the rest.

Really? So why does GRUB need any extra-partition space?

Because there's no room for filesystem reading code in the MBR. Especially not for reading all the formats Linux supports at once (what if you want to boot from FAT, ext3 and zfs?)

If you're thinking grub should load the code from some fixed space in the Linux partition, then every single FS would have to agree to reserve that space. Including the ones like JFS that come from elsewhere.

Why wouldn't you be able to retrieve the boot sector of an extended partition? Obviously some operating systems (Windows) will assume they're booting off a primary partition and break unless their boot sector is tweaked, but this isn't inevitable.

In my understanding, a partition having a boot sector is a DOS convention, that other filesystems don't necessarily follow. I think 512 bytes at the start may be mostly guaranteed, but again, you're not going to read things like reiserfs in that little space, so you're back to having the same problem.

Which is why it should load a second stage from a system or other partition.

It can't read it from "other partition" because if there is a filesystem there, it has to understand it, and 446 bytes is not enough.

If you mean a special, reserved partition, then that reduces the number of primary partitions for other purposes to 3, which creates compatibility issues. And if there are 4 primary ones already, you're screwed.

Resuming: the way x86 computers boot sucks, and boot loaders have to be written with those constraints in mind. The whole "embedding area" is a horrible hack, but the alternatives have significant issues as well.

Re:WTF is the "embedding area"?!

[thsths]

> Though adobe in this one looks like they deserve to be slapped around a bit, if the conjecture is accurate.

Adobe deserve to be slapped around a bit (and then a bit more). Period.

Otherwise I think the problem is (again) the BIOS. It only loads the 1st sector to boot, when 63 sectors (or 2048 with EFI) are reserved. Back in the old days you could just fit some FAT16 code in there to find the DOS image - but only at the expense of error handling. Nowadays you have to load the next stage from a fixed position - and the only position that is certainly fixed are the other 62 sectors. So they are the logical place for a boot loader.

You could add a boot partition, but with only 4 partitions available, that would use up a very limited resource. And I guess even if you put a boot partition into the first 63 sectors (which is now perfectly possible), Adobe would still overwrite it (and Windows would possibly freak out).

Re:Solution:

[EvanED]

A few years ago this would have been a much more fair question... now it's just troll/flamebait. I run as a limited user at both work and home, and for the most part it's installers and a couple other apps you'd expect which need admin rights.

(Even when Vista was new I kept a log of all the elevations I gave in a month or so, and with a couple exceptions (one of which has been since fixed and one of which was a stupid utility I didn't really need) they were basically on-par with what you'd need to 'sudo' to do in Linux.)

It is free for all region

[Technomancer]

While MBR has some function, the rest of sectors between MBR and the first partition was always a great area.
Many MBR viruses put their stuff there. Many stupid programs use it to store DRM data, so they can check whether they were copied to other computer
If GRUB is using this region too, it is equally stupid. There is no protocol for allocating this area and there is no guarantee that this data is not going to be overwritten by any other stupid program.
So nothing to see here, move aling, it is just Core Wars between stupid programs.
GRUB developers should have known better.

Re:It is free for all region

[sjames]

There is a fairly strong convention there that userspace data goes in partitions and boot loaders low-level stuff go outside of partitions. The "unused" sectors on track 0 have long been considered as reserved for boot loader. It's even in the original specs.

Yeah, viruses use that space sometimes, but by nature a virus ignores boundaries anyway, DRM, that is, software that hides itself from the user and makes the computer malfunction (by not doing the owner's bidding) is just a special case of virus.

Another example of DRM fail

[Andorin]

From the article:

At least some occurrences of this are with software which writes a signature to the embedding area which hangs around even after uninstallation (even with one of those tools that tracks everything the installation process did and reverses it, I gather), so that you cannot uninstall and reinstall the application to defeat a trial period.

So once again DRM is fucking with peoples' abilities to use their computers. Except this particular bit of DRM doesn't just screw with Windows; it could potentially screw with every OS on your drive (or screw with your ability to access them, at any rate).

Yeah, it's not conventional DRM, but it's a form of DRM in that it restricts the user in some arbitrary way (and, I ought to add, breaks something else in the process... that too should be part of the definition of DRM).

Re:Solution:

[mysidia]

This is not a problem for the most important Linux systems which are not dual boot.

Most systems that are dual boot are workstations, not servers. Meaning the person who uses the system every day is most likely using Linux.

I think the solution is for the Linux installer to create Windows icons and a Start menu item group with two things.... A "boot Linux" icon (for launching loadlin)

And a "fix grub" icon, for fixing grub, no matter what some dastardly windows program has done to it.

Re:So that's what happened...

[The MAZZTer]

IIRC there's a part of grub.cfg that is marked with comments to not be auto-replaced when grub takes inventory of your linux kernel versions. Put the Windows stuff in there.

Not surprised

[Murdoch5]

Got to say this isn't surprising at all. Windows has never favored the dual boot setup. In the mind of Microsoft, there product should be the only one to touch the drive and thats it. Personally I run 2 dual boot setups. 1 on my notebook and 1 on my desktop. The amount of times that Windows has chosen to just over write grub and leave me with no way to get into Linux is amazing. What Microsoft should do to show there a team player is put code into the install to detect a grub install and then append the correct entry into the grub file to setup the dual boot.

I know this will very likely never happen but it would be a good step to be taken by Microsoft.

Re:Move along

[Anonymous Coward]

Wrong, GRUB belongs in the MBR, not in some unpartioned space that is not supposed to be of use, if they have a problem with that, just keep that thing (GRUB) small or create a partition.

LILO is immune to this.

[John Hasler]

And yes, LILO is still supported and under development. LILO 23

Re:LILO is immune to this.

[X0563511]

... which is better than adding 3 lines to /boot/menu.list or /boot/grub.conf how?

I still see to fail why GRUB2 is a big deal (right now at least).

Re:LILO is immune to this.

[rodgerd]

I failed to see what the big benefit of GRUB was in the first place. It adds a huge amount of complexity for standard Intel boxes, minimal benefits, and when it was first jammed into distros, regressed all sorts of use cases (such as booting from broken software RAIDs).

Much like the Linux audio subsystems, it's a tail of throwing out something that works for 90% of users, replacing it with something of dubious virtue, and then declaring the remaining problems too hard to solve and moving on to the Next Big Thing (GRUB 2 in this case), while giving you a pile of new and insane problems to deal with.

Re:Solution:

[makomk]

Of course, Flexnet is apparently quite capable of making Windows unbootable too, at least if you're using TrueCrypt. Say no to badly-designed DRM!

Re:Move along

[osu-neko]

Does grub have any more reason to be there these other companies?

It does if I put it there. Nothing should be automatically written into partitioned space. Partitioning defines what areas of the disk I want to be automatically written to using whatever scheme I define by setting the partition type. Anything outside that, I'm free to manage any way I please. I can put a block-oriented FORTH program there if I like, individually managing "screen" loads and saves in the FORTH code. Or whatever. The point is, they're my blocks to do with as a like, and nothing should be written there except what I explicitly write there.

Among other things, it does mean that if I choose to write GRUB data there, it should be perfectly safe there. If it isn't, that's a serious bug in whatever program overwrote the unpartitioned block(s).

FLEXnet, Adobe's rootkit

[Animats]

The big headache is FLEXnet, Adobe's "license manager". It's a specialized rootkit that gives the remote licensing system access to the machine at a low level. Which is why it tends to break things a Windows application shouldn't be able to break. On Windows, it runs a background service and contacts a remote server frequently, sending undocumented information to the remote server and accepting update commands to change software already on the computer.

FLEXnet is the successor to FlexLM, a licensing system from the 1980s. It started as a UNIX product. It's been owned at various times by Highland, Globetrotter, Macrovision, and Thoma Cressey Bravo. It was unreliable in the 1990s, and the passage of time does not seem to have improved things.

In general, it's best to avoid buying Adobe products which install the FLEXnet license server.

Re:Solution:

[tinkerghost]

The second one is that if these apps need to be able to write to that section of the disk, they're going to ask for elevation.

OK, I can see AV software requiring raw disk access. I can't see why it would need to be able write to that section of the disk if there is no virus there.

Of the 3 programs listed, none are anti-virus. HP's software is for heavy duty keycard/usb dongle access to the computer - it might be trying to secure the bootstrap - however if that's what it's doing it should be replacing grub not just writing to the disk.

PC Angel is backup/recovery software ... WTF does it need raw disk access? It's not like your computer is accidentally going to be writing files outside the partition.

Adobe's netflex is their DRM. It's obvious why they want to write their information outside the partition - to make it harder to discover & alter - but I'll tell you that if I found a program doing that - I'd yank it off of any network I was running. You want to run on my networks, you color within the lines. I'm not wasting my time hunting down why a chunk of software is writing where it's going to be hard for my AV software to check it, I'm yanking it & tossing it in the trash.

Yeah, just a great idea to toss your proprietary code chunks into random places on the hard drive that 'nobody uses anyway'. It's a file system for a reason.

Unfortunately, the only company that's going to get any flak over this is Adobe. People are going to get work stations with the HP software installed & installing the netflex software will break it. Once that happens, Adobe will get called by "big important companies" and bitched at. HP & PC angel will merrily go on their way with only a few 'fringe crackpots' having an issue with their software.

Re:Move along

[Jorl17]

Oh, yes! Much like .NET is cross-platform! And the Windows API!!


Oh, wait...

"built his house upon the sand"

[alizard]

The whole point behind VMs is to make the host as reliable and stable as possible and put the flakier OS and software in a VM so when it crashes and burns, all one has to do is start the VM, not try to rebuild file structures and apps from scratch. Your post suggests you're not quite clear on the concept.

Unless you honestly believe that "Son of Vista" is more reliable and stable than Linux. In which case, I recommend you get help from a competent mental health professional.

Re:"built his house upon the sand"

[Jerry]

That's because Linux is 100% as vulnerable to ... Linux uses security by "obscurity"...

You really have things backwards. Linux source code is GPL freely available for anyone to inspect. Windows source is proprietary and secret, which Gates testified before Congress was necessary because it was a national resource that should be kept secret for security reasons ... until Gates gave the Chinese copies of the XP source because it was their price for Microsoft to do business in China. So, it is Microsoft that practices "security by obscurity".

Actual security? The 1,000,000 + zombies that are appearing on the giant bot farms discovered every so often are compromised Windows boxes, not Linux or Mac OS X boxes. Ballmer himself put the Linux desktop market share at around 10% and called Linux a greater competitive threat than Apple. With that percentage and, according to you Linux is equally as vulnerable, then why isn't 100,000 of those zombies Linux boxes?

And, if Linux is so easy to compromise then why did professional hackers spend more than 6 months last year just to capture only 700 Linux boxes using brute force password cracking when, according to you, all they had to do was spend a day or two to lure a few hundred thousand Linux users to their porn site honey pot?

Morons are those who drink Microsoft's Kool-aide and become brainless human zombies chanting MS Technical Evangelists astroturf postings as if they are fact.

Re:"built his house upon the sand"

[sumdumass]

Actually, 99.5% is a little high. Try something closer to 80%.

And yes, I'm speaking from the experience of cleaning systems where windows has Zero day vulnerabilities recently discussed here where simply going to an infected website was enough not to mention programs masquerading as legitimate installed programs because the legitimate programs draws it's window frames from IE and it's almost identical when it pops up saying "infection found, do you want to delete it". and lets not mention the recent network solutions app infections recently discussed here too.

We have a few sites that all web access is audited with log trails that even reveal what is types into word processors (the owners are anal about security and yes, working there is hell too). We tracked a few infections back to the ads served on some legitimate sites where the system was infected before the user had a chance at clicking anything. Of course our AV caught the infections but not before the winsock stack got hosed and we needed to reset it with the netsh command. And I stress again, this was with no user interaction other then going to a legitimate website for official work use as our tracking software showed. And this was also only a couple of weeks ago.

Turbo Tax Did It First

[McD]

We've been down this road before. In 2003, Intuit's Turbo Tax (for tax year 2002) pulled the same stunt, indiscriminately overwriting sectors at the beginning of the disk (outside any partition) and trashing people's bootloaders.

All in the futile pursuit of DRM. That's reason enough for me to use Tax Cut, instead, every year since.

Re:Solution:

[Beelzebud]

Or how about I continue to dual-boot, and use my PC the way I want to?

Re:Solution:

[jedidiah]

> Or how about I continue to dual-boot, and use my PC the way I want to?

You know the drill. Microsoft isn't going to cooperate with that. Now it seems so of their stooges will also "help".

Re:Solution:

[cgenman]

If I'm not mistaken, Flex is required for Photoshop, Illustrator, Dreamweaver, Flash, InDesign, and After Effects. Except for After Effects, you won't find any real professional-level alternatives for any of them.

Try telling upper management that you banned your $100 an hour designers, artists, and developers from the tools they need to do their jobs, because you were worried about bootloader compatibility and proper code behaviors.

Re:Move along

[Dahamma]

The way most other boot loaders have done it (including the original GRUB). Put enough code in the MBR to load the rest of the code and config out of a second location. The smart ones actually use a real partition for that, though, so no one overwrites it.

Re:Solution:

[jedidiah]

> Yea, article is somewhat trollish, all three apps listed are server apps, and who the fuck would dual boot a server?

In a "grown up" OS, the server apps don't run as Administrator.

Re:Reinstall GRUB

[couchslug]

"You and your grandmother likely have different definitions of "hard"."

As long as she's satisfied, I'll retain my definition.

Re:Solution:

[Yaa 101]

Say no to any DRM'd shit!

Nothing new

[eggman9713]

This has been a problem with older versions of Dreamweaver. As part of the copy protection, it would write data to the space between the MBR and the first partition. Steve Gibson talked about it on Security Now episode 132 (circa 2008) when discussing how this issue fubar'd TrueCrypt (unless you had a recovery CD) just after it came out with its whole-disk encryption ability.

Re:Malicious by definition?

[MrLint]

Are you implying that GRUB, which is a bootloader, and whose code is available, is writing in the boot area in an undocumented fashion?

One would presume that a bootloader is supposed to write in the boot area. One is not likely to presume a userland app in a high level OS is writing in the boot area.

Re:Move along

[Nimey]

Heh, funnily enough that's exactly what Windows 7 does. If you install it to an empty drive, it'll create two partitions - one small one (a couple hundred megs?) for the boot loader, and the rest for Windows itself.

Flexlm rant

[dbIII]

Flexlm is about as evil a piece of software I've ever seen. It only exists to punish the innocent that have actually paid for the licence and to fleece the software vendors that have paid for this bit of rubbish that is easier to circumvent than it is to use. Due to compatibility bugs I'm still running a fucking RedHat7.2 machine just to feed the other Centos5 machines a licence - so one machine doing nothing but burning electricity and handing out a licence. Running it in a VM would of course void the licence, as would one of the many simple workarounds to disable flexlm.
A later MS Windows version I had the misfortune to use had a Y2K bug in 2008! With an update our perpetual licences were marked as expired in 2000. It took two weeks to get a fix out of Macrovision.

Re:Solution:

[Anonymous Coward]

What about Gimp as a Photoshop replacement?

Re:Solution:

[Alex Belits]

Virtualization is the last refuge of a horrendously mis-engineered operating system.

Re:Solution:

[ozmanjusri]

Sounds like it solved a lot of problems then?