Sophos Free A-V For Mac May Kill Time Machine Backups

kdawson writes "Herewith the tale of the instantaneous loss of 19 months of Time Machine backup data, with the possible involvement of a fresh install of Sophos's new free Mac A-V package. Sophos support has been contacted but has not responded as of this writing."

Comments

seems about right to me

[waterwingz]

you sometimes get what you pay for.

Re:seems about right to me

[Anonymous Coward]

It's free and there is no warranty!

Now, click here to get your free virus scan and up to date protections...... BITCH!

Re:seems about right to me

[helix2301]

We had Enterprise Sophos and it broke everything and it let every virus threw. They just make a bad Anti Virus payed or unpayed.

Only if you tell it to delete them

[Anonymous Coward]

As he apparently did. Perhaps it wasn't clear enough, but it's not like it just randomly did it.

Also, backups are backups. He can just create new ones.

Re:Only if you tell it to delete them

[Anonymous Coward]

But he lost 9 months of his Mac life!!

That was his legacy, all gone now.

Re:Only if you tell it to delete them

[Gubbe]

If his backups were that important, maybe he should have backed them up!

Re:Only if you tell it to delete them

[Ecuador]

Also, backups are backups. He can just create new ones.

Exactly, that is what I don't understand. I have to use a Mac at work, but I've never tried Time Machine since I use rsync on everything - even Windows machines. But in any case, if TM "backs up" your data, you end up with your original data + a backup with the point being you can lose one of the two and still have your data. So what happened here? He lost his backup, then what about his original data? How did he lose all his work when only the backup is gone?
Also, he probably messed things up by killing processes etc. Next time use rsync - nothing happens if you kill it while it's working.
And finally, A/V on a Mac? Seriously? Why just throttle down your cpu to half its frequency and just PRETEND you are running A/V software. You will have the same chance getting viruses, plus no problems like the one we are discussing now.

Re:Only if you tell it to delete them

[Sockatume]

He says that it was "irreplacable data". The whole point of a backup is that the data is trivially replacable, because it has been duplicated. I suspect his backup routine is rather like this one.

Re:Only if you tell it to delete them

[gnasher719]

Exactly, that is what I don't understand. I have to use a Mac at work, but I've never tried Time Machine since I use rsync on everything

That's why you don't understand. Time Machine keeps historical data around, so you can have say a laptop with a 250 GB drive, a 2 TB backup drive, and everything that was ever on your laptop drive will be on your backup drive. Like the OP said: 19 months of historical data. Time Machine is basically backup for the current state, plus history.

Re:Only if you tell it to delete them

[Sockatume]

If you need that historical data, it should be kept on some kind of permanent storage, which is backed up. Relying on old backups to keep it around after you deleted it is not a valid approach, any more than filing your old records in the trashcan is.

Re:Only if you tell it to delete them

[Ecuador]

Hmm... So it would be a bit like me using rsync without the --delete option so that data that gets deleted is not erased from the backup and then I go and ERASE data that I NEED from my working copy, since, you know, it is "backed up" ???
Hate to break it to the OP if that is the case, but keeping a single copy of your data cannot be called "backup" in any way. The whole situation sounds idiotic, as a historical backup that can get corrupted in various ways used as your single data store is LESS safe than not having a backup at all. But then again we are talking about a Mac user and kdawson no less... ;)

Re:Only if you tell it to delete them

[0100010001010011]

Not just that, it includes snapshots.

Say you want to go back and look at your resume as it was 4 months ago. Or you're working on a project and you want to see what it looked like before you made a big change.

Re:Only if you tell it to delete them

[Ecuador]

Yes, I get it, I was using Amanda over a decade ago. And it is exactly the reason I said it is more liable to corruption than just having your files somewhere. So it provides more functionality than just a backup copy PROVIDED THAT you don't go deleting your original files - otherwise you have the extra historical functionality but at a great risk.

Re:Only if you tell it to delete them

[Anonymous Coward]

Ever heard of version control? Time Machine is not backup. It lets you view history. The clue is in the name. If someone deleted my git repository I would be pissed off, and now you are saying "you should have kept all your old files in the working copy."

Re:Only if you tell it to delete them

[Ecuador]

Time Machine is not backup.

That is what *I* am saying. The way OP is using it, it is like a repository, he can't claim "his backup is deleted" when he is using a backup tool as a repository.
Oh, because Time Machine apparently is not supposed to be something like svn/git. According to apple:

Time Machine will automatically back up your entire Mac, including system files, applications, accounts, preferences, music, photos, movies, and documents. But what makes Time Machine different from other backup applications is that it not only keeps a spare copy of every file, it remembers how your system looked on a given day — so you can revisit your Mac as it appeared in the past.

Note the "spare copy" - you are not meant to delete files you need from your working directories.

Re:Only if you tell it to delete them

[aristotle-dude]

He says that it was "irreplacable data". The whole point of a backup is that the data is trivially replacable, because it has been duplicated. I suspect his backup routine is rather like this one.

You really don't get it do you? Time machine is not just a backup, it is a lot like a repository such as SVN. It lets you find a previous revision of a file or project and save it out somewhere else to do a comparison or undo changes you decide that you don't want. It is an incremental backup.

Re:Only if you tell it to delete them

[Altus]

That's all well and good as long as you notice it deleted your backups before your hard drive crashes. And it assumes that you don't realize the next day that you need to revert some file to a previous version, or you deleted some file by mistake 4 weeks ago that you really needed.

Backups are there to protect against the unexpected, so while these might be low probability events, they are exactly what your backups are there for.

Re:Only if you tell it to delete them

[Anonymous Coward]

Except that it's designed so that older backups eventually expire, depending on usage. There is a finite limit to what it can store and using it as long term storage is a bad idea.

Re:Only if you tell it to delete them

[aliquis]

His mac was his current backup backup.

Re:Only if you tell it to delete them

[aristotle-dude]

Except that it's designed so that older backups eventually expire, depending on usage. There is a finite limit to what it can store and using it as long term storage is a bad idea.

That is all very interesting but the same thing eventually happens to source repositories since storage space is finite and performance of a repository decreases over time if you don't archive once in a while.

For all intents and purposes, it is unlimited from a consumer perspective since you are not going to want to go back more than a year anyway and it is a good idea to have a time machine backup drive that is considerably larger than your system directories + home folders.

Re:Only if you tell it to delete them

[aliquis]

It's not a one-state backup. It's regular snapshots for all time backwards.

So no, he may not have lost his current state of the data, but he lost the history of his data.

Re:Only if you tell it to delete them

[aliquis]

"I heard you liked backups so I .."?

Sure he could had done it twice, but it probably wasn't _THAT_ important. Just disturbing that it was lost.

He probably don't want to take manual historical data backups, that's why he use time machine in the first place ... He could do it on a raid array but that wouldn't had helped if time machine fucked up or some program decided to delete the data.

To take a backup copy of the time machine disk every now and then would probably be the best.

Re:Only if you tell it to delete them

[aliquis]

He don't need it. But say he code some project and there's not much reason to keep a copy of the state from two weeks ago, but he _COULD_ see it if he wanted to.

It's not up to you to decide what he want to do with his data or why.

He wanted it for some reason (or maybe he didn't, but he still think it was bad that it was lost, if nothing else just because it happened.)

Re:Only if you tell it to delete them

[aliquis]

.. maybe he could have had the time machine backup in an HFS+/HFSX image on a network mounted ZFS disk pool and have that one take snapshots every now and then. Handy for going back when time machine fails ;D

Re:Only if you tell it to delete them

[aliquis]

The whole idea of using it is:
1) You may not know in advance that you're going to do something where you would had wanted to keep the old version or need a backup.
2) It don't waste space as much as taking a whole damn copy of your files every time they are ever changed does.

Stupid.

Not needed

[nacturation]

With a little sophostry installed from Sophos, backups are a thing of the past. You will now never lose a file either due to virus, trojan, or simple human error. Want to revert to how your essay looked 12 hour ago? You no longer need to! Sophos magically takes care of all errors and mistakes for you ahead of time, freeing you up to work effortlessly and error-free on your gorgeous Mac without the constant file churning that Time Machine used.

Loss of data, backups disabled without warning?

[Anonymous Coward]

Sounds like a virus, you should install AV

Sophos

[webmistressrachel]

Compared to Norton, Symantec, and the other system-strangling solutions available for virus detection, Sophos is definitely the leading provider. When I was at college (10 years ago), their software scanned everything coming in and going out, and yet hardly slowed the systems down at all (yes, if you had a local machine Admin account you could end the process and prove this!)

I would be surprised if this turned out to be true.

Re:Sophos

[Anonymous Coward]

Norton is made by Symantec, they are not separate entities. Sophos is a leading provider? Never even heard of them.

Re:Sophos

[scdeimos]

You obviously don't move in corporate circles.

Re:Sophos

[Anonymous Coward]

and yet hardly slowed the systems down at all

I'd have believed you if you didn't say the above bit. Sophos is not the fastest anti-virus out there by any stretch.

Re:Sophos

[webmistressrachel]

If you're a government, educational institution, or a large corporation, you've definitely heard of them.

If you're a troll on /. with no real experience working in IT, then of course you haven't heard of them.

Re:Sophos

[webmistressrachel]

Sophos DID hardly slow the systems down at all, it maxed out 1% CPU time most of the time, and flagged "dodgy" executables as they landed in the network share, before allowing Windows even to load icons from them. I realising I'm biting YET ANOTHER troll, but what else is there that one can do to fight bullshit and misinformation? Anybody else got experience using Sophos in an NT4 Workstation / Server environment "back in the day" wanna back me up?

Re:Sophos

[datapharmer]

Yep. It used to work acceptably well. As in the CPU use was justified. It wouldn't catch everything but it also wouldn't require a dedicated box just to handle email and filesystem scanning. That said, it is hardly sufficient these days and I haven't used it in years. We run active scanning on client machines and lock them down with group policy, then they roll back to disk images on reboot. We block problematic file extensions from email (with the exception of pdfs and documents) and run everything through a dedicated mailserver that scans everything coming in and going out. We also scan all external media when it is mounted (we keep usb etc as mounting read only which also helps to keep things from moving around if we were to get an infection). Periodic scans are run on network drives and periodic checks are done on client machines by booting with external media. Router and NAS are monitored for unusual activity and sysadmins are alerted if it is detected. We don't have virus problems.

RTFA First

[Caraig]

After looking through the article, while the user seems to have erred in taking Sophos and Time Machine both at their word -- I need to re-read the part he was talking about VMs, something there didn't sound right but I'm not sure what -- and been a little too quick with the OK button, it does strike me as odd that Sophos didn't drop some kind of error when it tried to write to the backup file.

How does Sophos do this?

[MarchHare]

He tried to open a quarantined file, once with the 'cat' command
and once with vi, as root, and both times Sophos warned him and
prevented him from proceeding. Now, the code for the 'cat'
command is quite simple, it basically just does a open(2)
of the file and then issues a series of read(2). My question
is: Does Sophos actually intercept the system calls in order
to make sure no application opens an infected file? If so,
wouldn't that introduce a HUGE performance penalty on the
everything happening on the machine, since these system calls
are so crucial?

Re:How does Sophos do this?

[0123456]

If so, wouldn't that introduce a HUGE performance penalty on the everything happening on the machine, since these system calls are so crucial?

Uh, it's anti-virus software: of course it introduces a huge performance penalty when accessing files. Otherwise, how would you know that it was doing anything?

Re:How does Sophos do this?

[bill_mcgonigle]

Yes.

Really, though, on a Mac, it should have a mode that makes it noop unless it's a Microsoft Office app running.

Re:How does Sophos do this?

[scdeimos]

I'd always thought that most AV's hook into file system drivers so that their operation is hidden from the application layer. On Windoze at least they're called "file system filter drivers."

Re:How does Sophos do this?

[goombah99]

Mac extended attributes tell the OS when not to open a file. For example com.apple.quarentine get's tagged onto every file you download from the internet unless it's of a set of known safe file types. If you have os 10.6 try typing ls -loe@ in your downloads folder. When you edit a file the mac file system also tags it as changed so it knows it will need to back it up without having to go checksum compare every file like rsync checksums do. Thus it's perfectly possible that the virus software could intercept every file open.

What I don't like about this is that when I compile code, every time I run it, a waring message gets written to the system log unless I also code sign it before I run it. I can see why this is really good for me and consumers in general, so I put up with it.

Moreover, macs also check to see if any executable has a sandbox before it launches as well.

so there are lots of hooks.

Re:How does Sophos do this?

[Trogre]

Well... yes. That's how every single real-time virus protection suite works. Files are scanned for viruses (using lookup tables and/or heuristics) before being passed back to the application.

That's also why for quite some time my company policy has been at least two CPU cores per computer - one for the virus scanner and the OS/apps can have the rest.

Re:How does Sophos do this?

[flonker]

Yes, and yes. That is why all AV software sucks.

What do you suggest as an alternative? Remember, people have grown to expect real-time protection.

Re:How does Sophos do this?

[vtcodger]

***What do you suggest as an alternative? Remember, people have grown to expect real-time protection.***

Good question. In this case a disable network-virus scan-backup-re-enable network scheme without real-time protection might have worked better, but it is hardly bulletproof. It's a little late to point out that it would have been better to have alternating external backup drives -- more to protect against hardware failure than software issues. And that won't work if you don't know the backup drive has been trashed. You won't notice until both are trashed. And of course nothing protects against zero-day exploits.

I don't have an answer other than unplug the network cable. Or go back to 1980 and try again with a lot less clever and a LOT more secure so that viruses become impossible. Or both.

Re:How does Sophos do this?

[am 2k]

That's also why for quite some time my company policy has been at least two CPU cores per computer - one for the virus scanner and the OS/apps can have the rest.

That doesn't make sense. When the scanner kicks in, the application is blocked on the open() call until the scanner is finished analyzing the file, so your second CPU does nothing, and vice versa.

Re:How does Sophos do this?

[duguk]

If so, wouldn't that introduce a HUGE performance penalty on the everything happening on the machine, since these system calls are so crucial?

Uh, it's anti-virus software: of course it introduces a huge performance penalty when accessing files. Otherwise, how would you know that it was doing anything?

What I've never understood, is why? Why not just check on writing; and reading on removable drives?

Re:How does Sophos do this?

[bill_mcgonigle]

Maybe they run background scans?

I won't pretend to understand core coherency under Windows, but if they have one of those network traffic interceptors, conceivably every other thread in a multi-connection webpage load could get scheduled to a different core and get interleaved between scanning and transferring.

Probably smarter just to benchmark it than reason it out.

Or, if they do the standard corporate thing and keep RAM low and swap like mad, the second core can run the memory manager. ;)

Re:How does Sophos do this?

[Chelloveck]

Mac extended attributes tell the OS when not to open a file. For example com.apple.quarentine get's tagged onto every file you download from the internet unless it's of a set of known safe file types.

Yes, but it's not something that's done by intercepting system calls. The com.apple.quarantine attribute is only respected by apps like Finder which are specifically looking for it. If you just use something like 'cat' in a terminal window you can still view the file without getting the "ZOMG! This is from teh interwebz!" dialog.

As a test I downloaded an app from the big bad internet. Double-click with Finder and I get the dialog. Cancel out, pull up Terminal, and cd into Foo.app/Contents/MacOS. I can cat the executable and I can run it with './Foo'. I try again in Finder and I get the dialog again, so I know the attribute hasn't somehow gotten cleared.

I don't doubt that there's a way for anti-virus programs to hook into the open(2) system call, but that's not how quarantine works. It's just a feature of Finder and friends.

Re:How does Sophos do this?

[sjames]

It's a bit of a problem in Windows since anything could potentially be an executable including an email or a Word document (gee, thanks MS!).

Re:How does Sophos do this?

[goombah99]

Mac extended attributes tell the OS when not to open a file. For example com.apple.quarentine get's tagged onto every file you download from the internet unless it's of a set of known safe file types.

Yes, but it's not something that's done by intercepting system calls. The com.apple.quarantine attribute is only respected by apps like Finder which are specifically looking for it.

No this is not true. While the finder does a pop-up for these the system does check this attribute. You can see this for example when you launch code you compiled yourself, even from the bash command line. Look in the 10.6 OS system.log and behold there is a warning that the code is not signed. No finder involved; the finder is simply more vocal, but it's the system that is checking things.

Re:How does Sophos do this?

[Trogre]

I think you misunderstood. More CPU cores won't make that application, or more correctly that thread, any faster per se. They will, however allow other threads to continue running while the first CPU core is tied up with CPU-intensive virus scanning.

Re:How does Sophos do this?

[bjb]

What I've never understood, is why? Why not just check on writing; and reading on removable drives?

When virus definitions are updated, it is possible that a file that was written in the past is now considered infected. As well, the file could have been written to the disk without the antivirus software's knowledge (could have not been loaded, killed/crashed, etc).

Still, I agree with you that most of the time we have already scanned the darn file..

Re:How does Sophos do this?

[exomondo]

What I've never understood, is why? Why not just check on writing; and reading on removable drives?

2 things:

firstly they may keep track of which files have been scanned with the current up-to-date DB and not re-scan them on read, though it may be slower to do the lookup than to just do the scan.

secondly think about the situation described in TFA, he went to open a scanned, identified and infected file.

Re:How does Sophos do this?

[exomondo]

What I don't like about this is that when I compile code, every time I run it, a waring message gets written to the system log unless I also code sign it before I run it.

You don't like that unsigned code gives you a warning?

SOME GUY LOST SOME FILES

[wampus]

Not sure why, film at 11.

Re:SOME GUY LOST SOME FILES

[gmhowell]

Because that 'some guy' is the infamous kdawson.

Re:SOME GUY LOST SOME FILES

[jonbryce]

Time machine had backed up a virus, so Sophos killed the entire Time machine backup image to get rid of it.

Re:SOME GUY LOST SOME FILES

[david_thornley]

It's the media effect. If we invade another country and accidentally kill a few tens of thousands of civilians, and suffer hundreds of casualties, it won't be presented as effectively as the death of the single journalist who got shot in all of this.

Mess up a few hundred random computer dudes, and nobody may hear of it. Don't even in the slightest mess with a /. editor, or lots of people will know.

Re:SOME GUY LOST SOME FILES

[Em Adespoton]

"Time machine had backed up a virus, so Sophos killed the entire Time machine backup image to get rid of it."

Not quite what happened, according to the article.

Time machine had backed up a virus, so Sophos blocked the user from meddling with it and stated it could not automatically remove the virus. The user then attempted to work around both Sophos and Time machine and discovered that not only did he remove the virus, he corrupted his TM plist, which meant that it lost the record of what files belonged in what snapshot. As a result, during the next backup, the plist was created from scratch, Time Machine wiped out all files not in the current backup (as they were dead weight, not belonging to any snapshot), and voila... empty TM.

Assuming this is true....

[8127972]

... Then this is a serious hit to Sophos as they have a very good reputation. Having said that, AFAIK this is their first Mac app. So perhaps it needed more QA before release. Until more reports of this phenomenon appear, I'd reserve judgment. However it might be wise for Sophos to get out front of this issue before the spin gets out of control.

Re:Assuming this is true....

[osssmkatz]

It isn't their first Mac app. They've been selling it to businesses before now, but businesses don't generally use Time machine, and would never execute a deletion command using an antivirus on a backup archive while it was running. Not sure whether this is an OS bug, or a sophos bug, or whether if he had allowed the command to finish, it would have worked fine. (Maybe it was just taking a long time.) --Sam

Re:Assuming this is true....

[baddaybeav]

we've used the business side of it for over a year, major performance headaches... as to the time machine part, if my memory serves, time machine creates one large file (like tar, but a lot more advanced) it saw the "virus" in the one large file, didn't differentiate that and deleted what it saw as the "file containing the bad stuff" now that he's written data to the drive he's lost any good chance at recovery... I guess we'll need a time machine time machine soon.

Re:Assuming this is true....

[Yjerkle]

No, each time machine backup is a folder that mirrors the root of your hard drive. Each file is separate on the time machine drive. Space is shared for unchanged files and folders between backups using hard links.

Re:Assuming this is true....

[mug funky]

TM had the privileges to stop Sophos fucking this guy's shit up. Sophos should probably have been aware of the existence of Time Machine and perhaps had a specific behavior or at least prompt for it (as TM comes with the OS IIRC - i'm not a mac guy and never use TM when i'm on one).

blame sophos?

blame apple?

let the shitstorm begin.

Re:Assuming this is true....

[zippthorne]

No, it's separate files. You can browse it using finder or terminal.

Unless you're backing up a filevault protected home directory. Then it handles it in just about the stupidest way possible: it saves the whole honking encrypted image as one big file.* And despite the fact that it doesn't decrypt the image, it still only works if you're logged in and the image is open.

*If you're set up as sparse images, then you do a little better. But still, no incremental backups for you. If a file changes, you have to copy the *whole* thing, because good encryption won't make it obvious which bits of the file are different. Also, I'm not sure it can tell which files are, say, disk cache for the browser....

Re:Assuming this is true....

[kdawson]

FYI, I'm not using filevault, just individual files to be backed up... but TM uses sparsebundles in ways I don't begin to understand. One respondent via Twitter suggested that Sophos may have simply been in the process of deleting the entire sparsebundle -- i.e. the entire lot of backups -- when I killed its process. No idea if this is correct. I hope Sophos eventually provides some insight.

Re:Assuming this is true....

[larry bagina]

yes, one large file which is actually a sparse disk image.

Re:Assuming this is true....

[Anonymous Coward]

They have a whole disk encryption product for macs. They have been hawking it for a while, but finally released it in August. Its crap. Does not lend itself to a secure system of deployment. Absolute enterprise fail. But do bear in mind, their AV product is not their only mac product.

Re:Assuming this is true....

[Rosyna]

yes, one large file which is actually a sparse disk image.

it's a sparse disk image bundle thingy. Which uses a bunch of 8MB files, not one file. from the hdiutil man page:

  By default, UDSP images grow one megabyte at a time.
                                                    Introduced in 10.5, UDSB images use 8 MB band files
                                                    which grow as they are written to.. -imagekey
                                                    sparse-band-size=size can be used to specify the
                                                    number of 512-byte sectors that will be added each
                                                    time the image grows. Valid values for SPARSEBUNDLE
                                                    range from 2048 to 262144 sectors (1 MB to 128 MB).

                                                    The maximum size of a SPARSE image is 128 petabytes;
                                                    the maximum for SPARSEBUNDLE is just under 8
                                                    exabytes (2^63 - 512 bytes minus 1 byte). The
                                                    amount of data that can be stored in either type of
                                                    sparse image is additionally bounded by the filesys-
                                                    tem in the image and by any partition map. compact
                                                    can reclaim unused bands in sparse images backing
                                                    HFS+ filesystems. resize will only change the vir-
                                                    tual size of a sparse image. See also USING PERSIS-
                                                    TENT SPARSE IMAGES below.

Re:Assuming this is true....

[Anonymous Coward]

Have you double checked to make sure that you can't still see the backup history using the native Time Machine browser app? In my experience with TM failure, one symptom included a sudden change in the amount of free/used space reported - not unlike your experience - see below for more details.

One of the reasons I switched to Mac was because I liked the Time Machine concept. I use a Seagate USB drive plugged into a Macbook Pro. A few weeks in, Time Machine reports that it is unable to complete a backup. Multiple days later, I was unable to a) fix the TM backups, b) fix the TM file system, c) backup my backup data - despite the fact that TM would still let me browse the data just fine. Somewhere in the sparsebundle there was a bad file, and this kept TM from completing further backups, or from letting me save the still browsable data in a way that would let me re-import it later. Apple support told me to format the drive and live with losing my backup history.

End result: I haven't run a backup in 196 days, according to TM.

Conclusion: Time Machine sucks. Apple support knows very little about sparsebundles.

Re:Assuming this is true....

[Rosyna]

One thing. directly connected hard drives do not use sparse bundles if FileVault is not on,.

Re:Assuming this is true....

[uglyduckling]

Really, really - no. Time Machine backups are sparse bundles, which looks like a file unless you mount it as a volume. Just like those 'dmg' files you download to install an application. It's possible that you're using a really old version or have some options set to use a folder, but sparse bundles are the default on a new Snow Leopard backup schedule.

Re:Assuming this is true....

[uglyduckling]

Actually, we might be both right - I've seen another post that suggests that TM uses folders on a directly connected drive, although I'm pretty sure that before I moved to a DIY Time Capsule (USB external connected to Airport Extreme) I still had sparse bundles. YMMV.

Re:Assuming this is true....

[uglyduckling]

Blame Sophos. Sparse bundles are a key feature of the Apple filing system and really, really useful. Sophos should know all about them. This would be akin to a Linux AV that could look inside .tar.gz files but would nuke the whole archive if one file inside was questionable, without making that absoluely clear to the user.

Re:Assuming this is true....

[jonbryce]

That's how it presents itself to the user, but it does this magic inside a Sparsebundle image file.

Re:Assuming this is true....

[LoganDzwon]

You guys are explaining the behavior on 10.5, 10.6 is more intelligent about it. FileVault home directories will get backup as sparesbundles instead of sparesdisks . The different is the former uses mutiple 8 meg files. Up date only cache, only the files that contain that data will be updated.

Re:Assuming this is true....

[ilsaloving]

If your backup drive is a locally accessible drive, Time Machine stores your HD data to the backup drive as files, folders, and (I think) lots of hardlinks. That's how time machine is designed to work. You don't have direct file system access to a volume when you access it over a network, so Time Machine fakes it by creating a sparse bundle on the destination volume, mounts THAT as a 'local' hard drive, and chugs along.

Re:Assuming this is true....

[Anonymous Coward]

As clear as say, the average kdawson-selected slashdot summary?

Re:Assuming this is true....

[bill_mcgonigle]

No, it's separate files. You can browse it using finder or terminal.

Yeah, that's what I see here on 10.5 as well - at one point I had my rsnapshot backing up a Mac's Time Machine 'latest' tree.

From the other comments here it sounds like 10.6 might have gone with sparse bundles for all of its backups? Maybe to enable encryption?

I dunno, Apple has abandoned my wife's Apple hardware. Her Mini will get turned into a mythfronend when Lion is shipped. Too bad the iLife analogs on Linux are terrible (quite featureful, but the UI's stink).

Re:Assuming this is true....

[tlhIngan]

Unless you're backing up a filevault protected home directory. Then it handles it in just about the stupidest way possible: it saves the whole honking encrypted image as one big file.* And despite the fact that it doesn't decrypt the image, it still only works if you're logged in and the image is open.

*If you're set up as sparse images, then you do a little better. But still, no incremental backups for you. If a file changes, you have to copy the *whole* thing, because good encryption won't make it obvious which bits of the file are different. Also, I'm not sure it can tell which files are, say, disk cache for the browser....

How do you expect it to work? DO you expect it to copy files from your encrypted home directory to an unencrypted storage area? FileVault uses disk images to handle encrypted home directories, so the files hit the disk encrypted. Time Machine treats the FileVault image as a single file (or a collection of files with sparsebundles). It can't go and individually take the files out of the FileVault and encrypt them (in case the filenames reveal information), just like it shouldn't copy the files to the backup store unencrypted.

Being logged in, however, allows Time Machine to do proper indexing for TM-aware apps so you can go back and recover individual files in your FileVault (e.g., should you hose a iSync contacts merge, you can go backwards and recover the record prior to the merge). Otherwise it'll be more of an all or nothing restore instead of being able to support partial recovery.

Re:Assuming this is true....

[kdawson]

My backup disk is a Time Capsule, whose internal disk is connected only wirelessly. Probably doesn't count as direct-connected. Does it therefore use sparse bundles?

Re:Assuming this is true....

[zippthorne]

Here is what I would expect:

If it does it without decrypting the contents, then the user shouldn't need to actually be logged in in order to get the benefit of the backup. It should back them up if it's changed since the last backup regardless of whether the user is logged at the precise moment backup begins.

OR,

It could require the user to be logged in, because it needs access to the unencrypted files. Obviously, it would encrypt the backup itself. The benefit here would that a logged-in filevault user would have the full functionality of time-machine that an unencrypted user has.

But it doesn't do either of those. It requires the user to be logged in, but still does a data-agnostic backup. a filevault user does NOT get a nicely indexed time machine. They can still explore the backup through the finder, but the nifty pick a file, then get any previous version of that file feature is missing.

As a result of which I've made the decision to generally work with an unencrypted account and have a separate, encrypted account for financials, to keep the amount of data that would need to be manually searched to a minimum.

Re:Assuming this is true....

[zippthorne]

In 10.6, browser cache does not get backed up for a non-filevault user....

My Time Capsule instantaneously loses...

[Slutticus]

...data all the time. I thought this was a feature. Even my non-techie wife knows what a "corrupt sparsebundle" is....

Re:My Time Capsule instantaneously loses...

[tibit]

Something must be broken then in your setup somewhere, because I use a Time Capsule, recently upgraded from 500GB to WD Green 2TB, and never had a single data loss/corruption issue. I'm using it with a MBP and an iMac, and have used it with OS X 10.5, and now 10.6. Not a single problem, apart from running out of room on the 500GB drive and having to upgrade.

Re:My Time Capsule instantaneously loses...

[gmhowell]

How did you get data from the old drive to the new one? I have the 1 TB model, but that won't last forever.

Re:My Time Capsule instantaneously loses...

[mug funky]

Trash your preferences!

flash the P-ROM!

buy more RAM!

i can't help you! ...well, that's the usual order of responses i get from mac techies.

Re:My Time Capsule instantaneously loses...

[tibit]

I didn't. Simply reinitialized the time machines on the new drive.

Transferring data would have been trivial. All you need is SATA-USB or SATA-Firewire adapter. Procedure I'd use:

1. Format the new drive with same format as the one in time capsule (remove it, check whether it's HFS+ journaled or not, put it back).

2. Hook up time capsule via gigabit ethernet, hook up new drive via USB/Firewire.

3. Disable time machine.

4. Mount both drives.

5. Copy all files over using a *recent* rsync, with xattrs/acls and whatnot enabled so that metadata stays intact.

6. Unmount, power off, move new drive into the time capsule.

I'd think that would do it.

Re:My Time Capsule instantaneously loses...

[Anonymous Coward]

Don't forget to repair permissions (because they just spontaneously go bad, right?). Oh, and use a tool to run the daily/weekly/monthly cron jobs if you leave your mac off/sleeping at night. Because you know, those unix cron jobs are critical and your system will bog down if they're not run! /sarcasm

Of course all of the above is complete BS.

Re:My Time Capsule instantaneously loses...

[Anonymous Coward]

http://support.apple.com/kb/ht1427

Re:My Time Capsule instantaneously loses...

[gmhowell]

I think step five might be the secret sauce.

Exclude Backups from A/V Scanning

[Anonymous Coward]

Obviously.

Duh!

combo of bad apple, bad sophos, and stupid user.

[GNUALMAFUERTE]

The closest I've ever come to AV software has been running clamav on a Slackware machine acting as a mail server, but I do understand how they work. It doesn't look like it was the AV's fault.

Well, it was in a way, AV software is a braindead solution to a problem that shouldn't exist. Use only properly signed software from trusted sources in a secure platform, that's a real solution.

Anyway, this guy killed both Sophos and the Time Machine process in the middle of a backup, while they were both trying to access his backup disk.

Backup disks should never be treated in that way, and you should actually never sync against your only copy of a backup. That is plain stupidity. Backups should be done in two stages:

Active Data -> Backup server -> Offline backup.

Connecting your only copy of your backup to where your precious data is means you have both copies of your information connected and mounted in a single computer. That's beyond stupid.

Anyway, it seems like Apple's fault. I've used Rsync for ages. You can kill an rsync process, and recover from where you started, but I can see how cheaper backup alternatives might screw everything up if you killed them in the middle of an operation.

I don't know how data is stored on TM's timecapsules, but it doesn't seem to be transactional or secure, based on the way this guy lost so much data in a split second.

I guess my policy of staying away of anything proprietary, and using server-class, proven backup solutions in the proper way (data -> backup server -> offline storage), using fully transactional solutions, and always backing up to separate instances on the second stage (instead of replacing) is the only solution, as I've never lost a byte, while I keep hearing terrible stories of data loss, empty backups and massive filesystem corruption (yeah, mostly from windows/mac users).

"Time Machine"

[Anonymous Coward]

Can somebody explain to me what the Hell "Time Machine" is in this context?

Re:"Time Machine"

[Jeremy Erwin]

It's a daemon that copies files that have changed in the last hour to an second hard drive. It's useful for casual development work, and the GUI client is intuitive. I've also used it to recover files after they've been over-wriiten by buggy programs. It's also come in handy for certain games-- if the autosaved game file from today is less interesting than the autosaved game file from yesterday, or two weeks ago, I can recover the older files.

Yes, you can get the same effect by running VMS, or Git, or adhering to a regular backup schedule, but this makes it easy. All you have to do is make sure that your backup hard drive is connected, and turned on.

Re:"Time Machine"

[Anonymous Coward]

With the important distinction that time machine backups integrate with things like iphoto, to allow you to browse old versions of these application's data. And also that you can easily re-install your OS from a time machine backup (plus the install DVD of course), which would probably come in very handy in the case of an HD crash or having your entire machine stolen.

Cost/benefit

[flyingfsck]

AV on a UNIX machine is a bad idea in more ways than one. By definition, AV programs go about deleting files. Obviously this can corrupt a system. So the risk of incurring virus damage must significantly outweigh the risk of incurring antivirus damage. On any UNIX system, it it is still best not to have AV.

Re:Cost/benefit

[Anonymous Coward]

>>By definition, AV programs go about deleting files.

ugh what? sorry just plain wrong.
By definition, AV programs protect against malicious code being executed or read. Anything it does after that is just a setting (or a bad default..)

Re:Cost/benefit

[del_diablo]

Well, zerodays attacks can not be detected.
And the only thing the AV will do is to scan for Windoze viruses, and Mac before it got the X in OS X.
So its more or less completely useless, except for helping the poor mass of sheeps that should never be allowed to use a computer because of their stupidity.

Re:combo of bad apple, bad sophos, and stupid user

[Anonymous Coward]

Well, it was in a way, AV software is a braindead solution to a problem that shouldn't exist. Use only properly signed software from trusted sources in a secure platform, that's a real solution.

So.. You are never allowed to download something and try it out, unless it's from a trusted source. Exactly how are normal people supposed to get their programs into said trusted sources? Should we perhaps have an "app store" for all software, putting a few large entities in control of what is acceptable or not?

I also enjoy your naive belief that virus can only spread by downloading and running infected code. This is not 1989. Comprimosed web pages, exploitng holes in browsers and browser add-ons, infected non-executable files exploitng holes in applications, and autonomous worms exploiting holes in networked applications and operating systems, are by far the biggest infection vector, for all platforms.

You probably consider running OpenBSD with the minimum number of activated services, pf configured for maximum security, and an external firewall between your system and the internet a good and acceptable solution for everyone, but most people would disagree.

Your solution is not a solution, any more than building customized computers that can only run a specific set of pre-installed and custom made software would be a solution.

It is possible to go without AV software and still have a very low risk of infection, even on Windows, if you are careful. But the problem it is there to solve is a real one.

No problems here with Sophos and Time Machine

[Anonymous Coward]

I've installed it on my Mac, and run it alongside Time Machine without any difficulties. It even found some occurrences of the EICAR test file and handled them appropriately.

The initial Time Machine backup was admittedly slower than normal, but I haven't noticed any impact since.

if there are no viruses on OSX, why use?

[jsepeta]

if there are no viruses on OSX, why use an antivirus program? don't we have to wait for OSX to be compromised first?

Re:if there are no viruses on OSX, why use?

[Anonymous Coward]

If you have a multi-OS network you don't wanna be carrier for some silly virus that has no idea what it's doing on your machine. Personally, I use ClamAV.

I am actually not surprised

[fluch]

The time machine stores the back up files on an external hard drive in a specific way such that can perform the backup task and the possible restore task effectively. In order to this to work noone should modify or delete any data stored in the backup location. This will most likely corrupt the backup.

The author of the article told Sophos AV to delete files from within the time machnien backup location ... well, of course one can expect that it messes things up.

10 years is a long time

[Anonymous Coward]

Software that was at the bleeding edge a decade ago might not be that now. Hell, 2 years is pretty long time in AV business and 10 years is long enough for larger changes in the enviroment itself (the amount of data, the type of data, the bottlenecks, the type of malware, tec. all can change in that time).

I don't know much about Sophos: They might or might not be a leading provider. But 10 years old examples aren't really all that relevant when it comes to any software, especially one that includes such a cat-and-mouse play as AV...

Re:10 years is a long time

[webmistressrachel]

Yes, but I was actually illustrating that Sophos has a very long history of writing quality bug-free software for mission-critical environments, like Governments, Educational Institutions, and large corporations.

The chances of their software not functioning as intended and screwing up systems or backups are far smaller than their lesser counterparts, Symantec et al, and the whole article smells of Troll Fat.

Re:10 years is a long time

[Anonymous Coward]

Bug free? Sophos? I hardly think so. We switched from Symantec to Sophos about 6 months ago (80,000 machines). It was switching one set of problems for another set. For example Sophos can't easily be put in an image without "re-installing" part of it after the image comes up. If you don't do that just right, it will lock up the machines built from the image for 10 minutes at a time randomly throughout the day. It frequently gets false positives and says it "quarantined" something when it actually took no action other than to log something. It IS better than Symantec was. But it is hardly bug or problem free.

What the?

[ledow]

First, we get an article that consists of one idiot posting on a blog who openly admits that he clicked delete himself on the popup and thus caused the problem in the first place. If it had been a critical set of Windows backups, the same thing would have happened, or even the System Restore folders.

Then, I realise it's an article by kdawson who I have deliberately blocked because all their submissions have glaring errors and omissions or are nothing more than rumour, but they've handed it off to another person to post on the site. I BLOCKED kdawson for a reason. Don't start slipping their posts around that block which you enable me to use yourselves.

Re:combo of bad apple, bad sophos, and stupid user

[vtcodger]

***Well, it was in a way, AV software is a braindead solution to a problem that shouldn't exist. Use only properly signed software from trusted sources in a secure platform, that's a real solution.***

Uh, Yeah. ... Of Course.

Now that you have solved that problem for us, what are you going to tackle next? World Peace? Finding economists who understand economics? Keeping sociopaths out of political office?

You do understand that the trusted sources solution is utterly impractical once you allow access outside of a closed, rigidly controlled, local network, right?

and the other 95%

[Anarchduke]

of computer users don't care about macs

Lost what, exactly?

[lga]

kdawson complains about having lost nineteen months of 'mac life' but what was there to lose? These were backups. They weren't the only location of the files in question, and if there were files stored only in Time Machine, are you also one of those people that keep important files in the trash can?

I'm not saying there isn't a problem if Sophos deleted the backups, just that it isn't that big a deal.

Re:Lost what, exactly?

[Anonymous Coward]

Time machine is historical backups. So this means yes, like most time machine users, he has files ONLY on his time machine.

Time machine is for when you go "Oh crap I deleted that document last week but I still need it. Let me just go back a week + 1 day and restore it!"

At the moment he may not be missing anything but when he needs a file from a week, month, year ago then he'll realize what he lost.

That said using time machine as your only backup solution is a bad idea and he should have also been doing daily/weekly standard backups (with a program like SuperDuper!)

Re:combo of bad apple, bad sophos, and stupid user

[Anarchduke]

I don't run active antivirus at all, the trick is never to touch the internet explorer browser. Another tip is don't download a bunch of pirated program and run them without scanning them first. I suggest malwarebytes.

I also keep a copy of combofix on a usb drive just in case.

Timothy

[metrix007]

Please never refer to yourself as an editor. Ever.

Not such a bad thing

[Gothmolly]

The guys sounds like a complete douche and fanboi - drooling on about it being Unix, and having root, and having the 'cat' command. You bent over for Steve Jobs buddy, and not you're finding Macs are just computers too. Sorry for the loss of your innocence.

Backup

[Lord Lode]

IMHO a backup of something important should be done with the simplest method possible. Put it on a medium (optical, HD, ...) and put the medium in a cupboard to never touch anymore. Why trust a program of which you don't know exactly what it does and that can be influenced by other programs as turns out now?

Not their first Mac app

[zerofoo]

It's not their first Mac app - we've been running Sophos AV (corporate, non-free) for over 3 years. It supports Windows, Mac OS, and Linux. -ted

Re:combo of bad apple, bad sophos, and stupid user

[GNUALMAFUERTE]

Not true. I use Free Software. I was a Slackware user for ages (version 3 through 12, then I switched to Ubuntu). I trust the community. I've never gotten malware into my machine. Security bugs? Sure. They were all promptly fixed.

So, don't say that something that has been a reality for 20 years isn't possible, you sound stupid.

Re:combo of bad apple, bad sophos, and stupid user

[GNUALMAFUERTE]

Come on dude.

Use a modern, secure operating system. Use only free software that has been reviewed by the community. Peer-reviewing works, you know?

I only use Free Software. We review everything that goes in those repositories. It's simple, and it works.

Don't use privative software, don't download from untrustworthy sources. Easy.

It's key to read the instructions

[Anonymous Coward]

If you're using Time Machine and you think it'll keep files you've deleted from your original drive around forever, you're mistaken. Time Machine focuses on staying current; if you run out of space on your Time Machine volume, it starts deleting old backups to make room for the new ones. It assumes that since you deleted it, you don't want it anymore. It'll keep it around for a while as a side effect of how it works and as a convenience, but it's not the priority.

It also defeats the whole purpose of backing up: redundancy.

* If something isn't in two or more places, it's not backed up.
* If something is irreplaceable and it's not backed up, you're an idiot.
* If you're an idiot and you lose data, too bad so sad.

Re:It's key to read the instructions

[joh]

Time Machine is too easy to use. Many users even use it for archiving deleted files or older versions of files. This is madness. A Backup is not an archive. As soon as you start to rely on parts of your backup as a source of data that is not elsewhere anymore you deserve everything that may happen to you. But you'll never get this into the brains of users. Give them a backup system that is easy to use and they will use it for letting it archive stuff.

Used just as a plain silly backup system TM is great. Setup is as easy as plugging in some cheap external drive and clicking "OK" in the TM dialog popping up then. Best OS feature ever.

Re:combo of bad apple, bad sophos, and stupid user

[idontgno]

Also, don't ever accidentally subject yourself to zero-day exploits in your browser, which means never browse any valid website compromised by malware pushers without the knowledge or consent of the website owner.

In other words, connect your computer only to a fantasy Internet powered by the carbon-offsetting power of unicorn farts and good wishes.

Yes, the world is out to get you. Not you personally, of course; you're not that interesting. Just you as part of the entire gamut of possible malware victims. The same way that a cluster bomb doesn't care if it kills you, but insisting you're cluster-bomb-proof is still naive and silly.

Let me get it right...

[drolli]

The virus scanner asked him whether to delete the files, he clicked "yes" and thats it? So what would should the program have done?

Re:combo of bad apple, bad sophos, and stupid user

[GNUALMAFUERTE]

My browser runs as a non-privileged user on a secure Unix system. The process itself doesn't have write permission on any executable file, not even itself.

That user is != to my actual user, so it won't even get to my docs or other information. It'll only affect my browser, which can write nowhere but it's own home directory. If something like that happened, restarting my browser and killing any process it might have spawn would be enough.

Exclude

[Anonymous Coward]

There are options to exclude files/directories from the real-time and full scan. The smart move would be to exclude Time Machine backups from the virus scanning. This is something that should have been included as a option in the virus scanner, just like most Windows virus scanners usually exclude the Windows System Restore (shadowcopy) directories.

Not everything is a file, everything is an exe!

[Anonymous Coward]

Gates et al just got confused about the Unix approach, where everything is a file, and thought it would be helpful to go even further and make everything an executable. "Helping", just like Clippy used to "help".

Cheers,

Re:combo of bad apple, bad sophos, and stupid user

[Em Adespoton]

My browser runs as a non-privileged user on a secure Unix system. The process itself doesn't have write permission on any executable file, not even itself.

That user is != to my actual user, so it won't even get to my docs or other information. It'll only affect my browser, which can write nowhere but it's own home directory. If something like that happened, restarting my browser and killing any process it might have spawn would be enough.

I presume you also do this with your torrent client, IM client, email client, etc? As well as having Adobe Reader under its own account?

I tried doing this for a while... having a separate user for each process that accessed the internet, and for each one that was a major exploit target. However, it became too much of a pain, as there was no process integration, and tossing stuff into the shared bin to transfer files between parts of the filesystem proved to be too annoying -- so I went back to a single userland and an AV solution, which has been much less annoying in the long run.

Re:combo of bad apple, bad sophos, and stupid user

[GNUALMAFUERTE]

Dude, what are you running, windows?

I use GNU/Linux. I Don't need an AV solution. Process intercommunication is solved by standard means, namely Dbus. Only my browser runs in a different account.

I don't use Adobe Reader, I use Evince. PDF is not an exploit vector on my platform.

Your problem is crappy software, get rid of it.

Discount Nike Shox at http://www.shoes-vip.com

[Anonymous Coward]

Nice written,Look forward your updates,I also like this,thank you,Cheap Nike Shoes , Nike Shox On Sale at shoes-vip.com,Discount and Free Shipping,Save43%-60%

Comment from G. Cluley of Sophos

[kdawson]

I've added a comment from Sophos's Graham Cluley to the end of the blog post. He/they have been quite responsive, especially given that the free A-V product comes without official support. Apparently I am the only one ever to have reported such a problem with Time Machine.