Slashdot Mirror
Linux Equivalents for Novell's "Filer"?
Josh Berkus asks: "One of my clients would love to convert their entire office to Linux. But one thing keeps them tied to a NetWare file server: a little utility called 'Filer' Filer allows the sysadmins to retrieve deleted and overwritten files, up to a week after the event. With 70 secretaries using that server, that ability is crucial. I've looked around the Internet, but cannot find anything quite equivalent in the Linux world, except maybe hourly backups and that's a pain. What the client really needs is a utility/mod for the filesystem or network layer that archives files instead of over-writing or deleting them. I had one kernel hacker offer to write me something like that, but my client does not want to be a test case. Note, that we have nothing against Netware, it's just that this client has historically not been able to get good Novell support. Anybody know anything that's already in production like this? Is Reiser working on this for ReiserFS?" This sounds deceptively easy. If this was a personal machine, this would be easy, but the showstopper is that this has to work as a share meaning that this "trashcan" like functionality needs to be implemented at the filesystem level. While I can understand the submittors desire to not be a test case, if there doesn't exist a ready-made solution to this particular problem, how difficult would it be to add this on to ext2/ext3, Reiserfs or some other suitable open source filesystem and test it for reliability?
Perhaps you could look at cvsfs? Don't know if it actually works, but meh..
http://sourceforge.net/projects/cvsfs/
stuff
Here's a project that supposedly does that. They use a daemon that physically deletes the files after a while. It's not as cute as in Novell, where erased files were physically overwritten only when the system really needed the space.
Slashdot community, please notice: I am looking for a girlfriend.
Nave H. Weiss
$ su /bin/rm /trash /bin/rm /trash /bin/rm
# rm
# mkdir
# cat >
mv $1 $2
^D
# chmod a+x
-- The morphemes of your disquisition are ascertainable, but they have eschewed an ambit of transpicuous exposition.
I know i saw a library a while ago on freshmeat that supplies a new unlink() function. This version will move the file into a trashcan area rather than just delete it. it could be worth investigating. after a quick search, it's apparently called libtrash.
I don't really know the software, so i can't vouch for it, but it seems to be taking a sensible approach. of course, the whole idea breaks if you use the kernel nfsd, so be careful with that.
#define F(x) int main(){printf(#x,10,#x);}
F(#define F(x) int main(){printf(#x,10,#x);}%cF(%s))
that scans for new files appearing and hardlinks them to the /trash (or whatever) directory and removes stuff from there that's older than a week and has a reference count of 1
kthx
This would also have the advantage that is someone deleted even part of a file the could even get that back later.
Make nightly (incremental) backups on a 14 day rotation. Putting them on a large file storage system.
Than create some perl scripts to retrieve the files when someone wants to.
$>getback letter_to_customer*
(0) letter_to_customer_20020927
(1) letter_to_customer_20020928
(2) letter_to_customer_20020930
Which one (enter for all)?
Of course a GUI would be more office friendly.
All the parts would be off the shelf so you can say it is not a test case. And you would have a backup system that had day to day value.
I'm guessing this may be out of your budget range, but you should know about Network Appliance filers' snapshotting capability. The filers create online "backups" by saving the state information about a filesystem. You could have snapshots of the state of the filesystem every day for a week and several times a day. I think they support something like a dozen snapshots at a time.
.snapshot directory, choose the latest image that has your file intact, and copy it back to the live filesystem. It's perfect for those belly churning "Ooops" situations.
All this does take up space on the filer, but only the changed blocks have to be saved. As long as your churn is pretty low it's not that much extra space.
If you lose or munge a file, all you have to do is cd to a special
Snapshots are also used for backups, so that you are always backing up a totally static image of the filesystem. No files changing midway through.
Yeah, filers are spendy, but when it comes to fast, reliable, easy to administer file service I really believe they are the best.
Other than that, I think Veritas added a similar capability to VxFS, but I don't recall it being quite as elegant.
Sarah
The key post was this one which pointed me at using:-
Very scary. Had to boot into a rescue system off CD first, mount HDD RO, then get enough tools to be able to backup the HDD to a remote box. Unmount disk and then run reiserfsck. Pray for a bit. Got all the db files back (and because they where in a directory that was deleted, they had all the correct names as well (once I'd found the directory)).
There was some minor file-system corruption for files that had been written frequently, but this was taken care of by restoring the previous backup and checking everything else from the RPM db. So, all-in-all not an experience that you would want to do on a daily basis, but definately worth it to restore 2 months of lost work (why oh why don't people use backups?).
As far as kernel hooks for undeleting data, the mailing list link above contains several discussions about this, but the general feeling seems to be that it is possible but Linus doesn't want it (see here.
The only Good System is a Sound System
Filer is in fact a DOS based frontend utility for managing files on Novell Netware servers. Being a DOS utility, it is obvious that filer is as old as the hills but is still very useful. Filer's functionality has been included in the GUI tools that Novell has developed over the years and is included in the outstanding NWAdmin utility and the less appreciated Console 1 tool. Today, the utility Salvage is even included in the Netware Client and is accessed from a context sensitive menu on the client machine's desktop.
Salvage, the feature's proper name, is a tremendously powerful feature of Netware. Basically, when files are deleted from Netware volumes, they are not truely deleted. They are unlinked or renamed and become invisible to the clients but, the deleted files remain on the Netware volumes. When the Salvage utility is invoked, these files can be displayed, selected and instantly restored.
Deleted files remain on the volume until the volume runs "out of space", at which time the oldest deleted files are automatically purged as necessary to allow space for new files. It is important to note that the space consumed by deleted files is not reflected in any of the client utilities and does not count in disk quotas. This means that from the clients perspective the volume will show that there are 30 Gigs free despite of the fact that the volume is in fact full, due to deleted files.
Another interesting note is that deleted files retain their rights attributes. Files invisible to a certain user cannot be seen or restored by that user if they are deleted. Only users that had the appropriate rights before the file was deleted can manipulate these deleted files.
The deleted files can also be purged manually from individual directories or the entire volume. Purging can also be configured to purge the deleted files immediately which is the recommended configuration of temp directories.
This beloved feature of Netware has always been admired. The user community has always requested it in other OSes but, as yet, the only thing to even come close is an NT/2000 add-on called Network Undelete, from the folks at Executive Software, the same people that brought us Diskeeper. Unfortunately, it's still not quite the same.
Several posts have stated that this should be a simple thing to implement. I cannot speak to the ease or difficulty of implementing this feature. However, one does have to wonder how easy it would really be. Considering that Salavge is such an old feature on Netware, that there have been so many requests for it in other OSes and yet Netware is still the only OS to offer it, one must conclude that it is not really so easy to implement.
If you have found a developer that is willing to try to implement such a feature, I strongly encourage you to get them going, regarless of whether you want to be a test case or not. The community would love and appreciate this feature in Linux and any other OS.
oh go buy yourself a NetApp and be happy
they are the reason VA Linux made a mistake
(check out the VA board....)
basically you can set snapshots for hourly minutes or whatever but you have to be careful about space
nice machines
regards
john jones
Another example of technology that works not being left alone due to the 'lets change everything and create new problems that will need more technology to solve'.
It's not freaking fashion. If a Commodore 64 is doing a job somewhere, leave it alone. It's going to work for another 20 years.
Since I learned Unix on the big shared machine, I always thought it was built into the OS from the beginning. Was almost dissapointed when I installed Linux and found out it was something they added.
I don't remember all the particulars, but they had a "Charon" daemon that handled the deleted files. You had a pretty good chance up to a couple of days of recovering files, then they were permanently gone.
Someone is working on this for FreeBSD, but I haven't seen a Linux equivalent.
"Tax preparation software eliminates errors your[SIC] may make...." From IRS home page.
This needs to be very user oriented and transparent so that it doesn't fail when you need it most.
set up a daemon looking at new files (use fam, of course) and create a hard link on a mirror directory as soon as they're created. When the file is unlink()ed, it won't be deleted since there's still another link to it. Add a cron job that scans the mirror directory and moves any file with a link count of 1 to the 'just deleted' folder. the rest is routine, hourly/dayly/weekly rotations, old ones really deleted, etc.
-Kz-
They are called snapshots. Many NAS boxes do it and they are VERY helpful. They keep me from doing restores from tape when someone deletes a file. I just grab it from last night's snapshot. I do a nightly snapshot as well as a weekly so that I keep up to a month's worth for each volume (free-space willing...). That way I can go back to a file 3 weeks old without calling tapes back from off-site storage.
Not to be a complete twit - but I woulden't replace the Novell server. If it's stable and does what you want then I'd just let it be. Put your enenrgies into adding value rather than replacing somthing that works with something else. Perhaps a Linux based firewall is needed? Or Linux based VPN so people can get into the office network from home?
Office types love VPN from home - though I'd sugjest OpenBSD for the job over Linux, both would work fine and make people *very* happy if done correctly.
Cheers and good luck.
Moneyed corporations, non-working 'poor' and criminal prisoners are turning productive citizens into tax-slaves.
Uhmm.. I think you replied to the wrong message.
Slashdot community, please notice: I am looking for a girlfriend.
Nave H. Weiss
I don't have the details yet, but: I am evaluating NAS solutions. The two Linux systems that have journaled filesystems also use XFS, and also have snapshot capabilities. I don't know if it's something they've added to their systems, or if it's inherent in XFS. Can somebody who's already directly playing with XFS fill me in on this?
Wow, so you company had a shitty sys admin that didn't do backups like he was supposed to, and others got fired for it? Maybe they should have fired the dumbshit sys admin that didn't do backups.
Sign me up, I would like to work for your company.
Doesn't the Linux Logical Volume Management software do snapshotting? Ok, not perfect, but it ought to work.
a ck up.html
http://www.tldp.org/HOWTO/LVM-HOWTO/x136.html
http://www.tldp.org/HOWTO/LVM-HOWTO/snapshots_b
Not really efficent for hourly, but perhaps it could be made to work, also look at rsync or some of the options for cp.
Plato seems wrong to me today
So, how often do you create backups? Every 5 minutes, 10, 20, what? In the real world, where everyone makes mistakes (gasp! make a mistake!?!), the end-user most likely made a significant change to a document, only to overwrite the changes or delete the document by accident. The NetWare salvage tool is an excellent tool for "undoing" these kind of mistakes while not wasting time retyping the entire document, or restoring the document from backup, and then pulling together the changes that were overwritten from that point (plus, as the sysadmin, I dont have to spend time restoring the document)...
Dont be so quick to criticize, it makes you look like an idiot, and turns people off to asking these types of questions in a public forum.
This presentation (section A network "Recycle bin") and this post give some information about it. It probably needs some work, it seems to be mostly experimental.
keeps copies of all deleted, renamed, truncated or overwritten files
works with all applications on the system
file system independent
versioning (!): all versions of a file are kept in the trash directory
very configurable
But what about only using SymLinks. Then write a little scripty that checks to see if the symlink has been deleted. if it has put a counter on the file and do what you must to the file saved safely on the server...
You can put the list of files in /etc/ld.so.preload, and it'll affect all apps. This is used by the Gentoo Sandbox.
I guess that libtrash is the asnwer.
Slashdot community, please notice: I am looking for a girlfriend.
Nave H. Weiss
More seriously, most Linux-ians and Slahsdotters have never worked with a properly designed, engineered, and installed Netware network, so they don't know what that system is (was?) capable of.
sPh
There is a patch for Samba that provides similar functionality for Linux. Read this nifty article:
l
http://www.amherst.edu/~bbstone/howto/samba.htm
I like having an extra harddrive in my systems, and just run nightly backups to the hard drive. guess you could do this more often, say every 1 hour...
In my work this is very useful, I to remember the convenience of FILER, SALVAGE. But Windows Servers don't offer a way to save deletes from a network share, and any third party apps seem to cause the blue screen of death. So I'll stick with an extra hard drive. I use Linux, but find it too cumbersome to use daily for simple file sharing, and Print serving. I'll be using at home until I have more time to spend with my server at work, which may be a long time to come....:(
Cheers.
And it has worked since the first verion of the OS. Now, admitedly it's not a network file server, but it does preform the same function for the user--they think they deleted the file (thrown it out) but they can still rescue it until they run out of disk space and 'empty the trash can.' But, I would suggest that in this case what you really need is an addition/mofification to the samba codebase which would allow your windows SMB shares (what it sounds like you really care about) to not delete files when the appropriate network APIs are called, but rather move the files to a temporary directory which could opperate as a FIFO queue or a 'empty manually when it gets full' system.