Virtualization is the thing. I was fortunate to be able to do this early on (5+ years ago) and I learned a few things along the way:
1) Memory is the thing. VMware and the other hypervisors are really good at making the most out of memory (ballooning, shared memory, de-dupe, etc.), but RAM is cheap now. My setup has 16GB and I can do just about anything I want with this.
2) Disk is even more the thing. My setup is a cluster, but even if it wasn't I'd still use some sort of external disk solution. I have two in my environment. For my "dev and cheap" stuff I use a cheapo windpc+opensolaris/indiana ZFS based NAS box. You can build yourself something that rivals enterprise class stuff for dirt cheap. My other solution is a discard from one of my customer. I have an (older) netapp array. It's clustered and does all sorts of fun stuff. At one point it was an enterprise class solution, so for my lab it's more than enough. You use external storage because your VM's then become transportable. The server you use today, might not be the same tomorrow. Moving the bits around from one server to another is a pain. Having them on external storage makes it really easy: fire up the new server, connect it to the storage, shut down the VM's on the old server, start them up again on the new server, retire the old server. And if you have compatible architectures, you might not even need to do the shutdown the VM's at all.
3) Get your servers from the company you work for (or a friend works for). My lab is Dell 2950 based. I was able to get the servers for next to nothing from a company that had depreciated them and was willing to sell them for nothing. They are not the most current, bestest, fastest thing, but guess what - the time when CPU and memory are the bottlenecks for most stuff is over. Even three year old hardware can run most current OSs more than adequately for a home lab. Granted, if you are going to playing games in your "lab", then this isn't going to work. But then again, it would turn this whole discussion moot anyways.
Have you checked out http://zimbra.com/? It's a complete OSS email, contact, calendaring solution. I'm not sure about the Entourage integration, but it does have outlook integration in the commercial version. I'd be surprised that it doesn't work. It does come with an AJAX interface that's well worth looking into as an addition/replacement for entourage.
----
tachijuan http://amuyu.com/
Storage is all that I do for a living. Here's a quick summary of how you could do it:
1) First you have to make sure that the drives you have your data on are going to be able to give you the read rates that you need. I highly suggest that you go with a raid array of some sort. Preferrably one with substantial cache in front of it. The raid controller should be smart enough to do what's called "read ahead" caching. That reduces the read miss ratio and speeds up sequential (i.e. backup) applications tremendously.
2) Get whatever tape drive you choose. Base your decision on the speed of the device - use only the native performance. LTO is either 15mb/s or 16mb/s depending on whos drive you get. It is safe to assume that you will get about 1.2x compression. So for LTO that would get you either 18mb/s or 19.2mb/s. Assuming you get the 15mb/s drive you can realistically expect to get ~64GB/hr per drive.
3) You have to get some sort of advanced backup package to support those rates. I would suggest that you go with either Veritas NetBackup, Legato Networker, SyncSort Backup Express, or any of the enterprise class products. Don't go with cheap software - in general they do not have the performance coding necessary to move data at very high rates efficiently. This is a hard choice, but if you stick with the three I told you, you should do fine.
4) Get a library that can handle several drives, so that you can use them in parallel.
5) Put each tape drive on a separate scsi bus, or if you go with fibre channel put at most 3 drives on the channel. There's a ton of way of architecting this side of the house, but in general if you stick with those numbers, you should be fine.
6) Try not to send data over the network, even with GigE, the effective rates are going to be drastically slower than those of direct attached devices. GigE also severely impacts the server - tcp/ip overhead is a bear for high throughput environments. There are ways around this, but that ouside the scope of this email:-).
That should do it for you for the traditional backup methodology. There are other ways of doing backups - making mirrors that you can split of. Taking snapshots..... There are a ton of products that can help you on this. Some of them are software based packages that sit on the server with the data. Some of them are hardware/software devices that sit on a SAN or a NAS. Again, going into this is quite lengthy, but it can be done. I have a customer for which we are doing over 1TB/hr backups using a combination of lots of tools. You problem is quite a bit simpler.
Slashdot Mirror
Virtualization is the thing. I was fortunate to be able to do this early on (5+ years ago) and I learned a few things along the way:
1) Memory is the thing. VMware and the other hypervisors are really good at making the most out of memory (ballooning, shared memory, de-dupe, etc.), but RAM is cheap now. My setup has 16GB and I can do just about anything I want with this.
2) Disk is even more the thing. My setup is a cluster, but even if it wasn't I'd still use some sort of external disk solution. I have two in my environment. For my "dev and cheap" stuff I use a cheapo windpc+opensolaris/indiana ZFS based NAS box. You can build yourself something that rivals enterprise class stuff for dirt cheap. My other solution is a discard from one of my customer. I have an (older) netapp array. It's clustered and does all sorts of fun stuff. At one point it was an enterprise class solution, so for my lab it's more than enough. You use external storage because your VM's then become transportable. The server you use today, might not be the same tomorrow. Moving the bits around from one server to another is a pain. Having them on external storage makes it really easy: fire up the new server, connect it to the storage, shut down the VM's on the old server, start them up again on the new server, retire the old server. And if you have compatible architectures, you might not even need to do the shutdown the VM's at all.
3) Get your servers from the company you work for (or a friend works for). My lab is Dell 2950 based. I was able to get the servers for next to nothing from a company that had depreciated them and was willing to sell them for nothing. They are not the most current, bestest, fastest thing, but guess what - the time when CPU and memory are the bottlenecks for most stuff is over. Even three year old hardware can run most current OSs more than adequately for a home lab. Granted, if you are going to playing games in your "lab", then this isn't going to work. But then again, it would turn this whole discussion moot anyways.
Have you checked out http://zimbra.com/? It's a complete OSS email, contact, calendaring solution. I'm not sure about the Entourage integration, but it does have outlook integration in the commercial version. I'd be surprised that it doesn't work. It does come with an AJAX interface that's well worth looking into as an addition/replacement for entourage.
----
tachijuan
http://amuyu.com/
Here's a copy of the email I sent to cybrthg:
:-).
Storage is all that I do for a living. Here's a quick summary of how you could do it:
1) First you have to make sure that the drives you have your data on are going to be able to give you the read rates that you need. I highly suggest that you go with a raid array of some sort. Preferrably one with substantial cache in front of it. The raid controller should be smart enough to do what's called "read ahead" caching. That reduces the read miss ratio and speeds up sequential (i.e. backup) applications tremendously.
2) Get whatever tape drive you choose. Base your decision on the speed of the device - use only the native performance. LTO is either 15mb/s or 16mb/s depending on whos drive you get. It is safe to assume that you will get about 1.2x compression. So for LTO that would get you either 18mb/s or 19.2mb/s. Assuming you get the 15mb/s drive you can realistically expect to get ~64GB/hr per drive.
3) You have to get some sort of advanced backup package to support those rates. I would suggest that you go with either Veritas NetBackup, Legato Networker, SyncSort Backup Express, or any of the enterprise class products. Don't go with cheap software - in general they do not have the performance coding necessary to move data at very high rates efficiently. This is a hard choice, but if you stick with the three I told you, you should do fine.
4) Get a library that can handle several drives, so that you can use them in parallel.
5) Put each tape drive on a separate scsi bus, or if you go with fibre channel put at most 3 drives on the channel. There's a ton of way of architecting this side of the house, but in general if you stick with those numbers, you should be fine.
6) Try not to send data over the network, even with GigE, the effective rates are going to be drastically slower than those of direct attached devices. GigE also severely impacts the server - tcp/ip overhead is a bear for high throughput environments. There are ways around this, but that ouside the scope of this email
That should do it for you for the traditional backup methodology. There are other ways of doing backups - making mirrors that you can split of. Taking snapshots..... There are a ton of products that can help you on this. Some of them are software based packages that sit on the server with the data. Some of them are hardware/software devices that sit on a SAN or a NAS. Again, going into this is quite lengthy, but it can be done. I have a customer for which we are doing over 1TB/hr backups using a combination of lots of tools. You problem is quite a bit simpler.