How Would You Make a Distributed Office System?

Necrotica writes "I work for a financial company which went through a server consolidation project approximately six years ago, thanks to a wonderful suggestion by our outsourcing partner. Although originally hailed as an excellent cost cutting measure, management has finally realized that martyring the network performance of 1000+ employees in 100 remote field offices wasn't such a great idea afterall. We're now looking at various solutions to help optimize WAN performance. Dedicated servers for each field office is out of the question, due to the price gouging of our outsourcing partner. Wide area file services (WAFS) look like a good solution, but they don't address other problems, such as authenticating over a WAN, print queues, etc. 'Branch office in a box' appliances look ideal, but they don't implement WAFS. So what have your companies done to move the data and network services closer to the users, while keeping costs down to a minimum?"

Comments

erm..

[Anonymous Coward]

Or, in other words, how do i put servers in branch offices without putting servers in branch offices?

If you solve that one let me know...it's been bothering me a while too...

Not enough information.

[Anonymous Coward]

Financial companies, at least in my State, have very specific requirements for storing and transmitting data. Without knowing what your specific needs are, I have no answer other than "Define your problem".

The reality is other companies, such as yourself, exist and function probably better. If that indeed is the case, perhaps a friendly lunch with another IT staff member might help you.

I've consolidated offices and I've also pushed out servers to remote offices. It all depends on the need of the client. Examples

1. Client wanted 99.999% uptime and the only way I could get that was to have their servers in a data center. We moved them and uptime has been great.

2. Client wanted fast file access. We setup DFS with WIndows 2003 over a WAN link (T1) the client has never been happier.

So, to answer your question, it depends on your needs.

Hmm

[moogied]

Dedicated servers for each field office is out of the question, due to the price gouging of our outsourcing partner

Find a new partner.

Amazing

[obeythefist]

Some basic truths.

IT costs money. I'm sorry that your outsourcer had some bad ideas. But your management must understand that IT services aren't free, and the health of your company depends on it's infrastructure.

Without knowing the specifics, the only low cost suggestion I can provide is converting desktop PC's into Linux servers, thus providing you with the distributed server network you need. Of course, the boxes will be underpowered and fall over all the time (yay desktop hardware), but if you really want to cut costs, there you have it. For backups, put in extra hard disk and backup to disk, it beats nothing at all.

Pixie dust

[c0d3h4x0r]

Think happy thoughts, and sprinkle some pixie dust over your IT infrastructure, and all your problems will be solved.

But whatever, you do, don't fire your incompetent outsourcing partner or actually invest in beefing up your IT resources. Both of those paths are DOOMED, DOOOOOOMED, I say!

Re:No Good Solution

[chappel]

I was really impressed with the improvements we got by implementing some 'smoke and mirrors' from Riverbed (http://www.riverbed.com/). Granted, we've got some reasonably adequate bandwidth to start with, but it dropped the WAN traffic to our large (500 user) remote site by a good 80%. They seemed mighty expensive for a plain dell server with CentOS, but there's no arguing with results. /reminds self to look into riverbed stock

Too little too late

[armada]

I suggest you pay more attention to the data itself. Do an comprehensive and brutaly unbiased audit of what data/resources are needed by whom. You would be amazed at how much of your infrastructure is either superfulous or capricious. Once you do this then you at least have a smaller mountain to climb.

So, here's your answer:

[SanityInAnarchy]

Either consolidate your servers, or don't.

Exactly what costs were you thinking of saving by consolidating? If it's just the cost of building and maintaining those physical servers, then here is the cold, hard truth: You are paying less for less service. Put servers at each branch office if you'd rather pay more for more service.

You get what you pay for.

Now, if it's other problems that are keeping you from setting up those dedicated boxes, realize that these are other problems. Identify them, and bring them back to Ask Slashdot. We're Slashdot, we're not psychic.

If it's your outsourcing partner gouging prices, dump them for an outsourcing partner which doesn't gouge prices, or do it in-house.

If it's the inability to manage all those servers, get them to talk to each other, etc, that's a more interesting technical problem that Slashdot might be able to help solve.

There are a few exceptions -- you might be able to get away with something like Coda or AFS, though I don't know how well that scales to crappy bandwidth. But if so, that would imply that your only problem is managing strictly filesystem data -- it doesn't help at all if the problem is access to, say, an intranet webapp. So again, we need details, if we are to find the clever exceptions.

Otherwise, upgrade your bandwidth, and/or outsource your actual application servers to someone who can scale. If it's just web/email/docs, Google can do that. Otherwise, find someone who specializes in what you're doing (our SVN is run by cvsdude.com), or bite the bullet and buy some virtual servers.

Re:No Good Solution

[eazeaz]

We use riverbed appliances at all our remote offices. They take about an hour to install and are damn near like magic. I just pulled some statistics from one of our remote offices. Over the last 30 days, we had a reduction in data flow of 95% 6.3GB of data went over the T1 instead of 129.3GB We can run applications over a T1 and users do not know that they are not local. They allowed us to go from DS-3 to T1 lines without any user complaints.

Welcome to the cross roads...

[moorley]

Of a good a idea that worked well in one area but is not ready for full adoption. Wide Are Network has too much latency to simply turn local office systems global.

Your company is trying to cheat their development model. Rather than setup a distributed IT application they have simply tried to distribute a small office network worldwide. If you look back to the tried and true OSI model. 7 layers. The 7 layer model doesn't speak of Network File Sharing, it speaks of Hardware and Application. TCP/IP (which we have taken quite for granted) is around/below the application level. If you have an application that runs at the TCP/IP level you are good to go.

I have setup distributed systems for several ISPs in the late 90's. We didn't think about what we were doing or why it worked. It looked like we could long haul anything we wanted. A little lag in sending mail or a few extra milliseconds to authenticate LDAP is no big thang. The Internet is distributed by nature. Sometimes DNS was a little slow but that was acceptable for 56k modems and DSL customers. But we spent 2 years working on a central web based administration/billing/customer support application with 1 SQL base in the center. We didn't distribute the application and have it write to the SQL base directly or move files around.

But you can't distribute the file layer. SANs in a local building have had some of the same problems. Any lag affects all applications and you solve it by throwing a big fat fiber backbone in the local building, but it break downs when you try to long haul over WAN links.

If your company is thinking it can sneak around coming up with a decent workflow model, and then implementing that in an application by simply given MS Office and Exchange (or whatever they have employed) to everybody they are sadly mistaken.

But worry not. You are not alone. Many business execs scratch their heads as to why the simply can't share out MS Project and their Excel Spreadsheets to 25 plus people teams and it will work fine. You still need to do the leg work of figuring out the work flow and reducing that to a transaction based system centrally located. That's it. All we've done in the last 20 years is replaced printouts with emails and spreadsheets, and the night operator (a job I used ta do) with scripts (or procedures) that dynamically update or run every 10-15 minutes. You still need a central system and then distribute parts of it, or have slim down interface that everyone can use remotely. Look at how a bank does it, just good ole dumb terminals.

No magic bullets yet. We need faster broadband and much lower latency before you can share out at the file layer using a network stack meant for transaction based appilications.

Let yourself off the hook. No mortal IT person can turn this tide....

You need local servers to reduce the latency. You need some decent thought on the application, not the OS and Office Suite. Good luck!