What Goes into an Enterprise Network?

Komi asks: "I work for a big semiconductor company, and I'm part of a group that is spear heading the Linux movement here. Right now everyone uses Sun machines to design, but you can get a cheaper Linux x86 machine that is four times faster. So it is my job to prove that Linux works. The problem is that I'm an analog circuit designer stuck in the role of sysadmin. So I need some advice on what goes into a network. It won't be that large right now, but it has to be scalable for up to a couple of hundred machines. If this works, then hopefully we'll convince all designers at my company to make the switch."

"Here's the hardware that I am planning on getting:

• 2 servers:
  
  These would hold the home accounts and tools, as well as serve out NIS, NTP, etc. I know I'll need a lot of hard drive space (2x72GB SCSI each), but do I need a lot of memory? (It's 4GB RDRAM max.) Should the processor be fast, or dual?
  
  
• 3 batch machines:
  
  These would be a small compute farm running LFS or something. Jobs would get queued up and run continuously. So these should be dual CPU with lots of memory, probably 4GB each. Any other particular details?
  
  
• 10 desktop machines:
  
  These would be on the designers and developers desktops. These should be reasonably fast (~2GHz) single CPU machines with probably need at least 2 GB RAM. The simulations we run do not benefit from dual CPUs. They probably don't even need SCSI. I'm thinking a $2k PC should work.
  
  
• 1 Itanium server:
  
  This would be to play around on to test our 64-bit applications. The only advantage of 64-bit is applications using huge amounts of data.

We plan to run Red Hat 8.0 on these machines. Is there anything I'm missing? I don't have much redundancy in the servers. I plan to do backups to DVDs. Is this asking for trouble? Any further advice would be appreciated."

Comments

What Goes into an Enterprise Network?

[Anonymous Coward]

What Goes into an Enterprise Network?

Dilithium?

Biggest troll ever?

[WasterDave]

Either this is the biggest troll ever, or you're deeply in the shit. Assuming it's the latter, for now, I shall toss my orb and see where it lands:

*HIRE A REALLY GOOD SYSADMIN*

You're horrendously out of your depth and there are shedloads of really good sysadmins around who need jobs. Take someone on for three months to look at the problem properly. Advice 2:

*DON'T BUY AN ITANIUM MACHINE*

There is simply no point, particularly if you don't really know what you're going to use it for.

Cheers,
Dave

Re:Biggest troll ever?

[Zapman]

It's a big troll, sure. However it is also a chance to dispense some good advice:

1) There's a difference between PC's and 'Server Class' hardware. The biggest is testing. It will work, and it can be supported easily. Drivers are nice and available (generally speaking). Usually dual proc, usually RAID enabled. You can use RAID to speed up read access, but almost no one does. They use it for redundancy (in case a disk flakes out on you). How much money you spend depends on how much downtime costs. If it really costs, you need RAID 1+0 or 0+1. Go with hardware based RAID if you can.

2) Sun hardware. There are many more advantages to sun hardware than what's obvious. Never over look what a good support organization can do. You pay for it, but if something fried, I can have a part in my hands 4 hours later. Sun's low end desktop's are nothing to write home about. However, if you've got Ultra60's or SunBlade 1000 or 2000's, that's some really class hardware. You can do some supprising things with it. [1]

3) Dual procs. On Desktops, even if your simulations don't benifit from dual proc, if they take a while, and they eat that 1 CPU, you'll be happy to have a second (web browsing, etc). On your servers, it's effectivly a must.

4) RAM. On the servers, crank it. On the desktops, you should probably crank it.

5) Cost. If your work is anything like mine, you have 'capital' money, and 'O and M' money. When in doubt, over spec the machines, so that your less likely to have to request more money from the 'capital' pool than you initially quoted. "Going back to the well" viewed poorly.

6) NIS. NIS is evil and the plague. If your in a relativly local office with good connectivity, it's alright. If you try to spread it over WAN links, you're going to get hurt at some point.

7) NTP? Why run a seperate server when you don't have too. Leverage what's already in use in the company. This leads to my last point (and what was the best point of the parent)

8) Get yourself a real sysadmin. These are decisions that s/he is experienced in, and paid to do. Your trial by fire that would come from this will probably drive you insane. Good sysadmins are a rareish breed. I know, I am one. There are a fair number of good ones out of work now. Find one.

[1] The reason largely has to do with cache. Sun chips made in the last 2ish years have 8 MEGS of cache on them (that's even mirrored so it's 16 in total, but you can only use 8). We built a GIS app, and field tested it on Sun and Intel hardware. The intel hardware could deal with 1 to 4 users with less resources than the sun box could. However, the the sun box kept growing up to several hundered users, while the intel box started thrashing hard after 10 or so. We compared a dual US3 box to a dual Xeon P4.

Re:Biggest troll ever?

[Komi]

This is not a troll. The issue is money. We can't afford to hire anyone, or to buy hardware. But we have a source of free loaner equipment. Our deal was to prove that their machines work, that way when design groups start getting money again, they will buy it from a proven source. So they asked us what we need, and I have to compile a list. This is a proof of concept on zero budget (except my salary I suppose). And it has to be all linux, because that's the deal.

Sysadmins will be hired for this once money gets freed up and we can prove to groups that linux works. A later post was correct that there are really two issues. a) Getting everyone to switch to linux, and b) getting designers to put linux on their desktop. We really only care about b), but by the nature of the deal, we have to prove a) and b). Also we don't care about the cost of switching a design group over to linux either. That's someone else's job. We just show that the end result works.

And finally, we do need 64-bit machines. Some of the programs we run use huge ammounts of data that need 64-bit to address them. So if we're getting free loaner equipment, then why not play with an Itainium? :)

I appreciate the advice from everyone.

Thanks,
Komi

You're facts aren't quite straight.

[pmz]

So I need some advice on what goes into a network. It won't be that large right now, but it has to be scalable for up to a couple of hundred machines.

1) You had better find some damn fine PCs to replace those Suns, because a couple hundred PCs can make your life miserable due to lots of random breakage.

...you can get a cheaper Linux x86 machine that is four times faster.

2) This is not true (unless you found Pentiums with SPECfp of over 3000!). If you buy the right-sized computers for your task, the hardware costs won't be a dominating part of your budget. Human costs and non-OS commercial licensing will be, regardless of your platform choice.

Whenever people say that Linux is absolutely outright cheaper then commercial UNIX, then I'm pretty convinced they haven't figured out all the costs involved. Also, I'm not convinced they understand just how simple maintaining a Solaris box can be, for example, due to sunsolve.sun.com, ample documentation, optional support out the wazoo, etc.

Before you go blazing these new trails, just stop and think for a minute. Put aside the zealotry and really think hard about what is and is not cost effective. Regardless of your choice, you really need to be convinced it is the right one.

Re:You're facts aren't quite straight.

[shaka999]

I wonder what side of the aisle your on? Are you a sys admin or an end user?

Like the author of this email I work at a larger semiconductor company. We are in the middle of switching from Sun to Linux. The price/performance difference is huge. There is more than a 10X difference in price between a DELL box and slower Sun machine.

We have been up on Linux for over a year and so far haven't had many issues. Ok, well we have one issue.

Re:You're facts aren't quite straight.

[Anonymous Coward]

I also work at a company that does chip design. While it's true that the software costs are overwhelmingly huge in this space, that has not seemed to blind our money minders to the cost of the hardware. An inexpensive dual cpu PC will complete the same job as a Sun 420 in about a third of the time. This is empirical fact. The PCs also install in about half of the time a sun takes to jumpstart. We still have quite a few Suns, but the engineers only use them for huge memory jobs that won't fit in the RAM available on the PCs.

As far as your concerns over reliability, we buy name brand PCs that are meant to be servers, not crappy integrator machines or desktops. I find that the hardware reliability on these machine is at least as good as the Suns.

That said, we *have* to have the Suns for those jobs. We use Suns for infrastructure stuff as well. And some of the engineers cannot *live* without a Sun on their desktop, though they all whine they their desktops are too slow to run a browser.

My feeling is that until Hammer hits the streets it's going to take a mixed environment to get the job done. Certainly when we have an option to run Linux servers with many GBs of RAM we won't be buying more Suns.

Re:Feasibility Study...

[shaka999]

Your missing one metric which my company has found to be the most important!

QUALITY!

Because of the reduced price and increased performance of our DELL XEON boxes we are able to run more simulations on our circuits. This allows us to check a greater range of operating conditions thus improving the overall quality of our product.

The BIG Never in Enterprise Networks

[Inexile2002]

Never let Captain Kirk talk to the main computer. Every damn time he does he tricks it into self destructing. You'd think he doesn't want the Enterprise to have a network...

DAMMIT Jim! I'm a Doctor not a UNIX admin!

Hardware doesn't make the enterprise software does

[MerlynEmrys67]

Well you have listed some trivial hardware requirements, what you haven't said are things like: 1) Does your application that the designers use to do their daily work exist on Linux, does it run as well, is as fully featured, cost the same amount of money... if the answer is NO then this is a non-starter 2) How are you going to handle signon, login, desktop managment, etc. 3) Backup is a big issue 4) Frankly 2 72 GB hard drives isn't enterprise or scalable. Look into RAID, LVM, and other options to make the hard drive system more reliable 5) The Linux solution isn't 4X cheaper, frankly it is significantly more expensive... you have all ready purchased the current solution correct, so the cost to maintain it is 0 (well not really but still) vs. having to buy this list of hardware and very possibly new software licenses (you have the solaris licenses right now correct, probably not Linux ones.. if they exist, see point 1) So the cost of this system going forward is significantly higher than the current solution Other than that, go for it... just remember it is much easier to tell you to spec it out and then say "We can't spend that kind of money" rather than tell you No up front

Think TASKS not BOXES!!

[crmartin]

The first thing you've `got to do is stop thinking about how you're going to buy a couple of boxes and that'll make your network, because, Bullwinkle, that trick never works. Except, at least, for those of us who consult for a living, because we often get gigs out of saving someone's shorts from the George Foreman.

Now, back up and think about this:

• who will use the machines on the network?
  
• what will they be doing?
  

In your case, you're talking primarily about engineers, and they are primarily (for job functions) going to be doing engineering ... which means (this is not sarcasm) that they will spend anywhere from 2-4 hours a day interacting with their tools of choice for circuits and engineering, and the remainging time with web browsers, email programs, etc., particularly including word processors or the like. Since you're starting with a Sun network, you at least have confidence that everything people would normally use is UNIX-able.

Now, on you EXISTING network, measure what a few users do for at least a few days. If you've got admin on, you should be able to extract information from the logs. This will give you a chance to get at how much load there really is.

Next task: establish some of your "non-functional" requirements. In particular, how long can response time be for your most important tools, how long can you afford to have the system as a whole be unavailable, and how much work (an hour, half a day, a week?) can you afford to lose. Divide all of those by two and make them your basic "service level agreement" -- which is simply a statement of the service you promise the users, it doesn't have to be fancy.

Here are some reasonable values, from experience, but YMMV: most people will put up with the whole system being unavailable for an hour, they want half-second response time from specialized tools and more like about 4 seconds on a web page, and engineers hate losing ANYTHING but usually don't get too pissed off if it's less than a couple of hours work and doesn't happen very often.

Next: what's the environment? Do you have to think about firewalling yourself from the rest of the network? (Don't assumme just because you're inside the corporate firewall that you're protected. Get AND READ the corporate security policy, as well as talking with the admins who own the network as a whole.) How will you do backups? How do you fit into the corporate disaster planning scheme? (Lots of people forget that one, but just look into what happened to the Wall Street Journal on 9/11 to see how essential it really is.) This analysis will give you a good idea what you need.

And now, having said all that, it will turn out that what you're going to need is (1) a "big enough" file server with 5/4 RAID and a good periodic backup onto "archival media" like tapes or writeable CDs; (2) one workstation good enough for all your applications, and with at least a years' room for growth, for each desktop (plan to buy at leasy one for a spare, and set it up "hot" so a single failure doesn't slow anyone down"); (3) a smallish box as a print server (if you manage your own email, it can often go onto this); and (4) a firewall box or a router (betcha 50 cents Canadian that the company will insist on this.)

Plan for a full week, plus one day per user workstation, for installation. That is, with 4 users, plan on 5 + 4 = 9 days for two people.

All the other stuff, like using NIS, NFS, Kerberos, etc, will more or less fall out if you get these steps right first.

Why fight the whole war at once

[jbolden]

I'm not a system admin but it seems like you are confusing two different battles:

1) Getting the whole company moved over to Linux for everything

2) Getting engineer workstations running on x86s so you can get 4x the speed.

(2) is a much easier battle to fight than (1). Don't spec a whole Linux solution for everything, spec out a Linux solution for the workstations that allows them to work with the Suns. There you can make the cost difference really obvious. Reliability isn't a big deal.... Your software vendor might even give you the test software in hopes of the license switch down the line. In the back of your mind you can keep the total Linux solution but your strategy should be to take out the Suns piece by piece by piece.

Total overhauls come down from above not up from below. Incrimental change that overtime turns into a total overhaul comes up from below. You don't sound like you have anywhere near the juice to get a total overhaul through the company regardless of how good your analysis is.

not only hardware

[Ludoo]

as a previous poster said already, hardware is not the most important factor. you will eventually find yourself working on old or semi-obsolete hardware anyway, so getting top stuff is not a priority, especially given the number of users.
What I would concentrate in is:

• a single source for authentication (login) and profiling (groups, home dirs location, etc.); study pam a bit, a good option is to store everything in ldap and use pam_ldap; if security is a primary concern, consider kerberos
• network file sharing; you don't want your users' data scattered around on every desktop (your management costs will increase dramatically, and your backup strategy will be much more complex); nfs is quick and easy, but offers only decent performance and poor security; a good (but complex) alternative is openafs or IBM's DFS (which is the evolution of afs
• centralized backup on a single server, possibly running amanda so that you can backup different servers on a single medium; mondo rescue is a good option to backup systems periodically on bootable cds for quick recovery;
• standard distro, eg pick Redhat or Debian or whatever, based on a number of factors like ease of automating installation, software distribution and package management options, etc., and stick with it; reme,ber that you have to know your patricular distro well to handle emergencies (and emergencies DO happen);
• standard desktop, eg pick one of gnome or kde, develop suitable policies and management strategies, and stick with it; one of the factors in deciding a desktop is the toolkit used and its licensing, if you intend to develop custom software in the future;
• software distribution strategy, plan or at least try to learn a bit about possible ways to handle updates and software installation on your desktops (and servers); you can automate package management (apt or rpm) or enterprise software (red carpet or rhn);
• printing system, again for printing you have different options: lprng, cups, etc; check what printers/plotters you already have in house and if they're supported by printing systems;
• 
  Just a quick overview, to sum it up I would second the advice somebody else gave you in a previous posting: hire a decent sysadmin and plan things with him.

"analog circuit designer...sysadmin"

[duplicate-nickname]

LOL...these are the type of people Windows admins have been putting up with for years, and now you *nix guys can start dealing with them.

"Hi, I was a desktop support tech, now I have been thrown into the job of managing our Windows network, how do I install that Active Directory thing?"

Windows has had the burnden of bad, inexperienced sysadmins for years, now Linux can share in the joy as it's more widely deployed.