Ahhhh, the dot bomb days. Those were the days that I had to price out the cheapest machines we could get.. Everyone I knew that had other comparable positions were ordering big ass multiprocessor more bucks than I made in a year machines.. They all had the killer app, with the plan to make billions, and had millions in venture capital coming in, and blew it on shit like huge offices, poorly placed advertising, catered lunches, masseuses, and ungodly high salaries for the "executives" who did absolutely nothing. Pretty much fluffing their ego's, and not much else.
A year later, they were calling me asking for jobs.
Porn is definately the right business to stay in. People always like their smut.:) I'm going to be hitting my bosses up for about $60k in new equipment next week. It may or may not fly, but it will be fun to play with if it does.:)
The flames are no problem./. is much lighter than FidoNet (aka Fight-o-net) was back in the day.:) I fully expect a bit of bashing once in a while. I just take the liberty to answer 'em, if I'm in the mood.:)
This guy didn't say he was transfering CD images daily.
You'll see a *HUGE* difference between trying to transfer a CD image on a 100baseT hubbed LAN, and a 100baseTX switched LAN.
Take my office for example.
Transfering a 300MB file from a Linux server to a Windows workstation through a hub takes several minutes.
Transfering the same file on a switched LAN between two 100baseTX connected machines takes a small fraction of that time.
Transfering the same file on the same switched LAN between two 1000baseTX connected machines takes the same as the 100baseTX machines.
Your protocol more than likely isn't going to saturate 100Mb/s. If someone does a transfer like this, sometimes I'll pull up the bandwidth graphs. They'll almost never use up 100Mb/s doing it. Doing 10 simultaniously may.
But the question is, what's he trying to do? His statement was asking about doing it at home, just to do it, and I say "don't bother".
Someone at my office got a new Mac with a 1000baseTX card. They wanted to know about upgrading the LAN to support it. I asked him what he was really trying to do. Browse the Internet, and send 10 to 15Mb worth of files at a time. Nope, it's not worth it.
But sure, if you have money to burn, put GigE fiber between all your workstations, and then you can be all impressed with yourself every time you look at what you've done. You can brag to your friends "I spent $10k on my LAN" Why? Your bandwidth still peaks well under 100Mb/s.
According to the original story, "I've had a whole-house audio/video distribution project on the back-burner for a while now". He's looking to stream audio and video between computers in his house. Does he need 1000baseTX? No. He can get away with 1Mb/s or less between machines. So if he has 10 workstations simultaniously streaming, he may saturate at 10Mb network, 100Mb is fine for him. A switched 100baseTX network is ideal.
What you're probably encountering at 100base is the limitation of what your computer can send, or some crappy hardware on your LAN. Ya, that SMC hub isn't state of the art. Spend a few bucks on a good switch before you say "We should convert to GigE". Sure as hell I wouldn't recommend running it on that Linksys "workgroup switch". Ya, that's a bright solution for improving your network. Maybe if you're upgrading from a SMC hub, or one the no-name specials..
But hey, you probably also spent the few hundred bucks more for a Cyrix 6x86 PR233, and were screaming how great it was. "My Cyrix is so much faster than your Intel P200, who cares if it crashes all the time":)
I understand what you're saying. I have a few mirrors of sites that aren't mine, just in case they go away, so I can still see the content.:)
We didn't intentionally make it hard to do for you, it just worked out that way.
It actually becomes a big problem when people start spidering our servers all the time. That's a minor problem with Voyeurweb right now. Since it's all public, there's only so much we can do. So we have thousands of people spidering everything every day. The biggest problem came about when their spiders hit the message boards. Tens of thousands of hits to some of those CGI's just aren't very helpful. So, when we see it happening, we set firewall rules against them, so they only get a few requests in, and then can't request the message boards any more.
For Quantum, there isn't much I can do. We can discuss this more in private Email. Email me at jwsmythe (at) voynetworks.com.
Maybe if the particular site you're referencing is still laying around somewhere, I can dig it up for you. If you're lucky, it's on one of the machines I'm working on right now.:) When we moved fans.redclouds.com and community.redclouds.com from San Diego to Los Angeles, we transfered the sites up, and shut down the web servers. I'm getting ready to refurbish them, and put them up as mirrors of each site. Well, two of them. The other two I have evil plans for.:) If the site happens to be on there, you're in luck.
Well, "host" isn't necessarly an accurate term. We are Voyeurweb, among other things.:)
Through some connections several years ago, Igor hooked up with us. We used to just run small porn sites, but then Igor hooked up with us to do Watchcams.com (rest in peace).
I'm trying to make the clear distinction that we didn't "buy" Voyeurweb. There was a rumor about that for a while. It's still Igor's. We (well, my bosses) made a partnership with him, but he never lost control over what goes on the site.:)
The majority of our network and bandwidth are for Voyeurweb. Imagine that, a big free site, and a bunch of horny fuckers on the Internet. It's a great combinations for using up bandwidth.:)
Right now (well, at least on Mondays), we peak somewhere just over 1Gb/s across 3 cities (Tampa, Florida; New York, NY; Los Angeles, California). We found that we *HAD* to spread out across major cities, providers start bogging down. Tampa is particularly bad. i don't quite understand all the dirt with it, but it has something to do with Sprint handling all the last mile links in the city, so even though we are in a colo facility with a direct connection to our provider, Sprint gets overwhelmed, and our bandwidth would go flat at about 400Mb/s. Now if we detect any city doing that, we shunt some traffic over to other cities.
We're working on something else to do this more efficiently for us. Think of it like a local director on crack.:) Hopefully no one will even notice, other than getting slightly better speeds.
Sometimes I impress myself with Funbags.:) You don't know what kind of nightmare that is to have running. I had to write our own sync program to keep all the servers updated. Normally we'd use rsync, but it's too big for that. Every time we'd try to run it, it would suck up several hundred Mb of ram, and then pretty much kill the machine, and still never have copied the first file. And that was 2 years ago. So there's another 2 years worth of content in there now.
Just made a new revision to it's sync program, so now it works even better. We were having problems with one of the guys who reprocesses the contrs to put on Funbags putting trailing spaces on some (but not all) filenames. Either I was going to fix thousands of filenames, and update the associated HTML (yada, yada, yada), or fix the sync, so I did.:)
Today, I'm working on a little something new for Voyeurweb. Igor dropped me one of his "I have this great idea" Emails, which means a bunch of programming for me.:) It'll probably go live sometime this coming week. We're still working out bugs. Well, not programming bugs exactly, mostly we're going back and forth changing the look and function.
"I want this.."
"Ok how about this.."
"Can we add this?"
"Sure"
"Ok, how about this?"
"That'll mean rewriting everything we just did. Ok, it'll be ready tommorrow.":)
At least writing Igor's stuff is somewhat fun, and I know people like it when it's done well. It's not like I'm writing forms and CGI's just to collect data "Please enter your demographics here, and you could win $20"..
When we served a decent chunk of our assets (before we hooked up with Igor), we had about 40 servers in an office. The office's heat didn't even work. All we'd do is open up the server room door, and set the A/C up a little bit. There were two large units which kept the room cold. We'd just set the thermostat up to about 76 (from it's normal 60), and have the A/C fan circulate the heat throughout the offices.
I remember a long long (LONG) time ago, I was joking with one of the VP's at a company that is long since gone. We were talking about Tb arrays. A 1Tb array would have cost about $100k at the time. Now we can do 3Tb for less than $8k. Ahhhh, times change.
It isn't completely inconceivable to have *HUGE* storage. Its pretty easy to do 50Tb arrays, and you could put 3 or 4 on a machine pretty easily. Well, easily if you have an enterprise budget.:) I don't have a budget that big, nor is my wallet so thick that it needs to shed a few pounds. I wonder how soon filesystems will catch up with the space we could use. As drive sizes increase, these arrays will become more and more affordable to the general masses.
Hell, I still remember my first 20Mb hard drive, thinking "how will I ever use all this space". Most programs I had fit on one or two 1.44Mb floppy disks. I could back up everything I had on a box of floppies (and did frequently).
Read it again. He has 4 machines with 900Gb space each.
I have a 3Tb array on a machine, and a second on the way. That doesn't include all the smaller arrays we have. I'm afraid to calculate our total storage across all the machines. Worse than that, I'm not going to go through 100+ machines with large storage, and add them all up.
But hey, who's going to use all that storage anyways.:) Oh I forgot. Porn sites.
Actually, our biggest arrays are for backups, and staging for the larger sites. One of them is for a counters databases (masterstats.com), 15 75Gb very fast drives. It's not up yet, we're still deciding how to allocate the drives, and partition it.
Two of the good sized ones are on mail servers, mostly for redundancy, but it was cheap enough to get large drives while we were at it.
Our vendors love us. We buy lots of drives when we ask for them.
"Hi Mr. Vendor, we need two Quad Xeon machines with 24Gb RAM and two arrays with 15 drives each."
It seems like almost every time I'm standing around in the computer department looking at networking hardware, a clueless customer is asking a clueless sales guy about stuff. The sales guy will say something stupid, and I'll correct him. Then I'll help the clueless customer save a bunch of cash, helping him with what he needs, rather than what they wanted to sell him.
Who cares if he didn't spend a bunch of cash. He's a *HAPPY* customer now, knowing he got the right thing.
We use GigE fiber for our server networks, and pass up between 400Mb/s and 600Mb/s on high traffic days from each one.
The one thing I can say is that you'll probably never use it. There's really no need at this time. most protocols aren't any good at sucking up that much bandwidth on a single stream.
I've had many people prove this to me. They'll transfer files as single transfers. They can use up to about 10Mb/s. But if they transfer lots of files, they can use lots more. Try it through a switch that you can monitor bandwidth on. Through FTP, SMB, SCP, or whatever, you won't use up 100Mb/s. But, running multiple concurrent sessions, you can try to come close.
Heroinewarrior has a library called "firehose", which uses up all the available bandwidth, and will stripe across multiple connections to use up more. So, if you have 3 100Mb/s cards in a machine, you can come close to transfering at 300Mb/s.
You should also consider the other factors. Can your machine really send that fast? Is your hard drive fast enough to send over 100Mb/s ?? A nice fast SCSI drive, or a SATA drive can do it, but most IDE drives will fall short (specs be damned, try it in real life).
I transfer stuff around on the GigE lan all the time. We do exceed 100Mb/s, but it's usually with multiple machines.
The highest bandwidth usage machines we have are voyeurweb.com . They send out 150Mb/s through TEQL (Linux kernel option) combined 100baseTX cards, with several copies of thttpd running.
thttpd is a web server that is very small, and works very efficently. Apache has one process per connection, but thttpd has one process for everyone. Well, at least theoretically. It was around 80Mb/s of regular web site files, that it started flaking out. So, we run 4 copies of it on seperate IP's and let it scream.
As for our network, I'll outline our largest network.
We have a 1Gb/s uplink to Level3. This goes to a Cisco Catalyst 3508 (8 GBIC ports).
The remaining 7 GBIC ports go to 7 switches, mostly Cisco Catalyst 3550-48 (48 100Mb/s ethernet, 2 GBIC), and the servers are attached to the 100Mb/s ports. We have one Dell switch, which does 1000baseTX on all the ports, and a few machines with 1000baseTX cards. They can't pull anything resembling 1000Mb/s between each other. it simply doesn't happen. Honestly, doing transfers through http, ftp, or scp doesn't ever use over 100Mb/s on individual transfers. Sure, we can do it with multiple concurrent transfers, but at home, how many hundred or thousand users are you really trying to supply?
For home, you'll never use it. 100Mb/s is usually overkill. I set up my house with 802.11b, and at 11Mb/s peak, I see no difference than my old house, where we had copper run to every room and a Catalyst 2924 managing it. 11Mb/s is more than sufficent for a home network.
Spend your money on a *GOOD* 100Mb/s switch. I highly recommend Cisco, like a 2924, which you should be able to get relatively cheap used. Even if you put GigE cards in the machines, you can at least monitor your bandwidth now, and see what you really use. If you start flat-lining at 100Mb/s (bandwidth graphs make things really obvious), then you could consider upgrading.
I took a short (20 minute) job today, which involved fixing a customer's Cisco Catalyst 2924. There was an enable password set, but no one knew what it was. They wanted to make some network changes, most of which involved changing a couple port configurations. Zzz...
So I, not responsible for the lost password, took the "punishment" for the old admin loosing the password. Aparently the guy doesn't work for them anymore or whatever. Hell, I got paid for an hour, what do I care.:)
I hope this changes their strategy of putting in secret passwords. They're into security enough to know that is very dangerous. Secrets are not well kept, and someone will always leak.
I see a reply which says what I was thinking. I'm not 100% positive, and I don't have my Cisco books in front of me, but if I remember right, you can't do full duplex on a hub, only half duplex. Assuming your laptop or whatever monitoring box, wasn't sending anything at all, simply sniffing, you'd still be forcing your high-load server down to 100base half, which is probably a bad thing.
Ok, not probably, it is. a bad thing.:) I've fixed several connections for high load servers (and high load uplinks) because they came up as 100Mb half, instead of 100Mb full. Changing the switch port to 100mb full (or telling the offending machine to do the same) dramatically increases the performance.
We did a server move the other day. All the machines were plugged into a Dell GigE switch (which I don't recommend to anyone), one of the machines decided to negotiate a 100Mb half duplex connection, and it was pathetically slow. The guy who develops for those machines was bitching about the throughput on that single machine. I got on, switched the machine to 100Mb full, and then changed it to full duplex, and did the same for the switch. His performance came right back to where he expected it.
This isn't a problem in most environments. Normal humans only mess with machines doing less than 1Mb/s. I suspect there's more than one reader on here who deals with higher traffic though.:)
Your monitoring machine would still cause problems, even if you think it's just listening. There are probably broadcasts, if only ARP requests, which will induce at least a few responses from your monitoring machine, unless you specifically set it up to answer nothing.
You wouldn't want to do this to a link with any substantial traffic.
I guess I just see things in terms of the networks I work with a lot. Throwing 80+Mb/s through a hub may not be the wisest choice.:) Ok, it may not be entirely possible. (I'll leave it to someone with a higher Cisco cert than me to sort out the rest of that one)
My prefered way to do it is just have a port monitor another. But we use Cisco extensively, so it's really easy for us.:) I just have a bit of a problem monitoring our GigE uplinks. Not too many PC's or laptops can sniff 500+Mb/s, and most dont come with GigE fiber ports.:)
Actually, you sound like a kid who just got a job at a company who has 500+ employees, and wants to sniff their traffic.
You'll learn and get caught. But who am I to stop you from a life experience.:)
ethereal is great. It's proven to be lots of fun.:) tethereal is great too (comes with Ethereal). tcpdump is the grand-daddy of all packet sniffers, so it's kinda handy to know how to use it.
For wireless, I use Wellenreiter and Kismet.
Sitting in a major Las Vegas hotel, only a few floors up from the casino, I turned on my laptop, hoping to find an access point I could get online with (damned hotel didn't provide Internet access). I heard two AP's, and caught a couple IP's going by. I assigned myself an IP which appeared to not be used, and fired up ethereal.
I saw text for several of the casino machines going by. It was the text to be updated to the displays, including windows paths to where the files originated from (I believe). It was all in plain text. I noted down what I saw for a few minutes, shut down the laptop, and proceeded to lose for the rest of the night in the casino. Hey, that's what Vegas is for, right?:)
After I got home, I dug around for something resembling an admin contact at the casino, and advised him of what I saw. It would have probably been pretty easy to push my own updates to the machines. What would I say though?
"Gambing is an addiction, quit now." "This game is rigged, move on." "This is the droid you are looking for." "With a 97% chance of losing, did you really want to play this game?"
or, I guess
"I'm a spiffy keen elite haxor type person, props to my homeyz" haha
A lot of our old old employees were really stupid, and would fill out those "put your email for free stuff" links back in the days. Spam people trade lists all the time, so once you start getting spam, it's hopeless for it to stop. The occasional one will remove you from their list, but most of them continue to trade forever.
My box itself, I write to a lot of people. Lots of people write to us with various problems, and some of them are in the spam business, or share with someone that is. One of my addresses that's still in use has been used for many years, so I get plenty there. That's most of what gets filtered into the spam box now. Some of my alises got hit really hard, so I simply phased them out over time.
I guess it's all in the business you're in. Normal folks probably don't get as much spam as we do.
I feel really sorry for Hotmail, even if they are a M$ company. People use Hotmail addresses as throw-away accounts, so they must get absolutely hammered with spams, even if the accounts they're being sent to are undeliverable.
And that's just one month. I killed my accounts that were getting the majority of it, so this is maybe 20% of what I was getting before.
Our server overall is receiving 21,000 spams and 17,000 viruses per day. I can imagine what larger services do. A 1Gb limit is insane. But hey, if they have the budget for it, who's to say "no".
Of course, if Microsoft was to launch the same thing, everyone would be bashing it.:)
I'm a robotic underwater lumberjack and I'm okay, I sleep all night and I work all day.
Chorus: He's a robotic underwater lumberjack and he's okay, He sleeps all night and he works all day.
I cut down trees, I eat my lunch, I go to the lavatory. On Wednesdays I go shopping And have buttered scones for tea.
Fish: He cuts down trees, he eats his lunch, He goes to the lavatory. On Wednesdays he goes shopping And has buttered scones for tea.
Chorus: He's a robotic underwater lumberjack and he's okay, He sleeps all night and he works all day.
I cut down trees, I skip and jump, I like to press wild flowers. I put on women's clothing, And hang around in bars.
Fish: He cuts down trees, he skips and jumps, He likes to press wild flowers. He puts on women's clothing, And hangs around in bars?
Chorus: He's a robotic underwater lumberjack and he's okay, He sleeps all night and he works all day.
I cut down trees, I wear high heels, Suspendies and a bra. I wish I'd been a girlie, Just like my dear pappa.
Fish: He cuts down trees, he wears high heels? Suspendies...and a bra?
Chorus:...he's a robotic underwater lumberjack and he's okay, He sleeps all night and he works all day....he's a robotic underwater lumberjack and he's OKAAAAAAAAAAYYY. He sleeps all night and he works all day.
My girlfriend put a red sheet on our futon couch, 'cause everything I owned was black. I was happy with black. The TV, Stereo (and assorted equipment) matched the couch and chairs. Well, the TiVo stands out. One fuckin' silver box with all my nice black stuff. That's form and function, right?
She still doesn't agree with the idea that the grey candle holders were enough variety. Or the black&white photo of a rose was art.
If I could remember the name of the company, I would have listed it.
The really bad part is that they were running security tests against some of our servers for one of the credit card companies, to prove that we were secure. The credit card companies get a wild hair up their @ss occasionally, and want to know servers handling any sort of personal information are secure. They were sending me a self-evaluation, which was a single PDF in the self extracting zip. They were getting upset that I was telling them that I never received it. Well, I hadn't. Even if I had turned on notification of received viruses, I would have ditched it. For a while the server quarantined viruses, but it ended up wasting a whole lot of space, so I stopped even doing that.
The whole matter was being relayed through the bank handling the merchant account. Aparently that was one of the last mistakes they made, before they got a new company to handle their testing. They sent me PDF's, and had a much more professional report.
Both of them were still stupid about their reports. They'd cite every possible Apache exploit, regardless of platform. They were citing all kinds of Win32 and OS/2 exploits, and couldn't quite comprehend that we weren't using either. I think they just run `nmap -sV -O [hostname]`, and have it cross reference a database of exploits collected from BugTraq.
Providers should do this now. Email worms are such a hazard, and if everyone with a mail server took a few minutes to protect their users, these worms wouldn't be a problem.
We use MailScanner (from http://mailscanner.info) to protect our server. It's free, so this isn't an ad or something. It can use a whole variety of virus scanners. We update our dat files twice daily, to make sure the largest window of opportunity for these worms is only 12 hours. We also disallow all executable attachments, plus all kinds of extra filtering. MailScanner is very configurable, so you can have it block the virus and silently reject the message (that's what we do), or notify the users. Our users got tired of getting the reports, so I stopped sending them.
There's only one instance I can think of where a real message was blocked, but it was someone at a "internet security" company sending me an executable. They asked if I received it, and when they said it was a self-extracting archive (some_form.exe), I was like "Are you stupid? How do *I* know your machine isn't infected with something." Of course, I'm on a Linux box, so I'm not really worried about their silly Windows viruses, but it was the principle.
Slashdot Mirror
Well, there's an amazing waste of time and money. But hey, it's their money, not mine.
I'm tempted to call for a minute or two, even if it's an international call. It'll break the monotony of office conversation.
"Ya last night I called a cell phone in England, hanging from a balloon. What did you do?"
Ahhhh, the dot bomb days. Those were the days that I had to price out the cheapest machines we could get.. Everyone I knew that had other comparable positions were ordering big ass multiprocessor more bucks than I made in a year machines.. They all had the killer app, with the plan to make billions, and had millions in venture capital coming in, and blew it on shit like huge offices, poorly placed advertising, catered lunches, masseuses, and ungodly high salaries for the "executives" who did absolutely nothing. Pretty much fluffing their ego's, and not much else.
A year later, they were calling me asking for jobs.
Porn is definately the right business to stay in. People always like their smut.
The flames are no problem.
Most users won't see the difference.
:)
This guy didn't say he was transfering CD images daily.
You'll see a *HUGE* difference between trying to transfer a CD image on a 100baseT hubbed LAN, and a 100baseTX switched LAN.
Take my office for example.
Transfering a 300MB file from a Linux server to a Windows workstation through a hub takes several minutes.
Transfering the same file on a switched LAN between two 100baseTX connected machines takes a small fraction of that time.
Transfering the same file on the same switched LAN between two 1000baseTX connected machines takes the same as the 100baseTX machines.
Your protocol more than likely isn't going to saturate 100Mb/s. If someone does a transfer like this, sometimes I'll pull up the bandwidth graphs. They'll almost never use up 100Mb/s doing it. Doing 10 simultaniously may.
But the question is, what's he trying to do? His statement was asking about doing it at home, just to do it, and I say "don't bother".
Someone at my office got a new Mac with a 1000baseTX card. They wanted to know about upgrading the LAN to support it. I asked him what he was really trying to do. Browse the Internet, and send 10 to 15Mb worth of files at a time. Nope, it's not worth it.
But sure, if you have money to burn, put GigE fiber between all your workstations, and then you can be all impressed with yourself every time you look at what you've done. You can brag to your friends "I spent $10k on my LAN" Why? Your bandwidth still peaks well under 100Mb/s.
According to the original story, "I've had a whole-house audio/video distribution project on the back-burner for a while now". He's looking to stream audio and video between computers in his house. Does he need 1000baseTX? No. He can get away with 1Mb/s or less between machines. So if he has 10 workstations simultaniously streaming, he may saturate at 10Mb network, 100Mb is fine for him. A switched 100baseTX network is ideal.
What you're probably encountering at 100base is the limitation of what your computer can send, or some crappy hardware on your LAN. Ya, that SMC hub isn't state of the art. Spend a few bucks on a good switch before you say "We should convert to GigE". Sure as hell I wouldn't recommend running it on that Linksys "workgroup switch". Ya, that's a bright solution for improving your network. Maybe if you're upgrading from a SMC hub, or one the no-name specials..
But hey, you probably also spent the few hundred bucks more for a Cyrix 6x86 PR233, and were screaming how great it was. "My Cyrix is so much faster than your Intel P200, who cares if it crashes all the time"
I understand what you're saying. I have a few mirrors of sites that aren't mine, just in case they go away, so I can still see the content. :)
.
:) When we moved fans.redclouds.com and community.redclouds.com from San Diego to Los Angeles, we transfered the sites up, and shut down the web servers. I'm getting ready to refurbish them, and put them up as mirrors of each site. Well, two of them. The other two I have evil plans for. :) If the site happens to be on there, you're in luck.
We didn't intentionally make it hard to do for you, it just worked out that way.
It actually becomes a big problem when people start spidering our servers all the time. That's a minor problem with Voyeurweb right now. Since it's all public, there's only so much we can do. So we have thousands of people spidering everything every day. The biggest problem came about when their spiders hit the message boards. Tens of thousands of hits to some of those CGI's just aren't very helpful. So, when we see it happening, we set firewall rules against them, so they only get a few requests in, and then can't request the message boards any more.
For Quantum, there isn't much I can do. We can discuss this more in private Email. Email me at jwsmythe (at) voynetworks.com
Maybe if the particular site you're referencing is still laying around somewhere, I can dig it up for you. If you're lucky, it's on one of the machines I'm working on right now.
Well, "host" isn't necessarly an accurate term. We are Voyeurweb, among other things. :)
:)
:)
:) Hopefully no one will even notice, other than getting slightly better speeds.
:) You don't know what kind of nightmare that is to have running. I had to write our own sync program to keep all the servers updated. Normally we'd use rsync, but it's too big for that. Every time we'd try to run it, it would suck up several hundred Mb of ram, and then pretty much kill the machine, and still never have copied the first file. And that was 2 years ago. So there's another 2 years worth of content in there now.
:)
:) It'll probably go live sometime this coming week. We're still working out bugs. Well, not programming bugs exactly, mostly we're going back and forth changing the look and function.
:)
Through some connections several years ago, Igor hooked up with us. We used to just run small porn sites, but then Igor hooked up with us to do Watchcams.com (rest in peace).
I'm trying to make the clear distinction that we didn't "buy" Voyeurweb. There was a rumor about that for a while. It's still Igor's. We (well, my bosses) made a partnership with him, but he never lost control over what goes on the site.
The majority of our network and bandwidth are for Voyeurweb. Imagine that, a big free site, and a bunch of horny fuckers on the Internet. It's a great combinations for using up bandwidth.
Right now (well, at least on Mondays), we peak somewhere just over 1Gb/s across 3 cities (Tampa, Florida; New York, NY; Los Angeles, California). We found that we *HAD* to spread out across major cities, providers start bogging down. Tampa is particularly bad. i don't quite understand all the dirt with it, but it has something to do with Sprint handling all the last mile links in the city, so even though we are in a colo facility with a direct connection to our provider, Sprint gets overwhelmed, and our bandwidth would go flat at about 400Mb/s. Now if we detect any city doing that, we shunt some traffic over to other cities.
We're working on something else to do this more efficiently for us. Think of it like a local director on crack.
Sometimes I impress myself with Funbags.
Just made a new revision to it's sync program, so now it works even better. We were having problems with one of the guys who reprocesses the contrs to put on Funbags putting trailing spaces on some (but not all) filenames. Either I was going to fix thousands of filenames, and update the associated HTML (yada, yada, yada), or fix the sync, so I did.
Today, I'm working on a little something new for Voyeurweb. Igor dropped me one of his "I have this great idea" Emails, which means a bunch of programming for me.
"I want this.."
"Ok how about this.."
"Can we add this?"
"Sure"
"Ok, how about this?"
"That'll mean rewriting everything we just did. Ok, it'll be ready tommorrow."
At least writing Igor's stuff is somewhat fun, and I know people like it when it's done well. It's not like I'm writing forms and CGI's just to collect data "Please enter your demographics here, and you could win $20"..
When we served a decent chunk of our assets (before we hooked up with Igor), we had about 40 servers in an office. The office's heat didn't even work. All we'd do is open up the server room door, and set the A/C up a little bit. There were two large units which kept the room cold. We'd just set the thermostat up to about 76 (from it's normal 60), and have the A/C fan circulate the heat throughout the offices.
Yup, that's me.
You want to spider all the Quantum sites? Do you realize how extensive that is?
Exactly what are you looking for from it (other than collecting all the porn)? Maybe there's something we can integrate into the site for you.
I remember a long long (LONG) time ago, I was joking with one of the VP's at a company that is long since gone. We were talking about Tb arrays. A 1Tb array would have cost about $100k at the time. Now we can do 3Tb for less than $8k. Ahhhh, times change.
It isn't completely inconceivable to have *HUGE* storage. Its pretty easy to do 50Tb arrays, and you could put 3 or 4 on a machine pretty easily. Well, easily if you have an enterprise budget.
Hell, I still remember my first 20Mb hard drive, thinking "how will I ever use all this space". Most programs I had fit on one or two 1.44Mb floppy disks. I could back up everything I had on a box of floppies (and did frequently).
Read it again. He has 4 machines with 900Gb space each.
I have a 3Tb array on a machine, and a second on the way. That doesn't include all the smaller arrays we have. I'm afraid to calculate our total storage across all the machines. Worse than that, I'm not going to go through 100+ machines with large storage, and add them all up.
But hey, who's going to use all that storage anyways.
Actually, our biggest arrays are for backups, and staging for the larger sites. One of them is for a counters databases (masterstats.com), 15 75Gb very fast drives. It's not up yet, we're still deciding how to allocate the drives, and partition it.
Two of the good sized ones are on mail servers, mostly for redundancy, but it was cheap enough to get large drives while we were at it.
Our vendors love us. We buy lots of drives when we ask for them.
"Hi Mr. Vendor, we need two Quad Xeon machines with 24Gb RAM and two arrays with 15 drives each."
MMmmm ya.. I'm sure that was fun. I can just see you explaining to the power company the power consumption needs of your home computer room.
Sales people are generally clueless.
It seems like almost every time I'm standing around in the computer department looking at networking hardware, a clueless customer is asking a clueless sales guy about stuff. The sales guy will say something stupid, and I'll correct him. Then I'll help the clueless customer save a bunch of cash, helping him with what he needs, rather than what they wanted to sell him.
Who cares if he didn't spend a bunch of cash. He's a *HAPPY* customer now, knowing he got the right thing.
900Gb? Childs play. We just ordered our second 3Tb array.
It's 1000baseSX or 1000baseLX
Use 1000baseLX, have a GigE connection to friends and family miles away.
We use GigE fiber for our server networks, and pass up between 400Mb/s and 600Mb/s on high traffic days from each one.
The one thing I can say is that you'll probably never use it. There's really no need at this time. most protocols aren't any good at sucking up that much bandwidth on a single stream.
I've had many people prove this to me. They'll transfer files as single transfers. They can use up to about 10Mb/s. But if they transfer lots of files, they can use lots more. Try it through a switch that you can monitor bandwidth on. Through FTP, SMB, SCP, or whatever, you won't use up 100Mb/s. But, running multiple concurrent sessions, you can try to come close.
Heroinewarrior has a library called "firehose", which uses up all the available bandwidth, and will stripe across multiple connections to use up more. So, if you have 3 100Mb/s cards in a machine, you can come close to transfering at 300Mb/s.
You should also consider the other factors. Can your machine really send that fast? Is your hard drive fast enough to send over 100Mb/s ?? A nice fast SCSI drive, or a SATA drive can do it, but most IDE drives will fall short (specs be damned, try it in real life).
I transfer stuff around on the GigE lan all the time. We do exceed 100Mb/s, but it's usually with multiple machines.
The highest bandwidth usage machines we have are voyeurweb.com . They send out 150Mb/s through TEQL (Linux kernel option) combined 100baseTX cards, with several copies of thttpd running.
thttpd is a web server that is very small, and works very efficently. Apache has one process per connection, but thttpd has one process for everyone. Well, at least theoretically. It was around 80Mb/s of regular web site files, that it started flaking out. So, we run 4 copies of it on seperate IP's and let it scream.
As for our network, I'll outline our largest network.
We have a 1Gb/s uplink to Level3. This goes to a Cisco Catalyst 3508 (8 GBIC ports).
The remaining 7 GBIC ports go to 7 switches, mostly Cisco Catalyst 3550-48 (48 100Mb/s ethernet, 2 GBIC), and the servers are attached to the 100Mb/s ports. We have one Dell switch, which does 1000baseTX on all the ports, and a few machines with 1000baseTX cards. They can't pull anything resembling 1000Mb/s between each other. it simply doesn't happen. Honestly, doing transfers through http, ftp, or scp doesn't ever use over 100Mb/s on individual transfers. Sure, we can do it with multiple concurrent transfers, but at home, how many hundred or thousand users are you really trying to supply?
For home, you'll never use it. 100Mb/s is usually overkill. I set up my house with 802.11b, and at 11Mb/s peak, I see no difference than my old house, where we had copper run to every room and a Catalyst 2924 managing it. 11Mb/s is more than sufficent for a home network.
Spend your money on a *GOOD* 100Mb/s switch. I highly recommend Cisco, like a 2924, which you should be able to get relatively cheap used. Even if you put GigE cards in the machines, you can at least monitor your bandwidth now, and see what you really use. If you start flat-lining at 100Mb/s (bandwidth graphs make things really obvious), then you could consider upgrading.
That punishment doesn't necessarly fit the crime.
:)
I took a short (20 minute) job today, which involved fixing a customer's Cisco Catalyst 2924. There was an enable password set, but no one knew what it was. They wanted to make some network changes, most of which involved changing a couple port configurations. Zzz...
So I, not responsible for the lost password, took the "punishment" for the old admin loosing the password. Aparently the guy doesn't work for them anymore or whatever. Hell, I got paid for an hour, what do I care.
I hope this changes their strategy of putting in secret passwords. They're into security enough to know that is very dangerous. Secrets are not well kept, and someone will always leak.
I see a reply which says what I was thinking. I'm not 100% positive, and I don't have my Cisco books in front of me, but if I remember right, you can't do full duplex on a hub, only half duplex. Assuming your laptop or whatever monitoring box, wasn't sending anything at all, simply sniffing, you'd still be forcing your high-load server down to 100base half, which is probably a bad thing.
:) I've fixed several connections for high load servers (and high load uplinks) because they came up as 100Mb half, instead of 100Mb full. Changing the switch port to 100mb full (or telling the offending machine to do the same) dramatically increases the performance.
:)
Ok, not probably, it is. a bad thing.
We did a server move the other day. All the machines were plugged into a Dell GigE switch (which I don't recommend to anyone), one of the machines decided to negotiate a 100Mb half duplex connection, and it was pathetically slow. The guy who develops for those machines was bitching about the throughput on that single machine. I got on, switched the machine to 100Mb full, and then changed it to full duplex, and did the same for the switch. His performance came right back to where he expected it.
This isn't a problem in most environments. Normal humans only mess with machines doing less than 1Mb/s. I suspect there's more than one reader on here who deals with higher traffic though.
Your monitoring machine would still cause problems, even if you think it's just listening. There are probably broadcasts, if only ARP requests, which will induce at least a few responses from your monitoring machine, unless you specifically set it up to answer nothing.
You wouldn't want to do this to a link with any substantial traffic.
:) Ok, it may not be entirely possible. (I'll leave it to someone with a higher Cisco cert than me to sort out the rest of that one)
:) I just have a bit of a problem monitoring our GigE uplinks. Not too many PC's or laptops can sniff 500+Mb/s, and most dont come with GigE fiber ports. :)
I guess I just see things in terms of the networks I work with a lot. Throwing 80+Mb/s through a hub may not be the wisest choice.
My prefered way to do it is just have a port monitor another. But we use Cisco extensively, so it's really easy for us.
Actually, you sound like a kid who just got a job at a company who has 500+ employees, and wants to sniff their traffic.
:)
:) tethereal is great too (comes with Ethereal). tcpdump is the grand-daddy of all packet sniffers, so it's kinda handy to know how to use it.
:)
You'll learn and get caught. But who am I to stop you from a life experience.
ethereal is great. It's proven to be lots of fun.
For wireless, I use Wellenreiter and Kismet.
Sitting in a major Las Vegas hotel, only a few floors up from the casino, I turned on my laptop, hoping to find an access point I could get online with (damned hotel didn't provide Internet access). I heard two AP's, and caught a couple IP's going by. I assigned myself an IP which appeared to not be used, and fired up ethereal.
I saw text for several of the casino machines going by. It was the text to be updated to the displays, including windows paths to where the files originated from (I believe). It was all in plain text. I noted down what I saw for a few minutes, shut down the laptop, and proceeded to lose for the rest of the night in the casino. Hey, that's what Vegas is for, right?
After I got home, I dug around for something resembling an admin contact at the casino, and advised him of what I saw. It would have probably been pretty easy to push my own updates to the machines. What would I say though?
"Gambing is an addiction, quit now."
"This game is rigged, move on."
"This is the droid you are looking for."
"With a 97% chance of losing, did you really want to play this game?"
or, I guess
"I'm a spiffy keen elite haxor type person, props to my homeyz" haha
A lot of our old old employees were really stupid, and would fill out those "put your email for free stuff" links back in the days. Spam people trade lists all the time, so once you start getting spam, it's hopeless for it to stop. The occasional one will remove you from their list, but most of them continue to trade forever.
My box itself, I write to a lot of people. Lots of people write to us with various problems, and some of them are in the spam business, or share with someone that is. One of my addresses that's still in use has been used for many years, so I get plenty there. That's most of what gets filtered into the spam box now. Some of my alises got hit really hard, so I simply phased them out over time.
I guess it's all in the business you're in. Normal folks probably don't get as much spam as we do.
I feel really sorry for Hotmail, even if they are a M$ company. People use Hotmail addresses as throw-away accounts, so they must get absolutely hammered with spams, even if the accounts they're being sent to are undeliverable.
$ du -sh *
:)
92M SPAM
And that's just one month. I killed my accounts that were getting the majority of it, so this is maybe 20% of what I was getting before.
Our server overall is receiving 21,000 spams and 17,000 viruses per day. I can imagine what larger services do. A 1Gb limit is insane. But hey, if they have the budget for it, who's to say "no".
Of course, if Microsoft was to launch the same thing, everyone would be bashing it.
I was trying to make first post.
I'm still waiting for a good
I'm a robotic underwater lumberjack and I'm okay,
...he's a robotic underwater lumberjack and he's OKAAAAAAAAAAYYY.
I sleep all night and I work all day.
Chorus: He's a robotic underwater lumberjack and he's okay,
He sleeps all night and he works all day.
I cut down trees, I eat my lunch,
I go to the lavatory.
On Wednesdays I go shopping
And have buttered scones for tea.
Fish: He cuts down trees, he eats his lunch,
He goes to the lavatory.
On Wednesdays he goes shopping
And has buttered scones for tea.
Chorus: He's a robotic underwater lumberjack and he's okay,
He sleeps all night and he works all day.
I cut down trees, I skip and jump,
I like to press wild flowers.
I put on women's clothing,
And hang around in bars.
Fish: He cuts down trees, he skips and jumps,
He likes to press wild flowers.
He puts on women's clothing,
And hangs around in bars?
Chorus: He's a robotic underwater lumberjack and he's okay,
He sleeps all night and he works all day.
I cut down trees, I wear high heels,
Suspendies and a bra.
I wish I'd been a girlie,
Just like my dear pappa.
Fish: He cuts down trees, he wears high heels?
Suspendies...and a bra?
Chorus:...he's a robotic underwater lumberjack and he's okay,
He sleeps all night and he works all day.
He sleeps all night and he works all day.
My girlfriend put a red sheet on our futon couch, 'cause everything I owned was black. I was happy with black. The TV, Stereo (and assorted equipment) matched the couch and chairs. Well, the TiVo stands out. One fuckin' silver box with all my nice black stuff. That's form and function, right?
She still doesn't agree with the idea that the grey candle holders were enough variety. Or the black&white photo of a rose was art.
You got exactly why I quoted it. :)
If I could remember the name of the company, I would have listed it.
The really bad part is that they were running security tests against some of our servers for one of the credit card companies, to prove that we were secure. The credit card companies get a wild hair up their @ss occasionally, and want to know servers handling any sort of personal information are secure. They were sending me a self-evaluation, which was a single PDF in the self extracting zip. They were getting upset that I was telling them that I never received it. Well, I hadn't. Even if I had turned on notification of received viruses, I would have ditched it. For a while the server quarantined viruses, but it ended up wasting a whole lot of space, so I stopped even doing that.
The whole matter was being relayed through the bank handling the merchant account. Aparently that was one of the last mistakes they made, before they got a new company to handle their testing. They sent me PDF's, and had a much more professional report.
Both of them were still stupid about their reports. They'd cite every possible Apache exploit, regardless of platform. They were citing all kinds of Win32 and OS/2 exploits, and couldn't quite comprehend that we weren't using either. I think they just run `nmap -sV -O [hostname]`, and have it cross reference a database of exploits collected from BugTraq.
Providers should do this now. Email worms are such a hazard, and if everyone with a mail server took a few minutes to protect their users, these worms wouldn't be a problem.
We use MailScanner (from http://mailscanner.info) to protect our server. It's free, so this isn't an ad or something. It can use a whole variety of virus scanners. We update our dat files twice daily, to make sure the largest window of opportunity for these worms is only 12 hours. We also disallow all executable attachments, plus all kinds of extra filtering. MailScanner is very configurable, so you can have it block the virus and silently reject the message (that's what we do), or notify the users. Our users got tired of getting the reports, so I stopped sending them.
There's only one instance I can think of where a real message was blocked, but it was someone at a "internet security" company sending me an executable. They asked if I received it, and when they said it was a self-extracting archive (some_form.exe), I was like "Are you stupid? How do *I* know your machine isn't infected with something." Of course, I'm on a Linux box, so I'm not really worried about their silly Windows viruses, but it was the principle.