> How long until the laws of (current) economics > catch up with Google, and they can no longer > afford to do the right thing?
It could be quite a while. Google is profitable, and the click-through rate on the ads that you *CAN* purchase from them (clearly demarcated as ads) is phenomenal. They're doing fine.
> Does anyone have any insight into Google's > money situation? Where the money comes from?
Google stays profitable by aggressively negotiating bandwidth from several suppliers. The guy who runs the network there is a former coworker of mine. In fact, I'm logged into his computer right now:-).
> Are they are taking losses on traffic? Could > they economically handle disillutioned surgers > from all the other search engines?
See above. In short, yes, but this depends on the economic climate and the willingness of the networks to play ball.
> Or is it just that the other search engines > will do anything for a buck?
IMHO, yes.
Realistically, when was the last time someone asked you to Yahoo! or Altavista their next blind date? Google is a societal totem and if they fell prey to financial weakness, they would be snapped up immediately. John Doerr, Larry Page, and Sergey Brin have not allowed that to happen to their creation. I salute them, and all of my friends and coworkers who went to work for them. It is a great product and makes its own markets.
Your actions represent the worst of the Slashdot community. The poster expressed a valid, provocative opinion, and the best you can do is try to suppress it?
If you have no counter-argument, you could at least leave the post visible enough for someone who does. Microsoft may be bad for Linux, Wal-Mart is bad for inefficient mom-and-pop shops and perhaps the diversification of our economy and culture. Is that better or worse? Your moderation does not adequately answer this question.
I sincerely hope I'll see your action in metamoderation!
I've done MySQL, PostgreSQL, and Oracle DBA on live sites which were heavily beaten on and had, as customers at various times, Olivetti Inc., CNN, CSPAN, NBC, Fox, Reuters, Bloomberg, Business Week, NOAA, and the Department of Defense. Currently I run PostgreSQL for everything except Snort logging, because
0) I can tune it in about 5 minutes a day, from cron
1) it's cheaper than Oracle, with full transactions, and
2) I can only get Windows binaries of Snort that use MySQL. (eg. the one thing I use MySQL for)
I would not consider MySQL for serious applications because it doesn't do subselects (AFAIK) and its transaction support is fairly new (again AFAIK). Our applications DO need full transaction support for some applications that need versioned updating which only increments on success (hard to explain without showing you the code, which I can't under my NDA.)
I use MySQL for Snort because Roman says that its performance for ACID is the best, and my site seems to bear that out. MySQL 3.2x crumbled in production on a site I managed, and that soured me on the product.
Oracle is a BEAST. Properly tuned, it can blow the doors off of competitors, and OracleTool (http://www.oracletool.com/ is a terrific, free tool. But licensing costs for Oracle make it a tool that is best used in places where it's been in-place for years. I wouldn't adopt it at a new company. Since that's exactly what I work for these days, we use Postgres.
I doubt Wal-Mart will switch from Oracle (or DB2, or Sybase, or SQL Server) if they have a running installation of it. Most likely, it's on Sun if it's an Oracle installation; Blockbuster has a couple of E10K's (or did, last year, at Exodus Sunnyvale) for what appears to be their inventory control and billing system. If it ain't broke, why 'fix' it?
If you need replication, there's always RServ for Postgresql, right now. The number of sites which actually need multimaster replication is not great, and those that do seem to be running DB2 on a Parallel Sysplex or Oracle HA on a SunCluster, in my experience.
Red Hat already supports PostgreSQL. That version is called 'Red Hat Database'. I run Red Hat because I find ext3 to be a useful innovation, and I compile custom kernels; in my personal experience, the individual kernel maintainers for a given functionality (eg. LVM) can be hired for consultation or reached through mailing lists at a similar time-and-opportunity cost to what I'd encounter with a support contract from Red Hat.
Then again, I've been doing this for a while, and am probably not a 'representative' engineer... I've seen some pretty scary loads in my day, not what your average mope encounters, and my customers do not accept downtime.
We use Win2K servers where I work, too, because sometimes that's what is necessary to get the job done. I'd suggest that Wal-Mart's philosophy is closer to mine than to the average open-source hippie's, but with thousands of employees, you have to figure that many of them will be most familiar with Windows, and there is a lot of lock-in incentives for NT/2000/XP on the server side. It wouldn't make sense to expend effort and/or money to chase some mythical Linux 'savings' in many of the applications they find to be 'core'.
DDR ram, wow how about that. 500Mhz 64-bit processor, mmmhmm, how long has the Alpha been out now? And the Power3? Cheezy graphics advances, check...
Ok, ILM and every other CG house on earth is going Linux, maybe some people use Maya and Alias on SGI but god knows, it'll be a cold day in hell when I'd fork out for an SGI as a business expense looking at these specs. And it looks like the big moneymaking CG houses feel the same way.
Nice knowing you, SGI. It was fun while it lasted.
Eh? Any application that uses fork() will benefit.
Multithreading offers finer-grained parallelism, to be sure, but I guaran-damn-tee you that Apache will run almost twice as fast (all other conditions unbounded) and handle about twice the number of simultaneous clients on a two-processor Linux or Solaris system.
And Apache is NOT multithreaded (well, 1.x isn't, and 2.x is not what I would want to run on my production servers yet).
Similarly, my gnarly Perl and shell scripts that do lots of simultaneous-dispatch work benefit enormously from a second processor. Again this is in the absence of other bounding conditions, ie. network pooping out, etc.
Single-process single-thread applications probably won't benefit much. I don't know if Photoshop is multithreaded, but that's probably the only application that most high-end Mac users care about anyways;-).
They're not even making any money off the site AFAIK, unlike some sites that don't work (airline sites mostly) without IE5.5 and a lot of good luck.
IMHO it could be a lot worse, as well as a lot better. Usability nuts seem to forget how businesses actually work (which is to say, barely, on most days).
I run Linux full-time at home on my laptop, and use Windows full-time at work (mostly because Windows Media doesn't run natively in Linux, and Real is not representative under Linux of how it runs in Windows -- and our streaming media clients are the biggest source of support calls). Normally I just expect incompetent web design. By my standards, the SLOC website is not half bad, just wickedly slow.
square != quadrilateral for all values of quadrilateral.
So while you can take your UML diagrams, feed them to a program like Dia, and have it spit out XML representations of the semantic information in them, you will probably experience much less success feeding the resultant XML code directly to someone familiar with UML in their designing process.
More seriously, there are lots of languages derived from Latin. By your logic, we should all learn Latin. I did that. I still had to learn Spanish, German, and Italian in order to converse with speakers of those languages. And in fact, XML isn't even as useful as Latin;-)... (similarly, my favorite screwdriver doubles as a hammer and crowbar, but I own and use the latter tools anyways, because they do a better job)
If you are planning to represent state machines, object interactions, and other design idioms in a standardized form, UML is the current lingua franca. XML is a useful tool for communicating the semantics of (say) a UML diagram, an interlinked graph of information resources, or purchase orders. Is it a substitute for UML? Well, you try ordering a burrito in Latin in East LA and tell me how far you get.
But I typically poke around at least a little bit in any application I run to see if it's doing what it says. That's also why I run Microsoft products behind restrictive ACLs -- I can't see what they're trying to pull by looking at the code, so I am forced to explicitly restrict those bastards.
What can I say, I like to make sure my installation works before I hang my job security on it. I'm astounded that more people don't. Then again, I worked with grumpy old bastards like you and discovered that they were the ones whose installations stayed up and didn't get hacked. Must have made an impression...
At IBM, long before the Linux jihad started, I was told to use free software but audit the code and license first. That's what I've been doing ever since, although I don't work at IBM anymore, and haven't for years.
I write Perl too. I write Perl professionally and I do a professional job of it -- my code is organized into object-oriented modules that could pass for Java if it wasn't for the pointies and easier string manipulation. The documentation is Javadoc-style POD and it's kept up to date. Same thing goes for my PHP code, my C code, etc... I don't write Java code professionally anymore but that's where I picked up some of my documentation and OO habits, since I learned C++ as a scientific (computational) language. Anyways, discipline will see you through if you want it to. It's always more productive to just design and write the damn code than to hold meetings over which language to do it in.
However, Perl is really a language that I find more suited to "glue" than use on a continuing basis by teams of programmers. The vast majority of Perl (and a LOT of C++ for that matter) code I have seen used by teams was phased out MUCH faster than the equivalent PHP+C, Java, or Python code. Now with Inline.pm perhaps the horrible problems of using XS or Swig for wrapping "real" code are gone, but the fact remains, without a LOT of discipline and some bitter years of experience, Perl offers more chances for a programmer to take shortcuts. Again, great for quick glue jobs, but bad for long-lived projects. For what it's worth, the C++ code that got shitcanned was big on templates, gratuitous operator overloading, and all the other Perl-like shit that I hate in C++... if there's one good thing I can say about Java it's that it isn't C++.
But I *hate* the current Java platform with a seething passion. I don't really like C, and I can't hire a bunch of Common Lisp or Python programmers off the street. Ada? Probably, but who wants to deal with DoD curmudgeons? So every approach that I can see is a compromise in some respect or another. All the "good" ways to write code have their problems. Perl's problem is that it offers lazy programmers (oops, that was redundant) many, many ways to churn out spaghetti that "works"... for a while.
As un-fond as I am of fuddy-duddy computer scientists, their focus on formal correctness, proofs, and mathematical solutions to problems is really where it's at when you are solving a problem that hasn't been attacked before. If you're just re-inventing the wheel, Perl (or any other language) is fine, but if you need to parallelize development of some fast, bug-free innards code, it isn't what you want to release in.
I use Perl on a daily basis for configuration, scripts, etc. because it's the best tool for those jobs. Right now I am the only programmer on my team. When that changes again, I'm going to have to go back to a more easily maintained solution, which looks like it'll be PHP+C for most of my tasks. (did I mention that I hate Java with a seething passion? Too much work for too little payoff) This isn't too tough since PHP and Perl have equivalent means of using extension classes, etc. and PHP is actually easier than Perl to integrate directly with C (across-the-board, not just using Inline.pm, which I admit rules). Work with a large team doing development with some custom extension code, etc. and I'll bet dollars to donuts that you, too, will one day shrug your shoulders and admit that Perl is not right for all types of work. It's also a bit baroque and the syntax can be frustrating for people who aren't from a Unix background. I started out sharing your perspective years ago because Perl is SO very wonderful when you're working all by yourself. But it isn't so great when you're with a team. Especially if you are working on code that is inter-dependent with other programmers, it's important to be using the same or similar approaches to solving problems, and this is anathema to the TMTOWTDI credo at the heart of Perl. I'm not going to hedge on this claim -- I, too, make a LOT of money writing Perl and other code, I've done so for years, I appreciate the CPAN and Perl community, and I try to give back -- but I refuse to be a simp and just crow about how great Perl is for EVERY situation, when I have seen so much evidence to the contrary.
(It is still the greatest string manipulation and glue language known to mankind, though. And many of the one-offs I wrote in Perl have made my company tens of thousands of dollars. So the fundamental truth, "Use the right tool for the job" still holds!)
My girlfriend and I both use Red Hat 7.2 on my laptop. This is because it's easier to get all the networking set up (802.11b, DSL using PPPoE, IrDA) under Linux than under Windows, since the drivers are terribly buggy under the latter. She understands that sometimes web pages won't behave perfectly because the average HTML writer does not understand that people use platforms aside from Microsoft Windows, but since this is ostensibly my "work" machine, she's okay with that. Moreover, everything "just works" and when it doesn't I can log in remotely to fix it.
Contrast this to when I was just getting started... I expected people to know things or at least care why the computer acted the way it did.
Boy, was that a crock of shit!
I have nontechnical users merrily sending mail from Mutt and Pine on OpenBSD now because I simply give them a set of directions, say "It's not perfect, but it's a compromise, and in 3 years we've never been hacked; please play along nicely". Since my users all accomplish what they want to, they are happy, and since they're happy, I have more time to twiddle RAIDframe, play with Coda and Heartbeat, and generally nerd out.
The more experience I get, the less experience I expect my users to have, and the happier they are overall. Next week the marketing guy will be switching over to using RSA keys for SSH access from his cellular modem. I'm not kidding.
It doesn't have to be intimidating or nerdy to do the job right!
But if the credentialing scheme in place depends on Windows frontend servers being secure, you can damn well better bet that it will be dutifully serving up data to the wrong party.
Can't do much about that. I don't perform ANY core business functions on Microsoft server software, their history of getting brutally hacked and denying it is far too pervasive. (Yes, Sun and IBM are terrible too. Frankly, Red Hat and the OpenBSD Project are valuable to me not because they're "perfect", but because they're honest and prompt when they fuck up! I cut both organizations a new check every 6 months of my own free will, NOT because they try and force my company to. The checks come out of my after-tax salary; as far as I know the company has never paid a dime for either project's media.)
The consultants were probably lazy too, but don't get too overzealous to defend the most probable point of entry. I am somewhat less than surprised that a large gov't agency would screw up like this, although most of the dep'ts I work with at least have the sense to retain solid IT security consultants (I've met some very competent Lockheed employees, for example; I have no idea who was at fault in this incident).
moby_apache setup (php + perl + ssl)
on
Future Of IDS
·
· Score: 2
Lately I've been compiling Apache with just PHP, mod_ssl, and mod_auth_pgsql (no mod_perl anymore). Last time I did it was not TOO hard, though. You need to apply the SSL patches FIRST, THEN apply the mod_perl patches, and LAST add PHP.
At least, that worked for Apache 1.3.12 on FreeBSD 2.2. Like I said, it's been a while since I needed to prototype an Apache module (sooner or later it's best to move them all to C, IMHO).
I monitor 2 DS-3's, that's all I need to...
on
Future Of IDS
·
· Score: 4, Informative
I can't speak to higher-end solutions, because as I mentioned in my response, I suspect they'll already have an architecture in place (eg. when I was at IBM Burlington, before Snort was even born, the setup they had created for monitoring ingress and egress traffic was far beyond what I've seen before or since).
But for my live production hosts, dual-homed on UUNet and Qwest, and all monitored, Snort + Barnyard + ACID have kept up without clipping traffic or interfering with operations. And yes, we DO saturate both of those links on occasion (though not always).
That's all I can speak to. When I worked at XOOM we saw traffic up to about 0.75Gbps steady and never bothered running an IDS, just were real fucking careful about what went live and keeping everything audited. An HP OpenView installation with some sort of IDS support was looking like $300K in bills. We said "fuck that" and to this day I wouldn't do any differently.
But, my situation may be very different from yours. If you need a $20K solution and its presence saves you $40K, you sure as hell don't need my blessing to buy it!
CEO's like $$$
on
Future Of IDS
·
· Score: 4, Interesting
That made it pretty damn easy for me to push Snort where I work.
Only choads that are getting kickbacks from manufacturers are going to push for overpriced commercial solutions in shops that don't have an existing IDS installation or a compelling reason to use the packaged solutions (NetRanger, OpenView, their ilk).
A packet is a packet... NFR and Snort are both designed by well-respected engineers who are more interested in accuracy and correctness than in unit shifting. I trust them for that.
When you get right down to it, unless you're rolling in dough, why blow $20,000 per management station plus consulting costs to implement something your network administrator can probably set up in a week for free? (I know I can) It's stupid. Save the cash for your coke dealer or a rock for the missus.
Management console for Snort, take 2....
on
Future Of IDS
·
· Score: 3, Informative
Note: I fucked this up the first time by posting it as 'Extrans' when I meant to use HTML formatting. D'oh. Anyways, I've got karma to burn, so here's another whack at it...
See my earlier comment about ACID. Multisensor correlation and alert grouping, emailing of packet traces to offenders or CIO's, pretty much all you could ask for.
Try it. ACID homepage You may be pleasantly surprised at how easy Snort is to scale up. I have numerous sensors, all in production, all logging on all interfaces, all the time, and haven't had any major incidents on my subnet. I credit this partly to having early warning of when some idiot tries to attack my boxen, as well as to using Thoth for host monitoring, which makes it trivial to check that all my daemons are up-to-date, and all kernel patches are installed.
Someone pisses me off consistently, they get blackholed. This is something I'd recommend doing by hand, of course, but for people whose business I don't need or want, it's a great way to end the problem right then and there.:-)
Would somebody please mod this guy up?
on
Future Of IDS
·
· Score: 2
Both of the projects he's mentioning are brilliant. They encourage people to disclose alerts so that the numbers can be leveraged en masse to get vendors and ISPs off their asses.
This is the flipside to things like predictive agents and automated vulnerability testers. It improves security by social mechanisms. I'm going to look into using ARIS today, and if I can figure out how myNetWatchman works, I'll consider it too.
I hope other readers are forming solid opinions about where IDSes ought to be headed by reading posts to this article. It has been informative for me, at least. (and I've been around this stuff for a while... sometimes I see packet headers in my sleep:-))
a great management console for Snort...
on
Future Of IDS
·
· Score: 1, Redundant
See my earlier comment about ACID. Multisensor correlation and alert grouping, emailing of packet traces to offenders or CIO's, pretty much all you could ask for.
<p>
Try it. <a href="http://acidlab.sourceforge.net/">ACID homepage</a> You may be pleasantly surprised at how easy Snort is to scale up. I have numerous sensors, all in production, all logging on all interfaces, all the time, and haven't had any major incidents on my subnet. I credit this partly to having early warning of when some idiot tries to attack my boxen, as well as to using <a href="http://firedrake.org/thothproject/">Thoth</a > for host monitoring, which makes it trivial to check that all my daemons are up-to-date, and all kernel patches are installed.
<p>
Someone pisses me off consistently, they get blackholed. This is something I'd recommend doing by hand, of course, but for people whose business I don't need or want, it's a great way to end the problem right then and there.:-)
<p>
ACID and Barnyard for Snort users -- great stuff!
on
Future Of IDS
·
· Score: 5, Informative
I would have thought SecurityFocus could handle a/.'ing,
but I guess not. It's a shame since they are one of the good, unbiased sources for security info out there.
Anyways, I want to throw in a shill for ACID for anyone who runs Snort. It makes my job SO INCREDIBLY MUCH EASIER that, well, I bother to do it every day, maybe two or three times a day, and haven't had any major incidents to speak of. If you run Snort, you ought to log to a centralized database that can handle the traffic from all your sensors, and then grind through it with ACID for starters. Yes, you should keep a packet vault; yes, you should run Nessus; yes, you still need to use TripWire or Integrit for filesystems. But having a friendly, capable frontend to Snort sensors is a HUGE help.
If you're running a lot of sensors and they get a ton of attacks in production, you should also look into the Barnyard plugin for Snort. It's nice for keeping things from slowing down.
If I were to take a stab at what would MOST help IDS and ISS research in the near future, I'd guess at the integration of tools like Nessus and Snort with a predictive intelligent agent like Intravenous or similar. I wish I could comment intelligently on the article, but mostly I wanted people using Snort to be aware of HOW helpful the ACID frontend is, so that more people use it, and I have less subnets to blackhole;-).
Less cool at $3000
on
This is IT?
·
· Score: 5, Insightful
Being a bicyclist, I am partial to light, fast, cheap transportation. The Segway appears to be none of these. It is expensive, a brute-force solution to a non-problem IMHO. That's why I, at least, am underwhelmed.
Then again, I dislocated my shoulder last week on my bicycle while avoiding traffic. Maybe I can ride it again tomorrow, maybe not, but it has been quite painful and made it much harder to run errands around town (take the time to run an errand on a bike and double it; you've just arrived at the time to complete it, driving, if it's in downtown DC and you have to park). This device would make such injuries irrelevant. I'm sure it would be wonderful for elderly or infirm people who can't drive. So perhaps I am an "able-ist" in that I am biased to think about things as if I'll always be hale and healthy.
If the product is made affordable, it would be a lot nicer and less intrusive than a Lark or a Rascal for sure. But I don't see it as being quite as revolutionary as the car, simply because it does not radically increase carrying capacity, doesn't really offer commercially compelling advantages over a regular scooter, pair of feet, or a bicycle to balance out the cost... I don't see how this device would change the world for the average mope, but for some people it sounds like a godsend.
Attenuate your expectations, as this Dean Kamen seems to be telling us, and in context it is pretty neat. Not earth-shattering, but pretty neat, alright.
Actually, we're in DC, and I work on 3 of them. They're incredulous that things can actually get done, and I plan to keep them that way! More are lined up because of this.
maybe your arguments would make more sense. But I can't really be bothered to futz with the internals of Postgres or Apache or qmail or [...] every time I need to change something.
And the other employees of the company want to change things, our clients want things changed, no spec is ever firm, etc... eventually, at least in the line of work I'm doing, it dawns on just about everyone that the "logic" and the "presentation" are not only separate, but often so totally unrelated that one or two hooks will suffice. That's where my break happens.
The whole "MVC" idea says the same thing, in so many words. I'm just using the best tool for the job, and I've done this sooooooo many times, our business is soooooooo specialized, and our clients expect things sooooo fast (cause we always turn things out pretty fast) that, rather than being a conscious decision to eschew CMM or OOP or whatever the PHB buzzword of the week is, I simply got the damn job done, producing some reusable libraries along the way.
It's also worth noting that all of this *COULD* be in C/C++ or Perl alone, but I don't feel that Perl is good for anything other than glue, and I don't want to dive into a shit load of C code every time a client requests a change.
Anyways, that's just my job experience. Yours may be totally different, and probably is, if you write device drivers or microcode for a living, work on huge integrated apps, and so on. Then I'd probably want to stay in one language with *maybe* a scripting language for unit tests and prototyping interfaces.
In my opinion, the single biggest hurdle is acceptance by the client. If it's too late, or too buggy, or too rigid, no one will care whether it's in Java or PHP or Perl or Visual Fortran...
I write code on-demand because that's what I get paid to do. I've tried being super-neat and uni-language, sloppy as hell and multi-language, and have settled on the processes that work best for me to keep our clients happy.
I don't pretend to offer a solution for everyone in every situation, but in a small company with high-profile clients, results are the only things that matter. I find that using a fast, compiled "core" with flexible, highly-documented "glue" keeps me from having to work too much overtime, and allows other developers to get up to speed quickly.
Documentation is KEY to my strategy. Without it, I don't think any of my other comments are relevant. I'm not an OO fascist but it sure does make documenting and segmenting code easier, for run-of-the-mill business and controller apps.
I'd go insane if I tried to use a compiled language for frequently-changing applications (eg. web interfaces to purchasing systems, database large object manipulation & indexing, etc.) but likewise I'd grow old waiting for things to happen if the cores weren't in C.
Personally I dislike C++ because I find it harder to port than ANSI C. But I wrote plenty of it when I was learning how to program (before I learned Scheme and Perl and Java and got some latitude in my paradigms). Either C or C++ is great for speed and hooking into the OS or an Apache module or whatever. It's also less tempting for admins or rookies to mess with C code because bad code may not compile.
On the other hand, object-oriented (or at least modular) PHP and Perl code, and decently-written Java code, is much easier to adapt to changing demands. I stick with PHP and Perl, myself, and I use Perl as kind of a glue-core layer between C applications and PHP interfaces. If I had more time I'd probably write the hooks as PHP extensions, but Perl is just so damn powerful when you use it right.
I'm not sure what people are thinking when they specify that everything on a project "needs" to be done in Java or whatever. It's not particularly hard to use my code, since it's all just calls to libraries that automagically do the hard stuff. More importantly, I tend to use POD or Javadoc-style comments in everything, and don't put it anywhere else. That way I am forced to keep it all up to date, because that's where *I* remind myself what arguments to feed what methods!
Slashdot Mirror
> How long until the laws of (current) economics
:-).
> catch up with Google, and they can no longer
> afford to do the right thing?
It could be quite a while. Google is profitable, and the click-through rate on the ads that you *CAN* purchase from them (clearly demarcated as ads) is phenomenal. They're doing fine.
> Does anyone have any insight into Google's
> money situation? Where the money comes from?
Google stays profitable by aggressively negotiating bandwidth from several suppliers. The guy who runs the network there is a former coworker of mine. In fact, I'm logged into his computer right now
> Are they are taking losses on traffic? Could
> they economically handle disillutioned surgers
> from all the other search engines?
See above. In short, yes, but this depends on the economic climate and the willingness of the networks to play ball.
> Or is it just that the other search engines
> will do anything for a buck?
IMHO, yes.
Realistically, when was the last time someone asked you to Yahoo! or Altavista their next blind date? Google is a societal totem and if they fell prey to financial weakness, they would be snapped up immediately. John Doerr, Larry Page, and Sergey Brin have not allowed that to happen to their creation. I salute them, and all of my friends and coworkers who went to work for them. It is a great product and makes its own markets.
Your actions represent the worst of the Slashdot community. The poster expressed a valid, provocative opinion, and the best you can do is try to suppress it?
If you have no counter-argument, you could at least leave the post visible enough for someone who does. Microsoft may be bad for Linux, Wal-Mart is bad for inefficient mom-and-pop shops and perhaps the diversification of our economy and culture. Is that better or worse? Your moderation does not adequately answer this question.
I sincerely hope I'll see your action in metamoderation!
0) I can tune it in about 5 minutes a day, from cron
1) it's cheaper than Oracle, with full transactions, and
2) I can only get Windows binaries of Snort that use MySQL. (eg. the one thing I use MySQL for)
I would not consider MySQL for serious applications because it doesn't do subselects (AFAIK) and its transaction support is fairly new (again AFAIK). Our applications DO need full transaction support for some applications that need versioned updating which only increments on success (hard to explain without showing you the code, which I can't under my NDA.)
I use MySQL for Snort because Roman says that its performance for ACID is the best, and my site seems to bear that out. MySQL 3.2x crumbled in production on a site I managed, and that soured me on the product.
Oracle is a BEAST. Properly tuned, it can blow the doors off of competitors, and OracleTool (http://www.oracletool.com/ is a terrific, free tool. But licensing costs for Oracle make it a tool that is best used in places where it's been in-place for years. I wouldn't adopt it at a new company. Since that's exactly what I work for these days, we use Postgres.
I doubt Wal-Mart will switch from Oracle (or DB2, or Sybase, or SQL Server) if they have a running installation of it. Most likely, it's on Sun if it's an Oracle installation; Blockbuster has a couple of E10K's (or did, last year, at Exodus Sunnyvale) for what appears to be their inventory control and billing system. If it ain't broke, why 'fix' it?
If you need replication, there's always RServ for Postgresql, right now. The number of sites which actually need multimaster replication is not great, and those that do seem to be running DB2 on a Parallel Sysplex or Oracle HA on a SunCluster, in my experience.
Red Hat already supports PostgreSQL. That version is called 'Red Hat Database'. I run Red Hat because I find ext3 to be a useful innovation, and I compile custom kernels; in my personal experience, the individual kernel maintainers for a given functionality (eg. LVM) can be hired for consultation or reached through mailing lists at a similar time-and-opportunity cost to what I'd encounter with a support contract from Red Hat.
Then again, I've been doing this for a while, and am probably not a 'representative' engineer... I've seen some pretty scary loads in my day, not what your average mope encounters, and my customers do not accept downtime.
We use Win2K servers where I work, too, because sometimes that's what is necessary to get the job done. I'd suggest that Wal-Mart's philosophy is closer to mine than to the average open-source hippie's, but with thousands of employees, you have to figure that many of them will be most familiar with Windows, and there is a lot of lock-in incentives for NT/2000/XP on the server side. It wouldn't make sense to expend effort and/or money to chase some mythical Linux 'savings' in many of the applications they find to be 'core'.
As always, YMMV.
DDR ram, wow how about that. 500Mhz 64-bit processor, mmmhmm, how long has the Alpha been out now? And the Power3? Cheezy graphics advances, check...
Ok, ILM and every other CG house on earth is going Linux, maybe some people use Maya and Alias on SGI but god knows, it'll be a cold day in hell when I'd fork out for an SGI as a business expense looking at these specs. And it looks like the big moneymaking CG houses feel the same way.
Nice knowing you, SGI. It was fun while it lasted.
Eh? Any application that uses fork() will benefit.
;-).
Multithreading offers finer-grained parallelism, to be sure, but I guaran-damn-tee you that Apache will run almost twice as fast (all other conditions unbounded) and handle about twice the number of simultaneous clients on a two-processor Linux or Solaris system.
And Apache is NOT multithreaded (well, 1.x isn't, and 2.x is not what I would want to run on my production servers yet).
Similarly, my gnarly Perl and shell scripts that do lots of simultaneous-dispatch work benefit enormously from a second processor. Again this is in the absence of other bounding conditions, ie. network pooping out, etc.
Single-process single-thread applications probably won't benefit much. I don't know if Photoshop is multithreaded, but that's probably the only application that most high-end Mac users care about anyways
They're not even making any money off the site AFAIK, unlike some sites that don't work (airline sites mostly) without IE5.5 and a lot of good luck.
IMHO it could be a lot worse, as well as a lot better. Usability nuts seem to forget how businesses actually work (which is to say, barely, on most days).
I run Linux full-time at home on my laptop, and use Windows full-time at work (mostly because Windows Media doesn't run natively in Linux, and Real is not representative under Linux of how it runs in Windows -- and our streaming media clients are the biggest source of support calls). Normally I just expect incompetent web design. By my standards, the SLOC website is not half bad, just wickedly slow.
YMMV...
square != quadrilateral for all values of quadrilateral.
;-)... (similarly, my favorite screwdriver doubles as a hammer and crowbar, but I own and use the latter tools anyways, because they do a better job)
So while you can take your UML diagrams, feed them to a program like Dia, and have it spit out XML representations of the semantic information in them, you will probably experience much less success feeding the resultant XML code directly to someone familiar with UML in their designing process.
More seriously, there are lots of languages derived from Latin. By your logic, we should all learn Latin. I did that. I still had to learn Spanish, German, and Italian in order to converse with speakers of those languages. And in fact, XML isn't even as useful as Latin
If you are planning to represent state machines, object interactions, and other design idioms in a standardized form, UML is the current lingua franca. XML is a useful tool for communicating the semantics of (say) a UML diagram, an interlinked graph of information resources, or purchase orders. Is it a substitute for UML? Well, you try ordering a burrito in Latin in East LA and tell me how far you get.
But I typically poke around at least a little bit in any application I run to see if it's doing what it says. That's also why I run Microsoft products behind restrictive ACLs -- I can't see what they're trying to pull by looking at the code, so I am forced to explicitly restrict those bastards.
What can I say, I like to make sure my installation works before I hang my job security on it. I'm astounded that more people don't. Then again, I worked with grumpy old bastards like you and discovered that they were the ones whose installations stayed up and didn't get hacked. Must have made an impression...
At IBM, long before the Linux jihad started, I was told to use free software but audit the code and license first. That's what I've been doing ever since, although I don't work at IBM anymore, and haven't for years.
I write Perl too. I write Perl professionally and I do a professional job of it -- my code is organized into object-oriented modules that could pass for Java if it wasn't for the pointies and easier string manipulation. The documentation is Javadoc-style POD and it's kept up to date. Same thing goes for my PHP code, my C code, etc... I don't write Java code professionally anymore but that's where I picked up some of my documentation and OO habits, since I learned C++ as a scientific (computational) language. Anyways, discipline will see you through if you want it to. It's always more productive to just design and write the damn code than to hold meetings over which language to do it in.
However, Perl is really a language that I find more suited to "glue" than use on a continuing basis by teams of programmers. The vast majority of Perl (and a LOT of C++ for that matter) code I have seen used by teams was phased out MUCH faster than the equivalent PHP+C, Java, or Python code. Now with Inline.pm perhaps the horrible problems of using XS or Swig for wrapping "real" code are gone, but the fact remains, without a LOT of discipline and some bitter years of experience, Perl offers more chances for a programmer to take shortcuts. Again, great for quick glue jobs, but bad for long-lived projects. For what it's worth, the C++ code that got shitcanned was big on templates, gratuitous operator overloading, and all the other Perl-like shit that I hate in C++... if there's one good thing I can say about Java it's that it isn't C++.
But I *hate* the current Java platform with a seething passion. I don't really like C, and I can't hire a bunch of Common Lisp or Python programmers off the street. Ada? Probably, but who wants to deal with DoD curmudgeons? So every approach that I can see is a compromise in some respect or another. All the "good" ways to write code have their problems. Perl's problem is that it offers lazy programmers (oops, that was redundant) many, many ways to churn out spaghetti that "works"... for a while.
As un-fond as I am of fuddy-duddy computer scientists, their focus on formal correctness, proofs, and mathematical solutions to problems is really where it's at when you are solving a problem that hasn't been attacked before. If you're just re-inventing the wheel, Perl (or any other language) is fine, but if you need to parallelize development of some fast, bug-free innards code, it isn't what you want to release in.
I use Perl on a daily basis for configuration, scripts, etc. because it's the best tool for those jobs. Right now I am the only programmer on my team. When that changes again, I'm going to have to go back to a more easily maintained solution, which looks like it'll be PHP+C for most of my tasks. (did I mention that I hate Java with a seething passion? Too much work for too little payoff) This isn't too tough since PHP and Perl have equivalent means of using extension classes, etc. and PHP is actually easier than Perl to integrate directly with C (across-the-board, not just using Inline.pm, which I admit rules). Work with a large team doing development with some custom extension code, etc. and I'll bet dollars to donuts that you, too, will one day shrug your shoulders and admit that Perl is not right for all types of work. It's also a bit baroque and the syntax can be frustrating for people who aren't from a Unix background. I started out sharing your perspective years ago because Perl is SO very wonderful when you're working all by yourself. But it isn't so great when you're with a team. Especially if you are working on code that is inter-dependent with other programmers, it's important to be using the same or similar approaches to solving problems, and this is anathema to the TMTOWTDI credo at the heart of Perl. I'm not going to hedge on this claim -- I, too, make a LOT of money writing Perl and other code, I've done so for years, I appreciate the CPAN and Perl community, and I try to give back -- but I refuse to be a simp and just crow about how great Perl is for EVERY situation, when I have seen so much evidence to the contrary.
(It is still the greatest string manipulation and glue language known to mankind, though. And many of the one-offs I wrote in Perl have made my company tens of thousands of dollars. So the fundamental truth, "Use the right tool for the job" still holds!)
My girlfriend and I both use Red Hat 7.2 on my laptop. This is because it's easier to get all the networking set up (802.11b, DSL using PPPoE, IrDA) under Linux than under Windows, since the drivers are terribly buggy under the latter. She understands that sometimes web pages won't behave perfectly because the average HTML writer does not understand that people use platforms aside from Microsoft Windows, but since this is ostensibly my "work" machine, she's okay with that. Moreover, everything "just works" and when it doesn't I can log in remotely to fix it.
Contrast this to when I was just getting started... I expected people to know things or at least care why the computer acted the way it did.
Boy, was that a crock of shit!
I have nontechnical users merrily sending mail from Mutt and Pine on OpenBSD now because I simply give them a set of directions, say "It's not perfect, but it's a compromise, and in 3 years we've never been hacked; please play along nicely". Since my users all accomplish what they want to, they are happy, and since they're happy, I have more time to twiddle RAIDframe, play with Coda and Heartbeat, and generally nerd out.
The more experience I get, the less experience I expect my users to have, and the happier they are overall. Next week the marketing guy will be switching over to using RSA keys for SSH access from his cellular modem. I'm not kidding.
It doesn't have to be intimidating or nerdy to do the job right!
But if the credentialing scheme in place depends on Windows frontend servers being secure, you can damn well better bet that it will be dutifully serving up data to the wrong party.
Can't do much about that. I don't perform ANY core business functions on Microsoft server software, their history of getting brutally hacked and denying it is far too pervasive. (Yes, Sun and IBM are terrible too. Frankly, Red Hat and the OpenBSD Project are valuable to me not because they're "perfect", but because they're honest and prompt when they fuck up! I cut both organizations a new check every 6 months of my own free will, NOT because they try and force my company to. The checks come out of my after-tax salary; as far as I know the company has never paid a dime for either project's media.)
The consultants were probably lazy too, but don't get too overzealous to defend the most probable point of entry. I am somewhat less than surprised that a large gov't agency would screw up like this, although most of the dep'ts I work with at least have the sense to retain solid IT security consultants (I've met some very competent Lockheed employees, for example; I have no idea who was at fault in this incident).
Lately I've been compiling Apache with just PHP, mod_ssl, and mod_auth_pgsql (no mod_perl anymore). Last time I did it was not TOO hard, though. You need to apply the SSL patches FIRST, THEN apply the mod_perl patches, and LAST add PHP.
At least, that worked for Apache 1.3.12 on FreeBSD 2.2. Like I said, it's been a while since I needed to prototype an Apache module (sooner or later it's best to move them all to C, IMHO).
I can't speak to higher-end solutions, because as I mentioned in my response, I suspect they'll already have an architecture in place (eg. when I was at IBM Burlington, before Snort was even born, the setup they had created for monitoring ingress and egress traffic was far beyond what I've seen before or since).
But for my live production hosts, dual-homed on UUNet and Qwest, and all monitored, Snort + Barnyard + ACID have kept up without clipping traffic or interfering with operations. And yes, we DO saturate both of those links on occasion (though not always).
That's all I can speak to. When I worked at XOOM we saw traffic up to about 0.75Gbps steady and never bothered running an IDS, just were real fucking careful about what went live and keeping everything audited. An HP OpenView installation with some sort of IDS support was looking like $300K in bills. We said "fuck that" and to this day I wouldn't do any differently.
But, my situation may be very different from yours. If you need a $20K solution and its presence saves you $40K, you sure as hell don't need my blessing to buy it!
That made it pretty damn easy for me to push Snort where I work.
Only choads that are getting kickbacks from manufacturers are going to push for overpriced commercial solutions in shops that don't have an existing IDS installation or a compelling reason to use the packaged solutions (NetRanger, OpenView, their ilk).
A packet is a packet... NFR and Snort are both designed by well-respected engineers who are more interested in accuracy and correctness than in unit shifting. I trust them for that.
When you get right down to it, unless you're rolling in dough, why blow $20,000 per management station plus consulting costs to implement something your network administrator can probably set up in a week for free? (I know I can) It's stupid. Save the cash for your coke dealer or a rock for the missus.
See my earlier comment about ACID. Multisensor correlation and alert grouping, emailing of packet traces to offenders or CIO's, pretty much all you could ask for.
Try it. ACID homepage You may be pleasantly surprised at how easy Snort is to scale up. I have numerous sensors, all in production, all logging on all interfaces, all the time, and haven't had any major incidents on my subnet. I credit this partly to having early warning of when some idiot tries to attack my boxen, as well as to using Thoth for host monitoring, which makes it trivial to check that all my daemons are up-to-date, and all kernel patches are installed.
Someone pisses me off consistently, they get blackholed. This is something I'd recommend doing by hand, of course, but for people whose business I don't need or want, it's a great way to end the problem right then and there. :-)
Both of the projects he's mentioning are brilliant. They encourage people to disclose alerts so that the numbers can be leveraged en masse to get vendors and ISPs off their asses.
:-))
This is the flipside to things like predictive agents and automated vulnerability testers. It improves security by social mechanisms. I'm going to look into using ARIS today, and if I can figure out how myNetWatchman works, I'll consider it too.
I hope other readers are forming solid opinions about where IDSes ought to be headed by reading posts to this article. It has been informative for me, at least. (and I've been around this stuff for a while... sometimes I see packet headers in my sleep
See my earlier comment about ACID. Multisensor correlation and alert grouping, emailing of packet traces to offenders or CIO's, pretty much all you could ask for.a > for host monitoring, which makes it trivial to check that all my daemons are up-to-date, and all kernel patches are installed.
:-)
<p>
Try it. <a href="http://acidlab.sourceforge.net/">ACID homepage</a> You may be pleasantly surprised at how easy Snort is to scale up. I have numerous sensors, all in production, all logging on all interfaces, all the time, and haven't had any major incidents on my subnet. I credit this partly to having early warning of when some idiot tries to attack my boxen, as well as to using <a href="http://firedrake.org/thothproject/">Thoth</
<p>
Someone pisses me off consistently, they get blackholed. This is something I'd recommend doing by hand, of course, but for people whose business I don't need or want, it's a great way to end the problem right then and there.
<p>
Anyways, I want to throw in a shill for ACID for anyone who runs Snort. It makes my job SO INCREDIBLY MUCH EASIER that, well, I bother to do it every day, maybe two or three times a day, and haven't had any major incidents to speak of. If you run Snort, you ought to log to a centralized database that can handle the traffic from all your sensors, and then grind through it with ACID for starters. Yes, you should keep a packet vault; yes, you should run Nessus; yes, you still need to use TripWire or Integrit for filesystems. But having a friendly, capable frontend to Snort sensors is a HUGE help.
If you're running a lot of sensors and they get a ton of attacks in production, you should also look into the Barnyard plugin for Snort. It's nice for keeping things from slowing down.
If I were to take a stab at what would MOST help IDS and ISS research in the near future, I'd guess at the integration of tools like Nessus and Snort with a predictive intelligent agent like Intravenous or similar. I wish I could comment intelligently on the article, but mostly I wanted people using Snort to be aware of HOW helpful the ACID frontend is, so that more people use it, and I have less subnets to blackhole ;-).
Being a bicyclist, I am partial to light, fast, cheap transportation. The Segway appears to be none of these. It is expensive, a brute-force solution to a non-problem IMHO. That's why I, at least, am underwhelmed.
Then again, I dislocated my shoulder last week on my bicycle while avoiding traffic. Maybe I can ride it again tomorrow, maybe not, but it has been quite painful and made it much harder to run errands around town (take the time to run an errand on a bike and double it; you've just arrived at the time to complete it, driving, if it's in downtown DC and you have to park). This device would make such injuries irrelevant. I'm sure it would be wonderful for elderly or infirm people who can't drive. So perhaps I am an "able-ist" in that I am biased to think about things as if I'll always be hale and healthy.
If the product is made affordable, it would be a lot nicer and less intrusive than a Lark or a Rascal for sure. But I don't see it as being quite as revolutionary as the car, simply because it does not radically increase carrying capacity, doesn't really offer commercially compelling advantages over a regular scooter, pair of feet, or a bicycle to balance out the cost... I don't see how this device would change the world for the average mope, but for some people it sounds like a godsend.
Attenuate your expectations, as this Dean Kamen seems to be telling us, and in context it is pretty neat. Not earth-shattering, but pretty neat, alright.
Actually, we're in DC, and I work on 3 of them. They're incredulous that things can actually get done, and I plan to keep them that way! More are lined up because of this.
Dude, this is like saying that you shouldn't use VB or shell scripts because then you'll have to hire a [language Foo] guru.
Anyone who can understand C++ or real C code can pick up Perl or Python or PHP in a few hours.
maybe your arguments would make more sense. But I can't really be bothered to futz with the internals of Postgres or Apache or qmail or [...] every time I need to change something.
And the other employees of the company want to change things, our clients want things changed, no spec is ever firm, etc... eventually, at least in the line of work I'm doing, it dawns on just about everyone that the "logic" and the "presentation" are not only separate, but often so totally unrelated that one or two hooks will suffice. That's where my break happens.
The whole "MVC" idea says the same thing, in so many words. I'm just using the best tool for the job, and I've done this sooooooo many times, our business is soooooooo specialized, and our clients expect things sooooo fast (cause we always turn things out pretty fast) that, rather than being a conscious decision to eschew CMM or OOP or whatever the PHB buzzword of the week is, I simply got the damn job done, producing some reusable libraries along the way.
It's also worth noting that all of this *COULD* be in C/C++ or Perl alone, but I don't feel that Perl is good for anything other than glue, and I don't want to dive into a shit load of C code every time a client requests a change.
Anyways, that's just my job experience. Yours may be totally different, and probably is, if you write device drivers or microcode for a living, work on huge integrated apps, and so on. Then I'd probably want to stay in one language with *maybe* a scripting language for unit tests and prototyping interfaces.
lest you facilitate a dialogue with my supervisor.
my sourcebook for business-speak -- Action Item Man comics
yeah, you got me... I'll go back to my TPS reports now.
In my opinion, the single biggest hurdle is acceptance by the client. If it's too late, or too buggy, or too rigid, no one will care whether it's in Java or PHP or Perl or Visual Fortran...
I write code on-demand because that's what I get paid to do. I've tried being super-neat and uni-language, sloppy as hell and multi-language, and have settled on the processes that work best for me to keep our clients happy.
I don't pretend to offer a solution for everyone in every situation, but in a small company with high-profile clients, results are the only things that matter. I find that using a fast, compiled "core" with flexible, highly-documented "glue" keeps me from having to work too much overtime, and allows other developers to get up to speed quickly.
Documentation is KEY to my strategy. Without it, I don't think any of my other comments are relevant. I'm not an OO fascist but it sure does make documenting and segmenting code easier, for run-of-the-mill business and controller apps.
Again, this is just one programmer's experience.
I'd go insane if I tried to use a compiled language for frequently-changing applications (eg. web interfaces to purchasing systems, database large object manipulation & indexing, etc.) but likewise I'd grow old waiting for things to happen if the cores weren't in C.
Personally I dislike C++ because I find it harder to port than ANSI C. But I wrote plenty of it when I was learning how to program (before I learned Scheme and Perl and Java and got some latitude in my paradigms). Either C or C++ is great for speed and hooking into the OS or an Apache module or whatever. It's also less tempting for admins or rookies to mess with C code because bad code may not compile.
On the other hand, object-oriented (or at least modular) PHP and Perl code, and decently-written Java code, is much easier to adapt to changing demands. I stick with PHP and Perl, myself, and I use Perl as kind of a glue-core layer between C applications and PHP interfaces. If I had more time I'd probably write the hooks as PHP extensions, but Perl is just so damn powerful when you use it right.
I'm not sure what people are thinking when they specify that everything on a project "needs" to be done in Java or whatever. It's not particularly hard to use my code, since it's all just calls to libraries that automagically do the hard stuff. More importantly, I tend to use POD or Javadoc-style comments in everything, and don't put it anywhere else. That way I am forced to keep it all up to date, because that's where *I* remind myself what arguments to feed what methods!