Re:Oracle may be losing relevance
on
Oracle Linux Explored
·
· Score: 2, Interesting
> For me, Oracle is a non-starter. It's big, expensive, and reportedly has a high management overhead. So why would I bother?
you often wouldn't for a start-up - assuming modest data volumes postgresql is a great choice
But let's say that you've got a 200 gbyte table, and your query is doing table scans because the selection criteria identifies more than 5% of the data in the table. Ok, on db2 or oracle with partitioning, parallelism and very good query optimization that might take you, say, 2-5 seconds on $40k in hardware. Not too bad. How long for postgresql? 20x as long for the table scan and 4x as long for the serial activity. So, 160-400 seconds. Meanwhile you're pounding the snot out of your server.
Of course, you could buy a million dollar machine to crunch the data more quickly with your free database. But it would probably be cheaper, easier and faster to just buy a $20-40k database.
And sure, you could keep the data in 200 separate tables and use a union-all view to concatenate them together. Postgresql will do this part. Assuming you want to keep your data in 200 tables, and assuming that you can take the performance when you do have to scan through them all - and it tries to union them all together.
So, yeah - postgresql is a great database, and I'd probably want to use that too if I was in your shoes. But in my shoes I've got a lot of data, and need to scan tons of it quickly - to ensure my users get subsecond response time. Saving money on postgresql here ultimately loses money in customer revenue.
Personally, these days I prefer db2 for data warehousing and business intelligence - since even the lowest-end products include partitioning, great query optimization, materialized views, etc - everything you need to run a 5TB data warehouse. And when you get into frequent changes in a multi-terabyte environment db2 seems easiest to work with.
Anyhow, $160k for a four cpu license does sound expensive. In db2-land that would probably be around $120 for the top-end product, but either oracle or db2 could also come out far cheaper - depending on which features exactly you want to use.
And note that this could still be cheaper than using mysql. Here's how: mysql doesn't have GA partitioning, parallelism, query rewrite, a descent optimizer, etc. So, if you've got an analytical application in which you've got tables with hundreds of millions of rows - and often have to access 5-10% of that data then mysql will force you to do table scans. DB2 and Oracle will easily use partitioning instead - giving you a 10-20x performance gain. They'll also divide the work up between the four CPUs - giving you another 4x performance gain. In the analytical application space you may also have very complex queries - that will definitely performance far faster on oracle or db2. How much? Difficult to quantify. Bottomline - in this scenario db2/oracle will give you a 40-80x performance benefit for this type of app over mysql.
So, $160k for oracle to go on $80k in hardware probably sounds expensive. But it's extremely cheap compared to paying for another 40-80x more hardware to get equiv performance out of mysql.
Also, another reason why people go for something like oracle & db2 when they don't really need to: labor is typically more expensive than the software or hardware. And you really don't want to work in an enterprise in which people actually select products based on what's best for each system - rather than attempt to achieve some consistency. It's way too expensive to support *every* database product out there, much cheaper to just support oracle or db2.
Of course, if postgresql can do everything you need (you're a small company) then great - I'd stay on just that and nothing else. But when your postgresql/mysql 100 gbyte database is bogging down - and you're being told to spend $200k on a sun 8-way machine, it's probably time to just move to oracle or db2.
right, because transactional engines are:
- because they are trivial
- because the need for consistency with the rest of the storage engines isn't important
- because developers are a dime a dozen
- because healthy and productive open source projects are such a breeze to create
- because none of the existing developers in the employ of oracle are encumbered by ip agreements and most importantly:
- because
Some people wonder if you have enough money to pay for food - since television is obviously higher than food on mazlo's pyramid.
Then others wonder if your children will be well-developed without access to mtv, american idle, and various reality and sit-com programs.
And others still will be concerned that your abnormal productivity - that extra 2-4 hours a day of learning how to play the guitar, remodeling your home, etc will all come to a bad end eventually.
Personally, I'm just relieved to avoid 40-80 minutes of advertising a day along with 80-160 minutes of stupid programming.
And of course, playing games with your daughter and wife is a great family experience. Watching television isn't. It's no more of a family activity than sleeping in the same house is.
> There's no need to go back to the stone age. Just use prepared statements/parameterized queries
There are often quite a few benefits of going with stored procedures besides the ability to improve security. Right now I'm on a project in which another team is developing an application layer on my database - and we've now found out that they are almost sql-illiterate. Meanwhile, they're trying to write trending queries, etc - that are often over 200 lines long - that produce incorrect results at 100x the speed the query should take. This is *after* we gave them a set of queries that we wrote that worked fine - they had to tweak them to work in their environment.
So, all queries are being replaced by stored procedures that: 1. validate input 2. leave an audit trail of start & stop time, number of rows returned, all arguments, return code, etc
I'm on a call right now in which that team is trying to solve huge query problems - and I'm just moving each problem query into an API. If they have problems with that API, fine - we'll fix it. We'll also see the performance problems immediately. And there's no risk of them returning data for the wrong customer.
And sure, the stored procedure language is primitive - but then again we're just using it as a thin wrapper for a query - so it isn't like it needs to be extremely powerful. And sure, this code isn't portable. But back to "thin wrapper description" - if we need to migrate these to another feature-rich database it won't be a problem: 90% of the code is just the query itself.
Anyway - back to sql injection. Another team also wanted to access the same data warehouse. I took a look at their code and saw that it was completely rife with sql injection vulnerabilities. Rather than tell them that they need to fix all of this AND rewrite their queries to hit our data model - I told them to just use our APIs. Those same ones mentioned above? Yep, completely reusable by this second project, completely safe against sql injection, complete auditing, complete validation AND reduces their codebase by about 30%. The only way it could be better is if we were able to use python or ruby as the stored procedure language.
> I'm curious, what virtualization platform did you use? Which guest operating systems?
Wish I could tell you, but I can't.
i have a similar problem with virtualization
on
To Grid Or Not To Grid?
·
· Score: 4, Informative
My management is similarly obsessed with virtualization: they want to lower admin costs, lower lab costs, etc through this simple technology.
So, rather than move everything over to lpars I took a simple step - purchased a large virtualization-oriented server highly touted as perfect for this, and moved over a single app, with the goal of putting two apps on this server. Along the way I learned:
- io virtualization sucks for io-heavy applications
- the tools to determine how much of the cpu your app is getting at a given moment stink
- memory virtualization in which you resize application memory is primitive and almost useless
- there were no guidelines for optimization of the server - just recommendations to try it
hundreds of different ways and leave it on the best settings
- basic setup of the machine required wading through tons of jargon that even the os engineers didn't seem to know well
- out of the box - a single app on the new virtualization server performed more slowly than it did on a free seven year-old server
- some of the most heavily-advertised virtualization features of the product just don't work
- virtualization of multiple busy apps onto the same server is mostly a waste of money
- virtualization of multiple mostly idle app (failover servers, test servers, demo servers, etc) should work very well
- we spent at least $25k on labor just to create something that was a slam dunk
- I'm glad that we started with a small prototype - and didn't waste a ton of cash moving everything over immediately the way some management hoped
- I think in the end we'll get multiple apps working on this box just fine. BUT - we will have spent more money on this scenario than by simply purchasing separate systems. We may recoup a savings if we move enough idle systems onto virtual boxes.
As a result of this experience my team now knows more about virtualization than any other people in the division, we now have a production server supporting it, my management is now cool on this technology, and there is no risk of being forced to migrate critical servers over quickly to the virtual world. I'd call that a success.
I think that you're right - that grid is in a hype cycle right now. So - there are quite a few disappointments to be had along the way to its implementation. For example - if your workload is heavily transactional - you're really not going to get much benefit. In this example oracle supports grids - but it is really more about failover than performance. If you roll your own or use a more sophisticated product you can be safe in assuming that you'll hit unexpected issues, a gap between vendor marketecture & what you really need, and possibly the pain of having a vendor talking directly to your management.
You might want to consider having management fund a small prototype to prove out the benefits. Then let them see that they can achive perhaps better availability but worse performance at a very high cost through this approach.
good luck
but are these features washingmachine-proof?
on
Top Ten Geek Wallets
·
· Score: 2, Funny
All these gadgets are fine, until my wife throws the pants & wallet together into the wash.
note, in case my wife reads this: not complaining that someone else in the house does my wash, just need to not waste money on a wallet that can't survive the handling.
> people want to connect with the games of their youth.
how about a few other reasons, like:
- these games are quick to pick up - you don't have to blow eight hours to just learn the game
- many of them lived & died on gameplay - which many newer, more media-rich games blow
> 2) they've got Robotron 2084 for hell's sake. robotron 2084!!. and time pilot.
Man, i loved both of those games. Especially Robotron. What a great workout - I could hit three million in that game over the period of thirty minutes - by which time I got killed due to numb fingers. I'd love to play that more but I'd really want the good controls.
or a company that has nearly infinite profitability and can dangle vast financial rewards in front of the nose of its entire staff.
now, simplify your process:
- "portfolio management" (ie, determining which projects move forward and which get cancelled) can be self-managing through the developers own self-interests
- developer productivity is handled through developer self-interest
- project scheduling is handled through developer self-interest
man, there's all kinds of benefits to this: just like the Mythical Man Month showed - the cost of management/liaison/coordination overhead is a killer. And so is the cost of mediocre or poor developers. So, if you have a large staff of self-directed, self-motivated and expert developers motivated by huge financial rewards it will *really* pay off.
All you need to do is to start with the spherical cow, er infinitely profitable company, and some insightful management. And the rest is a cakewalk.
> Regarding the number of meetings, I only have one formal meeting a week, but can spend several hours a week with a couple > other guys talking over the specifics of whatever we're working on. Could be considered "meetings", even though they don't > involve sitting around a table and going through an agenda.
right, so it isn't a meeting if you're not sitting down.
so, the best way to reduce meetings is to stand up when having "work-oriented get-togethers"? Sounds so simple, I wish we had figured that one out before!;-)
> The thing is, we have a highly distributed database, the network is not constantly available, and updates (within constraints) > may happen simultaneously at different places in the network and be reconciled later. And most of the nodes run on small > embedded systems, while some nodes are large, centralised severs.
sounds like a good problem:-)
There are some database combinations that might work - db2, for instance, has a little brother product that runs on palmtops and is sometimes used for configurations like this. But the asynchronous updates are a challenge - especially if they are bidirectional. If not, then an ETL approach might work for you - it's like batch replication.
> I would like to read more about the things you mention, especially data modelling as a discipline, and the > things you listed: "dynamic attributes", "snowflake schema" etc. Suggest any other good books?
Off the top of my head I can't recommend a general book on data modeling. Ralph Kimball has the Data Warehouse Toolkit - which covers data modeling for reporting databases extremely well. David C. Hays has Data Modeling patterns - which covers reusable models the best. The Data Model Resource Book by Len Silverston isn't bad either. These reusable model books might be a useful complement to a book on theory.
Note that once you leave strict relational modeling then you get into unofficial terminology. So, David Hays refers to the "universal modeling pattern" in which you have a generic entity, a relationship type entity and two many-to-many relationships between them. This allows you to support multiple hierarchies, networks, and other relationships - for example in describing organizations and people. Very, very flexible approach. But there is no single common term for this technique. Likewise, when I mention dynamic attributes I'm referring the technique of storing attributes as key-value pair strings along with some metadata (type, length, etc). This is very useful when your data requirements change too quickly to make modeling changes and updates. But again, no common terminology here either.
Yep - data modeling is now out of vogue, and so it is mostly done by programmers.
Unfortunately, programmers are seldom good data modelers (if only because they seldom take the time to really learn the discipline). You can take a look at some of the very good books on data modeling patterns (I recommend the one by David Hays).
But there's no substitute for experience when you get into a tight spot - knowing when to break the old rules and use something like:
- a star or snowflake schema
- a hierarchy or network
- dynamic attributes
- materialized query tables
- triggers
- etc all takes experience and knowledge. One thing that I often do when in a spot like yours - is to define a good object model at least - then use that as the basis for persistence apis. These apis can be stored procedures (best case) or views (worst case). Then (assuming you have a dba that will be owning this code) the dba is free to remap that api layer to the physical tables. Change is inevitable, but at least this way you've got one role that has access to all the code that needs to change, and hopefully your api interfaces will encapsulate the code well.
Unfortunately, stored procedure use is on the decline. But as long as you take a cautious approach with them they're not a bad thing.
You mentioned 3 different objectives:
- kill monsters
- socialize
- create a story
Of which only the first you stated that a computer does it better than a table top. But really, the computer only helps with the logistics of getting together. It hardly does as good a job of killing as you can do on the table top - where subtles of movement, position, psychology, exhaustion and various forms of trickery are so poorly handled.
So, yeah if you just want to rack up some bland kills the computer kicks butt. But if you want to do something really memorable with friends in a game - do it over a tabletop.
For example, I just recently got together with some good friends this summer. We're all about 40 now, used to play heavily between the ages of 16 & 25. We laughed so hard at those shared adventures. The best was the story of the "Chimes of Hunger" - in which the chimes are run - and every character in the room is reduced to a single-minded eating machine - mindlessly eating whatever he has on hand. The GM asked everyone to show him where, on their character sheet, they had marked food. One player had none, but asserted that of course he was carrying food. This character was about a tenth level mage with a homonculous. The GM peeked over his GM Screen and said "homonculous is practically chicken", a reference to an olds buggs bunny. Anyhow, while the homonculous was fighting to get away the mage ate him, inflicting a good dozen points of damage on himself, and leaving himself covered with blood. We thought it was very funny at the time, but apparently the character's owner didn't. He was the only one of all of us that couldn't remember this adventure at all.
Anyhow, still enjoying some of those memories on occasion twenty years later.
> I, for one, would not bring my violin with me to launch rockets with
well, you got me there - there are certainly instruments I wouldn't want my kids to travel with - pipe organs, xylaphones, even violins. Still, kids shouldn't be so specialized - drag the ukelele, tenor guitar, etc with you then. That's what they used to often do way back in the day.
> More generally, I don't understand why playing a gameboy while bored implies anything about what other activities > one engages in while not bored.
The problem is that most children lack the discpline to stop eating candy or playing video games - and so will do them to the exclusion of everything else given the opportunity. Kind of like a rat eating crack. As a parent you typically don't want a kid who at the age of eighteen is an academic and social failure and who's only skill is in some rpg. Not a good way to start as an adult. So, you've got to make sure that they moderate these activities. A few hours a week is harmless in my opinion - but there are millions of kids out there playing 20-40 hours a week.
> Maybe it'd be a good thing to expose your kids to those and let them carry on their own ideas after they're done?
Yeah, that's the big question. But if you ever have kids you'll find that a close temptation becomes a constant source of problems - either you completely give in to the kids or you face frequent debates. Many people break down, wish they didn't have a television, and live with it on every night.
It's so much easier to just get rid of it. And once you get rid of it for 6+ months it's almost impossible to go back. After listening to public radio, books on tape, reading books to each other, etc - the stupidy of almost all programming becomes intolerable.
But I'm not a nazi about it - my kids watch television occasionally when at their friend's home. And I don't care in the least - it keeps it down to an hour a week or so.
> I could very easily be misinterpretting something here, but were the kids there of their own volition, or did > someone (say, a father) drag them out there for some "fun outdoors"?
No idea. But kids often prefer, if given the chance, to do nothing but play video games, eat candy bars, skip school then maybe play with matches and guns. Your job as a parent is to guide your kids away from that kind of behavior and help them eventually become adults.
Part of becoming an adult involves doing more than playing a gameboy. Actually getting outside. Perhaps those kids were bored - though if they didn't have the opportunity to play the 5000th hour on a gameboy they might actually have noticed some cool in the park or field. Might have played tag, might have found an interesting ant hill to check out, etc, etc.
Quite a few people are rightfully, in my opinion, concerned about the overscheduling of children by their parents. These kids are in so many different activities that they don't get a chance to become parents. Well, that's the over-achieving parent's trap. I suppose the under-achieving parent's trap is to let the kid overschedule their life with nothing more that gameboy - so that videogames crowd everything else out like weeds.
> Sure, I imagine it'd be fairly straight forward to sing or play music, but wouldn't that be a little awkward/embarrassing at a venue like that?
I'm not the original poster, so I can't say for sure. But there's nothing awkward about kids playing music.
> Did it ever occur to you that perhaps those kids playing their video games just weren't interested in launching rockets?
Oh sure, they aren't interested in launching rockets. Or reading a book. Or playing a musical instrument. Or singing. Or hiking. Or playing a game of baseball with the neighbor kids. Or any else almost.
They are interested in playing a simple game for thousands of hours - after hitting diminishing returns in the first two.
As a parent, in my opinion letting a kid drag around a gameboy and play it for thousands of hours instead of actually living life is the same as letting him skip all meals and eat nothing but candy bars instead.
> Now, my kids have cable, computer with the net and half a dozen consoles. I work on limiting it, but it is tough.
Yep - constant availability of gaming consoles, flash and other online games and television is like putting racks of candy bars all over your house. Six year olds shouldn't be eating a non-stop diet of chocolate and fried potatoes all day, nor should they be sitting on their asses playing video games and watching television all day. The challenge is that in some communities (especially suburbs) a couple of hours of this kind of play a day is the norm. And in that situation restricting your kids has got to be tough.
But I know of many households that restrict kids to four or less hours of electronic games & television a week. In my household we ditched television broadcasting (cable, dish, antenna) fifteen years ago and have *never* regretted it. We rent dvds a couple of times a month, that's it.
Sure, it means that kids don't get to watch their MTV when they were eight years old, but they did read "The Wind in the Willows" instead, they did learn how to play musical instruments, juggle, explore the local trails, wrestle, play with the dog, play with their friends, etc. All far better ways for kids to spend their time.
The 5% is what I found to be the difference when making system calls. The overhead of invoking python is pretty trivial when you're then executing a ten second external system check. But even beyond that easy comparison - many of the modules are very fast - traversing directories, etc, is very quick.
> It says they are maybe 1.7 times faster than CPython, which is not that great, because CPython is incredibly slow and things > like Psyco can give pretty big speedups (say 10 to 100 according to their website). > It seems fundamentally impossible to make a language like Python or Ruby fast.
fast or slow are relative and somewhat meaningless terms.
I use python to transform tens of millions of rows of data every day in the running of a data warehouse. C would be faster for most of these processes, but not all - since python's io speed is the same as C's and these are io-heavy applications. But i'll trade a lot of that speed for the advantages in maintainability & speed of development that I get with python.
Likewise, small scripts that we use for monitoring disk space, monitoring processing queues, checking for data quality issues, etc - are fine in python. C would be faster, but probably only 5% most of the time. So, who cares about the difference between 1.0 second and 0.95 seconds?
And if I felt like writing a small gui on a windows box I'd prefer to work with python over.net or vb. It might be slower, or it might not. Either way it'll probably run fine on a 1ghz desktop.
Re:With all due respect to the man ...
on
Steve Irwin Dead
·
· Score: 0
> I know I'm ranting out of proportion to your specific post, and I apologize. It's just that this seems to be happening all > the freaking time lately, no matter how tenous the connection to the topic is, and it's driving me batty.
Unfortunately, it is happening more & more - just like public expressions of anger for the government happened more & more in the 60s. And they're now angry enough they there opinions have started spilling over in non-political discussions.
Americans weren't allowed to criticize the government's policy for a couple of years. No laws against it - just a ton of social pressure following the administration's direction at the top. Even thoughtful questioning of our motives would be met with criticisms of "sedition", "hating america", and "not supporting the troopos". You instantly made people nervous and worried or angry if you stated wacky things like "but wait, the UN chief weapon inspector said there are no WMDs in Iraq, why are we going in again?".
So, quite a few people in this country are freaking out, and for good reason: the controlling party has in just four years:
- thrown the US reputation into the toilet
- shown a clear interest in rolling back the bill of rights as much as possible
- put us deeply into dept and at China's mercy
- encouraged vast outsourcing of jobs
- gotten us embroiled in a war that we cannot win - thousands of miles from home
- been too bogged down in their oil war to address actual terrorist nations
People don't want to be drafted to fight iran, they don't want their neighbors getting killed fighting someone else's oil war, they don't want to see our people torturing civilians in other countries, they don't want their jobs oursourced overseas, they don't want to be unable to travel, they don't want to attract terrorists to their communities, and they don't want big brother monitoring them.
I didn't expect to see any anti-bush sentiments on this thread either. But I'm not surprised when they appear. Wait another year - when iraq is really up in flames - you'll see even more.
Hmm, sounds like you mean the opposite of Not-Invented-Here: it typically means that an organization doesn't respect a solution unless it was invented there.
But I think I agree with your thoughts: I've seen far better productivity come from small teams creating simple solutions than from much larger teams attempting to implement huge commercial platforms.
> That one was to actually vary the pay of the teachers and their tenure based upon the results of the standardized test progress of their students.
Right - think carefully about that just a little bit. If someone's pay is directly based on test scores, then... the teacher will want to get every kid that isn't promising kicked out of their class. Slightly slow child? Poor english? Minor health issues? Whatever, they're out of there! No time for charity - there are high scores to be earned!
> Beginning about the 4th grade students should no longer carry books. They should be issued laptops.
Great idea. And who's going to be paying to replace these laptops every year? You don't actually think that laptops are going to last more than a year (if that) in the hands of 9 year olds, do you?
> Schools are not under funded. They are grossly over funded. > Teachers are not under paid except in their early career years. > You get what you pay for.
You've got a bug in your code here fella.
> A note to the mods. This is the most on topic least troll and most informative listing you have ever read on this topic.
No, this is the one of the least on topic and least informative postings today. Your thinking is cloudy and you're ranting half of the time. You should probably start taking your meds again.
You claim responsibility for the No Child's Behind Left program. That's just precious - this is the program by which every school will eventually be a "failing school". See, eventually every school runs out of progress, every school will fail to get good grades out of some tiny minority sliver, and every school will fail to get 100% of their students over the bar.
I know the critics of the public schools are disgusted at the poor performance that some of them deliver. Then again - look at the poor performance that these critics deliver:
- poor grammer (see above posting)
- inability to pass tests they require of high school students (see Colorado Governor Owen's big testing failure)
- inability to work with numbers (see how NCLB will cause all schools to fail within next 5-7 years)
Testing is a good thing, no argument there. But giving testing numbers to the numerically illiterate (whether it is pointy-haired bosses in corporate america or ranting anti-school libertarians) just doesn't work. Here's a suggestion - lets take a look at the parents role a bit, ok? Why do we expect teachers to work miracles with kids that are allowed to play videogames, watch television, and play sports 4-5 hours a day? Where are the parents of all these poor-performing children?
>> A moment's panic may well cost someone thirty million.
> Christ. With that kind of debt one might as well flip out, blow up the RIAA's HQ and disappear into the middle-east or China. $30 million!
Look at how little companies *usually* end up paying when they are guilty of causing them bodily harm. Prisoners that were used for medical testing that got really screwed up back in the 70s received something like $40k! Compare that to $30 million. Clearly, taking those songs was 750 times as bad as screwing some persons health for the rest of their life.
Shit at this point I'd be inclined (in perhaps a rash moment of anger) of demanding that they raise it to $30 billion. Why not? It isn't like I'm going to come up with $30 million either - might as well make it totally assinine. Also raises the value of a copied song to $750,000,000 - or 3,750 times the value of a person's health. Really puts what is valued in our country into a useful perspective.
Slashdot Mirror
> For me, Oracle is a non-starter. It's big, expensive, and reportedly has a high management overhead. So why would I bother?
you often wouldn't for a start-up - assuming modest data volumes postgresql is a great choice
But let's say that you've got a 200 gbyte table, and your query is doing table scans because the selection criteria identifies more than 5% of the data in the table. Ok, on db2 or oracle with partitioning, parallelism and very good query optimization that might take you, say, 2-5 seconds on $40k in hardware. Not too bad. How long for postgresql? 20x as long for the table scan and 4x as long for the serial activity. So, 160-400 seconds. Meanwhile you're pounding the snot out of your server.
Of course, you could buy a million dollar machine to crunch the data more quickly with your free database. But it would probably be cheaper, easier and faster to just buy a $20-40k database.
And sure, you could keep the data in 200 separate tables and use a union-all view to concatenate them together. Postgresql will do this part. Assuming you want to keep your data in 200 tables, and assuming that you can take the performance when you do have to scan through them all - and it tries to union them all together.
So, yeah - postgresql is a great database, and I'd probably want to use that too if I was in your shoes. But in my shoes I've got a lot of data, and need to scan tons of it quickly - to ensure my users get subsecond response time. Saving money on postgresql here ultimately loses money in customer revenue.
Personally, these days I prefer db2 for data warehousing and business intelligence - since even the lowest-end products include partitioning, great query optimization, materialized views, etc - everything you need to run a 5TB data warehouse. And when you get into frequent changes in a multi-terabyte environment db2 seems easiest to work with.
Anyhow, $160k for a four cpu license does sound expensive. In db2-land that would probably be around $120 for the top-end product, but either oracle or db2 could also come out far cheaper - depending on which features exactly you want to use.
And note that this could still be cheaper than using mysql. Here's how: mysql doesn't have GA partitioning, parallelism, query rewrite, a descent optimizer, etc. So, if you've got an analytical application in which you've got tables with hundreds of millions of rows - and often have to access 5-10% of that data then mysql will force you to do table scans. DB2 and Oracle will easily use partitioning instead - giving you a 10-20x performance gain. They'll also divide the work up between the four CPUs - giving you another 4x performance gain. In the analytical application space you may also have very complex queries - that will definitely performance far faster on oracle or db2. How much? Difficult to quantify. Bottomline - in this scenario db2/oracle will give you a 40-80x performance benefit for this type of app over mysql.
So, $160k for oracle to go on $80k in hardware probably sounds expensive. But it's extremely cheap compared to paying for another 40-80x more hardware to get equiv performance out of mysql.
Also, another reason why people go for something like oracle & db2 when they don't really need to: labor is typically more expensive than the software or hardware. And you really don't want to work in an enterprise in which people actually select products based on what's best for each system - rather than attempt to achieve some consistency. It's way too expensive to support *every* database product out there, much cheaper to just support oracle or db2.
Of course, if postgresql can do everything you need (you're a small company) then great - I'd stay on just that and nothing else. But when your postgresql/mysql 100 gbyte database is bogging down - and you're being told to spend $200k on a sun 8-way machine, it's probably time to just move to oracle or db2.
> MySQL AB can just fork'em
right, because transactional engines are:
- because they are trivial
- because the need for consistency with the rest of the storage engines isn't important
- because developers are a dime a dozen
- because healthy and productive open source projects are such a breeze to create
- because none of the existing developers in the employ of oracle are encumbered by ip agreements
and most importantly:
- because
Yeah, living without television is tough.
Some people wonder if you have enough money to pay for food - since television is obviously higher than food on mazlo's pyramid.
Then others wonder if your children will be well-developed without access to mtv, american idle, and various reality and sit-com programs.
And others still will be concerned that your abnormal productivity - that extra 2-4 hours a day of learning how to play the guitar, remodeling your home, etc will all come to a bad end eventually.
Personally, I'm just relieved to avoid 40-80 minutes of advertising a day along with 80-160 minutes of stupid programming.
And of course, playing games with your daughter and wife is a great family experience. Watching television isn't. It's no more of a family activity than sleeping in the same house is.
> There's no need to go back to the stone age. Just use prepared statements/parameterized queries
There are often quite a few benefits of going with stored procedures besides the ability to improve security. Right now I'm on a project in which another team is developing an application layer on my database - and we've now found out that they are almost sql-illiterate. Meanwhile, they're trying to write trending queries, etc - that are often over 200 lines long - that produce incorrect results at 100x the speed the query should take. This is *after* we gave them a set of queries that we wrote that worked fine - they had to tweak them to work in their environment.
So, all queries are being replaced by stored procedures that:
1. validate input
2. leave an audit trail of start & stop time, number of rows returned, all arguments, return code, etc
I'm on a call right now in which that team is trying to solve huge query problems - and I'm just moving each problem query into an API. If they have problems with that API, fine - we'll fix it. We'll also see the performance problems immediately. And there's no risk of them returning data for the wrong customer.
And sure, the stored procedure language is primitive - but then again we're just using it as a thin wrapper for a query - so it isn't like it needs to be extremely powerful. And sure, this code isn't portable. But back to "thin wrapper description" - if we need to migrate these to another feature-rich database it won't be a problem: 90% of the code is just the query itself.
Anyway - back to sql injection. Another team also wanted to access the same data warehouse. I took a look at their code and saw that it was completely rife with sql injection vulnerabilities. Rather than tell them that they need to fix all of this AND rewrite their queries to hit our data model - I told them to just use our APIs. Those same ones mentioned above? Yep, completely reusable by this second project, completely safe against sql injection, complete auditing, complete validation AND reduces their codebase by about 30%. The only way it could be better is if we were able to use python or ruby as the stored procedure language.
> I'm curious, what virtualization platform did you use? Which guest operating systems?
Wish I could tell you, but I can't.
My management is similarly obsessed with virtualization: they want to lower admin costs, lower lab costs, etc through this simple technology.
So, rather than move everything over to lpars I took a simple step - purchased a large virtualization-oriented server highly touted as perfect for this, and moved over a single app, with the goal of putting two apps on this server. Along the way I learned:
- io virtualization sucks for io-heavy applications
- the tools to determine how much of the cpu your app is getting at a given moment stink
- memory virtualization in which you resize application memory is primitive and almost useless
- there were no guidelines for optimization of the server - just recommendations to try it
hundreds of different ways and leave it on the best settings
- basic setup of the machine required wading through tons of jargon that even the os engineers didn't seem to know well
- out of the box - a single app on the new virtualization server performed more slowly than it did on a free seven year-old server
- some of the most heavily-advertised virtualization features of the product just don't work
- virtualization of multiple busy apps onto the same server is mostly a waste of money
- virtualization of multiple mostly idle app (failover servers, test servers, demo servers, etc) should work very well
- we spent at least $25k on labor just to create something that was a slam dunk
- I'm glad that we started with a small prototype - and didn't waste a ton of cash moving everything over immediately the way some management hoped
- I think in the end we'll get multiple apps working on this box just fine. BUT - we will have spent more money on this scenario than by simply purchasing separate systems. We may recoup a savings if we move enough idle systems onto virtual boxes.
As a result of this experience my team now knows more about virtualization than any other people in the division, we now have a production server supporting it, my management is now cool on this technology, and there is no risk of being forced to migrate critical servers over quickly to the virtual world. I'd call that a success.
I think that you're right - that grid is in a hype cycle right now. So - there are quite a few disappointments to be had along the way to its implementation. For example - if your workload is heavily transactional - you're really not going to get much benefit. In this example oracle supports grids - but it is really more about failover than performance. If you roll your own or use a more sophisticated product you can be safe in assuming that you'll hit unexpected issues, a gap between vendor marketecture & what you really need, and possibly the pain of having a vendor talking directly to your management.
You might want to consider having management fund a small prototype to prove out the benefits. Then let them see that they can achive perhaps better availability but worse performance at a very high cost through this approach.
good luck
All these gadgets are fine, until my wife throws the pants & wallet together into the wash.
note, in case my wife reads this: not complaining that someone else in the house does my wash, just need to not waste money on a wallet that can't survive the handling.
I'm thinking about neoprene
> people want to connect with the games of their youth.
how about a few other reasons, like:
- these games are quick to pick up - you don't have to blow eight hours to just learn the game
- many of them lived & died on gameplay - which many newer, more media-rich games blow
> 2) they've got Robotron 2084 for hell's sake. robotron 2084!!. and time pilot.
Man, i loved both of those games. Especially Robotron. What a great workout - I could hit three million in that game over the period of thirty minutes - by which time I got killed due to numb fingers. I'd love to play that more but I'd really want the good controls.
or a company that has nearly infinite profitability and can dangle vast financial rewards in front of the nose of its entire staff.
now, simplify your process:
- "portfolio management" (ie, determining which projects move forward and which get cancelled) can be self-managing through the developers own self-interests
- developer productivity is handled through developer self-interest
- project scheduling is handled through developer self-interest
man, there's all kinds of benefits to this: just like the Mythical Man Month showed - the cost of management/liaison/coordination overhead is a killer. And so is the cost of mediocre or poor developers. So, if you have a large staff of self-directed, self-motivated and expert developers motivated by huge financial rewards it will *really* pay off.
All you need to do is to start with the spherical cow, er infinitely profitable company, and some insightful management. And the rest is a cakewalk.
> Regarding the number of meetings, I only have one formal meeting a week, but can spend several hours a week with a couple
;-)
> other guys talking over the specifics of whatever we're working on. Could be considered "meetings", even though they don't
> involve sitting around a table and going through an agenda.
right, so it isn't a meeting if you're not sitting down.
so, the best way to reduce meetings is to stand up when having "work-oriented get-togethers"? Sounds so simple, I wish we had figured that one out before!
> The thing is, we have a highly distributed database, the network is not constantly available, and updates (within constraints)
:-)
> may happen simultaneously at different places in the network and be reconciled later. And most of the nodes run on small
> embedded systems, while some nodes are large, centralised severs.
sounds like a good problem
There are some database combinations that might work - db2, for instance, has a little brother product that runs on palmtops and is sometimes used for configurations like this. But the asynchronous updates are a challenge - especially if they are bidirectional. If not, then an ETL approach might work for you - it's like batch replication.
> I would like to read more about the things you mention, especially data modelling as a discipline, and the
> things you listed: "dynamic attributes", "snowflake schema" etc. Suggest any other good books?
Off the top of my head I can't recommend a general book on data modeling. Ralph Kimball has the Data Warehouse Toolkit - which covers data modeling for reporting databases extremely well. David C. Hays has Data Modeling patterns - which covers reusable models the best. The Data Model Resource Book by Len Silverston isn't bad either. These reusable model books might be a useful complement to a book on theory.
Note that once you leave strict relational modeling then you get into unofficial terminology. So, David Hays refers to the "universal modeling pattern" in which you have a generic entity, a relationship type entity and two many-to-many relationships between them. This allows you to support multiple hierarchies, networks, and other relationships - for example in describing organizations and people. Very, very flexible approach. But there is no single common term for this technique. Likewise, when I mention dynamic attributes I'm referring the technique of storing attributes as key-value pair strings along with some metadata (type, length, etc). This is very useful when your data requirements change too quickly to make modeling changes and updates. But again, no common terminology here either.
Good luck
> I suspect it's a common scenario.
Yep - data modeling is now out of vogue, and so it is mostly done by programmers.
Unfortunately, programmers are seldom good data modelers (if only because they seldom take the time to really learn the discipline). You can take a look at some of the very good books on data modeling patterns (I recommend the one by David Hays).
But there's no substitute for experience when you get into a tight spot - knowing when to break the old rules and use something like:
- a star or snowflake schema
- a hierarchy or network
- dynamic attributes
- materialized query tables
- triggers
- etc
all takes experience and knowledge. One thing that I often do when in a spot like yours - is to define a good object model at least - then use that as the basis for persistence apis. These apis can be stored procedures (best case) or views (worst case). Then (assuming you have a dba that will be owning this code) the dba is free to remap that api layer to the physical tables. Change is inevitable, but at least this way you've got one role that has access to all the code that needs to change, and hopefully your api interfaces will encapsulate the code well.
Unfortunately, stored procedure use is on the decline. But as long as you take a cautious approach with them they're not a bad thing.
You mentioned 3 different objectives:
- kill monsters
- socialize
- create a story
Of which only the first you stated that a computer does it better than a table top. But really, the computer only helps with the logistics of getting together. It hardly does as good a job of killing as you can do on the table top - where subtles of movement, position, psychology, exhaustion and various forms of trickery are so poorly handled.
So, yeah if you just want to rack up some bland kills the computer kicks butt. But if you want to do something really memorable with friends in a game - do it over a tabletop.
For example, I just recently got together with some good friends this summer. We're all about 40 now, used to play heavily between the ages of 16 & 25. We laughed so hard at those shared adventures. The best was the story of the "Chimes of Hunger" - in which the chimes are run - and every character in the room is reduced to a single-minded eating machine - mindlessly eating whatever he has on hand. The GM asked everyone to show him where, on their character sheet, they had marked food. One player had none, but asserted that of course he was carrying food. This character was about a tenth level mage with a homonculous. The GM peeked over his GM Screen and said "homonculous is practically chicken", a reference to an olds buggs bunny. Anyhow, while the homonculous was fighting to get away the mage ate him, inflicting a good dozen points of damage on himself, and leaving himself covered with blood. We thought it was very funny at the time, but apparently the character's owner didn't. He was the only one of all of us that couldn't remember this adventure at all.
Anyhow, still enjoying some of those memories on occasion twenty years later.
> I, for one, would not bring my violin with me to launch rockets with
well, you got me there - there are certainly instruments I wouldn't want my kids to travel with - pipe organs, xylaphones, even violins. Still, kids shouldn't be so specialized - drag the ukelele, tenor guitar, etc with you then. That's what they used to often do way back in the day.
> More generally, I don't understand why playing a gameboy while bored implies anything about what other activities
> one engages in while not bored.
The problem is that most children lack the discpline to stop eating candy or playing video games - and so will do them to the exclusion of everything else given the opportunity. Kind of like a rat eating crack. As a parent you typically don't want a kid who at the age of eighteen is an academic and social failure and who's only skill is in some rpg. Not a good way to start as an adult. So, you've got to make sure that they moderate these activities. A few hours a week is harmless in my opinion - but there are millions of kids out there playing 20-40 hours a week.
> Maybe it'd be a good thing to expose your kids to those and let them carry on their own ideas after they're done?
Yeah, that's the big question. But if you ever have kids you'll find that a close temptation becomes a constant source of problems - either you completely give in to the kids or you face frequent debates. Many people break down, wish they didn't have a television, and live with it on every night.
It's so much easier to just get rid of it. And once you get rid of it for 6+ months it's almost impossible to go back. After listening to public radio, books on tape, reading books to each other, etc - the stupidy of almost all programming becomes intolerable.
But I'm not a nazi about it - my kids watch television occasionally when at their friend's home. And I don't care in the least - it keeps it down to an hour a week or so.
> I could very easily be misinterpretting something here, but were the kids there of their own volition, or did
> someone (say, a father) drag them out there for some "fun outdoors"?
No idea. But kids often prefer, if given the chance, to do nothing but play video games, eat candy bars, skip school then maybe play with matches and guns. Your job as a parent is to guide your kids away from that kind of behavior and help them eventually become adults.
Part of becoming an adult involves doing more than playing a gameboy. Actually getting outside. Perhaps those kids were bored - though if they didn't have the opportunity to play the 5000th hour on a gameboy they might actually have noticed some cool in the park or field. Might have played tag, might have found an interesting ant hill to check out, etc, etc.
Quite a few people are rightfully, in my opinion, concerned about the overscheduling of children by their parents. These kids are in so many different activities that they don't get a chance to become parents. Well, that's the over-achieving parent's trap. I suppose the under-achieving parent's trap is to let the kid overschedule their life with nothing more that gameboy - so that videogames crowd everything else out like weeds.
> Sure, I imagine it'd be fairly straight forward to sing or play music, but wouldn't that be a little awkward/embarrassing at a venue like that?
I'm not the original poster, so I can't say for sure. But there's nothing awkward about kids playing music.
> Did it ever occur to you that perhaps those kids playing their video games just weren't interested in launching rockets?
Oh sure, they aren't interested in launching rockets. Or reading a book. Or playing a musical instrument. Or singing. Or hiking. Or playing a game of baseball with the neighbor kids. Or any else almost.
They are interested in playing a simple game for thousands of hours - after hitting diminishing returns in the first two.
As a parent, in my opinion letting a kid drag around a gameboy and play it for thousands of hours instead of actually living life is the same as letting him skip all meals and eat nothing but candy bars instead.
> Now, my kids have cable, computer with the net and half a dozen consoles. I work on limiting it, but it is tough.
Yep - constant availability of gaming consoles, flash and other online games and television is like putting racks of candy bars all over your house. Six year olds shouldn't be eating a non-stop diet of chocolate and fried potatoes all day, nor should they be sitting on their asses playing video games and watching television all day. The challenge is that in some communities (especially suburbs) a couple of hours of this kind of play a day is the norm. And in that situation restricting your kids has got to be tough.
But I know of many households that restrict kids to four or less hours of electronic games & television a week. In my household we ditched television broadcasting (cable, dish, antenna) fifteen years ago and have *never* regretted it. We rent dvds a couple of times a month, that's it.
Sure, it means that kids don't get to watch their MTV when they were eight years old, but they did read "The Wind in the Willows" instead, they did learn how to play musical instruments, juggle, explore the local trails, wrestle, play with the dog, play with their friends, etc. All far better ways for kids to spend their time.
> I'd like links to the benchmarks that show that C's io is as slow python's and also you to point out the processes that are only 5% faster in C.
= 3
http://www.osnews.com/story.php?news_id=5602&page
The 5% is what I found to be the difference when making system calls. The overhead of invoking python is pretty trivial when you're then executing a ten second external system check. But even beyond that easy comparison - many of the modules are very fast - traversing directories, etc, is very quick.
> It says they are maybe 1.7 times faster than CPython, which is not that great, because CPython is incredibly slow and things
.net or vb. It might be slower, or it might not. Either way it'll probably run fine on a 1ghz desktop.
> like Psyco can give pretty big speedups (say 10 to 100 according to their website).
> It seems fundamentally impossible to make a language like Python or Ruby fast.
fast or slow are relative and somewhat meaningless terms.
I use python to transform tens of millions of rows of data every day in the running of a data warehouse. C would be faster for most of these processes, but not all - since python's io speed is the same as C's and these are io-heavy applications. But i'll trade a lot of that speed for the advantages in maintainability & speed of development that I get with python.
Likewise, small scripts that we use for monitoring disk space, monitoring processing queues, checking for data quality issues, etc - are fine in python. C would be faster, but probably only 5% most of the time. So, who cares about the difference between 1.0 second and 0.95 seconds?
And if I felt like writing a small gui on a windows box I'd prefer to work with python over
> I know I'm ranting out of proportion to your specific post, and I apologize. It's just that this seems to be happening all
> the freaking time lately, no matter how tenous the connection to the topic is, and it's driving me batty.
Unfortunately, it is happening more & more - just like public expressions of anger for the government happened more & more in the 60s. And they're now angry enough they there opinions have started spilling over in non-political discussions.
Americans weren't allowed to criticize the government's policy for a couple of years. No laws against it - just a ton of social pressure following the administration's direction at the top. Even thoughtful questioning of our motives would be met with criticisms of "sedition", "hating america", and "not supporting the troopos". You instantly made people nervous and worried or angry if you stated wacky things like "but wait, the UN chief weapon inspector said there are no WMDs in Iraq, why are we going in again?".
So, quite a few people in this country are freaking out, and for good reason: the controlling party has in just four years:
- thrown the US reputation into the toilet
- shown a clear interest in rolling back the bill of rights as much as possible
- put us deeply into dept and at China's mercy
- encouraged vast outsourcing of jobs
- gotten us embroiled in a war that we cannot win - thousands of miles from home
- been too bogged down in their oil war to address actual terrorist nations
People don't want to be drafted to fight iran, they don't want their neighbors getting killed fighting someone else's oil war, they don't want to see our people torturing civilians in other countries, they don't want their jobs oursourced overseas, they don't want to be unable to travel, they don't want to attract terrorists to their communities, and they don't want big brother monitoring them.
I didn't expect to see any anti-bush sentiments on this thread either. But I'm not surprised when they appear. Wait another year - when iraq is really up in flames - you'll see even more.
Hmm, sounds like you mean the opposite of Not-Invented-Here: it typically means that an organization doesn't respect a solution unless it was invented there.
But I think I agree with your thoughts: I've seen far better productivity come from small teams creating simple solutions than from much larger teams attempting to implement huge commercial platforms.
> That one was to actually vary the pay of the teachers and their tenure based upon the results of the standardized test progress of their students.
Right - think carefully about that just a little bit. If someone's pay is directly based on test scores, then... the teacher will want to get every kid that isn't promising kicked out of their class. Slightly slow child? Poor english? Minor health issues? Whatever, they're out of there! No time for charity - there are high scores to be earned!
> Beginning about the 4th grade students should no longer carry books. They should be issued laptops.
Great idea. And who's going to be paying to replace these laptops every year? You don't actually think that laptops are going to last more than a year (if that) in the hands of 9 year olds, do you?
> Schools are not under funded. They are grossly over funded.
> Teachers are not under paid except in their early career years.
> You get what you pay for.
You've got a bug in your code here fella.
> A note to the mods. This is the most on topic least troll and most informative listing you have ever read on this topic.
No, this is the one of the least on topic and least informative postings today. Your thinking is cloudy and you're ranting half of the time. You should probably start taking your meds again.
You claim responsibility for the No Child's Behind Left program. That's just precious - this is the program by which every school will eventually be a "failing school". See, eventually every school runs out of progress, every school will fail to get good grades out of some tiny minority sliver, and every school will fail to get 100% of their students over the bar.
I know the critics of the public schools are disgusted at the poor performance that some of them deliver. Then again - look at the poor performance that these critics deliver:
- poor grammer (see above posting)
- inability to pass tests they require of high school students (see Colorado Governor Owen's big testing failure)
- inability to work with numbers (see how NCLB will cause all schools to fail within next 5-7 years)
Testing is a good thing, no argument there. But giving testing numbers to the numerically illiterate (whether it is pointy-haired bosses in corporate america or ranting anti-school libertarians) just doesn't work. Here's a suggestion - lets take a look at the parents role a bit, ok? Why do we expect teachers to work miracles with kids that are allowed to play videogames, watch television, and play sports 4-5 hours a day? Where are the parents of all these poor-performing children?
>> A moment's panic may well cost someone thirty million.
> Christ. With that kind of debt one might as well flip out, blow up the RIAA's HQ and disappear into the middle-east or China. $30 million!
Look at how little companies *usually* end up paying when they are guilty of causing them bodily harm. Prisoners that were used for medical testing that got really screwed up back in the 70s received something like $40k! Compare that to $30 million. Clearly, taking those songs was 750 times as bad as screwing some persons health for the rest of their life.
Shit at this point I'd be inclined (in perhaps a rash moment of anger) of demanding that they raise it to $30 billion. Why not? It isn't like I'm going to come up with $30 million either - might as well make it totally assinine. Also raises the value of a copied song to $750,000,000 - or 3,750 times the value of a person's health. Really puts what is valued in our country into a useful perspective.