On one overly spammed account, 23 people are on the white list, I get no spam.
And how do you expect to receive a surprise email from a college buddy you lost touch with 10 years ago? White lists only work if you have a clearly defined set of people who you wish to receive email from - they do not allow the possibility that someone will legitimately send you an email without you having added that person's email address to your list.
If you get down to where it's conveniently stuffed waaay at the bottom (wow, seems the NYT likes being sensationalist as well), it says that all but $141,000 have been resolved after the investigation.
If you re-read the article, you will find that PwC (the external auditors) 'found $3.7 million of unreconciled purchases.' The laboratory claims that all but $121,000 worth of purchases have been reconciled. Until an external auditor verifies these claims, I'd be more willing to believe the larger figure. To me it sounds like the laboratory are trying to cover up the rorting that has been going on.
And I recall in the mid-80s when "clear cola" was launched, there were several different brands of soft drink available in Australia in similar, clear plastic "cans". They were kinda cool...
Re:They keep going on about
on
Dow vs. Parody
·
· Score: 2
Shouldn't parodies be allowed?
Parodies should be allowed, but the supposed parody web site did not look like a parody to me. It was trying to pass itself off as the real Dow site, which is not allowed. Fraud. Deception. etc...
This could take anywhere from a couple of weeks to a couple of years. Your best bet would be to find a programmer who has done something similar and feed him your specs. As others have mentioned, double his estimates to come up with a more likely figure.
Without reasonably detailed specs no-one here can give you any meaningful answer...
You may find that the "truth in advertising" regulations might have come into effect after several companies were caught trying such tricks. There have been many documented cases where advertisers have been even more deceitful than simply putting marbles in a bowl of soup.
"I save my mod points to mod up people who reply to my journal"
Damn, there seems to be morons thinking up new ways of abusing the moderation system every day.
Did you even look at his journal? Only three entries - the last dated August 20 2002. Ain't gonna be easy to post a reply to his journal to score the "free" karma. This discussion has been archived. No new comments can be posted.
It just means that the first 24 people didn't thorougly read the article.
But this is slashdot. You're not trying to tell me that we're meant to read the articles are you?
Re:RAID can mean different things...
on
IDE RAID Examined
·
· Score: 5, Informative
I don't quite understand where this Inexpensive crap came from. RAID was around long before IDE RAID controllers started showing up and of course SCSI RAID arrays almost always use very expesive disks. It's Redunant Array of Independent Disks, always has always will be.
It probably comes from the original reseach paper... A case for redundant arrays of inexpensive disks in the Proceedings of SIGMOD International Conference on Data Management, 1988. (Pages 109-116.) SCSI drives were an inexpensive option compared to other storage technologies that offered high performance and fail over safety.
Over time the acronym expansion was changed to become "redundant array of independent disks" as RAID become more popular (and affordable) for smaller systems.
Then Sun should be trying to get MS to create a JVM that complies with the standards rather than bitching about them not bundling any JVM at all. Sun wants MS to distribute Sun's JVM, which is what I am opposed to. They got their wish when they forced MS to stop bundling the "enhanced" (according to MS - not me, IMHO MS has no idea what a standard is they just pick and choose what bits they will implement) JVM which MS did. Now Sun is bitching about there being no JVM included in Windows. Let the damn users download Sun's JVM from Sun, or MS's version, or any other damn version they want. Don't make MS include one implementation over another - make them provide links at all the different JVMs available and let the users decide.
Not really - first Sun wants MS to stop distributing a "broken" JVM. That is fine, and I support them in their efforts to a point. However, now they are crying because MS isn't distributing any JVM at all and want MS to distribute SUN's JVM, instead of a JVM.
Sun have changed their cries from "your JVM is broken, stop distributing it" to "you stopped distributing a JVM you must include ours". Other companies have JVMs for Windows, why not include one of them? Why must it be Sun's implementation? Why not force MS to create a compliant, non-broken JVM?
Sun didn't like Microsoft including their own version of the Java VM with their OS so Sun took MS to court to stop them doing so.
Microsoft removed the JVM from their OS, as per Sun's demands.
When a user goes to a web-site that uses Java, the user has to download a JVM.
Now Sun is complaining that MS doesn't include a VM with their OS? Sun wants MS to distribute Sun's JVM with the OS? What about other vendors that produce JVMs?
So long as programming REQUIRES GREAT SKILL, it will take many hours and many inefficient projectes to train programmers properly. No matter how many books you've read, no matter how many pet projects you have on the side, it's rare to get any REAL programming experience until you're in the trenches, fighting requirements and deadlines.
However, merely having been "in the trenches" does not make a person a good quality developer. I have worked with (and unfortunately hired) some developers who have very long, impressive resumes, but couldn't write code to save their lives. Some of these guys have been simply attrocious and haven't even understood simple data structures or basic algorithms. OTOH I have worked with some guys who have only had 6 months "experience" who have been absolute geniuses and are able to solve the most intricate problems with clean, well thought out code in a relatively short period of time.
One developer who had 3+ years of "experience" got promoted to team lead and proceeded to screw up a $750K project followed by a $3M+ project (and lost the company at least two multi-million dollar per year clients in the process).
Some guys seem to have a natural ability, some guys work hard to develop their abilities and some just have no hope. I've worked with all three categories and unfortunately the naturals are few and far between, and the rest are split evenly between the other two camps.
First off its C....not c. Secondly you can't make that claim.
The poster was right - it is c. c is the speed of light, and as we all know, nothing can go faster than that, so it follows that Java is slower than c.
I have never seen a single project that was succesful because it followed a process.
Succesful projects seem to grow their own process.
It is true that porojects are not successful because they followed a process, but successful projects are generally ones that have followed a process. A good process which is followed by the entire team certainly helps. Projects can fail for a variety of reasons, but most of the failed projects I have seen have either not had any processes at all or the processes have not been followed by everyone in the team.
Processes do no have to be heavy-weight affairs like the complete RUP or CMM, but there should be something in place. Lightweight methodologies and processes are perfectly suitable for a large number of projects.
The biggest problem I have found has been getting the project team to follow the process that has been chosen. Developers generally don't like documentation, yet documentation is crucial for projects of any reasonable size. Specifications, design documentation, interfaces, data models, comments in code - these are the things that I had the most trouble getting developers to write, yet when the developers were working on someone else's code they screamed if any of these things were missing.
Having some simple rules about code formatting and keeping documentation up to date (no documentation at all is better than documentation that does not match the finished code!) and rules governing who checks code in/out helps move the project along and keeps everything on track.
Having said that, I think a lot of the problems addressed are real - I think they get solved by people, not processes though.
Yet processes can help the people solve the problems! Succesful projects seem to grow their own process: usually in response to problems casued by a lack of process (or faulty process) initially.
What the parent poster didnt realize is that when youre dealing with 3 tables, one of which has 52 million records in it, joins are the non-option. 4 day queries are not permitted.
Just a quick check of one our systems shows a table with 1.5 million records. Not a large table, but not trivial either. Due to the nature of the data and the structure of the database, it is necessary to execute queries containing 10-12 self-joins routinely. One report requires 16 joins in total. The slowest of these queries takes about 30 seconds, most are well under 2 seconds.
Sure, we could denormalise our data to make the queries simpler (multi-table joins scare DBAs for some reason) but we would lose a lot of the flexibility that our design affords us. We have no need to denormalise this database because we wrote more efficient queries and have paid careful attention to our indexes.
The largest table in our production database is under 10 million rows. We have simulated much larger sizes in our test environments though, and found our joins are not a problem. The performance with 10 times the data is still well within acceptable range for our requirements. (Most stored procedures execute under 5 seconds with this size.) If you have problems with a three table join, head back to school and learn how to do it properly!;-)
You say you are as religious as possible about indexing, but what about the structure of your queries? Are you indexing the right columns? You are using temp tables? Try getting rid of them - in many cases they slow things down. Make sure your queries are sargable. Make sure you are limiting the intermediate result-sets to be as small as possible. Understand the query execution plan. Make sure your server has enough memory! Are your server's settings tuned for your particular application? There are many, meny things to pay attention to, but you should not be having any problems with a 52 million row table in a three-table join!
When you start doing real joins with 10 or more tables (each with >1 million rows) then we'll talk about how to denormalise the data to improve performance.
It really depends on what data you are storing. How much data, how critical, what are you doing with it, etc...
Sometimes, Excel is good enough. Or XML. Or plain text files. Or a custom file format. etc...
If the people working on your project have only ever worked with databases, they will want to use databases for everything. Most stuff will fit into a database, but sometimes it is not apprpriate to do so - as other have mentioned here, storing images in a database is not always a good idea, but you would probably want to store the location of the images in a database.
If you need to be able to ship the data around to different machines/offices/clients/over the net/etc, then maybe an XML file will be best. Custom file formats may be appropriate in some cases too. (Though I'd lean towards a more open file format.)
But please, for the love of God, don't listen to those moron college profs who say normalization is key. That's all good and fun until you have a million records in each of four tables you need to join to provide a solution. That, or your data model charts take up 300 sq. ft of wall space.
Don't normalise your database at your own peril!
Learn how to properly normalise a database (3NF is usually good enough) and then learn how to write decent queries and tune your indexes appropriately. In some circumstances it may be worthwhile caching some data in extra tables which are refreshed periodically. In even rarer circumstances it may be necessary to denormalise the database, but always normalise it and only denormalise if you encounter some intractible performance issues.
We have some incredibly complex queries at my current client and we have managed to gain performance improvements by re-writing certain queries. Splitting complex queries up and using temp tables, derived tables, sub-queries, etc can help and you will have less of a problem with data anomalies which can creep into a denormalised structure.
Sometimes, well-thought out denormalisation can make a huge, positive impact on an application. Yes, it can be difficult to make sure you don't have any anomalies in your data, but with a rigourous design and development methodology, these problems can be minimised.
Don't denormalise for the sake of denormalising - the trick is to know when to break the rules and to do so very carefully.
Denormalisation is only one tool that can be used to improve the performance of a system, and of course, other options like more memory, faster CPUs and better code should be addressed first.
The answer to your question depends on a lot of variables. As others have mentioned, make sure you throw plenty of RAM into the server and that your disks and network are nice and speedy too. You then need to assess your particular requirements. What is the current load like (if you are running in a production environment already) or what is the simulated load in your testing environments? How well written are the stored procedures? How many users? What usage patterns do you expect? What response times are acceptable? How much data? How well-structured is the database? etc...
Poorly written code can kill a server far more quickly than slow processors or less CPUs. Make sure the developers know what they are doing and are not generating too many round-trips to the database and are writing efficient code. A major problem with my current project is that most of the stored procedures were written by C++ and VB coders who did not understand the set-based nature of SQL. Re-writing some of their stored procedures (removing their reliance on cursors for example) achieved massive performance improvements - in one case taking a 12-16 hour batch job down to 25-30 minutes, and bringing most 1 minute or so queries to under a second. A bigger/faster box would not have helped much in those situations.
Head on over to www.sql-server-performance.com for some help with finding your answer, though my suggestion is that if cost is such a big factor, buy a server with 2 really fast CPUs, but make sure it is expandable and you can then add another 2 CPUs to it if you need extra performance at a later stage.
Whatever you do though, make sure you have plenty of RAM.
Slashdot Mirror
So I'm still waiting for you to point one, ONE original idea by Microsoft.
;-)
Bob?
On one overly spammed account, 23 people are on the white list, I get no spam.
And how do you expect to receive a surprise email from a college buddy you lost touch with 10 years ago? White lists only work if you have a clearly defined set of people who you wish to receive email from - they do not allow the possibility that someone will legitimately send you an email without you having added that person's email address to your list.
If you get down to where it's conveniently stuffed waaay at the bottom (wow, seems the NYT likes being sensationalist as well), it says that all but $141,000 have been resolved after the investigation.
If you re-read the article, you will find that PwC (the external auditors) 'found $3.7 million of unreconciled purchases.' The laboratory claims that all but $121,000 worth of purchases have been reconciled. Until an external auditor verifies these claims, I'd be more willing to believe the larger figure. To me it sounds like the laboratory are trying to cover up the rorting that has been going on.
i am by no means a conousour, but i can taste a diffence between caned, and plastic bottled coke, and i like the caned one better.
;-) Thank God they have started selling Coke in glass again!
But Coke in glass bottles tastes even better still!
And I recall in the mid-80s when "clear cola" was launched, there were several different brands of soft drink available in Australia in similar, clear plastic "cans". They were kinda cool...
Shouldn't parodies be allowed?
Parodies should be allowed, but the supposed parody web site did not look like a parody to me. It was trying to pass itself off as the real Dow site, which is not allowed. Fraud. Deception. etc...
This could take anywhere from a couple of weeks to a couple of years. Your best bet would be to find a programmer who has done something similar and feed him your specs. As others have mentioned, double his estimates to come up with a more likely figure.
Without reasonably detailed specs no-one here can give you any meaningful answer...
A quick search on Google will find this and this andthis as three of the top four results...
You may find that the "truth in advertising" regulations might have come into effect after several companies were caught trying such tricks. There have been many documented cases where advertisers have been even more deceitful than simply putting marbles in a bowl of soup.
Every time I click the link in the article, I get http://www.shreveaudio.com...
"I save my mod points to mod up people who reply to my journal"
Damn, there seems to be morons thinking up new ways of abusing the moderation system every day.
Did you even look at his journal? Only three entries - the last dated August 20 2002. Ain't gonna be easy to post a reply to his journal to score the "free" karma. This discussion has been archived. No new comments can be posted.
It just means that the first 24 people didn't thorougly read the article.
But this is slashdot. You're not trying to tell me that we're meant to read the articles are you?
I don't quite understand where this Inexpensive crap came from. RAID was around long before IDE RAID controllers started showing up and of course SCSI RAID arrays almost always use very expesive disks. It's Redunant Array of Independent Disks, always has always will be.
It probably comes from the original reseach paper... A case for redundant arrays of inexpensive disks in the Proceedings of SIGMOD International Conference on Data Management, 1988. (Pages 109-116.) SCSI drives were an inexpensive option compared to other storage technologies that offered high performance and fail over safety.
Over time the acronym expansion was changed to become "redundant array of independent disks" as RAID become more popular (and affordable) for smaller systems.
Some references: here, here and here
Then Sun should be trying to get MS to create a JVM that complies with the standards rather than bitching about them not bundling any JVM at all. Sun wants MS to distribute Sun's JVM, which is what I am opposed to. They got their wish when they forced MS to stop bundling the "enhanced" (according to MS - not me, IMHO MS has no idea what a standard is they just pick and choose what bits they will implement) JVM which MS did. Now Sun is bitching about there being no JVM included in Windows. Let the damn users download Sun's JVM from Sun, or MS's version, or any other damn version they want. Don't make MS include one implementation over another - make them provide links at all the different JVMs available and let the users decide.
Not really - first Sun wants MS to stop distributing a "broken" JVM. That is fine, and I support them in their efforts to a point. However, now they are crying because MS isn't distributing any JVM at all and want MS to distribute SUN's JVM, instead of a JVM.
Sun have changed their cries from "your JVM is broken, stop distributing it" to "you stopped distributing a JVM you must include ours". Other companies have JVMs for Windows, why not include one of them? Why must it be Sun's implementation? Why not force MS to create a compliant, non-broken JVM?
Sun didn't like Microsoft including their own version of the Java VM with their OS so Sun took MS to court to stop them doing so.
Microsoft removed the JVM from their OS, as per Sun's demands.
When a user goes to a web-site that uses Java, the user has to download a JVM.
Now Sun is complaining that MS doesn't include a VM with their OS? Sun wants MS to distribute Sun's JVM with the OS? What about other vendors that produce JVMs?
Nah he'll post 10 pages about how he set his camcoder to time-lapse mode (!) so we can see the grass growing at high speed.
Why high speed? That would spoil all the fun.
Personally, I can't wait for his next 20k article on how to set up a wet-paint-drying-cam.
So long as programming REQUIRES GREAT SKILL, it will take many hours and many inefficient projectes to train programmers properly. No matter how many books you've read, no matter how many pet projects you have on the side, it's rare to get any REAL programming experience until you're in the trenches, fighting requirements and deadlines.
However, merely having been "in the trenches" does not make a person a good quality developer. I have worked with (and unfortunately hired) some developers who have very long, impressive resumes, but couldn't write code to save their lives. Some of these guys have been simply attrocious and haven't even understood simple data structures or basic algorithms. OTOH I have worked with some guys who have only had 6 months "experience" who have been absolute geniuses and are able to solve the most intricate problems with clean, well thought out code in a relatively short period of time.
One developer who had 3+ years of "experience" got promoted to team lead and proceeded to screw up a $750K project followed by a $3M+ project (and lost the company at least two multi-million dollar per year clients in the process).
Some guys seem to have a natural ability, some guys work hard to develop their abilities and some just have no hope. I've worked with all three categories and unfortunately the naturals are few and far between, and the rest are split evenly between the other two camps.
First off its C....not c. Secondly you can't make that claim.
The poster was right - it is c. c is the speed of light, and as we all know, nothing can go faster than that, so it follows that Java is slower than c.
8-)
I have never seen a single project that was succesful because it followed a process.
Succesful projects seem to grow their own process.
It is true that porojects are not successful because they followed a process, but successful projects are generally ones that have followed a process. A good process which is followed by the entire team certainly helps. Projects can fail for a variety of reasons, but most of the failed projects I have seen have either not had any processes at all or the processes have not been followed by everyone in the team.
Processes do no have to be heavy-weight affairs like the complete RUP or CMM, but there should be something in place. Lightweight methodologies and processes are perfectly suitable for a large number of projects.
The biggest problem I have found has been getting the project team to follow the process that has been chosen. Developers generally don't like documentation, yet documentation is crucial for projects of any reasonable size. Specifications, design documentation, interfaces, data models, comments in code - these are the things that I had the most trouble getting developers to write, yet when the developers were working on someone else's code they screamed if any of these things were missing.
Having some simple rules about code formatting and keeping documentation up to date (no documentation at all is better than documentation that does not match the finished code!) and rules governing who checks code in/out helps move the project along and keeps everything on track.
Having said that, I think a lot of the problems addressed are real - I think they get solved by people, not processes though.
Yet processes can help the people solve the problems! Succesful projects seem to grow their own process: usually in response to problems casued by a lack of process (or faulty process) initially.
What the parent poster didnt realize is that when youre dealing with 3 tables, one of which has 52 million records in it, joins are the non-option. 4 day queries are not permitted.
;-)
Just a quick check of one our systems shows a table with 1.5 million records. Not a large table, but not trivial either. Due to the nature of the data and the structure of the database, it is necessary to execute queries containing 10-12 self-joins routinely. One report requires 16 joins in total. The slowest of these queries takes about 30 seconds, most are well under 2 seconds.
Sure, we could denormalise our data to make the queries simpler (multi-table joins scare DBAs for some reason) but we would lose a lot of the flexibility that our design affords us. We have no need to denormalise this database because we wrote more efficient queries and have paid careful attention to our indexes.
The largest table in our production database is under 10 million rows. We have simulated much larger sizes in our test environments though, and found our joins are not a problem. The performance with 10 times the data is still well within acceptable range for our requirements. (Most stored procedures execute under 5 seconds with this size.) If you have problems with a three table join, head back to school and learn how to do it properly!
You say you are as religious as possible about indexing, but what about the structure of your queries? Are you indexing the right columns? You are using temp tables? Try getting rid of them - in many cases they slow things down. Make sure your queries are sargable. Make sure you are limiting the intermediate result-sets to be as small as possible. Understand the query execution plan. Make sure your server has enough memory! Are your server's settings tuned for your particular application? There are many, meny things to pay attention to, but you should not be having any problems with a 52 million row table in a three-table join!
When you start doing real joins with 10 or more tables (each with >1 million rows) then we'll talk about how to denormalise the data to improve performance.
8-)
It really depends on what data you are storing. How much data, how critical, what are you doing with it, etc...
Sometimes, Excel is good enough. Or XML. Or plain text files. Or a custom file format. etc...
If the people working on your project have only ever worked with databases, they will want to use databases for everything. Most stuff will fit into a database, but sometimes it is not apprpriate to do so - as other have mentioned here, storing images in a database is not always a good idea, but you would probably want to store the location of the images in a database.
If you need to be able to ship the data around to different machines/offices/clients/over the net/etc, then maybe an XML file will be best. Custom file formats may be appropriate in some cases too. (Though I'd lean towards a more open file format.)
But please, for the love of God, don't listen to those moron college profs who say normalization is key. That's all good and fun until you have a million records in each of four tables you need to join to provide a solution. That, or your data model charts take up 300 sq. ft of wall space.
Don't normalise your database at your own peril!
Learn how to properly normalise a database (3NF is usually good enough) and then learn how to write decent queries and tune your indexes appropriately. In some circumstances it may be worthwhile caching some data in extra tables which are refreshed periodically. In even rarer circumstances it may be necessary to denormalise the database, but always normalise it and only denormalise if you encounter some intractible performance issues.
We have some incredibly complex queries at my current client and we have managed to gain performance improvements by re-writing certain queries. Splitting complex queries up and using temp tables, derived tables, sub-queries, etc can help and you will have less of a problem with data anomalies which can creep into a denormalised structure.
Sometimes, well-thought out denormalisation can make a huge, positive impact on an application. Yes, it can be difficult to make sure you don't have any anomalies in your data, but with a rigourous design and development methodology, these problems can be minimised.
Don't denormalise for the sake of denormalising - the trick is to know when to break the rules and to do so very carefully.
Denormalisation is only one tool that can be used to improve the performance of a system, and of course, other options like more memory, faster CPUs and better code should be addressed first.
Doesn't Google run off of a huge Excel spreadsheet?
No, they use pigeons!
The answer to your question depends on a lot of variables. As others have mentioned, make sure you throw plenty of RAM into the server and that your disks and network are nice and speedy too. You then need to assess your particular requirements. What is the current load like (if you are running in a production environment already) or what is the simulated load in your testing environments? How well written are the stored procedures? How many users? What usage patterns do you expect? What response times are acceptable? How much data? How well-structured is the database? etc...
Poorly written code can kill a server far more quickly than slow processors or less CPUs. Make sure the developers know what they are doing and are not generating too many round-trips to the database and are writing efficient code. A major problem with my current project is that most of the stored procedures were written by C++ and VB coders who did not understand the set-based nature of SQL. Re-writing some of their stored procedures (removing their reliance on cursors for example) achieved massive performance improvements - in one case taking a 12-16 hour batch job down to 25-30 minutes, and bringing most 1 minute or so queries to under a second. A bigger/faster box would not have helped much in those situations.
Head on over to www.sql-server-performance.com for some help with finding your answer, though my suggestion is that if cost is such a big factor, buy a server with 2 really fast CPUs, but make sure it is expandable and you can then add another 2 CPUs to it if you need extra performance at a later stage.
Whatever you do though, make sure you have plenty of RAM.