One of the wisdoms I got from my education at HP Germany was "quality is tested into hardware". And with any real software, it is exactly the same. Or maybe should I say "even worse" ? The biggest investment is always testing, bug fixing, validation programs, test vectors, test vector generators, code reviews and so on. Initially the HP MPE operating system was very unreliable, but decades of bug fixing and refining made it into one of the most reliable operating systems.
Muharg. In Java you can't have arrays of structs. You can have an array of references which points to your 25 million individually allocated heap objects. Which is both a runtime and a massive storage overhead. Plus, it destroys your cache efficiency.
So, get off my lawn and stop saying stupid things in the presence of grown-ups !
Tell that to the armies of OLAP developers/analysts. They believe their RBBMS nail will answer all analytics questions. Just recently they have rediscovered batch-mode analytics and call it MapReduce. See TeraData Aster. How do you need ACID for a read-only database ???
Oh, yeah, score a point by making a different argument. Seymour Cray literally owned the supercomputer business (NSA, LLNL and that type of thing) until he died. Just recently IBM does some sort of supercomputing business again, but these are NOT mainframes. IBM supercomputing are massive clusters of Power CPUs. If Seymour Cray still lived, I am sure IBM would NOT be in supercomputing.
Well, I do think we have to differentiate between implementation quality, conceptual quality and manufacturing quality. The first one is definitely not very good on IBM mainframes. They are shipping products with much more bugs than Intel, HP or their own Power Unix people do. But the mainframe concept of a large centralized machine with lots of failure detection, failover, management, virtualization, massive I/O, clustering and so on is solid.
Just looking at the hardware and software cost is very short-sighted. Factor in operating, management, custom software development cost and the picture looks different. For example, if you don't have proper job control facilities, a single user can monopolize the system by accident and affect lots of different user groups/applications on the same system. Raw horsepower is only one of many important aspects.
As a qualifier, maybe the mainframe IBMers are especially shoddy. We never had these problems with their Power machines. These were actually excellent Unix machines - fast, reliable, very high quality. But DB/2 and mainframe, not so much. Thinking about it, maybe it indeed is a deliberate tactic to force customers into expensive support contracts.
Maybe you just read what I write: IBM delivers shoddy quality which must be fixed by their field engineers. MS does have their fair share of bugs, but they are definitely not as bad as IBM. So does Intel - their processors at least don't crap out on the integer operations.
IBM is a company dominated by the sales and finance guys. Technology is very much a nuisance to them, which they only need to make $$. At MS and Intel, there seem to be some technologists at the critical posts, even though the MBA-In-Chief of MS appears to "change" that, recently.
The difference is that I can get the typical MS product running in a reasonable manner without the aid of an MS field engineer. For example, when DB/2 competes against SQL Server, developers will disregard DB/2 quite early when they can't get proper connect times without the help of the IBM field engineer. Maybe DB/2 is actually better than SQL Server on the long run, but there won't be a long run because initially it looks like a very bad product.
If they ever let me close to a Cobol system, I am sure it will be infested with macro-based generic data structures pretty soon. There is a need for a Cobol Standard Library (of my own invention), I am quite positive !
Not entirely correct. Thomas Watson once decided to take out CDC and build a supercomputer. IBM would badmouth CDC "in support of our future machine". It turned out the single guy Seymour Cray beat the crap out of the army of IBM engineers.
Mr Watson was pissed but from then on focused on mainframes, not supercomputers.
Always good to see that a single capable and determined man can stand up to the sleazy multi-billion corporations and win !
At this point, Google CAN afford ccNuma. Facebook, too. Apparently, it still is not cost-efficient. Or there simply is no competitive pressure to go ccNUMA.
Actually, MapReduce is all the rage with companies like TeraData, who had an affiliation with BigIron in the past.
Actually, my impression of IBM is that they deliver quite crappy technology on a regular basis, because "we can fix that later on". Cynical colleagues suggested this being a business strategy with the objective of selling support. At a former employer we discovered a broken integer division instruction. When demonstrated to the IBM field engineer, they finally agreed to fix it by means of a microcode update.
Just recently I fought with DB/2 and only the field engineer could help this time, again.
IBM was always very much sales/service oriented and they still cannot compete with the "shrink-wrapped" companies like MS and Oracle. IBM is a relic of the past and will soon be history because it is simply much more efficient to perfect a product once instead of sending a field engineer to each and every customer to clean up the problems they haven't fixed during R&D time.
You prefer that everybody in this industry earns a precarious income and operates dirt-cheap stuff ? No high-end equipment whatsoever ? Yeah, that makes sense if you earn about as much as a McDonalds worker. Unfortunately, I do think your wish will come true.
Regarding IBM mainframes, they are worth their money and if you cannot see the difference to the cheap x86 stuff from Dell, that's your fault. Finally, every skilled factory worker stands in front of 500k Euros/dollars worth of hardware these days. You prefer that 10k$ Dell "server" ??
Well, men use FORTRAN. Real men are not beancounters, they work on some lethal machinery. Semi-real men like me at least use C++ to do beancounting, which they call "statistics".
I don't think that you actually understand what you write. The CPUs are definitely not commodity, even if it would make sense to emulate the S/390 instruction set on x86 CPUs. Which it doesn't from a reliability/serviceability perspective. And "supercomputers" were the "men's" toys to simulate the weather and how to evaporate Russian cities, amongst other belligerent stuff. Mainframe != supercomputer. Mr Watson of IBM once tried to disprove that and got his fingers burned when he tried to compete with Seymour Cray. Actually, his little army of engineers got their fingers burnt when they competed against that single guy Cray.
MapReduce is just the latest form of Batch Processing, with a fancy new name from the Googlers. So in 2005 Google discovered something IBM knew in 1955.
In the hands of a capable professional, objects can be at least as efficient as structures+procedures. Plus, you need a language with real destructors to immediately reclaim memory when you are done. I would argue that properly used classes and objects will improve your program in several dimensions. Think of automatically closed file and database handles when a method returns or the containing object is being destructed.
I currently work on a data analysis application which processes 32Mbyte/s of input data using C++ PER CORE. I tested it to linearly scale to 50 cores. And yeah, it works similar to a record-oriented Cobol program. Full-table scans are not as bad as they might sound. Index-oriented access is actually a very restrictive way of accessing large data sets and CS professionals should break out of the jail called "relational database". Not every lose part requires fixing by nail.
I am now at the middle of my life expectation and growing a bit smarter. And, I talk to a Cobol programmer then and now in the bus home. He works for an insurance company and will probably retire as a Cobol programmer for that company. He is a mathematician and I am a CS guy; I know much more than he about algorithms, C++, templates, macros, databases and whatnot.
But just recently I realized that Map-Reduce and "record-oriented processing" are actually very similar in that they do NOT consume voracious amounts of main memory. Both perform full-table-scans, in database parlance, which has unique advantages over index-based access for many scenarios.
That's all important if your data set is 100 times larger than your main memory. So the mainframers have that capability since 1955 and the C++ guys just discover this in the year 2005 or so ??
Cobol is here to stay for very systematic reasons very few people understand, including those with a CS degree and those developing in Cobol for a very long time. The latter do Cobol simply because it always paid nicely and there is absolutely no end in sight.
The term is "loop reordering". Essentially, you can iterate matrices row-wise or column-wise. If your matrix is stored row-wise, you better iterate it row-wise, or your cache locality will be very bad. Typical use case is matrix multiplication and solving linear equation systems. So that Mr Kuck of the uni of Illinois (now at Intel, still working !) created optimizing compilers which can do impressive things, including estimating how long a typical Fortran program run will take (surely that does not work for all categories of Fortran programs).
In addition to that Mr Kuck developed compilers which would automatically exploit parallelism of "vector" hardware like the machines designed by Seymour Cray. Note that there is no need to explicitly say "parallelize this loop" as you need to do with OpenMP; the compiler can figure that itself !
Again, this proves that new != better. Fortran still beats C++ in numerical processing and that is quite interesting, considering the fact that Fortran is one of the very first programming languages !
Google Mr Kuck and his papers regarding Fortran compilers - it's an impressive part of computer science from the CDC6600 to the present day !
Slashdot Mirror
One of the wisdoms I got from my education at HP Germany was "quality is tested into hardware". And with any real software, it is exactly the same. Or maybe should I say "even worse" ? The biggest investment is always testing, bug fixing, validation programs, test vectors, test vector generators, code reviews and so on. Initially the HP MPE operating system was very unreliable, but decades of bug fixing and refining made it into one of the most reliable operating systems.
Muharg. In Java you can't have arrays of structs. You can have an array of references which points to your 25 million individually allocated heap objects. Which is both a runtime and a massive storage overhead. Plus, it destroys your cache efficiency.
So, get off my lawn and stop saying stupid things in the presence of grown-ups !
I think you meant "Cobol programmers are on average 72 year old, bearded virgins" ?
Typo: must read "RDBMS"
Tell that to the armies of OLAP developers/analysts. They believe their RBBMS nail will answer all analytics questions. Just recently they have rediscovered batch-mode analytics and call it MapReduce. See TeraData Aster. How do you need ACID for a read-only database ???
Oh, yeah, score a point by making a different argument. Seymour Cray literally owned the supercomputer business (NSA, LLNL and that type of thing) until he died. Just recently IBM does some sort of supercomputing business again, but these are NOT mainframes. IBM supercomputing are massive clusters of Power CPUs. If Seymour Cray still lived, I am sure IBM would NOT be in supercomputing.
Well, I do think we have to differentiate between implementation quality, conceptual quality and manufacturing quality. The first one is definitely not very good on IBM mainframes. They are shipping products with much more bugs than Intel, HP or their own Power Unix people do. But the mainframe concept of a large centralized machine with lots of failure detection, failover, management, virtualization, massive I/O, clustering and so on is solid.
Just looking at the hardware and software cost is very short-sighted. Factor in operating, management, custom software development cost and the picture looks different. For example, if you don't have proper job control facilities, a single user can monopolize the system by accident and affect lots of different user groups/applications on the same system. Raw horsepower is only one of many important aspects.
As a qualifier, maybe the mainframe IBMers are especially shoddy. We never had these problems with their Power machines. These were actually excellent Unix machines - fast, reliable, very high quality. But DB/2 and mainframe, not so much. Thinking about it, maybe it indeed is a deliberate tactic to force customers into expensive support contracts.
Maybe you just read what I write: IBM delivers shoddy quality which must be fixed by their field engineers. MS does have their fair share of bugs, but they are definitely not as bad as IBM. So does Intel - their processors at least don't crap out on the integer operations.
IBM is a company dominated by the sales and finance guys. Technology is very much a nuisance to them, which they only need to make $$. At MS and Intel, there seem to be some technologists at the critical posts, even though the MBA-In-Chief of MS appears to "change" that, recently.
The difference is that I can get the typical MS product running in a reasonable manner without the aid of an MS field engineer. For example, when DB/2 competes against SQL Server, developers will disregard DB/2 quite early when they can't get proper connect times without the help of the IBM field engineer. Maybe DB/2 is actually better than SQL Server on the long run, but there won't be a long run because initially it looks like a very bad product.
If they ever let me close to a Cobol system, I am sure it will be infested with macro-based generic data structures pretty soon. There is a need for a Cobol Standard Library (of my own invention), I am quite positive !
Not entirely correct. Thomas Watson once decided to take out CDC and build a supercomputer. IBM would badmouth CDC "in support of our future machine". It turned out the single guy Seymour Cray beat the crap out of the army of IBM engineers.
Mr Watson was pissed but from then on focused on mainframes, not supercomputers.
Always good to see that a single capable and determined man can stand up to the sleazy multi-billion corporations and win !
Uhhm, Google and Facebook prove your arguments. It's all about business, forget these stupid CS and math courses.
At this point, Google CAN afford ccNuma. Facebook, too. Apparently, it still is not cost-efficient. Or there simply is no competitive pressure to go ccNUMA.
Actually, MapReduce is all the rage with companies like TeraData, who had an affiliation with BigIron in the past.
Actually, my impression of IBM is that they deliver quite crappy technology on a regular basis, because "we can fix that later on". Cynical colleagues suggested this being a business strategy with the objective of selling support. At a former employer we discovered a broken integer division instruction. When demonstrated to the IBM field engineer, they finally agreed to fix it by means of a microcode update.
Just recently I fought with DB/2 and only the field engineer could help this time, again.
IBM was always very much sales/service oriented and they still cannot compete with the "shrink-wrapped" companies like MS and Oracle. IBM is a relic of the past and will soon be history because it is simply much more efficient to perfect a product once instead of sending a field engineer to each and every customer to clean up the problems they haven't fixed during R&D time.
You prefer that everybody in this industry earns a precarious income and operates dirt-cheap stuff ? No high-end equipment whatsoever ? Yeah, that makes sense if you earn about as much as a McDonalds worker. Unfortunately, I do think your wish will come true.
Regarding IBM mainframes, they are worth their money and if you cannot see the difference to the cheap x86 stuff from Dell, that's your fault. Finally, every skilled factory worker stands in front of 500k Euros/dollars worth of hardware these days. You prefer that 10k$ Dell "server" ??
Sorry for messed-up formatting.
Most importantly, read the papers of David Kuck. A MUST for your professional development as a Computer Science graduate who wants to continue to work in this field (as opposed to doing "business" things). http://www.computer.org/portal/web/awards/pioneer-kuck http://www.dtjcd.vmsresource.org.uk/pdfs/dtj_v06-03_1994.pdf
Well, men use FORTRAN. Real men are not beancounters, they work on some lethal machinery. Semi-real men like me at least use C++ to do beancounting, which they call "statistics".
I don't think that you actually understand what you write. The CPUs are definitely not commodity, even if it would make sense to emulate the S/390 instruction set on x86 CPUs. Which it doesn't from a reliability/serviceability perspective. And "supercomputers" were the "men's" toys to simulate the weather and how to evaporate Russian cities, amongst other belligerent stuff. Mainframe != supercomputer. Mr Watson of IBM once tried to disprove that and got his fingers burned when he tried to compete with Seymour Cray. Actually, his little army of engineers got their fingers burnt when they competed against that single guy Cray.
Yeah, you ran Women's stuff on a Man's computer.
MapReduce is just the latest form of Batch Processing, with a fancy new name from the Googlers. So in 2005 Google discovered something IBM knew in 1955.
In the hands of a capable professional, objects can be at least as efficient as structures+procedures. Plus, you need a language with real destructors to immediately reclaim memory when you are done. I would argue that properly used classes and objects will improve your program in several dimensions. Think of automatically closed file and database handles when a method returns or the containing object is being destructed.
I currently work on a data analysis application which processes 32Mbyte/s of input data using C++ PER CORE. I tested it to linearly scale to 50 cores. And yeah, it works similar to a record-oriented Cobol program. Full-table scans are not as bad as they might sound. Index-oriented access is actually a very restrictive way of accessing large data sets and CS professionals should break out of the jail called "relational database". Not every lose part requires fixing by nail.
I am now at the middle of my life expectation and growing a bit smarter. And, I talk to a Cobol programmer then and now in the bus home. He works for an insurance company and will probably retire as a Cobol programmer for that company. He is a mathematician and I am a CS guy; I know much more than he about algorithms, C++, templates, macros, databases and whatnot.
But just recently I realized that Map-Reduce and "record-oriented processing" are actually very similar in that they do NOT consume voracious amounts of main memory. Both perform full-table-scans, in database parlance, which has unique advantages over index-based access for many scenarios.
That's all important if your data set is 100 times larger than your main memory. So the mainframers have that capability since 1955 and the C++ guys just discover this in the year 2005 or so ??
Cobol is here to stay for very systematic reasons very few people understand, including those with a CS degree and those developing in Cobol for a very long time. The latter do Cobol simply because it always paid nicely and there is absolutely no end in sight.
The term is "loop reordering". Essentially, you can iterate matrices row-wise or column-wise. If your matrix is stored row-wise, you better iterate it row-wise, or your cache locality will be very bad. Typical use case is matrix multiplication and solving linear equation systems. So that Mr Kuck of the uni of Illinois (now at Intel, still working !) created optimizing compilers which can do impressive things, including estimating how long a typical Fortran program run will take (surely that does not work for all categories of Fortran programs). In addition to that Mr Kuck developed compilers which would automatically exploit parallelism of "vector" hardware like the machines designed by Seymour Cray. Note that there is no need to explicitly say "parallelize this loop" as you need to do with OpenMP; the compiler can figure that itself ! Again, this proves that new != better. Fortran still beats C++ in numerical processing and that is quite interesting, considering the fact that Fortran is one of the very first programming languages ! Google Mr Kuck and his papers regarding Fortran compilers - it's an impressive part of computer science from the CDC6600 to the present day !