Nope. Always start with cat. It makes it easier to insert a new command in the pipeline if you need to do something before sorting. How many changes must you make to your version to cause it to run 'rev' before sorting? Now, what sort of changes do you make to the version that uses cat? With your version you must change 'sort' to 'rev' and then insert 'sort |'. With the cat version you need merely insert 'rev |'
Heh, also a useless use of uniq. Try sort -u quote*.txt
But uniq has the -c flag, which is terribly useful. sort doesn't have it. Why waste brain cells remembering "use uniq when you need -c, otherwise use sort -u" when instead you could just always use uniq?
If I was your boss, I'd want "scripts" and not "hacks," too.
The real question is, was your 4 line shell script only 4 lines long because it really was that simple, or was it because, as you say, it was a "hack?"
I'll take 10 lines of code I can understand over 1 line I can't, any day of the week.
Yeah, I skipped the discussion of block sizes and memory requirements, just like people generally skip such things when talking about Turing machines.
In the case of a Turing machine, it's not really relevant to mention how long the tape is (for example). It's definitely relevant to mention block sizes when discussing Huffman compression, because it has a major impact on its effectiveness.
Your distinction between "transformation" and "compression" is unclear and appears arbitrary.
I wasn't drawing a distinction between transformation and compression, but between transformation and entropy coding. "Compression" is the overall concept of reducing the number of bits required to represent the data.
The distinctions are quite clear. Transformation preserves the number of bits, but potentially can reduce the entropy. Coding can mean several things, the most common of which is entropy coding, which preserves the entropy while reducing representation size. "Notational re-engineering" is, I'm guessing, a lot like coding -- renaming symbols. Compression is what you get by combining all these things.
To someone with a rough understanding of compression, it might seem okay to equate all these things, but it's really not correct, any more than saying that a distributor and an alternator are really the same thing because they both happen to be parts of a car's electrical system. You can't replace either one with the other, and the car won't work without both of them.
Another thing that worries me is why all prepostrous claims are met with so much resistance....
If it's true, it will be accepted in the end. Scientists don't want to live in a fantasy world. It seems logical that preposterous claims are met with resistance, otherwise every nutcase who makes an outrageous statement would cause a massive waste of time as people try to validate it.
And anyway, nothing about turbo codes is preposterous. It's been around for years, and I really don't understand why this article is written as if there is some huge controversy around it.
That is, the compression algorithm which, if you fed it every possible combination of input data, would compress the data the best. The algorithm is called Huffman coding.
No. Huffman coding is only optimal if the block size grows to infinity. In that case, it can approach the Shannon limit. For finite data sets (i.e., all data sets), arithmetic coding performs slightly better on a per-symbol basis, because it is able to use fractional numbers of bits to represent symbols.
Huffman is never used on its own, except as a demonstration of basic compression algorithms. It's commonly used as the final step in entropy-coding the output symbols of another compressor, commonly a lossy compressor, like JPEG, MP3, or MPEG.
The problem is, in actual use Huffman coding is crap. Why? Because real data isn't random, it doesn't cover the entire space of possible data.
In actual coding, it is crap, but not for the reasons you listed. It's crap because it's infeasible to allow the block size to grow without bound, since the number of possible codes increases exponentially.
As someone else has pointed out in the threads, if you allow your compression scheme to be useful only for particular files, and allow the compression and decompression software to be arbitrarily complex, there's essentially no limit to how tight you can compress data.
There's a very clear limit, given by the Shannon entropy (which can vary widely, depending on the transformations you apply to the data). It is also limited by the quite obvious pigeonhole principle. You cannot pick an algorithm which will compress a given data set arbitrarily well.
So in general, there's no way to tell what the technical limit is on compression.
No. The limit is definitely the Shannon entropy. You cannot magically represent data using fewer bits than necessary (and Shannon's entropy tells you how many that is). What changes between compression algorithms is the transformations which are applied to the data, or in other words, the different ways of looking at it which reveal a lower entropy than other ways of looking at it. However, the limit is always the Shannon entropy. The transformations can toss out data which is irrelevant. Your playing card example is a good one -- the information which is relevant is which card it is and possibly its angle. You've applied a transformation to the data -- transformed it from a bitmapped image to a series of card values and positions. You can then compress those pieces of information with Huffman codes (if the entropy allows it).
Except that magnets can't disrupt capacitor-based storage devices such as SRAM or DRAM. Magnetic-based core memory technology came and went decades ago...
Lets see.. 2500 GB of RAM at $150 a gig (retail).. that $375K..
Do you think they're buying it in DIMMs or something? Yeah, somewhere in the depths of the mainframe, there is a giant board with 2500 1G DIMMs plugged into it. </sarcasm>
Not all memory is equivalent, you know. The price does seem a bit steep, though.
The 16 year old liked Pine for email, she would rather exit X to read email under Pine, before she learned about xterm.
Ahhh.. This reminds me of when *I* started using computers. I was 5 or 6, and my mother had purchased a Commodore 64 to do her programming assignments. She bought a few games, which my brother and I played. But I was always curious about what my mother did using the "blue screen" (when there's no program running on a C64, you see a blue screen with light blue border, running the BASIC interpretter).
I would always play my games for a little while then ask my mom if it was okay to "play with the blue screen." Which actually meant messing around with C64 BASIC. I learned about PRINT, and INPUT, and how to get a program to add numbers together, etc. Slowly my mom taught me how to write actual programs. Ahh... Those days were great!
It was my experience with "the blue screen" that first ignited my passion for technology. It's really quite sad that the term "blue screen" means something entirely different these days:-(
Its like claiming we'd be more free if the government could take away our right to free speech or public assembly at will.
It's absolutely nothing like that.
Under a license like the BSD license, a company could take the code and make modifications to it and not release the changes. Okay, their particular set of changes is not freely available. So what? This has not deprived you of anything. What did you have before the company made the changes, that you no longer have afterward?
Tell that to the college student who installs SP2 the night before the deadline for his midterm paper only to discover that his word processor doesn't work any more beause of "security issues."
Sorry, but anyone who makes major changes to the system when it's in the middle of doing something extremely important, is a fucking idiot.
"Yeah, I know I was planning on driving to Chicago tomorrow, but I thought there'd be no problem installing a new head gasket twenty minutes before I planned to leave... How was I supposed to know this was not a good idea?" Oh for Christ's sake.
When you stand around the water cooler talking about recents The Apprentice or Survivor episodes are you plagiarising?
No, because that is a discussion of facts, not ideas. Last I checked, most television shows were pretty much devoid of any sort of ideas. It's all shit.
How many jokes that you tell on a daily basis are ones you made up yourself?
Not many, and it's definitely plagiarism to retell someone else's joke without crediting them. However, nobody said that was a bad thing! "Plagiarism" is just a word, with a specific definition. It has a negative connotation, yes. But according to the technical meaning of the word, your example would be plagiarism.
I think it's quite true that blogs plagiarize each other. I also think it's not a bad thing at all. Of course, there are exceptions, such as people copy and pasting other people's entire comments, which I've seen on multiple occassions... It's a matter of personal judgment which things are "right" and which are "wrong."
Or perhaps I'm not an egoist who requires constant re-filling of my own self worth through others quoting me on the internet as BillaBong13? Anyone that gets upset over people not crediting their ideas on a blog needs to grow up.
I actually agree with you, but I was trying to point out that it's still plagiarism even if we don't get all upset over it. I still think it would be appropriate to link back to a blog where you first saw something...
Quite frankly I'd be more flattered that people are listening to what I have to say among the thousands of other voices than worried about "idea theft".
My point was that plagiarism isn't about "idea theft." It's about attribution theft, and theft of respect. People do not insist on attribution because it inflates their egos. They insist on it so that other people do not falsely earn respect by piggybacking on someone else's work. A subtle but definite difference.
How am I supposed to know where every bit of information I have picked up over my lifetime came from?
You're not really expected to memorize every source of everything you've ever learned. If you're writing a paper, however, it's expected that you'll put in the effort required to track down your sources.
There's a big difference between a casual conversation about a topic, and a thesis. "I heard this somewhere, I can't remember where..." doesn't cut it in a piece of academic writing.
3. Abusive or improper treatment; violation: a rape of justice.
Your degree of offense seems to indicate that you have some direct experience with rape, and we're all sympathetic to that, but the word was used in a proper sense. People tend to abuse the word "definitely" and use it improperly.
Scientists "steal" ideas - they modify other's ideas.
And they attribute them. Failing to do so will get you kicked off committees, fired from professorships, ejected from laboratories, and basically make you an unhireable laughing stock.
Yes, the dissemination of ideas is nothing but beneficial. But to fail to attribute is morally reprehensible. There comes a point when an ideas is so far-flung and well known that the attribution becomes implicit, but it is always there.
Notice that we don't list Maxwell in the bibliography of a paper on electromagnetism. However, we attribute Maxwell by naming his equations after him: Maxwell's Equations. The attribution is therefore eternal, in a sense.
Is Thomas Jefferson a plagarist because he didn't put references to the "original authors" of his ideas in the Declaration of Independance?
Yes. So he's a famous guy. Your point?
Eventually ideas become part of the mainstream consciousness.
Exactly, and at that point it is considered "common knowledge," and the rules change somewhat. However, even when an idea has become common knowledge, often it is still attributed, for the sake of respect for the originator. Behold, "Newton's Laws," "Maxwell's Equations," "Fermi-Dirac Statistics," "The Haber Process," etc. We attribute simply by mentioning.
It's also something that mostly applies to acadamia where people care about that sort of thing.
You're absolutely right. Notice that plagiarism is not illegal. It is considered wrong, but there's nothing illegal about failing to attribute ideas.
Perhaps you do not recognize the emotional need people have to be recognized for their profound ideas, because you have not contributed any profound ideas?
Slashdot Mirror
Nope. Always start with cat. It makes it easier to insert a new command in the pipeline if you need to do something before sorting. How many changes must you make to your version to cause it to run 'rev' before sorting? Now, what sort of changes do you make to the version that uses cat? With your version you must change 'sort' to 'rev' and then insert 'sort |'. With the cat version you need merely insert 'rev |'
Heh, also a useless use of uniq. Try sort -u quote*.txt
But uniq has the -c flag, which is terribly useful. sort doesn't have it. Why waste brain cells remembering "use uniq when you need -c, otherwise use sort -u" when instead you could just always use uniq?
The real question is, was your 4 line shell script only 4 lines long because it really was that simple, or was it because, as you say, it was a "hack?"
I'll take 10 lines of code I can understand over 1 line I can't, any day of the week.
In the case of a Turing machine, it's not really relevant to mention how long the tape is (for example). It's definitely relevant to mention block sizes when discussing Huffman compression, because it has a major impact on its effectiveness.
Your distinction between "transformation" and "compression" is unclear and appears arbitrary.
I wasn't drawing a distinction between transformation and compression, but between transformation and entropy coding. "Compression" is the overall concept of reducing the number of bits required to represent the data.
The distinctions are quite clear. Transformation preserves the number of bits, but potentially can reduce the entropy. Coding can mean several things, the most common of which is entropy coding, which preserves the entropy while reducing representation size. "Notational re-engineering" is, I'm guessing, a lot like coding -- renaming symbols. Compression is what you get by combining all these things.
To someone with a rough understanding of compression, it might seem okay to equate all these things, but it's really not correct, any more than saying that a distributor and an alternator are really the same thing because they both happen to be parts of a car's electrical system. You can't replace either one with the other, and the car won't work without both of them.
But it sounds like your nieces win... their minds back.
Obviously, he's not the one who did the mod. The Slashdot administrators must have done it on their database machine.
No responsible hobbyist or scientist would do something as idiotic as testing a worm on a net-connected machine.
As for cutting off the connections of those who download warez 24/7, I say fucking go for it. More bandwidth for my legitimate uses.
If it's true, it will be accepted in the end. Scientists don't want to live in a fantasy world. It seems logical that preposterous claims are met with resistance, otherwise every nutcase who makes an outrageous statement would cause a massive waste of time as people try to validate it.
And anyway, nothing about turbo codes is preposterous. It's been around for years, and I really don't understand why this article is written as if there is some huge controversy around it.
No. Huffman coding is only optimal if the block size grows to infinity. In that case, it can approach the Shannon limit. For finite data sets (i.e., all data sets), arithmetic coding performs slightly better on a per-symbol basis, because it is able to use fractional numbers of bits to represent symbols.
Huffman is never used on its own, except as a demonstration of basic compression algorithms. It's commonly used as the final step in entropy-coding the output symbols of another compressor, commonly a lossy compressor, like JPEG, MP3, or MPEG.
The problem is, in actual use Huffman coding is crap. Why? Because real data isn't random, it doesn't cover the entire space of possible data.
In actual coding, it is crap, but not for the reasons you listed. It's crap because it's infeasible to allow the block size to grow without bound, since the number of possible codes increases exponentially.
As someone else has pointed out in the threads, if you allow your compression scheme to be useful only for particular files, and allow the compression and decompression software to be arbitrarily complex, there's essentially no limit to how tight you can compress data.
There's a very clear limit, given by the Shannon entropy (which can vary widely, depending on the transformations you apply to the data). It is also limited by the quite obvious pigeonhole principle. You cannot pick an algorithm which will compress a given data set arbitrarily well.
So in general, there's no way to tell what the technical limit is on compression.
No. The limit is definitely the Shannon entropy. You cannot magically represent data using fewer bits than necessary (and Shannon's entropy tells you how many that is). What changes between compression algorithms is the transformations which are applied to the data, or in other words, the different ways of looking at it which reveal a lower entropy than other ways of looking at it. However, the limit is always the Shannon entropy. The transformations can toss out data which is irrelevant. Your playing card example is a good one -- the information which is relevant is which card it is and possibly its angle. You've applied a transformation to the data -- transformed it from a bitmapped image to a series of card values and positions. You can then compress those pieces of information with Huffman codes (if the entropy allows it).
Except that magnets can't disrupt capacitor-based storage devices such as SRAM or DRAM. Magnetic-based core memory technology came and went decades ago...
Do you think they're buying it in DIMMs or something? Yeah, somewhere in the depths of the mainframe, there is a giant board with 2500 1G DIMMs plugged into it. </sarcasm>
Not all memory is equivalent, you know. The price does seem a bit steep, though.
"Well, SCO sues you, and all your customers flee."
"Okay. Remind me not to do that."
Ahhh.. This reminds me of when *I* started using computers. I was 5 or 6, and my mother had purchased a Commodore 64 to do her programming assignments. She bought a few games, which my brother and I played. But I was always curious about what my mother did using the "blue screen" (when there's no program running on a C64, you see a blue screen with light blue border, running the BASIC interpretter).
I would always play my games for a little while then ask my mom if it was okay to "play with the blue screen." Which actually meant messing around with C64 BASIC. I learned about PRINT, and INPUT, and how to get a program to add numbers together, etc. Slowly my mom taught me how to write actual programs. Ahh... Those days were great!
It was my experience with "the blue screen" that first ignited my passion for technology. It's really quite sad that the term "blue screen" means something entirely different these days :-(
It's absolutely nothing like that.
Under a license like the BSD license, a company could take the code and make modifications to it and not release the changes. Okay, their particular set of changes is not freely available. So what? This has not deprived you of anything. What did you have before the company made the changes, that you no longer have afterward?
It's testimony like this that makes me proud to be a computer scientist. Even when we're wrong, we're actually right!
I think you mean toeing the line, and last time I checked, being accused of "toeing the line" was never a compliment.
I don't see any particularly vehement criticisms of this move by Microsoft.
Perhaps you should pull your head out of your ass and look around instead of assuming things incorrectly?
Sorry, but anyone who makes major changes to the system when it's in the middle of doing something extremely important, is a fucking idiot.
"Yeah, I know I was planning on driving to Chicago tomorrow, but I thought there'd be no problem installing a new head gasket twenty minutes before I planned to leave... How was I supposed to know this was not a good idea?" Oh for Christ's sake.
No, because that is a discussion of facts, not ideas. Last I checked, most television shows were pretty much devoid of any sort of ideas. It's all shit.
How many jokes that you tell on a daily basis are ones you made up yourself?
Not many, and it's definitely plagiarism to retell someone else's joke without crediting them. However, nobody said that was a bad thing! "Plagiarism" is just a word, with a specific definition. It has a negative connotation, yes. But according to the technical meaning of the word, your example would be plagiarism.
I think it's quite true that blogs plagiarize each other. I also think it's not a bad thing at all. Of course, there are exceptions, such as people copy and pasting other people's entire comments, which I've seen on multiple occassions... It's a matter of personal judgment which things are "right" and which are "wrong."
The theft is of credit, not ideas.
I actually agree with you, but I was trying to point out that it's still plagiarism even if we don't get all upset over it. I still think it would be appropriate to link back to a blog where you first saw something...
Quite frankly I'd be more flattered that people are listening to what I have to say among the thousands of other voices than worried about "idea theft".
My point was that plagiarism isn't about "idea theft." It's about attribution theft, and theft of respect. People do not insist on attribution because it inflates their egos. They insist on it so that other people do not falsely earn respect by piggybacking on someone else's work. A subtle but definite difference.
You're not really expected to memorize every source of everything you've ever learned. If you're writing a paper, however, it's expected that you'll put in the effort required to track down your sources.
There's a big difference between a casual conversation about a topic, and a thesis. "I heard this somewhere, I can't remember where..." doesn't cut it in a piece of academic writing.
3. Abusive or improper treatment; violation: a rape of justice.
Your degree of offense seems to indicate that you have some direct experience with rape, and we're all sympathetic to that, but the word was used in a proper sense. People tend to abuse the word "definitely" and use it improperly.
Take your crusade elsewhere.
What's the point of a secret trial? Why not just execute the person?
And they attribute them. Failing to do so will get you kicked off committees, fired from professorships, ejected from laboratories, and basically make you an unhireable laughing stock.
Yes, the dissemination of ideas is nothing but beneficial. But to fail to attribute is morally reprehensible. There comes a point when an ideas is so far-flung and well known that the attribution becomes implicit, but it is always there.
Notice that we don't list Maxwell in the bibliography of a paper on electromagnetism. However, we attribute Maxwell by naming his equations after him: Maxwell's Equations. The attribution is therefore eternal, in a sense.
Yes. So he's a famous guy. Your point?
Eventually ideas become part of the mainstream consciousness.
Exactly, and at that point it is considered "common knowledge," and the rules change somewhat. However, even when an idea has become common knowledge, often it is still attributed, for the sake of respect for the originator. Behold, "Newton's Laws," "Maxwell's Equations," "Fermi-Dirac Statistics," "The Haber Process," etc. We attribute simply by mentioning.
It's also something that mostly applies to acadamia where people care about that sort of thing.
You're absolutely right. Notice that plagiarism is not illegal. It is considered wrong, but there's nothing illegal about failing to attribute ideas.
Perhaps you do not recognize the emotional need people have to be recognized for their profound ideas, because you have not contributed any profound ideas?