I love your twisted logic here, justifying copyright infringement on the one hand because it promotes "freedom" but condemning it on the other because it promotes the "freedom" to disseminate. Orwell would love you.
I explicitly did not state my opinion on the matter. If you are going to base this conversation on assumptions, then the conversation is over.
Had I written "It is not surprising that the family of a murder victim might want to kill the murderer," would you have assumed that I am advocating vigilantism?
After all, as everyone here on/. knows, copyright violators are bad people who need to be punished. Right?
In one case, copyright is being used as a tool to protect the ability to freely disseminate and modify works. In the other case, it is being used as a tool to restrict freedom of distribution and prevent modification.
I'm not going to bother telling you my personal opinion on the matter, but given the differences in the two scenarios, it should not be surprising that freedom-minded people might be more supportive of the first use of copyright, and more critical of the second.
You apparently don't know what "dynamic programming" means. Dynamic programming is a general technique, usually applied to search, which caches the results of executions on problem subsets and then reuses those results when those subsets are encountered again. It has nothing to do with dynamic typing, or dynamic code, or whatever you're thinking about.
To put it extremely simply, dynamic programming means hash tables, although that's not really a complete definition.
But how about "Al Sanders"... a single CPU could would have more divides to reach that name, dividing names and finding if the target result would be more/less than the middle name.
Nitpicking here... I assume you're talking about a binary search. It takes about the same amount of time to find a name anywhere in the list, unless you happen to get lucky and land directly on it. The lower and upper search bounds still have to converge to a zero-width window, and that takes the same number of divisions no matter where in the list the target is. You can optimize slightly by checking if you've hit the target on each division, in which case you might terminate early, but that doesn't happen often, especially in a large list.
Unfolding the list into two lists, with the even elements in one list and the odd elements in the other, only saves you, at most, a single division. In short, searching a sorted list is not the type of operation that benefits greatly from multiple CPUs.
Well, hopefully they now have a more realistic risk factor when it comes to a single-source development platform produced by a notorious monopolist corporation.
I'm not trying to be abrasive; obviously the planners could not have known what the world would be like 10 years into the future. But the fact remains that that particular risk factor was grossly underestimated.
I really don't follow. You just said that this will probably cost them another 10 man years to fix. Sounds to me like they really blew the cost analysis.
You appearantly haven't understood all the factors that are involved with developing software in a commercial environment
Funny. It looks like they're the ones who committed the monumental screwup.
There is only one valid interpretation. The concept you are looking for here is "ambiguity." Just because something is ambiguous does not mean that all possible interpretations are correct.
Unless the two execution states overflow your L1 cache, in which case a HT CPU could run slower.
I was under the impression that most HT cores had double the L1 cache to ameliorate this exact problem, but perhaps I haven't done my research.
And if they DON'T, they sure as hell SHOULD.
Also, I've always thought it would be cool to allow a process/thread to allocate a certain portion of the cache for its exclusive use. I understand that this would make the caching logic insanely complicated, however.
if done correctly on a single threaded system, if one thread is in a wait state waiting on disc activity, then the CPU should jump threads and handle other tasks in the mean time.
That's precisely my point. One of those "other tasks" is another thread running your conversion. If you are maxing a CPU for long periods of time, you should be running on a dedicated box.
Wasn't the purpose of a DMA channel to mode data from one location to another while the CPU is performing other tasks?
Uhh, yes... But clearly, the thread which is waiting for that data still needs to wait. My entire point was that by multithreading your task you can eliminate the impact of a lot of that latency.
anything i write usually maxes out the processor at 100% for days at a time (i deal with huge data conversions)
so yeah i'd also like to know: what does it mean to me?
Well, if your data conversions are independent, multithreading might be of benefit to you if you have a hyperthreading processor.
And are you sure you are maxing the processor? Surely you have to wait for disk or network, at least some of the time. If more than 10% or so (number pulled from ass but based on empirical observations) of you time is spent waiting for latent devices, you can benefit from multithreading even on a plain vanilla single CPU system with no hyperthreading.
"What's" is a common contraction of "What does." The fact that it is used, heavily, in American speech is evidence enough of this. In British English, who knows.
Companies that have invested 50 man year in the development of VB6 applications are now faced with the fact that they will require to trow in another 10 man year just to make the transition to VB.Net.
Aww, poor babies. They picked a proprietary framework sourced by a single vendor, they are now reaping the consequences.
The developers who can transition to something new will prove their worth. The ones left bleeding in the wreckage were never really developers in the first place.
Presumably the meaning is best expressed by "Is Google Breaking Their Own Rules?" which translates into something like "Is someone (singular) at Google breaking rules established in general by others (plural) within Google?"
Consider the sentence, "Is Microsoft a shitty corporation?" Am I referring to a single individual at Microsoft? And yet "Are Microsoft a shitty corporation?" cannot be correct either, because then you might respond, "Yes, that corporation are very shitty" which is just wrong no matter how you look at it.
I'm not talking about who or how many people are breaking rules, I'm saying that as far as I've ever seen of the English language (in the US at least), corporations are singular entities.
Interesting idea to sort the letters! That would enhance bzip's compression efficiency somewhat. On the decoding side, it would be fairly simple to map the sorted words back to the originals. An ambiguity resolver based on, say, a third-order Markov model might be able to make the right choice most of the time (similar to some OCR cleanup techniques).
BTW, I sent you an email suggesting a few more data sets you might host on your site.
Anything where minute details don't matter will probably be lossy, but probably 80% or more will be text, code and other data that needs to be lossless.
Whether text compression needs to be lossless is actually debatable. I'm gonna veer a little off topic here, but hey, it's Slashdot...
Suppose you are compressing English text by Huffman encoding entire words at a time. However, people make typos, so the actual set of words to be encoded will be larger than a set where there were no typos. By first running a spell check you can slightly increase the compression efficiency. Strictly, this is "lossy" compression since the decoded data is not the same as the encoded data. But not only have you improved compression efficiency, you've fixed the typos.
For something like C code, it is not necessary to precisely reproduce the indentation style of the original code. It could be boiled down to a simple canonical format and then compressed with a C-grammar-aware method, then run through "indent" on the other end to recover reasonable indentation.
Compresison on any sort of data. Give me a good general-purpose compressor.
There isn't any such thing, unless you use a very narrow definition of "general purpose." To me, general purpose would imply that I could throw any sort of data at it I please, and it would do well, provided the data is not just random data. Since we're working with very vague definitions I can only give very vague suggestions.
To answer your specific questions, a good dictionary compressor is the Flate algorithm used by gzip. Very fast decompression, moderately fast compression, which can be tuned by a single parameter ("compression level" ranging from 0 to 9).
The filters depend again on the sort of data you are compressing. For non-photographic image data, the standard PNG linear filters, as well as the Paeth filter also supported by PNG, usually do quite well. For photographic data, just use high quality JPEG and consider it lossless (it could be argued that the amount of noise injected by JPEG is on the same order of magnitude as the noise in the digital imaging process, so who cares where the noise comes from, it's there whether you want it or not). For audio data, channel decorrelation followed by linear prediction is pretty much standard.
Bzip2 does well on any sort of data with repeating patterns, or nearly repeating patterns. In general, it works well on the sorts of data that a dictionary approach would work on, but it also works fairly well on audio data (much better than a dictionary approach, but still worse than if you had applied a filter specifically designed for audio data and then compressed with something simple like Flate or even just Huffman or Rice codes).
When bzip compressing many small files it is imperative to archive first and then compress, so that the 900k block sizes are used to their fullest advantage.
What does "everyone" have? It seems to me that if you're starting a mass data archival site, you can boost the number of algorithms by making them available on your site. At the bare minimum I would expect people to support zip, flate, bzip2, jpg, flac.
The various types of data you mention are not uniformly compressable by any single algorithm. Therefore your "compression ratio" criterion is dubious. Compression ratio on what sort of data?
Dictionary-based compression schemes work well on data which might be described as "linguistic," i.e., data which has some kind of grammar describing it. English text, machine code (binaries), source code, HTML, etc. It won't work very well at all on audio or image data, at least not without some kind of preprocessing (usually called filtering). Bzip2 does somewhat better on audio because it is not a dictionary-based scheme.
There's no really simple way to get everything you are asking for. Do you want to distribute archives of multiple, compressed audio files? Then the files should be compressed individually, then archived. OTOH, if you want to distribute a bunch of.txt files, you should do the opposite -- archive first, and THEN compress.
If you are really serious about saving disk space, there's simply no way you're going to have a single, universal compressed archival format. OTOH, if the main goal is to have a format everybody can deal with, and perhaps as a side effect get a little compression out of it, I'd just use plain ZIP.
If it was MS (if you use msn search or whatever their search engine is) made their products come out on top using such tactics then everyone would be trying to join the lynch mob. However if google does it its fine.
Microsoft has a long, bad history of stomping the little guy by bringing the monopoly cannons to bear. Google, to date, does not do such things. The two aren't really comparable.
That should be, "Are Google Breaking Their Own Rules," not "Is Google..."
Sure, that agrees now, but it still sounds bad. "Google are really cool!" WTF? Just because a corporation consists of multiple people doesn't mean it's plural. The headline should have been, "Is Google Breaking Its Own Rules?"
Oddly, this is the ONLY thing I get pedantic about when it comes to grammar.
And neither do any contests of this sort. Break it down by the types of people who might enter the contest:
1. White hats. Why would they do it? If they're any good, it'll just be a waste of time, and you can always set up your own server to practice with. There's not even any prize!
2. Black hats (I mean real ones, not script kiddies). They wouldn't bother either. Why expose the contents of your secret toolbox for no good reason? Any hack attempts (and successes) will be fully logged, revealing your secret exploits. That's no good, is it?
3. Script kiddies. Maybe they'll try, but they won't get in, unless the server is embarrassingly badly configured. If they do manage to crack it, what does that prove? That it's possible to set up a Linux box with terrible security if you happen to be incompetent?
I'm having a hard time figuring out exactly WHAT this contest is for. The only thing I can imagine (which a few other people have mentioned in this discussion) is that it's meant to enhance the image of Linux as a secure platform. So what -- so you've shown that if you do a good job configuring your box, you can keep out script kiddies. To put it bluntly, no shit.
I understand the argument, but it depends how far you're willing to take it. You could also wash your clothes by hand, if you really wanted to save resources.
I prefer to spread my resource optimization across many different areas so that I can splurge on a few things (like a dishwasher) which, although not as efficient as it could be, makes my evenings quite a bit easier.
To put it another way, I feel less guilty about indulging in a bit of luxury because I try to do so much in other ways.
Slashdot Mirror
I explicitly did not state my opinion on the matter. If you are going to base this conversation on assumptions, then the conversation is over.
Had I written "It is not surprising that the family of a murder victim might want to kill the murderer," would you have assumed that I am advocating vigilantism?
In one case, copyright is being used as a tool to protect the ability to freely disseminate and modify works. In the other case, it is being used as a tool to restrict freedom of distribution and prevent modification.
I'm not going to bother telling you my personal opinion on the matter, but given the differences in the two scenarios, it should not be surprising that freedom-minded people might be more supportive of the first use of copyright, and more critical of the second.
You apparently don't know what "dynamic programming" means. Dynamic programming is a general technique, usually applied to search, which caches the results of executions on problem subsets and then reuses those results when those subsets are encountered again. It has nothing to do with dynamic typing, or dynamic code, or whatever you're thinking about.
To put it extremely simply, dynamic programming means hash tables, although that's not really a complete definition.
It is only ambiguous because you are being intentionally boneheaded. There's a time and a place for geeky logical nitpicking, and this ain't it.
Nitpicking here... I assume you're talking about a binary search. It takes about the same amount of time to find a name anywhere in the list, unless you happen to get lucky and land directly on it. The lower and upper search bounds still have to converge to a zero-width window, and that takes the same number of divisions no matter where in the list the target is. You can optimize slightly by checking if you've hit the target on each division, in which case you might terminate early, but that doesn't happen often, especially in a large list.
Unfolding the list into two lists, with the even elements in one list and the odd elements in the other, only saves you, at most, a single division. In short, searching a sorted list is not the type of operation that benefits greatly from multiple CPUs.
I'm not trying to be abrasive; obviously the planners could not have known what the world would be like 10 years into the future. But the fact remains that that particular risk factor was grossly underestimated.
You appearantly haven't understood all the factors that are involved with developing software in a commercial environment
Funny. It looks like they're the ones who committed the monumental screwup.
When a person says something, the intended meaning is not ambiguous (unless you are a poet), although the words used to describe that meaning may be.
In this case it was intended to mean "What does it mean" and absolutely nothing else, your grammatical writhings notwithstanding.
There is only one valid interpretation. The concept you are looking for here is "ambiguity." Just because something is ambiguous does not mean that all possible interpretations are correct.
I was under the impression that most HT cores had double the L1 cache to ameliorate this exact problem, but perhaps I haven't done my research.
And if they DON'T, they sure as hell SHOULD.
Also, I've always thought it would be cool to allow a process/thread to allocate a certain portion of the cache for its exclusive use. I understand that this would make the caching logic insanely complicated, however.
That's precisely my point. One of those "other tasks" is another thread running your conversion. If you are maxing a CPU for long periods of time, you should be running on a dedicated box.
Wasn't the purpose of a DMA channel to mode data from one location to another while the CPU is performing other tasks?
Uhh, yes... But clearly, the thread which is waiting for that data still needs to wait. My entire point was that by multithreading your task you can eliminate the impact of a lot of that latency.
Well, if your data conversions are independent, multithreading might be of benefit to you if you have a hyperthreading processor.
And are you sure you are maxing the processor? Surely you have to wait for disk or network, at least some of the time. If more than 10% or so (number pulled from ass but based on empirical observations) of you time is spent waiting for latent devices, you can benefit from multithreading even on a plain vanilla single CPU system with no hyperthreading.
"What's" is a common contraction of "What does." The fact that it is used, heavily, in American speech is evidence enough of this. In British English, who knows.
Aww, poor babies. They picked a proprietary framework sourced by a single vendor, they are now reaping the consequences.
The developers who can transition to something new will prove their worth. The ones left bleeding in the wreckage were never really developers in the first place.
Hey, it's time for me to mail in my check to extend my right to freedom of speech...
Presumably the meaning is best expressed by "Is Google Breaking Their Own Rules?" which translates into something like "Is someone (singular) at Google breaking rules established in general by others (plural) within Google?"
Consider the sentence, "Is Microsoft a shitty corporation?" Am I referring to a single individual at Microsoft? And yet "Are Microsoft a shitty corporation?" cannot be correct either, because then you might respond, "Yes, that corporation are very shitty" which is just wrong no matter how you look at it.
I'm not talking about who or how many people are breaking rules, I'm saying that as far as I've ever seen of the English language (in the US at least), corporations are singular entities.
BTW, I sent you an email suggesting a few more data sets you might host on your site.
Whether text compression needs to be lossless is actually debatable. I'm gonna veer a little off topic here, but hey, it's Slashdot...
Suppose you are compressing English text by Huffman encoding entire words at a time. However, people make typos, so the actual set of words to be encoded will be larger than a set where there were no typos. By first running a spell check you can slightly increase the compression efficiency. Strictly, this is "lossy" compression since the decoded data is not the same as the encoded data. But not only have you improved compression efficiency, you've fixed the typos.
For something like C code, it is not necessary to precisely reproduce the indentation style of the original code. It could be boiled down to a simple canonical format and then compressed with a C-grammar-aware method, then run through "indent" on the other end to recover reasonable indentation.
There isn't any such thing, unless you use a very narrow definition of "general purpose." To me, general purpose would imply that I could throw any sort of data at it I please, and it would do well, provided the data is not just random data. Since we're working with very vague definitions I can only give very vague suggestions.
To answer your specific questions, a good dictionary compressor is the Flate algorithm used by gzip. Very fast decompression, moderately fast compression, which can be tuned by a single parameter ("compression level" ranging from 0 to 9).
The filters depend again on the sort of data you are compressing. For non-photographic image data, the standard PNG linear filters, as well as the Paeth filter also supported by PNG, usually do quite well. For photographic data, just use high quality JPEG and consider it lossless (it could be argued that the amount of noise injected by JPEG is on the same order of magnitude as the noise in the digital imaging process, so who cares where the noise comes from, it's there whether you want it or not). For audio data, channel decorrelation followed by linear prediction is pretty much standard.
Bzip2 does well on any sort of data with repeating patterns, or nearly repeating patterns. In general, it works well on the sorts of data that a dictionary approach would work on, but it also works fairly well on audio data (much better than a dictionary approach, but still worse than if you had applied a filter specifically designed for audio data and then compressed with something simple like Flate or even just Huffman or Rice codes).
When bzip compressing many small files it is imperative to archive first and then compress, so that the 900k block sizes are used to their fullest advantage.
What does "everyone" have? It seems to me that if you're starting a mass data archival site, you can boost the number of algorithms by making them available on your site. At the bare minimum I would expect people to support zip, flate, bzip2, jpg, flac.
Is lossy compression an option, or not an option?
Dictionary-based compression schemes work well on data which might be described as "linguistic," i.e., data which has some kind of grammar describing it. English text, machine code (binaries), source code, HTML, etc. It won't work very well at all on audio or image data, at least not without some kind of preprocessing (usually called filtering). Bzip2 does somewhat better on audio because it is not a dictionary-based scheme.
There's no really simple way to get everything you are asking for. Do you want to distribute archives of multiple, compressed audio files? Then the files should be compressed individually, then archived. OTOH, if you want to distribute a bunch of .txt files, you should do the opposite -- archive first, and THEN compress.
If you are really serious about saving disk space, there's simply no way you're going to have a single, universal compressed archival format. OTOH, if the main goal is to have a format everybody can deal with, and perhaps as a side effect get a little compression out of it, I'd just use plain ZIP.
Microsoft has a long, bad history of stomping the little guy by bringing the monopoly cannons to bear. Google, to date, does not do such things. The two aren't really comparable.
Sure, that agrees now, but it still sounds bad. "Google are really cool!" WTF? Just because a corporation consists of multiple people doesn't mean it's plural. The headline should have been, "Is Google Breaking Its Own Rules?"
Oddly, this is the ONLY thing I get pedantic about when it comes to grammar.
1. White hats. Why would they do it? If they're any good, it'll just be a waste of time, and you can always set up your own server to practice with. There's not even any prize!
2. Black hats (I mean real ones, not script kiddies). They wouldn't bother either. Why expose the contents of your secret toolbox for no good reason? Any hack attempts (and successes) will be fully logged, revealing your secret exploits. That's no good, is it?
3. Script kiddies. Maybe they'll try, but they won't get in, unless the server is embarrassingly badly configured. If they do manage to crack it, what does that prove? That it's possible to set up a Linux box with terrible security if you happen to be incompetent?
I'm having a hard time figuring out exactly WHAT this contest is for. The only thing I can imagine (which a few other people have mentioned in this discussion) is that it's meant to enhance the image of Linux as a secure platform. So what -- so you've shown that if you do a good job configuring your box, you can keep out script kiddies. To put it bluntly, no shit.
I prefer to spread my resource optimization across many different areas so that I can splurge on a few things (like a dishwasher) which, although not as efficient as it could be, makes my evenings quite a bit easier.
To put it another way, I feel less guilty about indulging in a bit of luxury because I try to do so much in other ways.
Replace outdated or inefficent appliances
Already done. My washer and drier use 1/4 the energy and water of a typical appliance. I just got an energy saving dishwasher, too.
Service your vehicle to ensure it is running at maximum efficency
Done. Every four months.
Purchase compact-flourescent lights
Every light in the house which can be, is.
Add insulation to your house
The previous owner took care of that. Visually inspected by myself. Great condition.
Install energy-efficent windows
This is next on my list. But if you think you can do it for anywhere close to$1500, you're insane. On this house the cost will be pushing $20,000.
Purchase renewable-resource energy credits from your energy provider
I do. I also spend extra for "Green" energy, which helps fund advances in renewable technology.
Invest in a renewable energy cooperative
I'm helping in the design process for a megawatt-sized solar facility, does that count?
Cover the cost differential to purchase biodiesel
I'm not sure exactly what you mean by that, but if you explain, I'll consider it.