If you did pick up some noise, it would be from the repeated DCT and inverse DCT, not from the quantization itself. The ideal DCT wouldn't introduce noise, as I understand it, but in real life there might be rounding errors.
From your explanation, as the quality setting is applied to jpegs, it sounds like recompressing a jpeg would not increase the loss if the same quantizer was used as was used on the original. Is that true?
That should be true.
JPEG, and the video codecs that I know about, all divide an image up into squares of 8 pixels by 8 pixels. Then each square is processed. For JPEG, the squares are run through the DCT.
So only the edited 8x8 squares should actually be affected.
So when I run GNU/Linux on one machine, and GNU/HURD on another, I should call one "Linux" and the other "HURD"?
Up to you, dude. RMS would have you call the second one "GNU" because the HURD is the kernel for the GNU operating system.
I'd probably call one "Linux" and the other "HURD" (sorry, RMS).
What if I install SunOS + Solaris, remove everything but the kernel, and somehow manage to get the thing to boot with only GNU tools? Is it still SunOS? Or is is GNU/SunOS?
I'll bet RMS would be happy if you called it "GNU/SunOS". If you actually made an install like that, I'd probably call it "GNU/SunOS" too, since you made such a point of ripping away everything but GNU tools.
Of course, you contradict yourself; as if the kernel were the most important part vs. The kernel really is the most important part Make your mind up... but of course, you can't.
Go back and read it again, and decide for yourself whether I can make up my mind or whether I was just trying to write in a non-boring way. Hint: I don't agree with RMS 100% about everything.
It's partially a political thing, but really a technical one; just as a server is useless without storage, Linux is useless without GNU.
And RMS's campaign to have everyone say "GNU/Linux system" instead of "Linux system" is doomed by GNU's very success. GNU tools are omnipresent. Since they are always there, few people will bother to mention them! Just as people might say "This box runs GNOME" but they won't usually bother to say "This box runs GNOME on top of XFree86".
As I said, anyone who actually says "GNU/Linux" is doing it out of respect for GNU and not because it's a useful way to describe a Linux system.
To compress a sample of audio or video data, you want to throw away details that don't matter. So you transform the data using a function, and then you apply a quantizer to throw away some of the data. The whole point of the transform function is to make a data set that is "safer" to quantize.
Most audio and video codecs use the DCT function, which decomposes an image into a series of waves. If you throw away some of the wave data, you get a similar series of waves, and hopefully humans won't notice the differences.
Applying the quantizer is the "lossy" step in lossy compression. Here's how you do it:
You divide the output data (from the transform function) by some integer number, the quantizer, and you use integer division. (Because you want to simplify the data, you actually want the fractional part discarded!) The bigger the quantizer, the more data is thrown away. A quantizer of 1 would give perfect quality, but very poor compression.
Here is a semi-real example:
You have some data to compress. Let's say it's eight bytes (maybe an audio sample for Ogg Vorbis).
After the DCT, it looks like this (using decimal numbers because it's easier):
355 244 33 192 43 9 3 8
Quantize with a quantizer of 8:
{8} 44 30 4 24 5 1 0 1
The {8} is intended to show that the quantizer has to be stored with the quantized data stream, so we know what quantizer was used so we can dequantize.
On dequantization, we just multiply by 8 again:
352 240 32 192 40 8 0 8
Then we feed these resulting numbers into the inverse DCT, and get back some data that is hopefully not full of artifacts.
When you are encoding, you quantize, and then you encode the resulting stream of numbers. Because the values of the numbers are smaller (e.g. 24, instead of 192) you can encode the values with fewer bits. And you can apply run-length compression to the quantized values; in my example it wouldn't buy you much, but in real examples you will often have a bunch of zeroes in a row.
Video compression dialogs (such as the JPEG options in "Save As" in The GIMP) often give you a slider for a quality percentage. This is actually controlling the quantizer. For JPEG, you have 31 possible settings for the quantizer, so there is no difference between 80% and 81%.
By the way, my personal experience with video compression is that a quantizer of 8 gives pretty good compression with very few visible artifcats. An 8 quantizer corresponds with about a 75% in the quality sliders (such as Save As in The GIMP).
Some afternoon when you are bored, save the same JPEG image over and over with different quality settings, and watch what happens with the artifacts. When you set the quality down to 2% (corresponding to a 31 quantizer) you get ugly, coarse blocks. The GIMP under Linux is particularly good for this because you can drag the quality slider and the preview updates instantly, so you don't have to save and re-open.
Yes, the site came up fine for me. And I actually wasn't too worried about Red Hat's servers getting Slashdotted; I think Red Hat has pretty good servers.
And I wasn't even looking for free karma; I already have plenty.
I know that many, many people just read the Slashdot comments and never click through the links, and I wanted them to see the announcement before asking questions answered in the announcement. I was hoping to head off some of the more obvious dumb questions.
Also, the URL provided is for a very temporary place: Ingo's personal stuff. A year from now people might read the archived comments on Slashdot, and I thought it would be nice to have the announcement in there, especially because the link probably won't work anymore. (Although I may have been wrong about the link not working anymore; there seems to be lots of older stuff on there that he never took down. Oh, well.)
However his die hard views seem strange. If Linus calls his kernel Linux and not gnuLinux then its called Linux. A name is a name. Who cares? I could call it Katzware! But its still Linux.
I agree that RMS has chosen an odd battle to fight with this GNU/Linux stuff. However, even RMS isn't trying to get Linus to change the name of the kernel.
The kernel is Linux. You can say "the Linux kernel" or you can just say "Linux". What annoys RMS is that people refer to their whole system as "my Linux system", as if the kernel were the most important part. So he wants people to say "my GNU/Linux system".
There is some justice in his request. If you count how many lines of code in a running system come from the GNU project, you will get a large number. And the compiler we use to build our Linux kernels is from the GNU project.
Presumably, if someone were to port the BSD userland to run over the Linux kernel, RMS would also be perfectly happy to hear people say "my BSD/Linux system".
All that said, RMS will find it to be a losing battle. When I am talking about my personal Linux system, I say "that's my Athlon XP system running Linux". The motherboard, hard disk, video card, and RAM are all pretty essential to my system's operations, and it would I suppose be more correct to say "that's my ASUS A7V333 Athlon XP system with a GeForce 4 and blah blah blah all running GNU/Linux". I just don't, though.
When I was running Windows 98, I usually said something like "my computer with Win98", as opposed to "my computer running Microsoft Windows 98 Second Edition". Most people can't be bothered to add on extra syllables.
The kernel really is the most important part, when you are tersely describing a computer, because it controls what software will run on that computer. Adding the "GNU/" prefix is more a sign of respect to the GNU project than a useful classifier that describes the system.
I'm trying to understand this fully, but I don't have all the background to do so.
From the announcement:
It is commonly known that x86 pagetables do not support the so-called executable bit in the pagetable entries - PROT_EXEC and PROT_READ are merged into a single 'read or execute' flag. This means that even if an application marks a certain memory area non-executable (by not providing the PROT_EXEC flag upon mapping it) under x86, that area is still executable, if the area is PROT_READ.
Is this limitation in the x86 processor MMU, or in the Linux kernel?
Is anyone working to fix this limitation? (E.g. AMD fixing it in 64-bit mode on the Opteron, or someone reworking the Linux kernel)
Furthermore, the x86 ELF ABI marks the process stack executable, which requires that the stack is marked executable even on CPUs that support an executable bit in the pagetables.
Does this statement imply that some x86 processors do the right thing with PROT_EXEC?
If Linus is willing to create a new ELF standard, could this problem be fixed? Would it be worth the pain of a new incompatible ELF standard? Are there any other problems that can only be fixed with a new version of ELF, so we can fix them together?
It would be sort of cool to get a new ELF standard, so that Linus can call the new kernel "3.0"!:-)
[Announcement] "Exec Shield", new Linux security feature
We are pleased to announce the first publically available source code release of a new kernel-based security feature called the "Exec Shield", for Linux/x86. The kernel patch (against 2.4.21-rc1, released under the GPL/OSL) can be downloaded from:
http://redhat.com/~mingo/exec-shield/
The exec-shield feature provides protection against stack, buffer or function pointer overflows, and against other types of exploits that rely on overwriting data structures and/or putting code into those structures. The patch also makes it harder to pass in and execute the so-called 'shell-code' of exploits. The patch works transparently, ie. no application recompilation is necessary.
Background: -----------
It is commonly known that x86 pagetables do not support the so-called executable bit in the pagetable entries - PROT_EXEC and PROT_READ are merged into a single 'read or execute' flag. This means that even if an application marks a certain memory area non-executable (by not providing the PROT_EXEC flag upon mapping it) under x86, that area is still executable, if the area is PROT_READ.
Furthermore, the x86 ELF ABI marks the process stack executable, which requires that the stack is marked executable even on CPUs that support an executable bit in the pagetables.
This problem has been addressed in the past by various kernel patches, such as Solar Designer's excellent "non-exec stack patch". These patches mostly operate by using the x86 segmentation feature to set the code segment 'limit' value to a certain fixed value that points right below the stack frame. The exec-shield tries to cover as much virtual memory via the code segment limit as possible - not just the stack.
Implementation: ---------------
The exec-shield feature works via the kernel transparently tracking executable mappings an application specifies, and maintains a 'maximum executable address' value. This is called the 'exec-limit'. The scheduler uses the exec-limit to update the code segment descriptor upon each context-switch. Since each process (or thread) in the system can have a different exec-limit, the scheduler sets the user code segment dynamically so that always the correct code-segment limit is used.
the kernel caches the user segment descriptor value, so the overhead in the context-switch path is a very cheap, unconditional 6-byte write to the GDT, costing 2-3 cycles at most.
Furthermore, the kernel also remaps all PROT_EXEC mappings to the so-called ASCII-armor area, which on x86 is the addresses 0-16MB. These addresses are special because they cannot be jumped to via ASCII-based overflows. E.g. if a buggy application can be overflown via a long URL:
then only ASCII (ie. value 1-255) characters can be used by attackers. If all executable addresses are in the ASCII-armor, then no attack URL can be used to jump into the executable code - ie. the attack cannot be successful. (because no URL string can contain the \0 character.) E.g. the recent sendmail remote root attack was an ASCII-based overflow as well.
With the exec-shield activated, and the 'cat' binary relinked into the the ASCII-armor, the following layout is created:
Remember the Klingons? That was spread out over several years.
No, I don't remember a cohesive story arc involving Klingons. I remember some episodes, spread over several years, that did together show some story progression. But B5 and DS9 had story arcs planned out in advance. B5 even had tremendous amounts of foreshadowing: since he knew what would happen in the future, JMS could put in lots of little signs and portents.
God knows we never saw the "save Earth against next to impossible odds!" plot before Crusade.
Heh. I may be stretching too far, especially given that Crusade has been off the air for years. Still, it's interesting that JMS has had two space SF series, one about a space station and one about saving the Earth, and Trek did DS9 and now this.
I find Enterprise the most interesting Star Trek series so far by a wide margin
You know, every Star Trek series has had great characters; great sets; great props; great special effects... I'd even say "great actors" although I mean "well suited to the show" as opposed to "the most talented of all actors".
All Trek needs for full greatness is great scripts. And we so seldom get them. Having Berman and his crew let go a little, and get more outside scripts, would be a good start.
J. Michael Straczynski, creator of Babylon 5, deserves credit for a bunch of creative new ideas... that were lifted for Star Trek.
Babylon 5 had a heavy story arc. Later, Deep Space 9 developed a story arc. Babylon 5 used CGI heavily when Trek was using models. Of course Trek now uses CGI; perhaps that one was inevitable, but they probably adopted it sooner because of the example of Babylon 5.
After Babylon 5, JMS had a short-lived series called Crusade. The ship in Crusade had a limited amount of time to find a cure that would save the lives of all humans on Earth. Now we find out that Enterprise is turning into Crusade -- they will have to go and stop the Xindi super-weapon.
And new hair styles? Given that Babylon 5 was famous for its wild hair styles, I was amazed they were hyping this.
All that said -- I'll try to hope. Stopping a superweapon is closer to "Trek with phasers" than preachy episodes like "Cogenitor". I'd like to see it be fun and exciting, with far less lecturing.
But I'm afraid that next week (the Borg prequel) is going to be the "jump the shark" episode.
As others have noted, if you currently have your CPU at 48 C, the new CPU probably will stay within spec using your existing cooling setup.
But depending on which CPU core your XP 1900+ has, the new CPU may not even be much hotter.
An XP 1900+ with a Palomino core dissipates about 60.7 Watts typical, 68 Watts max.
An XP 3000+ with a Barton core dissipates about 58.4 Watts typical, 74.3 Watts max. I should expect the new Athlon will be only slightly hotter than the Barton 3000+.
Oh by the way, the Thermal Design Power for a Pentium 4 at 3000 is 81.8 Watts. If we figure typical will be under 75% of that number, the typical heat dissipation of a Pentium 4 at 3000 will be similar to the AMD at 3000, and the worst-case slightly hotter.
well no I dont need a powerful CPU, a huge harddrive , etc etc.
Nor did I say you need them. The MobilePro has no hard drive and a weak CPU. An old Libretto from eBay has a small hard drive and a weak CPU. But both of those have relatively large screens (at least compared to a PDA), and touch-typable keyboards, and expandability (such as a PC Card slot). You want to do things like remotely control your computer, those features will be nice.
Why cant I pay bills and buy stuff with my PDA? why cant I check my email?
You can do all that, with a network enabled PDA that does 802.11b... but you will be doing it on a tiny screen. I used to have a Libretto with a 6" screen that was 640x480, and that's about as small as I'd like to go for web surfing.
Also you said you don't want to spend too much money. You will pay far more for the newest and shiniest PDA than for an old Libretto from eBay.
Your requirements are pretty stiff for a PDA. I use a PDA for these things: appointments, phone numbers and email addresses, todo lists, notes, reading novels, playing games, taking notes, and listening to music. Almost any PDA will do all these things well (maybe not the music part; lots of PDAs don't do music).
You could buy a Microsoft-based PDA, a PocketPC, and that might meet your specs. However, I hate Windows CE, much preferring the Palm system.
I have a Palm Tungsten T, and I love it. But there is no 802.11b for it yet. It should otherwise meet your specs, including the ability to play MP3 and Ogg files.
Personally, I think you would be better served with a really compact laptop. If you really want to check mail, upload/download files, listen to music, remotely run things on your desktop... a laptop really is the way to go. A Lindows Notebook PC is $800, and you can always install some other OS if you don't like Lindows. You can check eBay for a used tiny notebook. You could probably get a Libretto cheap, and that's really tiny!
You might even want to get a NEC MobilePro. That's a Windows CE based subnotebook: it's like a really big PDA with a really big screen and a really big keyboard. Jerry Pournelle uses one to take notes, and he gave it his "Cold, Dead Fingers" award (as in, if you want to take it away from him you will have to pry his cold dead fingers from around it).
If you insist on something in your pocket, maybe you should check out the Zaurus PDAs. Since they run real Linux, you can make them do lots of stuff, and I hear they are great. I've never used one yet so I don't know.
The fundamental reason why we get so much spam is this: it is almost free to send spam to us. The cost is so low, spamming is worth it even if there is a 0.00001% response rate.
To truly solve spam, we need to change that. It has to be no longer essentially free to spam us.
One way to solve this would be for some sort of Email Authority to impose a fee structure on sending and receiving emails. Wrong, wrong, wrong; the cure would be far worse than the disease. I want control of my email; I don't want bureaucrats in government to have any say over what I do with it. Suppose they decide they don't like me and I have to pay $25 per message to send email? (Anytime regulations get proposed, you should always ask yourself "What would happen if my worst enemy were in charge of applying these regulations?")
The correct way to solve this is for each of us to use a mail transfer agent that charges to accept email. Here is how it should work:
You specify an amount of money that it will cost a stranger to send you email. For most of us, five cents would be good, or even ten cents. If you are a famous person, such as Stephen King, you could set the bar higher to cut down the amount of email.
This scheme requires a ubiquitous micropayments system: it shouldn't cost $1 in fees to pay ten cents, and if I want to charge ten cents, you need to be able to pay it (the micropay system I use and the one you use need to be able to cooperate).
Your email client would have a button to click, that would refund the cost for a message you are glad to have received, and another button that adds the sender to your white list: anyone on that list can send you email for free.
(Note that you really don't want to trust the email headers to correctly identify who sent the email! Best would be for digital signatures, with GPG or something similar. If the signature matches your friend, then you trust that the message really is from your friend.)
I actually don't want to receive zero unsolicited messages. There are some messages I would like to get. Suppose it's a coupon for a store I'd like to shop at? But right now, they spam you with a fire hose; I get messages telling me how to increase my penis size, and messages telling me how to increase my breast size; and messages in languages I can't even read; and so on. They just don't care whether you would want the message. But if it cost them $10,000 to send emails to 100,000 people, they wouldn't be so random; they would try to send messages that the person might actually respond to.
Older HP DeskJet printers had slots for expansion cards. Most of the expansion cards were font cartridges (bitmapped fonts!) for use with WordPerfect or whatever. But there was a plug-in card for the DeskJet that made it understand PostScript. (Probably not very fast.) There have been "workgroup" DeskJet printers that understand PostScript also, such as the 1200C/PS.
By the way, the LaserJet 1200 does not contain genuine Adobe PostScript. It's some workalike.
A LaserJet 1200 is a great printer. I have one and I love it.
That RAM expansion slot is pretty standard. You can go to crucial.com and get a 64MB expansion for a LaserJet 1200 for about $30, which is a lot cheaper than an official HP expansion. I know because I did it.
Years ago, in the April issue of Computer Language magazine, there was an article that I consider the best April Fools computer-related story ever. Not only was it funny, it was actually educational!
It was on how to add a GOSUB statement to FORTH. (For those of you who never suffered under one of the old BASIC versions, GOSUB was the statement for calling a function which would return; as opposed to GOTO which would not return.)
About four different ways to write GOSUB were shown, each of which worked, and each one was better in some way than the previous one. Each one illustrated some aspect of FORTH.
The final version looked like this:
: GOSUB ; IMMEDIATE
This defines a function, GOSUB, that does nothing at all. Because it is IMMEDIATE, it does nothing at all during compile time, and thus generates no code. Because the default behavior of FORTH is that every statement is a function call, GOSUB is completely superfluous.
Another fond memory (heh) is the article from the early 80's on how to build your own 64KB random-access memory unit. Take 65,536 8-bit flip-flop chips, and install on a suitably large breadboard, connecting the various pins with lengths of wire, and soldering. The instructions called for a 1:65536 multiplexer and a 65536:1 demultiplexer for addressing the chips.
The author helpfully included calculations on power requirements and heat dissipation; I don't remember the exact numbers, but you could no doubt heat a room with this thing if it were somehow real.
"Chess doesn't even include the concept of diplomacy," they will say. "Why can't the two sides just appeal to some sort of Security Council that can veto the war and send both people home? And it's so violent. Oh sure, the pieces are 'captured'. Yeah, right. Everyone who has seen the first Harry Potter movie knows the brutal truth about what really happens when you 'capture' a piece in chess."
They might also point out that only one piece out of 16 on each side is female. Why not half? It's sexist as well as violent!
Spoilers for Kiki's Delivery Service. You Have Been Warned.
They made one drastic change: at the end, you hear Gigi's voice (Phil Hartman) saying "Kiki! Kiki!" And I think he even says "Can you hear me?"
This is a major change because it makes it clear that Kiki is once again able to understand Gigi. That was not at all in the original.
Why did Kiki ever become unable to understand Gigi? I researched this on the Web after I saw Kiki's Delivery Service for the first time, and what I found in an FAQ file was that Miyazaki added this plot point--it was not in the book he based the movie on. Miyazaki's idea was that Kiki could talk to Gigi because they were very close to each other, and now that they were growing apart (he, in his interest in the lady kitty; she, in her problems fitting in with her new life) they were no longer close enough to talk. She got her confidence back, and could fly again, but she couldn't ever be that close to Gigi again... but she could be that close to the black kitten, so she had a new close friend. Doors close in life, but new ones open too.
Miyazaki is a god, and I'm not, but this is the one place where I would overrule him if I could. I think his whole spin on the talking Gigi thing is just strange. So I actually kind of like the idea that once she got her confidence back, she could hear Gigi again. But it is a change, and a big one.
Do you remember when Enterprise was first introduced? We were promised it would be "Star Trek with phasers". In other words, lots of action, less "character development" episodes and other slow topics.
That recent "Stigma" episode (T'Pol has mind-meld disease) was as far from "Star Trek with phasers" as you can get. On the other hand, that recent "Canamar" epsiode (Con Air, in space) was pretty cool.
Here is the best hope for the series: Berman and Pillar have stopped writing all the episodes. Every time I watch Enterprise, I make careful note of who wrote the episode. The whole first season was purely written by Berman and Pillar. Recently, we have had a string of episodes written by other writers.
The key issue here, as far as I can tell, is whether the XFree86 guys were correct to kick Keith Packard out.
On the one hand, David Wexelblat has strong words about Keith Packard's actions:
What Keith has done is among the most low-class, unprofessional, and tactless things I have ever experienced in my professional career.
For Keith to blatantly lie to the Core Team about what he was doing is utterly unacceptable.
But what Keith is doing, at least how he's handled it, is just flat out wrong. It's literally dishonest, and morally repugnant. Doesn't mean that there aren't some valid issues to work, or that there is no need for branching, but (a) it remains to be proven, and (b) I'll be damned if I'll quietly accept it being done by someone who is lying to my face.
Whew. On the other hand, here's what Keith Packard has to say:
Some have suggested that this was a secret attempt to undermine the XFree86 project: this was not my intent. I have tried as hard as I can to work within the existing XFree86 structure.
It's hard to think that this is some kind of misunderstanding. Either Keith has been lying, or else he hasn't. It's impossible for us to really decide for ourselves, since the emails containing the alleged lying are not public.
David Wexelblat said:
There is an email thread documenting this. Some members of the BOD wanted to post the email, or quotes therefrom, with the announcement. I and some of the others were utterly uncomfortable doing that. I don't think anyone on the BOD or Core Team would have any issues with an independent audit of this email thread, if there are concerns about the veracity of what I say, but airing that in public isn't appropriate, IMHO.
I'd like to see someone I trust given the job of auditing those emails, and judging whether Keith Packard has in fact been lying.
P.S. A fork might be a good thing, in the end. Keith Packard says he believes his fork can attract more developers and improve more quickly than the status quo. If he can pull that off, we will all be better off. But unless he can clear his name, he may have trouble attracting developers.
Slashdot Mirror
If you did pick up some noise, it would be from the repeated DCT and inverse DCT, not from the quantization itself. The ideal DCT wouldn't introduce noise, as I understand it, but in real life there might be rounding errors.
From your explanation, as the quality setting is applied to jpegs, it sounds like recompressing a jpeg would not increase the loss if the same quantizer was used as was used on the original. Is that true?
That should be true.
JPEG, and the video codecs that I know about, all divide an image up into squares of 8 pixels by 8 pixels. Then each square is processed. For JPEG, the squares are run through the DCT.
So only the edited 8x8 squares should actually be affected.
I haven't tested this, though.
steveha
So when I run GNU/Linux on one machine, and GNU/HURD on another, I should call one "Linux" and the other "HURD"?
Up to you, dude. RMS would have you call the second one "GNU" because the HURD is the kernel for the GNU operating system.
I'd probably call one "Linux" and the other "HURD" (sorry, RMS).
What if I install SunOS + Solaris, remove everything but the kernel, and somehow manage to get the thing to boot with only GNU tools? Is it still SunOS? Or is is GNU/SunOS?
I'll bet RMS would be happy if you called it "GNU/SunOS". If you actually made an install like that, I'd probably call it "GNU/SunOS" too, since you made such a point of ripping away everything but GNU tools.
Of course, you contradict yourself; as if the kernel were the most important part vs. The kernel really is the most important part
Make your mind up... but of course, you can't.
Go back and read it again, and decide for yourself whether I can make up my mind or whether I was just trying to write in a non-boring way. Hint: I don't agree with RMS 100% about everything.
It's partially a political thing, but really a technical one; just as a server is useless without storage, Linux is useless without GNU.
And RMS's campaign to have everyone say "GNU/Linux system" instead of "Linux system" is doomed by GNU's very success. GNU tools are omnipresent. Since they are always there, few people will bother to mention them! Just as people might say "This box runs GNOME" but they won't usually bother to say "This box runs GNOME on top of XFree86".
As I said, anyone who actually says "GNU/Linux" is doing it out of respect for GNU and not because it's a useful way to describe a Linux system.
steveha
I can explain quantizers.
To compress a sample of audio or video data, you want to throw away details that don't matter. So you transform the data using a function, and then you apply a quantizer to throw away some of the data. The whole point of the transform function is to make a data set that is "safer" to quantize.
Most audio and video codecs use the DCT function, which decomposes an image into a series of waves. If you throw away some of the wave data, you get a similar series of waves, and hopefully humans won't notice the differences.
Applying the quantizer is the "lossy" step in lossy compression. Here's how you do it:
You divide the output data (from the transform function) by some integer number, the quantizer, and you use integer division. (Because you want to simplify the data, you actually want the fractional part discarded!) The bigger the quantizer, the more data is thrown away. A quantizer of 1 would give perfect quality, but very poor compression.
Here is a semi-real example:
You have some data to compress. Let's say it's eight bytes (maybe an audio sample for Ogg Vorbis).
After the DCT, it looks like this (using decimal numbers because it's easier):
355 244 33 192 43 9 3 8
Quantize with a quantizer of 8:
{8} 44 30 4 24 5 1 0 1
The {8} is intended to show that the quantizer has to be stored with the quantized data stream, so we know what quantizer was used so we can dequantize.
On dequantization, we just multiply by 8 again:
352 240 32 192 40 8 0 8
Then we feed these resulting numbers into the inverse DCT, and get back some data that is hopefully not full of artifacts.
When you are encoding, you quantize, and then you encode the resulting stream of numbers. Because the values of the numbers are smaller (e.g. 24, instead of 192) you can encode the values with fewer bits. And you can apply run-length compression to the quantized values; in my example it wouldn't buy you much, but in real examples you will often have a bunch of zeroes in a row.
Video compression dialogs (such as the JPEG options in "Save As" in The GIMP) often give you a slider for a quality percentage. This is actually controlling the quantizer. For JPEG, you have 31 possible settings for the quantizer, so there is no difference between 80% and 81%.
By the way, my personal experience with video compression is that a quantizer of 8 gives pretty good compression with very few visible artifcats. An 8 quantizer corresponds with about a 75% in the quality sliders (such as Save As in The GIMP).
Some afternoon when you are bored, save the same JPEG image over and over with different quality settings, and watch what happens with the artifacts. When you set the quality down to 2% (corresponding to a 31 quantizer) you get ugly, coarse blocks. The GIMP under Linux is particularly good for this because you can drag the quality slider and the preview updates instantly, so you don't have to save and re-open.
steveha
Well, since you asked...
Yes, the site came up fine for me. And I actually wasn't too worried about Red Hat's servers getting Slashdotted; I think Red Hat has pretty good servers.
And I wasn't even looking for free karma; I already have plenty.
I know that many, many people just read the Slashdot comments and never click through the links, and I wanted them to see the announcement before asking questions answered in the announcement. I was hoping to head off some of the more obvious dumb questions.
Also, the URL provided is for a very temporary place: Ingo's personal stuff. A year from now people might read the archived comments on Slashdot, and I thought it would be nice to have the announcement in there, especially because the link probably won't work anymore. (Although I may have been wrong about the link not working anymore; there seems to be lots of older stuff on there that he never took down. Oh, well.)
steveha
However his die hard views seem strange. If Linus calls his kernel Linux and not gnuLinux then its called Linux. A name is a name. Who cares? I could call it Katzware! But its still Linux.
I agree that RMS has chosen an odd battle to fight with this GNU/Linux stuff. However, even RMS isn't trying to get Linus to change the name of the kernel.
The kernel is Linux. You can say "the Linux kernel" or you can just say "Linux". What annoys RMS is that people refer to their whole system as "my Linux system", as if the kernel were the most important part. So he wants people to say "my GNU/Linux system".
There is some justice in his request. If you count how many lines of code in a running system come from the GNU project, you will get a large number. And the compiler we use to build our Linux kernels is from the GNU project.
Presumably, if someone were to port the BSD userland to run over the Linux kernel, RMS would also be perfectly happy to hear people say "my BSD/Linux system".
All that said, RMS will find it to be a losing battle. When I am talking about my personal Linux system, I say "that's my Athlon XP system running Linux". The motherboard, hard disk, video card, and RAM are all pretty essential to my system's operations, and it would I suppose be more correct to say "that's my ASUS A7V333 Athlon XP system with a GeForce 4 and blah blah blah all running GNU/Linux". I just don't, though.
When I was running Windows 98, I usually said something like "my computer with Win98", as opposed to "my computer running Microsoft Windows 98 Second Edition". Most people can't be bothered to add on extra syllables.
The kernel really is the most important part, when you are tersely describing a computer, because it controls what software will run on that computer. Adding the "GNU/" prefix is more a sign of respect to the GNU project than a useful classifier that describes the system.
steveha
I'm trying to understand this fully, but I don't have all the background to do so.
:-)
From the announcement:
It is commonly known that x86 pagetables do not support the so-called executable bit in the pagetable entries - PROT_EXEC and PROT_READ are merged into a single 'read or execute' flag. This means that even if an application marks a certain memory area non-executable (by not providing the PROT_EXEC flag upon mapping it) under x86, that area is still executable, if the area is PROT_READ.
Is this limitation in the x86 processor MMU, or in the Linux kernel?
Is anyone working to fix this limitation? (E.g. AMD fixing it in 64-bit mode on the Opteron, or someone reworking the Linux kernel)
Furthermore, the x86 ELF ABI marks the process stack executable, which requires that the stack is marked executable even on CPUs that support an executable bit in the pagetables.
Does this statement imply that some x86 processors do the right thing with PROT_EXEC?
If Linus is willing to create a new ELF standard, could this problem be fixed? Would it be worth the pain of a new incompatible ELF standard? Are there any other problems that can only be fixed with a new version of ELF, so we can fix them together?
It would be sort of cool to get a new ELF standard, so that Linus can call the new kernel "3.0"!
steveha
http://people.redhat.com/mingo/exec-shield/ANNOUNC E-exec-shield
./cat-lowaddr /proc/self/maps /lib/ld-2.3.2.so /lib/ld-2.3.2.so /lib/libc-2.3.2.so
_____________________________________________
[Announcement] "Exec Shield", new Linux security feature
We are pleased to announce the first publically available source code
release of a new kernel-based security feature called the "Exec Shield",
for Linux/x86. The kernel patch (against 2.4.21-rc1, released under the
GPL/OSL) can be downloaded from:
http://redhat.com/~mingo/exec-shield/
The exec-shield feature provides protection against stack, buffer or
function pointer overflows, and against other types of exploits that rely
on overwriting data structures and/or putting code into those structures.
The patch also makes it harder to pass in and execute the so-called
'shell-code' of exploits. The patch works transparently, ie. no
application recompilation is necessary.
Background:
-----------
It is commonly known that x86 pagetables do not support the so-called
executable bit in the pagetable entries - PROT_EXEC and PROT_READ are
merged into a single 'read or execute' flag. This means that even if an
application marks a certain memory area non-executable (by not providing
the PROT_EXEC flag upon mapping it) under x86, that area is still
executable, if the area is PROT_READ.
Furthermore, the x86 ELF ABI marks the process stack executable, which
requires that the stack is marked executable even on CPUs that support an
executable bit in the pagetables.
This problem has been addressed in the past by various kernel patches,
such as Solar Designer's excellent "non-exec stack patch". These patches
mostly operate by using the x86 segmentation feature to set the code
segment 'limit' value to a certain fixed value that points right below the
stack frame. The exec-shield tries to cover as much virtual memory via the
code segment limit as possible - not just the stack.
Implementation:
---------------
The exec-shield feature works via the kernel transparently tracking
executable mappings an application specifies, and maintains a 'maximum
executable address' value. This is called the 'exec-limit'. The scheduler
uses the exec-limit to update the code segment descriptor upon each
context-switch. Since each process (or thread) in the system can have a
different exec-limit, the scheduler sets the user code segment dynamically
so that always the correct code-segment limit is used.
the kernel caches the user segment descriptor value, so the overhead in
the context-switch path is a very cheap, unconditional 6-byte write to the
GDT, costing 2-3 cycles at most.
Furthermore, the kernel also remaps all PROT_EXEC mappings to the
so-called ASCII-armor area, which on x86 is the addresses 0-16MB. These
addresses are special because they cannot be jumped to via ASCII-based
overflows. E.g. if a buggy application can be overflown via a long URL:
http://somehost/buggy.app?realyloooooooooooooooooo oong.123489719875
then only ASCII (ie. value 1-255) characters can be used by attackers. If
all executable addresses are in the ASCII-armor, then no attack URL can be
used to jump into the executable code - ie. the attack cannot be
successful. (because no URL string can contain the \0 character.) E.g. the
recent sendmail remote root attack was an ASCII-based overflow as well.
With the exec-shield activated, and the 'cat' binary relinked into the the
ASCII-armor, the following layout is created:
$
00101000-00116000 r-xp 00000000 03:01 319365
00116000-00117000 rw-p 00014000 03:01 319365
00117000-0024a000 r-xp 00000000 03:01 319439
0024a000-00
Remember the Klingons? That was spread out over several years.
No, I don't remember a cohesive story arc involving Klingons. I remember some episodes, spread over several years, that did together show some story progression. But B5 and DS9 had story arcs planned out in advance. B5 even had tremendous amounts of foreshadowing: since he knew what would happen in the future, JMS could put in lots of little signs and portents.
God knows we never saw the "save Earth against next to impossible odds!" plot before Crusade.
Heh. I may be stretching too far, especially given that Crusade has been off the air for years. Still, it's interesting that JMS has had two space SF series, one about a space station and one about saving the Earth, and Trek did DS9 and now this.
I find Enterprise the most interesting Star Trek series so far by a wide margin
You know, every Star Trek series has had great characters; great sets; great props; great special effects... I'd even say "great actors" although I mean "well suited to the show" as opposed to "the most talented of all actors".
All Trek needs for full greatness is great scripts. And we so seldom get them. Having Berman and his crew let go a little, and get more outside scripts, would be a good start.
steveha
I hope they will install some cool fighters, and a huge Wave Motion Gun.
And there should be a countdown at the end of each episode: 43 days remain until Earth is destroyed!!
steveha
J. Michael Straczynski, creator of Babylon 5, deserves credit for a bunch of creative new ideas... that were lifted for Star Trek.
Babylon 5 had a heavy story arc. Later, Deep Space 9 developed a story arc. Babylon 5 used CGI heavily when Trek was using models. Of course Trek now uses CGI; perhaps that one was inevitable, but they probably adopted it sooner because of the example of Babylon 5.
After Babylon 5, JMS had a short-lived series called Crusade. The ship in Crusade had a limited amount of time to find a cure that would save the lives of all humans on Earth. Now we find out that Enterprise is turning into Crusade -- they will have to go and stop the Xindi super-weapon.
And new hair styles? Given that Babylon 5 was famous for its wild hair styles, I was amazed they were hyping this.
All that said -- I'll try to hope. Stopping a superweapon is closer to "Trek with phasers" than preachy episodes like "Cogenitor". I'd like to see it be fun and exciting, with far less lecturing.
But I'm afraid that next week (the Borg prequel) is going to be the "jump the shark" episode.
steveha
As others have noted, if you currently have your CPU at 48 C, the new CPU probably will stay within spec using your existing cooling setup.
But depending on which CPU core your XP 1900+ has, the new CPU may not even be much hotter.
An XP 1900+ with a Palomino core dissipates about 60.7 Watts typical, 68 Watts max.
An XP 3000+ with a Barton core dissipates about 58.4 Watts typical, 74.3 Watts max. I should expect the new Athlon will be only slightly hotter than the Barton 3000+.
Oh by the way, the Thermal Design Power for a Pentium 4 at 3000 is 81.8 Watts. If we figure typical will be under 75% of that number, the typical heat dissipation of a Pentium 4 at 3000 will be similar to the AMD at 3000, and the worst-case slightly hotter.
These numbers from:
http://users.erols.com/chare/elec.htm
steveha
well no I dont need a powerful CPU, a huge harddrive , etc etc.
Nor did I say you need them. The MobilePro has no hard drive and a weak CPU. An old Libretto from eBay has a small hard drive and a weak CPU. But both of those have relatively large screens (at least compared to a PDA), and touch-typable keyboards, and expandability (such as a PC Card slot). You want to do things like remotely control your computer, those features will be nice.
Why cant I pay bills and buy stuff with my PDA? why cant I check my email?
You can do all that, with a network enabled PDA that does 802.11b... but you will be doing it on a tiny screen. I used to have a Libretto with a 6" screen that was 640x480, and that's about as small as I'd like to go for web surfing.
Also you said you don't want to spend too much money. You will pay far more for the newest and shiniest PDA than for an old Libretto from eBay.
steveha
Perhaps by Palm OS 6, the POS will both be good enough
Now, now, PalmOS may not be perfect, but there's no reason to call it a "POS".
steveha
Your requirements are pretty stiff for a PDA. I use a PDA for these things: appointments, phone numbers and email addresses, todo lists, notes, reading novels, playing games, taking notes, and listening to music. Almost any PDA will do all these things well (maybe not the music part; lots of PDAs don't do music).
You could buy a Microsoft-based PDA, a PocketPC, and that might meet your specs. However, I hate Windows CE, much preferring the Palm system.
I have a Palm Tungsten T, and I love it. But there is no 802.11b for it yet. It should otherwise meet your specs, including the ability to play MP3 and Ogg files.
Personally, I think you would be better served with a really compact laptop. If you really want to check mail, upload/download files, listen to music, remotely run things on your desktop... a laptop really is the way to go. A Lindows Notebook PC is $800, and you can always install some other OS if you don't like Lindows. You can check eBay for a used tiny notebook. You could probably get a Libretto cheap, and that's really tiny!
You might even want to get a NEC MobilePro. That's a Windows CE based subnotebook: it's like a really big PDA with a really big screen and a really big keyboard. Jerry Pournelle uses one to take notes, and he gave it his "Cold, Dead Fingers" award (as in, if you want to take it away from him you will have to pry his cold dead fingers from around it).
If you insist on something in your pocket, maybe you should check out the Zaurus PDAs. Since they run real Linux, you can make them do lots of stuff, and I hear they are great. I've never used one yet so I don't know.
Good luck.
steveha
The fundamental reason why we get so much spam is this: it is almost free to send spam to us. The cost is so low, spamming is worth it even if there is a 0.00001% response rate.
To truly solve spam, we need to change that. It has to be no longer essentially free to spam us.
One way to solve this would be for some sort of Email Authority to impose a fee structure on sending and receiving emails. Wrong, wrong, wrong; the cure would be far worse than the disease. I want control of my email; I don't want bureaucrats in government to have any say over what I do with it. Suppose they decide they don't like me and I have to pay $25 per message to send email? (Anytime regulations get proposed, you should always ask yourself "What would happen if my worst enemy were in charge of applying these regulations?")
The correct way to solve this is for each of us to use a mail transfer agent that charges to accept email. Here is how it should work:
You specify an amount of money that it will cost a stranger to send you email. For most of us, five cents would be good, or even ten cents. If you are a famous person, such as Stephen King, you could set the bar higher to cut down the amount of email.
This scheme requires a ubiquitous micropayments system: it shouldn't cost $1 in fees to pay ten cents, and if I want to charge ten cents, you need to be able to pay it (the micropay system I use and the one you use need to be able to cooperate).
Your email client would have a button to click, that would refund the cost for a message you are glad to have received, and another button that adds the sender to your white list: anyone on that list can send you email for free.
(Note that you really don't want to trust the email headers to correctly identify who sent the email! Best would be for digital signatures, with GPG or something similar. If the signature matches your friend, then you trust that the message really is from your friend.)
I actually don't want to receive zero unsolicited messages. There are some messages I would like to get. Suppose it's a coupon for a store I'd like to shop at? But right now, they spam you with a fire hose; I get messages telling me how to increase my penis size, and messages telling me how to increase my breast size; and messages in languages I can't even read; and so on. They just don't care whether you would want the message. But if it cost them $10,000 to send emails to 100,000 people, they wouldn't be so random; they would try to send messages that the person might actually respond to.
steveha
Older HP DeskJet printers had slots for expansion cards. Most of the expansion cards were font cartridges (bitmapped fonts!) for use with WordPerfect or whatever. But there was a plug-in card for the DeskJet that made it understand PostScript. (Probably not very fast.) There have been "workgroup" DeskJet printers that understand PostScript also, such as the 1200C/PS.
By the way, the LaserJet 1200 does not contain genuine Adobe PostScript. It's some workalike.
It works for me, so I don't care.
steveha
steveha
A LaserJet 1200 is a great printer. I have one and I love it.
That RAM expansion slot is pretty standard. You can go to crucial.com and get a 64MB expansion for a LaserJet 1200 for about $30, which is a lot cheaper than an official HP expansion. I know because I did it.
steveha
Years ago, in the April issue of Computer Language magazine, there was an article that I consider the best April Fools computer-related story ever. Not only was it funny, it was actually educational!
It was on how to add a GOSUB statement to FORTH. (For those of you who never suffered under one of the old BASIC versions, GOSUB was the statement for calling a function which would return; as opposed to GOTO which would not return.)
About four different ways to write GOSUB were shown, each of which worked, and each one was better in some way than the previous one. Each one illustrated some aspect of FORTH.
The final version looked like this:
: GOSUB ; IMMEDIATE
This defines a function, GOSUB, that does nothing at all. Because it is IMMEDIATE, it does nothing at all during compile time, and thus generates no code. Because the default behavior of FORTH is that every statement is a function call, GOSUB is completely superfluous.
Another fond memory (heh) is the article from the early 80's on how to build your own 64KB random-access memory unit. Take 65,536 8-bit flip-flop chips, and install on a suitably large breadboard, connecting the various pins with lengths of wire, and soldering. The instructions called for a 1:65536 multiplexer and a 65536:1 demultiplexer for addressing the chips.
The author helpfully included calculations on power requirements and heat dissipation; I don't remember the exact numbers, but you could no doubt heat a room with this thing if it were somehow real.
steveha
You know, my attempt to be funny sort of worked and is a little bit funny.
:-)
Your reply, on the other hand, made me laugh out loud!
Someone please mod this up.
steveha
"Chess doesn't even include the concept of diplomacy," they will say. "Why can't the two sides just appeal to some sort of Security Council that can veto the war and send both people home? And it's so violent. Oh sure, the pieces are 'captured'. Yeah, right. Everyone who has seen the first Harry Potter movie knows the brutal truth about what really happens when you 'capture' a piece in chess."
They might also point out that only one piece out of 16 on each side is female. Why not half? It's sexist as well as violent!
steveha
Spoilers for Kiki's Delivery Service. You Have Been Warned.
They made one drastic change: at the end, you hear Gigi's voice (Phil Hartman) saying "Kiki! Kiki!" And I think he even says "Can you hear me?"
This is a major change because it makes it clear that Kiki is once again able to understand Gigi. That was not at all in the original.
Why did Kiki ever become unable to understand Gigi? I researched this on the Web after I saw Kiki's Delivery Service for the first time, and what I found in an FAQ file was that Miyazaki added this plot point--it was not in the book he based the movie on. Miyazaki's idea was that Kiki could talk to Gigi because they were very close to each other, and now that they were growing apart (he, in his interest in the lady kitty; she, in her problems fitting in with her new life) they were no longer close enough to talk. She got her confidence back, and could fly again, but she couldn't ever be that close to Gigi again... but she could be that close to the black kitten, so she had a new close friend. Doors close in life, but new ones open too.
Miyazaki is a god, and I'm not, but this is the one place where I would overrule him if I could. I think his whole spin on the talking Gigi thing is just strange. So I actually kind of like the idea that once she got her confidence back, she could hear Gigi again. But it is a change, and a big one.
steveha
Oops. My bad.
steveha
Do you remember when Enterprise was first introduced? We were promised it would be "Star Trek with phasers". In other words, lots of action, less "character development" episodes and other slow topics.
That recent "Stigma" episode (T'Pol has mind-meld disease) was as far from "Star Trek with phasers" as you can get. On the other hand, that recent "Canamar" epsiode (Con Air, in space) was pretty cool.
Here is the best hope for the series: Berman and Pillar have stopped writing all the episodes. Every time I watch Enterprise, I make careful note of who wrote the episode. The whole first season was purely written by Berman and Pillar. Recently, we have had a string of episodes written by other writers.
If they want to make us happy, they ought to get some scripts from actual SF authors. How about John E. Stith, David Weber, or Catherine Asaro? (I draw the line at Piers Anthony, though...)
steveha
The key issue here, as far as I can tell, is whether the XFree86 guys were correct to kick Keith Packard out.
On the one hand, David Wexelblat has strong words about Keith Packard's actions:
What Keith has done is among the most low-class, unprofessional, and tactless things I have ever experienced in my professional career.
For Keith to blatantly lie to the Core Team about what he was doing is utterly unacceptable.
But what Keith is doing, at least how he's handled it, is just flat out wrong. It's literally dishonest, and morally repugnant. Doesn't mean that there aren't some valid issues to work, or that there is no need for branching, but (a) it remains to be proven, and (b) I'll be damned if I'll quietly accept it being done by someone who is lying to my face.
Whew. On the other hand, here's what Keith Packard has to say:
Some have suggested that this was a secret attempt to undermine the XFree86 project: this was not my intent. I have tried as hard as I can to work within the existing XFree86 structure.
It's hard to think that this is some kind of misunderstanding. Either Keith has been lying, or else he hasn't. It's impossible for us to really decide for ourselves, since the emails containing the alleged lying are not public.
David Wexelblat said:
There is an email thread documenting this. Some members of the BOD wanted to post the email, or quotes therefrom, with the announcement. I and some of the others were utterly uncomfortable doing that. I don't think anyone on the BOD or Core Team would have any issues with an independent audit of this email thread, if there are concerns about the veracity of what I say, but airing that in public isn't appropriate, IMHO.
I'd like to see someone I trust given the job of auditing those emails, and judging whether Keith Packard has in fact been lying.
P.S. A fork might be a good thing, in the end. Keith Packard says he believes his fork can attract more developers and improve more quickly than the status quo. If he can pull that off, we will all be better off. But unless he can clear his name, he may have trouble attracting developers.
steveha