Offtopic, but how does your dtach program differ from the bash "disown" command? The only thing I see is that it looks like you can reattach to a detached program. I'm not sure if you can do that with the "disown" command. (I'm afraid I never had a reason to.)
On Intel chips there is no way to tell the difference between memory that can be read, and memory that can be executed.
Incorrect. Read Section 3.2.3 "Multi-segment Model" of the "IA-32 Intel® Architecture Software Developer's Manual: Volume 3: System Programming Guide". It lists both data and code segments quite clearly. This stuff has been around since the 386 days.
That is NOT what my Pentium manual says. My pentium manual (Section 3.2.3: Multi-segment Model) shows that a segment type breaks up into four bits. The first bit is Data or Code. The second bit is expand-down or conforming. The third byte is read/write access. The fourth is some useless "Access" flag to tell you if it's been accessed since you cleared the flag.
The manual says that you can have an execute only segment as long as you load the executable constants into some other segment. (Or alternatively, map a data segment over the constant pool.) You can have it executable and read only, but you can't write to it without a data segment mapped over the same location.
Now what I'm saying, is that the OS should delete the data segment after it's done loading the code. That way, the only access will be via the IP register and (optionally) read only access of the constants pool.
No, wait, I've got it now. The return pointer. Ouch, how could I be so blind? I forgot the fact that an interrupt or subroutine stores the pointer on the stack. The only problem I see is, shouldn't his data still be in a data segment? And if the data segment is not executable, he shouldn't be able to return to it, should he?
As I said, the actual security benefit of moving the stack is very limited. The perf consequences, by contrast, are drastic.
Ding, ding, ding! I think I've got it. You're saying that the stack is set to point to a relative address so that the CPU doesn't have to use a long pointer to access the stack. Use of a shorter pointer improves performance drastically because the CPU is saving itself quite a few lookups and offset computations. Right?
Basically, yes. A GPF occurs when the data is written outside of the currently selected segment. A new segment selector is required to write to a different segment. An execute-only segment tells the processor that a request for a write segment is invalid.
Also, GPFs are "General" protection faults. The concept itself is not tremendously descriptive on what circumstances it's thrown. Under Unix, they referred to it as a "segment violation".
A thread's stack must be readable and writable by the process, right. Therefore, on an x86, was required to be executable. Oops -- that means you can jump to an address on the stack and run code there.
Parsing...
Ok, I think I've got you now. Correct me if I'm wrong though, but isn't that a problem with the way that Windows executable are directly mapped to memory? IIRC, the SP register takes a selector. That selector should be able to point to any location in the GDT or LDT table. Hmmm... let me check my notes...
The x86 only has two bits per page, Readable and Writable. You can't mark a page of memory as readable without marking it as being executable, they are the same thing on the x86.
But that's my point. You shouldn't *need* to make code memory readable. Mark it as 100b, and make sure that the only register that references it is the Instruction Pointer. The processor will still get its instructions, and the OS will be safe and happy.
But what if my buffer overflow overwrote code already marked as Execute?
Then it would pass from a data segment into a code segment. If it passes into a code segment, the processor will throw a general protection fault. The OS will catch the General Protection Fault and kill the program. Keep in mind that standard mallocs should only be returning data segments of memory. So (in theory) the overflow can't be writing to execute memory.
The problem is that there are historical and legitimate reasons for executing "data" and so OSs allow it.
Which is where the NX flag won't help. You *still* need a way of telling the OS whether a segment should be executable or not. What the OS should allow is to allocate a data segment for code loading. Once loaded, a special kernel method is called to switch it to execute-only. That would support JITs, and prevent buffer overflows.
Furthermore, I may be wrong here, but I don't think the original x86 had the capability of marking code as data or code, so if that was done it was all in the OS.
The 8086 instruction set had no concept of memory protection. However, the 80386 introduced a full protected memory mode that was later extended by the 80486. Of course, back then everyone put one entry into the GDT (0-end) and set it to executable and data. Worked great for video games. It wasn't until someone got the bright idea of creating a protected mode OS for the PC (WOW!) that segment protection was used. Of course, they STILL didn't segment data and code. *grumble*
I still don't get it. So you allocate a section for code. It's initially marked as a read/write data or code segment (001b or 101b). Once the code is loaded, you mark it 100b (execute only) and it's 100% protected until you reallocate it. What's the problem here?
My understanding is that the real issue has been supporting self-morphing code. Since a lot of legacy code liked this performance trick (especially games), all segments were marked as data and code.
It wouldn't be a problem if OSes properly separated code and data segments. By marking it properly in the GDT or LDT, even an exploit in the kernel shouldn't allow arbitrary code to run. And if the write does continue into a code segment, a general protection fault should occur.
This NX thing sounds like just a bunch of hoopla to make Microsoft start doing their job. Unfortunately, I don't think Linux is much better.:-/
Ok, maybe I don't get this. How is this different from marking a block of memory as data, not code? The real problem is that certain OSes mark all memory as both code and data. Sure, it's easier on the bookkeeping, but it allows buffer overflows. If data was kept in data segments, and code was kept in code segment, the worst that would happen is a corrupted data segment, and/or a General Protection Fault.
The soviet system lacked rewards based on performance and ways to control those who were in control.
Not quite true. If you got into the graces of the Communists, you could get a really nice high rise apartment and have lots of money. Otherwise you had to live dirt poor. So there was an incentive to do well.
One has to wonder why we call it Quantum Encryption when it really has nothing to do with Encryption. From the article:
The aim is to produce a communication system that cannot be intercepted by anyone
If I understand their intent, they plan to use concepts like Quantum Entanglement to ensure that communication is shared only between the entangled particles. This is a very different concept from using the properties of Quantum Mechanics to scramble information in a reversible manner or creating computers capable of super-fast calculations.
I believe the poster's point was that 3Com should be able to make a driver for each kernel version (or key stable ones). It need not necessarily be supported (i.e., continually "beta"), but, just having the driver would be a good thing; --even for 3Com.
1. Recompiling and QAing for each kernel version is money out of 3Com's pocket. 2. What happens if I build a custom kernel? (Hint: It breaks permanently.)
As for your statement about communism; yes, my idea shares certain characteristics with it. However, the main reasons that (so-called) communist systems have failed is that they tried to make things work without rewards, and power was concentrated in the hands of few, with no real checks in place.
Actually, I should use the more correct term. What you propose is "socialism". A dash here and there isn't necessarily a bad thing, but funds like Social Security show how badly a government manages these things. The real problem is that such concepts are very idealistic and tend to fail when hit with the real world. For example, your suggestion raises the following questions:
1. Who decides what projects get money? 2. How do they decide how much money to give? 3. How do they prevent pie in the sky projects (e.g. a "real soon now" warp drive) from eating up the budget, year after year?
In the market, you have to show a return on money spent, so the first is solved. The second is decided based on the return formula. The third is handled by cost overruns. If a project gets too expensive with little chance of success, it gets killed.
Academia handles pie in the sky stuff more elegantly because a project can continue as long as it can find relatively small amounts of funding. And if other scientists/engineers believe the project to be a dud, it will live and die by the team originally assigned to it. That's not to say that there isn't tons of wonderful stuff that "should have been" or "could be", but there are times when things should be left alone and revisited with more knowledge and experience.
Can Realtek release an 8019 driver for WinXP? An AGP driver for Win95 (retail)?
You're confusing things. 3Com makes WinModems. Therefore they should be able to release a binary driver for Linux. But they can't. You know why? Because binary drivers are specific to the version of the kernel.
Your reasoning about 8019 and AGP are flawed, because they are core hardware components that are built according to open industry specifications. If RealTek released a motherboard with the new MXM feature, you're damn right I'd expect them to produce a driver.
OSS is also a chain reaction, which they fear. Imagine OSS does 99,5% of what a company needs to do. So they write the other 0,5% (as OSS, not their core business + they get community support and maintenance). Now suddnly some other company went from 99,2% -> 99,5%. So they write another 0,5%. Suddenly the snowball is rolling.
From a market perspective, this is a good thing. Lowering the barriers to entry forces companies to produce more innovation and cheaper products to keep ahead of the competition. Every company would *like* to have a monopoly, but monopolies inherently cause development to stagnate. Take phone companies as an example. AT&T had a monopoly and was able to keep rates the same with practically no need to technological development. The Feds suddenly allow competitors and Fibre Optics, DSL, lower prices, etc. develop.
YaST is a "glue" type of product that gave SuSE a competitive advantage. Releasing it as a commercial product would only have created one more difficult to install binary that would make your life easier if you could only get it installed.
The alternative would have been to be a technology licensor, and attempt to license it to RedHat and Mandrake. My guess, however, is that SuSE would have met with a "DIY mentality" and been shunned.
There's a flaw in your counter-argument. You're equating software development with patents. However, keeping source code closed does not in any way prevent a competitor from developing the same technology simultaneously. Thus the advantages of funding R&D while turning a profit still hold true.
What I envision is a system where funds are collected and pooled (like taxes), so that they can be spent on R&D and the like in a manner that all can benefit from. With the results available for everyone to use, there can be competition, and the market forces can work for us all instead of for the select few who control the patents etc.
This is communism, and it won't work without becoming just another pool of public money for governments to mismanage. The US has actually had a system in place to encourage funding for R&D. That system is academic research. Many companies pay a "smart guy" at a university to do a large amount of legwork for them at a low cost. This information is then usually available to the rest of the industry. Companies then turn around and complete the development of this research into a full product or technology.
Can Adobe create a single binary that installs, works, and takes advantage of Windows 9x, NT, 2000, and XP? Can Adobe create a single binary that does the same for RedHat 7.x-Fedora Core 2.x, SuSE 8.x-9.x, and Mandrake 9.x-10.x?
Can 3Com release a WinModem driver that will work for all Linux 2.x kernel versions?
These are conscious decisions by Open Source companies and programmers to make it difficult for Close Source software. Binary drivers are particularly sticky on this point, as Linus has stated that he wants to only promote drivers with the source available. But why fight? Linux is so close to being a pretty good OS, but it's so far because it won't make the leap to play nice. Either everyone plays by the OS rules, or the OS community will take its toys and go home.
It does cut both ways. And it's a sad state of affairs.:-(
Slashdot Mirror
Yes, they do. Why does everyone insist that they don't? I was reading about the execute vs. data segments back in the days of 386s!
Here's the manual if you don't believe me.
Offtopic, but how does your dtach program differ from the bash "disown" command? The only thing I see is that it looks like you can reattach to a detached program. I'm not sure if you can do that with the "disown" command. (I'm afraid I never had a reason to.)
On Intel chips there is no way to tell the difference between memory that can be read, and memory that can be executed.
Incorrect. Read Section 3.2.3 "Multi-segment Model" of the "IA-32 Intel® Architecture Software Developer's Manual: Volume 3: System Programming Guide". It lists both data and code segments quite clearly. This stuff has been around since the 386 days.
We seem to have split the thread.
That is NOT what my Pentium manual says. My pentium manual (Section 3.2.3: Multi-segment Model) shows that a segment type breaks up into four bits. The first bit is Data or Code. The second bit is expand-down or conforming. The third byte is read/write access. The fourth is some useless "Access" flag to tell you if it's been accessed since you cleared the flag.
The manual says that you can have an execute only segment as long as you load the executable constants into some other segment. (Or alternatively, map a data segment over the constant pool.) You can have it executable and read only, but you can't write to it without a data segment mapped over the same location.
Now what I'm saying, is that the OS should delete the data segment after it's done loading the code. That way, the only access will be via the IP register and (optionally) read only access of the constants pool.
No, wait, I've got it now. The return pointer. Ouch, how could I be so blind? I forgot the fact that an interrupt or subroutine stores the pointer on the stack. The only problem I see is, shouldn't his data still be in a data segment? And if the data segment is not executable, he shouldn't be able to return to it, should he?
As I said, the actual security benefit of moving the stack is very limited. The perf consequences, by contrast, are drastic.
Ding, ding, ding! I think I've got it. You're saying that the stack is set to point to a relative address so that the CPU doesn't have to use a long pointer to access the stack. Use of a shorter pointer improves performance drastically because the CPU is saving itself quite a few lookups and offset computations. Right?
Basically, yes. A GPF occurs when the data is written outside of the currently selected segment. A new segment selector is required to write to a different segment. An execute-only segment tells the processor that a request for a write segment is invalid.
Also, GPFs are "General" protection faults. The concept itself is not tremendously descriptive on what circumstances it's thrown. Under Unix, they referred to it as a "segment violation".
A thread's stack must be readable and writable by the process, right. Therefore, on an x86, was required to be executable. Oops -- that means you can jump to an address on the stack and run code there.
Parsing...
Ok, I think I've got you now. Correct me if I'm wrong though, but isn't that a problem with the way that Windows executable are directly mapped to memory? IIRC, the SP register takes a selector. That selector should be able to point to any location in the GDT or LDT table. Hmmm... let me check my notes...
The x86 only has two bits per page, Readable and Writable. You can't mark a page of memory as readable without marking it as being executable, they are the same thing on the x86.
:-/
But that's my point. You shouldn't *need* to make code memory readable. Mark it as 100b, and make sure that the only register that references it is the Instruction Pointer. The processor will still get its instructions, and the OS will be safe and happy.
Maybe I'm just being dense here.
But what if my buffer overflow overwrote code already marked as Execute?
Then it would pass from a data segment into a code segment. If it passes into a code segment, the processor will throw a general protection fault. The OS will catch the General Protection Fault and kill the program. Keep in mind that standard mallocs should only be returning data segments of memory. So (in theory) the overflow can't be writing to execute memory.
The problem is that there are historical and legitimate reasons for executing "data" and so OSs allow it.
Which is where the NX flag won't help. You *still* need a way of telling the OS whether a segment should be executable or not. What the OS should allow is to allocate a data segment for code loading. Once loaded, a special kernel method is called to switch it to execute-only. That would support JITs, and prevent buffer overflows.
Furthermore, I may be wrong here, but I don't think the original x86 had the capability of marking code as data or code, so if that was done it was all in the OS.
The 8086 instruction set had no concept of memory protection. However, the 80386 introduced a full protected memory mode that was later extended by the 80486. Of course, back then everyone put one entry into the GDT (0-end) and set it to executable and data. Worked great for video games. It wasn't until someone got the bright idea of creating a protected mode OS for the PC (WOW!) that segment protection was used. Of course, they STILL didn't segment data and code. *grumble*
I still don't get it. So you allocate a section for code. It's initially marked as a read/write data or code segment (001b or 101b). Once the code is loaded, you mark it 100b (execute only) and it's 100% protected until you reallocate it. What's the problem here?
My understanding is that the real issue has been supporting self-morphing code. Since a lot of legacy code liked this performance trick (especially games), all segments were marked as data and code.
It wouldn't be a problem if OSes properly separated code and data segments. By marking it properly in the GDT or LDT, even an exploit in the kernel shouldn't allow arbitrary code to run. And if the write does continue into a code segment, a general protection fault should occur.
:-/
This NX thing sounds like just a bunch of hoopla to make Microsoft start doing their job. Unfortunately, I don't think Linux is much better.
Ok, maybe I don't get this. How is this different from marking a block of memory as data, not code? The real problem is that certain OSes mark all memory as both code and data. Sure, it's easier on the bookkeeping, but it allows buffer overflows. If data was kept in data segments, and code was kept in code segment, the worst that would happen is a corrupted data segment, and/or a General Protection Fault.
If they call it the "NX-01", I'm gonna shoot somebody.
The soviet system lacked rewards based on performance and ways to control those who were in control.
Not quite true. If you got into the graces of the Communists, you could get a really nice high rise apartment and have lots of money. Otherwise you had to live dirt poor. So there was an incentive to do well.
One has to wonder why we call it Quantum Encryption when it really has nothing to do with Encryption. From the article:
The aim is to produce a communication system that cannot be intercepted by anyone
If I understand their intent, they plan to use concepts like Quantum Entanglement to ensure that communication is shared only between the entangled particles. This is a very different concept from using the properties of Quantum Mechanics to scramble information in a reversible manner or creating computers capable of super-fast calculations.
I believe the poster's point was that 3Com should be able to make a driver for each kernel version (or key stable ones). It need not necessarily be supported (i.e., continually "beta"), but, just having the driver would be a good thing; --even for 3Com.
1. Recompiling and QAing for each kernel version is money out of 3Com's pocket.
2. What happens if I build a custom kernel? (Hint: It breaks permanently.)
As for your statement about communism; yes, my idea shares certain characteristics with it. However, the main reasons that (so-called) communist systems have failed is that they tried to make things work without rewards, and power was concentrated in the hands of few, with no real checks in place.
Actually, I should use the more correct term. What you propose is "socialism". A dash here and there isn't necessarily a bad thing, but funds like Social Security show how badly a government manages these things. The real problem is that such concepts are very idealistic and tend to fail when hit with the real world. For example, your suggestion raises the following questions:
1. Who decides what projects get money?
2. How do they decide how much money to give?
3. How do they prevent pie in the sky projects (e.g. a "real soon now" warp drive) from eating up the budget, year after year?
In the market, you have to show a return on money spent, so the first is solved. The second is decided based on the return formula. The third is handled by cost overruns. If a project gets too expensive with little chance of success, it gets killed.
Academia handles pie in the sky stuff more elegantly because a project can continue as long as it can find relatively small amounts of funding. And if other scientists/engineers believe the project to be a dud, it will live and die by the team originally assigned to it. That's not to say that there isn't tons of wonderful stuff that "should have been" or "could be", but there are times when things should be left alone and revisited with more knowledge and experience.
P.S. Why is parent marked as Troll?
Can Realtek release an 8019 driver for WinXP? An AGP driver for Win95 (retail)?
You're confusing things. 3Com makes WinModems. Therefore they should be able to release a binary driver for Linux. But they can't. You know why? Because binary drivers are specific to the version of the kernel.
Your reasoning about 8019 and AGP are flawed, because they are core hardware components that are built according to open industry specifications. If RealTek released a motherboard with the new MXM feature, you're damn right I'd expect them to produce a driver.
OSS is also a chain reaction, which they fear. Imagine OSS does 99,5% of what a company needs to do. So they write the other 0,5% (as OSS, not their core business + they get community support and maintenance). Now suddnly some other company went from 99,2% -> 99,5%. So they write another 0,5%. Suddenly the snowball is rolling.
From a market perspective, this is a good thing. Lowering the barriers to entry forces companies to produce more innovation and cheaper products to keep ahead of the competition. Every company would *like* to have a monopoly, but monopolies inherently cause development to stagnate. Take phone companies as an example. AT&T had a monopoly and was able to keep rates the same with practically no need to technological development. The Feds suddenly allow competitors and Fibre Optics, DSL, lower prices, etc. develop.
YaST is a "glue" type of product that gave SuSE a competitive advantage. Releasing it as a commercial product would only have created one more difficult to install binary that would make your life easier if you could only get it installed.
The alternative would have been to be a technology licensor, and attempt to license it to RedHat and Mandrake. My guess, however, is that SuSE would have met with a "DIY mentality" and been shunned.
There's a flaw in your counter-argument. You're equating software development with patents. However, keeping source code closed does not in any way prevent a competitor from developing the same technology simultaneously. Thus the advantages of funding R&D while turning a profit still hold true.
What I envision is a system where funds are collected and pooled (like taxes), so that they can be spent on R&D and the like in a manner that all can benefit from. With the results available for everyone to use, there can be competition, and the market forces can work for us all instead of for the select few who control the patents etc.
This is communism, and it won't work without becoming just another pool of public money for governments to mismanage. The US has actually had a system in place to encourage funding for R&D. That system is academic research. Many companies pay a "smart guy" at a university to do a large amount of legwork for them at a low cost. This information is then usually available to the rest of the industry. Companies then turn around and complete the development of this research into a full product or technology.
Can Adobe create a single binary that installs, works, and takes advantage of Windows 9x, NT, 2000, and XP? Can Adobe create a single binary that does the same for RedHat 7.x-Fedora Core 2.x, SuSE 8.x-9.x, and Mandrake 9.x-10.x?
:-(
Can 3Com release a WinModem driver that will work for all Linux 2.x kernel versions?
These are conscious decisions by Open Source companies and programmers to make it difficult for Close Source software. Binary drivers are particularly sticky on this point, as Linus has stated that he wants to only promote drivers with the source available. But why fight? Linux is so close to being a pretty good OS, but it's so far because it won't make the leap to play nice. Either everyone plays by the OS rules, or the OS community will take its toys and go home.
It does cut both ways. And it's a sad state of affairs.