So while looking at the data collected, I had to wonder if some of the conclusions reached were not something of a matter of weighting - I saw some things pretty troubling about the WRK. Among the top of my list was a 99.8% global function count!!! I guess Microsoft uses a non-C linker-specific mechanism to isolate their functions, for instance by linking their code into modules. But yes, this is a troubling number.
This would explain some things like lower LOC count - after all, if you just have a bunch of global functions there's no need for a lot of API wrapping, you just call away. The lower LOC comes from the fact that WRK is s subset of Windows. It does not include device drivers, and the plug-and-play, power management, and virtual DOS subsystems.
Also, on a side note I would say another conclusion you could reach is that open source would tend to be more readable, with the WRK having a 33.30% adherence to code style and the others being 77-83%. That meshes with my experience working on corporate code, where over time coding styles change on more of a whim whereas in an open source project, it's more important to keep a common look to the code for maintainability. (That's important for corporate code too - it's just that there's usually no-one assigned to care about that). About 15 years ago I chanced upon code in a device driver that Microsoft distributed with something like a DDK that had comments written in Spanish. The situation in WRK is markedly better, but keep in mind that Microsoft distributes WRK for research and teaching.
I would wonder if you're just seeing a statistical wash-out. Are you dealing with data sets (tens of millions of lines and thousands of functions) that are so large, that patterns simply get washed out in the analysis? I don't think so. I can't now do an analysis for a counterexample, but I am pretty sure that if I run the same metrics on, say, the bottom 20% in terms of downloads, of Sourceforge projects I will very different results.
I didn't write the last part when I submitted the story, and, yes, the summary given here is comprehensible, because it appears out of context. What the sentence '..the structure and internal quality attributes of a working, non-trivial software artifact will represent first and foremost the engineering requirements of its construction, with the influence of process being marginal, if any.' means is that when you build something complex and demanding, say a dam or an operating system kernel, the end result will have a specific level of quality, no matter how you build it. For this reason the differences in the software built with a tightly-controlled proprietary software process and that built using an open-source process are not that big.
Re:Alternative solution for a trusted LAN
on
Multi-Threaded SSH/SCP
·
· Score: 2, Informative
Nc is useful, but it still involves the overhead of copying the data through it (once at the client and once at the server). Nowadays, in most settings this overhead can be ignored. But, given the fact that a well-behaved application will work with a socket exactly as well as with a pipe or a file descriptor, I thought it would be very elegant to be able to connect two instances of (say) tar through a socket. Hence the implementation of socketpipe. Socketpipe sets up the plumbing and then just waits for the programs to finish.
This is the setup using nc:
tar --pipe--> nc --socket--> nc --pipe--> tar
and this is the setup that socketpipe arranges:
tar --socket--> tar
Alternative solution for a trusted LAN
on
Multi-Threaded SSH/SCP
·
· Score: 5, Interesting
If you want to speed up transfers and you're working on a LAN you trust (i.e. you don't worry about the integrity and confidentiality of the data passing through it), you can dramatically increase throughput using socketpipe. Although the initial socketpipe communication setup is performed through client-server intermediaries such as ssh(1), the communication channel that socketpipe establishes is a direct socket connection between the local and the remote commands. This eliminates not only the encryption/description overhead, but also the copying between your processes and ssh or rsh.
main() { ptrace(PT_DENY_ATTACH, 0, 0, 0); sleep(10000); } ^D $ cc t.c $./a.out & [1] 411 $ gdb a.out -p $! GNU gdb 6.3.50-20050815 (Apple version gdb-563) (Wed Jul 19 05:17:43 GMT 2006) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "powerpc-apple-darwin"...Reading symbols for shared libraries.. done
/Users/dds/411: No such file or directory. Attaching to program: `/Users/dds/a.out', process 411. Segmentation fault
gdb exits with a segmentation fault, because the process has disallowed tracing. Adding your "l33t rootkit software" to the my example, is left as an exercise to the reader.
First, in most cases there's no need to spend time comprehending a large code base. You have to be selective, and the tools you'll use will depend on the problem you're facing.
If you're only trying to solve a specific bug, then, as other posters already commented, the debugger is your tool of choice.
If you must evaluate the code, then invest on metric collection tools, like CCCC.
If you want to reuse the code in another project, then you need to investigate packaging tools and techniques.
If the code's identifiers or its structure require refactoring, then try using a refactoring browser,
like CScout.
If the code's formatting and style brings you a headache, look at formatter,
like indent
Second, never underestimate the power of grep. Grep will often find things that other tools can't. It can search in documentation, comments, binary files, unparsable source code, and change logs. It can work on any language, and (with the help of find(1) and xargs(1) tools) traverse deep directory hierarchies. Unlike most other tools, grep doesn't need any setup, tuning, or configuration. Grep will often locate what you're looking for, in less time than what you need to download a more sophisticated tool.
That is the point. Maybe only a small percentage of those who use TrueCrypt use the nested encryption feature. So if you claim you're not using it, this is an entirely plausible statement.
If the passphrase is considered keys to a safe, and you are therefore likely to be forced to divulge it, then you can avoid trouble by using an encryption system, like TrueCrypt, that supports plausible deniability. Inside the encrypted volume, blank space is always filled with random data, which can also be another nested encrypted volume. Without the correct passphrase, nobody can prove that the random bits are anything more than random bits.
This used to happen more often than one would expect. In the 1980s I found portions of Ashton Tate's Framework II source code in "blank" sectors of floppy disks containing printer drivers. Those were the days where:
each application came with its own display and printer drivers,
people were using floppy disks to move around source code, and, worse,
other people had enough free time to trawl "blank" sectors for interesting tidbits.
While an open source Java port would be an interesting development, the real issue is whether Apple is interested to support Java on its platforms. A recent Javalobby article, titled So Long Apple. The Party's Over, gives several arguments supporting the position that Apple is doing a lot less than what it should in order to properly support Java. A high-quality implementation of the Java virtual machine needs all the help it can get from the underlying platform. For an illustration of this, see how slickly Java runs on Sun's Solaris. If Apple isn't interested to put its weight behind Java, it's unlikely that Sun will fill this role. Sun is putting a lot of effort to tune Java on the Windows platform; I doubt they have the resources and motivation to do the same with Apple's platform, due to the significantly lower market share of Mac OS X. So, while an open source port of Java is nice, full-hearted support from Apple would be a lot better.
Will my files survive 50 years of moving between storage media? Will I be able to view JPEG files in 50 years time? And, maybe even more critical: Will I be able to view/convert today's RAW files in 50 years time? No way, and I'd be willing to place a bet on this. The best you can do is to store them to a lossless format. I know, you will loose information through this process, but usefully processing RAW images requires much out-of-band bespoke knowledge that is unlikely to survive 50 years.
The article starts by presenting the preservation of photo negatives in a storehouse at 0 degrees Celsius and 25% RH, and then moves on discussing the problems of preserving inkjet photos. Photos printed on inkjets come from digital images. It is the bits of these images we want to preserve, not the printed photos. The nice thing with digital photos, is that if the printed photo fades, you can print it again. I was scanning some 20-year old negatives over the weekend, and I realized that they were irreparably scratched and darkened. (And don't get me started on the color distortions of printed 30-year old photos). With my digital photos I am reasonably sure that in 20 years I'll be able to print them in the same, or probably better quality.
The two real problems are:
Digital preservation. Will my files survive 50 years of moving between storage media? Will I be able to view JPEG files in 50 years time?
People who print their photos on inkjet printers and then delete (or loose) the digital version of the image. This is happening more often as digital cameras are increasingly bought by less IT-savvy people.
open source developers (and other freeware programmers in general) do what they do because they have nothing better to spend their time on.
This is by definition true for any activity we undertake. If there was something more profitable, enjoyable, pressing, useful to do, we (as rational thinkers) would be doing it.
Amazon has already deployed such a system under the name of Mechanical Turk. The idea is that humans assist computers, providing what is cutely named artificial artifical intelligence. You can read more about the concept in an article that ACM Queue run on May 2006.
-- Code Quality: The Open Source Perspective
If you plan to expose your application's GUI through a web browser, have a look at SWILL, the Simple Web Interface Link Library. With a couple of function calls you can add a web front-end to any C/C++ application. I've used it for adding a front end to the CScout source code analyzer and refactoring browser, and for implementing a wizard-like front-end for a stochastic production line optimization toolkit; I also supervised a student who worked on a SWILL-based gdb front end (unfortunatelly he didn't finish it).
SWILL is great for adding an interface to legacy code, because its impact on the application can be minimal. I wouldn't recommend its use if your GUI requirements are above what can be implemented in a dozen web pages.
I think you mean Google's AdSense technology, which is aimed at web publishers. The AdWords you mention are the ads that appear next to the Google search to search results. We should keep in mind that Google's AdSense lists are dynamically generated on the fly for each specific page request (see the source code in a page with AdSense. Google already knows (and stores) all your search queries. Do you really want it to also know all the pages you've been browsing in Wikipedia?
For me the three main ways I find information on the web are: Google, Wikipedia, and various digital libraries (in that order). Allowing Google to pry on the two first in concert would make the existing risks of Google regarding privacy considerably worse.
The site's design is a technical and usability disaster. It appears to be a mixture of JPEG page images grouped to look like a book, scanned documents in PDF format, huge PDF proofs complete with their crop marks, PDF files generated on the fly, and previously cached content. Opaque URLs, frames, gratuitous uses of Javascript, and botched internationalization complete the picture. A more simple design for the site would be a lot more usable and consume considerably less bandwidth and CPU power.
Unfortunately, too often non-technical managers get to make technical decisions and supervise web development. They invariably go for eye candy, ignoring usability and performance issues. Publishing legacy formats on the web is not easy, but the result really doesn't got to be this bad.
It's also safe to assume these are people with a lot of time on their hands.
It looks like the Washington Post's columnist thinks that having free time on your hands that you can dispense as you please, perhaps even helping volunteer efforts like Wikipedia, is the beginning of a slippery slope. I guess he's worrying will come next. Free thought?
This was the design of the TI-99/4A peripheral expansion box. You can see some photos here (look at the second row). The concept was neat in theory, but in practice it was a flop. The box would accept interchangable standard-sized expansion cards. Typical expansion cards were an RS-232 interface, 32k memory expansion, and a floppy disk controller. Each card was housed in its own metal casing, and all cards had the same form factor. However, the cards were bulkier and more expensive than anything comparable at that time. I think the cost of expanding the TI-99/4A was one of the reasons the computer failed in the market.
I just finished comparing the sizes of the DLLs the two programs use. The total size of the DLLs that the Internet Explorer loads is 44MB; the size of the DLLs for SeaMonkey is 11MB less: 33MB.
With shared libraries most package-level source code dependencies end-up as binary dependencies in the distribution, so the comparison is not as silly as you make it out to be. The idea of comparing Firefox on Windows with IE is neat; I'll run a test, after I press the submit button.
True, the DLLs in the first diagram are documented, but some in the last one aren't. Search for example for documentation on msoert2.dll or msasn1.dll on Microsoft's MSDN. The external DLLs and open source package dependencies do affect security, because an implementation error in them can create a security problem in the browser. You're right about not showing an SSL library for Mozilla, it wasn't listed as a dependency in its port, because it is part of the operating system base platform. A similar case could be made for some of the IE components.
A while ago I compared the number of dependencies to other components between Mozilla and the Internet Explorer. I thought that the free availability of many open source components would result in a much large number of dependencies (and therefore complexity) in Mozilla than in the IE. It turned out that the opposite was true. One explanation could be that, because Microsoft isn't obliged to publish the interfaces of internal Windows components and maintain backward compatibility, Microsoft developers have an easier time in creating internally reusable Windows components. Of course, in the long term, this strategy will backfire, as demonstrated by the travails of the Windows Vista release.
Slashdot Mirror
I didn't write the last part when I submitted the story, and, yes, the summary given here is comprehensible, because it appears out of context. What the sentence '..the structure and internal quality attributes of a working, non-trivial software artifact will represent first and foremost the engineering requirements of its construction, with the influence of process being marginal, if any.' means is that when you build something complex and demanding, say a dam or an operating system kernel, the end result will have a specific level of quality, no matter how you build it. For this reason the differences in the software built with a tightly-controlled proprietary software process and that built using an open-source process are not that big.
This is the setup using nc:
and this is the setup that socketpipe arranges:
If you want to speed up transfers and you're working on a LAN you trust (i.e. you don't worry about the integrity and confidentiality of the data passing through it), you can dramatically increase throughput using socketpipe. Although the initial socketpipe communication setup is performed through client-server intermediaries such as ssh(1), the communication channel that socketpipe establishes is a direct socket connection between the local and the remote commands. This eliminates not only the encryption/description overhead, but also the copying between your processes and ssh or rsh.
First, in most cases there's no need to spend time comprehending a large code base. You have to be selective, and the tools you'll use will depend on the problem you're facing.
Second, never underestimate the power of grep. Grep will often find things that other tools can't. It can search in documentation, comments, binary files, unparsable source code, and change logs. It can work on any language, and (with the help of find(1) and xargs(1) tools) traverse deep directory hierarchies. Unlike most other tools, grep doesn't need any setup, tuning, or configuration. Grep will often locate what you're looking for, in less time than what you need to download a more sophisticated tool.
That is the point. Maybe only a small percentage of those who use TrueCrypt use the nested encryption feature. So if you claim you're not using it, this is an entirely plausible statement.
This is a good point. An approach would be to do all the work on that volume under a virtual machine, which you then fill with zeroes.
If the passphrase is considered keys to a safe, and you are therefore likely to be forced to divulge it, then you can avoid trouble by using an encryption system, like TrueCrypt, that supports plausible deniability. Inside the encrypted volume, blank space is always filled with random data, which can also be another nested encrypted volume. Without the correct passphrase, nobody can prove that the random bits are anything more than random bits.
While an open source Java port would be an interesting development, the real issue is whether Apple is interested to support Java on its platforms. A recent Javalobby article, titled So Long Apple. The Party's Over, gives several arguments supporting the position that Apple is doing a lot less than what it should in order to properly support Java. A high-quality implementation of the Java virtual machine needs all the help it can get from the underlying platform. For an illustration of this, see how slickly Java runs on Sun's Solaris. If Apple isn't interested to put its weight behind Java, it's unlikely that Sun will fill this role. Sun is putting a lot of effort to tune Java on the Windows platform; I doubt they have the resources and motivation to do the same with Apple's platform, due to the significantly lower market share of Mac OS X. So, while an open source port of Java is nice, full-hearted support from Apple would be a lot better.
The two real problems are:
- Digital preservation. Will my files survive 50 years of moving between storage media? Will I be able to view JPEG files in 50 years time?
- People who print their photos on inkjet printers and then delete (or loose) the digital version of the image. This is happening more often as digital cameras are increasingly bought by less IT-savvy people.
These are important problems. However, on balance I think that the benefits of digital preservation are more than the risks.This is by definition true for any activity we undertake. If there was something more profitable, enjoyable, pressing, useful to do, we (as rational thinkers) would be doing it.
Amazon has already deployed such a system under the name of Mechanical Turk. The idea is that humans assist computers, providing what is cutely named artificial artifical intelligence. You can read more about the concept in an article that ACM Queue run on May 2006.
--
Code Quality: The Open Source Perspective
SWILL is great for adding an interface to legacy code, because its impact on the application can be minimal. I wouldn't recommend its use if your GUI requirements are above what can be implemented in a dozen web pages.
For me the three main ways I find information on the web are: Google, Wikipedia, and various digital libraries (in that order). Allowing Google to pry on the two first in concert would make the existing risks of Google regarding privacy considerably worse.
Unfortunately, too often non-technical managers get to make technical decisions and supervise web development. They invariably go for eye candy, ignoring usability and performance issues. Publishing legacy formats on the web is not easy, but the result really doesn't got to be this bad.
This was the design of the TI-99/4A peripheral expansion box. You can see some photos here (look at the second row). The concept was neat in theory, but in practice it was a flop. The box would accept interchangable standard-sized expansion cards. Typical expansion cards were an RS-232 interface, 32k memory expansion, and a floppy disk controller. Each card was housed in its own metal casing, and all cards had the same form factor. However, the cards were bulkier and more expensive than anything comparable at that time. I think the cost of expanding the TI-99/4A was one of the reasons the computer failed in the market.
I just finished comparing the sizes of the DLLs the two programs use. The total size of the DLLs that the Internet Explorer loads is 44MB; the size of the DLLs for SeaMonkey is 11MB less: 33MB.
With shared libraries most package-level source code dependencies end-up as binary dependencies in the distribution, so the comparison is not as silly as you make it out to be. The idea of comparing Firefox on Windows with IE is neat; I'll run a test, after I press the submit button.
True, the DLLs in the first diagram are documented, but some in the last one aren't. Search for example for documentation on msoert2.dll or msasn1.dll on Microsoft's MSDN. The external DLLs and open source package dependencies do affect security, because an implementation error in them can create a security problem in the browser. You're right about not showing an SSL library for Mozilla, it wasn't listed as a dependency in its port, because it is part of the operating system base platform. A similar case could be made for some of the IE components.
A while ago I compared the number of dependencies to other components between Mozilla and the Internet Explorer. I thought that the free availability of many open source components would result in a much large number of dependencies (and therefore complexity) in Mozilla than in the IE. It turned out that the opposite was true. One explanation could be that, because Microsoft isn't obliged to publish the interfaces of internal Windows components and maintain backward compatibility, Microsoft developers have an easier time in creating internally reusable Windows components. Of course, in the long term, this strategy will backfire, as demonstrated by the travails of the Windows Vista release.