What is interesting is that at least some of the
government labs, while not using clusters, are
using processor grids. As the parent post
notes, these have problems too.
I attended an unclassified talk on the
implementation of a neutron transport code at one
of the Labs. While some of the mathematics of
neutron transport was over my head, what impressed
me was the effort and creativity that went into
developing a massively scalable solution. The
person who developed this code came up with
a very elegant solution that tiles across
massive processor arrays.
As far as I can tell, the modus for high performance
computing seems to be to throw some very bright
hardworking people at the problem and hope that
they come up with a massively scalable
solution.
Another course would be to develop software,
languages and other tools to aid in parallel
programming. Beyond some libraries to support
parallel programming this kind of support does
not really exist. And since there is very little
market for massively parallel processors, there
does not seem to be much prospect for the
emergence of
this software.
There seems to be some historical
revisionism going on regarding the demise
of the "supercomputer industry". People are
coming out of the woodwork now saying that lack
of government support caused the great supercomputer
die off.
As Eugene Brooks predicted in his paper Attack
of the Killer Micros, the supercomputer
dieoff was caused by the increasing performance
of microprocessor based systems. Many of us now
own what used to be called supercomputers
(e.g., 3GHz Pentinum processors, capable of
hundreds of megaFLOPs).
The problem with supercomputers is that high
performance codes must be specially designed
for the supercomputer. This is very expensive.
As people were able to fill their needs with
high performance microprocessors they quit
buying supercomputers.
Many people who need supercomputer levels of
performance for specialized applications
(e.g., rendering Finding Nemo or The
Lord of the Rings) are able to use
walls of processors or clusters.
There are, of course, groups where putting
together off-the-shelf supercomputers will not
suffice. But these groups are few and far
between. As far as I can tell they consist of
the government and a few corporations doing
complex simulations. The problem is that this
is not much of a market. Even if the government
funds computer and interconnect architectural
research, there does not seem to be a market to
sustain the fruits of this research.
In the heyday of supercomputers there were those
who argued that when cheap supercomptuers were
available the market would develop. The problem
is, again, programming. High performance
supercomputer codes tend to be specialized for
the architecture. Also, no supercomputer
architecture is equally efficient for all
applications. It is difficult to build
a supercompter that is good at doing fluid
flow calculations for Boeing and VLSI netlist
simulation for Intel (the first applications
tends to be SIMD, the second, MIMD).
The end result of these problems
tends to suppress any emerging supercomptuer
market.
The reality right now seems to be that those
who are doing massive computation must build
specialized systems and throw a lot of talent
into developing specialized codes.
A number of postings have done an excellent job
of describing why open source is not a security
threat and, in fact, can be more secure because
the source code itself can be audited, signed,
etc...
This is a long discussion and I may have missed
it. But I have not seen
a mention of why Greenhills might be motivated
to make the claim that open source is a security
threat.
Greenhills sells proprietary compilers for
embedded systems. They also sell real time
operating system software. Their business
model is under threat by open source. This
is especially true in the area of real time
embedded operating systems. With their compilers
they can at least show that for a given embedded processor
their compiler produces better code (this is not
that hard to do against GNU for a number of
architectures). But with real time operating
systems (RTOS) they have less of an edge.
Linux is becoming more and more widely adopted.
Increasing numbers of people have experience
with Linux.
Greenhills, one can speculate,
fears a serious erosion of their RTOS business.
This business is probably bigger than their
compiler business (few people make much money
on compilers these days).
Taking a page from Microsoft and SCO they are
attacking Linux using FUD. If this also helps
their compiler business ("Who knows what trojans
open source compilers might generate?"), all the better.
Western Union money orders collected in Nigeria
are pretty safe for the scammers. The Nigerian
police are paid off, so it can be collected safely.
Re:OSS can't be everything...
on
Data Mining Goes 3D
·
· Score: 2, Informative
Wouldn't the work of a government-funded national lab be public domain if it ever were to be released?
As far as I know the Department of Energy labs,
which include the Sandia labs, Lawrence Livermore,
Los Alamos, are all managed by contractors.
The contractor does work for the government, but
frequently maintains co-ownership with
the government for the work performed.
I have worked with commercial contractors that
worked under similar arragements. The customer
paid the contractor for software development work,
but the contractor also owned a copy, which
tbey could sell to others. Only work that was
explicitly identified as proprietary was
exempted from this. Some consulting companies,
like Wind River, in its early days, have built
a significant amount of intellectual property
following this model. Once they build up a
software base they have a competitive advantage
in licensing it for new applications. The fact
that some software can be provided "off the shelf"
rather than developed provides an incentive for
the prospective customer to agree to co-ownership.
The organizations that manage the national labs
seem to take a similar approach. They also
own much of the intellectual information they
develop. Release of software into the public
domain at the University of California managed
labs requires a review by a UC office that is in
charge of licensing.
No surprise that Lockheed uses Sandia work
on
Data Mining Goes 3D
·
· Score: 2, Informative
Other firms, such as Lockheed Martin, also are starting to use the lab.
I don't find it surprising that Lockheed Martin
is one of the firms "starting to use the lab".
Lockheed Martin runs Sandia as a contractor for
the Department of Energy. Lockheed has a builtin
bias to show how applicable the work at Sandia
is.
In the case of run once applications that are CPU intensive,
like compilers, there is no question that Java
can be slow (see my web page on why it is reasonable
to consider compiling Java into native code)
I've been working on the design of a trade engine,
which can support multiple trading applications.
I'm an experienced C++ developer, but for the
trade engine I'm planning on using Java (running
on Linux and storing transactions in PostgreSQL).
One of the core issues in many financial
applications, like a trade engine, is that they
must record transactions in a database. Once
you start inserting into a database it is likely
that the database will become the bottleneck
(unless you are using something like an in
memory database (e.g., like the one from
TimesTen). So the performance advantage
that C++ can deliver over Java does not
seem to apply for database applications.
Sometimes database operations seem so slow that
I fell like programming in smoke signals would
make no difference.
The class libraries available for Java are more extensive
than any environment I've worked in. In
C++ you could only match Java's library by
purchasing all of Rogue Wave's libraries. And
even then, Rogue Wave can be rather obscure.
In addition to the huge class library, Java gives you garbage collection, which
makes development faster.
Java also provides many advantages to enterprise
applications. For example, you can use an
application server like Resin to start up your
services and to support remote calls (via
SOAP). You can also have dynamically generated
web pages that display portfolio postions and
other changing data.
This sure does not look like a flame bait post
to me. I'd say, rather, that its informative.
The poster is noting that Java, currently
on Solaris is popular for enterprise applications.
I'd say that's reasonable. Then they note that some
users are going to.NET. The fact that the poster
believes that the corporate customers are not
going to touch Mono in the near future is an
opinion, and perhaps one that corresponds to
reality.
I've been programming heavily in C++ for many years.
While I have a love hate relationship with C++'s
complexity, I never thought that I would use
Java heavily. But I've been working on more
web services applications that access a database.
The huge class library that is available with
Java is a great advantage when it comes to
developing these applications. Sure I can pay
thousands of dollars to Rogue Wave and get some
of the same features in C++ that I get for free using
the Sun and Apache Java libraries. But why? The
higher performance of C++ is of little use to
most
applications that references a database, since
these applications are usually bottlenecked in the
database. And when it comes to web services
(e.g., XML processing, servlets, dynamically
generated web pages), C++ cannot compare to
Java. And then there is garbage collection,
which makes develoopment faster.
There are still applications that I would not
write in Java. For example, compilers or other
algorithmically intensive applications that
are CPU bound. There are also times when I
can simplify my code by using C++ features like
operator overloading (see my wavelet
packet transform algorithm for example). But
these applications are now in such a minority
in the work I'm doing that I worry that my C++ "chops" will get
rusty.
I went to a talk by Stroustrup where he discussed
C++ (some cool algorithms to support
linear algebra computation), future developments and C+++ vs.
Java. He promotes C++ as a systems programming
language. For things like operating systems,
virtual machines, hardware drivers and compilers.
He trumpted C++ as the most widely used
programming language.
What does not seem to have occured to
Stroustrup is that systems level applications, where
C++ shines, are a small minority of the code
programmers write. My view is that C++'s
star is fading.
So yeah, I'd say Java is heavily used on Linux.
At work I'm part of a group developing a
distributed database application in Java
(this runs on top of a relational database,
so Java's performance is not an issue), hosted
on Linux. I'm
in the process of setting up a Linux/PostgreSQL
system on which to develop a financial trading
application, again in Java, using XML and
web services.
Liberal compared to whom?
Tom DeLay? The late Francisco Franco?
Hitchens likes to think of himself as a modern
George Orwell. A man who has turned away from
misguided leftist views toward more informed
right wing views.
After seeing communism and
various other forms of leftism during the civil
war in Spain (where Orwell was almost killed by
Soviet NKVD assassins), Orwell turned against
some of his leftist views. Out of this came
is masterpiece 1984, which was
inspired by Stalin's Soviet Union.
But there are a few things that "Hitch" misses
in his alcoholic fog as he compares himself to
Orwell:
Orwell was one of the greatest english stylists
of the twentieth century. Few people would make
a similar claim for "Hitch".
Orwell's turning away from the left did not
involve embracing people from the ultra-right
wing like Ken Starr and John Ashcroft.
Orwell did not move rightward for personal
financial gain. I would guess that "Hitch"
has found it more profitable to be a right
wing pundit than a poorly paid writer for
The Nation.
Rather than being a modern Orwell, as Hitchens
would like to believe himself to be, he is more like
David Horowitz, someone who went from shallow
left wing views to equally shallow right wing
views. Hitchens is a second rate writer seeking
a profitable gig.
I work at the Lawrence Livermore National Labs
in Livermore, California. At least at
Lawrence Livermore (LLNL) there are excellent
summer internship programs. Each "Directorate"
has a summer internship program. The directorates
include "Computation" (the directorate I work in),
materials science, engineering, physics, chemistry and molecular biology. Internships
are available for both undergrads and graduate
students. At the graduate student level there
are divisions that also hire people outside
of the sciences (like East Asia experts).
You need to
apply early, usually the year before (by December
31 in some cases) or early in the year (by
January or so). Some programs require letters
from your professors. So start early. At least at LLNL you need
to be a US citizen. There is a basic background
check for summer jobs.
Hiring at "The Labs" ebbs and flows, depending
on funding. If you take part in a summer
program and you impress people during your internship, there
is a better chance that you will be offered a
job on graduation.
The projects are interesting
and it is a good chance to get experience.
A friend of mine's daughter is working in an
internship in materials science. They are giving
her access to the MEMS silicon fabrication
facility (which, she tells me, costs more than
they are paying her for the summer). Summer
projects in computation in the past have
involved networking and cryptography.
The summer programs are closed for this year.
You can find them described on the LLNL web site
(llnl.gov). The directorates do the hiring, so
please don't send me your resume. I can't help in
this area.
There are a number of other national labs with
summer programs. Labs that come to mind are
Los Alamos (LLNL's "sister lab"), Oak Ridge,
Argonne, the Pacific Northwest National Lab.
There is also an Dept. of Energy engineering
lab in Idaho, but I don't know if they hire
summer students.
Finally, for what it's worth my heart goes out
to anyone who is graduating in CS or
engineering these days. In my twenty four
years working in this industry I have never seen
times like these (and that includes the 1992
down turn). As others have noted, the problem
is that there is unemployment among experienced
engineers, so this makes it doubly difficult
for new grads.
The irony is that from what I've read, hiring is booming
for engineers in India (see my essay
An Economics Question). This is one reason
I'm grateful to have a job at a National Lab.
My work requires a security clearance and it will
never be moved overseas.
"world's oldest sea water -- a vast, still pool unstirred for millennia, walled by steep ridges and lidded with ice." Bring on the "Jurassic Park" references."
For those of us who have studied that dread work,
the Necronomicon,
the truth is not Jurassic Part but the
the Elder Gods. Yes, my slashdot fellow readers,
what will be found are those who were here before
us. Trapped for millions of years behind the
walls of ice will be found those who came from
beyond. Behind the icy barriers they have
waited, only now to be awakened. We can at
least take heart in the fact that this is the
northern polar climbs. If it were the cold
icy regions of the south pole (where the
Mountains of Maddness lie), those released elder
Gods would come forth to gorge on penguins.
The horror! The rejoicing in Redmond! None
of us can question which operating system
Cthulhu would use! At least
we are spared this fate.
Ahh, now I understand: it's all innovating for
the customer. Once you find out what the customer
wants, then the innovation can begin. How silly
of me to have missed this.
The Microsoft interview style is to ask the
interviewee a constant stream of white board
programming problems and throught puzzles. While
this selects people with a certain level of
intelligence, it also selects people who can
think rapidly "on their feet".
Perhaps the end result is to select a
homogenious population of "Softies"
think fast, settle on an approach and then
hack it into code. Where a better approach
to product development
might be to think about the design, think about
some alternatives, discuss the design and then
implement it.
Many people agree that Microsoft software
evolves once it has been released. The
common example being a first product that is
inadequate, buggy and slow, eventually evolving
into something that becomes popular. Perhaps
this is a result of a culture of programmers
who believe they are very smart (after all, they
survived the Microsoft interview), think fast
and then entomb their initial half-baked
design in software.
I can see your point about privacy. It is true
that once you allow something to read email it
could be abused. But to balance this is the
fact that, at least for me, email would be useless
without a spam filter.
Privacy is not an issue in my case.
I use text only email on Linux (email never
touches my Windoz system for security reasons).
I run a
spam filter for my own email account, so it is
my program that reads my email, not someone elses.
I read my email on a shared Linux system run by the
ISP that hosts my domain (my ISP is webquarry.com).
As far as I know, the RBL approach would not work
in my case. I do discard some email one the basis of the domain name, which is far
less efficient than the RBL. My spam filter
keeps a log of some of the header information
from the email it discards. A fair amount of
spam is going through fixed domain names these
days (e.g., like the infamous tekmailer).
One of the problems I had with the commonly used
spam filters was that it was unclear to me how
to install them in the case where I am simply
piping my email to them. I was also concerned
about resource usage, since I am using a shared
system. So like a typical programmer I wrote
my own spam filter in C++. It is probably 80 to
90 percent efficient. Enough spam still gets
through that I'm going to take
another look at SpamAssassin and see if I can
get it to run with a "procmail" forward. It is
just too time consuming to constantly hack the
spam filter for the latest evil spammer trick
(recently they have been sending spam to my
email address from the other valid user on
my domain, where I don't check content).
I have noticed that black lists are indeed
effective. Many spammers now use "bullet proof"
spam hosts, so they use static domain names.
However, there has been an marked rise in zombie
systems sending spams. These are systems
that are infected by viruses and then used as
spam hosts. Since these systems come on line
rapidly (when they are infected) and then drop
out (when they are cleared of the virus or
booted off their ISP) it seems
unlikely that black lists will help.
At least in the spam stream I see, there is
more than 1-2 percent of the spam flow from
zombies. The best technique seems to be to
use a black list first and then content filter.
An a related topic in the parent post:
In a previous post, in another discussion, I
also suggested that the sophistication of
spam filters like SpamAssassin, which use
several algorithms to filter spam, would consume
lots of system resources. Another poster wrote
that these tools do not consume much in the way
of processor and memory resources. This seems
counter intuitive, but I don't have any contrary
evidence.
One of the most common measures that we see
quoted in statistics is the mean, or the average.
When there is a "bimodal" distribution, that is,
clumping at the high end and clumping at the low
end, the average is misleading. An example of
this can be found in the claims for the benifits of the
Bushies tax cuts. Bushie quoted the average
benifit. That factored in all of the rich people
who benified a lot. This was used to suggest
that those "average families" that Bushie appeared
with in photo-ops would benifit as well.
When there is a bimodal distribution a more
representative statistic is the median. If
you take a set of numbers, say the benifit
from the Bushie tax cut, and sort them, the median
is the number that is in the middle of this
set (e.g., if there are 101 numbers, the 50th number is the median). Here you don't get a bias from the high
end. If Bushie had quoted the median, instead
of the average, people would have immediately
realized that they were not getting much and
that wealthy people were getting a lot.
Which brings us to income statistics and the
point raised in the parent post. If the
median salary was quoted it seems likely that
computer industry salaries would have remained constant or
declined.
We are definitely seeing bleaker times in the
computer industry than I have seen in my
twenty plus year career. And I include the
1992 downturn. However I think that part of
what is happening is that the factors that
have effected other job catagories are now
effecting "knowledge worker" jobs.
My guess is that if you were to look at the
median, not the average, income of individual
workers, you would see a decline over the last
twenty years.
The government frequently quotes household or
family income, and again, they quote the average.
During the last twenty years some classes of
workers have done much better than others. In
particular, until 2000 those
of us in the computer industry were on the winning
side. Many of us saw our salaries increase
dramatically. In the 1990s few would have
believed that unemployment in the computer
industry would be higher than overall
unemployment.
For those outside of the "knowledge worker"
catagory household income has increased
or remained steady, I theorize, because increasing
numbers of women entered the workforce. Espeically
in urban areas on the coasts where two incomes
was necessary to survive as a middle class
family.
I find it interesting that few people ever
talk about the individual worker median income,
adjusted for inflation (the common
statistic quoted is household income). Quoting
adjusted individual median income might
throw into question the fantasy in the US that
we are such a wealthy country. What people would
find is that while we are, indeed, a wealthy country, over the last twenty years the wealth has
been increasingly concentrated in the hands of
a smaller and smaller faction of the population.
As the economist Paul Krugman wrote, this is
"class warfare". But it warfare practiced by
the rich and the powerful on everyone else.
Increasingly we are seeing the executive branch
(e.g., the departments that report to the
President) either not publish statistics or
publish misleading or partial statistics.
This is true for many departments that previously
prided themselves on non-partisanship.
The job forecasts and market outlook for programmers
and software engineers did not mention anything
about outsourcing. Could this be because outsourcing
is a senstive political topic that the current
administration is vulnerable on? I found it odd
reading that job growth for programmers would
be about the same as job growth overall, without
any mention of why such tepid job prospects
were being forecast. In fact, I found nothing
about low wage competition for "knowledge
worker" jobs.
Then there is the issue of job catagories.
Apparently the job prospects for "software
engineers" were bright, while those for programmers
were mediocre.
I have never worked in an environment where
someone did design and someone else implemented
this design in software. Yes I've had customers
provide a broad outline of what they wanted,
sometimes in terms of system components, but
the engineering of large software systems
is closely tied to their implementation. So
as far as I'm concerned the division between
"programmer" and "software engineer" does not
exist. In fact some of the problems
encountered in offshore outsourcing involve
the attempt to separate software engineering
from programming. Those contracting for low wage programming must
provide detailed documentation
that describes exactly what they want and how
they want it done. Even then sometimes the software that is
delivered is not adequate.
I now see that the comments on operating
system authorship vs. kernel authorship
are on target. In responding to my email
Ken Brown seems to admit that Linus wrote
the Linux kernel.
Perhaps I need to go back an review my books
on operating systems. Perhaps I am the one
in error here. But to me an operating system
is as I have described it in my reply to
Ken Browns reply to my initial email. This
is included below.
Again, all this may simply be a waste of time.
Ken Brown seems to have latched on to a position
that most in the computer science and engineering
community feel is silly. He seems unlikely to
admit he is wrong. My comparision of Ken Brown
to Peter Duesberg is probably appropriate.
Date: Sat, 5 Jun 2004 14:19:45 -0700 To: kenbrown@adti.net Subject: Re: Linux, Linus and so on...
Dear Ken:
Perhaps the problem is that, my view, Microsoft has perverted the
meaning of operating system. What Microsoft calls an "operating
system" comes with a vast array of applications. In my view the
operating system called Windows NT is the software system described
in "Inside Windows NT" by Helen Custer and David N. Cutler, 1992.
As far as I am concerned, an operating system is software that:
Manages access to devices like disk, graphic displays, the
keyboard and, in modern systems, the network interface, sound and
high performance graphic devices.
Manages memory. Modern operating systems provide virtual memory,
that gives each application the illusion that it is running in
its own address space.
Manages processes (applications). An operating system provides
support for switching process execution when a process accesses a
physical device and when its allocated time slice expires.
File system. The file system exists above the device management
software.
Network protocol stack. The network protocol stack, like the
file system, exists above the software that provides abstract
access to the network interface hardware.
In theory some of these components, like the protocol stack and the
file system, could be application level, not operating system
level. However, in practice they are built into the operating
system for performance reasons. And in the case of the file system,
there is usually integration with the virtual memory subsystem.
If you open a text book on operating systems, an "operating system"
is what I have described above. The Minix book is one example. On
Slashdot people have cautioned me to refer to this as a "kernel". I
guess that I'm from "the old school". To me what I have listed
above is an operating system. "Kernel" does not have as exact a
definition in my understanding. For example, one view of an
operating system kernel might be that it does not include the file
system and the network stack.
The window system, user shells, file system utilities, compilers,
text or graphic editors are not part of the operating system.
To return to the original point: whether you want to call it an
operating system or a "kernel" there is no question in my mind that
Linus Torvalds wrote Linux. As I noted, this operating system has
evolved. Compared to the current operating system, the initial
operating system was relatively simple. It was well within what a
dedicated, talented software engineer could implement. As I've
noted, most UNIX software engineers will agree with my view.
As Linus Torvalds would be the first to acknowledge, he is a small
part of the Linux community. He has managed the integration and
release of the components that I have listed above as "operating
system" (or, if you must, "kernel"). The fact that he has been
willing to be inclusive in this process has sometimes been listed as
the re
OK, this was probably a waste of time. Ken
Brown is so over the top that he has to have
an agenda. But for what it is worth, here is
a copy of an email I sent him (with minor
changes)
Date: Fri, 4 Jun 2004 17:54:31 -0700 To: kenbrown@adti.net Subject: Linux, Linus and so on...
Dear Mr. Brown:
I am a software engineer with over twenty three years of
experience. Much of my background is in the design and
implementation of system software, including compilers and runtime
support. If you are interested you can find my resume at my domain.
I am not in the Linux fanatic camp. I use Linux, but in many ways I
am disappointed with its popularity. I would much rather that the
freeBSD operating system, which I regard as superior, had Linux's
popularity. But popularity is the result of many factors, some of
which have nothing to do with technology.
As a highly experienced software engineer and someone who is not a
Linux fanatic, let me state that I absolutely believe that Linus
Torvalds wrote Linux. And I also fully believe Prof. Tanenbaum when
he states that Linux was not "stolen" from Minux.
Linux has been many years in development. The initial operating
system was not the operating system that exists today. Linux has
evolved over the years and many people have contributed to this
evolution. As you note, Linus was young when he wrote Linux and I
have no doubt that he too has evolved into a skilled and expert
operating system designer and implementer. Linux is now far better
for this experience and the work of its many contributors.
I find it rather odd that you, who are not an expert in software,
are arguing against a large community of people who are experts, that
Linus did not write Linux. I would hazard a guess that most UNIX
systems programmers find your arguments silly. I also find it
interesting that many of your arguments mirror those that have been
put forward by Microsoft, a company that clearly finds Linux a
threat.
There is nothing wrong with charging for
access to a spam black list. You could even
charge on the basis of the bandwidth that the
customer used. So big customers that hit the
database a lot pay, people who it it only a few
hundred times a day pay little or nothing.
My impression however is that Spamhaus is
compromised by their association with IronPort,
which provides a system for high speed mass
mailing. Yeah I know that these are only
supposed to be used by "the good guys". But
that assumes that we agree on what a "good guy"
is.
Perhaps I'm being too hard on Spamhaus and
IronPort. But like many you, I get hundreds
of spams a day. It has gotten to the point
where having domain based email that comes
to my Linux shell exacts a cost, even with
my spam filter in place. So I don't look
kindly on anyone who supports mass e-mail,
whether they are IronPort or some spammer in the
Bahamas. Given this view, I don't look kindly
on those who associate with spammers (ah mass
emailers), and that
includes Spamhaus.
At the very minimum Spamhaus fails the
"Caeser's wife" test. They support a spam
blacklist database and they are owned by a
company that sells mass mailing services.
Seems like a built in conflict of interest.
I would never advocate violence and torture,
but the first thing that occured to me when
I read this is
brand Darl!. Perhaps the Microsoft logo
tatooed onto one of his ass cheeks. Then
he can remember who 0wns his ass.
And Darl (and bodyguards), this is satire, not a suggestion that anyone violate your ass.
Crude, I know. Sorry.
Back to more mature discussions...
As time goes on DSPAM (and SpamAssassin for that
matter) become more and more sophisticated,
incorporating more complex algorithms. What I also
find striking is that many of these algorithms
appear to be compute intensive. These spam filters
seem to be designed for server side ISP level
email filtering. I would expect that a computer
would have to be dedicated to running
this anti-spam software.
Also, as a number of posters have noted,
configuring these spam filters takes some
effort and education on the part of the
user.
This level of resource comsumption is fine
for an ISP, but it seems problematic for
a single user or someone using a shared system.
I only use Linux shell based email to avoid
worms and viruses. I've had my domain since
1995, so I got a lot of spam. As the tide
of SPAM increased, overcoming the primative
SPAM filter I was using I looked at both
SpamAssassin and DSPAM. But these tools do not
appear appropriate for a shared Linix system
like the one that hosts my domain.
So, in classic "reinvent the wheel fashion",
I wrote my own SPAM filter. It is just a
simple (compared to these tools) rule based
filter. It filters out enough SPAM that email
it not totally useless. This email filter
can be found here.
The email filter is written in C++ in an attempt
to minimize resource usage. It is published
as open source.
I too would think that a portfolio of past work
would demonstrate to a prospective employer that
I can engineer software. I have published
thousands of lines of Java and C++ code on
my web site
www.bearcave.com.
What constantly strikes me as odd is that people
I've interviewed with are more or less
unwilling to look at my work (note that this
work belongs to me and has been done
on my own time). And those who
do
interviews that consists largely of
programming problems still insist in
having me code up their little algorithm on
the white board.
As I note on the web page linked to above,
I think that in part this is because the
interviewer does not know what else to ask.
When I have been a hiring manager I've looked for
people who really loved software and had
engineered large systems. Given this they can
learn what ever else I need them to know.
Slashdot Mirror
What is interesting is that at least some of the government labs, while not using clusters, are using processor grids. As the parent post notes, these have problems too.
I attended an unclassified talk on the implementation of a neutron transport code at one of the Labs. While some of the mathematics of neutron transport was over my head, what impressed me was the effort and creativity that went into developing a massively scalable solution. The person who developed this code came up with a very elegant solution that tiles across massive processor arrays.
As far as I can tell, the modus for high performance computing seems to be to throw some very bright hardworking people at the problem and hope that they come up with a massively scalable solution.
Another course would be to develop software, languages and other tools to aid in parallel programming. Beyond some libraries to support parallel programming this kind of support does not really exist. And since there is very little market for massively parallel processors, there does not seem to be much prospect for the emergence of this software.
There seems to be some historical revisionism going on regarding the demise of the "supercomputer industry". People are coming out of the woodwork now saying that lack of government support caused the great supercomputer die off.
As Eugene Brooks predicted in his paper Attack of the Killer Micros, the supercomputer dieoff was caused by the increasing performance of microprocessor based systems. Many of us now own what used to be called supercomputers (e.g., 3GHz Pentinum processors, capable of hundreds of megaFLOPs).
The problem with supercomputers is that high performance codes must be specially designed for the supercomputer. This is very expensive. As people were able to fill their needs with high performance microprocessors they quit buying supercomputers.
Many people who need supercomputer levels of performance for specialized applications (e.g., rendering Finding Nemo or The Lord of the Rings) are able to use walls of processors or clusters.
There are, of course, groups where putting together off-the-shelf supercomputers will not suffice. But these groups are few and far between. As far as I can tell they consist of the government and a few corporations doing complex simulations. The problem is that this is not much of a market. Even if the government funds computer and interconnect architectural research, there does not seem to be a market to sustain the fruits of this research.
In the heyday of supercomputers there were those who argued that when cheap supercomptuers were available the market would develop. The problem is, again, programming. High performance supercomputer codes tend to be specialized for the architecture. Also, no supercomputer architecture is equally efficient for all applications. It is difficult to build a supercompter that is good at doing fluid flow calculations for Boeing and VLSI netlist simulation for Intel (the first applications tends to be SIMD, the second, MIMD). The end result of these problems tends to suppress any emerging supercomptuer market.
The reality right now seems to be that those who are doing massive computation must build specialized systems and throw a lot of talent into developing specialized codes.
A number of postings have done an excellent job of describing why open source is not a security threat and, in fact, can be more secure because the source code itself can be audited, signed, etc...
This is a long discussion and I may have missed it. But I have not seen a mention of why Greenhills might be motivated to make the claim that open source is a security threat.
Greenhills sells proprietary compilers for embedded systems. They also sell real time operating system software. Their business model is under threat by open source. This is especially true in the area of real time embedded operating systems. With their compilers they can at least show that for a given embedded processor their compiler produces better code (this is not that hard to do against GNU for a number of architectures). But with real time operating systems (RTOS) they have less of an edge. Linux is becoming more and more widely adopted. Increasing numbers of people have experience with Linux.
Greenhills, one can speculate, fears a serious erosion of their RTOS business. This business is probably bigger than their compiler business (few people make much money on compilers these days). Taking a page from Microsoft and SCO they are attacking Linux using FUD. If this also helps their compiler business ("Who knows what trojans open source compilers might generate?"), all the better.
Western Union money orders collected in Nigeria are pretty safe for the scammers. The Nigerian police are paid off, so it can be collected safely.
Wouldn't the work of a government-funded national lab be public domain if it ever were to be released?
As far as I know the Department of Energy labs, which include the Sandia labs, Lawrence Livermore, Los Alamos, are all managed by contractors. The contractor does work for the government, but frequently maintains co-ownership with the government for the work performed.
I have worked with commercial contractors that worked under similar arragements. The customer paid the contractor for software development work, but the contractor also owned a copy, which tbey could sell to others. Only work that was explicitly identified as proprietary was exempted from this. Some consulting companies, like Wind River, in its early days, have built a significant amount of intellectual property following this model. Once they build up a software base they have a competitive advantage in licensing it for new applications. The fact that some software can be provided "off the shelf" rather than developed provides an incentive for the prospective customer to agree to co-ownership.
The organizations that manage the national labs seem to take a similar approach. They also own much of the intellectual information they develop. Release of software into the public domain at the University of California managed labs requires a review by a UC office that is in charge of licensing.
Other firms, such as Lockheed Martin, also are starting to use the lab.
I don't find it surprising that Lockheed Martin is one of the firms "starting to use the lab". Lockheed Martin runs Sandia as a contractor for the Department of Energy. Lockheed has a builtin bias to show how applicable the work at Sandia is.
In the case of run once applications that are CPU intensive, like compilers, there is no question that Java can be slow (see my web page on why it is reasonable to consider compiling Java into native code)
I've been working on the design of a trade engine, which can support multiple trading applications. I'm an experienced C++ developer, but for the trade engine I'm planning on using Java (running on Linux and storing transactions in PostgreSQL).
One of the core issues in many financial applications, like a trade engine, is that they must record transactions in a database. Once you start inserting into a database it is likely that the database will become the bottleneck (unless you are using something like an in memory database (e.g., like the one from TimesTen). So the performance advantage that C++ can deliver over Java does not seem to apply for database applications. Sometimes database operations seem so slow that I fell like programming in smoke signals would make no difference.
The class libraries available for Java are more extensive than any environment I've worked in. In C++ you could only match Java's library by purchasing all of Rogue Wave's libraries. And even then, Rogue Wave can be rather obscure. In addition to the huge class library, Java gives you garbage collection, which makes development faster.
Java also provides many advantages to enterprise applications. For example, you can use an application server like Resin to start up your services and to support remote calls (via SOAP). You can also have dynamically generated web pages that display portfolio postions and other changing data.
This sure does not look like a flame bait post to me. I'd say, rather, that its informative. The poster is noting that Java, currently on Solaris is popular for enterprise applications. I'd say that's reasonable. Then they note that some users are going to .NET. The fact that the poster
believes that the corporate customers are not
going to touch Mono in the near future is an
opinion, and perhaps one that corresponds to
reality.
I've been programming heavily in C++ for many years. While I have a love hate relationship with C++'s complexity, I never thought that I would use Java heavily. But I've been working on more web services applications that access a database. The huge class library that is available with Java is a great advantage when it comes to developing these applications. Sure I can pay thousands of dollars to Rogue Wave and get some of the same features in C++ that I get for free using the Sun and Apache Java libraries. But why? The higher performance of C++ is of little use to most applications that references a database, since these applications are usually bottlenecked in the database. And when it comes to web services (e.g., XML processing, servlets, dynamically generated web pages), C++ cannot compare to Java. And then there is garbage collection, which makes develoopment faster.
There are still applications that I would not write in Java. For example, compilers or other algorithmically intensive applications that are CPU bound. There are also times when I can simplify my code by using C++ features like operator overloading (see my wavelet packet transform algorithm for example). But these applications are now in such a minority in the work I'm doing that I worry that my C++ "chops" will get rusty.
I went to a talk by Stroustrup where he discussed C++ (some cool algorithms to support linear algebra computation), future developments and C+++ vs. Java. He promotes C++ as a systems programming language. For things like operating systems, virtual machines, hardware drivers and compilers. He trumpted C++ as the most widely used programming language. What does not seem to have occured to Stroustrup is that systems level applications, where C++ shines, are a small minority of the code programmers write. My view is that C++'s star is fading.
So yeah, I'd say Java is heavily used on Linux. At work I'm part of a group developing a distributed database application in Java (this runs on top of a relational database, so Java's performance is not an issue), hosted on Linux. I'm in the process of setting up a Linux/PostgreSQL system on which to develop a financial trading application, again in Java, using XML and web services.
Hitchens is a liberal claims the parent post
Liberal compared to whom? Tom DeLay? The late Francisco Franco?
Hitchens likes to think of himself as a modern George Orwell. A man who has turned away from misguided leftist views toward more informed right wing views.
After seeing communism and various other forms of leftism during the civil war in Spain (where Orwell was almost killed by Soviet NKVD assassins), Orwell turned against some of his leftist views. Out of this came is masterpiece 1984, which was inspired by Stalin's Soviet Union.
But there are a few things that "Hitch" misses in his alcoholic fog as he compares himself to Orwell:
Rather than being a modern Orwell, as Hitchens would like to believe himself to be, he is more like David Horowitz, someone who went from shallow left wing views to equally shallow right wing views. Hitchens is a second rate writer seeking a profitable gig.
I work at the Lawrence Livermore National Labs in Livermore, California. At least at Lawrence Livermore (LLNL) there are excellent summer internship programs. Each "Directorate" has a summer internship program. The directorates include "Computation" (the directorate I work in), materials science, engineering, physics, chemistry and molecular biology. Internships are available for both undergrads and graduate students. At the graduate student level there are divisions that also hire people outside of the sciences (like East Asia experts).
You need to apply early, usually the year before (by December 31 in some cases) or early in the year (by January or so). Some programs require letters from your professors. So start early. At least at LLNL you need to be a US citizen. There is a basic background check for summer jobs.
Hiring at "The Labs" ebbs and flows, depending on funding. If you take part in a summer program and you impress people during your internship, there is a better chance that you will be offered a job on graduation.
The projects are interesting and it is a good chance to get experience. A friend of mine's daughter is working in an internship in materials science. They are giving her access to the MEMS silicon fabrication facility (which, she tells me, costs more than they are paying her for the summer). Summer projects in computation in the past have involved networking and cryptography.
The summer programs are closed for this year. You can find them described on the LLNL web site (llnl.gov). The directorates do the hiring, so please don't send me your resume. I can't help in this area.
There are a number of other national labs with summer programs. Labs that come to mind are Los Alamos (LLNL's "sister lab"), Oak Ridge, Argonne, the Pacific Northwest National Lab. There is also an Dept. of Energy engineering lab in Idaho, but I don't know if they hire summer students.
Finally, for what it's worth my heart goes out to anyone who is graduating in CS or engineering these days. In my twenty four years working in this industry I have never seen times like these (and that includes the 1992 down turn). As others have noted, the problem is that there is unemployment among experienced engineers, so this makes it doubly difficult for new grads.
The irony is that from what I've read, hiring is booming for engineers in India (see my essay An Economics Question). This is one reason I'm grateful to have a job at a National Lab. My work requires a security clearance and it will never be moved overseas.
"world's oldest sea water -- a vast, still pool unstirred for millennia, walled by steep ridges and lidded with ice." Bring on the "Jurassic Park" references."
For those of us who have studied that dread work, the Necronomicon, the truth is not Jurassic Part but the the Elder Gods. Yes, my slashdot fellow readers, what will be found are those who were here before us. Trapped for millions of years behind the walls of ice will be found those who came from beyond. Behind the icy barriers they have waited, only now to be awakened. We can at least take heart in the fact that this is the northern polar climbs. If it were the cold icy regions of the south pole (where the Mountains of Maddness lie), those released elder Gods would come forth to gorge on penguins. The horror! The rejoicing in Redmond! None of us can question which operating system Cthulhu would use! At least we are spared this fate.
Ahh, now I understand: it's all innovating for the customer. Once you find out what the customer wants, then the innovation can begin. How silly of me to have missed this.
rather than deeply.
The Microsoft interview style is to ask the interviewee a constant stream of white board programming problems and throught puzzles. While this selects people with a certain level of intelligence, it also selects people who can think rapidly "on their feet".
Perhaps the end result is to select a homogenious population of "Softies" think fast, settle on an approach and then hack it into code. Where a better approach to product development might be to think about the design, think about some alternatives, discuss the design and then implement it.
Many people agree that Microsoft software evolves once it has been released. The common example being a first product that is inadequate, buggy and slow, eventually evolving into something that becomes popular. Perhaps this is a result of a culture of programmers who believe they are very smart (after all, they survived the Microsoft interview), think fast and then entomb their initial half-baked design in software.
I can see your point about privacy. It is true that once you allow something to read email it could be abused. But to balance this is the fact that, at least for me, email would be useless without a spam filter.
Privacy is not an issue in my case. I use text only email on Linux (email never touches my Windoz system for security reasons). I run a spam filter for my own email account, so it is my program that reads my email, not someone elses. I read my email on a shared Linux system run by the ISP that hosts my domain (my ISP is webquarry.com).
As far as I know, the RBL approach would not work in my case. I do discard some email one the basis of the domain name, which is far less efficient than the RBL. My spam filter keeps a log of some of the header information from the email it discards. A fair amount of spam is going through fixed domain names these days (e.g., like the infamous tekmailer).
One of the problems I had with the commonly used spam filters was that it was unclear to me how to install them in the case where I am simply piping my email to them. I was also concerned about resource usage, since I am using a shared system. So like a typical programmer I wrote my own spam filter in C++. It is probably 80 to 90 percent efficient. Enough spam still gets through that I'm going to take another look at SpamAssassin and see if I can get it to run with a "procmail" forward. It is just too time consuming to constantly hack the spam filter for the latest evil spammer trick (recently they have been sending spam to my email address from the other valid user on my domain, where I don't check content).
I have noticed that black lists are indeed effective. Many spammers now use "bullet proof" spam hosts, so they use static domain names. However, there has been an marked rise in zombie systems sending spams. These are systems that are infected by viruses and then used as spam hosts. Since these systems come on line rapidly (when they are infected) and then drop out (when they are cleared of the virus or booted off their ISP) it seems unlikely that black lists will help.
At least in the spam stream I see, there is more than 1-2 percent of the spam flow from zombies. The best technique seems to be to use a black list first and then content filter.
An a related topic in the parent post:
In a previous post, in another discussion, I also suggested that the sophistication of spam filters like SpamAssassin, which use several algorithms to filter spam, would consume lots of system resources. Another poster wrote that these tools do not consume much in the way of processor and memory resources. This seems counter intuitive, but I don't have any contrary evidence.
One of the most common measures that we see quoted in statistics is the mean, or the average. When there is a "bimodal" distribution, that is, clumping at the high end and clumping at the low end, the average is misleading. An example of this can be found in the claims for the benifits of the Bushies tax cuts. Bushie quoted the average benifit. That factored in all of the rich people who benified a lot. This was used to suggest that those "average families" that Bushie appeared with in photo-ops would benifit as well.
When there is a bimodal distribution a more representative statistic is the median. If you take a set of numbers, say the benifit from the Bushie tax cut, and sort them, the median is the number that is in the middle of this set (e.g., if there are 101 numbers, the 50th number is the median). Here you don't get a bias from the high end. If Bushie had quoted the median, instead of the average, people would have immediately realized that they were not getting much and that wealthy people were getting a lot.
Which brings us to income statistics and the point raised in the parent post. If the median salary was quoted it seems likely that computer industry salaries would have remained constant or declined.
We are definitely seeing bleaker times in the computer industry than I have seen in my twenty plus year career. And I include the 1992 downturn. However I think that part of what is happening is that the factors that have effected other job catagories are now effecting "knowledge worker" jobs. My guess is that if you were to look at the median, not the average, income of individual workers, you would see a decline over the last twenty years.
The government frequently quotes household or family income, and again, they quote the average. During the last twenty years some classes of workers have done much better than others. In particular, until 2000 those of us in the computer industry were on the winning side. Many of us saw our salaries increase dramatically. In the 1990s few would have believed that unemployment in the computer industry would be higher than overall unemployment.
For those outside of the "knowledge worker" catagory household income has increased or remained steady, I theorize, because increasing numbers of women entered the workforce. Espeically in urban areas on the coasts where two incomes was necessary to survive as a middle class family.
I find it interesting that few people ever talk about the individual worker median income, adjusted for inflation (the common statistic quoted is household income). Quoting adjusted individual median income might throw into question the fantasy in the US that we are such a wealthy country. What people would find is that while we are, indeed, a wealthy country, over the last twenty years the wealth has been increasingly concentrated in the hands of a smaller and smaller faction of the population. As the economist Paul Krugman wrote, this is "class warfare". But it warfare practiced by the rich and the powerful on everyone else.
Increasingly we are seeing the executive branch (e.g., the departments that report to the President) either not publish statistics or publish misleading or partial statistics. This is true for many departments that previously prided themselves on non-partisanship.
The job forecasts and market outlook for programmers and software engineers did not mention anything about outsourcing. Could this be because outsourcing is a senstive political topic that the current administration is vulnerable on? I found it odd reading that job growth for programmers would be about the same as job growth overall, without any mention of why such tepid job prospects were being forecast. In fact, I found nothing about low wage competition for "knowledge worker" jobs.
Then there is the issue of job catagories. Apparently the job prospects for "software engineers" were bright, while those for programmers were mediocre.
I have never worked in an environment where someone did design and someone else implemented this design in software. Yes I've had customers provide a broad outline of what they wanted, sometimes in terms of system components, but the engineering of large software systems is closely tied to their implementation. So as far as I'm concerned the division between "programmer" and "software engineer" does not exist. In fact some of the problems encountered in offshore outsourcing involve the attempt to separate software engineering from programming. Those contracting for low wage programming must provide detailed documentation that describes exactly what they want and how they want it done. Even then sometimes the software that is delivered is not adequate.
I now see that the comments on operating system authorship vs. kernel authorship are on target. In responding to my email Ken Brown seems to admit that Linus wrote the Linux kernel.
Perhaps I need to go back an review my books on operating systems. Perhaps I am the one in error here. But to me an operating system is as I have described it in my reply to Ken Browns reply to my initial email. This is included below.
Again, all this may simply be a waste of time. Ken Brown seems to have latched on to a position that most in the computer science and engineering community feel is silly. He seems unlikely to admit he is wrong. My comparision of Ken Brown to Peter Duesberg is probably appropriate.
Dear Ken:
Perhaps the problem is that, my view, Microsoft has perverted the meaning of operating system. What Microsoft calls an "operating system" comes with a vast array of applications. In my view the operating system called Windows NT is the software system described in "Inside Windows NT" by Helen Custer and David N. Cutler, 1992.
As far as I am concerned, an operating system is software that:
Manages access to devices like disk, graphic displays, the keyboard and, in modern systems, the network interface, sound and high performance graphic devices.
Manages memory. Modern operating systems provide virtual memory, that gives each application the illusion that it is running in its own address space.
Manages processes (applications). An operating system provides support for switching process execution when a process accesses a physical device and when its allocated time slice expires.
File system. The file system exists above the device management software.
Network protocol stack. The network protocol stack, like the file system, exists above the software that provides abstract access to the network interface hardware.
In theory some of these components, like the protocol stack and the file system, could be application level, not operating system level. However, in practice they are built into the operating system for performance reasons. And in the case of the file system, there is usually integration with the virtual memory subsystem.
If you open a text book on operating systems, an "operating system" is what I have described above. The Minix book is one example. On Slashdot people have cautioned me to refer to this as a "kernel". I guess that I'm from "the old school". To me what I have listed above is an operating system. "Kernel" does not have as exact a definition in my understanding. For example, one view of an operating system kernel might be that it does not include the file system and the network stack.
The window system, user shells, file system utilities, compilers, text or graphic editors are not part of the operating system.
To return to the original point: whether you want to call it an operating system or a "kernel" there is no question in my mind that Linus Torvalds wrote Linux. As I noted, this operating system has evolved. Compared to the current operating system, the initial operating system was relatively simple. It was well within what a dedicated, talented software engineer could implement. As I've noted, most UNIX software engineers will agree with my view.
As Linus Torvalds would be the first to acknowledge, he is a small part of the Linux community. He has managed the integration and release of the components that I have listed above as "operating system" (or, if you must, "kernel"). The fact that he has been willing to be inclusive in this process has sometimes been listed as the re
OK, this was probably a waste of time. Ken Brown is so over the top that he has to have an agenda. But for what it is worth, here is a copy of an email I sent him (with minor changes)
Dear Mr. Brown:
I am a software engineer with over twenty three years of experience. Much of my background is in the design and implementation of system software, including compilers and runtime support. If you are interested you can find my resume at my domain.I am not in the Linux fanatic camp. I use Linux, but in many ways I am disappointed with its popularity. I would much rather that the freeBSD operating system, which I regard as superior, had Linux's popularity. But popularity is the result of many factors, some of which have nothing to do with technology.
As a highly experienced software engineer and someone who is not a Linux fanatic, let me state that I absolutely believe that Linus Torvalds wrote Linux. And I also fully believe Prof. Tanenbaum when he states that Linux was not "stolen" from Minux.
Linux has been many years in development. The initial operating system was not the operating system that exists today. Linux has evolved over the years and many people have contributed to this evolution. As you note, Linus was young when he wrote Linux and I have no doubt that he too has evolved into a skilled and expert operating system designer and implementer. Linux is now far better for this experience and the work of its many contributors.
I find it rather odd that you, who are not an expert in software, are arguing against a large community of people who are experts, that Linus did not write Linux. I would hazard a guess that most UNIX systems programmers find your arguments silly. I also find it interesting that many of your arguments mirror those that have been put forward by Microsoft, a company that clearly finds Linux a threat.
Yours,
Wintermute
Several people have posted that I've confused Spamhaus with SpamCop. Sorry. It was careless on my part. My appologies to Spamhaus.
There is nothing wrong with charging for access to a spam black list. You could even charge on the basis of the bandwidth that the customer used. So big customers that hit the database a lot pay, people who it it only a few hundred times a day pay little or nothing.
My impression however is that Spamhaus is compromised by their association with IronPort, which provides a system for high speed mass mailing. Yeah I know that these are only supposed to be used by "the good guys". But that assumes that we agree on what a "good guy" is.
Perhaps I'm being too hard on Spamhaus and IronPort. But like many you, I get hundreds of spams a day. It has gotten to the point where having domain based email that comes to my Linux shell exacts a cost, even with my spam filter in place. So I don't look kindly on anyone who supports mass e-mail, whether they are IronPort or some spammer in the Bahamas. Given this view, I don't look kindly on those who associate with spammers (ah mass emailers), and that includes Spamhaus.
At the very minimum Spamhaus fails the "Caeser's wife" test. They support a spam blacklist database and they are owned by a company that sells mass mailing services. Seems like a built in conflict of interest.
I would never advocate violence and torture, but the first thing that occured to me when I read this is brand Darl!. Perhaps the Microsoft logo tatooed onto one of his ass cheeks. Then he can remember who 0wns his ass.
And Darl (and bodyguards), this is satire, not a suggestion that anyone violate your ass.
Crude, I know. Sorry. Back to more mature discussions...
As time goes on DSPAM (and SpamAssassin for that matter) become more and more sophisticated, incorporating more complex algorithms. What I also find striking is that many of these algorithms appear to be compute intensive. These spam filters seem to be designed for server side ISP level email filtering. I would expect that a computer would have to be dedicated to running this anti-spam software.
Also, as a number of posters have noted, configuring these spam filters takes some effort and education on the part of the user.
This level of resource comsumption is fine for an ISP, but it seems problematic for a single user or someone using a shared system.
I only use Linux shell based email to avoid worms and viruses. I've had my domain since 1995, so I got a lot of spam. As the tide of SPAM increased, overcoming the primative SPAM filter I was using I looked at both SpamAssassin and DSPAM. But these tools do not appear appropriate for a shared Linix system like the one that hosts my domain. So, in classic "reinvent the wheel fashion", I wrote my own SPAM filter. It is just a simple (compared to these tools) rule based filter. It filters out enough SPAM that email it not totally useless. This email filter can be found here. The email filter is written in C++ in an attempt to minimize resource usage. It is published as open source.
I too would think that a portfolio of past work would demonstrate to a prospective employer that I can engineer software. I have published thousands of lines of Java and C++ code on my web site www.bearcave.com.
What constantly strikes me as odd is that people I've interviewed with are more or less unwilling to look at my work (note that this work belongs to me and has been done on my own time). And those who do interviews that consists largely of programming problems still insist in having me code up their little algorithm on the white board. As I note on the web page linked to above, I think that in part this is because the interviewer does not know what else to ask.
When I have been a hiring manager I've looked for people who really loved software and had engineered large systems. Given this they can learn what ever else I need them to know.