The truth of the matter is that it is very hard to support random other languages on VMs written for certain languages.
All these dynamic languages do one thing or another that puts a hole in your plan. Ruby with it's continuations is right up there but Python with "modify anything fundamental anytime" isn't much better. The native environment has a huge headstart.
Myself, it works good for me on a storage server. There are some edge cases left, for example I could provoke a panic by combining ZFS on a 3-disk array with automatically parking the disks on timeout. Accessing the filesystem with power down disks didn't go so well. This was in 9-current, though.
ZFS is still the only thing out there providing all of: - snapshots - compression - attributes such as compression can be turned on and off by directory tree - a "filesystem-aware raid", aka something that avoids the RAID hole - and as mentioned optional extra redundancy (more than one disk can die) and the checksumming. That means, for example, you can have your filesystem like raid-5 but some directories as redundant as raid-6
Until Linux gets BTRFS (and I'm not sure how complete it is with regards to all those features) it's the best thing for a storage server out there.
%%
Apple's diversion either means they do their own thing (ZFS seems excessively hard to integrate) or is based on patent concerns, or the former because of the latter.
But if you look at the core of it, NetApp tries to claim patents on everything that does filesystem-integrated snapshots (as opposed to the lame LVM raw device layer snapshots in Linux). Reimplementing a filesystem with snapshots, whether you call it ZFS or not, won't help here.
Memtest86 tests much less of the memory than you think. It is 100% no-load. It does find outright broken memory cells but it does nothing if the memory interface runs unreliably.
To test your memory interface under stress you use a program named "Superpi", you run the "32M" test. It is available for Linux and runs on FreeBSD. I find a lot more problems with SuperPi than with memtest, a lot of memtest-stable machines don't actually work right once you stress-test.
%%
To test the CPUs/cores, you use "MPrime" or "Prime95" (same thing). It is the hardest load test that the overclocking record chasers have found, and they try very hard to find more and more nasty tests to proof that their competitors' overclock is not valid. They do this all day long, you should profit from their research.
You run MPrime with one instance per core. Available for Linux, IIRC also works on FreeBSD.
Be warned that the CPU temperature during MPrime will raise to levels that no other program I am aware of reaches. That's the point. MPrime also has a very high amount of plausibility checking on it's intermediate results. The combination of those two factor is why it is such an effective hardware test.
%%
So, in summary:
Run: 1) MPrime for 36 hours (all cores simultaneously, one MPrime each) 2) 24 hours of memtest86 3) a whole bunch of SuperPi 32M.
If there is any 3D graphics ever used you also run Futuremark's 3DMark (Windoze only).
Oh, and you will have to note the CPU temperature that you get during that mprime run and never exceed that temperature during everyday work from then on. This usually isn't a problem since mprime will heat your CPU like nothing else.
Good luck. Notebooks in particular, and cheap ready-made desktops not distributed by Dell tend to fail this outright. If any of these steps fail you can't pass any important data through this computer, it can and sooner or later will scramble you harddrive contents, silently, so that you backup USB drive already has the corrupted version by the time you notice.
All of us cursed GNU creeping featurism in the commandline utilities Funny. I tend to curse the lack of features in non-GNU utilities. For instance, grep -o.
Well, it goes both ways. GNU dropped `tail -r` which is pretty bad since at the time many scripts were written that was the only portable way to reverse the lines in a file.
But anyway, this was part of the "what this is not" intro. If you run this you get GNU *utils.
So, can I expect debian packages of BSD userland soon?
What you probably want to do is have a FreeBSD userland in a chroot.
That way you have binaries built in Debian/FreeBSD binaries at the root. You have regular Linux binaries in/compat/linux (which isn't a chroot, it falls through to the regular filesystem). And if you want to keep your shellscripts portable and/or need something from FreeBSD userland you put that into a/var/chroot/freebsd.
If said scenario still exists, there needs to be some long, loud, and pointed complaining engaged in on X's mailing lists.
My attempts to even get a single new Xorg bug fixed ended up in frustration and me feeling sure that I have been lied to by the developer in question when he said it works on his installation. I went through all the trouble of doing a git version Xorg build (there's no single module-tree with a central build anymore and that Python metabuilder they have did not go through for me). Only to find that the bug behaved the same way in their devhead.
I haven't been very active in the FreeBSD project since I now work for a company that uses Linux. The pressure from things that don't work at work is off. I'd like to dig my teeth into something that is broken for me every day. But even though Xorg fits that criteria (too well...) they pretty much excluded themselves from the list.
And I got a hunch that I am not the only one who feels that way. Why would people who value correctness join a project that doesn't? Human nature says otherwise.
Maybe a second fork from pre-license change XFree86, emphasizing on correctness instead of features and instead of support for hardware that long-time computer professionals don't use in the first place is the way to go.
This sounds insane to people who approach this from the usual angle. Linux has a lot more support for all the junk and semi-junk hardware out there, but some of the GNU core Unix userland is of questionable quality. All of us cursed GNU creeping featurism in the commandline utilities and GNU libc problems at some time or another. You would think people want the Linux kernel and the FreeBSD Unix userland. So why go the other way round?
There are very specific needs being addressed by using the FreeBSD kernel inside a Debian.
FreeBSD's ports system for third-party applications only has a devhead, and that has caused an increasing number of problems. FreeBSD has stable branches and releases for kernel, for "core Unix" userland including binutils and gcc/g++, but not for third-party applications. At the time that this was created it was great, because what we wanted at the time was a stable base system to do "server stuff" with, and the ports/applications were just for accessing the things, a light desktop that didn't do much except run xterm and emacs.
Today, I see two main problems with what worked a few years back:
1) those "server style" third-party applications aren't sitting flat on a Unix anymore. They are stacks of dependencies of considerable depths. It's not an apache with mod_cgi and the base perl system anymore.
2) some third-party applications became very aggressive lately and can be unusable in their newest releases. Many people bash GNOME and/or KDE, myself my favorite target is Xorg. The Xorg server has caused the most headaches across all my Linux and FreeBSD machines in the last years.
So, here's the trick. FreeBSD only has one branch in ports, so even if you use an older -STABLE release branch of the FreeBSD core system you still get the newest releases of third-party applications via ports. That's why my *most* stable OS (FreeBSD) had caused me the most headaches lately, because it upgrades me to the newest Xorg *first*, not last like it should.
I don't want to distract too much from the point of this posting by giving reasons why people want the FreeBSD kernel, let's just say there are enough of us. But no matter how much you want the FreeBSD kernel, many see increasing problems with ports/applications for the reasons I gave.
Debian provides stable branches for all applications, and that makes some people who don't generally like Linux still go "PLING!".
In addition to all that, Debian's packaging system, and the way that it is kept working (few package screwups upgrading), the way that it integrated/etc/* file management are simply first class and blow other Linuxes out of the water, too. Debian's packaging is the best out there, I haven't seen anyone challenge that notion in a long time.
So, very suddenly you have a demand for the FreeBSD kernel in a Debian application provision system and here we are.
%%
(BTW, what blows my mind for real is that FreeBSD is now partially sold based on driver availability. Because they kept their NDIS windoze driver integration system alive and maintained when Linux didn't. That is... something, I have to think about it)
People mentioned the self-checks and debugging features that used to be turned on in FreeBSD development branches and beta releases.
Self-checks, which are the major source of kernel slowdowns in those kernel options, are not turned on in the 8.0 release candidates.
Debugging is on, but unless you are very short of memory it should not cause a noticeable slowdown.
FreeBSD's slowness in these benchmarks can be attributed to two factors:
1) the compiler. The GPL v3 is unacceptable for FreeBSD, so newest GCCs cannot be used as the base compiler. Users can choose to install a new gcc on their own (as a port) and then even go and compile all ports and even the base system with that new compiler (some parts might not have been cleaned up to comply with new language strictness that might have come with new gccs).
2) threading, as in the userland threading library support. It is very hard to tell whether there is some performance problem in FreeBSD's thread libraries or whether applications just happen to have been optimized and tested only with Linux.
That happens a lot and you can also see Solaris with it's M:N threading fail miserably for some threading benchmarks that do dump things, such as just creating and destroying threads at a high rate.
%%
The problem of "thread benchmarks" benchmarking bogus things and/or just having been written with Linux' thread model in mind has been going on for 12 something years now. Benchmarking thread systems in a realistic manner is very difficult. In real-world applications you don't create and destroy threads at a high rate and you minimize locking. Benchmarking this is almost as hard as benchmarking programming languages.
I haven't benchmarked threading libraries, knowing that it will take time that I don't have right now. I can tell you, however, that just the I/O subsystem in FreeBSD, as in filesystems and networking, isn't any slower than Linux. Not to mention GbE and today's disks are too slow to really show an OS difference for most tests anyway.
%%
The real question of I/O subsystems will come in when ZFS+Zraid is standard in FreeBSD and BTRFS is standard in Linux. In a couple of years from now nobody will understand why we ran today with no snapshots, with the raid hole (from block layer raid systems) and without transparent compression in the subtrees where we want it.
But these filesystems are complicated and there's some real performance difference visible.
%%
An area completely overlooked in the benchmark is the VM subsystem. Namely - what happens when you overload your RAM and paging commences? Linux used to make very bad choices here (dropping readonly pages, which is a wise thing as such, at rates about 10 times higher than wise).
FreeBSD used to be the go-to OS on memory shortage thanks to John Dyson's VM work (backed by a large database company that provided support and a realistic benchmarking environment during that development).
But today? Nobody knows. I'm not aware of any benchmarks that you can download that simulate memory stress and map the tradeoffs that the OS makes.
In general, the biggest obstacle to improve Linux, FreeBSD and everybody's else's OS performance is a lack of high-quality benchmarks.
Why don't people develop more benchmarks? Because they get annoyed that today, in 2009, no realistic OS benchmark will show a single number as a result. All OS benchmarks today can only make a map of what tradeoffs the OS chose, what part of the running application suite got the short end in favor of what other part. This isn't sexy and publishers don't like it.
People like reality reduced to single numbers, but in the area os OS benchmarks (and language benchmarks) that party is over.
Myself, I found myself gasping many years ago when I benchmarked network I/O versus userland CPU load. I hammered a couple of GbE interfaces while at the same time running moderate memory-intensive CPU benchmarks (with no network access from those CPU lo
which one was the latest version which didn't special-code with the Alt/meta key?
Longer story:
the special-coding that apparently both tightvnc and realvnc introduced works fine - when I actually use a keyboard. But I send x11 events to a VNC window and that does not work.
So which version would be the last one to treat the ALT/Meta key normally?
And would I need to downgrade the client, the server, either or both?
Of course, if you can offer a programming solution to simulate the Alt key modifier when sending x11 keystroke or mouse events, that would work even better.
It is bejond me how anybody who studied what AMD's 64 bit extensions actually do (the "nice" prefix trick) and has observed some of the tricky coding issues on pre-386 MS-DOS can want AMD's extension ton win over IA-64. You people are crazy. Well, no probably most of you don't even know how AMD's extension works.
IA-64 is about as much as a bitch when it comes to compilers as it gets, but it is truly the right thing to do (and I part-time work on a compiler).
One thing is clear: this guy (the writer) is clueless about the technical implications. The UK article is most interesting, thanks for the link. But it's really obvious from this piece, too.
Then, I wonder what IBM owning Java instead of Sun would change about the fact that Java sucks, from a performance, a memory and in many people's opinion from a language design standpoint? And will IBM's ownership magically repair Java in all the browsers?
A real-time GC is not in the least what we need. A real-time GC makes the pauses go away. We don't care about the pauses, we're not interactive. But the real-time GC creates an even bigger overall CPU- and run-time overhead. While the visible pauses go away, the application is slower.
That is the basic problem about many of these discussions: there is no magic bullet. There are GC schemes more or less suitable for some tasks, but not for others. It is tradeoffs, guys.
BTW, at my former employer I had a system written in C with parts using the Boehm GC. It was faster than the malloc/free variant (the application's part could switch at compile time). The Boehm GC would screw ITA's system royally, though.
We could very well write a GC specially coded for our application, but that is a lot of work. And since we can do without system-visible memory allocation when answering search requests, we prefer to that and we hide the data bulk from the GC when cleaning up the systems in-between activities.
GC is a hard problem, just as manual memory management is. For none of these you can buy or download a perfect solution.
Re:This problem isn't as hard as they make it soun
on
Common Lisp: Inside Sabre
·
· Score: 2, Insightful
You are abolutely right, it's easy.
Unfortunately you're just talking flights and not fares. [Hint: if you think you pay a price for a flight you board like you do in a Bus, you're wrong].
I am working for ITA and like to comment on some issue brought up here:
1) As said, we talk Orbitz here, and not SABRE. Currently, Orbitz
uses our software for domestic US flights, not for international.
2) Our engine does not use a functional programming style, rather the
opposite. Still, we found that Lisp is a great advantage. While
each hacker here has own preferences why he/she likes Lisp, key
elements (I see) are:
2a) macros, especiallly macros that allow us to define new iteration
constructs. C programmers can thing of being able to write their own
for/while/if as seem appropriate for the task as hand. Especially
with-[whatever] constructs, but also nice tricks with
destructuring-bind.
2b) scope, working annonymous functions with static scope. Kind of
Java's inner classes but in 1/10 of the codelines.
2c) said destructuring-bind which frees your from a lot of boring and
error-prone tasks of tree parsing with a snap.
2d) compile-time computing, a key element to make our software fast
without cluttering it up by expensing manually written source code by
a factor of 100 or by inventing ad-hoc code generators which need to
be debugged after they broke your system for weeks. Macros that can
use the full language at compile time and macros that can "walk" their
argument when passed at compile-time to find interesting things to do
with them. Also see define-compiler-macro to get an idea what makes
Lisp code fast while maintaining elegance (use with care, though).
2e) safety. A language without optional overflow checking of integers
is a toy at best and dangerous at worst.
2f) debugging and testing with the read-eval-print-loop (REPL). Like
the gdb prompt for evaluating code, but you can use the native
language and you have the full language. Or better like a shell where
thing's aren't echoed in ASCII and need to be re-parsed, but you get
the real objects you can play with (send message as defined in your
system). The debuggers in Allegro and CMUCL are rather crappy, IMHO,
but the REPL and ultra-fast re-compilation and loading of single
functions (standard feature of every Lisp) -used for debugging print
statements- make more than up for that.
Keep in mind that everyone of our Lisp hackers can contribute a Lisp
of similar length, this is just what *I* like.
For the record, I like C++, but I couldn't absord all the application--specific knowledge I need while spending my day figuring out C++ specialities and keeping them swapped in. C++ is for full-time C++ coders only.
Owning a NeXTStation in 1991, I cannot share this enthusiasm.
The Unix below was quite rooten, performance-wise, from a porting standpoint and from correctness and freedom of gcc warning issues in include files.
The great GUI and its builder were shipped without documentation until Simson Garfinckle finally wrote his book. You were supposed to use their 5-day training and fill the holes with an expensive support contract.
Hardware interface was also notoriously bad:
- Ethernet and SCSI really had sub-standard
performance. SCSI couldn't use many of the
available PC disks at the time.
- No slots, obviously.
- Only 8 RAM slots in the station (=32 MB).
- Non-upgradable Video card, and that with
display postscript and its tendency to do
single-pixel updates.
I originally bought it mostly because it came with some nice commercial software packages bundled, because it was faster than a SPARC at the same price and I liked (and still do) Objective-C. PC Unixes were crap at the time. But soon the rotten Unix was what killed it for me, I bought a SPARC 18 months later (both systems from my personal money), sold the NeXT and was happy (until Solaris-2.1 made me switch to free PC Unixes). In retrospective, it had been better to buy the slower SPARC in first place and most of the commercial software sucked anyway and you weren't supposed to update the stuff that came bundled (as if much NeXT software ever got into several releases...).
It is obvious that Tim doesn't properly seperate between specifying APIs and ABIs on one hand and a specific implementation on the other. The TCP/IP stack example shows this: To end the DOS/early-Windows chaos, people should have agreed (or forced to) APIs and ABIs, it wouldn't be required to use one single implementation.
However, in several ways D3D as an API does what Game programmers want where OpenGL does not. For example, it has better control over texture loading, which is important for games that try to be more beautiful and/or more outdoor - oriented than Quake. The step from Glide to OpenGL/D3D initially widened the speed gap between Quake and games like Unreal, Starsiege Tribes etc, mostly because texture loading was suboptimal and couldn't be controlled/tuned as in Glide.
Microsoft quickly extends D3D in ways the game programmers want, while the OpenGL consortium moves much slower and while moving it has to weigth the concerns of "serious" (non-game) 3D applications as well.
There are some more far-sighted programmers like Carmack and Chris Hecker who think (my interpretation) that using a single-vendor API will cause more trouble in the long term than is worth the quick improvements now, but few people enjoy the luxury of allowed to be stubborn.
Sweeney has to position his product relative to Quake, he needs to do some things better than Quake and (engine-wise) he lost ground while moving from Unreal vs. Q2 to UT vs. Q3A. OpenGL needs a lot of "right" tuning by the vendor and Quake is often used as the benchmark. It is obvious that Sweeney *needs* the better control features of D3D to get a product that is different from the new id engine while still performing well.
I'd like to make clear that I'm absolutely on the OpenGL side here, I'm just trying to point out that Sweeney isn't necessarily evil (although his post about the MS splitup is rather stupid IMHO and I can't agree with most of his programming language observations either).
I have two working Symbolics Lisp machines to give away for free in Hamburg/Germany. I will even ship in northern Germany to get a new home for them now that I'm moving to a smaller house.
I'd like to know your/the FSF's position on Java. Obviously Sun's policy regarding development of the core language (through standard processes) and access to the JDK lacks what we want (both the GPL - style free software people and the "OpenSource"/BSD people).
On the other hand, some people say that Java is a great opportunity for free software, namely the ease of code sharing, the improved trust one can set into a library (compared to a C library), portability and some abstraction features (passive reflection, inner classes, namespaces). Those people argue that Java will make a project model of great coders doing core libraries and less programming-mad people using them to create applications in their non-programming/ non-computing domain easier and/or more effective.
Questions: - Do your think that free Java tools reached the critical mass to continue use of Java even when we can't use Sun's tools anymore? - Do you think the free software Java movement is big enough to split the language in the extreme case, renaming it and doing something that is "just" compatible to Sun's Java? - Do you share the above opinion that Java might make free software projects more effective (especially desktop projects)? - What is considered "linking" in the GPL sense for a Java program? Putting the class files into the same zip/jar file?
The recent discussion about the Mattel Webfilter countersoftware raised some interesting points, namely: - A (free) software license may only be valid when money was paid - A (free) software license may require a (hand-written) signed document to be non-retractable
This could be solved by central trusted instances that free software authors hand their copyright over (signing a document and receiving a dollar), so that users of this trusted software know that the original author may not retract the license. They have to trust the central authority that could retract it, but a trusted instance would be a big improvement over random software authors. Also, we may have several such central instances, each package "sells" the copyright to one of them and sells a non-retractable license to several others.
For GNU software, the FSF already does this, at least the signing part.
Questions: - Do you consider paying the dollar (or such) for the license and/or copyright transfer, just in case it may proof required? - Would the FSF accept such a role of central authority for free software that is covered not by the GPL, but GPL-compatible licenses, such as the BSD (without advertising clause) or MIT/X11 licenses? - Would the FSF be in a role to actively build a network of such trusted instances?
For me, this looks like an attempt to work around the need to put out documentation.
While Matrox released enough docu to make GLX on the G-400 full-functional, Nvidia put out sources that don't make full use of the cards and no documentation to improve it.
I think this is a bit unfair. Johnc programs a game *engine*. Other people take the engine and produce those kind of games they think they can sell best.
The problem here is that the kind of games that require the most interesting engine are those that need collision detection and architecture seem from near. There is not much use of beauty in - say - a flight simulator's ground graphics. When you don't need the beauty, many of the most interesting programming tasks wouldn't be needed.
Hence the best engine hackers work on indoor/ground games where things fly and hit something. Commercially successful exploration of this technology usually requires that these objects represent projectiles.
Slashdot Mirror
... those galaxies are just running away from Justin Bieber.
[firefox user's voice] ... but my extensions didn't break. You sure this is a real upgrade?
The truth of the matter is that it is very hard to support random other languages on VMs written for certain languages.
All these dynamic languages do one thing or another that puts a hole in your plan. Ruby with it's continuations is right up there but Python with "modify anything fundamental anytime" isn't much better. The native environment has a huge headstart.
We should all move to LLVM.
FreeBSD is about to release 8.0, with ZFS, and ZFS has officially been labeled production-ready:
http://www.freebsd.org/news/status/report-2009-04-2009-09.html#FreeBSD/ZFS
Myself, it works good for me on a storage server. There are some edge cases left, for example I could provoke a panic by combining ZFS on a 3-disk array with automatically parking the disks on timeout. Accessing the filesystem with power down disks didn't go so well. This was in 9-current, though.
ZFS is still the only thing out there providing all of:
- snapshots
- compression
- attributes such as compression can be turned on and off by directory tree
- a "filesystem-aware raid", aka something that avoids the RAID hole
- and as mentioned optional extra redundancy (more than one disk can die) and the checksumming. That means, for example, you can have your filesystem like raid-5 but some directories as redundant as raid-6
Until Linux gets BTRFS (and I'm not sure how complete it is with regards to all those features) it's the best thing for a storage server out there.
%%
Apple's diversion either means they do their own thing (ZFS seems excessively hard to integrate) or is based on patent concerns, or the former because of the latter.
But if you look at the core of it, NetApp tries to claim patents on everything that does filesystem-integrated snapshots (as opposed to the lame LVM raw device layer snapshots in Linux). Reimplementing a filesystem with snapshots, whether you call it ZFS or not, won't help here.
Memtest86 tests much less of the memory than you think. It is 100% no-load. It does find outright broken memory cells but it does nothing if the memory interface runs unreliably.
To test your memory interface under stress you use a program named "Superpi", you run the "32M" test. It is available for Linux and runs on FreeBSD. I find a lot more problems with SuperPi than with memtest, a lot of memtest-stable machines don't actually work right once you stress-test.
%%
To test the CPUs/cores, you use "MPrime" or "Prime95" (same thing). It is the hardest load test that the overclocking record chasers have found, and they try very hard to find more and more nasty tests to proof that their competitors' overclock is not valid. They do this all day long, you should profit from their research.
You run MPrime with one instance per core. Available for Linux, IIRC also works on FreeBSD.
Be warned that the CPU temperature during MPrime will raise to levels that no other program I am aware of reaches. That's the point. MPrime also has a very high amount of plausibility checking on it's intermediate results. The combination of those two factor is why it is such an effective hardware test.
%%
So, in summary:
Run:
1) MPrime for 36 hours (all cores simultaneously, one MPrime each)
2) 24 hours of memtest86
3) a whole bunch of SuperPi 32M.
If there is any 3D graphics ever used you also run Futuremark's 3DMark (Windoze only).
Oh, and you will have to note the CPU temperature that you get during that mprime run and never exceed that temperature during everyday work from then on. This usually isn't a problem since mprime will heat your CPU like nothing else.
Good luck. Notebooks in particular, and cheap ready-made desktops not distributed by Dell tend to fail this outright. If any of these steps fail you can't pass any important data through this computer, it can and sooner or later will scramble you harddrive contents, silently, so that you backup USB drive already has the corrupted version by the time you notice.
All of us cursed GNU creeping featurism in the commandline utilities
Funny. I tend to curse the lack of features in non-GNU utilities. For instance, grep -o.
Well, it goes both ways. GNU dropped `tail -r` which is pretty bad since at the time many scripts were written that was the only portable way to reverse the lines in a file.
But anyway, this was part of the "what this is not" intro. If you run this you get GNU *utils.
So, can I expect debian packages of BSD userland soon?
What you probably want to do is have a FreeBSD userland in a chroot.
That way you have binaries built in Debian/FreeBSD binaries at the root. You have regular Linux binaries in /compat/linux (which isn't a chroot, it falls through to the regular filesystem). And if you want to keep your shellscripts portable and/or need something from FreeBSD userland you put that into a /var/chroot/freebsd.
If said scenario still exists, there needs to be some long, loud, and pointed complaining engaged in on X's mailing lists.
My attempts to even get a single new Xorg bug fixed ended up in frustration and me feeling sure that I have been lied to by the developer in question when he said it works on his installation. I went through all the trouble of doing a git version Xorg build (there's no single module-tree with a central build anymore and that Python metabuilder they have did not go through for me). Only to find that the bug behaved the same way in their devhead.
I haven't been very active in the FreeBSD project since I now work for a company that uses Linux. The pressure from things that don't work at work is off. I'd like to dig my teeth into something that is broken for me every day. But even though Xorg fits that criteria (too well...) they pretty much excluded themselves from the list.
And I got a hunch that I am not the only one who feels that way. Why would people who value correctness join a project that doesn't? Human nature says otherwise.
Maybe a second fork from pre-license change XFree86, emphasizing on correctness instead of features and instead of support for hardware that long-time computer professionals don't use in the first place is the way to go.
This sounds insane to people who approach this from the usual angle. Linux has a lot more support for all the junk and semi-junk hardware out there, but some of the GNU core Unix userland is of questionable quality. All of us cursed GNU creeping featurism in the commandline utilities and GNU libc problems at some time or another. You would think people want the Linux kernel and the FreeBSD Unix userland. So why go the other way round?
There are very specific needs being addressed by using the FreeBSD kernel inside a Debian.
FreeBSD's ports system for third-party applications only has a devhead, and that has caused an increasing number of problems. FreeBSD has stable branches and releases for kernel, for "core Unix" userland including binutils and gcc/g++, but not for third-party applications. At the time that this was created it was great, because what we wanted at the time was a stable base system to do "server stuff" with, and the ports/applications were just for accessing the things, a light desktop that didn't do much except run xterm and emacs.
Today, I see two main problems with what worked a few years back:
1) those "server style" third-party applications aren't sitting flat on a Unix anymore. They are stacks of dependencies of considerable depths. It's not an apache with mod_cgi and the base perl system anymore.
2) some third-party applications became very aggressive lately and can be unusable in their newest releases. Many people bash GNOME and/or KDE, myself my favorite target is Xorg. The Xorg server has caused the most headaches across all my Linux and FreeBSD machines in the last years.
So, here's the trick. FreeBSD only has one branch in ports, so even if you use an older -STABLE release branch of the FreeBSD core system you still get the newest releases of third-party applications via ports. That's why my *most* stable OS (FreeBSD) had caused me the most headaches lately, because it upgrades me to the newest Xorg *first*, not last like it should.
I don't want to distract too much from the point of this posting by giving reasons why people want the FreeBSD kernel, let's just say there are enough of us. But no matter how much you want the FreeBSD kernel, many see increasing problems with ports/applications for the reasons I gave.
Debian provides stable branches for all applications, and that makes some people who don't generally like Linux still go "PLING!".
In addition to all that, Debian's packaging system, and the way that it is kept working (few package screwups upgrading), the way that it integrated /etc/* file management are simply first class and blow other Linuxes out of the water, too. Debian's packaging is the best out there, I haven't seen anyone challenge that notion in a long time.
So, very suddenly you have a demand for the FreeBSD kernel in a Debian application provision system and here we are.
%%
(BTW, what blows my mind for real is that FreeBSD is now partially sold based on driver availability. Because they kept their NDIS windoze driver integration system alive and maintained when Linux didn't. That is ... something, I have to think about it)
People mentioned the self-checks and debugging features that used to be turned on in FreeBSD development branches and beta releases.
Self-checks, which are the major source of kernel slowdowns in those kernel options, are not turned on in the 8.0 release candidates.
Debugging is on, but unless you are very short of memory it should not cause a noticeable slowdown.
FreeBSD's slowness in these benchmarks can be attributed to two factors:
1) the compiler. The GPL v3 is unacceptable for FreeBSD, so newest GCCs cannot be used as the base compiler. Users can choose to install a new gcc on their own (as a port) and then even go and compile all ports and even the base system with that new compiler (some parts might not have been cleaned up to comply with new language strictness that might have come with new gccs).
2) threading, as in the userland threading library support. It is very hard to tell whether there is some performance problem in FreeBSD's thread libraries or whether applications just happen to have been optimized and tested only with Linux.
That happens a lot and you can also see Solaris with it's M:N threading fail miserably for some threading benchmarks that do dump things, such as just creating and destroying threads at a high rate.
%%
The problem of "thread benchmarks" benchmarking bogus things and/or just having been written with Linux' thread model in mind has been going on for 12 something years now. Benchmarking thread systems in a realistic manner is very difficult. In real-world applications you don't create and destroy threads at a high rate and you minimize locking. Benchmarking this is almost as hard as benchmarking programming languages.
I haven't benchmarked threading libraries, knowing that it will take time that I don't have right now. I can tell you, however, that just the I/O subsystem in FreeBSD, as in filesystems and networking, isn't any slower than Linux. Not to mention GbE and today's disks are too slow to really show an OS difference for most tests anyway.
%%
The real question of I/O subsystems will come in when ZFS+Zraid is standard in FreeBSD and BTRFS is standard in Linux. In a couple of years from now nobody will understand why we ran today with no snapshots, with the raid hole (from block layer raid systems) and without transparent compression in the subtrees where we want it.
But these filesystems are complicated and there's some real performance difference visible.
%%
An area completely overlooked in the benchmark is the VM subsystem. Namely - what happens when you overload your RAM and paging commences? Linux used to make very bad choices here (dropping readonly pages, which is a wise thing as such, at rates about 10 times higher than wise).
FreeBSD used to be the go-to OS on memory shortage thanks to John Dyson's VM work (backed by a large database company that provided support and a realistic benchmarking environment during that development).
But today? Nobody knows. I'm not aware of any benchmarks that you can download that simulate memory stress and map the tradeoffs that the OS makes.
In general, the biggest obstacle to improve Linux, FreeBSD and everybody's else's OS performance is a lack of high-quality benchmarks.
Why don't people develop more benchmarks? Because they get annoyed that today, in 2009, no realistic OS benchmark will show a single number as a result. All OS benchmarks today can only make a map of what tradeoffs the OS chose, what part of the running application suite got the short end in favor of what other part. This isn't sexy and publishers don't like it.
People like reality reduced to single numbers, but in the area os OS benchmarks (and language benchmarks) that party is over.
Myself, I found myself gasping many years ago when I benchmarked network I/O versus userland CPU load. I hammered a couple of GbE interfaces while at the same time running moderate memory-intensive CPU benchmarks (with no network access from those CPU lo
Short question to VNC exports:
which one was the latest version which didn't special-code with the Alt/meta key?
Longer story:
the special-coding that apparently both tightvnc and realvnc introduced works fine - when I actually use a keyboard. But I send x11 events to a VNC window and that does not work.
So which version would be the last one to treat the ALT/Meta key normally?
And would I need to downgrade the client, the server, either or both?
Of course, if you can offer a programming solution to simulate the Alt key modifier when sending x11 keystroke or mouse events, that would work even better.
Thanks!
It is bejond me how anybody who studied what AMD's 64 bit extensions actually do (the "nice" prefix trick) and has observed some of the tricky coding issues on pre-386 MS-DOS can want AMD's extension ton win over IA-64. You people are crazy. Well, no probably most of you don't even know how AMD's extension works.
IA-64 is about as much as a bitch when it comes to compilers as it gets, but it is truly the right thing to do (and I part-time work on a compiler).
One thing is clear: this guy (the writer) is clueless about the technical implications. The UK article is most interesting, thanks for the link. But it's really obvious from this piece, too.
Then, I wonder what IBM owning Java instead of Sun would change about the fact that Java sucks, from a performance, a memory and in many people's opinion from a language design standpoint? And will IBM's ownership magically repair Java in all the browsers?
[I work for ITA]
A real-time GC is not in the least what we need. A real-time GC makes the pauses go away. We don't care about the pauses, we're not interactive. But the real-time GC creates an even bigger overall CPU- and run-time overhead. While the visible pauses go away, the application is slower.
That is the basic problem about many of these discussions: there is no magic bullet. There are GC schemes more or less suitable for some tasks, but not for others. It is tradeoffs, guys.
BTW, at my former employer I had a system written in C with parts using the Boehm GC. It was faster than the malloc/free variant (the application's part could switch at compile time). The Boehm GC would screw ITA's system royally, though.
We could very well write a GC specially coded for our application, but that is a lot of work. And since we can do without system-visible memory allocation when answering search requests, we prefer to that and we hide the data bulk from the GC when cleaning up the systems in-between activities.
GC is a hard problem, just as manual memory management is. For none of these you can buy or download a perfect solution.
You are abolutely right, it's easy.
Unfortunately you're just talking flights and not fares. [Hint: if you think you pay a price for a flight you board like you do in a Bus, you're wrong].
Hi,
I am working for ITA and like to comment on some issue brought up here:
1) As said, we talk Orbitz here, and not SABRE. Currently, Orbitz
uses our software for domestic US flights, not for international.
2) Our engine does not use a functional programming style, rather the
opposite. Still, we found that Lisp is a great advantage. While
each hacker here has own preferences why he/she likes Lisp, key
elements (I see) are:
2a) macros, especiallly macros that allow us to define new iteration
constructs. C programmers can thing of being able to write their own
for/while/if as seem appropriate for the task as hand. Especially
with-[whatever] constructs, but also nice tricks with
destructuring-bind.
2b) scope, working annonymous functions with static scope. Kind of
Java's inner classes but in 1/10 of the codelines.
2c) said destructuring-bind which frees your from a lot of boring and
error-prone tasks of tree parsing with a snap.
2d) compile-time computing, a key element to make our software fast
without cluttering it up by expensing manually written source code by
a factor of 100 or by inventing ad-hoc code generators which need to
be debugged after they broke your system for weeks. Macros that can
use the full language at compile time and macros that can "walk" their
argument when passed at compile-time to find interesting things to do
with them. Also see define-compiler-macro to get an idea what makes
Lisp code fast while maintaining elegance (use with care, though).
2e) safety. A language without optional overflow checking of integers
is a toy at best and dangerous at worst.
2f) debugging and testing with the read-eval-print-loop (REPL). Like
the gdb prompt for evaluating code, but you can use the native
language and you have the full language. Or better like a shell where
thing's aren't echoed in ASCII and need to be re-parsed, but you get
the real objects you can play with (send message as defined in your
system). The debuggers in Allegro and CMUCL are rather crappy, IMHO,
but the REPL and ultra-fast re-compilation and loading of single
functions (standard feature of every Lisp) -used for debugging print
statements- make more than up for that.
Keep in mind that everyone of our Lisp hackers can contribute a Lisp
of similar length, this is just what *I* like.
For the record, I like C++, but I couldn't absord all the application--specific knowledge I need while spending my day figuring out C++ specialities and keeping them swapped in. C++ is for full-time C++ coders only.
Owning a NeXTStation in 1991, I cannot share this enthusiasm.
The Unix below was quite rooten, performance-wise, from a porting standpoint and from correctness and freedom of gcc warning issues in include files.
The great GUI and its builder were shipped without documentation until Simson Garfinckle finally wrote his book. You were supposed to use their 5-day training and fill the holes with an expensive support contract.
Hardware interface was also notoriously bad:
- Ethernet and SCSI really had sub-standard
performance. SCSI couldn't use many of the
available PC disks at the time.
- No slots, obviously.
- Only 8 RAM slots in the station (=32 MB).
- Non-upgradable Video card, and that with
display postscript and its tendency to do
single-pixel updates.
I originally bought it mostly because it came with some nice commercial software packages bundled, because it was faster than a SPARC at the same price and I liked (and still do) Objective-C. PC Unixes were crap at the time. But soon the rotten Unix was what killed it for me, I bought a SPARC 18 months later (both systems from my personal money), sold the NeXT and was happy (until Solaris-2.1 made me switch to free PC Unixes). In retrospective, it had been better to buy the slower SPARC in first place and most of the commercial software sucked anyway and you weren't supposed to update the stuff that came bundled (as if much NeXT software ever got into several releases...).
Martin
They don't release the source for the working
version, they release source for a non-working
experimental new implementation.
Same as with Netscape/Mozilla. No fixing of
the quite fine 3.x, just major new stuff few
people want or even can work on.
It's amazing how exactly they reproduce Netscape's
steps/mistakes. No problems for Sun, "OpenSource"
will get the blame anyway.
Martin
What exactly are you missing?
It is obvious that Tim doesn't properly seperate between specifying
APIs and ABIs on one hand and a specific implementation on the other.
The TCP/IP stack example shows this: To end the DOS/early-Windows
chaos, people should have agreed (or forced to) APIs and ABIs, it
wouldn't be required to use one single implementation.
However, in several ways D3D as an API does what Game programmers want
where OpenGL does not. For example, it has better control over
texture loading, which is important for games that try to be more
beautiful and/or more outdoor - oriented than Quake. The step from
Glide to OpenGL/D3D initially widened the speed gap between Quake and
games like Unreal, Starsiege Tribes etc, mostly because texture
loading was suboptimal and couldn't be controlled/tuned as in Glide.
Microsoft quickly extends D3D in ways the game programmers want, while
the OpenGL consortium moves much slower and while moving it has to
weigth the concerns of "serious" (non-game) 3D applications as well.
There are some more far-sighted programmers like Carmack and Chris
Hecker who think (my interpretation) that using a single-vendor API
will cause more trouble in the long term than is worth the quick
improvements now, but few people enjoy the luxury of allowed to be
stubborn.
Sweeney has to position his product relative to Quake, he needs to do
some things better than Quake and (engine-wise) he lost ground while
moving from Unreal vs. Q2 to UT vs. Q3A. OpenGL needs a lot of
"right" tuning by the vendor and Quake is often used as the benchmark.
It is obvious that Sweeney *needs* the better control features of D3D
to get a product that is different from the new id engine while still
performing well.
I'd like to make clear that I'm absolutely on the OpenGL side here,
I'm just trying to point out that Sweeney isn't necessarily evil
(although his post about the MS splitup is rather stupid IMHO and I
can't agree with most of his programming language observations
either).
Martin
I have two working Symbolics Lisp machines to give
away for free in Hamburg/Germany. I will
even ship in northern Germany to get a new home
for them now that I'm moving to a smaller house.
Martin Cracauer cracauer@seagull.cons.org
I'd like to know your/the FSF's position on Java. Obviously Sun's
policy regarding development of the core language (through standard
processes) and access to the JDK lacks what we want (both the GPL -
style free software people and the "OpenSource"/BSD people).
On the other hand, some people say that Java is a great opportunity
for free software, namely the ease of code sharing, the improved trust
one can set into a library (compared to a C library), portability and
some abstraction features (passive reflection, inner classes,
namespaces). Those people argue that Java will make a project model
of great coders doing core libraries and less programming-mad people
using them to create applications in their non-programming/
non-computing domain easier and/or more effective.
Questions:
- Do your think that free Java tools reached the critical mass to
continue use of Java even when we can't use Sun's tools anymore?
- Do you think the free software Java movement is big enough to split
the language in the extreme case, renaming it and doing something
that is "just" compatible to Sun's Java?
- Do you share the above opinion that Java might make free software
projects more effective (especially desktop projects)?
- What is considered "linking" in the GPL sense for a Java program?
Putting the class files into the same zip/jar file?
The recent discussion about the Mattel Webfilter countersoftware
raised some interesting points, namely:
- A (free) software license may only be valid when money was paid
- A (free) software license may require a (hand-written) signed
document to be non-retractable
This could be solved by central trusted instances that free software
authors hand their copyright over (signing a document and receiving a
dollar), so that users of this trusted software know that the original
author may not retract the license. They have to trust the central
authority that could retract it, but a trusted instance would be a big
improvement over random software authors. Also, we may have several
such central instances, each package "sells" the copyright to one of
them and sells a non-retractable license to several others.
For GNU software, the FSF already does this, at least the signing
part.
Questions:
- Do you consider paying the dollar (or such) for the license and/or
copyright transfer, just in case it may proof required?
- Would the FSF accept such a role of central authority for free
software that is covered not by the GPL, but GPL-compatible
licenses, such as the BSD (without advertising clause) or MIT/X11
licenses?
- Would the FSF be in a role to actively build a network of such
trusted instances?
Martin Cracauer cracauer@freebsd.org
For me, this looks like an attempt to work around
the need to put out documentation.
While Matrox released enough docu to make GLX
on the G-400 full-functional, Nvidia put out
sources that don't make full use of the cards and
no documentation to improve it.
Just bought a G-400, for exactly this reason.
I think this is a bit unfair. Johnc programs a game *engine*. Other people take the engine and produce those kind of games they think they can sell best.
The problem here is that the kind of games that require the most interesting engine are those that need collision detection and architecture seem from near. There is not much use of beauty in - say - a flight simulator's ground graphics. When you don't need the beauty, many of the most interesting programming tasks wouldn't be needed.
Hence the best engine hackers work on indoor/ground games where things fly and hit something. Commercially successful exploration of this technology usually requires that these objects represent projectiles.