So specifically what we've been doing is taking every binary in the system and assigning it a layer number, which is a rank in a directed acyclic graph. There's about 5,500 binaries in the system.
oh, they know about acyclic graphs, good. I bet they have only 5500 levels in it. On a side note:
Debian GNU/Linux provides more than a pure OS: it comes with over 15490 packages, precompiled software bundled up in a nice format for easy installation on your machine... (http://www.debian.org/)
So why keep re-inventing the wheel, just ship vista with msdpkg/msapt...
To me it looks like the current DVD drives reached the reasonable limits at DVD+/-R and RW. The write speed is only x2 better than that of x4 drive, so not much reason to upgrade. The DL is unlikely to matter, as media is very expensive and higher capacity drives are on the way. However, what happened to the MtRainer/EasyWrite support? None of the drives still support it? Looks like NEC 4550 does support DVD-RAM, but the drive is still not available in the US and it's not clear how it will compare to DVD+MRW in terms of usability. For example, it would be cool to boot from Knoppix, customize it a bit (small things like wireless keys, printers, proxies, NIS, etc.) and keep configuration right on the same media.
It's amazing that.Net is even mentionned. Product of the company that couldn't deliver _anything_ technically excellent is compared to something which design was at least given a lot of thought, something that was implemented by several vendors and ported to several platforms, received a lot of feedback and made a few release cycles. As much as I don't think Java is a great thing, I simply wouldn't even consider.Net as a viable alternative. Microsoft spent no time to improve quality of their compilers (I have some acquitance with their c++ "98% compliant" implmentations), but spends a lot of time adding "features" with unclear semantics, even though the plain C++ or Java has a lot of dark corners when it comes to,for example, multithreading. I don't give a damn about any of their fantastic extensions, first show me the semantics and prove it allows to write correct programs!
Re:The real 90s versus outdated 00s software
on
Java Is So 90s
·
· Score: 1
You probably from msdos background, as with the bad software you described you can trivially fix it with limits ("limit vmemory 256m"). But the good system should allow you to use all resources of you machine -- shouldn't limit you. With java, it is possible to, for example, pass "-Xmx400000m" to sun's java 1.5 (just tried on my box) -- first thing it does is to pre-allocate 400GB, making eclipse to use 800MB of RAM. Not a very scalable system if you need tweak memory settings for every program, I much prefer programs that take memory as they need it.
Re:The real 90s versus outdated 00s software
on
Java Is So 90s
·
· Score: 1
Okay, name ONE major Java application that you are proud of (as a Java fan). I know two: Eclipse & Netbeans. Both sucks at speed/memory footprint compared to Emacs (I agree they have other very nice features and can be useful for a lot of things). Both easily run out of memory when compile a big project. Name one Java implementation where I can have a vector with a billion of elements. Name one serious game written in Java (I like BookWorm -- Comodore64 class game, I tried Quake2 -- too bad didn't work for me on Linux). Name one compiler written in Java that is competitive with GCC... Yes, the language itself is nice, it's just impossible to implement it efficiently.
And I do not agree that sizeof(Object) should not bother developers: actually it matters a lot in many projects. If you don't need locking, don't need virtual tables, don't need garbage collection -- you don't want to pay for all this stuff.
With C (and to some extent with C++) I can predict cache footprint of a function, can ask questions about page misses, memory locality, can rewrite code to use SSE2 intinsics, etc. With Java such questions have no meaning as you have a very little control over it!
And about Google searches -- not surprisingly they write software in C/C++ and don't run on HPUX (I think).
Re:The real 90s versus outdated 00s software
on
Java Is So 90s
·
· Score: 2, Interesting
Oh, come on! What you say may be true on an average desktop with the usual 2% cpu load, but if you have all your CPUs 100% busy where do you get any time for "background"? Next, ever tried any Java IDE? NetBeans,Eclipse -- try that against Xemacs! It's only 100 times more responsive and doesn't crash with "NULL pointer exception" while collecting compilation output. The problem with Java is that you pay A LOT for stuff you don't use. How may bytes is Object on 64 bit platform? What is the price of Object allocation vs C-style allocation on stack? And it's not going away with a better compiler, because language itself guarantees certain things (that are expensive). Another problem is the memory model of Java. The C program can use as much virtual memory as it is allowed (i.e. in most cases the it is a priori bounded by the virtual memory size), on Java, if I understand correctly, you should set beforehand how much you will allow it to use (and the one I tried didn't allow for more than 2GB on 32 bit linux). Can I set the limit to 1TB on 64bit? In C program it make perfect sense as *I* control memory, on Java I'll need to do tricks to clean the memory or it'll eat everything.
Also, please disclose which airlines use Java for realtime purposes!
I didn't follow all the whole event, but how antivirus companies reacted to this? Was there one that prevented infection? I guess the rest should find some excuses for betraying their customers?
I have two identical high-end dual cpu desktops, both with HT enabled sitting on my desk. One runs win-xp, the other a 64 bit Linux. The thing I observe every day is how windows scheduler sucks. I don't know for how long marketing dept. of MSFT knows about HT, but their OS definitely doesn't know about it yet (start update in subversion or compilation in VC -- go to drink some coffee, as computer is unusable). On Linux, on the other hand, HT really improves both responsiveness and throughput. I'm waiting to test quad- dual-core box with HT enabled;)
I agree it's very useful. Callgrind && kcachegrind usually give a lot of insight into the code. I wonder if callgrind can be hacked to produce nice seq. UML diagrams (it knows when objects are created and what messages are sent -- should be a very nice addition to the analysis).
it would be interesting to see what is the overhead of the "Object" for different JDKs on 32/64 bits implementation? If class {int x,y;} takes 128 bytes, why would I care about speed of the memory allocation? They also mention thread specific allocators which are nice, but only if your program is nowhere close to max memory.
One thing people seems to forget is mmap(). When you can mmap your 1TB drive into the address space you can do many things much cleaner/more efficient. For example, GNU grep supports "--mmap" flag that sometimes produces a significant performance boost.
Second, 64bit CPU, hopefully, means death of 32bit filesizes because of lazy programmers. 2^64 bytes should be enough for everybody! It's also good that in 64 bit mode AMD/Intel are not slower -- that would be a major roadblock in their acceptance (though Nocona was slower than Northwood, yet people ate that)
A few years ago I compared several OSes to see how they perform vs Linux on my computer: I tried Solaris (8 iirc) and FreeBSD 3.x. FreeBSD looked okay, until I tried a simple program that first mmap() a file much bigger than the RAM and then qsort() it. Solaris accomplished it and system was responsive, Linux did the same, but disk I/O made system quite irresponsive, as for FreeBSD, well, it... rebooted! Of course, I expected more from a system that was proud about its MM design, and uninstalled it after checking that it's a know bug. I think it was fixed in 4.x, but I lost much of my interest.
I'm also quite suspicious about ports collections: every time I tried to install/compile something big (like xemacs, KDE...) it failed for me: with messages like "too many open files" after a night of compilation, I don't understand how people may like such a system. I also wonder what happens if you're forced to use not-the-latest distro? I heard that it can be quite a big pain (as with perl modules, many people assume that you always run yesterday's pre-alpha pre-release of their wonderful program.)
From my personal experience, I agree that it is true on SPARC (~20-30% slowdown). However, it seems to be more complicated on x86_64. I have access only to Intel's processors and most of the programs I deal with are _not_ slower than 32 bit analogs. In many instances you can clearly see they take twice as much memory, but run at least as fast as 32 bit analog (the example I talk is a pointer-heavy Union-Find implementation). Is it memory prefetch, double number of registers or something else? Hard to say, but it (just) works.
- why don't you order a 8-way Opteron? Each CPU supports 4 DDR modules and 2GB ones are available (probably $1K/piece). This gives you 8*4*2=64GB -The largest SATA drive on Pricewatch is ~400GB. You need 160 of them to reach your limit, but again, I don't see why it's not possible... Of course, you more likely to call Sun/IBM with such an order.
First of all, according to man page, "ll" is a printf modifier for "long long". So the fact it works for int64_t is a simple coincidence that int64_t is defined as "long long int". If "long long" were 128 bit, your program would work incorrectly. I completely agree that if you _know_ how many bits you need and rely on that, you should use explicit types like int64_t; however there are situations where you prefer int/long: for example you have atol() but no atoint64(), fseek()'s offset is "long", etc. (yes, I know, there is fsetpos()), etc.
I actually have a theory for Intel's decision about sizeof(long) on win-x86_64. There is a compiler flag "-mcmodel", which when set to "small" puts your app/data to the lower 2GB. So I think one company asked another to have a way to break as little stuff as possible to make old code at least to compile. I don't know what they do with pointers, but probably there is a trick (like "short pointers" mode). Of course, I hope it's NOT true and there is a more valid reason to make type sizes different from the rest of the 64 bit world (linux, IA64(?), sparc64, dec, etc.)
isn't the header an iso C99 feature? Which flags you specify for printf/scanf? Finally, if you write a portable program, can you rely that int64_t available?
With kernel 2.6.11 I had no problem malloc()'ing 2^47 bytes (128TB) ! Memory overcommitting is on, of course. While it seems like an unneeded feature now, remember that W$ limitation means you cannot mmap() stuff >16TB, and this will be a painful limitation in a year or two (1TB IDE disks will soon be launched, I heard).
In addition, I was _really_ surprised to see that Intel's compiler still keeps "long" to 4 bytes on windows (didn't check, but so says their doc). With NO standard integer type for 64 bit, programming is set to be no fun on x86_64 under windows.
Oh, sweet memories. My board was quite stable and definitely passed BurnP6 and kernel compilation test. With unofficial BIOS it supports io-apic and some basic overclocking. Maybe it's not the speediest board, but it gave me 0 trouble so far (unlike some other, like LeadTek with their damn capacitors). I also discovered I could plug 1GB of ECC SDRAM in it - try to do it with other cheap kind of MB (I guess credits here are due to SiS chipset). Overall, it was nice and cheap board at the time.
my router logs all in/out connections and keeps bandwith utilization statistics. Last morning it informed me there is a new firmware update (so it called home). It is also capable to establish VPN tunnels via IPsec so it can send anything it likes without any possibility to examine content. Does it spy on me? Who knows..., but I started to think about installing a normal Linux box instead.
Given that fdopen() isn't working on fd > 255 (at least on Sol8), it means any program/library that may encounter many file descriptors cannot use stdio. So how do you do thread safe I/O? How do you simulate fflush, vfprintf, etc? Write your own libc?
Note that I talked about FILE, not fd (there is no problem with file descriptors). On the other hand, FILE is a struct defined in the following way on Solaris 8 (cut from 'gcc -E' for something that includes stdio.h)
The _file here has type "unsigned char". That's it, at most 256 files open at the same time. And I guess backward compatibility prevents them from changing it to int. But think about it: I work on a box with 16GB memory and 8 Sparc CPUs, compile purely 64bits code and CANNOT have more than 256 FILEs! Why these limitations from the dawn of computer era are still there?
Sorry, can't comment about other tools you mentioned, don't you think/proc & lsof & valgrind provide comparable functionality?
You're completely right, the user experience is terrible on Solaris. A lot of their tools have bugs (sad experience of putting 80GB of data on a tape - the only tar that worked accross Sparc/OSF/Linux, was, of course, GNU tar from my Debian distro). I also don't understand their attitude towards 64bits: they hype it left and right, but when you take a look -- most of their tools I _needed_ to support large files didn't do it out of the box (tar, gzip, sh, etc.). Even more: why awk cannot handle more than 512 fields? Why regex crashes inside libc on patterns ~40KB? Why can't you have more than 255 FILE* opened ? I really cannot think of any SUN's software that I would say, "oh, cool!"; any Linux distro provides much better environment. People say kernel is more robust, I cannot comment on that: indeed it crashes not very frequently, but so do our Dells (300 days uptime so far). In addition we have 4 times more Dells each 4 times faster than Sparc.
That's very interesting how SUNW will proceed: they still have tons of money and quality people, but what are they all doing?
why not make knoppix work on DVD+/-RW ? They are so cheap, fast and capacity is quite adequate. with a little help from hardware, number of rewrites can be reasonably high.
Recently I realized just how easy it is to extend Knoppix if you're using DVD. First of all, you can burn Knoppix ISO on DVD and get a bootable system. It makes sense because DVD are generally faster than CD and system will use only 1/7 of the space so seek times are much lower. (Just for completness: to burn ISO on DVD+R/W, you just say "growisofs -Z/dev/dvd=/cdimage/knoppix.iso", et voila, it's done; may also work with "-R/W", but I'm not sure). Another nice feature of growisofs is that it can write the next session: "growisofs -M/dev/dvd -r/usr/local/games" and all your games can be found in the root catalog (i'm sure mkisofs has an option to put it elsewhere) So, you start with standard knoppix and add about 3.8GB of your favorite software: no need to repackage and create compressed isofs.
BTW, it also probably means that a malicious application can in theory alter Knoppix if it's on CDR/W, esp. if mkisofs and growisofs are included.
Slashdot Mirror
To me it looks like the current DVD drives reached the reasonable limits at DVD+/-R and RW. The write speed is only x2 better than that of x4 drive, so not much reason to upgrade. The DL is unlikely to matter, as media is very expensive and higher capacity drives are on the way. However, what happened to the MtRainer/EasyWrite support? None of the drives still support it? Looks like NEC 4550 does support DVD-RAM, but the drive is still not available in the US and it's not clear how it will compare to DVD+MRW in terms of usability. For example, it would be cool to boot from Knoppix, customize it a bit (small things like wireless keys, printers, proxies, NIS, etc.) and keep configuration right on the same media.
It's amazing that .Net is even mentionned. Product of the company that couldn't deliver _anything_ technically excellent is compared to something which design was at least given a lot of thought, something that was implemented by several vendors and ported to several platforms, received a lot of feedback and made a few release cycles. As much as I don't think Java is a great thing, I simply wouldn't even consider .Net as a viable alternative. Microsoft spent no time to improve quality of their compilers (I have some acquitance with their c++ "98% compliant" implmentations), but spends a lot of time adding "features" with unclear semantics, even though the plain C++ or Java has a lot of dark corners when it comes to,for example, multithreading. I don't give a damn about any of their fantastic extensions, first show me the semantics and prove it allows to write correct programs!
You probably from msdos background, as with the bad software you described you can trivially fix it with limits ("limit vmemory 256m"). But the good system should allow you to use all resources of you machine -- shouldn't limit you. With java, it is possible to, for example, pass "-Xmx400000m" to sun's java 1.5 (just tried on my box) -- first thing it does is to pre-allocate 400GB, making eclipse to use 800MB of RAM. Not a very scalable system if you need tweak memory settings for every program, I much prefer programs that take memory as they need it.
Okay, name ONE major Java application that you are proud of (as a Java fan). I know two: Eclipse & Netbeans. Both sucks at speed/memory footprint compared to Emacs (I agree they have other very nice features and can be useful for a lot of things). Both easily run out of memory when compile a big project. Name one Java implementation where I can have a vector with a billion of elements. Name one serious game written in Java (I like BookWorm -- Comodore64 class game, I tried Quake2 -- too bad didn't work for me on Linux). Name one compiler written in Java that is competitive with GCC...
Yes, the language itself is nice, it's just impossible to implement it efficiently.
And I do not agree that sizeof(Object) should not bother developers: actually it matters a lot in many projects. If you don't need locking, don't need virtual tables, don't need garbage collection -- you don't want to pay for all this stuff.
With C (and to some extent with C++) I can predict cache footprint of a function, can ask questions about page misses, memory locality, can rewrite code to use SSE2 intinsics, etc. With Java such questions have no meaning as you have a very little control over it!
And about Google searches -- not surprisingly they write software in C/C++ and don't run on HPUX (I think).
Oh, come on! What you say may be true on an average desktop with the usual 2% cpu load, but if you have all your CPUs 100% busy where do you get any time for "background"? Next, ever tried any Java IDE? NetBeans,Eclipse -- try that against Xemacs! It's only 100 times more responsive and doesn't crash with "NULL pointer exception" while collecting compilation output. The problem with Java is that you pay A LOT for stuff you don't use. How may bytes is Object on 64 bit platform? What is the price of Object allocation vs C-style allocation on stack? And it's not going away with a better compiler, because language itself guarantees certain things (that are expensive). Another problem is the memory model of Java. The C program can use as much virtual memory as it is allowed (i.e. in most cases the it is a priori bounded by the virtual memory size), on Java, if I understand correctly, you should set beforehand how much you will allow it to use (and the one I tried didn't allow for more than 2GB on 32 bit linux). Can I set the limit to 1TB on 64bit? In C program it make perfect sense as *I* control memory, on Java I'll need to do tricks to clean the memory or it'll eat everything.
Also, please disclose which airlines use Java for realtime purposes!
I didn't follow all the whole event, but how antivirus companies reacted to this? Was there one that prevented infection? I guess the rest should find some excuses for betraying their customers?
I have two identical high-end dual cpu desktops, both with HT enabled sitting on my desk. One runs win-xp, the other a 64 bit Linux. The thing I observe every day is how windows scheduler sucks. I don't know for how long marketing dept. of MSFT knows about HT, but their OS definitely doesn't know about it yet (start update in subversion or compilation in VC -- go to drink some coffee, as computer is unusable). On Linux, on the other hand, HT really improves both responsiveness and throughput. I'm waiting to test quad- dual-core box with HT enabled ;)
I agree it's very useful. Callgrind && kcachegrind usually give a lot of insight into the code. I wonder if callgrind can be hacked to produce nice seq. UML diagrams (it knows when objects are created and what messages are sent -- should be a very nice addition to the analysis).
it would be interesting to see what is the overhead of the "Object" for different JDKs on 32/64 bits implementation? If class {int x,y;} takes 128 bytes, why would I care about speed of the memory allocation? They also mention thread specific allocators which are nice, but only if your program is nowhere close to max memory.
One thing people seems to forget is mmap(). When you can mmap your 1TB drive into the address space you can do many things much cleaner/more efficient. For example, GNU grep supports "--mmap" flag that sometimes produces a significant performance boost.
Second, 64bit CPU, hopefully, means death of 32bit filesizes because of lazy programmers. 2^64 bytes should be enough for everybody! It's also good that in 64 bit mode AMD/Intel are not slower -- that would be a major roadblock in their acceptance (though Nocona was slower than Northwood, yet people ate that)
I'm also quite suspicious about ports collections: every time I tried to install/compile something big (like xemacs, KDE...) it failed for me: with messages like "too many open files" after a night of compilation, I don't understand how people may like such a system. I also wonder what happens if you're forced to use not-the-latest distro? I heard that it can be quite a big pain (as with perl modules, many people assume that you always run yesterday's pre-alpha pre-release of their wonderful program.)
From my personal experience, I agree that it is true on SPARC (~20-30% slowdown). However, it seems to be more complicated on x86_64. I have access only to Intel's processors and most of the programs I deal with are _not_ slower than 32 bit analogs. In many instances you can clearly see they take twice as much memory, but run at least as fast as 32 bit analog (the example I talk is a pointer-heavy Union-Find implementation). Is it memory prefetch, double number of registers or something else? Hard to say, but it (just) works.
- why don't you order a 8-way Opteron? Each CPU supports 4 DDR modules and 2GB ones are available (probably $1K/piece). This gives you 8*4*2=64GB ...
-The largest SATA drive on Pricewatch is ~400GB. You need 160 of them to reach your limit, but again, I don't see why it's not possible
Of course, you more likely to call Sun/IBM with such an order.
First of all, according to man page, "ll" is a printf modifier for "long long". So the fact it works for int64_t is a simple coincidence that int64_t is defined as "long long int". If "long long" were 128 bit, your program would work incorrectly. I completely agree that if you _know_ how many bits you need and rely on that, you should use explicit types like int64_t; however there are situations where you prefer int/long: for example you have atol() but no atoint64(), fseek()'s offset is "long", etc. (yes, I know, there is fsetpos()), etc.
I actually have a theory for Intel's decision about sizeof(long) on win-x86_64. There is a compiler flag "-mcmodel", which when set to "small" puts your app/data to the lower 2GB. So I think one company asked another to have a way to break as little stuff as possible to make old code at least to compile. I don't know what they do with pointers, but probably there is a trick (like "short pointers" mode). Of course, I hope it's NOT true and there is a more valid reason to make type sizes different from the rest of the 64 bit world (linux, IA64(?), sparc64, dec, etc.)
isn't the header an iso C99 feature? Which flags you specify for printf/scanf? Finally, if you write a portable program, can you rely that int64_t available?
With kernel 2.6.11 I had no problem malloc()'ing 2^47 bytes (128TB) ! Memory overcommitting is on, of course. While it seems like an unneeded feature now, remember that W$ limitation means you cannot mmap() stuff >16TB, and this will be a painful limitation in a year or two (1TB IDE disks will soon be launched, I heard).
In addition, I was _really_ surprised to see that Intel's compiler still keeps "long" to 4 bytes on windows (didn't check, but so says their doc). With NO standard integer type for 64 bit, programming is set to be no fun on x86_64 under windows.
Oh, sweet memories. My board was quite stable and definitely passed BurnP6 and kernel compilation test. With unofficial BIOS it supports io-apic and some basic overclocking. Maybe it's not the speediest board, but it gave me 0 trouble so far (unlike some other, like LeadTek with their damn capacitors). I also discovered I could plug 1GB of ECC SDRAM in it - try to do it with other cheap kind of MB (I guess credits here are due to SiS chipset).
Overall, it was nice and cheap board at the time.
my router logs all in/out connections and keeps bandwith utilization statistics. Last morning it informed me there is a new firmware update (so it called home). It is also capable to establish VPN tunnels via IPsec so it can send anything it likes without any possibility to examine content. Does it spy on me? Who knows..., but I started to think about installing a normal Linux box instead.
Given that fdopen() isn't working on fd > 255 (at least on Sol8), it means any program/library that may encounter many file descriptors cannot use stdio. So how do you do thread safe I/O? How do you simulate fflush, vfprintf, etc? Write your own libc?
Note that I talked about FILE, not fd (there is no problem with file descriptors).
/proc & lsof & valgrind provide comparable functionality?
On the other hand, FILE is a struct defined in the following way on Solaris 8 (cut from 'gcc -E' for something that includes stdio.h)
struct __FILE
{
ssize_t _cnt;
unsigned char *_ptr;
unsigned char *_base;
unsigned char _flag;
unsigned char _file;
unsigned __orientation:2;
unsigned __ionolock:1;
unsigned __filler:5;
};
The _file here has type "unsigned char". That's it, at most 256 files open at the same time. And I guess backward compatibility prevents them from changing it to int. But think about it: I work on a box with 16GB memory and 8 Sparc CPUs, compile purely 64bits code and CANNOT have more than 256 FILEs! Why these limitations from the dawn of computer era are still there?
Sorry, can't comment about other tools you mentioned, don't you think
You're completely right, the user experience is terrible on Solaris. A lot of their tools have bugs (sad experience of putting 80GB of data on a tape - the only tar that worked accross Sparc/OSF/Linux, was, of course, GNU tar from my Debian distro). I also don't understand their attitude towards 64bits: they hype it left and right, but when you take a look -- most of their tools I _needed_ to support large files didn't do it out of the box (tar, gzip, sh, etc.). Even more: why awk cannot handle more than 512 fields? Why regex crashes inside libc on patterns ~40KB? Why can't you have more than 255 FILE* opened ? I really cannot think of any SUN's software that I would say, "oh, cool!"; any Linux distro provides much better environment. People say kernel is more robust, I cannot comment on that: indeed it crashes not very frequently, but so do our Dells (300 days uptime so far). In addition we have 4 times more Dells each 4 times faster than Sparc.
That's very interesting how SUNW will proceed: they still have tons of money and quality people, but what are they all doing?
why not make knoppix work on DVD+/-RW ? They are so cheap, fast and capacity is quite adequate.
with a little help from hardware, number of rewrites can be reasonably high.
Recently I realized just how easy it is to extend Knoppix if you're using DVD. First of all, you can burn Knoppix ISO on DVD and get a bootable system. It makes sense because DVD are generally faster than CD and system will use only 1/7 of the space so seek times are much lower. (Just for completness: to burn ISO on DVD+R/W, you just say "growisofs -Z /dev/dvd=/cdimage/knoppix.iso", et voila, it's done; may also work with "-R/W", but I'm not sure). Another nice feature of growisofs is that it can write the next session: /dev/dvd -r /usr/local/games" and all your games can be found in the root catalog (i'm sure mkisofs has an option to put it elsewhere)
"growisofs -M
So, you start with standard knoppix and add about 3.8GB of your favorite software: no need to repackage and create compressed isofs.
BTW, it also probably means that a malicious application can in theory alter Knoppix if it's on CDR/W, esp. if mkisofs and growisofs are included.