Dude, you're calling bullshit on Willy Tarreau. It's his right not to believe me. Or maybe he misunderstood me. Or maybe he wanted more info. Anyway, there is no reason I could not be called bullshit.
It's like saying "That's all very interesting Mr Torvalds, but you don't really expect me to believe you wrote a kernel yourself do you?". Hmmm please don't compare me to Linus. Linus is a magician. I'm just an awkward assistant;-)
Was there any overlap between the address space of the two 8088s? Could you transfer data between them via any method? Yes, the address space was flipped around the 512kB limit. The second CPU booted from 7FFF:0 and had its interrupt table at 8000:0 (but I didn't use it).
The "transfer" method between the two CPU was simply the DEBUG tool with which I inserted bytes by hand, and could consult the data at will. For instance, a simple counter like this was easily observable :
I remember I expected that the second CPU would have slowed the first one down spinning like this (there was no cache), but in fact it was not very noticeable. Most probably because the CPU itself was very slow and possibly not limited by memory bandwidth which was pretty decent with 120ns RAMs on a 8 MHz machine.
Feel free to think what you want. Those are just very good memories I thought people would find interesting to share.
"Also, modding a 8088 motherboard to accept a second 8088 on the 8087 socket was definitely fun. There was no cache coherency problems at that time. You just had to invert A19 to make the second one boot at 512 kB and the bus arbiter let them work in parallel. It was really cool to have an 8088-SMP:-)"
The second 8808 would boot ROM BIOS, and start spitting up on the default monitor, copy the intrupt table to 08000, but would only have 128k to work with, the first intrrupt it hit, to load from a disk, would start both processors running on return from the INT. In fact you could just easyly hang this box with INT-20. both processors would be executing the same code at the same time. It would only be a matter of seconds before some interlock hung the machine. It would be a really fun thing to watch. But WORK? NIFMY.
I think you simply don't understand what you're talking about. Having two processors run in parallel doesn't necessarily mean to have MS-DOS-SMP! It means that you boot on the first CPU while a small wire maintains RST low on the second one. Then you start DEBUG and you put your "jmp far" at 7FFF:0 (FFFF:0 xor 2^19). You make it jump to 7000:0 where you put the code to be run by the second CPU. Having it simply make characters blink on the CGA frame buffer was terribly amazing. Can you imagine? (maybe you can't). You quit debug, you type "DIR" and while the lines scroll, you still see your character blink in the top right corner, and you know that it's your second CPU's heart which is still beating. Then you restart debug and you stop the CPU by inserting a "JMP $" (EB FE) and change the code to make it output a bip on the speaker. Then you quit, you start a stupid game, and you still hear you heart beat in the speaker (until one game reprograms the PIT to use the speaker output). You don't need to support any interrupts at all. Busy loops are quite fine.
It is THAT which is kinda cool. It's not having an SMP MS-DOS. For the same reason, this guy certainly spent hundreds of hours porting Minix to his machine. Do you think he tried to make something terribly useful ? No, of course. He just wants to build something to play with. There are people who do this for their pleasure, perhaps you just cannot get that, and I don't criticize. But had I any time for this, I would without a doubt work again with a 8088 to do such useless games. It was the only intel-family CPU which was compatible with a soldering iron and discrete TTL !
Do you catch a lot of people with this one? OR
I don't want to catch anyone. People are free not to believe me and to think I'm a liar. What's the problem. If they think so, they cannot even understand what I find cool in this amusement, nor share anything on the subject with me. No need to talk with them. On the other hand, maybe the guy on this site would have told me "hey, you cheated, it's too easy to build an SMP machine from two real CPUs, try with TTL instead!".
Do you catch more people with the Sanyo BIOS rewrite?
Same response as above.
8k certainly isnt enough room to fix the video card IO problem.
You're completely wrong on this. So much wrong that the initial bios was able to display "Disk error" if no diskette was inserted in. And there was no "problem" with the video card, it's just that the CRT controller addressed it in a funny way which finally made it easier to display characters than draw lines. The only feature it lacked was the ability to write to floppy disks. It was only a problem with games which had to write scores. But quite frankly, having the ability to start a text-mode tetris on this machine was very cool !
When I still was a teen, I used to spend full week-ends doing such nerd stuff. I wrote a PC-compatible BIOS for my Sanyo-MBC550 (eg: here: http://www.seasip.info/VintagePC/sanyo.html/), and was the happiest person of the world when I first got MS-DOS 5.0 to boot on it !
I also designed a simple microcontroller-based robot from printer parts just for fun, and I was really impressed when I saw it turn around the whole room for the first time (it could detect obstacles by sending ultrasonic pulses).
Also, modding a 8088 motherboard to accept a second 8088 on the 8087 socket was definitely fun. There was no cache coherency problems at that time. You just had to invert A19 to make the second one boot at 512 kB and the bus arbiter let them work in parallel. It was really cool to have an 8088-SMP:-)
Those were project during which the time did not exist. I can imagine that this guy spends his whole spare time on his project without noticing the night come, then the day... Sometimes I wish I still had that much spare time!
Sincere kudos to him and great respect for his work! Willy
Isn't the standard thing to do to append the source IP you (pound or whatever) see to the existing contents of the header (if the header exists) separated by commas? There should be no need for a separate header. Technically speaking, the proxy should add another X-Forwarded-For header after existing ones, and it is recommended that it merges them all by separating them with commas, though it's not strictly required. It's up to the server to read all values in all X-Forwarded-For headers and use the last one for instance.
The comma is just used in HTTP to compact multiple headers into one, and has no special meaning for this particular header. There are headers which are not allowed to be repeated multiple times (eg: Content-Length, Transfer-Encoding,...). Apache 1.3 still happily merges multiple Host headers into one, BTW;-)
1. Let various ISPs know that you're about to do this,
2. Do something to trigger a DDOS,
3. Track which machines the attacks are coming from, (basically, log the source of every packet aimed at your IP address)
4. shut down and clean every machine that is shown to be part of the DDOS
5. (profit???) #3 will not work if the source IPs are spoofed. With that large of a botnet, it's not even required that the machines are able to establish TCP connections. If we have, say, 50 millions machines with an average 128 kbps upstream, sending random packets will produce 6.4 Terabits/s at full throttle! Nobody can handle this. You can isolate an entire continent just with random packets. So I'm not sure that watching the IPs will bring anything.
The real source of the problem is microsoft selling an easy-to-use, insecure OS with too many fancy gadgets which nobody can reasonably maintain in a safe state. The single concept of an anti-virus should not even exist in the first place. It's a fix for the symptoms and not for the cause. The real fix would be to educate users into not being too much demanding for ease of use. Noone would like a car which does not need a key to start up, because it would get stolen. Why do they accept an OS which does not ask them for a correct password ?
I live in France and in France we have accentuated letters. They are present on our keyboards too, and we naturally use them when typing text. But I would hate it to have to use accents in domain names. This is plain silly. Uppercase letters do not have the accents, lower case letters have. What character should I type for a given domain name? And given that the domain names remain case insensitive, the ambiguity remains forever. Not to mention having to type characters you can't type from your keyboard, or that you can't even name or identify!
Let me repeat it, this is plain stupid. I would like the domain names to remain english 7bit-only. Yes it has limitations and it is inconvenient to many people, but those people have overcome the problem (otherwise they would not even connect to the net). Now we will enlarge the problem, and for everyone.
Plain stupid move. Sounds like marketting from registrars.
I've also been doing this on my laptop for about 2 years now. I have no use of the 64 MB video RAM on it, and I could dedicate 56 of them to the swap. While it has a very low read speed (8 MB/s), write speed is fast, and seek times are NULL. You have to remember that when you are swapping to a disk, you're very lucky if you can even reach 8 MB/s, as all the time is lost in seeks. Swapping is one of the most random patterns on disks, and disks are extremely slow at this job.
I really see a speed difference between VRAM swap and disk swap, but I try to use them very rarely anyway.
It's not a matter of not being geeks, it's a matter of buying the right tools to keep safe. I know about people who know they have lots of viruses, but they refuse to install anti-virus because their system is so much compromised that no AV software can run on their system, and they do not want to reinstall because they don't have the CD (!)
Those bastard are just LAZY.
On the opposite side, there are lots of people afraid of getting their computer compromised and who buy whatever is needed to protect it and have little or no problem. Those people know nothing about their computer, but they protect themselves and are not threats to the rest of the net.
When you don't want people to get into your house, you buy a lock and install it. Why not do it on your PC ?
Zombies identified as such should get "fined" by getting disconnected from the Net for anything other than a limited set of known sites which they may need to fix their problem (basically, only editors sites).
What if his shiny new windows gets rooted and turned into a bot ? Will he get accused of doing all the attacks himself ? They'd better install a logging appliance between his PC and the Net and let him use a more secure OS!
Second: ignore the ass above who said dump Java. Modern hotspots have made Java as fast or faster than C/C++. The guy is not up to date.
ROTFL! You made my day, really! Every person who tried to demonstrate this stupid statement to me finished by giving pointers to benchmarks computing PI in different languages, or meaningless things like this.
But when it comes to real work where you need to allocate memory every 5 instructions, it's amazing how the gap is big between both! Also, the large memory footprint considerably reduces CPU caches efficiency. A big company I know is replacing a big web app running on about 20 CPU (coded in C) with this brillant technology... More than 200 CPU, and half a terabyte of RAM. You can surely imagine that the poor CPUs caches will always thrash with such a large dataset! So maybe with 10 times the CPUs and RAM your crap is "as fast or faster than C/C++", but please stop spreading dangerous statement as above to people who are starting a project !
Six: Use AJAX wherever you can. The response time for an AJAX function is amazing and it is really not that hard to do Basic AJAX.
I discovered who you are : you are a developper who always tests his code on local network and never through the net. You need 100 microseconds to perform an HTTP request on the LAN, but 100 ms over a DSL link. IF you use AJAX to fill a table with 100 values, it will require 10 ms on your LAN (immediate response), but 10 seconds over a DSL link. Good! What a smart choice! You should use AJAX only to avoid reloading complete pages, but not wherever you can as you stated above. Also, the fact that you only considered the function's response time is a clear indication that you don't care at all about the end-user experience.
Hint: if you cannot afford to buy a modem to try your apps through the net, at least start with 2% packet loss between your server and client over the LAN, using this rule on your server (assuming you use linux), it will mimmick one of the problems people encounter through the net (though it will not add latency) :
$ iptables -I OUTPUT -p tcp --sport 80 -m random --average 2 -j DROP
And do not list to us what apps you have written, surely if people have been using them, they don't anymore.
an article I wrote last year about application scaling using load balancing may help you. It will not solve your problems but will certainly help you with the concepts, best practises and traps to avoid, which is a good starting point.
Also, what you need is to perform benchmarks frequent during all the cycle of development of your applications. Using traffic generators, you will simulate a lot of users and see how your application/database behaves. And believe me, it never breaks where you expected it to ! On the first run, it's almost always caused my too much memory usage. Then you optimize it (decrappify it in fact), then you break on concurrency (threads, processes, file descriptors, sockets,...), then you optimize it again and break on I/O in the database. Then you have to rewrite all your requests, and when you finally saturate the frontend servers with 1% of your target load, you realize that you have to rewrite everything using a faster language. But at least, you will be able to save time by starting on a few cheap servers, for the time needed to translate the code for version 2.
You talked about 60000 users. If it's a population of 60000 users, it's not much. If it's 60000 concurrent users, it's a huge load and you will have to educate yourself in network and operating system tuning, because tuning the app alone will not be enough.
The smallest hard drive you can find if you stolled into a best buy is like a 160 GB so what's the point. Except that small embedded and silent systems do not use hard disks but Flash memories. That makes a great difference in available size. For instance:
$ df Filesystem 1k-blocks Used Available Use% Mounted on rootfs 243694 187329 53849 78%/
On such systems, you find Opera more often than Firefox !
I've used this method in around 1990 when I was a kid to ensure that my little browser could not use my computer even if he knew the password. The password were stored as combinations of (letter, hit time). Of course they were not ciphered, but the principle was there. And I can say it's really effective. Nobody can type your password like you.
The first ten reactions are about how slow Java is. Wake up! It's 2007, NOT 1997. The world has changed. Don't you keep your knowledge up to date? Are you amateurs? And no, a benchmark from 2001 doesn't count too.
No of course it isn't, and that's why BEA does not mind proposing you only a few hundreds of CPUs and as many gigs of RAM to port to weblogic an application which has been running well in pure C on at least a few tens of CPUs...
This must be because they don't know java yet, not because the technology is horribly slow.
It always gets me nervous to read what people can write on those technos. It looks like they have never seen them in use in enterprise. Maybe on your PC to print "hello world" it looks as fast as C, but stop being naive and look how many kilowatts are burnt by machines running java in enterprises. Then you may reconsider your statement.
For one this will let you run X86 DOS applications on a SPARC for example.
I'd like you to point me to the support page for VMWare on SPARC... oh wait that's cause there isn't one. QEMU can't even run most applications on a SPARC.
In 1995, I used a PC emulator written in pure C which ran well on sparc. I even extended it to emulate 386/486 in real mode. I believe it was called "pcemu". It was fun by that time to be able to do this !
100% agree. There are a lot of missing features in C which require assembly nowadays :
- bit rotations => have to be coded by ORing bit shifts and masks - bit counting (FFS/FLS) => have to be cleverly coded to avoid iterations - locking => you need the lock instructions/prefixes of CPUs - prefetching => idem - memory barriers => idem
And the list is long. What do we do instead ? typically, bit operations are done by hand with 100 times slower operations and using jumps and loops while a single instruction almost always exists. Locking accesses is performed through calling functions from heavy and slow libraries such as pthreads.
It always strikes me to see code doing such a thing : pthread_mutex_lock(&counter_lock); counter++; pthread_mutex_unlock(&counter_lock);
While in assembly you would be able to do this on many platforms :
LOCK INC counter
Which is basically what is done in the mutex_lock (with added tests).
We really need some C extensions to access the lower level, but unfortunately, the trend is the other way:-(
"The fact is that for less than 10000 values, the mergesort becomes faster in his own code."
Ok, I agree on all your other points, but this is one always irritates me. Only someone very theoretical would mention that for 10000 (integer) values or less this sort is sub-optimal. Come on, how many ms would that sort really take? For less than 10000 values you could use a 286 and get it over relatively quickly.
I almost laughed all the way from my chair when I heard this in college. Same mistake. Nobody cares for values that low. It could be O(N^2) for all I care.
Are you kidding ???
Obviously you don't know what sort algorithms are use for ! Any real time system or multi-task subsystem (application or kernel) needs a scheduler to very quickly sort timers in order to wake up tasks at the right moment without scanning the whole list. For instance, in a proxy like Squid, you would need something like 10000 struct timeval to handle 10000 connections. If the algorithm takes milliseconds to sort 10000 values, your proxy will be the slowest on earth. Worse, if it takes as much time when you have only 10 or 100 connections, it is a real shame. Sorting 10000 values should take microseconds in such applications.
The same is true for kernel code. Did you ever wonder how the socket timeouts are handled ? When you have to update your timeouts at every packet, you need something very fast, clearly not an O(n^2) algorithm, nor bitfast
First, there are multiple bugs in the code itself :
1) everywhere, it is assumed that 2^16 = 65535 and not 65536. Arrays are allocated for this size too, but are used with 65536 entries. (un)fortunately for him, the space is probably allocated anyway so the program does not crash on this.
2) a similar bug in the random number generator carefully avoids dangerous values, because he generates numbers between min and max by doing a modulo on max-min, which prevents max from being produced. This is not a big deal anyway, just that it hides the bug.
3) he assumes everywhere that "long" is unsigned and "short int" too. This makes the code SegFault on Linux with negative array indexes. The fix is obvious, though.
And most of all : times are measured in optimal conditions. Indeed, this algorithm is just a radix sort with a large radix (2^16). A radix algorithm is efficient when the number of elements is high compared to the radix, because at every level, all radix entries have to be scanned for existing values. In his example, he sorts 4 million elements. His code is faster than the merge sort on 4 million entries, just as would be any radix sort. But on smaller sets, it still takes the same time per level, because it has to scan all 65536 entries even if there are only 100 numbers. The fact is that for less than 10000 values, the mergesort becomes faster in his own code.
When measuring algorithms complexity, we use the O(x) notation. The time per iteration is never considered. When the time matters (as in his example), we should consider the time per iteration (T), and the total time (t) will follow:
t = T . complexity
In complexity, his algorithm is O(N) on 16 bits, but the time per iteration is very high. Other sorting algorithms are O(N.log(N)). So the total time is :
t1 = T1 . O(N) for his algorithm, and : t2 = T2 . O(N.log(N)) for the mergesort.
But when T1 > T2.log(N), he looses, which is almost always because log(N) is small for 16 bits, and T2 too because operations are very simple. In his case, T1 is very high because it represents the time required to scan 65536 values.
It's amazing that he fell in such a beginner's trap. I suppose that he's still a student, which would explain the total lack of portability and newbie's bugs in the code.
Now you know that freedom according to FSF is just a matter of dictatorship. They just want to enforce their view of the world on others. Think twice before following those mad activists too closely...:-/
TFA talks about 2000 servers equipped with 10 Gbps network cards. 11.38 Pbps is 11380 Tbps or 11380000 Gbps. This means that each server has 569 network interfaces !! This is total bullshit. If they had said they had 10*2000*2 = 40 Tbps, it would have been based on more real (though irrelevant) data.
I hate it when ignorant journalists post meaningless data for public consumption.
Slashdot Mirror
Willy
The "transfer" method between the two CPU was simply the DEBUG tool with which I inserted bytes by hand, and could consult the data at will. For instance, a simple counter like this was easily observable :
7000:0000 mov cs:[100], ax
7000:0004 inc ax
7000:0005 jmp 0
I remember I expected that the second CPU would have slowed the first one down spinning like this (there was no cache), but in fact it was not very noticeable. Most probably because the CPU itself was very slow and possibly not limited by memory bandwidth which was pretty decent with 120ns RAMs on a 8 MHz machine.
Willy
I really think your blowing smoke on this one:
Feel free to think what you want. Those are just very good memories I thought
people would find interesting to share.
"Also, modding a 8088 motherboard to accept a second 8088 on the 8087 socket :-)"
was definitely fun. There was no cache coherency problems at that time. You
just had to invert A19 to make the second one boot at 512 kB and the bus arbiter
let them work in parallel. It was really cool to have an 8088-SMP
The second 8808 would boot ROM BIOS, and start spitting up on the default monitor, copy the intrupt table to 08000, but would only have 128k to work with, the first intrrupt it hit, to load from a disk, would start both processors running on return from the INT. In fact you could just easyly hang this box with INT-20. both processors would be executing the same code at the same time. It would only be a matter of seconds before some interlock hung the machine. It would be a really fun thing to watch. But WORK? NIFMY.
I think you simply don't understand what you're talking about. Having two processors
run in parallel doesn't necessarily mean to have MS-DOS-SMP! It means that you boot
on the first CPU while a small wire maintains RST low on the second one. Then you
start DEBUG and you put your "jmp far" at 7FFF:0 (FFFF:0 xor 2^19). You make it
jump to 7000:0 where you put the code to be run by the second CPU. Having it simply
make characters blink on the CGA frame buffer was terribly amazing. Can you imagine?
(maybe you can't). You quit debug, you type "DIR" and while the lines scroll, you
still see your character blink in the top right corner, and you know that it's
your second CPU's heart which is still beating. Then you restart debug and you stop
the CPU by inserting a "JMP $" (EB FE) and change the code to make it output a bip
on the speaker. Then you quit, you start a stupid game, and you still hear you heart
beat in the speaker (until one game reprograms the PIT to use the speaker output).
You don't need to support any interrupts at all. Busy loops are quite fine.
It is THAT which is kinda cool. It's not having an SMP MS-DOS. For the same
reason, this guy certainly spent hundreds of hours porting Minix to his machine. Do
you think he tried to make something terribly useful ? No, of course. He just wants
to build something to play with. There are people who do this for their pleasure,
perhaps you just cannot get that, and I don't criticize. But had I any time for this,
I would without a doubt work again with a 8088 to do such useless games. It was the
only intel-family CPU which was compatible with a soldering iron and discrete TTL !
Do you catch a lot of people with this one? OR
I don't want to catch anyone. People are free not to believe me and to think I'm a
liar. What's the problem. If they think so, they cannot even understand what I find
cool in this amusement, nor share anything on the subject with me. No need to talk
with them. On the other hand, maybe the guy on this site would have told me "hey,
you cheated, it's too easy to build an SMP machine from two real CPUs, try with TTL
instead!".
Do you catch more people with the Sanyo BIOS rewrite?
Same response as above.
8k certainly isnt enough room to fix the video card IO problem.
You're completely wrong on this. So much wrong that the initial bios was able to
display "Disk error" if no diskette was inserted in. And there was no "problem"
with the video card, it's just that the CRT controller addressed it in a funny
way which finally made it easier to display characters than draw lines. The only
feature it lacked was the ability to write to floppy disks. It was only a problem
with games which had to write scores. But quite frankly, having the ability to
start a text-mode tetris on this machine was very cool !
BTW, conc
I can imagine the pleasure he got doing this.
:-)
When I still was a teen, I used to spend full week-ends doing such nerd stuff.
I wrote a PC-compatible BIOS for my Sanyo-MBC550 (eg: here: http://www.seasip.info/VintagePC/sanyo.html/),
and was the happiest person of the world when I first got MS-DOS 5.0 to boot on it !
I also designed a simple microcontroller-based robot from printer parts
just for fun, and I was really impressed when I saw it turn around the
whole room for the first time (it could detect obstacles by sending
ultrasonic pulses).
Also, modding a 8088 motherboard to accept a second 8088 on the 8087 socket
was definitely fun. There was no cache coherency problems at that time. You
just had to invert A19 to make the second one boot at 512 kB and the bus arbiter
let them work in parallel. It was really cool to have an 8088-SMP
Those were project during which the time did not exist. I can imagine that this
guy spends his whole spare time on his project without noticing the night come,
then the day... Sometimes I wish I still had that much spare time!
Sincere kudos to him and great respect for his work!
Willy
The comma is just used in HTTP to compact multiple headers into one, and has no special meaning for this particular header. There are headers which are not allowed to be repeated multiple times (eg: Content-Length, Transfer-Encoding,
Willy
2. Do something to trigger a DDOS,
3. Track which machines the attacks are coming from, (basically, log the source of every packet aimed at your IP address)
4. shut down and clean every machine that is shown to be part of the DDOS
5. (profit???) #3 will not work if the source IPs are spoofed. With that large of a botnet, it's not even required that the machines are able to establish TCP connections. If we have, say, 50 millions machines with an average 128 kbps upstream, sending random packets will produce 6.4 Terabits/s at full throttle! Nobody can handle this. You can isolate an entire continent just with random packets. So I'm not sure that watching the IPs will bring anything.
The real source of the problem is microsoft selling an easy-to-use, insecure OS with too many fancy gadgets which nobody can reasonably maintain in a safe state. The single concept of an anti-virus should not even exist in the first place. It's a fix for the symptoms and not for the cause. The real fix would be to educate users into not being too much demanding for ease of use. Noone would like a car which does not need a key to start up, because it would get stolen. Why do they accept an OS which does not ask them for a correct password ?
I live in France and in France we have accentuated letters. They are
present on our keyboards too, and we naturally use them when typing
text. But I would hate it to have to use accents in domain
names. This is plain silly. Uppercase letters do not have the accents,
lower case letters have. What character should I type for a given
domain name? And given that the domain names remain case insensitive,
the ambiguity remains forever. Not to mention having to type characters
you can't type from your keyboard, or that you can't even name or
identify!
Let me repeat it, this is plain stupid. I would like the domain names
to remain english 7bit-only. Yes it has limitations and it is inconvenient
to many people, but those people have overcome the problem (otherwise they
would not even connect to the net). Now we will enlarge the problem, and
for everyone.
Plain stupid move. Sounds like marketting from registrars.
Willy
Hi,
I've also been doing this on my laptop for about 2 years now. I have
no use of the 64 MB video RAM on it, and I could dedicate 56 of them
to the swap. While it has a very low read speed (8 MB/s), write speed
is fast, and seek times are NULL. You have to remember that when you
are swapping to a disk, you're very lucky if you can even reach 8 MB/s,
as all the time is lost in seeks. Swapping is one of the most random
patterns on disks, and disks are extremely slow at this job.
I really see a speed difference between VRAM swap and disk swap, but
I try to use them very rarely anyway.
Willy
It's not a matter of not being geeks, it's a matter of buying the right tools to keep safe. I know about people who know they have lots of viruses, but they refuse to install anti-virus because their system is so much compromised that no AV software can run on their system, and they do not want to reinstall because they don't have the CD (!)
Those bastard are just LAZY.
On the opposite side, there are lots of people afraid of getting their computer compromised and who buy whatever is needed to protect it and have little or no problem. Those people know nothing about their computer, but they protect themselves and are not threats to the rest of the net.
When you don't want people to get into your house, you buy a lock and install it. Why not do it on your PC ?
Zombies identified as such should get "fined" by getting disconnected from the Net for anything other than a limited set of known sites which they may need to fix their problem (basically, only editors sites).
Willy
What if his shiny new windows gets rooted and turned into a bot ? Will he get accused of doing all the attacks himself ? They'd better install a logging appliance between his PC and the Net and let him use a more secure OS!
Second: ignore the ass above who said dump Java. Modern hotspots have made Java as fast or faster than C/C++. The guy is not up to date.
ROTFL! You made my day, really! Every person who tried to demonstrate this stupid statement to me finished by giving pointers to benchmarks computing PI in different languages, or meaningless things like this.
But when it comes to real work where you need to allocate memory every 5 instructions, it's amazing how the gap is big between both! Also, the large memory footprint considerably reduces CPU caches efficiency. A big company I know is replacing a big web app running on about 20 CPU (coded in C) with this brillant technology... More than 200 CPU, and half a terabyte of RAM. You can surely imagine that the poor CPUs caches will always thrash with such a large dataset! So maybe with 10 times the CPUs and RAM your crap is "as fast or faster than C/C++", but please stop spreading dangerous statement as above to people who are starting a project !
Six: Use AJAX wherever you can. The response time for an AJAX function is amazing and it is really not that hard to do Basic AJAX.
I discovered who you are : you are a developper who always tests his code on local network and never through the net. You need 100 microseconds to perform an HTTP request on the LAN, but 100 ms over a DSL link. IF you use AJAX to fill a table with 100 values, it will require 10 ms on your LAN (immediate response), but 10 seconds over a DSL link. Good! What a smart choice! You should use AJAX only to avoid reloading complete pages, but not wherever you can
as you stated above. Also, the fact that you only considered the function's response time is a clear indication that you don't care at all about the end-user experience.
Hint: if you cannot afford to buy a modem to try your apps through the net, at least start with 2% packet loss between your server and client over the LAN, using this rule on your server (assuming you use linux), it will mimmick one of the problems people encounter through the net (though it will not add latency) :
$ iptables -I OUTPUT -p tcp --sport 80 -m random --average 2 -j DROP
And do not list to us what apps you have written, surely if people have been using them, they don't anymore.
Hi,
l ications%20scalable%20with%20LB.pdf
...), then you optimize it again and break on I/O in the database. Then you have to rewrite all your requests,
an article I wrote last year about application scaling using load balancing may help you. It will not solve your problems but will certainly help you with the concepts, best practises and traps to avoid, which is a good starting point.
You can get read it online here : http://1wt.eu/articles/2006_lb/ or you can download it as a PDF here :
http://www.exceliance.fr/en/ART-2006-making%20app
Also, what you need is to perform benchmarks frequent during all the cycle of development of your applications. Using
traffic generators, you will simulate a lot of users and see how your application/database behaves. And believe me, it
never breaks where you expected it to ! On the first run, it's almost always caused my too much memory usage.
Then you optimize it (decrappify it in fact), then you break on concurrency (threads, processes, file descriptors,
sockets,
and when you finally saturate the frontend servers with 1% of your target load, you realize that you have to rewrite
everything using a faster language. But at least, you will be able to save time by starting on a few cheap servers, for
the time needed to translate the code for version 2.
You talked about 60000 users. If it's a population of 60000 users, it's not much. If it's 60000 concurrent users, it's
a huge load and you will have to educate yourself in network and operating system tuning, because tuning the app alone
will not be enough.
Good luck!
Willy
Willy
Wait, it's far more common for carefully written apps to hack IE than the reverse !
I've used this method in around 1990 when I was a kid to ensure that my little browser could
not use my computer even if he knew the password. The password were stored as combinations of
(letter, hit time). Of course they were not ciphered, but the principle was there. And I can
say it's really effective. Nobody can type your password like you.
Willy
No of course it isn't, and that's why BEA does not mind proposing you only a few hundreds of CPUs and as many gigs of RAM to port to weblogic an application which has been running well in pure C on at least a few tens of CPUs...
This must be because they don't know java yet, not because the technology is horribly slow.
It always gets me nervous to read what people can write on those technos. It looks like they have never seen them in use in enterprise. Maybe on your PC to print "hello world" it looks as fast as C, but stop being naive and look how many kilowatts are burnt by machines running java in enterprises. Then you may reconsider your statement.
Willy.
In 1995, I used a PC emulator written in pure C which ran well on sparc. I even extended it to emulate 386/486 in real mode. I believe it was called "pcemu". It was fun by that time to be able to do this !
Willy
100% agree. There are a lot of missing features in C which require assembly nowadays :
:-(
- bit rotations => have to be coded by ORing bit shifts and masks
- bit counting (FFS/FLS) => have to be cleverly coded to avoid iterations
- locking => you need the lock instructions/prefixes of CPUs
- prefetching => idem
- memory barriers => idem
And the list is long. What do we do instead ? typically, bit operations are done by hand with 100 times slower operations and using jumps and loops while a single instruction almost always exists. Locking accesses is performed through calling functions from heavy and slow libraries such as pthreads.
It always strikes me to see code doing such a thing :
pthread_mutex_lock(&counter_lock);
counter++;
pthread_mutex_unlock(&counter_lock);
While in assembly you would be able to do this on many platforms :
LOCK INC counter
Which is basically what is done in the mutex_lock (with added tests).
We really need some C extensions to access the lower level, but unfortunately,
the trend is the other way
Willy
Are you kidding ???
Obviously you don't know what sort algorithms are use for ! Any real time system or multi-task
subsystem (application or kernel) needs a scheduler to very quickly sort timers in order to wake
up tasks at the right moment without scanning the whole list. For instance, in a proxy like
Squid, you would need something like 10000 struct timeval to handle 10000
connections. If the algorithm takes milliseconds to sort 10000 values, your proxy will be
the slowest on earth. Worse, if it takes as much time when you have only 10 or 100 connections,
it is a real shame. Sorting 10000 values should take microseconds in such applications.
The same is true for kernel code. Did you ever wonder how the socket timeouts are handled ?
When you have to update your timeouts at every packet, you need something very fast,
clearly not an O(n^2) algorithm, nor bitfast
Willy
Don't use this code !
:
:-)
First, there are multiple bugs in the code itself :
1) everywhere, it is assumed that 2^16 = 65535 and not 65536. Arrays are allocated for this size too, but are used with 65536 entries. (un)fortunately for him, the space is probably allocated anyway so the program does not crash on this.
2) a similar bug in the random number generator carefully avoids dangerous values, because he generates numbers between min and max by doing a modulo on max-min , which prevents max from being produced. This is not a
big deal anyway, just that it hides the bug.
3) he assumes everywhere that "long" is unsigned and "short int" too. This makes the code SegFault on Linux with negative array indexes. The fix is obvious, though.
And most of all : times are measured in optimal conditions. Indeed, this algorithm is
just a radix sort with a large radix (2^16). A radix algorithm is efficient when the number
of elements is high compared to the radix, because at every level, all radix entries have to
be scanned for existing values. In his example, he sorts 4 million elements. His code is faster
than the merge sort on 4 million entries, just as would be any radix sort. But on smaller sets,
it still takes the same time per level, because it has to scan all 65536 entries even if there
are only 100 numbers. The fact is that for less than 10000 values, the mergesort becomes faster in his own code.
When measuring algorithms complexity, we use the O(x) notation. The time per iteration is never
considered. When the time matters (as in his example), we should consider the time per iteration (T), and the total time (t) will follow
t = T . complexity
In complexity, his algorithm is O(N) on 16 bits, but the time per iteration is very high. Other sorting algorithms are O(N.log(N)). So the total time is :
t1 = T1 . O(N) for his algorithm, and :
t2 = T2 . O(N.log(N)) for the mergesort.
But when T1 > T2.log(N), he looses, which is almost always because log(N) is small for 16 bits, and T2 too because operations are very simple. In his case, T1 is very high because it represents the time required to scan 65536 values.
It's amazing that he fell in such a beginner's trap. I suppose that he's still a student, which would explain the total lack of portability and newbie's bugs in the code.
Well tried anyway
Willy
Now you know that freedom according to FSF is just a matter of dictatorship. They just want to enforce their view of the world on others. Think twice before following those mad activists too closely... :-/
Willy
TFA talks about 2000 servers equipped with 10 Gbps network cards.
11.38 Pbps is 11380 Tbps or 11380000 Gbps. This means that each
server has 569 network interfaces !! This is total bullshit. If
they had said they had 10*2000*2 = 40 Tbps, it would have been
based on more real (though irrelevant) data.
I hate it when ignorant journalists post meaningless data for public
consumption.
Willy