In Switzerland, almost every adult male in the country is enrolled into the militia and keeps at least one gun at home. Gun deaths are very low (but higher than some. other countries with more rigid ownership laws). Like Canada, most of the guns in Switzerland are rifles although of a military types. I don't know about permissions relating to handguns though.
In Russia, they have strict gun ownership and few persons have guns at home. This doesn't stop the Mafya from carrying weapons from handguns though AK47s though.
This is not about some little company existing only in one country that was unfortunate enough to be hit on in another.
Dowjones has several Australian offices and it markets services there.
What is dangerous though is that it sets some dodgy precedents for reporting of less savoury regimes in, for example, Zimbabwe that like to keep criticism to a minimum. The BBC could certainly be hit in court for their reporting of Mugabe's administration (in earlier times they certainly had a bureau there(. One man's reporting is another's unjustified criticism.
Eurex/Xetra supports Win and Solaris clients for the front end. However some members have clients running on a variety of plaforms (including OS/X) but as long as they connect via a Win or Solaris connection server to route the transactions, nobody is bothered.
Re:The day of a single very powerful CPU is over..
on
End In Sight For Alpha
·
· Score: 2
First there may be some confusion between multiprocessing and clusters. You can cluster anything, you just need some good software. Clustering started with the VAX architecture, a high-end CISC machine. It could run on a lot of different machines and I'm interested in what IBM are planning with their emulation of the distributed lock manager. Cross system resource locking is fundemental to all kinds of projects (such as cluster wide file systems).In any case, if HPaq stand by their commitments (to the DOD amongst others), they will get VMS up on Itanium and this means VMS clustering. As most of the hard work was done during the Alpha port, getting it up should be easy. Only performance will be a question as much of VMS is still written in VAX assembler and then 'compiled' for the Alpha. VAX assembler is pretty high level so this is feasible.
Multprocessing is different. You just need some good locking primitives and ways to keep caches coherent with the necessary connections brought out of the chip. If the architecture doesn't support multiprocessing, it is very, very hard to graft it on later.
You are lucky, my biggest client uses OpenVMS, and although HPaQ wants to move them to Itanium, it will be a while before Itanium works reliably enough under VMS (or any other system at the moment).
Linux is fine and great but we need something that works like a VMS cluster to run things. In particular, we need the distributed lock manager and a cluster wide file system. HPaQ promises OpenVMS on Itanium, but that will take time. On the other hand, they are committed to supporting VMS for another 18 years or so because of DoD contracts that they inherited.
Re:The day of a single very powerful CPU is over..
on
End In Sight For Alpha
·
· Score: 5, Informative
Um, the Alpha was ideal for that. Dick Sites, the main architect used to work for Cray and he used a lot of ideas to make Alphas work well together. You want 16 processors, it will do it and do it well.
I like the idea of your solution but any exchange arbitarily allocating a delay factor would be hung, drawn and quartered by the participants. If two orders come in at the same price, it is the first in that gets the trade.
The participants do not see the source code, but they receive enough information so they understand how a bid and an offer order can be matched and who gets priority. Fairness is an important issue here.
Try comparing like with like, compare say Star Office 5.2 with Office 97. Office 97 was great for writing letters, simple letters, however when you took it to anything significant, it was slow, it crashed and the user interface plain lied about what the output would look like.
I guess you took the usual short cut and worked with small documents like business letters rather than long documents. You might be fairer to your students by giving them prepared large documents and spreadsheets to be changed.
Office 2000 is much better and is a reasonably solid product, although it still has problems with columns and object placement (forget about its DTP pretensions, MS Publisher does this much better).
Office 2000 is a great product, unless you have to pay for it (or the extra memory/faster processors). This is why some people only upgraded Office this year, because when you have several thousand users, upgrades are not to be lightly taken. Especially if it means machine upgrades and more memory.
Open Office isn't the best, but the price is right (remember we don't see educational discounts in the real world) and for a lot of cases it can replace Microsoft Office. If this means I can junk 5 copies of Office and just keep one for special ocassions, that is already a saving.
I have Ogle running on my 500MHz P3 Dell notebook without problems. The graphics card is an ATI Mobility P, which is sloooooowwwwww....
I recompiled Ogle plus the libraries like libdvdread and dvdcss to ensure the code was P3 optimised, installed the ATI x driver from source forge (with the xvideo extensions) and the performance is as good as PowerDVD under Win98 (no jumps). The kernel is RH 2.4.18-18.8 with the low latency stuff enabled.
If only they would fix the GUI, then I would be very happy.
Actually, the journalled flat files use 2PC for synchronisation. RMS journalling allows use to build up a recovery unit consisting of a number of operations on one or more files that are located somewhere in the clustered file system. This is directly analogous to an operation on several tables of a database. It is just uch faster. We don't need a full database manageemnt system, so we don't 'pay' for functionality that we don't need.
Regrettably, this means those words Referential Integrity are a bit of a misnomer because it is up to us to ensure that our database is consistent. Luckily everything is done inside recovery units so if an operation fails, the database is rolled back to a consistent point. However we have to rely on extremely thorough testing to ensure correct operation.
Again, all of what we do could be done by any half decent relational database management system. Just a lot slower.
When the system started in 89, hardware was much slower. At the same time we only had about 15,000 contracts traded. Now the hardware is much faster, but we couldn't switch to other approaches because a) project risk and b) the market has increased to about 750,000 contracts traded per day and for each contracted traded, there may be at least 10 times as many system transactions because of automatic order book operations that may not result in trades. All order entry was initiate by individual traders when the system started, this is no more the case.
Sounds like a nice little system and I like the way you can distribute information across different data centres. This is one of our limiting factors, we need a cluster for processing and clusters are limited as regards the distance between systems.
Once a TSN has been allocated, the system will write off the info to a journal file which is split across systems. The TSNs allow all the logs to be tied together, but they are kept partitioned to reduce interaction as during processing, state changes may be written to the journal to show how the transaction is progressing. Each transaction may decompose into many internal transactions, however for us they are poretty reliable. The latency of running off system would cripple the application, so we try to keep a transaction on a single system, but with enough external state to allow recovery in case of a failure.
In our case, the intermediate systems act as transaction routers, sending a transaction to the host node currently responsible for that product type. The intermediate systems don't know about each other so they can't assign transaction numbers and they certainly can't maintain state.
Nice suggestion, we have thought about it but unfortunately, it is not useful. In essence, the transactions must be serialised. It doesn't matter where the transactions come from, each must be 'stamped' by the exchange. Can two trasactions occur simultaneously, well yes they may be presented simultaneously to two different NICs or to different cluster nodes.. The system clock resolution (100ns), may not be sufficient to differentiate between them. Transactions are routed directly to the host node currently responsible for the particular order book by the intermediate systems. Each node will allocate a transaction sequence number by grabbing an exclusive lock on the TSN resource, incrementing the value and then downgrading the lock. The lock status info goes flying around the cluster for every transaction received by any node. Outside the TSN a lot of things can and do happen in parallel but at various points, information must be synchronised, such as the cash, the contracts traded and shares needed for margining.
Any single system can be completely powered off without any loss of accepted transactions (actually we would get a processing stall during the cluster failover but that is just a matter of seconds, but the information is safe).
There were rpms for 7.x (I think x > 3) when I downloaded my 8.0 rpms yesterday. Even better there were also rpms for 8.0 against xft. This really, really looks better under Linux.
Running gas circuits around would be problematic, but ehat about using chilled water instead as an intermediary.
Many large scale a/c systems produce chilled water for cooling the air. Sometimes the chilled water gets circulated, for example for conditioning areas. This water would also provide a convenient media for dumping the surplus heat from a refrigerator. It still remains two stage, but there isn;t the air buffer between the stages.
You are quite right. I didn't go into full details with URLs as many of the/.ers may be a little allergic to the capital markets after the recent troubles.
Actually as regards down time, sometimes it can save money. This is why volatility interrupts were introduced so that sudden slides (such as those caused by the idiots mixing their prices and quantities) can be caught before the market falls too far.
After speaking with floor-traders, the measures taken by EUREX were to handle specific issues that they had with electronic trading. The end result is a system that is definitely more transparent and certainly less prone to error than an open outcry market (those screwed up pieces of paper sitting on the pit floor represent lost trades). Whether it is more efficient at reaching a fair market price is another issue which is outside the current discussion.
However, the end result means that we hope that the electronic market operates in a way that can be easily described to traders, so they trust it. The failure modes must also be well understood because if the traders (non-IT people) and their bosses don't trust the system, they will take their business elsewhere.
I take your point but first, the client isn't trusted and keeps no stateful information other than the session itself and information for browsing lists. This is a policy in the system not to trust the client.
The system uses a central arbiter to say who does what when and to record who did what when. The arbiter must be secure and therefore must be under exchange rather than member control.
The locks in this case are between independent systems, they must be. Individual systems have multiple processors, but still we want 'out of the box' for real redundancy. Even a Z series has single points of failure. Our concept had to be full independence from single points of failure so state must be exported to other boxes.
It isn't impossible to run such a system on a PC, just don't expect the same throughput. A PC may be fast but how quickly can it communicate state to other PCs? At the same time it should be rattling of disk I/Os.
The problem is in a system where a piece of information is universally relevant, say how much money I have available to trade with and although the market may be decomposed into a number of order-books, i.e., for each product. However, we still have to be certain that I can afford to buy both 'apples' and 'oranges' whilst maintaining the cash in one place. Another issue is that of the relationship between products, for example I may have a CALL and a PUT option on BMW, the option may expire at three month intervals over the next year or so. People want to make trades made up of combinations of CALLs and PUTs in a product with different expiry dates and strike prices.
This means that it isn't possible to split the option over several systems, it must match on one system in case of combination trades. If it happens to be a big day for that product (say Annual Report Time), then volume will be very high. If it is an interesting day for the economy, say election time, then whoops, there goes our performance across all products.
Now if a transaction should fail, it becomes very important (legally so) that all transactions are unwound in the order that they were made.
The distributed lock manager was rather a neat piece of technology that Digital came up with for clustering VMS. It is sufficiently neat that there is a project to try and emulate it for Linux, interestingly enough one project from IBM. It allows for five different levels of lock to be held on a resource and each lock to be associated with a value. VMS uses it extensively for their clustered file system (one of the better ones). We use it for hierarchical locking of the order books (each product, CALL/PUT and strike for options and expiry/delivery date combination). The order books are sorted in price/time priority.
I have built smaller/simpler systems for other markets using databases and PC servers, using modern techniques. However looking at the monstrosity that I started working on about 12 years ago, I can't think of radical improvements without changing the exchange regulations, particularly with regards to those pesky regulations. I guess the best would be to convert it to Linuz but run it in multiple VMs on a Z-series mainframe.
I think you misundertand, not only must the transactions be uniquely identified (this is easy), but the original ordering must be maintained (hard). Simply time stamping transactions would not be enough.
Currently the TSN is assigned through a cluster-wide 'semaphore' maintained by the distributed lock manager. However, one system at any time has the responsibility for logging the transactions (although the job can 'fail-over' to any other system. The design of the system means that every state change must be written out of the system so that if an individual system dies, the others can continue from the same point with no loss of information permitted unless a major disaster occurs.
Oh and you can forget databases as they tend to be rather slow. Recovery unit journalled ISAM files was the only way fast enough.
There may be a lot of CompSci Theory on this subject but there is very little that is relevant when you want a highly reliable system with several thousand transactions per minute.
Oh and this particular system is running the trading at CBOT, EUREX and XETRA.
When you are a bank, an exchange or something similar you want to uniquely sequence every transaction. Why, well if you sold A and used the proceeds to buy B and then sell B to buy C and one of those transactions fails, you need to unroll the following transactions.
So you need to tag every transaction with a unique sequence number. This is really, really difficult when you don't have a single system with an amazing I/O throughput to assign those numbers.
A Google type solution uses a lot of execution units each with limited I/O capability. Queries may be parallelised without much interaction. In my example, every transaction must be synchronised. It doesn't matter if the application is spread over a cluster, the nodes must still coordinate to assign the sequnce number.
I agree though with your point about adding better cluster management though to open source operating systems. However, this is much more difficult than improvements to a standalone system because how many people can afford to run a cluster of say 4 or more systems for playing around.
Disbandment isn't required, just decimation (/10).
Whereas the US doesn't want to be part of any international organisation that it can not dominate, many other western countries have no objection. This is why the EU works. Hell, there are some major rows there, but it is better that they take place in Brussels/Strasbourg than the Somme.
The orginal principle of NATO is all for one meaning that no country needs to be able to defend itself because it's partners will help. This significantly reduces military spending and allows money to be blown on other more useful things than killing people.
It is however very interesting because such forms of punishments against corporations are very rare. A much discussed point within these hallowed walls is the effect of a breakup of Microsoft. In other countries there have also been discussions about what to do to companies that have misbehaved badly (think of the cases where deaths have arisen out of corporate policy).
The fines mechanism doesn't seem adequate when a company has been involved in major wrong-doing (i.e., causing deaths) as a matter of policy, rather than as individual action. At the same time, a major corporation likes to to wave the employment it provides in front of politicians to discourage actions against them. Here I'm thinking more about companies such as Union Carbide in Bhopal, India and the operator of the English channel ferry that sank because the policy was to leave the loading doors open until the ferry was underway.
IG Farben remains relevant and it would be a useful exercise to study what worked and what didn't. Some board members were imprisoned, all others in head office lost their jobs but the orinary workers could largely continue.
I don't regard IG Farben's 'punishment' as wildly successful either but it seems to be the best example that we have.
Destroying the company was not an option. The technology and the jobs that it created were vital to the revitalisation of Germany. After the war, many persons didn't want to repeat what had happened after the Treaty of Versailles (WW I) which crippled Germany economically and created many of the problems leading to the rise of the Nazis.
Instead, the company was split and the head-office disbanded. The constituent companies continued (they had their own identities as they started as independent entities) and most people kept their jobs.
Yes, there is still a small office in Frankfurt administering the IG Farben mark, but the consitutent companies are now competitors. The IG Farben building has now passed to Frankfurt University after the US military draw down.
Actually, IG Farben was dismantled after the war. They were intimately tied with the Nazi party and profited greatly from the war and the slave labor system.
After the war, IG Farben's HQ in Frankfurt was taken over by the US Army and the company split up. The process of denazification could not be complete though because the plants were vital to the reconstruction of post-war Germany.
What is relevant is that this was the first "Corporate Death Penalty" of a major corporation for moral reasons.
SCSI is nice but you pay a lot extra for it. IDE fails more, but I suspect that a lot of this is connected with overheating. Someone who buys a SCSI drive is likely to put it into a better housing but the usual home box tends to be undercooled. I don't know how this one runs but from what people are saying, it seems to be ok.
The main problem with file loads over a network is the lack of multicast protocols for file transfer (I know there are ways, but they are experimental).
Interestingly enough, these boxes also have firewire which can be used as another form of cluster interconnect.
Slashdot Mirror
In Russia, they have strict gun ownership and few persons have guns at home. This doesn't stop the Mafya from carrying weapons from handguns though AK47s though.
Dowjones has several Australian offices and it markets services there.
What is dangerous though is that it sets some dodgy precedents for reporting of less savoury regimes in, for example, Zimbabwe that like to keep criticism to a minimum. The BBC could certainly be hit in court for their reporting of Mugabe's administration (in earlier times they certainly had a bureau there(. One man's reporting is another's unjustified criticism.
Eurex/Xetra supports Win and Solaris clients for the front end. However some members have clients running on a variety of plaforms (including OS/X) but as long as they connect via a Win or Solaris connection server to route the transactions, nobody is bothered.
Multprocessing is different. You just need some good locking primitives and ways to keep caches coherent with the necessary connections brought out of the chip. If the architecture doesn't support multiprocessing, it is very, very hard to graft it on later.
Linux is fine and great but we need something that works like a VMS cluster to run things. In particular, we need the distributed lock manager and a cluster wide file system. HPaQ promises OpenVMS on Itanium, but that will take time. On the other hand, they are committed to supporting VMS for another 18 years or so because of DoD contracts that they inherited.
Um, the Alpha was ideal for that. Dick Sites, the main architect used to work for Cray and he used a lot of ideas to make Alphas work well together. You want 16 processors, it will do it and do it well.
They ran Linux quite nicely. Many Digital engineers had input into Linux
The participants do not see the source code, but they receive enough information so they understand how a bid and an offer order can be matched and who gets priority. Fairness is an important issue here.
I guess you took the usual short cut and worked with small documents like business letters rather than long documents. You might be fairer to your students by giving them prepared large documents and spreadsheets to be changed.
Office 2000 is much better and is a reasonably solid product, although it still has problems with columns and object placement (forget about its DTP pretensions, MS Publisher does this much better).
Office 2000 is a great product, unless you have to pay for it (or the extra memory/faster processors). This is why some people only upgraded Office this year, because when you have several thousand users, upgrades are not to be lightly taken. Especially if it means machine upgrades and more memory.
Open Office isn't the best, but the price is right (remember we don't see educational discounts in the real world) and for a lot of cases it can replace Microsoft Office. If this means I can junk 5 copies of Office and just keep one for special ocassions, that is already a saving.
I recompiled Ogle plus the libraries like libdvdread and dvdcss to ensure the code was P3 optimised, installed the ATI x driver from source forge (with the xvideo extensions) and the performance is as good as PowerDVD under Win98 (no jumps). The kernel is RH 2.4.18-18.8 with the low latency stuff enabled.
If only they would fix the GUI, then I would be very happy.
Regrettably, this means those words Referential Integrity are a bit of a misnomer because it is up to us to ensure that our database is consistent. Luckily everything is done inside recovery units so if an operation fails, the database is rolled back to a consistent point. However we have to rely on extremely thorough testing to ensure correct operation.
Again, all of what we do could be done by any half decent relational database management system. Just a lot slower.
When the system started in 89, hardware was much slower. At the same time we only had about 15,000 contracts traded. Now the hardware is much faster, but we couldn't switch to other approaches because a) project risk and b) the market has increased to about 750,000 contracts traded per day and for each contracted traded, there may be at least 10 times as many system transactions because of automatic order book operations that may not result in trades. All order entry was initiate by individual traders when the system started, this is no more the case.
Once a TSN has been allocated, the system will write off the info to a journal file which is split across systems. The TSNs allow all the logs to be tied together, but they are kept partitioned to reduce interaction as during processing, state changes may be written to the journal to show how the transaction is progressing. Each transaction may decompose into many internal transactions, however for us they are poretty reliable. The latency of running off system would cripple the application, so we try to keep a transaction on a single system, but with enough external state to allow recovery in case of a failure.
In our case, the intermediate systems act as transaction routers, sending a transaction to the host node currently responsible for that product type. The intermediate systems don't know about each other so they can't assign transaction numbers and they certainly can't maintain state.
Any single system can be completely powered off without any loss of accepted transactions (actually we would get a processing stall during the cluster failover but that is just a matter of seconds, but the information is safe).
There were rpms for 7.x (I think x > 3) when I downloaded my 8.0 rpms yesterday. Even better there were also rpms for 8.0 against xft. This really, really looks better under Linux.
Many large scale a/c systems produce chilled water for cooling the air. Sometimes the chilled water gets circulated, for example for conditioning areas. This water would also provide a convenient media for dumping the surplus heat from a refrigerator. It still remains two stage, but there isn;t the air buffer between the stages.
Actually as regards down time, sometimes it can save money. This is why volatility interrupts were introduced so that sudden slides (such as those caused by the idiots mixing their prices and quantities) can be caught before the market falls too far.
After speaking with floor-traders, the measures taken by EUREX were to handle specific issues that they had with electronic trading. The end result is a system that is definitely more transparent and certainly less prone to error than an open outcry market (those screwed up pieces of paper sitting on the pit floor represent lost trades). Whether it is more efficient at reaching a fair market price is another issue which is outside the current discussion.
However, the end result means that we hope that the electronic market operates in a way that can be easily described to traders, so they trust it. The failure modes must also be well understood because if the traders (non-IT people) and their bosses don't trust the system, they will take their business elsewhere.
The system uses a central arbiter to say who does what when and to record who did what when. The arbiter must be secure and therefore must be under exchange rather than member control.
The locks in this case are between independent systems, they must be. Individual systems have multiple processors, but still we want 'out of the box' for real redundancy. Even a Z series has single points of failure. Our concept had to be full independence from single points of failure so state must be exported to other boxes.
It isn't impossible to run such a system on a PC, just don't expect the same throughput. A PC may be fast but how quickly can it communicate state to other PCs? At the same time it should be rattling of disk I/Os.
This means that it isn't possible to split the option over several systems, it must match on one system in case of combination trades. If it happens to be a big day for that product (say Annual Report Time), then volume will be very high. If it is an interesting day for the economy, say election time, then whoops, there goes our performance across all products.
Now if a transaction should fail, it becomes very important (legally so) that all transactions are unwound in the order that they were made.
The distributed lock manager was rather a neat piece of technology that Digital came up with for clustering VMS. It is sufficiently neat that there is a project to try and emulate it for Linux, interestingly enough one project from IBM. It allows for five different levels of lock to be held on a resource and each lock to be associated with a value. VMS uses it extensively for their clustered file system (one of the better ones). We use it for hierarchical locking of the order books (each product, CALL/PUT and strike for options and expiry/delivery date combination). The order books are sorted in price/time priority.
I have built smaller/simpler systems for other markets using databases and PC servers, using modern techniques. However looking at the monstrosity that I started working on about 12 years ago, I can't think of radical improvements without changing the exchange regulations, particularly with regards to those pesky regulations. I guess the best would be to convert it to Linuz but run it in multiple VMs on a Z-series mainframe.
Currently the TSN is assigned through a cluster-wide 'semaphore' maintained by the distributed lock manager. However, one system at any time has the responsibility for logging the transactions (although the job can 'fail-over' to any other system. The design of the system means that every state change must be written out of the system so that if an individual system dies, the others can continue from the same point with no loss of information permitted unless a major disaster occurs.
Oh and you can forget databases as they tend to be rather slow. Recovery unit journalled ISAM files was the only way fast enough.
There may be a lot of CompSci Theory on this subject but there is very little that is relevant when you want a highly reliable system with several thousand transactions per minute.
Oh and this particular system is running the trading at CBOT, EUREX and XETRA.
So you need to tag every transaction with a unique sequence number. This is really, really difficult when you don't have a single system with an amazing I/O throughput to assign those numbers.
A Google type solution uses a lot of execution units each with limited I/O capability. Queries may be parallelised without much interaction. In my example, every transaction must be synchronised. It doesn't matter if the application is spread over a cluster, the nodes must still coordinate to assign the sequnce number.
I agree though with your point about adding better cluster management though to open source operating systems. However, this is much more difficult than improvements to a standalone system because how many people can afford to run a cluster of say 4 or more systems for playing around.
Whereas the US doesn't want to be part of any international organisation that it can not dominate, many other western countries have no objection. This is why the EU works. Hell, there are some major rows there, but it is better that they take place in Brussels/Strasbourg than the Somme.
The orginal principle of NATO is all for one meaning that no country needs to be able to defend itself because it's partners will help. This significantly reduces military spending and allows money to be blown on other more useful things than killing people.
The fines mechanism doesn't seem adequate when a company has been involved in major wrong-doing (i.e., causing deaths) as a matter of policy, rather than as individual action. At the same time, a major corporation likes to to wave the employment it provides in front of politicians to discourage actions against them. Here I'm thinking more about companies such as Union Carbide in Bhopal, India and the operator of the English channel ferry that sank because the policy was to leave the loading doors open until the ferry was underway.
IG Farben remains relevant and it would be a useful exercise to study what worked and what didn't. Some board members were imprisoned, all others in head office lost their jobs but the orinary workers could largely continue.
I don't regard IG Farben's 'punishment' as wildly successful either but it seems to be the best example that we have.
Instead, the company was split and the head-office disbanded. The constituent companies continued (they had their own identities as they started as independent entities) and most people kept their jobs.
Yes, there is still a small office in Frankfurt administering the IG Farben mark, but the consitutent companies are now competitors. The IG Farben building has now passed to Frankfurt University after the US military draw down.
After the war, IG Farben's HQ in Frankfurt was taken over by the US Army and the company split up. The process of denazification could not be complete though because the plants were vital to the reconstruction of post-war Germany.
What is relevant is that this was the first "Corporate Death Penalty" of a major corporation for moral reasons.
The main problem with file loads over a network is the lack of multicast protocols for file transfer (I know there are ways, but they are experimental).
Interestingly enough, these boxes also have firewire which can be used as another form of cluster interconnect.