But you failed to mention that for N-way SMP servers Event-driven + few threads == BEST.
If you want to utilize the performance available in a multiprocessor high performance network server you actually need to use threads, although in this case each thread is handling a set of events. 2 threads per CPU max is a nice rough measure.
This is the nasty little secret you fail to mention with the twisted python implementation - SMP will be underutilized.
SEDA is very cool, but forces a particular granularity of messaging via the queues that I fidn annoying and can cause unnecessary work when it is hard to break processing of events into discrete stages.
At any rate, threaded programming is here to stay. We just need to use it correctly (thread per connection is DEAD BEEF)
Going back to MY point.. For every 1 legitimate use of P2P there's 100 others who would use it to pirate/warez things that don't belong to them. Period! Until P2P can implement DRM, it's shark food to the MPAA and RIAA attorneys.
You are right. I am subject to two implicit assumptions:
1. despite attempts to force control over all information, strong encryption and tunnneling (SSH/OpenSSL/HTTPS/etc) will prevent total access to all information.
2. totatlitarian bootstrap methods for hardware level system verification (aka Palladium / TCPA) will fail in the market place ensuring the continued logetivity of consumer computing equipment of general purpose.
I am willing to bet my time, efforts, and meager monetary resources on such assertions. I am convinced we will win this conflict of interest concerning the flow of digital information in fully decentralized peer networks.
If you had clicked through a few of the links you would have come across BitTorrent which is currently running a widescale distribution of the latest RedHat release ISO images using a decentralized swarming distribution network.
Re:CriticalMassinpeernetworks-P2P overcomes physic
on
Cringely on P2P
·
· Score: 2
Grab a local map. Plot your position on it. Now draw a line from that point back to your ISP's connection with the internet...
You missed the point. When you combine swarming distribution, decentralized resource discovery, and strong encryption you cannot tell what is going across the wire. It is opaque data, and that is what the Microsoft group is talking about. Once you remove any centralized points of weakness for the content industries to attack and monitor you lose the ability to not only track where data is going between peers, but also to even tell what it is that they are trading.
There is no doubt it will be interesting, but you are right: the legal stick is and will continue to be the most popular stick with which the RIAA/MPAA will beat peer networks.
Another reason that fully decentralized, open source peer networking infrastructure is not only advantageous, but effectively required to sidestep the legal implications of a far profit corporation in the US where contributory and vicarious copyright infringement is a real threat (most countries have NO contributory or vicarious clauses to their copyright laws)
Critical Mass in peer networks
on
Cringely on P2P
·
· Score: 5, Informative
One thing Cringley hints at is a coming boom in popularity and capability of truly decentralized peer networks. It is the fully and highly decentralized network architectures that the Microsoft group credits most with resilience against any kind of legal, technological or political attacks.
We are starting to see some of these technologies emerge, awaiting integration into flexible infrastructure that allows fast, easy and efficient distribution of data, content or otherwise, between peers on a local and global scale.
The end result will be a combination of a number of technologies seamlessly interoperating like:
You can't possibly be thinking that you could write and maintain and DEBUG a 5 million line Python code base for gods sake. And anything in java over about 10,000 lines becomes unrunnable on any current machine.
I don't want to get into a language flameware, but I firmly believe in the mantra: "The best tool for the job"
If you wrote the code in Python, it wouldn't be 5 million lines! Put performance critical code in C or C++ compiled natively. Integrate with Python or Java via the available interfaces. Distribute processing using RMI/RPC if the system grows too big. You enhance scalability and maintainability this way.
C and C++ both have their place in the world, but Java and Python and similar tools are proving incredibly usefull for solving a wide range of computing tasks quickly and efficiently.
I am still going to be writing C++ code for a long time into the future, but that doesn't mean I think everyone should spend the significant effort required to become proficient with C++, nor do I think C++ is even a good tool for many of the coding projects out there.
I used to think this was the case. Then I tried to teach a friend of mine how pointers work. I swear, some people just cannot grasp the concept of indirection via pointers and memory addresses. It is simply pure alien to them.
To really understand pointers you need to understand pointer arithmetic, multiple indirection, stack and heap storage, function pointers and byte alignment.
All that combined adds up into some pretty complex concepts. Joel mentions VB programmers repeatedly in his article, and I can assure you that regardless of how pointers are taught, many of the VB programmers out there will simply not "get it".
In a known environment it becomes trivial. All you need to know are 1. indexes, and 2. tables sizes. There is a whole volume of information on ordering the clasues of a SQL query based on these criteria (and the type of clause itself) that will get you an optimal query without any use of profilers and crap.
Yes, a good DB programmer can look at a query, and given the indexes and table sizes, order the clauses optimally. It isnt that hard, you just need to do this for every environment that differs.
it's a criticism of C++ and C-style strings, a legacy that we are stuck with due to APIs
Exactly, which is why later in my post I mention the need to abandon the old ways and start fresh with clean abstractions unencumbered by old cruft.
I like to code in c++ myself, but there is no way I would ever recommend it to anyone these days given the presence of better tools like Java or Python, etc.
Proper abstractions avoid unintended side-effects by presenting a clean view of the intent and function of a given interface, and not just a collection of methods or structures.
When I read what Joel wrote about "leaky abstractions" i saw a peice complaining about "unintended side-effects". I don't think the problem is with abstractions themselves, but rather the implementation.
He lists some examples:
1. TCP - This is a common one. Not only does TCP itself have peculiar behavior in less than ideal conditions, but it is also interfaced with via sockets, which compound the problem with an overly complex API.
If you were to improve on this and present a clean reliable stream transport abstraction is would likely have a simple connection establishment interface and some simple read/write functionality. Errors would be propagated up to a user via exceptions or event handlers. But the point I want to make is that This problem can be solved with a cleaner abstraction.
2. SQL - This example is a straw man. The problem with SQL is not the abstraction it provides, but the complexity of dealing with unknown table sizes when you are trying to write fast generic queries. There is no way to ensure that a query runs fastest on all systems. Every system and environment is going to have different amounts and types of data. The amount of data in a table, the way it is indexed, and the relationship between records is what determines a queries speed. There will always be manual performance tweaking of truly complex SQL simply because every scenario is different and the best solution will vary.
3. C++ string classes. I think this is another straw man. Templates and pointers in C++ are hard. That is all there is too it. Most Visual Basic only coders will not be able to wrap their minds around the logic that is required to write complex c++ template code. No matter how good the abstractions get in C++, you will always have pointers, templates, and complexity. Sorry Joel, your VB coders are going to have to avoid c++ forever. There is simply no way around it. This abstraction was never meant to make things simple enough for Joe Programmer, but rather to provide an extensible, flexible tool for the programmer to use when dealing with string data. Most of the time this is simpler, sometimes it is more complex (try writing your own derived string class - there are a number of required constructors you must implement which are far from obvious) but the end result is that you have a flexible tool, not a leaky abstraction.
There are some other examples, but you see the point. I think Joel has a good idea brewing regarding abstractions, complexity, and managing dependencies and unintended side-effects, but I do not think the problem is anywhere near as clear cut as he presents. As a discipline software engineering has a horrible track record of implementing arcane and overly complex abstractions for network programming (sockets and XTI) generic programming (templates, ref counting, custom allocators) and even operating systems API's (POSIX).
Until we can leave behind all of the cruft and failed experiments of the past, start new with complete and simple abstractions that do not mask behavior, but rather recognize it and provide a mechansim to handle it gracefully, we will run into these problems.
Luckily, such problems are fixable - just write the code. If joel were right and complex abstractions were fundamentally flawed, that would be a dark picture indeed for the future of software engineering (it is only going to grow ever more complex from here kids - make no mistake about it).
Re:Other OS P2P technologies
on
Gnutella2?
·
· Score: 3, Interesting
are there other open P2P networks out there?
Yes, the ALPINE Network uses a UDP based social discovery mechanism to implement fast, effective searches with minimal bandwidth and dual NAT support.
Some of the features include:
- High concurrent connection support (over 10,000). - Adaptive configuration for enhanced accuracy and quality of responses. - True peer to peer network. No hierarchy, no central servers. - Low communication overhead (small UDP packets, no forwarding). - Module support to allow extensions to query and transport operations.
The company I work for provides ehanced telecom services (sorry, no names, just in case they don't want this information public) that has hundreds of call centers across the US and thousands of full time operators.
We use commodity linux systems on dell hardware to drive dialogic record/playback/synth on carrier T1's and our switches. We also use linux based systems for some limited text to speach operations as well.
I would put a rough estimate on the number of these systems at around 200 with 4-16 T1's per machine.
I think they have quickly come to realize that the more they persue this software the more it spreads. Remember when 2600 was forced to remove their copy of the software? It immediately sprouted up on hundreds of hundreds of sites, all listed in a number of different forums (including/.)
The DMCS provides some nice protections against liability for ISP's, but the anti-reverse engineering aspects of it relating to copyright content controls are rediculous and need to be nullified ASAP.
what about continuing the routing process over the Internet? Are Internet ARP tables generally updated fast enough for this?
There are a number of details, but the short of it is that a mobile IP stack takes care of all routing to a dedicated internet endpoint which serves as your connection to the net. This means you can actually transfer traffic for a single TCP session over multiple wireless links simultaneously. You can read more about mobile IP by searching google...
A number of assumptions are at play: The access points you're crossing among are all on the same LAN segment, they all have the same "name"/SSID, and your network switches can handle the idea of an MAC address that was coming from one port suddenly coming from another
You did not understand the description correctly. You can associate with multiple access points on different subnets / LANs, with different SSID's, using a different BSSID, and even on different channels at the same time if you use channel/ssid/wep hopping
These are not Apple Airports, but orinoco silver cards for transmit (hopping) and cisco cards for rfmon mode receive.
Nothing is instantaneous. Twenty five years ago, computers performed orbit calculations "nearly instantaneously,"...
Ok, if you want to be pedantic, it takes microseconds to switch between states required for association with an access point.
So yes, this would be plenty fast to seamlessly associate with multiple access points and use them all cocncurrently for sending mobile IP datagram packets. And more than fast enough to quickly hop between networks individually if you wanted to do so...
Anyone know how quickly 802.11 devices can connect, authenticate, route and transfer?
Nearly instantaneously. If you implement channel, bssid, ssid, and wep key hopping at the kernel level you can communicate datagram style without problems. There is no need to use DHCP, as you can intelligently assume an available address and start transmitting datagrams (for a mobile IP stack, for example) instantly.
The really cool uses will come when the integrated peer network / wireless network applications become popular an tandem with pervasive 802.11 deployment in homes and offices.
But can anyone explain why the cell-phone and other wireless vested companies won't destroy this in the same way the telcos/cable have stagnated broadband?]
The reason is that people are creating the infrastructure for themselves. We do not need to rely on the telco's to deploy DSL DSLAMS for broadband. We dont need the cable companies to install digital cable for our net access.
We can go to the store, and for a few hundred bucks outfit our homes and computers with wireless equipment that can interoperate with any of the millions of AP's out there at the click of a button.
I hope they are deathly afraid, because this is not expensive infrastructure in the hands of goliath monopolies, but emergent infrastructure arising from the collective contributions of millions of individuals buying commodity equipment.
This is a great example of how pervasive, open wireless hotspots can empower individual communication in unexpected ways.
It will not be long before this kind of saturation is common in all the metropolitan areas (previous studies have placed wireless growth at double the current deployment by 2005)
The biggest potential uses and applications are centered around peer network integration that support the style of personal, interactive communication people crave.
There are a few projects working towards this goal like the Janus Wireless Project. This will provide not just increased internet access reliability and throughput (using multiple AP's and simultaneous associations) but also tight integration with common peer network services, like file sharing, music broadcasting using a broadcast FEC transport and playlists, even Voice over IP.
This kind of infrastructure has to be built by philantropist coders, as the business model is lacking, however, this makes it all the more tuned to what users will want, and the resulting networks in full control of those who generously provide the hardware and network connectivity (such as the Personal Telco Project.
I can only begin to imagine the possible applications of a robust, open wireless network coupled with integrated peer network services and good internet connectivity. This will be one of the most interesting and innovative areas of growth in the near future.
How do you engineer a data transport protocol that prevents a DoS?
You can never eliminate it completely (anyone can still flood you off the network given enough bandwidth)
But what you can do is make it difficult. There are a few characteristics that should be implemented.
1. Make any kind of DoS attack require at least as much bandwidth on the DoS source as the bandwidth it interrupts. Meaning, the only usefull DoS is a flood, which no one can protect against anyway. In gnutella, a DoS can have a huge impact on wasted bandwidth by using very little sending bandwidth (the network effect of relaying bogus queries for example).
2. Make communication direct, so that all nodes know whom a given packet is coming from. In gnutella, packets are forwarded, and you never know if it is legitimate or not. Is this person malicious? Or are they relaying data from a peer downstream that is malicious? Having communication direct allowsd you to terminate connections to those you do not want to communicate with.
Relaying nodes wouldn't have to compute this because...
Any node which relays traffic is a fundamental weakness. I do not like overlay relay networks at all for this reason. You never know (really) where a packet originated from, and this makes it very hard to protect against certain kinds of attacks (from well connected peers injecting bogus traffic)
Well, on my DSL line we were given 10.n.n.n addresses...
I hope this was all your internal subnet. No broadband connections that I am aware of provision with internal IP's (except for a snafu at att broadband which transposed internal modem IP's with external client IP's)
In the real world everyone gets a public IP from their provider, and the NAT is usually a linksys cable modem / DSL router. In rare cases you might see linux or bsd systems used as NAT gateways.
You can work around NAT problems, but it is not exactly trivial for TCP.
in addition to LimeWire (and presumably others?) ability to drop most DoS-style packets, it's as much an arms race as the internet in general.
It is vastly different because with a small amount of DoS traffic injected into a gnutella network you can waste very large amounts of bandwidth overall. For most internet DoS attacks you need at least as much traffic to DoS as the pain imposed. I.e. it is a linear relationship, whereas DoS attacks in Gnutella are exponential. The gnutella clients have come a long way, but they will never be perfect because the protocol itself is flawed, and prevents robust resistance to DoS attacks (how can you tell if a given query packet is valid or not? You can't if it is done with any degree of stealth)
A supernode architecture, sure. And why not?
No reason at all. In fact, the super peer networks currently appear to have the optimal balance of decetralization and centralization mixed into one. The limited centralization improves query efficiency and reduces bandwidth, while the decentralized nature of ad-hoc super peer delegation allows for much greater scalability with very little dedicated infrastructure.
These will probably be the "sweet spot" topology/architecture for a number of years...
But this is a good bit different than a true mesh topology designed to optimize routes based on routing locality, etc, that you seemed to mention in the very first post. I suppose you could consider a super peer architecture as something akin to a mesh topology, but it is a stretch.
Alright, I'm really tired. So if none of this makes sense, assume it's my fault.
Peer networks are deceivingly simple at first glance, and wickedly complicated in the details. It all makes sense, but that doesn't mean it cant be confusing at times.:-)
Nat vs unNAT: Treat the NAT as an uplink that we should try to limit connections through.
What do you mean? NAT users only upload? NAT users only perform outgoing connections? There is a lot of documentation on the experiences developers have had when dealing with NAT. The majority of broadband users are behind NAT, and so using their connections effectively is critical to making a functional network.
NAT also comes into play not only for creating connections, but also maintaining connection state in the translation tables. You cannot even assume connections will stay open for arbitrary preiods of time or else you will inevitably start losing connections...
I'd wager that copying intelligence on connection ("Here's everything I know about the network around me") would endow newcomers with a good base to start off with.
That sounds like a good way to eat up bandwidth. One of the characteristics of most mesh routing protocols is that they do not lower overhead, they simply transition it from the overlay broadcast part of a protocol to the mesh maintenance layer. For example, maintaining optimal mesh topologies consumes a lot more bandwidth than the simple connect to a peer and send a query, pass it on style. So in this manner you are using much less bandwidth for queries, but much more bandwidth maintaining the mesh network over which queries are sent.
There's nothing fragile about this topology: it's a runtime dynamic mesh topology - exactly like Gnutella's now.
Gnutella is incredibly fragile. Gnutella sucks. Gnutella is incredibly easy to cripple with DoS attacks that flood queries into the network (among many other possible methods). So I do not think this is a valid comparison.
If you are relying on volatile peers to forward data through a mesh network you also need to provide robust recovery that does not consume extravagent amounts of bandwidth. Again, this is much harder than you think.
The sole difference is that groups of peers would try to actually group themselves by network-proximity
There is a lot of good info about subtle side effects of assuming Really you could think of this as ultrapeers agreeing amongst themselves as to which of them will actually route outside a group.
Then you are talking about a super node architecture, which is used in a number of peer networks and is much more efficient than broadcast & forward protocols like gnutella. But this is also a far cry from a true mesh routing protocol optimized for efficient bandwidth utilization.
Slashdot Mirror
Event-driven GOOD & Threads BAD
But you failed to mention that for N-way SMP servers Event-driven + few threads == BEST.
If you want to utilize the performance available in a multiprocessor high performance network server you actually need to use threads, although in this case each thread is handling a set of events. 2 threads per CPU max is a nice rough measure.
This is the nasty little secret you fail to mention with the twisted python implementation - SMP will be underutilized.
SEDA is very cool, but forces a particular granularity of messaging via the queues that I fidn annoying and can cause unnecessary work when it is hard to break processing of events into discrete stages.
At any rate, threaded programming is here to stay. We just need to use it correctly (thread per connection is DEAD BEEF)
Going back to MY point.. For every 1 legitimate use of P2P there's 100 others who would use it to pirate/warez things that don't belong to them. Period! Until P2P can implement DRM, it's shark food to the MPAA and RIAA attorneys.
....
You are right. I am subject to two implicit assumptions:
1. despite attempts to force control over all information, strong encryption and tunnneling (SSH/OpenSSL/HTTPS/etc) will prevent total access to all information.
2. totatlitarian bootstrap methods for hardware level system verification (aka Palladium / TCPA) will fail in the market place ensuring the continued logetivity of consumer computing equipment of general purpose.
I am willing to bet my time, efforts, and meager monetary resources on such assertions. I am convinced we will win this conflict of interest concerning the flow of digital information in fully decentralized peer networks.
But I could be wrong
If you had clicked through a few of the links you would have come across BitTorrent which is currently running a widescale distribution of the latest RedHat release ISO images using a decentralized swarming distribution network.
For a limited picture of what future decentralized peer networks can accomplish you need only use your imagination.
This is still relatively new technology with a lot of room for growth and extensible uses.
Who else do you know with a mobile wifi hotspot van ?
Grab a local map. Plot your position on it. Now draw a line from that point back to your ISP's connection with the internet ...
You missed the point. When you combine swarming distribution, decentralized resource discovery, and strong encryption you cannot tell what is going across the wire. It is opaque data, and that is what the Microsoft group is talking about. Once you remove any centralized points of weakness for the content industries to attack and monitor you lose the ability to not only track where data is going between peers, but also to even tell what it is that they are trading.
There is no doubt it will be interesting, but you are right: the legal stick is and will continue to be the most popular stick with which the RIAA/MPAA will beat peer networks.
Another reason that fully decentralized, open source peer networking infrastructure is not only advantageous, but effectively required to sidestep the legal implications of a far profit corporation in the US where contributory and vicarious copyright infringement is a real threat (most countries have NO contributory or vicarious clauses to their copyright laws)
One thing Cringley hints at is a coming boom in popularity and capability of truly decentralized peer networks. It is the fully and highly decentralized network architectures that the Microsoft group credits most with resilience against any kind of legal, technological or political attacks.
... and many others.
We are starting to see some of these technologies emerge, awaiting integration into flexible infrastructure that allows fast, easy and efficient distribution of data, content or otherwise, between peers on a local and global scale.
The end result will be a combination of a number of technologies seamlessly interoperating like:
- distributed hash tables
- decentralized search
- swarming distribution
- wireless networks
It is nice to see the word get out: You cannot control the flow of digitial information in decentralized peer networks!
You can't possibly be thinking that you could write and maintain and DEBUG a 5 million line Python code base for gods sake. And anything in java over about 10,000 lines becomes unrunnable on any current machine.
...
I don't want to get into a language flameware, but I firmly believe in the mantra: "The best tool for the job"
If you wrote the code in Python, it wouldn't be 5 million lines! Put performance critical code in C or C++ compiled natively. Integrate with Python or Java via the available interfaces. Distribute processing using RMI/RPC if the system grows too big. You enhance scalability and maintainability this way.
C and C++ both have their place in the world, but Java and Python and similar tools are proving incredibly usefull for solving a wide range of computing tasks quickly and efficiently.
I am still going to be writing C++ code for a long time into the future, but that doesn't mean I think everyone should spend the significant effort required to become proficient with C++, nor do I think C++ is even a good tool for many of the coding projects out there.
To each his own
hey're just so often poorly explained.
I used to think this was the case. Then I tried to teach a friend of mine how pointers work. I swear, some people just cannot grasp the concept of indirection via pointers and memory addresses. It is simply pure alien to them.
To really understand pointers you need to understand pointer arithmetic, multiple indirection, stack and heap storage, function pointers and byte alignment.
All that combined adds up into some pretty complex concepts. Joel mentions VB programmers repeatedly in his article, and I can assure you that regardless of how pointers are taught, many of the VB programmers out there will simply not "get it".
In a known environment it becomes trivial. All you need to know are 1. indexes, and 2. tables sizes. There is a whole volume of information on ordering the clasues of a SQL query based on these criteria (and the type of clause itself) that will get you an optimal query without any use of profilers and crap.
Yes, a good DB programmer can look at a query, and given the indexes and table sizes, order the clauses optimally. It isnt that hard, you just need to do this for every environment that differs.
it's a criticism of C++ and C-style strings, a legacy that we are stuck with due to APIs
Exactly, which is why later in my post I mention the need to abandon the old ways and start fresh with clean abstractions unencumbered by old cruft.
I like to code in c++ myself, but there is no way I would ever recommend it to anyone these days given the presence of better tools like Java or Python, etc.
Proper abstractions avoid unintended side-effects by presenting a clean view of the intent and function of a given interface, and not just a collection of methods or structures.
When I read what Joel wrote about "leaky abstractions" i saw a peice complaining about "unintended side-effects". I don't think the problem is with abstractions themselves, but rather the implementation.
He lists some examples:
1. TCP - This is a common one. Not only does TCP itself have peculiar behavior in less than ideal conditions, but it is also interfaced with via sockets, which compound the problem with an overly complex API.
If you were to improve on this and present a clean reliable stream transport abstraction is would likely have a simple connection establishment interface and some simple read/write functionality. Errors would be propagated up to a user via exceptions or event handlers. But the point I want to make is that This problem can be solved with a cleaner abstraction.
2. SQL - This example is a straw man. The problem with SQL is not the abstraction it provides, but the complexity of dealing with unknown table sizes when you are trying to write fast generic queries. There is no way to ensure that a query runs fastest on all systems. Every system and environment is going to have different amounts and types of data. The amount of data in a table, the way it is indexed, and the relationship between records is what determines a queries speed. There will always be manual performance tweaking of truly complex SQL simply because every scenario is different and the best solution will vary.
3. C++ string classes. I think this is another straw man. Templates and pointers in C++ are hard. That is all there is too it. Most Visual Basic only coders will not be able to wrap their minds around the logic that is required to write complex c++ template code. No matter how good the abstractions get in C++, you will always have pointers, templates, and complexity. Sorry Joel, your VB coders are going to have to avoid c++ forever. There is simply no way around it. This abstraction was never meant to make things simple enough for Joe Programmer, but rather to provide an extensible, flexible tool for the programmer to use when dealing with string data. Most of the time this is simpler, sometimes it is more complex (try writing your own derived string class - there are a number of required constructors you must implement which are far from obvious) but the end result is that you have a flexible tool, not a leaky abstraction.
There are some other examples, but you see the point. I think Joel has a good idea brewing regarding abstractions, complexity, and managing dependencies and unintended side-effects, but I do not think the problem is anywhere near as clear cut as he presents. As a discipline software engineering has a horrible track record of implementing arcane and overly complex abstractions for network programming (sockets and XTI) generic programming (templates, ref counting, custom allocators) and even operating systems API's (POSIX).
Until we can leave behind all of the cruft and failed experiments of the past, start new with complete and simple abstractions that do not mask behavior, but rather recognize it and provide a mechansim to handle it gracefully, we will run into these problems.
Luckily, such problems are fixable - just write the code. If joel were right and complex abstractions were fundamentally flawed, that would be a dark picture indeed for the future of software engineering (it is only going to grow ever more complex from here kids - make no mistake about it).
are there other open P2P networks out there?
Yes, the ALPINE Network uses a UDP based social discovery mechanism to implement fast, effective searches with minimal bandwidth and dual NAT support.
Some of the features include:
- High concurrent connection support (over 10,000).
- Adaptive configuration for enhanced accuracy and quality of responses.
- True peer to peer network. No hierarchy, no central servers.
- Low communication overhead (small UDP packets, no forwarding).
- Module support to allow extensions to query and transport operations.
You can read an overview of how alpine works here . There is also a frequently asked questions and plenty of developer information .
Enjoy!
The company I work for provides ehanced telecom services (sorry, no names, just in case they don't want this information public) that has hundreds of call centers across the US and thousands of full time operators.
We use commodity linux systems on dell hardware to drive dialogic record/playback/synth on carrier T1's and our switches. We also use linux based systems for some limited text to speach operations as well.
I would put a rough estimate on the number of these systems at around 200 with 4-16 T1's per machine.
I still have this old DeCSS source mirror online from way back when this fight started.
/.)
I think they have quickly come to realize that the more they persue this software the more it spreads. Remember when 2600 was forced to remove their copy of the software? It immediately sprouted up on hundreds of hundreds of sites, all listed in a number of different forums (including
The DMCS provides some nice protections against liability for ISP's, but the anti-reverse engineering aspects of it relating to copyright content controls are rediculous and need to be nullified ASAP.
what about continuing the routing process over the Internet? Are Internet ARP tables generally updated fast enough for this?
There are a number of details, but the short of it is that a mobile IP stack takes care of all routing to a dedicated internet endpoint which serves as your connection to the net. This means you can actually transfer traffic for a single TCP session over multiple wireless links simultaneously. You can read more about mobile IP by searching google...
A number of assumptions are at play: The access points you're crossing among are all on the same LAN segment, they all have the same "name"/SSID, and your network switches can handle the idea of an MAC address that was coming from one port suddenly coming from another
You did not understand the description correctly. You can associate with multiple access points on different subnets / LANs, with different SSID's, using a different BSSID, and even on different channels at the same time if you use channel/ssid/wep hopping
These are not Apple Airports, but orinoco silver cards for transmit (hopping) and cisco cards for rfmon mode receive.
Nothing is instantaneous. Twenty five years ago, computers performed orbit calculations "nearly instantaneously," ...
Ok, if you want to be pedantic, it takes microseconds to switch between states required for association with an access point.
So yes, this would be plenty fast to seamlessly associate with multiple access points and use them all cocncurrently for sending mobile IP datagram packets. And more than fast enough to quickly hop between networks individually if you wanted to do so...
Anyone know how quickly 802.11 devices can connect, authenticate, route and transfer?
Nearly instantaneously. If you implement channel, bssid, ssid, and wep key hopping at the kernel level you can communicate datagram style without problems. There is no need to use DHCP, as you can intelligently assume an available address and start transmitting datagrams (for a mobile IP stack, for example) instantly.
There are a number of other interesting wireless projects which provide some cool / usefull features to 802.11 wifi networks:
NoCat Networks which implement QoS controls on user traffic giving priority to authenticated users.
Janus Wireless is working to improve mobile IP connectivity and integrated peer network services
IRIS which was mentioned recently and is perfectly suited for integration itno wireless networks for large amounts of reliable, distributed data storage.
MIT's GRID routing project which is probably the most similar.
The really cool uses will come when the integrated peer network / wireless network applications become popular an tandem with pervasive 802.11 deployment in homes and offices.
But can anyone explain why the cell-phone and other wireless vested companies won't destroy this in the same way the telcos/cable have stagnated broadband?]
The reason is that people are creating the infrastructure for themselves. We do not need to rely on the telco's to deploy DSL DSLAMS for broadband. We dont need the cable companies to install digital cable for our net access.
We can go to the store, and for a few hundred bucks outfit our homes and computers with wireless equipment that can interoperate with any of the millions of AP's out there at the click of a button.
I hope they are deathly afraid, because this is not expensive infrastructure in the hands of goliath monopolies, but emergent infrastructure arising from the collective contributions of millions of individuals buying commodity equipment.
This changes the game entirely...
... good luck getting your packets routed beyond the gateway unless your MAC is registered.
or spoofed:
ifconfig eth0 hw ether 01:02:03:04:05:06
This is a great example of how pervasive, open wireless hotspots can empower individual communication in unexpected ways.
.
It will not be long before this kind of saturation is common in all the metropolitan areas (previous studies have placed wireless growth at double the current deployment by 2005)
The biggest potential uses and applications are centered around peer network integration that support the style of personal, interactive communication people crave.
There are a few projects working towards this goal like the Janus Wireless Project . This will provide not just increased internet access reliability and throughput (using multiple AP's and simultaneous associations) but also tight integration with common peer network services, like file sharing, music broadcasting using a broadcast FEC transport and playlists, even Voice over IP.
This kind of infrastructure has to be built by philantropist coders, as the business model is lacking, however, this makes it all the more tuned to what users will want, and the resulting networks in full control of those who generously provide the hardware and network connectivity (such as the Personal Telco Project
I can only begin to imagine the possible applications of a robust, open wireless network coupled with integrated peer network services and good internet connectivity. This will be one of the most interesting and innovative areas of growth in the near future.
How do you engineer a data transport protocol that prevents a DoS?
...
You can never eliminate it completely (anyone can still flood you off the network given enough bandwidth)
But what you can do is make it difficult. There are a few characteristics that should be implemented.
1. Make any kind of DoS attack require at least as much bandwidth on the DoS source as the bandwidth it interrupts. Meaning, the only usefull DoS is a flood, which no one can protect against anyway. In gnutella, a DoS can have a huge impact on wasted bandwidth by using very little sending bandwidth (the network effect of relaying bogus queries for example).
2. Make communication direct, so that all nodes know whom a given packet is coming from. In gnutella, packets are forwarded, and you never know if it is legitimate or not. Is this person malicious? Or are they relaying data from a peer downstream that is malicious? Having communication direct allowsd you to terminate connections to those you do not want to communicate with.
Relaying nodes wouldn't have to compute this because
Any node which relays traffic is a fundamental weakness. I do not like overlay relay networks at all for this reason. You never know (really) where a packet originated from, and this makes it very hard to protect against certain kinds of attacks (from well connected peers injecting bogus traffic)
Well, on my DSL line we were given 10.n.n.n addresses ...
:-)
I hope this was all your internal subnet. No broadband connections that I am aware of provision with internal IP's (except for a snafu at att broadband which transposed internal modem IP's with external client IP's)
In the real world everyone gets a public IP from their provider, and the NAT is usually a linksys cable modem / DSL router. In rare cases you might see linux or bsd systems used as NAT gateways.
You can work around NAT problems, but it is not exactly trivial for TCP.
in addition to LimeWire (and presumably others?) ability to drop most DoS-style packets, it's as much an arms race as the internet in general.
It is vastly different because with a small amount of DoS traffic injected into a gnutella network you can waste very large amounts of bandwidth overall. For most internet DoS attacks you need at least as much traffic to DoS as the pain imposed. I.e. it is a linear relationship, whereas DoS attacks in Gnutella are exponential. The gnutella clients have come a long way, but they will never be perfect because the protocol itself is flawed, and prevents robust resistance to DoS attacks (how can you tell if a given query packet is valid or not? You can't if it is done with any degree of stealth)
A supernode architecture, sure. And why not?
No reason at all. In fact, the super peer networks currently appear to have the optimal balance of decetralization and centralization mixed into one. The limited centralization improves query efficiency and reduces bandwidth, while the decentralized nature of ad-hoc super peer delegation allows for much greater scalability with very little dedicated infrastructure.
These will probably be the "sweet spot" topology/architecture for a number of years...
But this is a good bit different than a true mesh topology designed to optimize routes based on routing locality, etc, that you seemed to mention in the very first post. I suppose you could consider a super peer architecture as something akin to a mesh topology, but it is a stretch.
Alright, I'm really tired. So if none of this makes sense, assume it's my fault.
Peer networks are deceivingly simple at first glance, and wickedly complicated in the details. It all makes sense, but that doesn't mean it cant be confusing at times.
Nat vs unNAT: Treat the NAT as an uplink that we should try to limit connections through.
What do you mean? NAT users only upload? NAT users only perform outgoing connections? There is a lot of documentation on the experiences developers have had when dealing with NAT. The majority of broadband users are behind NAT, and so using their connections effectively is critical to making a functional network.
NAT also comes into play not only for creating connections, but also maintaining connection state in the translation tables. You cannot even assume connections will stay open for arbitrary preiods of time or else you will inevitably start losing connections...
I'd wager that copying intelligence on connection ("Here's everything I know about the network around me") would endow newcomers with a good base to start off with.
That sounds like a good way to eat up bandwidth. One of the characteristics of most mesh routing protocols is that they do not lower overhead, they simply transition it from the overlay broadcast part of a protocol to the mesh maintenance layer. For example, maintaining optimal mesh topologies consumes a lot more bandwidth than the simple connect to a peer and send a query, pass it on style. So in this manner you are using much less bandwidth for queries, but much more bandwidth maintaining the mesh network over which queries are sent.
There's nothing fragile about this topology: it's a runtime dynamic mesh topology - exactly like Gnutella's now.
Gnutella is incredibly fragile. Gnutella sucks. Gnutella is incredibly easy to cripple with DoS attacks that flood queries into the network (among many other possible methods). So I do not think this is a valid comparison.
If you are relying on volatile peers to forward data through a mesh network you also need to provide robust recovery that does not consume extravagent amounts of bandwidth. Again, this is much harder than you think.
The sole difference is that groups of peers would try to actually group themselves by network-proximity
There is a lot of good info about subtle side effects of assuming Really you could think of this as ultrapeers agreeing amongst themselves as to which of them will actually route outside a group.
Then you are talking about a super node architecture, which is used in a number of peer networks and is much more efficient than broadcast & forward protocols like gnutella. But this is also a far cry from a true mesh routing protocol optimized for efficient bandwidth utilization.