I'm one of the Squid developers and I have some experience with FreeBSD:)
FreeBSD-6 and FreeBSD-7 both rock for Squid (and my squid-2 fork, cacheboy.)
FreeBSD-7 is pretty scarily scalable when it comes to web stuff. I'm working on threading cacheboy/squid-2 over the next few months enough to take advantage of the parallelism that the FreeBSD guys have introduced into -7 and -current. I've got some test code here for fully transparent web interception caching with FreeBSD-current, and some stuff to use FreeBSD's fantastic POSIX AIO support.
Generators in different areas running at different speeds, for example.
This is why you'll notice the generator RPM and output frequency is measured in two or three decimal places. They'll slowly vary the speed of the generators to bring everything back in-phase.
Just stuff the AS numbers of the BGP anomaly detection systems into the path you're using to hijack and voila! They'll never see it.
The attack uses spoofed AS paths which include the AS numbers of the ASes in the -return path- of your hijacked traffic. It works because the default eBGP behaviour is to drop routes w/ an AS in the path that matches theirs (loop detection!)
Its not fool-proof, but you -can- reasonably selectively remove ASes from receiving the announcements.
Furthermore, if you know the topology near the network you're hijacking, you could figure out all the exit (transit) ASes, spoof those so the announcement never makes it out to the general internet and hijack the traffic near them. Dense peering relationships at multiple places around the internet == your friend in this method.
I call bullshit. The increasing amount of related-to-public-infrastructure stuff that has some access to the internet, even temporarily, makes an e-9/11 completely feasible.
I mean, hell, all it would take is one upset hacker getting an infection vector into say, the local water or electricity company/utility. People scare easily - it'd be possible to deploy something like this to scare the hell out of the populace with minimum physical impact.
People sometimes forget that you can infect a network without requiring that network to be connected to the internet 24/7; you just need some way to sneak code in behind the lines.
I'd hate to see what would happen if someone decided to attack the SCADA systems on mine sites via this method; having a small number of mines off-line could trigger an economic disaster.
Of course, there's not enough information in TFA nor in the research.
If you give more of math to boys, and they develop better at it, do you know if you've challenged them and developed them to their maximum? If you take some away and redistribute it to the girls (or across racism/cultural/religious/socioeconomic/etc) then are you still challenging -any- of them to their maximum?
I'm white, and I'm whining because I don't want to see more dumb people.
.. I'd love to benchmark a properly busy copy of Squid on an SSD. That, or some usenet software. So far people talk about the SSDs being survivors but noone's really published hard figures showing how they degrade over time.
As others have said, we don't really know the long term issues related to pulling out these sorts of genetic 'defects'.
It can be more subtle then you think - imagine being able to control for depression, but by doing so you remove all those potential periods in peoples' lives where heading towards depression causes them to completely change life direction into something more positive? Or if you can control for manic depressive behaviour, or mild forms of obsessive compulsive disorder, or heck, autism - a lot of reasonably interesting people had these sorts of conditions develop, and not only do we still not understand WHY they develop, we don't understand the impact these predispositions have to some of the very extreme "positive" developments by humanity.
To me, controlling for all of the above will probably result in standardised worker bees being developed. They may be very smart, they may have nothing wrong with them, but they may lack the strange bends which tips people to consider new and weird ideas. Of course, this stuff can't be proven (yet); its really begging for some more research into GAs..
I personally think that one of the keys to understanding what we're capable of as a species, both positive and negative, needs us to focus not on what happens when things go brilliantly right, but all of the strange conditions when things go wrong.
I suggest you start publicly mocking the government and what you believe they're doing wrong.
Just see what happens.
The trouble with America (looking in from the outside) is that American rights are being eroded whilst the populace are lead to believe nothing bad is happening. The fact that its _occuring_ is the problem, not how good or bad you think it is. You're being lead to think that its not bad. Hilarious.
China is always a fun one. I tell you what; how about you dream up a way for what, > 1 billion people to continue living and breathing. Points for logical deduction with evidence backing up your claims (logic itself isn't enough.)
Just remember, you get more money now farming gold on WoW than you do growing rice, so what do you do if you want to get ahead..
Bullshit. Its a great example of PHP, MySQL -and- caching.
Stock PHP and MySQL by themselves would be (a) useless, and (b) unable to keep up with the load. Their architecture doesn't distribute the content all over the world; trying to keep MySQL servers in-sync across the entire planet would be hilarious. Trying to convince PHP in its stock form to generate that much content would require an enormous amount of servers and some form of SQL caching layer because MySQL isn't designed for that - hence why people roll memcached in a lot of situations.
(There are places which run a very, very hacked up PHP to get stupendously high speeds out of it, but it ain't your daddy's PHP..)
It exists. Its called "validators". There are strong and weak validators. You can Vary on your validators, and thus have multiple copies of the same object but in different forms (so given a text document, you can have it in different languages, compressed/uncompressed, etc.)
Your browser will then quite happily ask the origin server (which may not be the "origin" origin) for an object and provide validators. (Last-Modified -> If-Modified-Since; ETag->If-None-Match) which the origin (or the cache which is pretending to be the origin) can check against its local copy and then return a "yes, use your local copy" or "no, don't bother."
Its all there, right now, in HTTP/1.1. I swear. People just don't have a clue how to use caching, they've been bitten by the difference between "expiry" and "revalidation", and they just turn off all hope of caching. Maybe they're scared; maybe their job is to sell bits; maybe they're just clueless about it and turning off caching fixed an obscure problem. In any case, its right there in HTTP/1.1 and you can use it any time you like.
Its not a big deal if they wanted to introduce dynamic ad type content at the edge. Oh it would be - we'd have to finish implementing ESI in a useful fashion (the current implementation in Squid-3 is not usable.) You just have to know what you're doing and be willing to learn about caching. Caching isn't evil, honest.:)
They've built a scalable solution to their problem space. People should really sit down and define their problem and solution spaces before they build things.
* it was available; because * a lot of it was put into the ground and bought by other companies; which * went busted.
Also, laying dark fibre capacity inter and intracity is way, way different to last-mile access. You have to realise that the US market is full of government-granted monopolies which make laying last-mile access not just prohibitively expensive but a political issue. Damn!
Mail servers with ORDB configured will delay accepting mail until it gets a reply from ORDB. If it can't reach ORDB (ie, it doesn't give a response) then it may delay -all- incoming mail. ORDB would have to return "OK" to all requests to keep peoples' mail happy.
Dropping an "OK" rule means mail flows fine for ORDB-poking mail servers, but requires the ORDB guys to keep doing it; there's no motivation for the site administrators to remove it.
Dropping a "SPAM" rule means admins have to figure out whats busted,a nd remove ORDB from their mail configuration.
Uhm, only in the case of financial bilateral peering agreements. Don't misunderstand the overall problem - its financial - with other issues such as "network capacity", "available upstream bandwidth on the DOCSIS cable modem infrastructure" and similar issues.
Even massive amounts of P2P between their clients, not ever leaving their network, costs them money.
Adrian (No CCIE, but I've been working with SP networks of sorts since 1997.)
there's plenty of oppertunities for local content. The trouble is that our last-mile infrastructure sucks, and there's currently no money to overhaul it to make it, well, not suck.
Our distances are huge, and our densities are horrible.
Up until recently (a couple of years ago) there was only one real option to get bandwidth to each exchange to hook up your non-Telstra (incumbent telco) DSLAM - and that was with Telstra connectivity. That cost a $LOT. Nowdays PIPE, UECOMM and others are running fibre everywhere and hopefully we'll see the models changing slightly.
I don't know how much slashdot readers know about IP networking (ok, I'll assume not a lot), but almost all ISPs in Australia backhaul all their DSL traffic as layer 2 (PPPoE -> (PPPo)L2TP -> {IP|ATM|whatever}) and aggregate it at a few|one point per city. This makes for a very inefficient traffic model when it comes to scaling.
If you're backhauling all your traffic back to one location onto an enormous virtual dial box (Cisco 10000 in some circumstances, Cisco 72xx's in others, Linux l2tpns at a few:) then it doesn't matter whether you're using international bandwidth, national bandwidth or talking to another ISP customer. At 1mbit/sec upload thats only 1000 customers uploading to another thousand customers for a gigabit ethernet port to be saturated.
Considering how much all of that equipment costs to -deliver packets to end users-, having 1000 users saturate a gigabit of connectivity when your profit margins are only $10 or $20 a month is just not economically feasable.
The reason EU can pull this off is that they have higher densities than the US, they seem happier to move to ethernet inside buildings rather than overload the old cable TV infrastructure, and there seems to be less exclusive contracts for areas like you have in the US.
Finally, its not a case of net neutrality. If the iTunes content was saturating 75%-80% of an ISPs network then they'd be -stupid- -not- to try and monetize that. iiNet are offering it for free because of marketing. The traffic isn't free (although caching traffic is nice, Akamai or not:) but its going to be a small %age of their overall traffic.
.. iiNet run the free local traffic stuff because:
* people don't leave their computers on 24/7 downloading from the free traffic zone; and * its basically free marketing for what they already had (apple content coming off their local akamai farm.)
The problem bounces between "not enough transit" and "not enough backhaul capacity between clients and the DSL aggregation point." If everyone's running P2P, even if its just between each other and people on the local internet exchange, they're still having to shovel gigabits of traffic around for very little revenue.
If clients started rushing massive amounts of traffic between other clients on the (much cheaper than transit) local peering fabric then their oversubscribed DSL delivery network starts getting taxed. That's why you don't get free local peering traffic anymore - they're still doing the accounting to know what kind of traffic you're doing, but the cost of shovelling a gigabit + of p2p traffic to a few % of their clientbase was just not economical.
Transit isn't the expensive bit anymore. Its getting the damned traffic to the end user through DSL and Cable, combined with 20 year old dialup aggregation models which Just Work for DSL aggregation in Australia.
Thats not as true. The trouble with 4-gig-in-32-bit-mode is the memory map breakdown. There's reserved space, generally above 3gig, for PCI devices and such. This covers RAM which you actually can't touch, PAE or not.
64-bit mode shuffles the memory map around substantially so the device windows are up the top of the 64 bit address space (IIRC); far out of the way of the RAM we're putting in machines now.
Its relatively hit and miss what size window you'll find in the top gig. Its as large as half a gigabyte on some motherboards, which gives you that "3.5gig available" type message you get in 32 bit mode.
Legacy hardware "handles" PAE mostly fine - the technique is called "bounce buffers". Similar to the technique used by OS developers to support ISA devices in machines with > 16mb RAM..:) Its legacy drivers and legacy software thats the problem. The OS can remap the memory space to be the lower 4 gig (whatever the first region is called, its been years since I"ve meddled in this) and then setup DMA to occur to that. The trouble is that you have to keep a region of memory in the first 32 bits available for device DMA, and shuffling data to and from this first 32 bits of space can drop performance significantly.
Oh for gods sake people. Just watch the damned episodes if you can. If you can't then email NBC and tell them why (Doesn't work under Mac. Doesn't work under Linux. Doesn't work with my browser. etc.)
If you just whinge here on slashdot and don't watch the episodes then you're not going to appear in their statistics. You -want- to appear in their statistics. Tell your friends about it. Get people to watch stuff. Whining about it not working -just right- for your situation doesn't help.
You -want- the statistics to reflect that there's interest in this service. You -want- the executives to notice that people are using it, that there's non-windows people using it, that people are actually providing constructive feedback to them. Sheesh!
Hi, I'm in Australia and I'm slightly older than you (28).
I'd suggest writing a letter to your local representative and to the local and state newspapers. Write, write, write. Explain your situation, explain how you were treated.
If you work in a corporate environment you might find that your computer build has "extra" root SSL certificate authority keys installed. the operation then is:
* intercept all SSL communications * decrypt the traffic, present a root SSL CA key, which the browser will go "yup, thats fine, I trust you" * proxy server does deep inspection and sends it off re-encrypted to the origin server.
You think this is evil but then think about this: how do you scan SSL downloads for viruses and javascript?
Slashdot Mirror
Hi!
I'm one of the Squid developers and I have some experience with FreeBSD :)
FreeBSD-6 and FreeBSD-7 both rock for Squid (and my squid-2 fork, cacheboy.)
FreeBSD-7 is pretty scarily scalable when it comes to web stuff. I'm working on threading cacheboy/squid-2 over the next few months enough to take advantage of the parallelism that the FreeBSD guys have introduced into -7 and -current. I've got some test code here for fully transparent web interception caching with FreeBSD-current, and some stuff to use FreeBSD's fantastic POSIX AIO support.
Its all lookup up, up, up from here. :)
I'm not sure what world you're in but BGP peers do not route-filter everywhere.
Generators in different areas running at different speeds, for example.
This is why you'll notice the generator RPM and output frequency is measured in two or three decimal places. They'll slowly vary the speed of the generators to bring everything back in-phase.
Just stuff the AS numbers of the BGP anomaly detection systems into the path you're using to hijack and voila! They'll never see it.
The attack uses spoofed AS paths which include the AS numbers of the ASes in the -return path- of your hijacked traffic. It works because the default eBGP behaviour is to drop routes w/ an AS in the path that matches theirs (loop detection!)
Its not fool-proof, but you -can- reasonably selectively remove ASes from receiving the announcements.
Furthermore, if you know the topology near the network you're hijacking, you could figure out all the exit (transit) ASes, spoof those so the announcement never makes it out to the general internet and hijack the traffic near them. Dense peering relationships at multiple places around the internet == your friend in this method.
Got a reference?
I call bullshit. The increasing amount of related-to-public-infrastructure stuff that has some access to the internet, even temporarily, makes an e-9/11 completely feasible.
I mean, hell, all it would take is one upset hacker getting an infection vector into say, the local water or electricity company/utility. People scare easily - it'd be possible to deploy something like this to scare the hell out of the populace with minimum physical impact.
People sometimes forget that you can infect a network without requiring that network to be connected to the internet 24/7; you just need some way to sneak code in behind the lines.
I'd hate to see what would happen if someone decided to attack the SCADA systems on mine sites via this method; having a small number of mines off-line could trigger an economic disaster.
It is all completely in the realm of possibility.
You mean, like _CISCO_ ?
(Cisco offer precisely the above btw.)
Of course, there's not enough information in TFA nor in the research.
If you give more of math to boys, and they develop better at it, do you know if you've challenged them and developed them to their maximum? If you take some away and redistribute it to the girls (or across racism/cultural/religious/socioeconomic/etc) then are you still challenging -any- of them to their maximum?
I'm white, and I'm whining because I don't want to see more dumb people.
.. I'd love to benchmark a properly busy copy of Squid on an SSD. That, or some usenet software. So far people talk about the SSDs being survivors but noone's really published hard figures showing how they degrade over time.
Weird!
Please read http://en.wikipedia.org/wiki/Permittivity and search for related material on the electromagnetic properties of space.
As others have said, we don't really know the long term issues related to pulling out these sorts of genetic 'defects'.
It can be more subtle then you think - imagine being able to control for depression, but by doing so you remove all those potential periods in peoples' lives where heading towards depression causes them to completely change life direction into something more positive? Or if you can control for manic depressive behaviour, or mild forms of obsessive compulsive disorder, or heck, autism - a lot of reasonably interesting people had these sorts of conditions develop, and not only do we still not understand WHY they develop, we don't understand the impact these predispositions have to some of the very extreme "positive" developments by humanity.
To me, controlling for all of the above will probably result in standardised worker bees being developed. They may be very smart, they may have nothing wrong with them, but they may lack the strange bends which tips people to consider new and weird ideas. Of course, this stuff can't be proven (yet); its really begging for some more research into GAs..
I personally think that one of the keys to understanding what we're capable of as a species, both positive and negative, needs us to focus not on what happens when things go brilliantly right, but all of the strange conditions when things go wrong.
I suggest you start publicly mocking the government and what you believe they're doing wrong.
Just see what happens.
The trouble with America (looking in from the outside) is that American rights are being eroded whilst the populace are lead to believe nothing bad is happening. The fact that its _occuring_ is the problem, not how good or bad you think it is. You're being lead to think that its not bad. Hilarious.
China is always a fun one. I tell you what; how about you dream up a way for what, > 1 billion people to continue living and breathing. Points for logical deduction with evidence backing up your claims (logic itself isn't enough.)
Just remember, you get more money now farming gold on WoW than you do growing rice, so what do you do if you want to get ahead..
Bullshit. Its a great example of PHP, MySQL -and- caching.
Stock PHP and MySQL by themselves would be (a) useless, and (b) unable to keep up with the load. Their architecture doesn't distribute the content all over the world; trying to keep MySQL servers in-sync across the entire planet would be hilarious. Trying to convince PHP in its stock form to generate that much content would require an enormous amount of servers and some form of SQL caching layer because MySQL isn't designed for that - hence why people roll memcached in a lot of situations.
(There are places which run a very, very hacked up PHP to get stupendously high speeds out of it, but it ain't your daddy's PHP..)
It exists. Its called "validators". There are strong and weak validators. You can Vary on your validators, and thus have multiple copies of the same object but in different forms (so given a text document, you can have it in different languages, compressed/uncompressed, etc.)
Your browser will then quite happily ask the origin server (which may not be the "origin" origin) for an object and provide validators. (Last-Modified -> If-Modified-Since; ETag->If-None-Match) which the origin (or the cache which is pretending to be the origin) can check against its local copy and then return a "yes, use your local copy" or "no, don't bother."
Its all there, right now, in HTTP/1.1. I swear. People just don't have a clue how to use caching, they've been bitten by the difference between "expiry" and "revalidation", and they just turn off all hope of caching. Maybe they're scared; maybe their job is to sell bits; maybe they're just clueless about it and turning off caching fixed an obscure problem. In any case, its right there in HTTP/1.1 and you can use it any time you like.
Adrian
(I'm a Squid developer.)
Its not a big deal if they wanted to introduce dynamic ad type content at the edge. Oh it would be - we'd have to finish implementing ESI in a useful fashion (the current implementation in Squid-3 is not usable.) You just have to know what you're doing and be willing to learn about caching. Caching isn't evil, honest. :)
They've built a scalable solution to their problem space. People should really sit down and define their problem and solution spaces before they build things.
Adrian
(I'm one of the Squid developers.)
Google has purchased dark fibre because:
* it was available; because
* a lot of it was put into the ground and bought by other companies; which
* went busted.
Also, laying dark fibre capacity inter and intracity is way, way different to last-mile access. You have to realise that the US market is full of government-granted monopolies which make laying last-mile access not just prohibitively expensive but a political issue. Damn!
Try explaining to people how far away from anything useful Perth, Western Australia is.
Mail servers with ORDB configured will delay accepting mail until it gets a reply from ORDB. If it can't reach ORDB (ie, it doesn't give a response) then it may delay -all- incoming mail. ORDB would have to return "OK" to all requests to keep peoples' mail happy.
Dropping an "OK" rule means mail flows fine for ORDB-poking mail servers, but requires the ORDB guys to keep doing it; there's no motivation for the site administrators to remove it.
Dropping a "SPAM" rule means admins have to figure out whats busted,a nd remove ORDB from their mail configuration.
Uhm, only in the case of financial bilateral peering agreements. Don't misunderstand the overall problem - its financial - with other issues such as "network capacity", "available upstream bandwidth on the DOCSIS cable modem infrastructure" and similar issues.
Even massive amounts of P2P between their clients, not ever leaving their network, costs them money.
Adrian
(No CCIE, but I've been working with SP networks of sorts since 1997.)
Thats not it at all.
:) then it doesn't matter whether you're using international bandwidth, national bandwidth or talking to another ISP customer. At 1mbit/sec upload thats only 1000 customers uploading to another thousand customers for a gigabit ethernet port to be saturated.
:) but its going to be a small %age of their overall traffic.
there's plenty of oppertunities for local content. The trouble is that our last-mile infrastructure sucks, and there's currently no money to overhaul it to make it, well, not suck.
Our distances are huge, and our densities are horrible.
Up until recently (a couple of years ago) there was only one real option to get bandwidth to each exchange to hook up your non-Telstra (incumbent telco) DSLAM - and that was with Telstra connectivity. That cost a $LOT. Nowdays PIPE, UECOMM and others are running fibre everywhere and hopefully we'll see the models changing slightly.
I don't know how much slashdot readers know about IP networking (ok, I'll assume not a lot), but almost all ISPs in Australia backhaul all their DSL traffic as layer 2 (PPPoE -> (PPPo)L2TP -> {IP|ATM|whatever}) and aggregate it at a few|one point per city. This makes for a very inefficient traffic model when it comes to scaling.
If you're backhauling all your traffic back to one location onto an enormous virtual dial box (Cisco 10000 in some circumstances, Cisco 72xx's in others, Linux l2tpns at a few
Considering how much all of that equipment costs to -deliver packets to end users-, having 1000 users saturate a gigabit of connectivity when your profit margins are only $10 or $20 a month is just not economically feasable.
The reason EU can pull this off is that they have higher densities than the US, they seem happier to move to ethernet inside buildings rather than overload the old cable TV infrastructure, and there seems to be less exclusive contracts for areas like you have in the US.
Finally, its not a case of net neutrality. If the iTunes content was saturating 75%-80% of an ISPs network then they'd be -stupid- -not- to try and monetize that. iiNet are offering it for free because of marketing. The traffic isn't free (although caching traffic is nice, Akamai or not
.. iiNet run the free local traffic stuff because:
* people don't leave their computers on 24/7 downloading from the free traffic zone; and
* its basically free marketing for what they already had (apple content coming off their local akamai farm.)
The problem bounces between "not enough transit" and "not enough backhaul capacity between clients and the DSL aggregation point." If everyone's running P2P, even if its just between each other and people on the local internet exchange, they're still having to shovel gigabits of traffic around for very little revenue.
If clients started rushing massive amounts of traffic between other clients on the (much cheaper than transit) local peering fabric then their oversubscribed DSL delivery network starts getting taxed. That's why you don't get free local peering traffic anymore - they're still doing the accounting to know what kind of traffic you're doing, but the cost of shovelling a gigabit + of p2p traffic to a few % of their clientbase was just not economical.
Transit isn't the expensive bit anymore. Its getting the damned traffic to the end user through DSL and Cable, combined with 20 year old dialup aggregation models which Just Work for DSL aggregation in Australia.
(The capatcha? word here is "scaled". Amusing.)
(I don't work for iiNet.)
Thats not as true. The trouble with 4-gig-in-32-bit-mode is the memory map breakdown. There's reserved space, generally above 3gig, for PCI devices and such. This covers RAM which you actually can't touch, PAE or not.
:) Its legacy drivers and legacy software thats the problem. The OS can remap the memory space to be the lower 4 gig (whatever the first region is called, its been years since I"ve meddled in this) and then setup DMA to occur to that. The trouble is that you have to keep a region of memory in the first 32 bits available for device DMA, and shuffling data to and from this first 32 bits of space can drop performance significantly.
64-bit mode shuffles the memory map around substantially so the device windows are up the top of the 64 bit address space (IIRC); far out of the way of the RAM we're putting in machines now.
Its relatively hit and miss what size window you'll find in the top gig. Its as large as half a gigabyte on some motherboards, which gives you that "3.5gig available" type message you get in 32 bit mode.
Legacy hardware "handles" PAE mostly fine - the technique is called "bounce buffers". Similar to the technique used by OS developers to support ISA devices in machines with > 16mb RAM..
Oh for gods sake people. Just watch the damned episodes if you can. If you can't then email NBC and tell them why (Doesn't work under Mac. Doesn't work under Linux. Doesn't work with my browser. etc.)
If you just whinge here on slashdot and don't watch the episodes then you're not going to appear in their statistics. You -want- to appear in their statistics. Tell your friends about it. Get people to watch stuff. Whining about it not working -just right- for your situation doesn't help.
You -want- the statistics to reflect that there's interest in this service. You -want- the executives to notice that people are using it, that there's non-windows people using it, that people are actually providing constructive feedback to them. Sheesh!
Hi, I'm in Australia and I'm slightly older than you (28).
I'd suggest writing a letter to your local representative and to the local and state newspapers. Write, write, write. Explain your situation, explain how you were treated.
If you work in a corporate environment you might find that your computer build has "extra" root SSL certificate authority keys installed. the operation then is:
* intercept all SSL communications
* decrypt the traffic, present a root SSL CA key, which the browser will go "yup, thats fine, I trust you"
* proxy server does deep inspection and sends it off re-encrypted to the origin server.
You think this is evil but then think about this: how do you scan SSL downloads for viruses and javascript?
Adrian
(A squid hacker.)