* and # are pretty cool, but for vim so is [i and [^i (show the first occurance of the keyword under the cursor in the minibuffer, recursing through #includes, and ^i is "jump to"). Also ^Wi for opening it in a new window.
For your system you have to remember their history numbers, and it's a lot more typing. Up arrow or ^R to search for it, then just keep hitting ^O, you don't even have to hit enter.
strace: Finding out why that program is failing with an error, but not explaining what the error is about. For example if your program dies with "Not such file or directory" but doesn't say what it was trying to find, strace will generally show a syscall near the end that failed with -ENOENT. strace is also useful for finding out where on earth this program has decided to log.
tcpdump: Watching what networking a program is doing. Often useful for figuring out why packets aren't going where you expect, or if it's the local or remote machine that's having problems.
gdb/valgrind: Both are useful for looking at why a program crashes.
if strace isn't verbose enough, theres also ltrace, which can show you how a program is interacting with a specific (or all) libraries.
More awesomely, if you have found something in your history with ^R or up arrow or whatever, then you can press ^O to "execute this line and put the next line in the history onto the command line". Thus:
vi foo.c
make ./foo
^Rvi^O^O^O^O^O^O^O^O^O^O^O^O^O^O
and so on.
Java couldn't be installed by my distro, and while I could install it by hand, it's annoying to do by hand. And to make matters worse, to install by hand required you to play EULA hopscotch through Sun's site to download it.
And because Java itself wasn't in any of the distro repositories, no program that depended on java would be in the repositories either. So any java program I wanted to run was going to be annoying to install, so I would often try and find an alternative in a more usefully supported language that could be managed by my package manager.
The Java browser plugin, requiring java was also a pain to install, so I never bothered with sites that required a java plugin -- it was too much hassle.
So all in all, I rarely, if ever, had any java software on my machine, and I suspect this is true of a lot of people. I suspect this had a chilling effect on Java under Linux/*BSD's, or on websites as plugins since people would tend to use/support/develop on software that was more easily installable.
Vista's theme reminds me of the eye candy from Enlightenment circa 2000ish. Sure it's a bit better delivered (eg transparency is actual transparency in vista, not faked), but the "black matte" look was all the rage 10 years ago and the excessive overuse of shadows, 3dish effects and transpancy and tinting are all there...
The problem with dark fiber is that it never goes where you want it to. Sure theres heaps of it around in various areas of CBD's (but not past the building you care about), or long distances between cities, but it'll probably coz you a whole heap of money to actually get it from where the fiber is to where you need it to be.
And then you have to assume that the dark fiber has actually been maintained sufficiently that it's worth using. Dark fiber often is left in the ground and ignored and when you go to use it you discover it doesn't work anymore.
One of the major things to consider is that TCP's performance is also bound by the recieve window size[2]. Since TCP can only send more data as it's ACK'd due to this window, RTT plays a factor.
So also consider the "bandwidth delay product"[1] to where you're connecting to and compare this to your recieve window size. Consider checkking to make sure increasing your recieve window size won't get you better performance. This needs to be done at the end of the connection that's recieving the data, which for a checkin is likely to be in the UK...
Another thing is that TCP slows down whenever it sees loss as it considers loss to be due to congestion. The amount of time it takes to recover from this is based on round trips (ie latency again). This has a far more complicated formula to work out, see my webpage http://www.wand.net.nz/~perry/max_download.php for a simple calculator to work it out for you.
Since you're in India I suspect that the delay to your UK collegues is probably going to end up going the "wrong" way around the world and will therefore due to the speed of light in fibre your latency is going to be high.
[1]: Easily calculated by the obvious formula bandwidth*delay=recieve_window_size. Perhaps a more useful version of this is rwin/delay = maximum_tcp_bandwidth. [2]: Not to be confused with the congestion window which is a different concept.
We do this on undernet for Cservice (X) accounts. Although we let people login to them as soon as they're created, we have people daily check them all to see if there are obvious duplicates. We have various scripts that highlight people doing stupid things (10 people registering with varients on the same usernames?). We don't have to deal too much with people using these accounts to spam, but we do have them use them for flooding.
There's a massive incentive for your compeditors to review your applications. They don't want you to get that patent that you could use to prevent them from making money. If I was IBM I'd have a team of people reviewing all of Microsoft's patents, it gives IBM an insight into what Microsoft is doing, and if you can block Microsoft from getting patents then it stops you having to deal with the issue later. Of course Microsoft would be reviewing all of IBM's patents too...
Microsoft are just as scared of VoIP as the carriers, with Skype, Google Talk and everyone else jumping on the VoIP bandwagon, it's another application "space" that Microsoft haven't entered and therefore can't control. By releasing their own "Skype killer" it means that they can control how people use this service, and make sure that Microsoft products are the way to do it.
In the past killer apps on the Internet have doubled their bandwidth usage every 2 weeks, usually for at least 6 months. If IPv6 does aquire a killer app[1], then will ISP's and companies have time to react? How quickly could your ISP buy equipment that supports IPv6? How quickly could your company roll out new IPv6 enabled firewalls?
There are technologies like 6to4[2], and Teredo[3] that will automatically tunnel IPv6 over the v4 internet, even if you are behind NAT and don't have a realworld IPv4 address. However intermediate systems that know nothing of IPv6 can't easily firewall, properly prioritise for QoS, or transparently proxy the traffic inside these tunnels. If you don't have IPv6 support for your network infrastructure, how do you know who on your network *does* have support for it and is using it to bypass your firewalls? Rumour has it that 6to4 and Teredo are both enabled by default on Windows Vista.
The WAND visualisation (lovingly called BSOD by the people who use it) is very interesting to watch. We use it on the Universities/16, and we see all kinds of neat patterns ranging from background scans from viruses, to highly sophisticated scans obviously looking for infectable machines.
The visualisation supports a "darknet" mode where it can show all traffic that isn't being responded to by internal machines, showing scans on other useless traffic (on our capture point it shows up heaps of NTP traffic going to an old NTP server that has been decommissioned).
The visualisation is fully customisable by a series of plugins for things such as layouts (for the left (internal) and right (external) networks), and colours (letting you colour traffic based on the type of traffic).
You can see infected machines on it as a cone of traffic, port scans as a sparkling of different colours to one machine. You can see that different parts of the Internets address space have different protocol mixes (P2P and HTTP interestingly don't have the same patterns). You very quickly get a feel for what "normal" traffic looks like, and can see at a glance if something on the network isn't working right. It's fascinating to watch, and even a layperson can easily see what's going on and understand what's happening. It makes great eyecandy for investors and managers too:)
We're almost ready for a new release supporting a lot more really cool features, including the ability to choose colours based on BPF expressions, tonnes of performance improvements, new plugins such as a geoip layout module.
Download it and it a go (the URL is in the parent post), and let us know if you have any suggestions, we're really keen on new ideas to extend it with.
I help out on the Undernet IRC Network. We have automated tools that detect botnets, but what can we do after we've detected them? Email their ISP's? They in general don't care. Talk to the FBI? They don't care either. Ban (Gline) them from the network? We get DDoS'd for the trouble, either directly by the kiddie taking revenge, or even indirectly by just having to live with the constant synflood of thousands of DDoS drones still trying constantly to reconnect to our servers.
Finding out who these people are isn't hard, we often know who they are, and even where they live, but nobody cares. These kiddies start by playing around DDoSing a few IRC servers here or there, but then they move on to bigger things like extortion rackets etc. Almost all of the people being put away for various High profile Cybercrimes have at one stage or another been well known by IRC administrators, but nobody cares until they've turned their sights on bigger fish than IRC networks.
Here in New Zealand, we have a choice. We can choose not to follow the US and not buy any DVD's, or follow along, and buy DVD's in our region, later and more expensively than if we brought them off Amazon when they were released in the US.
Is that a choice? I don't know. Hollywood holds the rest of the world hostage with it's content. What the content providers are worried about as you so rightly point out is that one day everybody else will hold them hostage dictating that if they don't release their content in a way that everyone agrees with nobody will buy it.
I have my own little internet mapping project ( http://tr.meta.net.nz/ ) which is designed in a similar way (people run traceroute nodes on their machines and information is merged together to provide pretty graphs). I wrote it because people would say "I can't get to this site, can anyone else get to it?" This lets you type in the hostname of a machine and it will take lots of traceroutes from around the Internet and merge them into a graph that you can use to figure out which particular segments of the Internet can/can't access it and where they all get tripped up. Or you can see that you're going via an international route, where as almost everyone else is going across a local exchange point. Also as it has AS# information on it you can determine who's fault it is.
tr produces a "small" section of the Internet (it doesn't map the entire thing) but it produces it in a way that can be interpreted by anyone savvy in network administration. It's mostly based in NZ (as thats what I care about, and thats where I have contacts where people are happy to run tr nodes) but it does show how the NZ Internet works extremely well, and provides reasonable detail to the rest of the Internet.
RFC3484 describes what it/should/ be doing. It has all sorts of criteria to select addresses, although I think most of them could be replaced with just "used the longest matching prefix, and make sure it's in the same scope, and try not to use deprecated addresses". They have an idea of a table that provides preferences for prefixes to determine that lets you override whats happening. The Kernel people want to use the routing table for this (which I can understand, it's a nicer solution imho), but someone needs to write a patch to do it:)
Anyway, with proper RFC3484 support, the kernel should choose 6to4 addresses to talk to 6to4 hosts, and 6bone addresses to talk to everyone else.
I completely agree, and run 6to4 here at home (as I do control my gateway). 6to4 works well, and I've wiki'd my experiences at http://wlug.org.nz/6to4
My biggest problem at the moment is that Linux doesn't do particularly good source address selection for IPv6 addresses, in fact it uses the most recently added address to an interface, which if you have 6to4 *and* a slow, laggy tunnel which takes ages to initialise, then all the source addresses on your packets will be via the slow, laggy tunnel. Gnrrg.
I have a patch for this, however the LK people want a different solution (they want source address selection to be customisable via/sbin/ip route command), and I've not spent time getting my patch to work how they want yet.
If you use irc over SSL, you're in the clear from passive and undetectable monitoring.
Unless someone else on the same channel doesn't use SSL, or the servers don't use SSL from serverserver. or the irc server you connect to just logs everything.
If you want proper protection from the Bad Guys you should be doing end to end encryption.:)
From my reading of the page I linked to, it is in the Advanced Networking Packet for Windows XP, however it was added into SP2, so you don't need the extra networking pack if you have installed SP2. I could be wrong here. Quotes:
Windows XP SP2 includes the Internet Protocol version 6 (IPv6) that was included in the Advanced Networking Pack for Windows XP.
and
Windows XP SP2 includes the following updates to IPv6 that are included in the Advanced Networking Pack for Windows XP
Teredo is a way to give yourself a realworld IPv6 address, even though you are stuck behind NAT (and without cooperation from the NAT device, like uPnP requires).
Basically Teredo tunnels IPv6 packets over UDP, and relies on the fact that most NAT's reuse the same source port for all udp packets that you send that have the same source address internally.
All your application only need to support IPv6. There are Teredo implementations for Linux and FreeBSD and Teredo is built into Windows SP2. Teredo also supports two people both behind NAT to talk to each other directly in almost all common circumstances.
So go add IPv6 support to your applications, and recommend your users use Teredo to defeat NAT!
Like Glom or GNU enterprise. Both prefer to use postgres, but failing that, at least gnuenterprise can use sqllite for local database use (dunno about glom).
Both projects seem pretty good, they just need mindshare:)
Slashdot Mirror
* and # are pretty cool, but for vim so is [i and [^i (show the first occurance of the keyword under the cursor in the minibuffer, recursing through #includes, and ^i is "jump to"). Also ^Wi for opening it in a new window.
For your system you have to remember their history numbers, and it's a lot more typing. Up arrow or ^R to search for it, then just keep hitting ^O, you don't even have to hit enter.
strace: Finding out why that program is failing with an error, but not explaining what the error is about. For example if your program dies with "Not such file or directory" but doesn't say what it was trying to find, strace will generally show a syscall near the end that failed with -ENOENT. strace is also useful for finding out where on earth this program has decided to log.
tcpdump: Watching what networking a program is doing. Often useful for figuring out why packets aren't going where you expect, or if it's the local or remote machine that's having problems.
gdb/valgrind: Both are useful for looking at why a program crashes.
if strace isn't verbose enough, theres also ltrace, which can show you how a program is interacting with a specific (or all) libraries.
Also, you can use ~-/ too to refer to the previous directory. So: /tmp .
cd
cp ~-/foo.c
More awesomely, if you have found something in your history with ^R or up arrow or whatever, then you can press ^O to "execute this line and put the next line in the history onto the command line". Thus:
./foo
vi foo.c
make
^Rvi^O^O^O^O^O^O^O^O^O^O^O^O^O^O
and so on.
I've used that before, and used the names of isotopes to distinguish replacements of a machine (hydrogen, deuterium, tritium)
Java couldn't be installed by my distro, and while I could install it by hand, it's annoying to do by hand. And to make matters worse, to install by hand required you to play EULA hopscotch through Sun's site to download it.
And because Java itself wasn't in any of the distro repositories, no program that depended on java would be in the repositories either. So any java program I wanted to run was going to be annoying to install, so I would often try and find an alternative in a more usefully supported language that could be managed by my package manager.
The Java browser plugin, requiring java was also a pain to install, so I never bothered with sites that required a java plugin -- it was too much hassle.
So all in all, I rarely, if ever, had any java software on my machine, and I suspect this is true of a lot of people. I suspect this had a chilling effect on Java under Linux/*BSD's, or on websites as plugins since people would tend to use/support/develop on software that was more easily installable.
Vista's theme reminds me of the eye candy from Enlightenment circa 2000ish. Sure it's a bit better delivered (eg transparency is actual transparency in vista, not faked), but the "black matte" look was all the rage 10 years ago and the excessive overuse of shadows, 3dish effects and transpancy and tinting are all there...
The problem with dark fiber is that it never goes where you want it to. Sure theres heaps of it around in various areas of CBD's (but not past the building you care about), or long distances between cities, but it'll probably coz you a whole heap of money to actually get it from where the fiber is to where you need it to be.
And then you have to assume that the dark fiber has actually been maintained sufficiently that it's worth using. Dark fiber often is left in the ground and ignored and when you go to use it you discover it doesn't work anymore.
One of the major things to consider is that TCP's performance is also bound by the recieve window size[2]. Since TCP can only send more data as it's ACK'd due to this window, RTT plays a factor.
So also consider the "bandwidth delay product"[1] to where you're connecting to and compare this to your recieve window size. Consider checkking to make sure increasing your recieve window size won't get you better performance. This needs to be done at the end of the connection that's recieving the data, which for a checkin is likely to be in the UK...
Another thing is that TCP slows down whenever it sees loss as it considers loss to be due to congestion. The amount of time it takes to recover from this is based on round trips (ie latency again). This has a far more complicated formula to work out, see my webpage http://www.wand.net.nz/~perry/max_download.php for a simple calculator to work it out for you.
Since you're in India I suspect that the delay to your UK collegues is probably going to end up going the "wrong" way around the world and will therefore due to the speed of light in fibre your latency is going to be high.
[1]: Easily calculated by the obvious formula bandwidth*delay=recieve_window_size. Perhaps a more useful version of this is rwin/delay = maximum_tcp_bandwidth.
[2]: Not to be confused with the congestion window which is a different concept.
We do this on undernet for Cservice (X) accounts. Although we let people login to them as soon as they're created, we have people daily check them all to see if there are obvious duplicates. We have various scripts that highlight people doing stupid things (10 people registering with varients on the same usernames?). We don't have to deal too much with people using these accounts to spam, but we do have them use them for flooding.
There's a massive incentive for your compeditors to review your applications. They don't want you to get that patent that you could use to prevent them from making money. If I was IBM I'd have a team of people reviewing all of Microsoft's patents, it gives IBM an insight into what Microsoft is doing, and if you can block Microsoft from getting patents then it stops you having to deal with the issue later. Of course Microsoft would be reviewing all of IBM's patents too...
Microsoft are just as scared of VoIP as the carriers, with Skype, Google Talk and everyone else jumping on the VoIP bandwagon, it's another application "space" that Microsoft haven't entered and therefore can't control. By releasing their own "Skype killer" it means that they can control how people use this service, and make sure that Microsoft products are the way to do it.
In the past killer apps on the Internet have doubled their bandwidth usage every 2 weeks, usually for at least 6 months. If IPv6 does aquire a killer app[1], then will ISP's and companies have time to react? How quickly could your ISP buy equipment that supports IPv6? How quickly could your company roll out new IPv6 enabled firewalls?
There are technologies like 6to4[2], and Teredo[3] that will automatically tunnel IPv6 over the v4 internet, even if you are behind NAT and don't have a realworld IPv4 address. However intermediate systems that know nothing of IPv6 can't easily firewall, properly prioritise for QoS, or transparently proxy the traffic inside these tunnels. If you don't have IPv6 support for your network infrastructure, how do you know who on your network *does* have support for it and is using it to bypass your firewalls? Rumour has it that 6to4 and Teredo are both enabled by default on Windows Vista.
[1]: My picks are P2P/VoIP, since NAT makes both tricky (although not necessarily impossible).
[2]: http://www.wlug.org.nz/6to4
[3]: http://www.wlug.org.nz/Teredo
The WAND visualisation (lovingly called BSOD by the people who use it) is very interesting to watch. We use it on the Universities /16, and we see all kinds of neat patterns ranging from background scans from viruses, to highly sophisticated scans obviously looking for infectable machines.
:)
The visualisation supports a "darknet" mode where it can show all traffic that isn't being responded to by internal machines, showing scans on other useless traffic (on our capture point it shows up heaps of NTP traffic going to an old NTP server that has been decommissioned).
The visualisation is fully customisable by a series of plugins for things such as layouts (for the left (internal) and right (external) networks), and colours (letting you colour traffic based on the type of traffic).
You can see infected machines on it as a cone of traffic, port scans as a sparkling of different colours to one machine. You can see that different parts of the Internets address space have different protocol mixes (P2P and HTTP interestingly don't have the same patterns). You very quickly get a feel for what "normal" traffic looks like, and can see at a glance if something on the network isn't working right. It's fascinating to watch, and even a layperson can easily see what's going on and understand what's happening. It makes great eyecandy for investors and managers too
We're almost ready for a new release supporting a lot more really cool features, including the ability to choose colours based on BPF expressions, tonnes of performance improvements, new plugins such as a geoip layout module.
Download it and it a go (the URL is in the parent post), and let us know if you have any suggestions, we're really keen on new ideas to extend it with.
I help out on the Undernet IRC Network. We have automated tools that detect botnets, but what can we do after we've detected them? Email their ISP's? They in general don't care. Talk to the FBI? They don't care either. Ban (Gline) them from the network? We get DDoS'd for the trouble, either directly by the kiddie taking revenge, or even indirectly by just having to live with the constant synflood of thousands of DDoS drones still trying constantly to reconnect to our servers.
Finding out who these people are isn't hard, we often know who they are, and even where they live, but nobody cares. These kiddies start by playing around DDoSing a few IRC servers here or there, but then they move on to bigger things like extortion rackets etc. Almost all of the people being put away for various High profile Cybercrimes have at one stage or another been well known by IRC administrators, but nobody cares until they've turned their sights on bigger fish than IRC networks.
Here in New Zealand, we have a choice. We can choose not to follow the US and not buy any DVD's, or follow along, and buy DVD's in our region, later and more expensively than if we brought them off Amazon when they were released in the US.
Is that a choice? I don't know. Hollywood holds the rest of the world hostage with it's content. What the content providers are worried about as you so rightly point out is that one day everybody else will hold them hostage dictating that if they don't release their content in a way that everyone agrees with nobody will buy it.
I have my own little internet mapping project ( http://tr.meta.net.nz/ ) which is designed in a similar way (people run traceroute nodes on their machines and information is merged together to provide pretty graphs). I wrote it because people would say "I can't get to this site, can anyone else get to it?" This lets you type in the hostname of a machine and it will take lots of traceroutes from around the Internet and merge them into a graph that you can use to figure out which particular segments of the Internet can/can't access it and where they all get tripped up. Or you can see that you're going via an international route, where as almost everyone else is going across a local exchange point. Also as it has AS# information on it you can determine who's fault it is.
r osoft.com.png . google.com.png o ot-servers.net.png
tr produces a "small" section of the Internet (it doesn't map the entire thing) but it produces it in a way that can be interpreted by anyone savvy in network administration. It's mostly based in NZ (as thats what I care about, and thats where I have contacts where people are happy to run tr nodes) but it does show how the NZ Internet works extremely well, and provides reasonable detail to the rest of the Internet.
Some interesting examples:
Microsoft: http://tr.meta.net.nz/output/2005-04-08_20:08_mic
Google: http://tr.meta.net.nz/output/2005-05-16_10:14_www
F root server anycast: http://tr.meta.net.nz/output/2005-05-16_10:16_f.r
RFC3484 describes what it /should/ be doing. It has all sorts of criteria to select addresses, although I think most of them could be replaced with just "used the longest matching prefix, and make sure it's in the same scope, and try not to use deprecated addresses". They have an idea of a table that provides preferences for prefixes to determine that lets you override whats happening. The Kernel people want to use the routing table for this (which I can understand, it's a nicer solution imho), but someone needs to write a patch to do it :)
Anyway, with proper RFC3484 support, the kernel should choose 6to4 addresses to talk to 6to4 hosts, and 6bone addresses to talk to everyone else.
I completely agree, and run 6to4 here at home (as I do control my gateway). 6to4 works well, and I've wiki'd my experiences at http://wlug.org.nz/6to4
/sbin/ip route command), and I've not spent time getting my patch to work how they want yet.
My biggest problem at the moment is that Linux doesn't do particularly good source address selection for IPv6 addresses, in fact it uses the most recently added address to an interface, which if you have 6to4 *and* a slow, laggy tunnel which takes ages to initialise, then all the source addresses on your packets will be via the slow, laggy tunnel. Gnrrg.
I have a patch for this, however the LK people want a different solution (they want source address selection to be customisable via
Unless someone else on the same channel doesn't use SSL, or the servers don't use SSL from serverserver. or the irc server you connect to just logs everything.
If you want proper protection from the Bad Guys you should be doing end to end encryption.
Why not?
Quotes:
and
-- IPv6
Use Teredo and whatever protocol you like.
Teredo is a way to give yourself a realworld IPv6 address, even though you are stuck behind NAT (and without cooperation from the NAT device, like uPnP requires).
Basically Teredo tunnels IPv6 packets over UDP, and relies on the fact that most NAT's reuse the same source port for all udp packets that you send that have the same source address internally.
All your application only need to support IPv6. There are Teredo implementations for Linux and FreeBSD and Teredo is built into Windows SP2. Teredo also supports two people both behind NAT to talk to each other directly in almost all common circumstances.
So go add IPv6 support to your applications, and recommend your users use Teredo to defeat NAT!
Like Glom or GNU enterprise. Both prefer to use postgres, but failing that, at least gnuenterprise can use sqllite for local database use (dunno about glom). :)
Both projects seem pretty good, they just need mindshare