Consider a question we had about FreeBSD support this week: is the gethostbyname() re-entrant? Well, in Solaris it is, at least as documented in the man page. In Linux (and most other OS's) it's not, you need to use the gethostbyname_r() function instead. With FreeBSD, it wasn't documented - ARGH! But fortunately I just spent an hour looking at the source, and found out the answer. Which is: there is no support for this in FreeBSD. FreeBSD pundits need to GET OVER IT.
This sort of thing happens ALL THE TIME for me. Wonder how something works? Just look and see!!!
I spent a frustrating FOUR HOURS working with Oracle support trying to figure out if their text import tool would handle input data with newlines (answer: "no, but you can buy 8i and it will"). If I'd had the source, I would have known much quicker!!! I ended up writing a loader in Perl which was just as fast and actually worked. Gotta love Perl.
Another example: I tried to run an FTP server from a chroot() jail on a Solaris box, and it kept failing to be able to create threads. After much painful experimenting, I was able to deduce that it was because Solaris pthreads use memmap() to create stacks for new threads, and/dev/mem was not available in the jail. Incidentally, this makes Solaris a pain in the ass for threaded chroot programs! I don't want MY MEMORY available to chroot programs, so I have to use malloc() for the thread stacks - what a hassle. Again, with source code I could have found this out in minutes.
So why would you choose an operating system which is endorsed by the vendor when you could have one you can investigate?
Re:Component reuse and glib in non-graphical progr
on
GTK+ without X!
·
· Score: 2
But glib isn't really that good. I mean, does anybody but me actually read the source?
For instance, if the system runs out of memory, an error is invoked. Now, the error handler exits with a message by default. I suppose there's some way to change this, but you had better exit, because at least some of the data structures do not check for success/failure.
In my mind, this means it CAN NOT be used for critical server applications, and probably SHOULD NOT be used for end-user applications.
Sure, it's nice for some things because of handy hashes, lists and so on, but it's not as well reviewed or specified as the things that Miguel de Icaza is comparing it to, like the standard C library or the POSIX API. (And to be sure, just use Perl or some other scripting language for anything that isn't a critical server application or an end-user application.)
The problem with this is the tragedy of the commons. You forget that if everyone is left to maximize their own desires, you end up with certain types of common goods being outside of reach.
It's the classic hill-climbing algorithm, and it works for some things (minimal spanning trees, fast food) but not for others (path finding, education). Even the most self-interested need to recognize that even for simple, idealized cases self-interest does NOT equal freedom! It is simply a different kind of freedom.
As a former empolyee at ARIN, and a current employee at the RIPE NCC, we get this question with fairly regular frequency. My short answer is always, "It can't be done." The long answer is, well longer.
The fundamental problem with using IP to establish location is that IP was designed to seperate physical topology from network topology. In other words, you're not supposed to know where hosts are at.
There are three main reasons in my experience why people want to do this:
Marketing. Companies want to do targetted advertisements and such. Using source IP is a bad way to do this - though there are some companies who use DNS tricks to do this with some success.
Legal Protection. Because of bogus laws in various locations around the world (Virginia, Afghanistan, wherever), companies want to avoid sending certain data to certain places. However, this almost always is mainly intended as a CYA (or "good faith" in legal circles), rather than to actually solve the problem.
Curiousity. Networking people do tend to have a certain amount of natural inquisitiveness. That's why they look at HTTP logs, for instance - not just to see where the best pron is.
Any even mildly determined individual can cross national boundaries with ease. Use telnet/ssh into a machine in another location. Also, ISP's transit data between countries (lots of countries in eastern Europe and the middle east use satallite downlinks with bases in North America, for instance).
There was a Spatial Relation BOF at the IETF in Australia this spring, but I don't know how that work is progressing. I expect that voluntary location is a solvable problem. You can also pinpoint someone's farthest possible location with simple speed of light calculations. Anything else I expect is either impossible or intractable.
The BSD license itself has been changed to remove the annoying advertising clause, and I suspect this is the case with the Kerberos license.
The BSD license basically says, "Do whatever you want, just don't claim you wrote it." From this interpretation, MS is within their legal rights as far as Kerberos goes. This ain't the GPL were're talking about here.
One thing that annoys me is the amount of space wasted by windows. The top right corner of most windows is blank, with the menu items crammed on the upper left. Throw in scroll bars, border frame, etc, and your 1600x1200 desktop disappears in overhead.
Not that I necesarily have a solution, just a complaint.:)
Nevertheless, most tools have handles that you can grip with a hand. Hammers, screwdrivers, scissors, wrenches, pencils, pens, cigarette lighters, spray cans, doorknobs, and so on. Yes, the details are different, but that doesn't mean it's meaningless to discuss common features:
1. Small enough to fit comfortably in a hand.
Or good, but not necessary features:
2. Good grip (i.e. not slippery).
Or potential problems to avoid:
3. Avoid untreated wood (may have splinters).
One size does not fit all, but there's no reason to reinvent the wheel for every program. Face it: interfaces were much lower quality in the 1980's when every application had to hand-roll an interface by hand. Users had to learn a totally new paradigm with every application. Yuck.
How can you say "elegant code follows where memory is pinched"?!?!?! Do you remember working with overlay files? What about writing self-modifying code to get a little more functionality out of an assembly program? How about grouping your functions by where they sit in memory rather than what they did for the program so you could use relative jumps and save a few bytes per branch? How about using video memory to store data?
*shudder*
And this is all from when I started coding, in 1987 or so. I don't remember when memory was really pinched, from the 8-bit days - things were way worse then. Imagine using the RAM on your floppy microcontroller for space. Not exactly what I would call elegant. More like an absolute hack-fest.
Palms have always had relatively large amount of memory for special purpose devices, ESPECIALLY when you factor in the huge amount of ROM they carry, allowing for lots of functionality built into system calls. A good design, mind you, but not having memory has never led to elegant design.
Actually, on our machines here (Ultra-2 boxes with Solaris 2.6), we periodically get applications that get stuck in Solaris' LWP kernel code and never terminate. Calls to _exit() don't terminate!!! Also, we have periodic kernel panics where the kernel dump reports multiple attempts to enter a mutex lock by multiple threads. Possibly a device driver problem, but who could know?
I do like not polluting my process id space with threads.
I have seen memory leaks with versions of Linux pthreads:
main is pthread_create() and pthread_join() in a tight loop
You know, CmdrTaco is in violation of the message as stated. And it's silly. You can't agree to something that you haven't seen yet, as is the case here (you agree by submitting the query).
It would be nice if it wrapped properly on an 80 column terminal, too. *sigh*
I downloaded it. It is large, slow, and extremely crash-prone. On the other hand, it did consume all available memory (I can't run both Outlook and Explorer on my desktop, since I only have a measly 64 Mbyte of RAM.)
It is, unfortunately, a fairly typical Microsoft product. In Windows, IE is better than Netscape (by which I mean it's faster, uses less memory, and crashes less often), because they really need for it to be, and because they can get in bed with the OS. In Solaris, they don't seem to care. It is actually bad enough to make me wonder why they bothered.
Addresses in Europe are allocated by RIPE-NCC, not ARIN. If anything, RIPE's policies are less restrictive than ARIN's. Addresses in the Pacific Rim are allocated by APNIC. APNIC's policies are actually fairly close to ARIN's.
All of these regional registries are open membership organizations, with public forums for comments and input. If you don't like it - join up! It's certainly not America using all the IP space by itself!
I suspect the problem in most cases is with the policies of the communication infrastructure (government and private) of the countries involved. Check it out before looking for the easy conspiracy theory.
What th'!?!? Maybe Al Gore went to IANA with a Secret Service agent and assured the IANA that either two/8's or their brains would be entered in the WHOIS database.
Seriously though, if you look at the curves of IP usage and routing table growth from 1993 or 1994, you'll see that the exponential growth was curbed, even though the Internet has grown significantly since then. That's not an accident.
From my personal point of view, I do get aggrevated that I can't get a static IP from my provider, though. IPv6, take me away!
As I understand it, Jon Postel came to Kim Hubbard (president of ARIN) last summer, and told her that ARIN needs to be ready to allocate IPv6 addresses ASAP.
When I started working at ARIN on 1998-11-23, there were no software requirements, no specs, nothing. Nada. Since then, one other engineer and myself have put together the software to process templates and update our database, provide IPv6 WHOIS functionality, and tools for the IP analysts to interact with the database directly.
As far as I know, no hard dates were ever given out. The biggest hold-up has always been, and continues to be, the IPv6 initial allocation policy. If you notice the complaints about the Class A's having been given out in the early days of IPv4, you may understand that getting allocation policy right actually is important!
Nevertheless, the message from the community has come through loud and clear at the RIPE meeting last week, for instance: "We don't care! Give me address space NOW!!!" So that's what's happening. Ready or not, here it comes.
Oh, a final FYI. There does need to be agreement on routing somewhere. Whether that's through a commercial entity, a government, or an entity like ARIN doesn't matter I suppose. But consider the implications of those three options, and maybe you'll see that we don't do such a bad job after all.
Shane Kerr Software Engineer ARIN
IPv4 shortage, private addresses, and IPv6
on
IP Address Shortage
·
· Score: 2
For those who suggest that using private addresses with NAT will handle the IPv4 number shortage, I would remind them that numerous IP features depend on end-to-end addresses. These include congestion control, and more importantly, IPSEC. Please see the following draft-RFC:
It's a pretty good read. Anyway, ARIN should be offering IPv6 addresses the 17th (next Monday) unless politics and policy get in the way. The registration folks are testing my code today.:)
Make sure your ISP is ready! And don't settle for a/128!
There is no requirement to be dual-homed (actually dual-homed networks are a sticky widget that hose up routing tables, and haven't been addressed at all in IPv6) to receive address space. There are different guidelines for ISP's and other organizations (or individuals). The intent it to allocate space to organizations that will actually use it.
This is a plea, from me personally, and also from me as an employee of ARIN.
Please stop using classful naming. Class A, B, and C really don't have much meaning these days. Use CIDR - it's more specific and just all-around better.
$0.10 Tutorial:
Class A is a/8 (e.g. 10.0.0.0/8) Class B is a/16 (e.g. 192.168.0.0/16) Class C is a/24 (e.g. 192.149.252.0/24) Single IP is a/32 (e.g. 206.170.14.74/32)
There you go. I'm sure you can figure out how other networks are specified. For instance, the network slashdot is on is a/23:
206.170.14.0/23
Isn't that better than saying "two class C's"?
Re:charge for IP, !4 domain name, money -> IPv6
on
IP Address Shortage
·
· Score: 1
Network Solutions no longer allocates IP space. This function has been handled in Europe by RIPE-NCC for something like 8 years, and in America by ARIN for the last 18 months. Both companies are not-for-profit, as is APNIC, the Asian-Pacific flavor.
NSI never charged for IP allocation. Actually, the IP allocation was funded by the domain registration fees, which prevented them from paying much attention to it.
I think you misunderstand the relationship between the various components here.
First, there is NSI's database, which contains information, such as your mailing address and so on, in addition to domain names and information (DNS server IP's and whatnow).
Second, there is the whois server. The web page which was recently changed only provided a nice, easy interface to the whois mechanism, AFIAK. Whois is an Internet standard, defined in RFC 954. It is a simple TCP/IP query/response protocol to find out information about Internet objects (networks, hosts, etc). NSI's whois server is generated by a selective dump of their registration database (leaving out credit card numbers, for instance).
And third, there is the DNS hierarchy. This includes a set of root servers, named 'A' through 'L' (I believe those are all that exist right now) in order to keep the names as short as possible, so they can fit in a single UDP packet (ick). These are distributed throughout the planet, with only one being run by NSI. NSI's role is to provide the zone file to the other root name servers. They do this by dumping the appropriate data from their database, as well as combining reverse addresses and other data from non-NSI top-level-domains (.mil,.gov,.uk,.ch,.de, etc).
With the ICANN switch, I'm pretty sure that NSI still generates the zone file for.com, but the other sources all send their zone files straight to ICANN. This should also allow the creation of additional top-level-domains, such as.www,.nom, and.xxx (my personal favorite).
I'm only on chapter two of this book. While it is an extremely impressive and educational source of knowledge, I think it could have stood a little better editing. For example, Recipe 1.2 refers the reader to the use of the && operator in Recipe 8.13, which has no such operator (nor the and operator)!
There have been at least two other errors, that I can't remember off the top of my head (nor find in the book on a quick scan, to be honest). All books have errors, but I'm only on chapter two, and minor details like this can be KILLERS in computer books. It's especially disappointing after the flawless presentation in Programming Perl.
Nevertheless, I do recommend it. It saved me hours on the project I'm working on now!
Slashdot Mirror
Consider a question we had about FreeBSD support this week: is the gethostbyname() re-entrant? Well, in Solaris it is, at least as documented in the man page. In Linux (and most other OS's) it's not, you need to use the gethostbyname_r() function instead. With FreeBSD, it wasn't documented - ARGH! But fortunately I just spent an hour looking at the source, and found out the answer. Which is: there is no support for this in FreeBSD. FreeBSD pundits need to GET OVER IT.
This sort of thing happens ALL THE TIME for me. Wonder how something works? Just look and see!!!
I spent a frustrating FOUR HOURS working with Oracle support trying to figure out if their text import tool would handle input data with newlines (answer: "no, but you can buy 8i and it will"). If I'd had the source, I would have known much quicker!!! I ended up writing a loader in Perl which was just as fast and actually worked. Gotta love Perl.
Another example: I tried to run an FTP server from a chroot() jail on a Solaris box, and it kept failing to be able to create threads. After much painful experimenting, I was able to deduce that it was because Solaris pthreads use memmap() to create stacks for new threads, and /dev/mem was not available in the jail. Incidentally, this makes Solaris a pain in the ass for threaded chroot programs! I don't want MY MEMORY available to chroot programs, so I have to use malloc() for the thread stacks - what a hassle. Again, with source code I could have found this out in minutes.
So why would you choose an operating system which is endorsed by the vendor when you could have one you can investigate?
But glib isn't really that good. I mean, does anybody but me actually read the source?
For instance, if the system runs out of memory, an error is invoked. Now, the error handler exits with a message by default. I suppose there's some way to change this, but you had better exit, because at least some of the data structures do not check for success/failure.
In my mind, this means it CAN NOT be used for critical server applications, and probably SHOULD NOT be used for end-user applications.
Sure, it's nice for some things because of handy hashes, lists and so on, but it's not as well reviewed or specified as the things that Miguel de Icaza is comparing it to, like the standard C library or the POSIX API. (And to be sure, just use Perl or some other scripting language for anything that isn't a critical server application or an end-user application.)
The problem with this is the tragedy of the commons. You forget that if everyone is left to maximize their own desires, you end up with certain types of common goods being outside of reach.
It's the classic hill-climbing algorithm, and it works for some things (minimal spanning trees, fast food) but not for others (path finding, education). Even the most self-interested need to recognize that even for simple, idealized cases self-interest does NOT equal freedom! It is simply a different kind of freedom.
The fundamental problem with using IP to establish location is that IP was designed to seperate physical topology from network topology. In other words, you're not supposed to know where hosts are at.
There are three main reasons in my experience why people want to do this:
- Marketing. Companies want to do targetted advertisements and such. Using source IP is a bad way to do this - though there are some companies who use DNS tricks to do this with some success.
- Legal Protection. Because of bogus laws in various locations around the world (Virginia, Afghanistan, wherever), companies want to avoid sending certain data to certain places. However, this almost always is mainly intended as a CYA (or "good faith" in legal circles), rather than to actually solve the problem.
- Curiousity. Networking people do tend to have a certain amount of natural inquisitiveness. That's why they look at HTTP logs, for instance - not just to see where the best pron is.
Any even mildly determined individual can cross national boundaries with ease. Use telnet/ssh into a machine in another location. Also, ISP's transit data between countries (lots of countries in eastern Europe and the middle east use satallite downlinks with bases in North America, for instance).There was a Spatial Relation BOF at the IETF in Australia this spring, but I don't know how that work is progressing. I expect that voluntary location is a solvable problem. You can also pinpoint someone's farthest possible location with simple speed of light calculations. Anything else I expect is either impossible or intractable.
Shane
http://www.masonhq.com
All the goodness of Perl, embedded in your HTML.
<cynic>
But of course, they'll never get advertising dollars from this...
</cynic>
The BSD license itself has been changed to remove the annoying advertising clause, and I suspect this is the case with the Kerberos license.
The BSD license basically says, "Do whatever you want, just don't claim you wrote it." From this interpretation, MS is within their legal rights as far as Kerberos goes. This ain't the GPL were're talking about here.
Shane
One thing that annoys me is the amount of space wasted by windows. The top right corner of most windows is blank, with the menu items crammed on the upper left. Throw in scroll bars, border frame, etc, and your 1600x1200 desktop disappears in overhead.
:)
Not that I necesarily have a solution, just a complaint.
Shane
Nevertheless, most tools have handles that you can grip with a hand. Hammers, screwdrivers, scissors, wrenches, pencils, pens, cigarette lighters, spray cans, doorknobs, and so on. Yes, the details are different, but that doesn't mean it's meaningless to discuss common features:
1. Small enough to fit comfortably in a hand.
Or good, but not necessary features:
2. Good grip (i.e. not slippery).
Or potential problems to avoid:
3. Avoid untreated wood (may have splinters).
One size does not fit all, but there's no reason to reinvent the wheel for every program. Face it: interfaces were much lower quality in the 1980's when every application had to hand-roll an interface by hand. Users had to learn a totally new paradigm with every application. Yuck.
Shane
How can you say "elegant code follows where memory is pinched"?!?!?! Do you remember working with overlay files? What about writing self-modifying code to get a little more functionality out of an assembly program? How about grouping your functions by where they sit in memory rather than what they did for the program so you could use relative jumps and save a few bytes per branch? How about using video memory to store data?
*shudder*
And this is all from when I started coding, in 1987 or so. I don't remember when memory was really pinched, from the 8-bit days - things were way worse then. Imagine using the RAM on your floppy microcontroller for space. Not exactly what I would call elegant. More like an absolute hack-fest.
Palms have always had relatively large amount of memory for special purpose devices, ESPECIALLY when you factor in the huge amount of ROM they carry, allowing for lots of functionality built into system calls. A good design, mind you, but not having memory has never led to elegant design.
Old Man Shane
I do like not polluting my process id space with threads.
I have seen memory leaks with versions of Linux pthreads:
- main is pthread_create() and pthread_join() in a tight loop
- function calls pthread_exit()
I don't know if this is still is a problem.Triple-DES actually works great. It will take that $80,000,000 computer 2^56 times as long to break - fine for a few years yet.
So this wouldn't cause any problems in NT?
C:\>cd \winnt
C:\WINNT>del *.*
You know, CmdrTaco is in violation of the message as stated. And it's silly. You can't agree to something that you haven't seen yet, as is the case here (you agree by submitting the query).
It would be nice if it wrapped properly on an 80 column terminal, too. *sigh*
I downloaded it. It is large, slow, and extremely crash-prone. On the other hand, it did consume all available memory (I can't run both Outlook and Explorer on my desktop, since I only have a measly 64 Mbyte of RAM.)
It is, unfortunately, a fairly typical Microsoft product. In Windows, IE is better than Netscape (by which I mean it's faster, uses less memory, and crashes less often), because they really need for it to be, and because they can get in bed with the OS. In Solaris, they don't seem to care. It is actually bad enough to make me wonder why they bothered.
Addresses in Europe are allocated by RIPE-NCC, not ARIN. If anything, RIPE's policies are less restrictive than ARIN's. Addresses in the Pacific Rim are allocated by APNIC. APNIC's policies are actually fairly close to ARIN's.
All of these regional registries are open membership organizations, with public forums for comments and input. If you don't like it - join up! It's certainly not America using all the IP space by itself!
I suspect the problem in most cases is with the policies of the communication infrastructure (government and private) of the countries involved. Check it out before looking for the easy conspiracy theory.
What th'!?!? Maybe Al Gore went to IANA with a Secret Service agent and assured the IANA that either two /8's or their brains would be entered in the WHOIS database.
Seriously though, if you look at the curves of IP usage and routing table growth from 1993 or 1994, you'll see that the exponential growth was curbed, even though the Internet has grown significantly since then. That's not an accident.
From my personal point of view, I do get aggrevated that I can't get a static IP from my provider, though. IPv6, take me away!
As I understand it, Jon Postel came to Kim Hubbard (president of ARIN) last summer, and told her that ARIN needs to be ready to allocate IPv6 addresses ASAP.
When I started working at ARIN on 1998-11-23, there were no software requirements, no specs, nothing. Nada. Since then, one other engineer and myself have put together the software to process templates and update our database, provide IPv6 WHOIS functionality, and tools for the IP analysts to interact with the database directly.
As far as I know, no hard dates were ever given out. The biggest hold-up has always been, and continues to be, the IPv6 initial allocation policy. If you notice the complaints about the Class A's having been given out in the early days of IPv4, you may understand that getting allocation policy right actually is important!
Nevertheless, the message from the community has come through loud and clear at the RIPE meeting last week, for instance: "We don't care! Give me address space NOW!!!" So that's what's happening. Ready or not, here it comes.
Oh, a final FYI. There does need to be agreement on routing somewhere. Whether that's through a commercial entity, a government, or an entity like ARIN doesn't matter I suppose. But consider the implications of those three options, and maybe you'll see that we don't do such a bad job after all.
Shane Kerr
Software Engineer
ARIN
Internet Transparency
It's a pretty good read. Anyway, ARIN should be offering IPv6 addresses the 17th (next Monday) unless politics and policy get in the way. The registration folks are testing my code today. :)
Make sure your ISP is ready! And don't settle for a /128!
Shane Kerr
Software Engineer
ARIN
There is no requirement to be dual-homed (actually dual-homed networks are a sticky widget that hose up routing tables, and haven't been addressed at all in IPv6) to receive address space. There are different guidelines for ISP's and other organizations (or individuals). The intent it to allocate space to organizations that will actually use it.
This is a plea, from me personally, and also from me as an employee of ARIN.
/8 (e.g. 10.0.0.0/8) /16 (e.g. 192.168.0.0/16) /24 (e.g. 192.149.252.0/24) /32 (e.g. 206.170.14.74/32)
/23:
Please stop using classful naming. Class A, B, and C really don't have much meaning these days. Use CIDR - it's more specific and just all-around better.
$0.10 Tutorial:
Class A is a
Class B is a
Class C is a
Single IP is a
There you go. I'm sure you can figure out how other networks are specified. For instance, the network slashdot is on is a
206.170.14.0/23
Isn't that better than saying "two class C's"?
Network Solutions no longer allocates IP space. This function has been handled in Europe by RIPE-NCC for something like 8 years, and in America by ARIN for the last 18 months. Both companies are not-for-profit, as is APNIC, the Asian-Pacific flavor.
NSI never charged for IP allocation. Actually, the IP allocation was funded by the domain registration fees, which prevented them from paying much attention to it.
ARIN's policy is now to assign /20 or larger blocks.
I think you misunderstand the relationship between the various components here.
.gov, .uk, .ch, .de, etc).
.com, but the other sources all send their zone files straight to ICANN. This should also allow the creation of additional top-level-domains, such as .www, .nom, and .xxx (my personal favorite).
First, there is NSI's database, which contains information, such as your mailing address and so on, in addition to domain names and information (DNS server IP's and whatnow).
Second, there is the whois server. The web page which was recently changed only provided a nice, easy interface to the whois mechanism, AFIAK. Whois is an Internet standard, defined in RFC 954. It is a simple TCP/IP query/response protocol to find out information about Internet objects (networks, hosts, etc). NSI's whois server is generated by a selective dump of their registration database (leaving out credit card numbers, for instance).
And third, there is the DNS hierarchy. This includes a set of root servers, named 'A' through 'L' (I believe those are all that exist right now) in order to keep the names as short as possible, so they can fit in a single UDP packet (ick). These are distributed throughout the planet, with only one being run by NSI. NSI's role is to provide the zone file to the other root name servers. They do this by dumping the appropriate data from their database, as well as combining reverse addresses and other data from non-NSI top-level-domains (.mil,
With the ICANN switch, I'm pretty sure that NSI still generates the zone file for
Does this mean IBM will finally port Visual Age for Java to something other than AIX, OS/2, and (of course) Windows? Neato!
I'm only on chapter two of this book. While it is an extremely impressive and educational source of knowledge, I think it could have stood a little better editing. For example, Recipe 1.2 refers the reader to the use of the && operator in Recipe 8.13, which has no such operator (nor the and operator)!
There have been at least two other errors, that I can't remember off the top of my head (nor find in the book on a quick scan, to be honest). All books have errors, but I'm only on chapter two, and minor details like this can be KILLERS in computer books. It's especially disappointing after the flawless presentation in Programming Perl.
Nevertheless, I do recommend it. It saved me hours on the project I'm working on now!