The problem is that in the naive implementation, the page copy is always necessary. A complicated implementation (in userspace) to take advantage of COW is more complicated than with explicit notification.
It should come as no surprise that a naive implementation is suboptimal. That's why it's naive. Given that zero-copy sockets is not on by default in FreeBSD, and that you have to explicitly enable it if you want to use it, presumably you've read the documentation and seen the big warning about exactly this issue. From man zero_copy(9):
The user should be careful not to overwrite buffers that have been written to the socket before the data has been freed by the kernel, and the copy-on-write mapping cleared. If a buffer is overwritten before it has been given up by the kernel, the data will be copied, and no savings in CPU utilization and memory bandwidth utilization will be realized.
From an application standpoint, the best way to guarantee that the data has been sent out over the wire and freed by the kernel (for TCP-based sockets) is to set a socket buffer size (see the SO_SNDBUF socket option in the setsockopt(2) manual page) appropriate for the application and network environment and then make sure you have sent out twice as much data as the socket buffer size before reusing a buffer. For TCP, the send and receive socket buffer sizes generally directly correspond to the TCP window size.
So it seems that the FreeBSD developers realized this long ago and perhaps aren't as moronic as Linus thinks.
I'd be willing to bet that unless they're threatened by a government entity, MS plans to integrate virtualization into a future version of Windows Server.
I have my own domain, and create a different address for each organization I correspond with. It's dual purpose: automatic filing, and I also never get any spam except on the addresses I post to mailing lists with. An unexpected benefit is that it makes phishing attempts painfully obvious.
I always chuckle a little when I get mail on my FreeBSD-ports account purporting to be from ebay, or Paypal going to great lengths (searching for the contact address on some software I wrote) to notify me about account problems;)
Yes, but the main page has boxes for both your ID and password. By the time you get to the sitekey page and see that it's wrong/missing, the phishing site already has your login information. That makes the whole thing just a pointless waste of time and an annoyance to have to enter the password a second time.
1. Content Industry files a complaint against unknown person with IP 127.0.0.1 2. attorney writes a letter to ISP 3. ISP laughs at the content industry; tells them that they're suing themselves
PostgreSQL 7 is ancient. There have already been multiple releases in the 8 series. I never worked significantly with 7, so I can't comment on it much, but with 8.x I haven't had the problems you've talked about.
It's not that old. I use 8.1 for new installations but have several production systems on 7.4.8 that haven't been migrated yet. 7.x is extremely stable -- as is 8.x so far.
The biggest changes in 8.x are with user/group (role) management. There's some nice new features too, but even 7.4 is a solid DB. 7.2 was good, but it was 7.3 and 7.4 when it started getting REALLY good.
I've never seen 8.x crash, even when I've thrown gigantic, deeply nested queries at it.
I've never seen a PostgreSQL backend crash (any version). Ever. Even when I was running 8.0 beta for my test environment. I've thrown some pretty wacky queries at it also.
I'm thinking it's more about Linux users who don't know the difference between a Bourne shell and bash, and assume that/bin/sh is bash.
I've seen lots of shell scripts that don't work because of that assumption, and I either fix them or change the first line to #!/usr/local/bin/bash. Though often, if they assume that/bin/sh is bash there are usually other assumptions made that cause it to break (i.e. make is GNU make, awk is gawk,/proc is mounted, etc.)
Sure you do not have to vacuum the InnoDB tablespace and it will internally re-use freed space, but the file itself will never shrink (which is the same as Postgres).
Normal (auto)vacuum won't release disk space, but VACUUM FULL will. It requires a table lock so it's not too good for highly concurrent databases, but if you really need the disk space you can get it back without having to re-create the table.
It's been extremely rare in my experience that you would need to do that (data sets tend to get bigger rather than smaller), but the ability is there in Postgres if you do.
You could always do full emulation in software, but that would be slow enough that the 10-party conference probably wouldn't work anyway.
I don't think there's any way to catch CPUID through processor virtualization (a la VMWare), since it's not a privledged instruction and doesn't touch any memory pages.
No black helicopters here -- just greed. This is the same company that created Kazaa and bundled a bunch of spyware with it. So enterprising hackers modified it to remove the spyware, as well as the built-in advertising banners, and released Kazaa Lite.
The Skype binary is encrypted to try to prevent a similar thing from happening (removal of ads, addition of features that are technically possible but they might want to limit for marketing reasons). Up until now Skype hasn't done any sufficiently annoying advertising to drive anyone to publicly break the encryption. However, as I predicted, it didn't take long for someone to bypass once there was a reason to.
This just in from your boss. Management heard of this web thing, and they've decided that all apps are to be ported to be developed and converted to web applications. They're confident that your model and controllers are portable, and since C and C# are really the same thing, you should have no trouble converting, right?
Management already decided that anything remotely web related (as in uses HTML or HTTP in any way) is under the perview of the Web Team, which is a separate department. They've done a few "intranet" web apps using ASP, and they all have clunky, barely useable interfaces that the users hate, Yet Another Login and Password (TM), and tend to break rather badly when you enter input that they weren't expecting. I haven't tried to exploit anything, but from some of the errors I've encountered I suspect that several are also vulnerable to SQL injection holes.
Perhaps this seems like a crazy situation to you, and that it'd never happen
Not so crazy, but if it does happen I at least know that it won't be my problem. My primary role is mostly support and server/network admin, I only do development to fill specific needs that aren't being addressed elsewhere and because I enjoy it. If they want to webify this all this stuff; fine by me -- all the source is in the Subversion server -- have fun!:)
...they hired on a head architect and he and another developer convinced management to move from Oracle Forms to J2EE.
Talk about going out of the frying pan into the fire... Though if given the choice I'd probably take J2EE over Oracle Forms anyway.
Be careful of the choices you make when developing your applications. Develop in a portable, forward thinking point of view, and security will follow.
The counter is also true. If you're not careful, you can overthink and overplan and end up with something too generic to be much use to anybody. Our accounting package (Oracle) has exactly that problem, it's designed to be able to be used by any type of business and so things that are specific to our type of business can be difficult to accomplish because we don't always fit the norm of what the designers were expecting. Sure you can fork out lots and lots of cash for consultants to come modify it, but that usually ends up being incompatible with some other modules down the line.
But if it WAS finished it would truly be an awesome tool. You might even build in a template toolkit, possibly even a content management system. And the whole application would be a tinly little 60k.so file or cgi. And it wouldn't care which! You could have your cake and eat it too. It would be both RAD and memory/CPU efficient. Why such a tool hasn't been created I do not know but it would be cool. Am I missing something? Maybe there is such a thing already?
Well, if you ever get around to such a beast, my feature request would be to also support running as a persistant FastCGI or SCGI daemon, and support either TCP/IP or UNIX domain socket connection to Apache (or any other web server that supports scgi). Shouldn't be too different from the ISAPI or Apache module modes, as in all those cases you have to be careful to clean everthing up and not leak memory.
For some reason I've been really liking Python for web development lately. I'm not sure why, maybe because it's just so damn flexible. Definitely not the most efficient approach though. I keep threatening to write an scgi wrapper that does page templates in straight C for speed, and supports running python code for page logic (it would keep an interpreter "hot" in memory to avoid startup costs). Maybe plug in optimized C modules for common tasks like session management.
It's nice to know that it's possible. (I didn't know C had that fancy interface and abstract stuff without the ++)
Modular programming has been around much longer than C++ and OOP in general. For languages like C, the source file is your module and the "interface" is the set of functions and their parameters (usually formally defined in a specification somewhere). To swap modules, you just compile a different source file that includes the same functions. It can be done a runtime also, by loading shared libraries. It's actually a lot easier to this run-time in C than C++, because with C++ you have to worry about function name mangling that varies from compiler to compiler.
Abstraction can be done in any language as it's a way of thinking rather than a language construct.
It's even possible to do objects and polymorphism in C -- use structs with function pointers. Define some macros to make life easier. The FreeBSD kernel makes heavy use of plain-C OOP, as does GTK.
But I'll bet my beer to your freedom that he didn't.
For which project? Some of the simpler ones are nothing more than GUI frontends to databases -- in those cases all of the logic happens on the database server, so there's nothing to separate. The client app is the platform-specific part, and the DB server is the platform-independent part.
For more complex projects; definitely. I'll either use a cross-platform library like wxWidgets (that needs C++ though), or make sure the Win32 calls are confined to the GUI modules. Other libraries I always use cross-platform; since my development box is BSD I have to be able to cross-compile it on either OS anyway. For example, libpq for database access, expat if I need XML, OpenSSL.
an app that uses 10's of megs!?! you mean like, on a SERVER!?!
No, on the desktop. The recent trend seems to be toward web apps on gigantic servers, which is one that I happen to disagree with. Web apps are good for two things: Horrible user interfaces, and wasting vast amounts of resources (you've got hundreds of client machines with gigahertz processors sitting there waiting!)
Eventually this trend will reverse -- it's a continuous cycle between thin client and thick client as the dominant philosophy. I prefer to split the difference -- keep business logic and data integrity in the DB server, but leverage the native interface and processing power of your clients.
Are your win32 calls supported by WinXP and Win2000?(probably)
Yes, and usually NT4 as well. Read the platform SDK docs, they tell you exactly which calls are supported on which platforms. As much as MS comes up with brain-dead designs sometimes, at least most of them are well documented.
How much effort would it take to port it to linux?
Not much. The simplest path would be just running them in Wine. I never use embedded IE or anything like that and try to keep things to calls that were supported on anything NT4 or better. Because of that all the basic system calls are supported natively by Wine and the program runs great.
A full port would require more work, but I split out the GUI code from the backend code so it's just a matter of replacing a few modules. All my UIs are based on a homegrown adaptive sizer model loosely inspired by GTK sizers, so it would translate well. For bigger projects I've been known to use wxWidgets which is cross-platform; it's mostly the smaller DB frontends that I go bare-metal on.
Are you helping lock your organisation onto a single software platform?
No, server side I use PostgreSQL on FreeBSD. Client desktops we're already locked in by accounting software; not much I can do about that in any case. For my stuff moving to another platform wouldn't be difficult as all of the data and rules for it are managed server-side.
Haha, well that may or may not be an upside, but I consider that a quality-assurance measure. After all, any competent and intelligent programmer or engineer will be able to figure it out. It's only the java-monkeys with paper certs and degrees in meta-theory who have never touched any real code (without 3+ layers of abstraction) in their life that will be lost.;)
I've played with a bunch of frameworks based on Java, Ruby, Python, etc... However for my last few projects I decided to go "old school". Since the target platform was Windows, that meant plain C and Win32 API. No MFC or anything. Staticly linked libpq if I need database access. Extra plus is that without C++ or COM frameworks, I can use mingw gcc on my BSD workstation to cross-compile.
It was a little more work up front, but I've gotten nothing but extremely positive responses about the interface. The application binary usually is under 50k, even the larger ones don't break the 100k barrier. They're extremely quick and responsive on modern machines, and still very usable on older ones. I like to do processing asynchronously (i.e. user types a few characters and a DB query kicks off in the background when they stop typing) and it keeps things snappy. It's pretty easy to literally run circles around all the bloated apps eating up tens of megs of memory or more.
While I'm on the subject, 60 Hz electromagnetic interference is also a hazard. I tested this back in college. I dropped a waterproof electromagnet into a fish tank filled with algae. I took samples every day for two months while I generated a (measured) 2 gauss (yes, two gauss) 60 Hz electromagnetic field. It caused the algae to grow much more dense and dark than the control samples.
How does that prove it's a hazard? The algae seemed to like it...
Silly FUD. The Edit menu is still available if you edit your gnome.prefs to include the line "UseMenuThatShallNotBeNamedBecauseItHasBeenDeclare dBad=1" in the "StupidAnachronisms" section.
Slashdot Mirror
It should come as no surprise that a naive implementation is suboptimal. That's why it's naive. Given that zero-copy sockets is not on by default in FreeBSD, and that you have to explicitly enable it if you want to use it, presumably you've read the documentation and seen the big warning about exactly this issue. From man zero_copy(9):
So it seems that the FreeBSD developers realized this long ago and perhaps aren't as moronic as Linus thinks.
I didn't say it was evil or a bad thing, just that they would do it.
Mini disc. Pffft. The Clik! Disk is here to stay!
Ugh, iomega crap. "Clik" as in "click-of-death" is more like it.
LS-120 forever!
Virtual Server does not come with windows
...yet.
I'd be willing to bet that unless they're threatened by a government entity, MS plans to integrate virtualization into a future version of Windows Server.
I have my own domain, and create a different address for each organization I correspond with. It's dual purpose: automatic filing, and I also never get any spam except on the addresses I post to mailing lists with. An unexpected benefit is that it makes phishing attempts painfully obvious.
;)
I always chuckle a little when I get mail on my FreeBSD-ports account purporting to be from ebay, or Paypal going to great lengths (searching for the contact address on some software I wrote) to notify me about account problems
Yes, but the main page has boxes for both your ID and password. By the time you get to the sitekey page and see that it's wrong/missing, the phishing site already has your login information. That makes the whole thing just a pointless waste of time and an annoyance to have to enter the password a second time.
How is having javascript that runs clientside to fiddle with the DOM more maintainable than static HTML?
Hmm, more like this:
1. Content Industry files a complaint against unknown person with IP 127.0.0.1
2. attorney writes a letter to ISP
3. ISP laughs at the content industry; tells them that they're suing themselves
First thing I do on a root login:
.bash_history && ln -s /dev/null .bash_history
rm -f
PostgreSQL 7 is ancient. There have already been multiple releases in the 8 series. I never worked significantly with 7, so I can't comment on it much, but with 8.x I haven't had the problems you've talked about.
It's not that old. I use 8.1 for new installations but have several production systems on 7.4.8 that haven't been migrated yet. 7.x is extremely stable -- as is 8.x so far.
The biggest changes in 8.x are with user/group (role) management. There's some nice new features too, but even 7.4 is a solid DB. 7.2 was good, but it was 7.3 and 7.4 when it started getting REALLY good.
I've never seen 8.x crash, even when I've thrown gigantic, deeply nested queries at it.
I've never seen a PostgreSQL backend crash (any version). Ever. Even when I was running 8.0 beta for my test environment. I've thrown some pretty wacky queries at it also.
I'm thinking it's more about Linux users who don't know the difference between a Bourne shell and bash, and assume that /bin/sh is bash.
/bin/sh is bash there are usually other assumptions made that cause it to break (i.e. make is GNU make, awk is gawk, /proc is mounted, etc.)
I've seen lots of shell scripts that don't work because of that assumption, and I either fix them or change the first line to #!/usr/local/bin/bash. Though often, if they assume that
Sure you do not have to vacuum the InnoDB tablespace and it will internally re-use freed space, but the file itself will never shrink (which is the same as Postgres).
Normal (auto)vacuum won't release disk space, but VACUUM FULL will. It requires a table lock so it's not too good for highly concurrent databases, but if you really need the disk space you can get it back without having to re-create the table.
It's been extremely rare in my experience that you would need to do that (data sets tend to get bigger rather than smaller), but the ability is there in Postgres if you do.
You could always do full emulation in software, but that would be slow enough that the 10-party conference probably wouldn't work anyway.
I don't think there's any way to catch CPUID through processor virtualization (a la VMWare), since it's not a privledged instruction and doesn't touch any memory pages.
No black helicopters here -- just greed. This is the same company that created Kazaa and bundled a bunch of spyware with it. So enterprising hackers modified it to remove the spyware, as well as the built-in advertising banners, and released Kazaa Lite.
The Skype binary is encrypted to try to prevent a similar thing from happening (removal of ads, addition of features that are technically possible but they might want to limit for marketing reasons). Up until now Skype hasn't done any sufficiently annoying advertising to drive anyone to publicly break the encryption. However, as I predicted, it didn't take long for someone to bypass once there was a reason to.
This just in from your boss. Management heard of this web thing, and they've decided that all apps are to be ported to be developed and converted to web applications. They're confident that your model and controllers are portable, and since C and C# are really the same thing, you should have no trouble converting, right?
Management already decided that anything remotely web related (as in uses HTML or HTTP in any way) is under the perview of the Web Team, which is a separate department. They've done a few "intranet" web apps using ASP, and they all have clunky, barely useable interfaces that the users hate, Yet Another Login and Password (TM), and tend to break rather badly when you enter input that they weren't expecting. I haven't tried to exploit anything, but from some of the errors I've encountered I suspect that several are also vulnerable to SQL injection holes.
Perhaps this seems like a crazy situation to you, and that it'd never happen
Not so crazy, but if it does happen I at least know that it won't be my problem. My primary role is mostly support and server/network admin, I only do development to fill specific needs that aren't being addressed elsewhere and because I enjoy it. If they want to webify this all this stuff; fine by me -- all the source is in the Subversion server -- have fun! :)
Talk about going out of the frying pan into the fire... Though if given the choice I'd probably take J2EE over Oracle Forms anyway.
Be careful of the choices you make when developing your applications. Develop in a portable, forward thinking point of view, and security will follow.
The counter is also true. If you're not careful, you can overthink and overplan and end up with something too generic to be much use to anybody. Our accounting package (Oracle) has exactly that problem, it's designed to be able to be used by any type of business and so things that are specific to our type of business can be difficult to accomplish because we don't always fit the norm of what the designers were expecting. Sure you can fork out lots and lots of cash for consultants to come modify it, but that usually ends up being incompatible with some other modules down the line.
But if it WAS finished it would truly be an awesome tool. You might even build in a template toolkit, possibly even a content management system. And the whole application would be a tinly little 60k .so file or cgi. And it wouldn't care which! You could have your cake and eat it too. It would be both RAD and memory/CPU efficient. Why such a tool hasn't been created I do not know but it would be cool. Am I missing something? Maybe there is such a thing already?
Well, if you ever get around to such a beast, my feature request would be to also support running as a persistant FastCGI or SCGI daemon, and support either TCP/IP or UNIX domain socket connection to Apache (or any other web server that supports scgi). Shouldn't be too different from the ISAPI or Apache module modes, as in all those cases you have to be careful to clean everthing up and not leak memory.
For some reason I've been really liking Python for web development lately. I'm not sure why, maybe because it's just so damn flexible. Definitely not the most efficient approach though. I keep threatening to write an scgi wrapper that does page templates in straight C for speed, and supports running python code for page logic (it would keep an interpreter "hot" in memory to avoid startup costs). Maybe plug in optimized C modules for common tasks like session management.
It's nice to know that it's possible. (I didn't know C had that fancy interface and abstract stuff without the ++)
Modular programming has been around much longer than C++ and OOP in general. For languages like C, the source file is your module and the "interface" is the set of functions and their parameters (usually formally defined in a specification somewhere). To swap modules, you just compile a different source file that includes the same functions. It can be done a runtime also, by loading shared libraries. It's actually a lot easier to this run-time in C than C++, because with C++ you have to worry about function name mangling that varies from compiler to compiler.
Abstraction can be done in any language as it's a way of thinking rather than a language construct.
It's even possible to do objects and polymorphism in C -- use structs with function pointers. Define some macros to make life easier. The FreeBSD kernel makes heavy use of plain-C OOP, as does GTK.
But I'll bet my beer to your freedom that he didn't.
For which project? Some of the simpler ones are nothing more than GUI frontends to databases -- in those cases all of the logic happens on the database server, so there's nothing to separate. The client app is the platform-specific part, and the DB server is the platform-independent part.
For more complex projects; definitely. I'll either use a cross-platform library like wxWidgets (that needs C++ though), or make sure the Win32 calls are confined to the GUI modules. Other libraries I always use cross-platform; since my development box is BSD I have to be able to cross-compile it on either OS anyway. For example, libpq for database access, expat if I need XML, OpenSSL.
an app that uses 10's of megs!?! you mean like, on a SERVER!?!
No, on the desktop. The recent trend seems to be toward web apps on gigantic servers, which is one that I happen to disagree with. Web apps are good for two things: Horrible user interfaces, and wasting vast amounts of resources (you've got hundreds of client machines with gigahertz processors sitting there waiting!)
Eventually this trend will reverse -- it's a continuous cycle between thin client and thick client as the dominant philosophy. I prefer to split the difference -- keep business logic and data integrity in the DB server, but leverage the native interface and processing power of your clients.
Are your win32 calls supported by WinXP and Win2000?(probably)
Yes, and usually NT4 as well. Read the platform SDK docs, they tell you exactly which calls are supported on which platforms. As much as MS comes up with brain-dead designs sometimes, at least most of them are well documented.
How much effort would it take to port it to linux?
Not much. The simplest path would be just running them in Wine. I never use embedded IE or anything like that and try to keep things to calls that were supported on anything NT4 or better. Because of that all the basic system calls are supported natively by Wine and the program runs great.
A full port would require more work, but I split out the GUI code from the backend code so it's just a matter of replacing a few modules. All my UIs are based on a homegrown adaptive sizer model loosely inspired by GTK sizers, so it would translate well. For bigger projects I've been known to use wxWidgets which is cross-platform; it's mostly the smaller DB frontends that I go bare-metal on.
Are you helping lock your organisation onto a single software platform?
No, server side I use PostgreSQL on FreeBSD. Client desktops we're already locked in by accounting software; not much I can do about that in any case. For my stuff moving to another platform wouldn't be difficult as all of the data and rules for it are managed server-side.
...and yes, the code is well-commented.
Haha, well that may or may not be an upside, but I consider that a quality-assurance measure. After all, any competent and intelligent programmer or engineer will be able to figure it out. It's only the java-monkeys with paper certs and degrees in meta-theory who have never touched any real code (without 3+ layers of abstraction) in their life that will be lost. ;)
I've played with a bunch of frameworks based on Java, Ruby, Python, etc... However for my last few projects I decided to go "old school". Since the target platform was Windows, that meant plain C and Win32 API. No MFC or anything. Staticly linked libpq if I need database access. Extra plus is that without C++ or COM frameworks, I can use mingw gcc on my BSD workstation to cross-compile.
It was a little more work up front, but I've gotten nothing but extremely positive responses about the interface. The application binary usually is under 50k, even the larger ones don't break the 100k barrier. They're extremely quick and responsive on modern machines, and still very usable on older ones. I like to do processing asynchronously (i.e. user types a few characters and a DB query kicks off in the background when they stop typing) and it keeps things snappy. It's pretty easy to literally run circles around all the bloated apps eating up tens of megs of memory or more.
While I'm on the subject, 60 Hz electromagnetic interference is also a hazard. I tested this back in college. I dropped a waterproof electromagnet into a fish tank filled with algae. I took samples every day for two months while I generated a (measured) 2 gauss (yes, two gauss) 60 Hz electromagnetic field. It caused the algae to grow much more dense and dark than the control samples.
How does that prove it's a hazard? The algae seemed to like it...
Try lsdvd or ifo_dump. Then just glance at the list and see which title has the longest run time.
Silly FUD. The Edit menu is still available if you edit your gnome.prefs to include the line "UseMenuThatShallNotBeNamedBecauseItHasBeenDeclare dBad=1" in the "StupidAnachronisms" section.
Somebody mod +10 So True
Yeah, I think that was even the topic of an episode of Law and Order: SUV a few weeks ago.