Yeah that is a pain, most of the memory utilities; ps, top etc simply show the resident set size of the application including shared libraries which leads you to massively over specify RAM, which isn't really that bad a thing I suppose other than spending too much money.
As I mentioned, we're using Solaris I profiled each of the major applications; metacity, panel, nautilus, open office etc using the pmem utility to see how much of each is shared. Turns out that the worst case is around 80% shared and it gets better from there.
It tells you in detail which shared libraries are in place and what's used by a specific instance of a binary:
Note: had to modify the output quiet heavily to get it past the slashdot lameness filter.
So there's 1.8Mb of code loaded and of that, 168k is what this instance of ksh uses for itself, the rest (93%) is shared with all the other instances of ksh. Most of the mem utilities will record the RAM used as 1.8Mb or 1.78Mb. It just shows how inefficiently individual desktop workstations use memory and how Unix systems *should* be used.
In this case I'm talking about a Linux PC, a Solaris/HP-UX/AIX workstation, doesn't matter as long as it has Grid Engine[1] and the application you want to run installed.
We had a load of Sun workstations sitting 98% idle on engineers desks, we took them away, put them into racks and installed the applications the engineers used onto an NFS server so they were available to all the machines. You could do the same with Linux boxes, and if I was putting new systems in I'd probably use Linux.
An application is started by 2 scripts. The first script is run on the login host from a Gnome menu and all it does is submit a request to run a job on the grid using the Grid Engine qsub command, the second script is the job which you want to run and it starts the application, OpenOffice as an example. The Grid Engine system copies the submitted run script over to a queue on the least loaded machine and runs it, up pops OpenOffice on your desktop.
It's really very simple and nothing to do with Java, though it's worth noting that the systems set up as grid nodes should all be configured identically so it doesn't matter which the application starts on.
It's a highly scalable system, you can go from 10 users to 1000 by adding login servers, grid nodes and network capacity, all completely transparently. Grid engine handles the high availability of the execution hosts automatically and a round robin DNS entry or IP failover on the login servers can add high availability to those as well.
[1] Any network queueing and load balancing system should be able to do the job, but SGE is easy to use, easy to get hold of, is free and runs on most platforms.
You see, the login servers run only the user interface; Gnome and a few small applications like the calculator and xterm. Everything else, including the office apps, web browser, interactive terminal sessions is farmed out to application servers managed by Grid Engine.
The situation you describe needing 2GB of ram is because they run the applications directly on the login servers, which is simply the wrong way to do it.
If you run everything on the login servers you need more RAM, more CPUs, more and faster disks, more and faster nework interfaces all of which make the login servers larger and much more expensive to purchase maintain and replace. Because they are expensive, adding more when they are overloaded is a costly pain.
2 single CPU systems with 1Gb of RAM each cost less than half the price of a single dual CPU system with 2Gb of RAM. Take advantage of this.
The system we have in place runs on ex-desktop Sparc 10 workstations, the login servers have 512Mb of RAM and handle 15 sessions each as Gnome on Solaris consumes 128Mb rather than the 90Mb on Intel and 25Mb per user rather than 18mb, the grid node application servers have 256Mb each.
1st layer the displays, old obsolete PCs running an X server.
2nd layer login servers with plenty of memory and CPU which serve an easy to use user interface like Gnome, but which do *NOT* run any heavyweight applications.
2 machines should be plenty to serve desktops to 40 users and give you some redundancy.
3rd layer an array of load balanced application servers run the heavyweight applications. Sun Grid Engine provides the load balancing and high availability. These can be PCs off the desktop if they have the horsepower.
4th layer - 2 NFS servers (again, redundancy) to hold all your applications and data.
This architecture works spectacularly well for Linux and Unix systems, it takes full advantage of their abilities. e.g. A single Gnome login is around 90Mb of RAM but with Linux, 80-95% of every application is shared code so the second and each additional Gnome login to the server only consumes 18Mb of RAM. 40 people at 90 + 40*18 = 810Mb of RAM, you get that in a desktop these days, go for 2 boxes and put half the users on each.
The same is true for Open Office, 90% shared so you create a couple of Open Office servers and they run all of your instances of the application, it completely eliminates the long startup time OO seems to have, qsub the job onto the grid and bang, Office appears on your thin client running on the least loaded application server.
It's worth noting that this isn't new, we've been running this type of architecture forever using old rackmounted Sun workstations. It's "the one".
"Grid computing" is just the new buzzword for PHBs and arts graduates (yes, that's you, that is). So, no I can't truly say that I ever heard any stories about grid computing "in the good old days".
I could tell you all about network queueing systems though, home grown, freely available open source stuff and commercially available systems.
As I said, "Grid computing" is nothing new, it's been in use for decades by those of us who know better than to waste 95% of the computing power they've purchased.
Distributed computing and "grids" are only new to you.
No, really. It takes decades to rebuild small areas of cities and their basic design stays the same for centuries.
London, Paris, Rome etc are all based on cities of hundreds or thousands of years ago. The streets are the same, in the same place with the same or similar names.
We got a rash of "new towns" in the 50s-70s which are a disaster now, requiring a car to do the most trivial of things. They were designed for a specific technology you see.
In mean, FFS why put the wheels side by side and then include $4,300 worth of electronics to make it balance when you can put the wheels inline, save that cash and a load of weight but otherwise have identical performance *and* better brakes?
It's being stupid for stupid's sake.
Google found this for me in 0.1 seconds: http://www.zapworld.com/xtreme.htm
You don't have very much experience working in a corporate Unix environment, do you?
Who would use it? Every corporate I.T. dept on the planet which has Unix or Linux installed somewhere.
We have 400 hardware and software engineers who's only access to Unix is a Gnome login. Everything they do is remote to arrays of rackmounted Unix boxes. It saves a fortune every year.
Always the problem with these things. ssh display forwarding and lbxproxy can both reduce the bandwidth used by X11 but both increase the latency, sometimes to unacceptable levels.
On the corporate LAN we have 100Mbit switched and haven't noticed bandwidth being a problem. We have however noticed that both lbxproxy and ssh require more CPU in order to perform compression and buffering which *can* be a problem on a shared server if the number of concurrent sessions it can support drops by 20%.
I guess if you want X to your phone then it could be an issue, but that's a fairly niche market.
Dr Who's time is past and this is just sentimental nostalgia. It was fine for children 25 years ago but it's return will be nothing more than a disappointment.
Slashdot Mirror
It might be useful. The on board Zaurus software is just deadful.
Yeah that is a pain, most of the memory utilities; ps, top etc simply show the resident set size of the application including shared libraries which leads you to massively over specify RAM, which isn't really that bad a thing I suppose other than spending too much money.
As I mentioned, we're using Solaris I profiled each of the major applications; metacity, panel, nautilus, open office etc using the pmem utility to see how much of each is shared. Turns out that the worst case is around 80% shared and it gets better from there.
It tells you in detail which shared libraries are in place and what's used by a specific instance of a binary:
example:
# pmem 25628
25628: -ksh
Kbytes Resident Shared Private Permissions Mapped File
192 192 192 - read exec ksh
8 8 - 8 read write exec ksh
32 32 - 32 read write exec [heap]
664 664 664 - read exec libc.so.1
24 24 - 24 read write exec libc.so.1
8 8 - 8 read write exec [anon]
552 552 552 - read exec libnsl.so.1
32 32 - 32 read write exec libnsl.so.1
32 16 - 16 read write exec [anon]
16 16 16 - read exec libc_psr.so.1
16 16 16 - read exec libmp.so.2
8 8 - 8 read write exec libmp.so.2
8 8 - 8 read write exec [anon ]
40 40 40 - read exec libsocket.so.1
8 8 - 8 read write exec libsocket.so.1
8 8 8 - read exec libdl.so.1
128 128 128 - read exec ld.so.1
8 8 - 8 read write exec ld.so.1
16 16 - 16 read write exec [ stack ]
total Kb 1800 1784 1616 168
Note: had to modify the output quiet heavily to get it past the slashdot lameness filter.
So there's 1.8Mb of code loaded and of that, 168k is what this instance of ksh uses for itself, the rest (93%) is shared with all the other instances of ksh. Most of the mem utilities will record the RAM used as 1.8Mb or 1.78Mb. It just shows how inefficiently individual desktop workstations use memory and how Unix systems *should* be used.
In this case I'm talking about a Linux PC, a Solaris/HP-UX/AIX workstation, doesn't matter as long as it has Grid Engine[1] and the application you want to run installed.
We had a load of Sun workstations sitting 98% idle on engineers desks, we took them away, put them into racks and installed the applications the engineers used onto an NFS server so they were available to all the machines. You could do the same with Linux boxes, and if I was putting new systems in I'd probably use Linux.
An application is started by 2 scripts. The first script is run on the login host from a Gnome menu and all it does is submit a request to run a job on the grid using the Grid Engine qsub command, the second script is the job which you want to run and it starts the application, OpenOffice as an example. The Grid Engine system copies the submitted run script over to a queue on the least loaded machine and runs it, up pops OpenOffice on your desktop.
It's really very simple and nothing to do with Java, though it's worth noting that the systems set up as grid nodes should all be configured identically so it doesn't matter which the application starts on.
It's a highly scalable system, you can go from 10 users to 1000 by adding login servers, grid nodes and network capacity, all completely transparently. Grid engine handles the high availability of the execution hosts automatically and a round robin DNS entry or IP failover on the login servers can add high availability to those as well.
[1] Any network queueing and load balancing system should be able to do the job, but SGE is easy to use, easy to get hold of, is free and runs on most platforms.
You see, the login servers run only the user interface; Gnome and a few small applications like the calculator and xterm. Everything else, including the office apps, web browser, interactive terminal sessions is farmed out to application servers managed by Grid Engine.
The situation you describe needing 2GB of ram is because they run the applications directly on the login servers, which is simply the wrong way to do it.
If you run everything on the login servers you need more RAM, more CPUs, more and faster disks, more and faster nework interfaces all of which make the login servers larger and much more expensive to purchase maintain and replace. Because they are expensive, adding more when they are overloaded is a costly pain.
2 single CPU systems with 1Gb of RAM each cost less than half the price of a single dual CPU system with 2Gb of RAM. Take advantage of this.
The system we have in place runs on ex-desktop Sparc 10 workstations, the login servers have 512Mb of RAM and handle 15 sessions each as Gnome on Solaris consumes 128Mb rather than the 90Mb on Intel and 25Mb per user rather than 18mb, the grid node application servers have 256Mb each.
1st layer the displays, old obsolete PCs running an X server.
2nd layer login servers with plenty of memory and CPU which serve an easy to use user interface like Gnome, but which do *NOT* run any heavyweight applications.
2 machines should be plenty to serve desktops to 40 users and give you some redundancy.
3rd layer an array of load balanced application servers run the heavyweight applications. Sun Grid Engine provides the load balancing and high availability. These can be PCs off the desktop if they have the horsepower.
4th layer - 2 NFS servers (again, redundancy) to hold all your applications and data.
This architecture works spectacularly well for Linux and Unix systems, it takes full advantage of their abilities. e.g. A single Gnome login is around 90Mb of RAM but with Linux, 80-95% of every application is shared code so the second and each additional Gnome login to the server only consumes 18Mb of RAM. 40 people at 90 + 40*18 = 810Mb of RAM, you get that in a desktop these days, go for 2 boxes and put half the users on each.
The same is true for Open Office, 90% shared so you create a couple of Open Office servers and they run all of your instances of the application, it completely eliminates the long startup time OO seems to have, qsub the job onto the grid and bang, Office appears on your thin client running on the least loaded application server.
It's worth noting that this isn't new, we've been running this type of architecture forever using old rackmounted Sun workstations. It's "the one".
These palm shaped things are next to useless for any serious work.
"Grid computing" is just the new buzzword for PHBs and arts graduates (yes, that's you, that is). So, no I can't truly say that I ever heard any stories about grid computing "in the good old days".
I could tell you all about network queueing systems though, home grown, freely available open source stuff and commercially available systems.
As I said, "Grid computing" is nothing new, it's been in use for decades by those of us who know better than to waste 95% of the computing power they've purchased.
Distributed computing and "grids" are only new to you.
No big deal. You can get almost a Tb on to a single tape these days.
Not particularly. If you can't be bothered to perform Google searches I'm not going to do it for you.
In fact, distributed computing and "grids" are older than you are.
It's called heat.
e ne rgy/sol_thermal/powertower.html#storage
The technique's being used effectively by the Solar II experimental station in California.
http://rhlx01.rz.fht-esslingen.de/projects/alt_
"Free markets cause power blackouts?"
The free market tries to make money out of the infrastructure this means low maintenance, low investment. It's a recipe for blackouts.
Can't say we weren't warned though.
As far as I can see it's only ever moved or divided into smaller units.
Got an expensive mobile phone, watch, PDA? Carrying several hundred dollars? Doesn't matter if it's out of view. Zap, you're a prime target.
Or Hi-ho hi-ho.
Add it to your screensaver, make it loud enough for your neighbors to hear and see how many people are humming/whistling it by home time.
Indiana Jones is pretty effective, as is Mission Impossible.
No, really. It takes decades to rebuild small areas of cities and their basic design stays the same for centuries.
London, Paris, Rome etc are all based on cities of hundreds or thousands of years ago. The streets are the same, in the same place with the same or similar names.
We got a rash of "new towns" in the 50s-70s which are a disaster now, requiring a car to do the most trivial of things. They were designed for a specific technology you see.
In mean, FFS why put the wheels side by side and then include $4,300 worth of electronics to make it balance when you can put the wheels inline, save that cash and a load of weight but otherwise have identical performance *and* better brakes?
It's being stupid for stupid's sake.
Google found this for me in 0.1 seconds:
http://www.zapworld.com/xtreme.htm
And watch your phone go into meltdown when the extra load from the proxies make the performance drop by 20%.
When initially switched and now it saves nearly a hundred grand a year.
We don't do transactions... HTH...
You don't have very much experience working in a corporate Unix environment, do you?
Who would use it? Every corporate I.T. dept on the planet which has Unix or Linux installed somewhere.
We have 400 hardware and software engineers who's only access to Unix is a Gnome login. Everything they do is remote to arrays of rackmounted Unix boxes. It saves a fortune every year.
Always the problem with these things. ssh display forwarding and lbxproxy can both reduce the bandwidth used by X11 but both increase the latency, sometimes to unacceptable levels.
On the corporate LAN we have 100Mbit switched and haven't noticed bandwidth being a problem. We have however noticed that both lbxproxy and ssh require more CPU in order to perform compression and buffering which *can* be a problem on a shared server if the number of concurrent sessions it can support drops by 20%.
I guess if you want X to your phone then it could be an issue, but that's a fairly niche market.
And how to kick it off right from a gdm login?
Dr Who's time is past and this is just sentimental nostalgia. It was fine for children 25 years ago but it's return will be nothing more than a disappointment.
And not carrying a smart card? It isn't compulsory.
Send them a tenner and demand to see all of the information they have on you.