Slashdot Mirror
Distributed Computing For Businesses?
Meostro asks: "I've run several distributed computing clients in the past, and it felt nice to contribute to these projects that are saving the world. I recently became the admin of my company's branch datacenter, so my access to computing power has gone from 4 assorted boxes up to 20 servers and around 400 desktops. My ambition has grown by equal bounds, so I want to do some distributed processing on at least a few of these. We do food warehousing and distribution, so there's no protein folding, DSP or weather modelling work that would significantly contribute to our business. I've done some simple work with optimizing routes for our trucks, but our business doesn't seem to present many other problems that require lots of processing. What are some good business-specific tasks you have used distributed or grid computing to work through?"
Imagine the possibilities. . .
Look, I understand your impulse -- I really do. But you've been given a responsibility so put your energy into carrying it out as fully as possible. You're supposed to be contributing to your employer's business, not digging around in a new box of toys and introducing new points of failure.
Well, they must need the processing power for something. After all they didn't buy 20 servers and 400 workstations just to look cool. Maybe they need the processing power for suff like groupware, inventory control, accounting, human resources and general "work stuff".
While these functions may not require grid computing per se, they do require a fair bit of modern computing power. Perhaps you should forgo the grid computing and use the right tool for the job at hand.
By the way, it seems that you don't have enough to do at your new job. Perhaps you should ask your boss what would be the best way to allocate company computing resources and your leet admin skillz. I'm sure he wouldn't have thought of Slashdot.
You could just do what the rest of admins with too much processing power do - benchmark the re-encoding process of whatever the latest movie thats out.
Although there are various other worthless projects that you could do instead (e.g. S.E.T.I), as you mentioned there are a lot off good projects you could contribute too as well..
How much publcity would your company get if you discovered a full/partial cure for cancer? I doub't they'f fire you for doing something as significat as than (or even firing you for using up CPU time to try and cure something like that). -- btw i'm a cancer veteran.
It's silly to waste time optimizing something that dosn't require it. If there is no need you can think of for using distributed processing why bother? The key is simplicity. Robustness stems from simplicity. The more unneccesary things you add to the system, the more break points you have. Do you really want to spend all day trouble shooting a problem which is cause by something that's not needed in the first place? If you feel the need to waste time read slashdot and maybe do some coding.
What are some good business-specific tasks you have used distributed or grid computing to work through?
I put our servers to work to find out what the most optimized route would be for me to take to sneak out of work early.
The only problem was there seemed to be some "quirk" in the software that kept suggesting to walk by the bosses door.
Once I get that fixed, I'll be on easy street.
Error: Sig not found.
On the 1 hand I agree with the above poster.
On the other hand, there may be something that fits your bill. But if you don't see it yet, just wait, get to know the business better, and keep your eye on all requirements documents, purchase requests, and so forth. Listen in on gripe and bluesky sessions. If it's coming, and you're plugged in and paying attention, you'll recognize the opportunity when it arrives.
We had turned our office into one of those for a while. Imagine 100 frame 3D views of hundreds of products.
Distributed computing is not free.
Compute the power costs of what you are proposing.
Knowingly wasting that sort of money is often a firing offense; at the scales you quote, we're easily talking hundreds a month and it may well exceed your personal salary. Not a great way to ensure you have a job tomorrow.
Look for other optimization problems in your organization. Maybe you should be using optimization algorithms to do things like work out what size box or crate to pack orders in, etc. Anything that's complex combinatorially (lots of permutations) might be a candidate for optimization. Scheduling is another area that can create complex problems.
Some of it just depends on the size of the company though. Even problems that exhibit a "combinatorial explosion" in complexity can be very easy to solve, for very small numbers of inputs.
I suggest you get started soon. Because once your boss finds out that you're using his computers for trite personal projects, you'll be unemployed.
Enjoy it while it lasts.
Don't think that a small group of dedicated individuals can't change the world. It's the only thing that ever has.
The only time I've used distributed computing for legitimate purposes was using distcc for compiling across multiple machines.
This only works if you're a software shop though ..
Let the user drive requirements, and you provide a solution that meets those requirements.
Sometimes the user doesn't ask to have something solved because they think that it can't be solved, or don't realize it is even possible. Find out by having the IT department ask everyone for their wishlist. Sometimes it is something simple (I want to be able to fax from my desktop apps), and sometimes it is earth shattering (CERN users wanting to share physics experiment data).
I have a close friend that is in the food shipping business.
She's said that her first job was trying to optimize the amount of stock in the warehouse at any given point, given that stock (in this case, frozen potatoes, hash browns, etc.) goes bad if stored too long. Yet, if they don't have supply, they miss out on tremoundous money making opportunities.
The problems would seem to be (at least with her work, and you can comment about how applicable this is to your situation):
1. Warehouse supply chain - prediction of needs;
2. Given several warehouses, where to move stock given disparate and sporadic ordering;
3. As you mentioned, routing trucks and how much they carry;
4. tracking unethical short-shipments (where the invoice says they shipped 12 cartons but they only really shipped 11);
5. tracking ethical short-shipments (where you order 12 and only get 11 and are properly invoiced for 11);
6. tracking suppliers on-time percentages, trucker's reliabilities, speeds, and costs;
7. doing predictive modeling of demand cycles given inputs of time of year, price of raw product, going rate (sales price) (and thus profit margin), availability of trucking and rail, etc.
This kind of stuff is why Walmart is huge (no flames on them being evil here, this is just a comment on their technial prowess in supply chain management). If you want to be successful, keep track of everything and run some queries to see what you could do to improve.
Unitarian Church: Freethinkers Congregate!
To put it the exact same way, but maybe more politely:
If there isn't a task that depends on massive amounts of computational horsepower, then doing distributed computing is not a goal you should be seeking for your boss.
(I can't count the number of times I have had to stop my manager from bringing in some new product (or vendor of a product) to buy without having a problem that needs this solution).
However, that being said: Don't close your mind to possibilities. Understand your companies business better. Think outside the box. Perhaps there actually is something that may not be a problem, yet could benefit from this kind (or some other nifty kind) of solution. Google didn't get where it is today by thinking a single or few big iron boxes were the only answer to the search engine game.
1) Optimization of goods warehouse layout, use to ensure that the fastest moving stock is the easiest to get at.
2) Optimization of shipping times. If this is done right then you will get either the slowest trucks or the trucks with the greatest loads out the door first.
3) Work out the most efficient use of your loading bay(s). If movement space is limited in your loading yard then this could be a problem.
4) Examine stock levels with reference to as many external factors as you can. If it's always cold in May in your part of the world then you don't want to be stocking up on popsicles. Also, examine stock levels with reference to annual local or international events and festivals.
5) Report generation (the bean-counters always love their reports). If a report takes thirty minutes to generate on your existing system then consider re-optimizing the code for running on a cluster and see if you can get the time down.
Ed Almos
Budapest, Hungary
The more corrupt the state, the more numerous the laws. - Tacitus, 56-120 A.D.
Thus the reason that any distributed project must contribute directly to the business. If that weren't the case, SETI would be running on everyone's desktop already because it's a pretty screensaver and it's completely worthless from a business perspective.
Try looking for your current problems, and then find some solutions to solve those problems.
In Soviet Russia the insensitive clod is YOU!
You need to make this a business case and get buy in from senior people in the company.
For example, the consequences of the work could get you or the company in big trouble. Constant use of CPU time will perhaps degrade performance of networks and desktop, or reduce the lifetime of computing components (e.g. extra swapping may reduce the life of hard drives on machines you're using), and not only that, cost time (e.g. support problems) and money (e.g. not only power requirements: but offices often don't have aircon enabled over the weekend, and excessive computing use may actually cause some problems with heating up the office).
On the other hand, if you put your case well, you could indicate how these costs and risks will actually pay back with the problems you're solving. You really do need to demonstrate concrete and business relevant outcomes: and possibly the different options for solving these. For example, the warehouse optimisation problem: why can't it be run on two commodity pc's: even if it is going to take an extra 2 weeks to finish the task, this may be considered safer alternative.
In summary: assess the problem you're trying to solve, look at it from all the business angles, come up with a proposal that takes into account the other options to give you an objective perspective, and if your case is good, put it forward, and ensure that you get buyin from others in the company: just don't go out and do it yourself alone.
When there's a sexy new hammer in the tool shop window, every problem looks like a nail.
~~~~~ BigLig2? You mean there's another one of me?
1. Run http://www.informatics.sussex.ac.uk/users/mmg20/dh e/ on all the boxes.
2. Evolve and use Skynet to run the business.
3. ?????
4. Profit!!!