Slashdot Mirror
Creating an Electronic Data Interchange System?
jgrumbles asks: "I've been in a PC support technician internship for the past 7 months for a polyethylene (plastic) pipe company which has doubled in size the past 4 years. I've been notified that management wants me to head a new initiative within the company. The main goal, in the beginning, is to basically restructure the slipshod EDI system they are using right now. The IT director even admits he should have had some training when implementing the system, but it was at the time of the boom so he had to do it as he went along. Are there any definitive EDI/E-Commerce information conglomerates, websites, listservs, groups, or other sources of information? My main mission will be to recreate the EDI system, which includes an AS/400 in a Windows environment, from scratch. Further down the road I'll be in charge of implementing technology in other areas such as getting RFIDs on every piece of pipe we ship in order to further automate tracking and billing. So, does anyone have suggestions on where to look for information and possible case studies?"
Boss: jgrumbles, our company is growing fast; we've doubled in size over the past 48 months. We need you to design, build and implement an EDI to replace our AS/400 system. Plan for expansion into RFID, shipping and automated tracking & billing. Would you mind using Ask Slashdot for guidance in this risky, company-wide endeavour?
Trolling is a art,
1)
Do you really need RFID that badly? It's pipe, for god's sake
2)
What the HELL are you talking about? Like, stuff used to control machines or what?
Show this to your friends and family that don't know what a real hacker is
Ok, this REALLY should have been one for google- even I admit that and I'm usually on the other side. Just how many hackers on slashdot do you think have even messed with EDI?
Still, given IBM's movement to bladeservers, I'd suggest whatever solution you come up with should eliminate the maintenance cost of the old iron, the AS/400. You'll save enough money there alone to purchase a fleet of blades, and even if you're running Oracle or MySQL, you're better off than DB2 on a single mainframe.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
Why do most people insist on recreating the wheel. I cannot imagine that your company needs some specialized version of EDI that no other company uses since that is the point of EDI (to make everything equal).
Tell your boss that buying standard software is usually cheaper than programming and supporting custom software to do the same job. Are you going to be writing the companies word processing software next. If so, quit now!
Try here
Error: Sig not found.
That is just overkill. The big boys have been messing with RFID and say that chances are they'll never RFID every single item. It's practical when marking containers and boxes but not on individual items.
Second, EDI is something very few people ever mess with because there are too many companies out there that can handle it for you. The trucking company I work with does EDI with some of our larger customers and it's completely automated and tied into our dispatching software. A company called TSi handles the transactions for us.
Don't re-invent what's already been invented. I'm betting whatever software your company uses for accounting already has a module that handles EDI.
Hello, Slashdot. I have a brain tumor and have decided that instead of going to a hospital, I'd try to operate on myself. I have the surgery part all worked out. I figure a hammer blow to my head and steak knife sterilize with hot water from the sink will do the trick. My question to Slashdot is two-fold: A. What open source life support systems are out there? and 2. Am I retarded?
..stay the fuck away from XML.
No, seriously. That's a sure sign that your employer is, in a word, without sense or direction. Sure, there's opportunity in chaos, but my 15 years in IT haven't prepared me to take on a challenge like that.
We do EDI. We do it big. It's complex. It's a pain. Hire consultants. They'll waste your money, but it's not the sort of place into which one hops with no experience. Heck, hire a vendor like (Covast|Mercator|Gentran|Amtrix) to do it end-to-end. You'd be much better off career-wise learning to track and manage a project like that rather than to do it yourself.
(We're not consultants, by the way. We're a distribution company.)
Amateurs discuss tactics. Professionals discuss logistics.
Holy [bleep]!
You're an intern PC Support Technician which is to say you're learning how to work a help desk, and you've been asked to lead a team in designing, building and deploying an EDI solution which typically runs on mainframes?
What's next? Is your company gonna ask the guy who changes lightbulbs to step up to the chaulkboard and sketch out the full specs for powering and cooling a 512 node beowulf cluster?
Something really smells here...
You seem to have mistaken us for the airheads you meet at cocktail parties.
Maybe I'm just mad because I can never adequately explain what I do for a living to the airheads I meet at cocktail parties.
-Peter
I've been in a PC support technician internship for the past 7 months for a polyethylene (plastic) pipe company which has doubled in size the past 4 years. I've been notified that management wants me to head a new initiative within the company.
This is where you step up and tell your Supervisor, IT Director, or the President that you're utterly unqualified for this task.
Unless you're some whiz experienced systems analyst with experience in EDI and supply chain that could only get a job as a PC Support Tech, you're in WAY over your head.
It's not a jab at you, I'm not looking down at you, but this isn't a simple problem and tends to be rather mission critical.
If the company wants you involved in this project, they should find a consultant to organize and lead it and have him hand off leg work to you.
Best of luck to you!
We're grudgingly letting Yahoo host it but if you want, come on over to the EDI-L mailing list. Messages are viewable by the public.
I have a few years of EDI experience as well, and it's my day job. I don't mind knowledge transfer either.
Mit der Dummheit kämpfen Götter selbst vergebens.
Knowing what to search for brings up these relevant links:
EDIFACT
X12
How Radio Frequency Identification Affects EDI
Integration for Logistics: RFID, EDI, XML, and Beyond
If you are using an off-the-shelf inventory/billing system they you should probably consider letting someone else handle the integration and format-translation.
I have implemented an EDI system from scratch at my previous company. It was based on EDIFACT and email, and had extensive tracking&tracing, status feedback, error handling. The major challenge in implementing and EDI system is the integration with your EDI partners. It took 3 months from start of testing to the first real EDI message getting through, and almost a year before the workflow was right. Another challenge is that touches on legal responsibility - who said what, why, when.
I believe that ROI was good. No more manually entering 5 batches of 100 items every day. And the deadlines were improved so the final information set could be imported half an hour before work was initiated.
As far as I know the system is still chugging along 5 years after I left the company.
Seriously though, you need to get someone that has massive experience doing this sort of thing and doing it in anger.
Lasers Controlled Games!
EDI is an interesting problem because it is one of those standards that we all just wish would go away. XML would work just as good, but if a EDI partner has a working system based on EDI... well then EDI it is.
I am the author of FreeB which is the first GPL medical billing engine available under the GPL. One of the standards that we support is a classic EDI standard, namely the X12 837p4010a medical billing standard. Hairy beast.
We have had a surprising amount of success using, of all things, PHP and Smarty. In order to handle the random variations that EDI interfaces call for you need a robust templating language. You must be able to seperate data from formating = templating. Smarty is an excellent templating engine! PHP/Smarty is very good at removing data from a database and then pumping them out in arbirary formats, which exactly what you need.
You could take a look at some of the FLOSS EDI libraries out there, but ultimately the feature you need is templating.
Regards,
Fred Trotter
You're a techie, but this is no reason to start off with a solely tech discussion. As a former techie, and more recently a consultant, here's my spin:
(1) get your boss to write down the business reasons for the change. Not technical reasons, business - cost savings, productivity increases, etc.
If there aren't well defined reasons, then you can't technically come up with a solution that's much use, as they haven't defined the problem for you.
(2) Get your boss to write down the goal of the project. This is usually a single statement. Eg if you worked in an oil company the goal might be "deploy a pipeline from turkey to russia that can carry 5m L oil per day". If he won't write it down, you write it, and get him to approve it formally.
If he can't or won't, then you don't have a baseline to judge all future decisions against. I.e. whwnever you have a question or problem, ask yourself does it help or hinder this goal.
(3) Figure out what the parameters of the deployment should be: think what, when, why, how, where. E.g. it should be completed within X months, should not cost more than $Y, should be availble in Z% of offices, should not take more time to complete a transaction than the exisiting system.
These parameters help define what you should and shouldn't do, how much resource to devote etc. Basically they give you the rules of the game.
(4) Figure out who the stakeholders are - ie who is affected, and who has power within the company. This typically includes the fincial director at a high level, and end-suers at the low level. Speak to the FD - he often has final say over everything, and has his own set of expectations different to the IT director. Speak to the users - they can suggest problems that need to be fixed, and things that work and they want kept.
Knowing the stakeholders and keeping them informed and happy is the biggest challenge.
No offense, but you say you're a 7m technician intern. This makes me suspicious they putting a core business function in your hands that if fucked up could kill the company.
I suggest after doing basic research, going back to the IT director and saying something along the lines of "the total cost of switching, impacts on the business and risk to business interruption are not clear. Yes the current solution might look expensive on paper, but it might be less expensive than swapping. We need to scope this in a bit more depth, and possibly engage outside experts for a small study to understand costs, benefits and impacts." then hand him a short list of potential expert companies that can help. by this stage you'll have called these companies, and in general terms described your problem, and checked out if than can help, how and will it cost. You'll have names, phone numbers and email addresses for all of them.
I agree with the others when they say EDI isn't something to take on alone. Use the experts. If you've never project managed before on any scale, I suggest you get help with that too - internally or externally.
Best of luck. this can be a great learning experience.
Ps if your company throws a wobbly at any of the things I've suggested above, tehn that means they're not serious. Find yourself another job, they don't pay you enough to deal with bad management on top of everything.
Congrats! Looks like the boss sees dynamism in you - give it your best effort, but be realistic and don't be afraid to ask money to implement an effective solution.
m l
.org to these names). They also have support for stuff like RosettaNet and RFID, but I'm not familiar with the specifics.
t &words=edi&imageField.x=0&imageField.y=0
. shtml
EDI can be mind-numbing. You'll need attention to detail, clear headed thinking, very good and very tightly controlled documentation, and a well thought out and documented processes for new EDI-partner implementation and existing-partner transitions.
For software, try getting something that simplifies life as much as possible. If your company is willing to spend a fair bit consider buying an off-the-shelf solutions.
webmethods: http://www.webmethods.com/ is a good solution. I've been using it for 5 years at work. It's got a good name since lots of people have had success with implementing EDI with it. The best open-to-the-public webMethods user website is http://wmusers.com/ - even if you don't use webMethods, searching that site for terms like "X.12" and "EDIFACT" will give you a sense of how others implement EDI. Here's an article by a friend of mine: "Create Positional Flat File Templates for the WmEDI Parser"
http://www.wmusers.com/ezine/2003apr_rmoser_1.sht
Note, this article is for ver 4.6 - I understand the current version of webMethods (ver 6.5) has a different, graphical, EDI parsing engine. Other articles:
http://www.wmusers.com/ezine/archives.shtml
The good thing about webMethods is it's got a visual-progamming IDE that simplifies life a great deal, and that it's really a webservices/XML-centric solution that also handles EDI - so in future, you can move forward to cXML, xCBL, ebXML, etc, which are XML-centric standards. (For the respective websites just append
The one thing webMethods does not do well is FTP file spooling for pickup - it does built-in FTP clients and an FTP server (can only submit documents to it), plus VAN and SFTP support. For FTP spooling, you'll need to run a separate FTP server.
webMethods competitors include TIBCO, Commerceone (conductor), Microsoft (with Biztalk), IBM and BEA (not sure what). There are also the old distinguished EDI translators like GenTran.
Google shows up a couple of open-source solutions as well:
http://edi4j.sourceforge.net/
http://mec-eagle.sourceforge.net/
Hm, maybe I should do a sourceforge search. Aha, quite a few matches:
http://sourceforge.net/search/?type_of_search=sof
Don't know how good these are though. If you evaluate them, try posting back here if you can.
Whatever solution you choose, when you do build a solution, try making a 3-layer
solution, with a middle layer of canonical data structures isolating the B2B/EDI standard from your backend-system standard. For eg: as mentioned in my article here:
http://www.wmusers.com/ezine/2002jul15_schauhan_1
Also, no matter how well you implement a standard, you'll always come across situations where you need to customize a B2B/EDI implementation because your trading partner sees the standard differently than you do - try building an architecture that enables easy customizations. Also it's important to implement automated integration and system testing (most people only try and automate unit testing) - as you build functionality, keep adding tests to your test suite.
The Business Reason for getting rid of legacy is to cut down on maintenance cost going forward. Sure, the old stuff still works; but as time goes on you'll find fewer and fewer programmers who want to work with it, or even have the faintest clue how it operates, because it was primarily written under the "find a job niche and protect it" school of design. What we're now finding out is that software and hardware, even the cheapest software and hardware, can be in use for decades. Thus, the good CIO will, when given the opportunity, redesign the entire system so that it is easily maintainable and extensible. So that, as you put it, new packages can be "bolted" into the system quickly and easily.
As for the complexity of the business rules, EDI has got to have the simplest set of business rules ever created. You have this well defined database on one side; well defined because of the meta data in the database, you know the data type and format of every field to start with. You have this well defined interface form on the other side, well defined because the standards organizations make it so. That's internal to your own IT department. Now you meet with the IT department from the business you want to send the data to. They've got their own well defined database format, but you don't have to know anything about that. All you have to do is come to an agreement of what standard form to use, and how to ship the data. From there on out the programming is a complete cake walk. You don't actually have to invent anything at all; in fact, asking other stakeholders for input is a really bad idea because they'll just insert a bunch of stuff that isn't needed into the standard.
The hardware and software that you do this with doesn't really matter as long as Database A puts out a report in format B that Database C can read and store. Yes, there may be thousands of columns per record in that report- but string manipulation is the easiest programming possible no matter what system you're using, or what programming language for that matter. And that's all data transformations are- string manipulation.
Don't get me wrong- I understand what you're saying about non-computer people not allowing you to do your job properly, we've all been there. But the closest the business should *ever* be allowed to get to the desgin of an EDI project is telling the CIO "We need to be able to send records from this system to this other company, make it happen", and from there on out, it should all be negotiation between IT departments, not lusers.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
look up ...
;-)
Trusted Links for Windows
or
Trusted Links for the I series (as400)
made by innovis