Slashdot Mirror
How Do I Make Sense of Microsoft Access?
Anthony Boyd asks: "I have a pretty good tool-set for LAMP work, but as I get into Microsoft jobs, I've started to wonder if I'm working with the best tools. In particular, I'm exploring an 'out of control' Microsoft Access setup, which has about 200 tables in 30 .mdb files, including some duplicated/outdated tables. I'd like to print the properties of each table (with the comments for each field), print the table list for each database, get info on the field types & relationships, and so forth. What tools do you suggest for trying to grok a large Access mess?"
Look for the Database Documenter in the tools menu...it will print out basically all of the information that you say you're looking for...
"What tools do you suggest for trying to grok a large Access mess?"
Coffee. Lots of coffee, and ignorance. It's bliss, or so I've heard.
How I mine for fish?!?
Sorry. For some reason after reading the title I couldn't think of anything else.
I have also done extensive work with LAMP and Microsoft (Access in particular). I actually started out with ASP and Windows before I learned all the LAMP stuff. IMHO, Access databases are by no means secure and what's more, they're very clumsy animals. It might be in your clients best interests to convert to LAMP/Linux, etc.
If you have that many access databases, you are probably misusing it. Import all that data into SQL Server, and start from there. There is no magic way to make sense of a database schema.. the best you'll do is grabbing a GUI that visualizes it.
I'm going to assume you've got Access 2003, because I can't remember what they changed from 2000 anymore :-)
The Microsoft Access Conversion Toolkit will give you some of the information you want, and can be used to query MDBs network-wide. If you just need to figure out the mess of a single chosen database, start using the built-in features of Access. Check the relationships and see if anything has been diagrammed out for you. If it has, then you have the ERD ready. If not, have fun figuring it out... Use the stuff in 'Tools->Analyze' to get more property and design information. Try right-clicking on a table/query/form/report and selecting 'Object Dependencies'. This will allow you to see what requires it to work, as well as what the object depends on. Lastly, I just start working through the code/macros (yuck). The object dependencies stuff won't check macros or VBA, so you have to check manually. Sometimes you'll find DAO/ADO code opening connections programmatically.
Best of luck to you! This will suck badly, in case you didn't figure that out already. Access provides an upsizing wizard that can help you upload your data to an MS-SQL server, but that will require you debug (ADP as frontend) / rewrite (VB.NET) the forms and stuff.
Despite what people say, Access does allow for security rights. However, it is not linked in any way to the machine or Active Directory. You use a modified shortcut to load the database with a security file. It works alright for most things, but there is no record-level security, and it sucks when you have 20 people signed in and you have to update the file. Also, supposedly there are cracks that break that security.
This all leads me to my next point for all who read: DO NOT USE ACCESS AS AN ENTERPRISE-LEVEL / MISSION CRITICAL DATABASE SUITE. Pay for a decent tool/programmer/dba/whatever if you really like your data. This application is just for personal / small-group data storage. There is a reason it comes with Office, and not SQL Server. Thank you.
And also get to know ADO and DAO, they let you "access" all the stuff you need.
but it does the job.
On the menu, pull up Tools, Analyzer, Documenter, and pick your criteria. Access is clumsy and not secure, but it's also what most organizations have. That doesn't keep the rest of us from attempting to subvert from within...
To consolidate the duplicate tables, build a query that replicates grep and/or another that replicates diff, and have fun from there. Somehow I'm sure that you know how to do this.
Even though I prefer to work with other platforms and venues, my Access skills have managed to keep me employed and the cats fed while I decide what to do next with my life.
I suggest alcohol, lots of alcohol.
Snowden and Manning are heroes.
"What tools do you suggest for trying to grok a large Access mess?"
A bottle of whiskey and a bottle of wine. Good luck.
Friends don't help friends install M$ junk.
I second TOAD, having used it on a daily basis with an absolutely massive (hundreds of tables, many millions of lines) Oracle-based system, and it's been the best way I've found of making sense of things. The "Schema Browser" function I find particularly helpful when I know vaguely what column I'm looking for, but not what table it's in. It's replaced a lot of the old "cheat sheets" I used to have pinned to every flat vertical surface in my cube.
I've heard it's a fairly expensive piece of software, but thankfully I don't pay for it. It might be tough to get your PHB to spring for it, if that's actually the case...but I've yet to use or even hear of a better way to work with really complex DB systems.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
Hmppf And here I thought it stood for "luscious"
There are a ton of commercial utilities and add-ons for MS Access (check out Access Advisor at your local bookstore), but most of those are just VB apps or ActiveX controls that just do what you could do yourself with a little Visual Basic. Once you have got the basics down from some online tutorials, Access Cookbook by Kurt Getz is a great investment.
.NET.
MS Access has a large community online, especially comp.databases.ms-access. Google is your friend - just about everything you'll ever want to do has already been done and has VB code examples online.
Here is a thread that has code demonstrating how to dump the contents of an Access database as DDL into text files:
comp.databases.ms-access: Exporting jet table metadata as text?
PS - If you are impatient with the limitations of VBA (aka "VB Classic"), there are Microsoft Office interop libraries that will let you automate Access Databases in
Access is frequently abused in the way you describe. Companies that have Office licenses often restrict distribution of the Access component, even if they are otherwise entitled to it, because of such abuse. Access is a very handy tool for a quick-and-dirty database design, so people use it for that - a lot. Pretty soon, you have little information islands all over the place, designed by amatuer DBAs, and containing gobs of misplaced but critical business data. I believe it is all another Nefarious Microsoft Plot (NGP) because when you switch to the solution for cleaning it all up - SQL server - your need for the software is so severe that you won't kick about the price, and expectations for performance are so low that SQL server easily passes muster. Of course, that's just the snide opinion of Yet Another Microsoft Detractor. 8)
"Even if you are on the right track, you'll get run over if you just sit there" - Will Rogers
Take the time NOW and convert everything to ANSI compliant SQL and then rely on front ends to do ALL non ANSI stuff.
:)
Do it NOW.
When your employer wakes up and decides not to remain with that closed proprietary bug ridden M$ stuff, they'll thank you profusely. Or they'll fire you and you'll come to my employer for a job and get hired right in.
--- Grow a pair, liberals... stop letting the Republicans bully you!
I've used ADO's OpenSchema method in the past to get raw lists of table schemas and relationships, including field descriptions.
Fdisk, Format and reinstall. Next time try the SQL desktop engine or Oracle.
The company I work for runs all of it's POS systems off of MS Access and it's given us nothing but problems. I would suggest switching to anything but Access. If you knew the number of corrupt databases and table property dropping we deal with over a months period (and that's only 72 stores worth, or 72 instances of Access) you'd think twice or think about switching immediately. We are constantly compacting and repairing the databases and it just generates a lot of extra work and takes up time that would better be spent fixing the POS code itself. Steer clear of Access if you want to keep large amounts of data (as it seems you do with that many mdb files and tables). People have argued with me over this, but Access just isn't meant for large scale enterprise applications. If you were a small business I could see it being appropriate but nothing above that.
I will forever be a student.
... I was forced to learn access.
The course was taught by "the new guy" who was a *NIX person (of course he got to teach the M$ classes and not the *NIX classes which would have made sense).
Anyway, one class he was showing us how to use objects, etc with what he had made prior to class... it didn't work. He made some comment like, "This worked 10 mins ago in my office." and tried to figure out what was wrong. A few french curses later and we got, "THAT IT! Class is over." And he walked out.
After working against access during the course I found that his reaction was appropriate. In fact, a class-mate that I worked with and I found out that access decides almost randomly when an object exists or not. In the end, we figured out that it had some relationship to the running context (which it shouldn't have), but I digress.
Since then I have purged all access knowledge from my mind. I think I'm the better for it. Or at least my sanity.
So, my advice would be to figure out how the system works, and convert it to a different DB. Maybe PostgreSQL? It's not like management will be able to figure it out.
UGH. I feel your pain, man. POS is right. No chance of migrating to MSSQL? You could rebuild the front end to access a local SQL server express DB then replicate changes to a central database for ease of reporting.. I know, easier said than done, but it would be well worth it to ditch access.
But NOT 200 tables in 30 mdb files. Way too much weirdness. Obviously, this ...thing...needs a complete redesign from the ground up. Whoever allowed this clusterfuck to happen needs to be shot.
Whatever tool you decide on for the back end, (Oracle, SQLServer, Postgres, whatever), I've had great success using Access as a front end. No data stored in it, but just the GUI and some queries/procedures/functions.
You could try using mdbtools ...
Access is about as simple as relational databases get.
All you have to do is learn SQL, then you will actually understand the implications of what the graphical 'query builder' is actually doing for you.
If you are not willing to do this, give up now. It will save you much frustration later, when you don't understand why something isn't working as you expected.
Access is actually quite a nice product when used as a kind of RAD tool. Nested forms are a particularly useful feature. It is one of the few Microsoft products that I have used that is genuinely 'best of class' for its narrow purpose.
Access is really a VERY simple database to use (and in many ways very limited). I would also suggest learning some VBA, as eventually, if you really want to do anything with access, you will need to write modules, or code attached as event handlers on some of the widgets (although there are prebuilt versions for common actions like moving through a record set).
Try Tora. It's a great database admin tool. There's a couple things Toad will do that Tora won't, but I've found it to be a nice substitute.
Free
Cross Platform
Open Source
supports Multiple Databases
Really quite a nice application.
I find that converting the DB to an Excel spreadsheet works wonders for my understanding of the DB.
.
.
.
I know this is posted as a joke, but I am serious.
.
.
.
See our company is laying off 10-15% of the workforce. And who do we NOT layoff? Yes, the DB manager who costs us $8 million in budget overruns (this is while we are cost cutting), and forced the most experienced guy in the DB group to quit. Yeah, she (the DB manager) stays. 100s of entries just not there in the DB. When I have a senior management presentation is have to generate 100s entries myself. They are generating the weekly reports by hand. I have no clue what is on my docket, because the DB is just F'ing wrong. So, when I need info my first act is to port it to something her hands haven't touched, and I can see what is missing. After all if you can't run reports it might as well be a spreadsheet.
.
Yeah, her job is safe.
.
Wonder why the stock is at $17.41 today. Yet we made ~$9 billion in profit in 2005.
Run away. Don't waste your time on these train wrecks. Just like 90% of the VB6 projects out there, it's a lost cause. You'll only end up loathing Microsoft for building tools that attract people with no skills to attempt to build things that should be way out of their league.
Access has it's place. Ignore the OS zealots here, in the real world Access is a viable solution in many business contexts and there is no direct OS equivalent that comes anywhere close, besides if you try to convert it all to LAMP you'll most probably go out of business in the time it will take you :-).
:-). Both of these are time-consuming to replace by a browser interface: the Forms will need extensive use of AJAX to reproduce the immediate responsive feel of an Access application, and Reports can take a lot of coding and even then reproducing pagenation and so forth can be problematic.
OK, firstly you need to convert the data from Access to SQL Server. This is essential: Access is really an excellent front-end system, but it's data handling sucks big time in a mult-user situation. The upsizing wizards included with it do a pretty good job, although you'll undoutably want to tweak manually. Upsizing then reviewing the database and adding relational integrity and other database rules is an essential first step. If you can't afford SQL Server, which is probably on of MS's best products, the MSDE will do if you don't have too many users. It's also a little known fact that Access will run as a front end to Postgres - although I've never tried this myself if you Google for it there's quite a few resources out there.
Having upsized the database you then have two choices. Review and modify the upsized MDB front-end, or create a new ADP project and convert. ADP's have some advantages, but you have to convert manually and rewrite the data access code. This does take time (although a suprising amount can be cut and pasted from between MDB and ADP). The choice here is heavily influence by how the old database has been written - I've seen some Access applications which are practically VBA applications and need to be rewritten susbtantially to use SQL Server even if left as MDBs, whereas others hardly need any changes at all. If you are picking up the application to support yourself and can afford the time the ADP approach is probably prefable as you'll get to know the code and iron out any junk.
You should indeed consider what use you can make of LAMP (or more accuratly WISP). Access's strengths lie in it's ability to support detailed responsive Forms for data entry and most particularly it's Reports where complex output can be generated remarkably quickly - generally it's RAD abilities blow LAMP and similar away for anything but a simpler application (and Ruby on Rails, that includes you
However in a business context it's quite usual for there to be a core group of users who are responsible for data entry and 'expert' use of the system, and a wider group of users who need just read-only access or some very simple data entry, generally for a limited number of screens. If this is the case it's a viable strategy to replace Access by a browser interface for these users. PHP runs happily on a windows server so all your LAMP skills can be applied quite readily. THE major advantage of replacing Access for the casual users is that you then no longer need to deploy Access, which will save you both licence money and support time.
If at a later date you have the time and motivation to convert more of the core user functionality from Access to browser then you can do that. A viable strategy is to convert the Forms but leave the complex reports in a Access as a 'reporting suite'. In many business setups it's quite common to find an 'expert user' who is capable of creating bespoke reports in Access. Handled correctly these people can be a valuable asset - generally I create an 'Adhoc' or 'Scratch' Access application for them (mdb is strongly preferred in this case so that objects are not created on the server) which they can use to generate bespoke reports. The core functionality is placed in a separate Access application which they do not modify.
Take-home message is to recognise that all these technologies have strengths and weaknesses and play
If your bosses will shell out for it, then Visual Studio 2005's Integration Services can take data from any number of MDBs, Excel files, text files, databases, etc and transform them into whatever you want. We recently used it to move a creaking Access DB (1 table, 165 columns!!!) into a (temporary, and slightly more normalised) SQL Server schema - we had cursors running to generate keys, data cleansing procedures plugged in, and best of all it ran at the click of a button, so we could test it very easily with copies of the data before the final rollout.
"If he were a plant, people would roll him up and smoke him."
Evolution of unanswerable questions:
Middle ages: How many angels can dance on the head of a pin?
1960's: How many roads must a man walk down?
2006: How do I make sense of Microsoft Access?
Quidquid latine dictum sit, altum sonatur.
We have a tool that's used for code review logging. It's written in Access (don't ask), by a developer who did it in his spare time.
The guy who wrote it didn't have a clue -- he ties fields directly to the DB (using the DB controls), so there's no transactions, you can't undo, you can't say, "No I didn't mean that... cancel".
On top of that, the guy couldn't design a UI to save his life, so users are continually corrupting the database, overwriting records when they mean to create new ones, etc.. And no, we can't change it, we're subs to the prime, and the prime has mandated the use of this piece of shit.
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
Take all the access .mdb files and dump them into an MS SQL 2003 database. Take that and scan it with Visio 2003 and create all the ERDs. Use Visio, or your favorite ERD editor to make a real database design (one that conforms to at least 3rd normal form).
Once you have a real database design (implemented in an modern RDBMS) and all the data in MS SQL than you can transform the data to fit the new ERD.
Neal
I'll agree with most of what Cruchan has to say, but want to add in a perspective that I've found important. Almost any business case that is capable of making good use of Access's capabilities is also capable of outgrowing Access's utility. Let me be more specific.
.mdb file and re-creating the data interface in HTML forms. The first is easy via an SQL dump, which Access supports. The second is more difficult because what you're essentially doing is Industrial Engineering.
Access's primary strength is that it allows a novice but intelligent user to store data in a database and create views with which they can examine, alter, and add to that data. This makes it very attractive for many small business owners to create methods of keeping track of customers, sales, products, whatever. If your business stays small, then you've saved yourself a bit of money and solved a paperwork headache.
If your business (venture, hobby, whatever) grows, then you will invariably run into Access's limitations. It's very easy to use, but a database program of any complexity will eventually run into programming errors that result in data corruption. They aren't everywhere, but when you run into one you're pretty well hosed. Microsoft may have fixed all of these kinds of bugs between their 2000 and 2003 release - I've somewhat gotten out of that kind of business - but I somehow doubt that Microsoft has changed its philosophy that much.
When you do run into that kind of problem, you have two choices. Keep a second paper trail of all of your changes so that you can fix the database when it hits that bug, or pay someone to migrate you to something more reliable. You would think that there would be an option to pay someone to fix your Access implementation, but by the time someone is willing to shell out money for this kind of thing you can pretty well guarantee that the flaws are in Microsoft's software, not in anything the user has done with it.
So on to the user's question - what do you do when you hit that tree and fly through your windshield? You have to remember that Accesss is a front end database management tool, not an actual database. What you need to replace isn't the actual storage of the information, but the routines that alter and display it. LAMP is an entirely viable idiom for this kind of change, even in a Windows environment. I run a LAMP environment on my laptop so I can develop and show off my web site designs while not online, and it's very reliable. Additionally, it allows remote access of information from many locations, although it takes a bit of skill to write something that can be altered from many locations at the same time.
I don't suggest WISP simply because any further growing will either lock you into Microsoft tools (many of which are highly suspect) or result in tedious and expensive searching for obscure features that allow you to attach other people's tools to the Microsoft Architecture.
Moving away from Access involves two things - migrating the data out of that
When you're talking about a 200 table database, you can quite readily start with the Database Documenter. It'll spit out a bunch of stuff that'll tell you what the formats of the tables are, but won't tell you how they're hooked together or what they're used for (unless the person who made the database was very, very professional). Then you need to have someone show you how the database is used. From that you can figure out what the inputs of the data process are, where the information is put, what is done with it, and what form and place it has to exist for it to be viewed and outputted.
From that you can generate charts that show how the various tables are connected, identify what the rules are (all cars shalt have a color), and get an idea for process flow.
No, this isn't easy. For really big databases it can take months. I was on a project where we were attempting to reconcile seven regional AAA databases into a central database and the entire project collapsed for the inability to get someone to tell us how the data was used. There just isn't enough information in a database dump to determine this.
Best of luck.
Wake up - the future is arriving faster than you think.
Until you get that POS replaced or moved to a real database, you can automate repair with a tool called jetcomp. It can repair many databases that Access's Compact and Repair and DAO's CompactDatabase method can't, and can be automated from the command line.
I went so far as to write a script to detect when the database has been corrupted by attempting to open it and checking the error message if it can't, and automatically backup and repair it using jetcomp. Our corruption problem finally went away when I found a system that had a slightly bad network connection and fixed it. Network problems become more apparent if you specify a larger packet size when pinging, like 400 bytes or so. We went from almost daily corruption to no corruption in over a month.
Yeah, you're very right. We actually use Jetcomp as well and it works much better than the other tools we have available to us. Network problems do always seem to be the culprit as storms and such hit even with surge protection they still get knocked out sometimes. Cutting the power in the middle of using the database and its processes can damage it no matter what. Not to mention the store employees aren't exactly computer savvy. When anything bad happens they hit everything and anything available to them(any options in error messages as well as the turning off and hitting any buttons on any devices in the go between) and that screws things up quite a bit. With all that in mind we are migrating to the free version of MSSQL here in the coming months so I suspect most of that will become a thing of the past (Dear god I hope so).
I will forever be a student.
If the former programmers were "macro happy" it is very difficult to decode, but with time, a lot of paper to map it out, and even more patients, you can get it done.
Good luck with your project. If you need any help, reply to this message and we can talk.
Randy
It's a common compromise in your situation.
You have too much code in the MDB to just dump it. But the Jet database just blows for anything but single user with a rock solid connection to storage.
You already own the MDBE (or SQL express or whatever it's called now). Use the upsizing wizard to move the data out of the MDB to an instance of MDBE, touch the login code (think about roles, don't give the access client code admin on the database), debug the things that break. Most things should work (poorly) executing the querys in Jet but getting the data from MDBE. For performance some querys will have to changed to passthru and hence rewritten in MSSQL parlance.
It's really not that big a deal and turns the evolution of the beast in the right direction. They stop digging. I've been there, but worse.
Start today. You should have a running Access frontend/SQL backend working in a matter of days, working as well as the Access version did in a couple of weeks.
You're main learning curve will be the admin tools for the backend. Don't let them scare you, this is stuff DBAs can manage so a good coder can grok it quickly (don't let yourself get drawn into learning to code in SQL yet). Let the upsizing wizard do it's stuff on few throwaway database instances (selecting deferent candidate tables for the backend move), look at the result in SQL admin, repeat untill all data is more or less where it belongs. The upsizing wizard will leave you with an MDB full of Access code/reports/forms and attached tables that look and act a lot like access tables. Don't worry about the voodoo that access is doing, just see how much of your app works. You might be surprised.
Only after taming the beast by taking it's data from it do you get to slay it.
You get to pick the tool(s) to replace the Access frontend with. HTF do you use Access in POS? PC on the counter? Managers reports?
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
Third normal form (absolutely no duplicate data) means you can't keep total on invoice. You must query and total line item. Line item in turn must query the pricing history table to return the correct sales price for the invoice date.
Putting a total on invoice and a price on line item both violate the third normal form but are still usually good ideas (there are exceptions where line items are added/removed constantly).
Immediatly fire anybody who speaks of normal forms higher then the third UNLESS they can produce a cognant arguement against de-normalizing thoughtfully. Those who speak of higher normal forms are usually net-negative producers.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
...have you had it totally, irreversably trash a database file yet?
The rest of it looks relatively simple and easy-to-use but is ever-so-prone to making exactly the kinds of spaghetti-farms which the OP is asking about.
AFAICT, those spaghetti-farms are a lock-in policy done with more stealth than is usual for MS. It's likely that the cheapest, most effective answer for anything beyond an instant fix is the total rewrite (in something standard and comprehensible).
Got time? Spend some of it coding or testing
This product, Total Access Analyzer,l
m l
http://www.fmsinc.com/products/analyzer/index.htm
has helped me get my head into new db's on several occasions.
Also by FMS, their Detective product
http://www.fmsinc.com/products/detective/index.ht
is great for figuring out where tables/code have diverged.
smile, it makes everyone else wonder what you're up to
If you have Visio Professional 2000 or higher you can Reverse Engineer the Access database into a printable ERD that will help you start finding your way around.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
- Pan Galactic Gargle Blaster
That about covers it.It must have been something you assimilated. . . .