Ask the Designers of D&D Fourth Edition
This past August, big news dropped in the tabletop gaming community: 2008 would see the release of a fourth edition of Dungeons and Dragons. Since then the official D&D Insider site, and communities like the excellent ENWorld, have been doing their best to keep us up to date on the ins and outs of the newest way to dungeon-delve. With the release just five months away, we've been given a chance to put some questions to the team developing the game. One question per post, if you would, and we'll make sure to pass the best questions on to the designers. Don't forget to ask about the online version of the D&D tools as well! We'll get their answers back to you as soon as we get them, so fire away.
It's been done. It's just not in the games you're familiar with, apparently.
;-)
A magic system for Fudge which works much the way you describe
the system in Ars Magica is quite similar
Here are some discussions about magic systems:
a discussion of different systems
another discussion, led off by Ron Edwards of the Sorcerer RPG
Speaking of Sorcerer, its magic is something else entirely. It's a largely outcome-based game rather than specifically action-based, and the magic system in it is quite a neat play on that.
GURPS, Rifts, and D&D pretty much follow the mystical grimoire approach. Ars Magica, White Wolf's Storyteller Series (Vampire, Mage, Werewolf, Wraith, Changeling, Hunters, etc), and some others take the combined skills approach. Still others have wholly different approaches. Here's a pretty good explanation of the theory of magic in Earthdawn which explains different ways magic can be used in that game, complete with disadvantages of some of them.
The Forge is very interesting reading material for anyone who's considered writing their own RPG. There's some advanced RPG jargon there so I'd suggest starting with the site glossary. It's not a site for arguing the merits or faults of different systems you've played although those might be used as support in discussing the design of new games.
Personally, I've played games with set spells, spell research to make new spells (as some versions of D&D let you do with the right GM). I've played ones that require a combination of skills (from two to five (five!) skills for every casting. Some require each spell being taken as a character advantage in an advantage/disadvantage slot balanced game. I've played on in which the game world has special words that are foreign to the players/characters that must be learned throughout the campaign which represent factors of a spell (speaking "large" + "fire" + "ball" + "at" + character's secret magic name results in that) and learning the words as an outsider is how to become a better mage. It becomes the whole point of some adventures.
I've even play tested one unpublished game in which the only magic was a link between two symbols dawn during a ritual trance. However, the link was so strong that whatever you did to one would happen to the other. You could talk into one, and someone in possession of the other could carry on a conversation with you. You could throw one safely in your fireplace while the other is inside an enemy's barrel of oil. You could lay one on the ground and step on it, and be transported to the other. However, if anyone unfriendly took over your other symbol, they could use it in reverse until one of the two was destroyed. If I ever give this game a name and publish a book, I hope you'll rush out to buy it.
So yes, there can be quite different magic systems in games. Many of them could be used in D&D, or you could try the other games.