The Vancian Wizard, Doomed Science-Hero

It seems there are two different ways of thinking about magic.

The first, though perhaps more recent school of thought, is that magic is essentially a scientific discipline that does not exist in the real world. In the world of fantasy, certain words, gestures, ingredients and so on will produce testable and replicable effects. For example, the same magic words will always produce a fireball when spoken. Magicians may not understand the mechanism behind their magic, any more than the modern-day layman understands the mechanisms of quantum physics. However, they understand the results of those mechanisms and know how to apply them in the art of spellcasting. Therefore, a wizard is essentially a scientist who has discovered that his school of study is especially easy to weaponize.
This form of magic appears in fantasy stories like Harry Potter, Magicka, Eberron, The Wheel of Time, and the Old Norse Sayings of the High One.

SCIENCE!

The second school of thought is that magic is fundamentally different to science. Magic has an undeniable presence in the world, but it cannot be empirically tested. It does not operate by rational laws. It moves in unexpected ways, not because our understanding of it is incomplete but because our entire rational-scientific model cannot encompass it. Magic is the unconscious, unprovable, and other vague wishy-washy concepts that are impossible to pin down or define.
This form of magic appears in fantasy stories like Game of Thrones, The Sandman, The Invisibles, the Conan stories, and many mythological texts. It is also the basis for any form of belief that claims magic is literally real, ranging from Aleister Crowley's esotericism to psychic TV and alternative medicine. Obviously homeopathy is something "beyond the realm of the empirical", because when you test it empirically you quickly find out that it's complete horseshit.

not science!

You'll notice that so far I haven't mentioned two of the most important texts (important to pretentious nerds, that is) of the wizardly canon: namely the Dying Earth and the D&D Magic-User. That's because the Vancian wizard occupies a space somewhere between these two schools, or in both schools at once.

In D&D, spellcasting is certainly an empirical affair. Every time you memorize Magic Missile, you get to cast Magic Missile once, and that spell is always the same. It would appear that magic in D&D is of the 'scientific' variety.
But on the other hand, there is so much more to magic than simply spellcasting. There are any number of supernatural beings with inexplicable spell-like abilities; magic items and artifacts with powers that are impossible to replicate; tricks, traps and puzzles of every kind that defy categorization according to the mean and piddling magic of the spellbook. This is very clear in the Dying Earth stories (where in fact the highest form of magic is not spellcasting but the binding of inscrutable alien beings to do one's will). It's also pretty clear in AD&D, both in the books (the liberal use of 'funhouse' tricks) and in actual play (because no DM worth his salt is going to let his adventure design be limited by some dorky system of magical metaphysics).

So although it's not stated explicitly, I feel like the premise behind Vancian spellcasting is this: that magic is vast, untameable and unfathomable - Type 2 in our survey above - but the system of fire-and-forget spellcasting represents a very miniscule effort by humankind to bite off one tiny part of that vastness and compress it down into a device that works the same way every time - Type 1.

This is why crafting a new spell is such an arduous process, and why many spells are named after the legendary wizard who created them. It's not that Tenser actually invented the floating disk so much as he discovered a very precise fragment of the seething unknown that can be reliably shaped into a floating disk. This was a process of research, but it was done in the face of something that inherently rejects being researched - a process of embattled trial and error, where no assumptions can ever be taken for granted. I imagine that Tenser's laboratory saw a great many Exploding Disks, Horrifying Disks, Cheese Disks and Floating Anti-Disks before he finally settled on the formula that he was seeking.

Even the method of spellcasting evokes a feeling of precarious uncertainty - in order to be a wizard, you have to let an alien meme burrow into your brain, and when you discharge it your memories of the meme will be wiped clean. Every day you memorize the same words and incantations, but every day they vanish from your mind. You can't tell me that there won't be long-term neurological consequences from that. And the very event of forgetting serves to challenge our precious rationality, to remind us that no matter how right we think we are, we are still trapped in a brain made of fallible meat. The original Vancian magic story, Mazirian the Magician, is a story about a man who sets out utterly sure of his intellectual superiority, and is gradually stripped of it - figuratively and literally - until he has nothing left and is devoured by evil trees.

Later, more or less every book, module and blog post would introduce new spells that PC wizards could potentially learn, and many of these spells are pretty great. But I also like the idea that the spells in the PHB are the only spells in existence. The world is ancient, and thousands upon thousands of magic-users have come before you, yet these spells are the only ones they have left behind. Of all the wonders that magic can work, only this handful have ever been bound into words that a human can have control over. No wonder Sleep is overpowered and Affect Normal Fires is terrible! These are not the powers of a well-read arcanist, but the blind fumblings of a thousand generations, crawling on hands and knees through a rat-maze as big as the multiverse.

This, then, is the Vancian wizard: a scientist, a Science Hero, one for whom the sword of reason is the greatest weapon - but tragically, a hero of rationality who lives in an inherently irrational world. No matter how he tries, his efforts are ultimately futile. The incomprehensible chaos of magic remains, huge and indifferent to his most rigorous testing. And one day, when civilisations fall and the flames of war sweep through the land, the books of magic will be burnt; the knowledge of spells will be wiped out; and the little piece of enlightenment that Tenser and Mordenkainen brought into the world will be gone, returned to the endless unknown from whence it came.