Old-school D&D players don't seem to dig the concept of a prestige class very much, probably because it makes character generation more cluttered and build-y and blah blah, I'm sure if you're reading this blog you know all that already and if you don't, well, there is surely someone posting about it on RPGnet at this very moment.
But: I think there is room for the concept of 'extra abilities/training that you tack onto your character' in OSR/DIY D&D play. Look at it this way: my FLAILSNAILS character Blodgrist has developed x-ray vision from drinking the blood of a mutant crab, and has a Sword of Rage that he stole from the goddess Kali. Both of those things arguably define his character more than his class (Fighter), but they aren't things I picked so much as they just happened to him (though I still had some agency because I chose to drink the blood and to steal the sword). And other people have been turned into talking dogs or cursed with the grief of a million mothers or some other thing. And there is this thing where Ian talks about how 'Paladin' can be something you sort of lay over the top of 'Fighter' on a temporary basis.
What I'm getting at is that Prestige Classes could be not too dissimilar from curses, magic items, mutations, and other things that happen to your character to make them who they are - which is kind of the core principle of character development in OSR, right? But there would be some differences from the PrCs in 3e or the Paragon Paths in 4e:
A Prestige Class is diegetic. Finding a prestige class in a splatbook in no way entitles you to pick that class. If you want to train as a Monk of the Spiral Fang then you don't buy the Ultimate Monk Handbook, you actually have to trek to the Spiral Fang Monastery and probably perform some quest to prove your worthiness, and then undergo intensive training for six months before you come out knowing kung fu or whatever.
Now, if you see a PrC in a book somewhere you can maybe convince your DM that this class exists somewhere in his/her world, but then maybe not because...
A Prestige Class is usually campaign-specific. Just as you can't expect every campaign to have a Deck of Many Things or a mad wizard who will give you mutations from Gamma World, you can't expect every campaign to include every prestige class. And furthermore, just as most DMs like to make up their own magic items, many of their prestige classes will only exist in that world. So if I'm running a Viking campaign maybe I cooked up a Berserker PrC, but someone else's science fantasy game will have Alchemical Gunsmith and a city-based game could have the Urban Ranger. But they could also get much more specific than that, like even 'Farseer of the Obsidian Tower' or 'Gullet Scavenger' 'Most Holiest Jester-Mendicant' etc.
I think this would be cool for FLAILSNAILS games because it would be another thing that you've picked up on your weird picaresque adventures through the multiverse. "Yeah, I got this sword in the Vats of Mazarin, this scar comes from a thoat stampede, and I learned to kill a man with a sewing needle from the Metassassins of Kwishtar."
A Prestige Class is mysterious. You can't plan to build your character around a certain PrC because you mostly don't know which ones are available. You go looking around the world and then you discover what's out there. The DM is not obliged to give you a PrC that you like any more than he/she is obliged to give you a particular magic item. (The DM can incorporate players' suggestions if they're good and fitting, though.)
A Prestige Class can have any rules that you like. It's DIY, so do whatever you want. Balance? Pfah! I spit on your balance!
A PrC could be just something that you fulfill the requirements for and then you get it. Like if you wanted to become an Animagus from Harry Potter, you don't take levels in that, you either are an Animagus or you aren't. Alternatively, a PrC could let you 'spend' XP to gain levels in that instead of your regular class. Presumably the amount of content that the DM wrote for the class would set a limit on how high you can go. Personally I'd prefer to limit them to 1-3 levels. I like the idea better if your base class remains the thing that you really are, and the PrC is just something on top of that. So maybe a general rule that you can't have more levels in your PrC than you do in your regular class.
A PrC could have stringent requirements of stat, class, race, etc. Though you should be careful not to have the player travel all the way to the monastery only to be told, "No, you need 13 Wisdom to join our order, you only have 12."
A PrC could happen due to wholly diegetic reasons - for example you could become a paladin because a priest charges you with a holy mission, and you don't have to expend any character resources on that any more than you'd pay extra to be able to wield a magic sword once you'd acquired it. Or it could be something that costs XP or levels - perhaps it's paid in advance so you get your powers as soon as your training is complete but you don't get anything the next time you level up.
Realistically you would get special training by actually spending time on training, but that's pretty boring unless you're already playing a game with lots of structured downtime.
I guess my vision of how PrCs would work is similar to everything else in FLAILSNAILS in that it's a total wild west; anything goes as long as everyone's having fun. If, on the other hand, you were running a closed-circuit campaign, then PrCs would be much more restricted and would serve to tie your players more closely into the setting.