The Cost of Magic?

If I were worldbuilding but wanted to avoid making magic inborn, (eg. "Mages are born with The Gift/The Source/The Awakening, etc.) what are some costs, restrictions, or otherwise dangerous setbacks to learning magic that might explain why every person doesn't study and practice wizardry to at least some extent?

Pact with mysterious entities of vague and mysterious (and possibly nefarious) purpouse.

There is a conspiracy of archmages to limit access to magic, but for entirely altruistic reasons: mixing magic is chaos.
Whenever a spell is cast, it releases an invisible cloud of waste mana that can take weeks or months to fully dissipate. Spells cast in areas with a high concentration of waste mana have random side effects, getting more severe and bizarre the more waste mana there is, and it gets MUCH worse when multiple spells directly interact.
Sure, you can have your magical industrial revolution, you can have a society where everyone learns basic magic and uses it constantly for frivolous things. That society is known as the Ancients, who are all dead from the side effects of magic, who are responsible for the world being so shit today and the existence of several types of monsters. NEVER AGAIN.

Conservation of energy baby

More powerful the spell the longer it takes to perform "must be said" incantations and after that words are forgotten. Thus forcing the most powerful mages to spent their entire lives relearning the spells.

This creates balance, which allows even normal people to effectively battle mages, because they are one-shot killers, then out of ammo for sometime.

Some spells can be prepared befpre hand, integrients bought and prepared but all in all it comes to incantations.

The ritual needed to access someone's latent magical abilities requires reagents that are exceptionally rare, such as:

>gems found only in the deepest mines
>alloyed metal that takes years to prepare
>branches from a single mystical tree
>the bones of a mage who died from natural causes
>raw mana, drawn into physical form by a sufficiently powerful wizard

Material costs. May extend to your own body and soul if you push it really far, or a strong spell backfires. Under these conditions, learning from your mistakes comes at a price too high for most to pay.

Or maybe the magic's loaned from a third party (like a deity, local spirit or demon) with ulterior motives or a fickle nature that don't make the deal worth it unless you're either desperate or in an excellent bargaining position.

It's risky. You don't have to go full Warhammer with this, but making miscasts and other failures dangerous is a good way to dampen it's popularity; almost no one would choose to be a computer programmer if there's a 5% chance your computer will explode violently or something every time you run the compiler.

It's focused. Magic doesn't do everything. It either has a focus (e.g. mind magic only) or has a significant hole in its control (e.g. can't affect anyone wearing anything made of steel). This not only justifies why everyone isn't a wizard, it also makes characters that are wizards not as OP.

Speaking of which, I'm going to go off on a tangent real quick. Making magic/mages rare does NOTHING to balance them against noncasters. As soon as there's a mage in the party, the rarity of magic is moot; they've got a magic dude, so what the setting says is normal is irrelevant. If anything, it makes PC mages more powerful as most systems have the only hard counter to mages be other mages.

It's subtle. Magic doesn't call down thunderstorms or conjure fireballs until you've dedicated the better part of four decades to mastering it. Instead, magic is a subtle force that normally needs others to help make it work. A typical mage can't fireball their way through an army of goblins or conjure up a wagon or produce food out of thin air, but their spells *can* give sizable advantages to an allied fighter, grant a carpenter steady hands and strong arms, and help secure a good harvest for farmers that season.

It takes restricted resources. If magic needs to be focused through a special crystal or burns stardust or something, and those resources only come from one area and their production is limited, everyone *can't* become mages regardless of how much they want to.

It's slow or takes special areas. This is more of a limitation for magic adventurers who often operate on strict time schedules like "I have three seconds till that orc bisects me with a sword."

Magic is extremely dangerous. The reason why the stereotypical mage is a bearded old wizard or a hunchbacked witch is because it takes a very, very long time to learn how to use it without blowing yourself up. Most people don't have that sort of dedication, and quite a few of those who try end up dead.

Magic that uses easily accessible emotion energy alters mind. You're angry and use that to make a fireball? If you keep goin' like that, you'll have a shitty temper. And then become a rabid pyro.
Magic that uses non-easily accessible Aether energy is just "good luck getting the needed number from RNG". Magic missile? Roll too low and it's a small magic arrow. Roll too high and it blasts AoE, maybe taking the mage with it.

Lots of up-front monetary costs and loads of paperwork to get your wizarding license. Not a lot of people have that sort of patience or money.

In my own setting, magic is very painful. And not just "for you", silly Baneposters, but for anyone learning it and using it.

Mechanically speaking, this means that when you cast a spell, you take the spell point cost in nonlethal damage that cannot be healed with magic, or convert the spell point cost's worth of nonlethal damage to lethal damage that cannot be healed with magic. The former is called Psychic Strain, and the latter called Mana Burn, which is quite literal: even the winner of a mage's duel may leave the field with smoke rising from the welts on his skin, not from the spells of his enemy but from his own magic. This replaces the normal Vancian system, such that your spellcasting resources are your hit points: use them wisely.

Thus for most people, magic is not worth the struggle. You study hard, you take painful burns, and all for only a few minor magical effects at first.

Besides that, I removed a lot of spells, such that magic does not make you do everything better than non-magic. In game terms, I wanted to remove all "boring" spells, where "boring" refers to anything modifying "the numbers": initiative, saves, DR, SR, AC, skills, HP (including temp HP), and the like, unless attached to another, more interesting effect. This effectively meant that the "buffer/debuffer mage" (which was the most effective spellcasting strategy in 3.5) does not work. Magic is a strange power, and not every person can apply it usefully to their everyday lives.

Furthermore, magic is restrictive. The state of mind required for each sort of magic is unique, so each person can only learn a closely-related subset of spells.

Finally, magic can drive a person insane. Instead of "mad as a hatter", people say "mad as a wizard": psychological damage is part and parcel of spellcasting.

tl;dr: magic is bad for physical and mental health and is difficult to learn, all while not being entirely useful in every situation.

Magic has a concept of "Equal and opposite reaction" or "Equivalent exchange". Most of the time, this is not a big deal. Creating a brief runaway exothermic reaction (IE, throwing a fireball) will just leave you hot, sweaty, and hungry, and might make you collapse if you push too much. Charm magic makes you feel bitter and grumpy and bitchy. That sort of thing. Grand, long-term magic, on the other hand, rarely has a cost up front unless you deliberately design it to, which is a really good idea, because magic will extract its price one way or another, and if you're skipping on the bill too much, it'll take its pound of flesh at the exact worst possible moment, generally in a way that makes the entire thing self-defeating or worse. Your house on chicken legs develops a need for blood sacrifices to keep it going, your food-creation circle starts eating the life force of people who eat from it, your flying city starts draining magic for miles around.
Half the study of magic is paying a huge up-front cost (Years of your life, your health, your relationships, your youth) in return for safer magic. Most people simply aren't willing to make the kind of sacrifice magic demands on a routine basis.

didnt someone make a story about a wizard explaining that gold coins = magic?

all magic has a price

everything you "buy" with magic, you have to "pay" for it, sooner or later. not with money necessarily, but with something of value. for simple magic, it might be just some gold or silver, or sacrificing a goat. for powerful spells, we're talking human sacrifice, precious memories, your own identity, things like that.

most ordinary folk avoid it the same way they avoid borrowing from loan sharks. it's easy to end up over your head in "debt". you can buy all sorts of things with magic, and then find yourself having to pay it back with interest. you have to be very patient and skilled to get the most out of magic while paying as little as possible. too many mages get drunk with power and then crash and burn when reality asks them to cough up. some are forced to resort to desperate and evil methods to keep themselves afloat.

Money is literally magic.

You have to actually collect the energy for the spell.

Basic cantrips work off food, so mages carry extra calories around. Before big battle, the mage just sits in a corner an shoves a bag of suggar into his mouth.

But for great spell, you need a long term power source. A wealthy mage may buy land at a river and built a mill-like aparatus, another one becomes a demi-god by eating an active volcano for 500 hundred years. And the BBEG tries to harnass the force of a comet, but it slowing down will cause a collide with cataclysmic consequences.

Add requirement to store those powers in some material and normal people have a way of fighting wizards by sabotaging their supply.

Holy magic requires an incredible amount of piety and comes with a constant temptation to fall to darkness.

Unholy magic is dangerous, slowly corrupts the user, and makes you fugly.

Elemental magic is rare, limited in utility, and passed down carefully within certain subsets of societies who live in the farthest, most inhospitable reaches of the planet.

Also, most magic takes from a minute to a week to cast, not five seconds.

Rumpelstiltskin? Is that you?

Spells cost materials you have to gather and mix, it takes time to memorize chants and incantations and gestures, some spells require certain days, celestial alignments, times, etc. to even work. Being a wizard is actually, for all the wonder and versatility of magic, fucking frustrating. It takes a long time to become proficient in this field due to level of work it requires to be at all remarkable.

Martials can wade into battle, swords and axes a-flying, rangers can leap about firing arrows and shit, but a wizard sits back, prepares his spells, does his rituals and lets it all come into play.

The way I see it, low level elemental spells, attacks like gusts of wind, flashes of flame, stuff like that, you can cast those with gestures, words of power, prayers to spirits or combustable admixtures of magical ingredients. You can do that on the battlefield with little need for preparation. But larger spells DO require preparation, maybe you need cover to engage in communication with some outer power or to call down some spirit of the air, maybe for the final confrontation with the big bad, you need to do a ritual three days in advance.

It also gives reason for why wizards would want to be adventurers, going out into the world to gather rare or dangerous ingredients, seeking old knowledge in far off places, visiting places of power. Wizards are always, always prepared with something because of how time consuming magic is, but that all takes ingredients and memory.

So I'm thinking a Highlander-like situation.

Everyone who is gifted SHARES the gift with all of the others in the world who were born with it. Maybe it stems from a few families who were the origin of magic bloodlines. There is only a finite amount of magic power in the world and the more control you gain over it, the more you draw away from the others. When a gifted magic user dies, the power increase is noticeable, if slight.

Therefore, mages are pretty much always in competition with eachother if they care about raw power. This has maybe lead to some nasty wars or genocides against more peaceful or humble magic users. It is also a reason that magic is not something you just play with. There are some who would kill you to absorb your power even for the smallest bit.

The fuel for magic only comes from the caster in a limited sense. The mystical 'fuel' that powers spells - especially powerful ones - are granted by otherworldly beings, nature spirits, and other powerful near-diety type creatures that exist beyond the veil the player's mundane universe. They do not give this power away freely - magi are an investment to them, which they then collect on once they become powerful. This may be by commanding them to do certain services that the magi themselves may be reluctant to do (destroying a dam to appease a nature spirit, which would destroy a village beneath it. Not only abhorrent to the magi, but very problematic legally if he's caught doing it).
Refusal to preform such actions might mean withholding the mystic might to cast spells easily, or even at all. This makes life difficult for a mage who may have spent a decade making powerful enemies who suddenly learn he doesn't have much of his arsenal to back him up [Such as Dr. Strange had to go through at one point, to give a /co related example].
Literacy itself might be outlawed - as it was in various times of history - to prevent the average man from learning things that might be used against the aristocracy (like weapon skills or making implements of war). Now imagine the fear a Lord might have of some guy in robes just waving his hands and bombarding his balcony with fireballs while he's having breakfast. It might make for an interesting scenario when magi and friends of magi are rebels fighting the local lords.
Policing one's own is also a possibility for this same reason - there are few magi as they themselves try to keep the scope of their powers hidden from the aristocrats who just might oppose them if they knew just how powerful they could be become.
Magi wars in the past destroyed a lot of civilization, causing people to irrationally fear magic in all forms - and magi in particular, making it difficult for them to survive day-to-day.