How do you quantify a card's power?

I don't think there's a magic formula, or indeed a Magic formula (cue rimshot, sobbing, gunfire), to calculate the abstract "power level" of an individual card, because power level varies massively depending on where you are.

My go-to example for this is Brainstorm, because Brainstorm's power stems more or less entirely from its favorable interactions with shuffle effects. If you run Brainstorm in a deck without shuffle outlets, it's actually pretty bad as a cantrip - but it's restricted in Vintage and a Legacy and Pauper staple because the selection it provides in conjunction with fetches is so strong.

There's also the issue that card advantage is quite a fuzzy metric and often not a very reliable one. It's certainly something that should be considered while designing cards, but it's by no means the sole determinant of card power. Faithless Looting is card disadvantage - even if you cast it and flash it back, you're down one card from the Looting itself. But it's one of the top twenty most played cards in Modern because it ditches dead cards for live ones, fills graveyards, and fuels synergies. So, while a strong understanding of card advantage is extremely useful, I'd caution you against trying to reduce every aspect of a card's power to some number of virtual cards. Soul Warden isn't getting you card advantage in any normal game state, but that's not what you're playing it for.

This is going to be a very frustrating answer, and I'm sorry for that, but yeah, the "look for similar cards and weigh it up" approach, plus a process of iteration, really is the most reliable way of getting balanced cards. Of course, this is complicated by the fact that some canonical Magic cards are strong or weak compared to what most designers would consider a good level of balance - consider Ancestral Recall versus Jace's Ingenuity, for example, and the eternal oscillation of the poor 3-damage burn spell between Lightning Bolt, Lightning Strike, and Open Fire.

To this end, for Constructed design, it really helps if you're at least passingly familiar with the competitive formats closest to the power level you'd like to design for. You don't need to be a hardcore tournament grinder, but watching some tournament footage from GPs and Pro Tours (there's plenty of it on YouTube) and scanning over deck lists to see what gets played can be a very useful exercise. For Limited design... well, we have plenty of avid drafters who can offer their input, but if you're at all able to play Draft or Sealed, even if it's just the occasional prerelease or casual draft, getting a feel for how valuable certain effects are in Limited is invaluable.

It does get easier with practice. Over time, you do develop a more instinctive feel for how strong some effects are, but it's less mystical talent and more being able to apply a set of thought experiments and design metrics more quickly. It's a cliche, I know, but the best way to learn is by doing.

... that said, I feel we're severely lacking in resources for designers who need help with balancing, because custom Magic writing tends to focus so heavily on inappropriate colors and rarities. I'm looking into putting together some pieces along these lines, and I'd encourage other experienced users to do the same.