Not the spells, but how and why magic works in the first place.
For non magic worlds, it's creating unique "races". For my home brew ttrpg system and world, I've come up with a few that I'm extra proud of that I've never seen anything like. There's one that I built from the ground up for my favorite player (shhh, DMs do have favorites, but it's a secret) based off of a picture in a magazine, which isn't totally unique because of that, but it's still one of my better interactive bits in that world.
Not the spells, but how and why magic works in the first place.
Emphasis mine; interested here. What about working on the "why" do you like the best? Is it the work of placing in "inherent" limitations, or the lore that you can work behind it, etc?
Well, when you look at our world, there's a deep underlying set of rules. We don't fully understand them, but they're there. Like, we know that pretty much everything in existence is the interaction of teeny tiny particles (this is shorthand, it's more complicated than that) exchanging energy.
The deeper the scientific method takes us into that, the weirder it gets, and so far all that's happened is uncovering yet another layer of things.
Quantum interactions look like magic
So, when world building for fiction, which is always a game of "what if" taken to an extreme, there's an opportunity to play with reality itself, to tear apart what we do have a good idea of and flip it around by replacing assumptions and premises.
Magic, at least in most forms it takes in creative works, bends reality. It breaks conservation it matter and energy, it defies what should be, based on the real world. So you get the chance to take that idea and tear it apart.
As example, if magic exists, how and why does it allow for things like the seemingly spontaneous creation of matter, ala something like a spell that creates a ball of acid. Is it transforming the air into acid, via rebuilding the atoms and molecules already there? Is it summoning the acid from somewhere else, and if so, where? Is it opening a portal to an infinite reservoir of acid on another plane?
By deciding that, by examining how you want magic to work on a fundamental level, you literally build your world from the smallest factor upwards. When you're doing that, the process essentially etches the ideas into your world as you build everything else, and it imprints on you, giving you a level of oneness with your world. It's like how learning a new language builds pathways in your brain that give you new ways of thinking, of looking at things.
The lore, the types of magic, the "races", nations, whatever, they all spring from that fundamental core of why magic is and how it forms the basis of that world.
It's something that you really don't find in any other kind of creation. World building is a form of art in and of itself, but one that gives other forms a place to stand. You don't have to world build that deep to write a book, or paint, or run a ttrpg, but there's always something missing if you don't.
Mind you, it is entirely possible to simply accept someone else's world building, or to use reality as that world, but doing so means learning from the outside in rather than the inside out, so it feels different
I'd probably say mundane ergonomics (eg.: how do your species use chairs, or if doors have knobs or bars) and general interpersonal processes: how does one find a mate, or raise a child in the setting, how are liars and cheaters treated, etc.