A small phone is useless unless your only goal with it is receiving calls. Even making calls on a small phone is difficult, never mind using the Internet. And losing a larger one still happens, a small one would just disappear like a sock in the dryer.
Sure, there are times when having a bigger phone makes sense, but it hurts a whole lot less when I'm in bed watching something and i drop it on my face.
A small phone is fine to take photo/video, send messages, make calls, view video content, and play games. Any other use case is more appropriate for another device.
With one exception, I've only had phone that would be classed as small by today's standards for the past ten years, the largest being a Pixel 4a and the smallest being a Sony X Compact. I preferred the size of the Sony.
I rarely make voice calls. I use my phone for messaging, light web browsing, notes, reading, watching videos, podcasts, navigation, occasional gaming, and photography when I've made the mistake of leaving the house without my real camera. I prefer smaller phones because they're easier to use one-handed.
That one exception is a Pixel 3a XL with a broken cellular modem I bought at a low price to test ROMs and mount on the handlebars of my bike where the vibration is likely to damage the camera. I do occasionally use it for media (video and books) where the larger screen is an advantage, but I would dislike carrying it in my pocket or using it for most tasks where I would have to use both hands.