Measurement systems, especially time measurement, is the one thing everybody should absolutely agree on for the sake of sanity of everyone. Of course the metric system is arbitrary bullshit but so is every other system, and metric at least tries to be less arbitrary to an extent. Yeah of course the most popular system is going to originate from a cultural hegemon but that's fine as long as everybody agrees on it.
Measurement systems being propagated throughout the world technically would constitute as colonialism I guess but honestly I see no sense in opposing the metric system other than pure spite.
I think that standardization is highly valuable, but there's absolutely no reason everyone everywhere should use the same systems.
I mean I guess there are exceptions like remote tribes living mostly independently of the global human civilization but other than that I can't see a reason why you wouldn't want a standardized measurement system.
The imperial system isn't based on day to day use either though. A yard is the length of some kings arm, not some universal signifyer. The foot is the length of Henry I's foot. An inch is three grains of barley laid end-to-end.
First time I remember my dad using yards was when I asked how far my family's dog could see. She was going blind. He replied 'about five yards' and I thought it meant back yards, and then thought she could see further than she could.
Having actually tried to teach dimensional analysis and conversion factors to students, the fact that we even have two commonly used systems is too many.
We've already crashed a spaceship because of this. The practical need for any group of people in regular communication to adopt a single unit system far outways any ideological objections to it.