Yea when this came out, not too many saw it. But it was awesome to be at a party or something and find someone else who had seen because you were almost certainly going to vibe out with each other over how awesome it was and what you thought about it and recounting which scenes happened when and how the film links them all together.
Apart from all the magician stuff and the core plot, I think an easy to miss aspect of the film is just how much it seems Chris/Jonathan had learnt from Memento and were keen to deploy in an even better more cohesive and dramatic way. I still think it's astounding the way the various threads of the film are not only told to the audience but told with respect to what the characters are learning at the same time.
Which of course strongly links with, what I think the film is in many ways about, namely the connection between magic and film making or story telling. How many "prestiges" are in "The Prestige" ... were we watching closely?
I'm not a big movie rewatcher myself, but this is one of those movies to watch again because you know the plot. It's extremely well crafted, and there are plenty of significant scenes with hidden meaning or foreshadowing to catch on the second watch if you keep your eyes open.
It really rewards rewatching IMO. You can appreciate the huge amounts of foreshadowing. The movie tells you exactly what's going on right from the beginning. But you're not really looking, because you want to be fooled.