For a little while I was like this. Turns out, this may work for cotton and synthetic fabrics, but don't do this for organic fabrics such as silk or wool.
I think it matters more with printed shirts. I've got one I love the design of so I don't run it through the dryer, just hand it over my shower curtain rod. Still toss it in the wash, though.