As an exhibition of notorious magician John Dee’s books continues, Philip Ball argues that he warrants a place in science's history