Learning to Map and Mapping to Learn