Learning to Map Anything, Anywhere, Anytime

What might it sound like here? How would you describe this place? Would it be unusual to see a large mammal if I took an early morning walk? These are all questions that are inherently spatial in nature and difficult to answer precisely. This talk surveys a decade of work on multi-modal remote sensing aimed at answering such questions at a global scale. I'll cover practical systems built on high-quality manual annotation, as well as more exploratory work that replaces manual labels with geotagged social media, satellite-ground image pairs, and species observation data to learn open-vocabulary, multimodal representations of place. Together these systems point toward a shared goal: understanding the Earth with a level of semantic, spatial, and temporal resolution that was previously impossible.

Host: Carlos Madrid Padilla

Dr. Nathan Jacobs is Vice Provost for Artificial Intelligence and a Professor of Computer Science & Engineering at Washington University in St. Louis, where he directs the Multimodal Vision Research Laboratory. His research develops learning-based algorithms and systems for understanding the visual world from large-scale, geotagged imagery, including images from social networks, outdoor webcams, and satellites. He received a National Science Foundation CAREER award in 2016 for his work at the intersection of computer vision, representation learning, and remote sensing, and his research has been funded by the National Institutes of Health (NIH), Defense Advanced Research Projects Agency (DARPA), Intelligence Advanced Research Projects Activity (IARPA), National Geospatial-Intelligence Agency (NGA), Army Research Laboratory (ARL), Air Force Research Laboratory (AFRL), Google, and others.