Monday, August 2, 2021 - 11:30am to 12:30pm
Event Calendar Category
Uncategorized
Speaker Name
David Fouhey
Affiliation
University of Michigan
Zoom meeting id
509011
Join Zoom meeting
https://mit.zoom.us/j/99879192114
Abstract
The long-term goal of my research is to let computers understand the physical world from images, including both 3D properties and how humans or robots could interact with things. This talk will cover two recent directions aimed at enabling this goal.
I will begin by talking about 3D reconstruction from two ordinary images where the camera pose is unknown and the views have little overlap -- think hotel listings. Computers struggle in this setting since standard techniques usually depend on many images, high overlap, known camera poses, or RGBD input. Nonetheless, humans seem to build a sense of a space from a few photos. We think the key to this ability is joint reasoning over reconstruction, camera pose, and correspondence. This insight is put into action with a deep learning architecture and optimization that produces a coherent planar reconstruction. Our system outperforms many baselines on Matterport3D, but there is plenty of room for new work in this exciting setting.
Then, I will focus on understanding what humans are doing with their hands. Hands are a primary means for humans to manipulate the world, but fairly basic information about what they're doing is often off-limits to computers (at least in unconstrained data). I'll describe some of our efforts on understanding hand state, including work on learning to segment hands and hand-held objects in images via a system that learns from large-scale video data.
Biography
David Fouhey is an assistant professor in the University of Michigan EECS department. He received a PhD in robotics in 2016 from CMU where he was an NSF and NDSEG fellow, then was a postdoctoral fellow at UC Berkeley. He has spent time at Oxford's Visual Geometry Group, INRIA Paris, and Microsoft Research. More information about him can be found here: http://web.eecs.umich.edu/~fouhey/.