 |
|
|
|
|
|
|
|
|
|
|
||
| (a) | (b) | (c) | (d) | (e) | (f) | (g) | (h) |
|
|
|
|
|
|
|
| (i) | (j) | (k) | (l) | (m) | (n) | (o) |
Marc Levoy has retired from Stanford University to lead a team at Google. This project is no longer active as a Stanford research project. However, many of the technologies developed during this project live on in products such as Google StreetView (e), the Lytro Camera (j), and Google's Jump camera (o).
The light field, first described in Arun Gershun's classic 1936 paper of the same name, is defined as radiance as a function of position and direction in regions of space free of occluders. In free space, the light field is a 4D function - scalar or vector depending on the exact definition employed. Light fields were introduced into computer graphics in 1996 by Marc Levoy and Pat Hanrahan. Their proposed application was image-based-rendering - computing new views of a scene from pre-existing views without the need for scene geometry. (A workshop on image-based modeling and rendering was held at Stanford in 1998.)
Since 1996, research on light fields has followed a number of lines. On the theoretical side, researchers have developed spatial and frequency domain analyses of light field sampling and have proposed several new parameterizations of the light field, including surface light fields and unstructured Lumigraphs. On the practical side, researchers have experimented with literally dozens of ways to capture light fields, ranging from camera arrays to kaleidoscopes, as well as several ways to display them, such as an array of video projectors aimed at a lenticular sheet. Researchers have also explored the relationship between light fields and other sampled representations of light transport, such as incident light fields and reflectance fields. At Stanford, we have focused on the boundary between light fields, photography, and high-performance imaging, an area we sometimes call computational photography. (A workshop on this theme was held at MIT in May of 2005.) However, computational photography has grown to become broader than light fields, and our research also touches on other aspects of light fields, such as interactive animation of light fields and computing shape from light fields.
The images above depict some of the work we have done in our laboratory on light fields:
(a) An image from our SIGGRAPH 1996 paper, showing an array of renderings of a 3D computer model of a buddha statuette (at top) and the transpose of the light field (at bottom). (b) The Stanford Spherical Gantry, a multi-purpose 4-degree of freedom motorized gantry we built for capturing light fields and bidirectional reflectance distribution functions (BRDFs) of small stationary objects. (c) We first demonstrated synthetic focusing of light fields in 1996. Here is our first continuous focusing, from 2002. The array of input images was captured by the same robot/camera rig used to capture a light field of Michelangelo's statue of Night in 1999. Focusing was done in software, producing this movie. (d) An array of 128 synchronized CMOS video cameras we built as part of the Stanford Multi-Camera Array project. (e) PhD student Gaurav Garg rides in a pickup truck with a sideways-looking video camera, which we used to construct multi-perspective panoramas of urban landscapes in the Stanford CityBlock project. Google's StreetView grew out of this project. (f) An array of 16 planar mirrors, into which we aimed a high-resolution camera and projector to produce 16 virtual cameras and projectors for our paper on synthetic aperture confocal imaging. (g) An array of curved mirrorlets, created by machining and chrome-plating an aluminum block. Arrays like this are an alternative to aiming a camera into an array of microlenses. (h) A photo-resister, which we used in conjunction with a video projector to capture slices of a light field using a technique called dual photography. (i) PhD student Ren Ng holds a medium format SLR camera into which he has inserted a microlens array between the sensor and main lens. The camera captures light fields instead of photographs. Here are papers describing the camera and its theory of operation. Ren's PhD dissertation, "Digital Light Field Photography," won the 2006 ACM Doctoral dissertation Award. (j) Ren is founder and current CEO of Lytro, a startup company that is commercializing his dissertation research. Pictured here is their camera, which captures a light field you can synthetically focus (a.k.a. digitally refocus) after capture. (k) By inserting a microlens array (circled in red) into a standard microscope, we create a light field microscope (LFM). From its images we can produce perspective flybys, focal stacks, and volume datasets at a single instant in time. Here is a paper describing the idea. (l) By inserting a second microlens array and video projector into the light path of the same microscope, we can control the incident light field falling on a specimen as well as the light field leaving it. In this example, we use our Light Field Illuminator (LFI) to change the characteristics of light falling on a single blond human hair. Here is a paper describing the idea. (m) We sometimes create light fields synthetically. This Venn diagram shows a taxonomy of the kinds of apertures that can occur in light fields, under a formulation we have devised of general linear cameras with finite (i.e. non-pinhole) apertures. (n) In another theoretical paper, we explore the relationship between light fields as used in the computer graphics community and the Wigner distribution commonly used in the wave optics community. (o) A ring of 16 synchronized Go-Pro cameras, built by Google as part of its Jump system and announced in May 2015 at Google I/O. Although not a product of our laboratory, this is a light field camera. Levoy's team at Google advised on its design and contributed software to the system.
(Listed in reverse chronological order. (Slides from papers may also be available on the web pages of those papers.)
A list of technical papers, with abstracts and pointers to additional information, is also available. Or you can return to the research projects page or our home page.