Skip to content

Assignment #2 – Light Field Camera (part I)

October 20, 2010

Extracted from Stanford’s Light Field and Computational Photography website:

“The light field, first described in Arun Gershun’s classic 1936 paper of the same name, is defined as radiance as a function of position and direction in regions of space free of occluders. In free space, the light field is a 4D function – scalar or vector depending on the exact definition employed. Light fields were introduced into computer graphics in 1996 by Marc Levoy and Pat Hanrahan. Their proposed application was image-based-rendering – computing new views of a scene from pre-existing views without the need for scene geometry… ”

The assignment consists of using a regular camera (or alternatively a 3D renderer) to capture a sample of the Light Field of a scene, and then (re)composing a particular setup (focus distance, and point of view), based on the captured images. The proposed experiment was to move the camera horizontally on small fixed steps, preferentially by using a robotic gantry – Lego Mindstorms is an option – to control this movement. I decided to postpone this to part II, starting my experiments with an image set from Stanford University and also the 3D rendered scene.

I used MatLab to combine the original photos with “shift and add” operations and refocus the images at different planes (orthogonal to the camera sensor). I was also able to create see-through effects with the 3D scene. I’d like to acknowledge Andy, Otkrist and Jessica, also from the Camera Culture Group, for some hints and the basic MatLab code from which I developed my own version.

For the first experiment, I used some sample light field images from the Stanford Database. I’m not including the original source images here because they can be obtained from the aforementioned URL. I used the rectified version of the Chess set and bellow are the results I obtained:

Chess board refocused at the first row.

Chess board refocused at the first row.

Chess board refocused at the middle row.

Chess board refocused at the middle row.

Chess board refocused at the last row.

Chess board refocused at the last row.

For the second experiment I used a virtual scene, rendered with the Unity3D game engine. I used a simple script to shift the virtual camera horizontally and record the rendered images. Bellow are the composed images showing a see-through effect. The camera is positioned in front of a grass “wall”, and there’s something behind it. Check the last image to find out what it is. This see-through effect is only possible because the original images where taken with a horizontal shift and the grass leaves form a (mostly) vertical pattern (on more complex patterns, such as bushes, I’d have to move the camera in both axis – horizontal and vertical). The original files can be seen and downloaded from the gallery at the end of the post (make sure you notice that the composed see-through image shows more details of the figure behind the grass than any of the original ones):

Focusing too close to the camera, nothing can be depicted from this one.

Focusing too close to the camera, nothing can be depicted from this one.

Focusing at the grass "wall", still not seeing what's hidden behind it

Focusing at the grass "wall", still not seeing what's hidden behind it

By focusing behind the grass "wall", it is possible to see the bird behind it.

By focusing behind the grass "wall", it is possible to see the bird behind it.

In the next post, I’ll show Part II of this assignment, detailing the results of the experiment with my own photos, and post the MatLab source code. Finally, here’s the gallery containing all the images (original and composed) from this Part I of Assignment #2.

Advertisements
2 Comments leave one →
  1. ekta permalink
    July 26, 2013 2:16 pm

    How to decide the focus distance “d” and scale in your code?

    • erickpassos permalink*
      August 22, 2013 11:47 pm

      experimentation, since it was a very naive approach, where I haven’t measured everything exactly, except for the spacing between shots.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: