Real-Time 2D to 3D Video Conversion on an FPGA

Technologies used:

  • Field Programmable Gate Arrays
  • Catapult-C HDL
  • Quartus Schematic Capture
  • Low-level Optimisation
  • Image Processing

Skills demonstrated:

  • Digital System Design
  • Discrete Mathematics

The Problem

A significant part of the cost of 3D filmmaking and viewing is the need for highly specialist cameras and displays. It is also extremely time consuming to to convert older 2D footage into 3D, with huge amounts of labour being consumed isolating and setting the depth of every object in view.

The Idea

FPGAs offer considerable advantages over traditional CPUs and GPUs in their very low power consumption and potential for huge parallelism. This made a custom logid device a good choice for a low cost processing platform.

The design of the algorithm centered around trying to generate a depth map from a single image. The solution to this problem was based on the idea that if a weak light source is placed near the camera, objects further away will appear darker. Darker areas can therefore be shifted less when the left and right eye images are constructed. A standard Sobel edge detector was incorporated to define the objects in a scene.

The Execution

Modules for the Altera FPGA were designed in Catapult-C. This is a C-style language with an emphasis on bitwise logic and other low-level hardware features.

These were connected together in Quartus, an industry standard FPGA programming tool, along with the standard modules to interface the camera and display. The 3D effect was created using an anaglyph - colouring red and blue images and using tinted glasses to send one to each eye. Variable sized buffers were used to shift parts of the camera input when they were displayed on the screen.

The Result

The resulting project produced a strong 3D effect, especially in scenes containing objects of contrasting colours. The resulting video was also produced in real time, with no delay between capturing and displaying each frame.