An Efficient And Simple C++ API for the Rasperry Pi Camera Module

For the past few days I've been messing around with my new raspberry pi camera modules (see earlier blog posts for excessive details) and part of that has involved putting together a nice and easy to use api to access the camera in c++ and read its frames. This post is a guide to installation, an overview of the very simple api and a description of the sample application.

One word of caution - as with any tinkering there is always a chance something will go wrong and result in a dead pi. If this worries you, back up first. I didn't bother, but I didn't have anything on there I was worried about losing!

Installation

Make sure you're on a recent Raspberry Pi build, and have a working Camera!

I'm assuming at this point you've got a camera module and it's working. If you've not set it up yet you may need to update your raspberry pi (depends when you bought it). I won't go over this process as it's been described 100 times already, but here's a link to get you going just in case:

http://www.raspberrypi.org/archives/3890

Once all is up and running type:

raspivid -t 10000

That should show you the raspberry pi video feed on screen for 10 seconds.

Get CMake

If you haven't already got it, you'll need cmake for building just about anything:

sudo apt-get install cmake

Download and install the latest 'userland-master'

This is the bit of the raspberry pi OS that contains the code for the camera applications and the various libraries they use. At time of writing it isn't supplied as part of the install, so you need to download, build and install it manually. To do so:

Download the latest userland-master.zip from here

Unzip it into your /opt/vc directory. You should now have a folder called /opt/vc/userland-master with various folders in it such as "host_applications" and "interfaces".

Change to the /opt/vc/userland-master folder, then build it with the following commands:

sudo mkdir build
cd build
sudo cmake -DCMAKE_BUILD_TYPE=Release ..
sudo make
sudo make install

Test everything worked by running raspivid again. You may see some different messages pop up (I got some harmless errors probably due to the build being so recent), but the crucial thing is that you still get the video feed on screen.

Download and build the PiCam API/Samples

The api and samples can all be downloaded here:
http://www.cheerfulprogrammer.com/downloads/picamtutorial/picamdemo.zip

Extract them into a folder in your home directory called 'picamdemo'. You should have a few cpp files in there, plus a make file and some shaders. 

Change to the folder and build the application with:

cmake .
make

Then run the sample with

./picamdemo

If all goes well you should see some text like this:

Compiled vertex shader simplevertshader.glsl:
<some shader code here>

Compiled fragment shader simplefragshader.glsl:
<some shader code here>

mmal: mmal_vc_port_parameter_set: failed to set port parameter 64:0:ENOSYS
mmal: Function not implemented
Init camera output with 512/512
Creating pool with 3 buffers of size 1048576
Init camera output with 256/256
Creating pool with 3 buffers of size 262144
Init camera output with 128/128
Creating pool with 3 buffers of size 65536
Init camera output with 64/64
Creating pool with 3 buffers of size 16384
Camera successfully created
Running frame loop

And your tv should start flicking between various resolutions of the camera feed like this:

(Edit - I've had some reports of the blogger you-tube link not working. You can see the full video here on proper you tube: http://www.youtube.com/watch?v=9bWJBSNxeXk) The API (and what it does!)

PiCam is designed to be very simple but also useful for image processing algorithms. Right now it lets you:

• Start up the camera with a given width, height and frame rate
• Specify a number of 'levels'. More on that later.
• Choose whether to automatically convert the camera feed to RGBA format

Basic Initialisation

All this is done just by calling StartCamera and passing in the right parameters. It returns a pointer to a CCamera object as follows:

CCamera* mycamera = StartCamera(512,512,30,1,true);

That's a 512x512 image at 30hz, with 1 level and rgba conversion enabled.

Reading

Once started you can extract frames from the camera by calling ReadFrame and passing in a buffer:

char mybuffer[512*512*4]

mycamera->ReadFrame(0,mybuffer,sizeof(mybuffer));

ReadFrame will return the number of bytes actually read, or -1 if there was an error. An error occurs either when there is no data available or your buffer is not large enough.

In addition to ReadFrame there are 2 functions: BeginReadFrame and EndReadFrame. These slightly more advanced versions are shown in the demo, and allow you to be more efficient by locking the actual camera buffer, using it, then releasing it. Internally ReadFrame is implemented using these functions.

Shutting down

Once done, call 'StopCamera'

Levels

In image processing it is often useful to have your data provided at different resolutions. Expensive operations need to be performed on low res images to run at a good frame rate, but you may still want higher res versions around for other operations or even just showing on screen. The PiCam api will do this for you automatically (for up to 3 additional levels). If we modify the StartCamera call to this:

CCamera* mycamera = StartCamera(512,512,30,4,true);

The system will automatically generate the main image plus an additional 3 down-sampled ones (at half res, quarter res and 1/8th res). These are then accessed by specifying a level other than 0 in the call to ReadFrame (or BeginReadFrame):

mycamera->ReadFrame(0,mybuffer,sizeof(mybuffer)); //get full res frame

mycamera->ReadFrame(1,mybuffer,sizeof(mybuffer)); //get half res frame

mycamera->ReadFrame(2,mybuffer,sizeof(mybuffer)); //get quarter res frame

mycamera->ReadFrame(3,mybuffer,sizeof(mybuffer)); //get 1/8th res frame

RGBA Conversions

For most purposes you'll want the data in a nice friendly RGBA format, however if you actually want the raw YUV data feed from the camera, specify false as the last parameter to StartCamera and no conversions will be done for you.

The demo application

The picamdemo application consists of the core camera code as these files:

• camera.h/camera.cpp
• cameracontrol.h/cameracontrol.cpp
• mmalincludes.h

A very simple opengl graphics api (which you are welcome to use/modify/change in any way you please):

• graphics.h/graphics.cpp

And the main demo app itself:

• picam.cpp

Which looks like this:

#include <stdio.h>
#include <unistd.h>
#include "camera.h"
#include "graphics.h"

#define MAIN_TEXTURE_WIDTH 512
#define MAIN_TEXTURE_HEIGHT 512

char tmpbuff[MAIN_TEXTURE_WIDTH*MAIN_TEXTURE_HEIGHT*4];

//entry point
int main(int argc, const char **argv)
{
    //should the camera convert frame data from yuv to argb automatically?
    bool do_argb_conversion = true;

    //how many detail levels (1 = just the capture res, >1 goes down by halves, 4 max)
    int num_levels = 4;

    //init graphics and the camera
    InitGraphics();
    CCamera* cam = StartCamera(MAIN_TEXTURE_WIDTH, MAIN_TEXTURE_HEIGHT,30,num_levels,do_argb_conversion);

    //create 4 textures of decreasing size
    GfxTexture textures[4];
    for(int texidx = 0; texidx < num_levels; texidx++)
        textures[texidx].Create(MAIN_TEXTURE_WIDTH >> texidx, MAIN_TEXTURE_HEIGHT >> texidx);

    printf("Running frame loop\n");
    for(int i = 0; i < 3000; i++)
    {
        //pick a level to read based on current frame (flicking through them every 30 frames)
        int texidx = (i / 30)%num_levels;

        //lock the chosen buffer, and copy it directly into the corresponding texture
        const void* frame_data; int frame_sz;
        if(cam->BeginReadFrame(texidx,frame_data,frame_sz))
        {
            if(do_argb_conversion)
            {
                //if doing argb conversion just copy data directly
                textures[texidx].SetPixels(frame_data);
            }
            else
            {
                //if not converting argb the data will be the wrong size so copy it in
                //via a temporary buffer just so we can observe something happening!
                memcpy(tmpbuff,frame_data,frame_sz);
                textures[texidx].SetPixels(tmpbuff);
            }
            cam->EndReadFrame(texidx);
        }

        //begin frame, draw the texture then end frame (the bit of maths just fits the image to the screen while maintaining aspect ratio)
        BeginFrame();
        float aspect_ratio = float(MAIN_TEXTURE_WIDTH)/float(MAIN_TEXTURE_HEIGHT);
        float screen_aspect_ratio = 1280.f/720.f;
        DrawTextureRect(&textures[texidx],-aspect_ratio/screen_aspect_ratio,-1.f,aspect_ratio/screen_aspect_ratio,1.f);
        EndFrame();
    }

    StopCamera();
}

That's the full code for exploiting all the features of the api. It is designed to loop through each detail level and render them in turn. At the top of the main function you will find a couple of variables to enable argb or change level count, and higher up you can see the frame size settings.

Questions? Problems? Comments?

I'm happy to answer any questions, hear any comments, and if you hit issues I'd like to fix them. Either comment on this blog or email me (wibble82@hotmail.com) with a sensible subject like 'pi cam problem' (so it doesn't go into the junk mail box!).

p.s. right at the end, here's a tiny shameless advert for my new venture - http://www.happyrobotgames.com/no-stick-shooter. If you like my writing, check out the dev blog for regular updates on my first proper indie title!