Testing out libcamera Software-ISP on RPi3/4

Introduction

In this post, we are going to test libcamera software-isp on RaspberryPi 3/4 hardware. The post is intended for the curious, to test out the SoftISP and perhaps hack on it, on a relatively easy-to-grab hardware platform.

The Software-ISP in libcamera

SoftISP in itself, is quite an independent project in libcamera itself. Currently, the ISP module merged in libcamera mainline, runs on CPU but efforts are in full-swing to develop and merge a GPU-based Software ISP in libcamera [1].

Ideally, you can use SoftwareISP on any platform by just adding the platform matching string, in the list of supportedDevices of simple pipeline handler and setting the SoftwareISP flag as true. We are doing to the same for RPi3/4, in order to test it.

Add ‘unicam’ CSI-2 receiver for RPi 3/4 in the list of simple pipeline handler’s list of supported devices. Additionally, set SoftwareISP flag to true, in order to engage it.

static const SimplePipelineInfo supportedDevices[] = {
    { "dcmipp", {}, false },
    { "imx7-csi", { { "pxp", 1 } }, false },
    { "intel-ipu6", {}, true },
    { "j721e-csi2rx", {}, true },
    { "mtk-seninf", { { "mtk-mdp", 3 } }, false },
    { "mxc-isi", {}, false },
    { "qcom-camss", {}, true },
    { "sun6i-csi", {}, false },
+   { "unicam", {}, true },
};

Build libcamera

We need to build libcamera now, without the Rpi3/4 pipeline handler but with simple pipeline handler. One can specify this by passing -Dpipelines=simple to meson.

($) meson setup -Dpipelines=simple -Dcam=enabled build
($) ninja -C build

The extensive build instructions are stated here .

Tip: You can pass --buildtype=release to meson for an perf-optimised build.

Test the build

Once a camera sensor is connected to RPi3/4 and libcamera has been build, one can easily if the above configuration took effect and you are ready to stream/route the frames from sensor to SoftwareISP.

($) ./build/src/apps/cam/cam --list

Now that you have verified that the camera pipeline is setup successfully, you can be sure that the frames captured by the camera sensor are routed through the SoftwareISP (instead of the bcm2835-isp present on RPi)

($) ./build/src/apps/cam/cam --camera 1 --capture

Tip: Pass one of --display, --sdl or --file=filename options to cam to direct processed frames to one of the sinks.

Oops! Streaming failed?

Oh, well, it did happen to me when the RPi was connected to IMX219 sensor. I happen to debug the issue and found that the simple pipeline handler does not take into sensor transforms into the account during configure(). A patch has been submitted upstream , hopfully should land soon!

Still facing issues? Get in touch at the upstream IRC channel or libcamera mailing list .

Happy hacking!