[SOLVED] Using atomic variable to storage cv::Mat object

Issue

I'm working in a application that's use opencv to capture frames and show it's in a window. The code use's the frame to record a video too, but, when I show the frame and record at the same time, the frame rate in screen decreases. Can I use a atomic variable to store the frames? like that:

std::atomic<cv::Mat> atomicFrame;
atomicFrame.store(newFrame);
cv::Mat sameFrame = atomicFrame.load();

My idea is uses the atomic variable to access the same frame at the same time in show_frame_on_screen function and record_frames function, but, when i set the variable in my header file, my cpp program get some errors.

A part of my header file:

#include <iostream>
#include <iomanip>
#include <chrono>
#include <ctime>
#include <thread>
#include <atomic>

class MainWindow : public QMainWindow
{
    Q_OBJECT

private:
    Ui::MainWindow *ui;
    QFutureWatcher<void> watcher_record_frame;
    QFutureWatcher<void> watcher_show_frame;
    QFutureWatcher<void> watcher_get_frame;
    QProcess *processKeypad;
    QTimer *savingTimer;
    QTimer *framesRecordTimer;

    cv::VideoWriter *videoWriter;

    std::queue<cv::Mat> framesQueue;
    std::atomic<cv::Mat> frameAtomic;
};

A part of my .cpp file:

#include "mainwindow.h"
#include "ui_mainwindow.h"

MainWindow::MainWindow(QWidget *parent)
    : QMainWindow(parent)
    , ui(new Ui::MainWindow)
{
    ui->setupUi(this);
}

In line ui->setupUi(this); i get the error when passing the this parameter:

mainwindow.cpp:10:17: error: cannot initialize a parameter of type 'QMainWindow *' with an rvalue of type 'MainWindow *'
ui_mainwindow.h:40:31: note: passing argument to parameter 'MainWindow' here

If I comment the line std::atomic<cv::Mat> frameAtomic; or change to std::atomic<int> frameAtomic; the error in cpp file disappear.

This application is running in a RaspBerry Pi 4 8gb, with raspbian OS.

EDIT: At this moment, my code have this functions to refresh the frames, show then and record then.

void MainWindow::update_frame()
{
    while (fvs.more())
    {
        cv::Mat frame = fvs.read();
        if (recording)
        {
            framesQueue.push(frame);
        }
        emit show_frame_on_screen(frame);
    }
}

void MainWindow::show_frame_on_screen(cv::Mat frame)
{
    cv::resize(frame, frame, cv::Size(1024, 600));
    QImage image(frame.data, frame.cols,
                 frame.rows, frame.step,
                 QImage::Format_BGR888);
    ui->layer_frame->setPixmap(QPixmap::fromImage(image));
}
void MainWindow::thread_record_frame()
{
    while (!framesQueue.empty()){
        cv::Mat newFrame = framesQueue.front();
        framesQueue.pop();
        newFrame = write_text_on_frame(newFrame);
        videoWriter->write(newFrame);
        if (!recording && framesQueue.empty()){
            savingTimer->stop();
            ui->layer_loading->setText("");
            videoWriter->release();
        }
    }
}

I'm using std::queue<cv::Mat> framesQueue; to storage all frames and record then.

Solution

std::atomic<cv::Mat> atomicFrame; makes little sense. cv::Mat is an n-dimensional array, and making it atomic isn't feasible.

Atomics are only meant for very small objects. For anything large, reading or writing the atomic object wouldn't be lock-free and might be no better than guarding all access with a std:mutex. Also, you're most likely interested in accessing small parts of the atomicFrame, not just replacing it in its entirety. std::atomic does not give you that option.

Instead, you should double-buffer, which means that you use two separate buffers to show_frame_on_screen and record_frames. This allows you to simultaneously draw and record on separate threads without contention. When a frame is done drawing, you swap the buffers.

std::atomic<cv::Mat> is not useful for this; you could instead use a pair of cv::Mat*, guarded by a std::atomic<bool>.

std::array<cv::Mat, 2> buffers;
cv::Mat* screen_buffer = &buffers[0];
cv::Mat* record_buffer = &buffers[1];
std::atomic<bool> request_frame = false;

void show_frame_on_screen() {
    // if an old request for a frame is still active, block until it isn't
    request_frame.wait(true);
    do_show(screen_buffer);
    request_frame.store(true);
    request_frame.notify_one();
}

There are two approaches to implementing record_frames:

// low-energy variant: only record a frame when needed
void record_frames() {
    request_frame.wait(false);
    do_record(record_buffer);
    std::swap(screen_buffer, record_buffer);
    request_frame.store(false);
    request_frame.notify_one();
}

// low-latency variant: never wait for the frame to be shown on screen, always draw
//                      (this variant is wait-free)
void record_frames() {
    do_record(record_buffer);
    if (request_frame.load()) {
        std::swap(screen_buffer, record_buffer);
        request_frame.store(false);
        request_frame.notify_one();
    }
}

As for the error caused by ui->setupUi(this);, it's unclear why this happens, without seeing your full code, or a minimal reproducible example.

Answered By - Jan Schultke

Answer Checked By - Katrina (WPSolving Volunteer)