Often the development of a computer vision project involves tweaking parameters of a technique to achieve the desired outcome. These parameters could be the thresholds of an edge detection algorithm or the brightness of an image, for instance. If you don’t use any graphical user interface (GUI) for tweaking these parameters, you need to stop your application, adjust your code, run the application again, evaluate, and repeat until it is good. That is tedious and time-consuming.
There are plenty of great GUI libs, e.g. Qt and imgui, that can be used together with OpenCV to allow you to tweak parameters during runtime. For using Qt with OpenCV on a Mac, check out this post. There might be cases, however, where you don’t have (or don’t want) the dependencies of such libs, e.g. you have not compiled OpenCV with Qt support, or you can’t use OpenGL. In such situations, all you need is a quick and hassle-free way of creating a GUI to tweak your algorithms.
That is the purpose of cvui. It is a C++, header-only and cross-platform (Windows, Linux and OSX) UI lib built on top of OpenCV drawing primitives. It has no dependencies other than OpenCV itself (which you are probably already using).
It follows the rule
One line of code should produce one UI component on the screen.
As a result, the lib has a friendly and C-like API with no classes/objects and several components, e.g. trackbar, button, text, among others:
How to use cvui in your application
In order to use cvui, you just include cvui.h in your project, give it an image ( i.e. cv::Mat ) to render components and you are done!
Basic “hello world” application
Let’s take a look at the capabilities of cvui by creating a simple hello-world application with some UI interactions. The application contains a button and a visual indicator showing how many times that button was clicked. Here is the code:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Hello World!"
int main(void)
{
cv::Mat frame = cv::Mat(200, 500, CV_8UC3);
int count = 0;
// Init a OpenCV window and tell cvui to use it.
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
// Fill the frame with a nice color
frame = cv::Scalar(49, 52, 49);
// Show a button at position (110, 80)
if (cvui::button(frame, 110, 80, "Hello, world!")) {
// The button was clicked, so let's increment our counter.
count++;
}
// Show how many times the button has been clicked.
// Text at position (250, 90), sized 0.4, in red.
cvui::printf(frame, 250, 90, 0.4, 0xff0000, "Button click count: %d", count);
// Update cvui internal stuff
cvui::update();
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC key was pressed
if (cv::waitKey(20) == 27) {
break;
}
}
return 0;
}
The result of the code above is the following:
To ensure cvui works properly with your project
- Call the initialization function
cvui::init()before rendering any components. - Call
cvui::update()once after all components are rendered.
Regarding the components used in the code above, the cvui::button() function returns true everytime the button is clicked, so you can conveniently use it in if statements. The cvui::printf() function works similarly to the standard C printf() function, so you can easily render texts and numbers on the screen using notations as %d and %s. You can also choose the color of the text using hex values as 0xRRGGBB, e.g. 0xFF0000 (red), 0x00FF00 (green) and 0x0000FF (blue).
A more advanced application
Now let’s build something a bit more sophisticated, but as easily as before. The application applies the Canny Edge algorithm to an image, allowing the user to enable/disable the technique and adjust its threshold values.
Step 1: Foundation
We start by creating an application with no UI elements. The use of the Canny Edge algorithm is defined by a boolean variable (use_canny), while the algorithm thresholds are defined by two integers (low_threshold and high_threshold). Using that approach, we must recompile the code every time we want to enable/disable the technique or adjust its thresholds.
The code for that application is the following:
#include <opencv2/opencv.hpp>
#define WINDOW_NAME "CVUI Canny Edge"
int main(int argc, const char *argv[])
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
cv::namedWindow(WINDOW_NAME);
while (true) {
// Should we apply Canny edge?
if (use_canny) {
// Yes, we should apply it.
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
// No, so just copy the original image to the displaying frame.
lena.copyTo(frame);
}
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC was pressed
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
The result is an application that either shows the original image (use_canny is false) or shows the detected edges (use_canny is true):
Step 2: Dynamically enable/disable the edge detection
Let’s improve the workflow by using cvui and adding a checkbox to control the value of use_canny. Using that approach, the user can enable/disable the use of Canny Edge while the application is still running. We add the required cvui code and use the cvui::checkbox function:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Canny Edge"
int main(void)
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
// Init a OpenCV window and tell cvui to use it.
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
// Should we apply Canny edge?
if (use_canny) {
// Yes, we should apply it.
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
// No, so just copy the original image to the displaying frame.
lena.copyTo(frame);
}
// Checkbox to enable/disable the use of Canny edge
cvui::checkbox(frame, 15, 80, "Use Canny Edge", &use_canny);
// Update cvui internal stuff
cvui::update();
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC was pressed
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
This small modification alone is already a time saver for testing the application without recompiling everything:
It might be difficult to see the rendered checkbox and its label depending on the image being used, e.g. image with a white background. We can prevent that problem by creating a window using cvui::window() to house the checkbox.
cvui renders each component at the moment the component function is called, so we must call cvui::window() before cvui::checkbox(), otherwise the window will be rendered in front of the checkbox:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Canny Edge"
int main(void)
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
// Init a OpenCV window and tell cvui to use it.
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
// Should we apply Canny edge?
if (use_canny) {
// Yes, we should apply it.
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
// No, so just copy the original image to the displaying frame.
lena.copyTo(frame);
}
// Render the settings window to house the UI
cvui::window(frame, 10, 50, 180, 180, "Settings");
// Checkbox to enable/disable the use of Canny edge
cvui::checkbox(frame, 15, 80, "Use Canny Edge", &use_canny);
// Update cvui internal stuff
cvui::update();
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC was pressed
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
The result is a more pleasant UI:
Step 3: Tweak threshold values
It is time to allow the user to select the values for low_threashold and high_threashold during runtime as well. Since those parameters can vary within an interval, we can use cvui::trackbar() to create a trackbar:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Canny Edge"
int main(void)
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
// Init a OpenCV window and tell cvui to use it.
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
// Should we apply Canny edge?
if (use_canny) {
// Yes, we should apply it.
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
// No, so just copy the original image to the displaying frame.
lena.copyTo(frame);
}
// Render the settings window to house the UI
cvui::window(frame, 10, 50, 180, 180, "Settings");
// Checkbox to enable/disable the use of Canny edge
cvui::checkbox(frame, 15, 80, "Use Canny Edge", &use_canny);
// Two trackbars to control the low and high threshold values
// for the Canny edge algorithm.
cvui::trackbar(frame, 15, 110, 165, &low_threshold, 5, 150);
cvui::trackbar(frame, 15, 180, 165, &high_threshold, 80, 300);
// Update cvui internal stuff
cvui::update();
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC was pressed
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
The cvui::trackbar() function accepts parameters that specify the minimum and maximum values allowed for the trackbar. In the example above, they are [5, 150] for low_threshold and [80, 300] for high_threshold, respectively.
The result is a fully interactive application that allows users to quickly and easily explore the tweaking of Canny Edge parameters, as well as enable/disable its use:
Below is the complete code for this application, without the comments. It shows that you don’t need many lines of code to produce a minimal (and useful) UI for your application:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Canny Edge"
int main(void)
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
if (use_canny) {
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
lena.copyTo(frame);
}
cvui::window(frame, 10, 50, 180, 180, "Settings");
cvui::checkbox(frame, 15, 80, "Use Canny Edge", &use_canny);
cvui::trackbar(frame, 15, 110, 165, &low_threshold, 5, 150);
cvui::trackbar(frame, 15, 180, 165, &high_threshold, 80, 300);
cvui::update();
cv::imshow(WINDOW_NAME, frame);
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
Conclusion
The cvui lib was created out of a necessity. It was not designed to be a full-blown solution for the development of complex graphical applications. It is simple and limited in many ways. However, it is practical, easy to use and can save you several hours of frustration and tedious work.
If you like cvui, don’t forget to check out its repository on Github, its documentation and all example applications (buildable with cmake).
