Often the development of a computer vision project involves tweaking parameters of a technique to achieve the desired outcome. These parameters could be the thresholds of an edge detection algorithm or the brightness of an image, for instance. If you don’t use any graphical user interface (GUI) for tweaking these parameters, you need to stop your application, adjust your code, run the application again, evaluate, and repeat until it is good. That is tedious and time-consuming.
There are plenty of great GUI libs, e.g. Qt and imgui, that can be used together with OpenCV to allow you to tweak parameters during runtime. For using Qt with OpenCV on a Mac, check out this post. There might be cases, however, where you don’t have (or don’t want) the dependencies of such libs, e.g. you have not compiled OpenCV with Qt support, or you can’t use OpenGL. In such situations, all you need is a quick and hassle-free way of creating a GUI to tweak your algorithms.
That is the purpose of cvui. It is a C++, header-only and cross-platform (Windows, Linux and OSX) UI lib built on top of OpenCV drawing primitives. It has no dependencies other than OpenCV itself (which you are probably already using).
It follows the rule
One line of code should produce one UI component on the screen.
As a result, the lib has a friendly and C-like API with no classes/objects and several components, e.g. trackbar, button, text, among others:
How to use cvui in your application
In order to use cvui, you just include cvui.h in your project, give it an image ( i.e. cv::Mat ) to render components and you are done!
Basic “hello world” application
Let’s take a look at the capabilities of cvui by creating a simple hello-world application with some UI interactions. The application contains a button and a visual indicator showing how many times that button was clicked. Here is the code:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Hello World!"
int main(void)
{
cv::Mat frame = cv::Mat(200, 500, CV_8UC3);
int count = 0;
// Init a OpenCV window and tell cvui to use it.
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
// Fill the frame with a nice color
frame = cv::Scalar(49, 52, 49);
// Show a button at position (110, 80)
if (cvui::button(frame, 110, 80, "Hello, world!")) {
// The button was clicked, so let's increment our counter.
count++;
}
// Show how many times the button has been clicked.
// Text at position (250, 90), sized 0.4, in red.
cvui::printf(frame, 250, 90, 0.4, 0xff0000, "Button click count: %d", count);
// Update cvui internal stuff
cvui::update();
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC key was pressed
if (cv::waitKey(20) == 27) {
break;
}
}
return 0;
}
The result of the code above is the following:
To ensure cvui works properly with your project
- Call the initialization function
cvui::init()before rendering any components. - Call
cvui::update()once after all components are rendered.
Regarding the components used in the code above, the cvui::button() function returns true everytime the button is clicked, so you can conveniently use it in if statements. The cvui::printf() function works similarly to the standard C printf() function, so you can easily render texts and numbers on the screen using notations as %d and %s. You can also choose the color of the text using hex values as 0xRRGGBB, e.g. 0xFF0000 (red), 0x00FF00 (green) and 0x0000FF (blue).
A more advanced application
Now let’s build something a bit more sophisticated, but as easily as before. The application applies the Canny Edge algorithm to an image, allowing the user to enable/disable the technique and adjust its threshold values.
Step 1: Foundation
We start by creating an application with no UI elements. The use of the Canny Edge algorithm is defined by a boolean variable (use_canny), while the algorithm thresholds are defined by two integers (low_threshold and high_threshold). Using that approach, we must recompile the code every time we want to enable/disable the technique or adjust its thresholds.
The code for that application is the following:
#include <opencv2/opencv.hpp>
#define WINDOW_NAME "CVUI Canny Edge"
int main(int argc, const char *argv[])
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
cv::namedWindow(WINDOW_NAME);
while (true) {
// Should we apply Canny edge?
if (use_canny) {
// Yes, we should apply it.
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
// No, so just copy the original image to the displaying frame.
lena.copyTo(frame);
}
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC was pressed
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
The result is an application that either shows the original image (use_canny is false) or shows the detected edges (use_canny is true):
Step 2: Dynamically enable/disable the edge detection
Let’s improve the workflow by using cvui and adding a checkbox to control the value of use_canny. Using that approach, the user can enable/disable the use of Canny Edge while the application is still running. We add the required cvui code and use the cvui::checkbox function:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Canny Edge"
int main(void)
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
// Init a OpenCV window and tell cvui to use it.
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
// Should we apply Canny edge?
if (use_canny) {
// Yes, we should apply it.
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
// No, so just copy the original image to the displaying frame.
lena.copyTo(frame);
}
// Checkbox to enable/disable the use of Canny edge
cvui::checkbox(frame, 15, 80, "Use Canny Edge", &use_canny);
// Update cvui internal stuff
cvui::update();
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC was pressed
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
This small modification alone is already a time saver for testing the application without recompiling everything:
It might be difficult to see the rendered checkbox and its label depending on the image being used, e.g. image with a white background. We can prevent that problem by creating a window using cvui::window() to house the checkbox.
cvui renders each component at the moment the component function is called, so we must call cvui::window() before cvui::checkbox(), otherwise the window will be rendered in front of the checkbox:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Canny Edge"
int main(void)
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
// Init a OpenCV window and tell cvui to use it.
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
// Should we apply Canny edge?
if (use_canny) {
// Yes, we should apply it.
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
// No, so just copy the original image to the displaying frame.
lena.copyTo(frame);
}
// Render the settings window to house the UI
cvui::window(frame, 10, 50, 180, 180, "Settings");
// Checkbox to enable/disable the use of Canny edge
cvui::checkbox(frame, 15, 80, "Use Canny Edge", &use_canny);
// Update cvui internal stuff
cvui::update();
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC was pressed
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
The result is a more pleasant UI:
Step 3: Tweak threshold values
It is time to allow the user to select the values for low_threashold and high_threashold during runtime as well. Since those parameters can vary within an interval, we can use cvui::trackbar() to create a trackbar:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Canny Edge"
int main(void)
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
// Init a OpenCV window and tell cvui to use it.
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
// Should we apply Canny edge?
if (use_canny) {
// Yes, we should apply it.
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
// No, so just copy the original image to the displaying frame.
lena.copyTo(frame);
}
// Render the settings window to house the UI
cvui::window(frame, 10, 50, 180, 180, "Settings");
// Checkbox to enable/disable the use of Canny edge
cvui::checkbox(frame, 15, 80, "Use Canny Edge", &use_canny);
// Two trackbars to control the low and high threshold values
// for the Canny edge algorithm.
cvui::trackbar(frame, 15, 110, 165, &low_threshold, 5, 150);
cvui::trackbar(frame, 15, 180, 165, &high_threshold, 80, 300);
// Update cvui internal stuff
cvui::update();
// Show everything on the screen
cv::imshow(WINDOW_NAME, frame);
// Check if ESC was pressed
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
The cvui::trackbar() function accepts parameters that specify the minimum and maximum values allowed for the trackbar. In the example above, they are [5, 150] for low_threshold and [80, 300] for high_threshold, respectively.
The result is a fully interactive application that allows users to quickly and easily explore the tweaking of Canny Edge parameters, as well as enable/disable its use:
Below is the complete code for this application, without the comments. It shows that you don’t need many lines of code to produce a minimal (and useful) UI for your application:
#include <opencv2/opencv.hpp>
#include "cvui.h"
#define WINDOW_NAME "CVUI Canny Edge"
int main(void)
{
cv::Mat lena = cv::imread("lena.jpg");
cv::Mat frame = lena.clone();
int low_threshold = 50, high_threshold = 150;
bool use_canny = false;
cv::namedWindow(WINDOW_NAME);
cvui::init(WINDOW_NAME);
while (true) {
if (use_canny) {
cv::cvtColor(lena, frame, CV_BGR2GRAY);
cv::Canny(frame, frame, low_threshold, high_threshold, 3);
} else {
lena.copyTo(frame);
}
cvui::window(frame, 10, 50, 180, 180, "Settings");
cvui::checkbox(frame, 15, 80, "Use Canny Edge", &use_canny);
cvui::trackbar(frame, 15, 110, 165, &low_threshold, 5, 150);
cvui::trackbar(frame, 15, 180, 165, &high_threshold, 80, 300);
cvui::update();
cv::imshow(WINDOW_NAME, frame);
if (cv::waitKey(30) == 27) {
break;
}
}
return 0;
}
Conclusion
The cvui lib was created out of a necessity. It was not designed to be a full-blown solution for the development of complex graphical applications. It is simple and limited in many ways. However, it is practical, easy to use and can save you several hours of frustration and tedious work.
If you like cvui, don’t forget to check out its repository on Github, its documentation and all example applications (buildable with cmake).
Looks elegant
If you want to use “make” instead of cmake, here is an example “Makefile”: (assuming cvui.h and lena.tiff are in the current directory):
all: cvuitest
CXXFLAGS += -std=c++11 -I. -g
LDFLAGS += -lopencv_imgproc -lopencv_highgui -lopencv_core
cvuitest: cvuitest.cpp
g++ $(CXXFLAGS) -o cvuitest cvuitest.cpp $(LDFLAGS)
Dear Satya,
Should we build CVUI as this link https://dovyski.github.io/cvui//build/ says, or can we directly use cvui.h file ? im confuse how to build this library. Also i want to know that are there any changes should do in MakeList file ?
Hi! cvui is a header-only lib, which in practice means you just put cvui.h along with your code and it will work. You don’t have to build it, only compile your code normally. The link you mentioned contains instructions to build the example programs that come along with cvui. You don’t need them for your application.
Dear, Satya,
Thank you for sharing and I tired it and did not have any problem in compiling and running
Thanks for letting me know.
It does not run properly for me, I get two windows with the same WINDOW_NAME and the buttons dont work properly. I am just trying the example. https://uploads.disquscdn.com/images/53f6366ab8ee933d0132047e1a5832c14c9af8f3225cc5cc561de404cfdc0e30.png
Nvm, that was in the debug mode in VS2017, I changed it to release mode and it is working perfectly. Any idea, why it is behaving so in debug mode ? are we rendering a “cvui” over the OpenCV window to get the functionality ?
Are you still facing that problem? If so, could you please open an issue on cvui’s Github repo (https://github.com/Dovyski/cvui/issues)?
Impressed CVUI works great so far. Except I need it to not only feature controls but host video frames with a thread. Right now they appear side by size when sized correctly but becomes annoying if they overlap.
Do you have any advice or examples to feature video in the window?
cvui works by rendering all its components into a frame, e.g. cv::Mat. What you could do is read each frame of your video and then display into the cvui frame using cvui::image(). I think that could work for your needs.
I’ve got the cv::Mat.: stored as “rawImage” frame.
Then
drawKeypoints(rawImage, keypoints, im_with_keypoints, Scalar(gui_v->valueBlue, gui_v->valueGreen, gui_v->valueRed), DrawMatchesFlags::DRAW_RICH_KEYPOINTS);
Then
imshow(“keypoints”, im_with_keypoints);
How would I adjust this to accomodate the cvui::image
Assuming you have a cv::Mat, e.g.
screen_mat, that represents your screen (the one being used by cvui), you can renderim_with_keypointsto it like:// Render im_with_keypoints to position (10, 10) in screen_mat
image(screen_mat, 10, 10, im_with_keypoints);
thank you – exactly; but I am only having difficulty with screen_mat
Where do es that come from. I’m using for e.g. “frame_x” sor screen mat, dont know how to make it
Take a look at this guide: https://dovyski.github.io/cvui/usage/ , in particular section 3 (Render cvui components). In that guide, the “screen mat” is called
frameinstead ofscreen_mat.cv::Mat frame = cv::Mat(cv::Size(640, 480), CV_8UC3);
image(frame, 10, 10, im_with_keypoints);
getting exceptions everytime