• Python 3 Demopack Download
Demo: geexlab-demopack-python3/opencv/04-webcam-face-detection/main.xml
• GeeXLab Downloads
• Forum thread (EN)

For the fourth and last article (previous articles are here, here and here), we’re going to look at face detection using OpenCV. So far, the OpenCV functions we used (image reading, webcam output) are basic functions we find in many softwares inlcuding GeeXLab. What makes OpenCV interesting is the availability of some computer vision algorithms such as face detection (or face recognition).

The face detection method used in the demo is based on the Haar Cascades. If, like me, you are not an expert of Haar cascades, here is what you need to know:

Object Detection using Haar feature-based cascade classifiers is an effective method proposed by Paul Viola and Michael Jones in the 2001 paper, “Rapid Object Detection using a Boosted Cascade of Simple Features”. It is a machine learning based approach in which a cascade function is trained from a lot of positive images (images of faces) and negative images (images without faces). It is then used to detect objects in other images.

More about Haar cascades can be found HERE.

OpenCV comes with pre-trained classifiers for faces and eyes we can use in our demo. These pre-trained classifiers are XML files stored in opencv/data/haarcascades/ folder. In the demo, the classifiers are stored in the demo/data/ folder:

– data/haarcascade_frontalface_default.xml
– data/haarcascade_eye.xml
– data/haarcascade_eye_tree_eyeglasses.xml

haarcascade_eye_tree_eyeglasses.xml is an eye detector with better handling of eyeglasses.

Once the classifiers (one classifier for faces, one for eyes) are initialized with XML files, they are used in detectMultiScale() function, that performs the detection.

Let see how to initialize the face and eyes classifiers and how to detect faces and eyes on a static image:

import cv2

demo_dir = gh_utils.get_demo_dir() 		
image = cv2.imread(demo_dir + 'data/pexels-people-03.jpg', cv2.IMREAD_COLOR)

# Parameter specifying how much the image size is reduced at each image scale.
fd_scaleFactor  = 1.1 

# Parameter specifying how many neighbors each candidate rectangle should have to retain it.
fd_minNeighbors  = 3 

# Initializes classifiers
face_cascade = cv2.CascadeClassifier(demo_dir + 'data/haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier(demo_dir + 'data/haarcascade_eye_tree_eyeglasses.xml')

# detectMultiScale requires an CV_U8 image (gray).
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Detects objects of different sizes in the input image. 
# The detected objects are returned as a list of rectangles.
image_faces = face_cascade.detectMultiScale(gray_image, fd_scaleFactor, fd_minNeighbors, minSize=(80, 80))

# detect eyes for each face.
for (x,y,w,h) in image_faces:
  roi_gray = gray_image[y:y+h, x:x+w]
  eyes = eye_cascade.detectMultiScale(roi_gray)
  for (ex,ey,ew,eh) in eyes:
    e = (x+ex, y+ey, ew, eh)
    image_eyes.append(e)

image_faces is a list of rectangles for all faces. image_eyes is a list of lists: list of all face eyes lists (for each face, there is a list of eyes rectangles).

In the FRAME script, faces and eyes rectangles can be rendered as follows for the static image (face and eyes detection is done once for a static image):

num_faces_detected = len(image_faces)
if (num_faces_detected > 0):
  gh_renderer.set_line_width(4.0)
  gh_gpu_program.bind(color_prog)

  # Face rectangles ---------------------
  #
  for (x,y,w,h) in image_faces:
    gh_gpu_program.uniform4f(color_prog, "color", 0.2, 0.5, 1.0, 1)
    draw_rectangle(x-texture_width/2, texture_height/2-y, w, h) 

    if (enable_eyes_detection == 1):
      num_eyes_detected = len(webcam_eyes)
      gh_gpu_program.uniform4f(color_prog, "color", 0, 1, 0, 1)

  # Eye rectangles ---------------------
  #
  for e in image_eyes:
    draw_rectangle(e[0]-texture_width/2, texture_height/2-e[1], e[2], e[3]) 

  gh_renderer.set_line_width(1.0)

 
For a dynamic image like the webcam, faces and eyes detection works in a similar way. The only difference is that detection is launched for every new webcam image.

 
I prepared a demo that shows faces + eyes detection on a static image and on the webcam output. I also added the possibility to add a hat on detected faces…

Remark: on my system, the demo takes more than 10 seconds to start. The guilty is the webcam initialization step (OpenCV)…

 
The OpenCV face detector works also with masks:

• Python 3 Demopack Download
Demo: geexlab-python3-demopack/opencv/03-webcam/main.xml
• GeeXLab Downloads
• Forum thread (EN)

For this third tutorial (part 1, part 2) about OpenCV in Python 3, we’re going to look at how to read the webcam output.

OpenCV has an object for video capturing from video files, image sequences or cameras: VideoCapture. Let’s see how to create a VideoCapture object and use it to grab a frame from the webcam.

INIT script:

import cv2

video_capture_device_index = 0
webcam = cv2.VideoCapture(video_capture_device_index)

ret, video_frame = webcam.read()

 
That’s all.

If you have several webcams, you can specify the webcam by its index. Most users have only one webcam, therefore the video_capture_device_index is set to 0.

The read() grabs, decodes and returns a video frame. The video_frame is an NumPy array and, thanks to the part 2 we know how to convert it in a GeeXLab texture and how to display it. But before creating the texture, we need to know the size of the video frame. One way to do it is to read the video capture properties:

frame_width = int(round(webcam.get(cv2.CAP_PROP_FRAME_WIDTH)))
frame_height = int(round(webcam.get(cv2.CAP_PROP_FRAME_HEIGHT)))

 
We can now create the texture and update it with the video frame:

PF_U8_RGB = 1
PF_U8_BGR = 2
PF_U8_RGBA = 3

# BGR U8 is the default pixel format of OpenCV images.
# We create an empty texture
frame_texture = gh_texture.create_2d(frame_width, frame_height, PF_U8_BGR)

# and we update the texture pixmap with the OpenCV image.
gh_texture.update_gpu_memory_from_numpy_img(frame_texture, video_frame)

 
In the FRAME script, we can update the the texture every 50 ms and display it every frame:

dt = gh_utils.get_time_step()

gh_texture.bind(frame_texture, 0)

video_dt = video_dt + dt
if (video_dt > 0.05):
  video_dt = 0
  ret, frame = webcam.read()
  gh_texture.update_gpu_memory_from_numpy_img(frame_texture, frame)

gh_camera.bind(camera_ortho)
gh_gpu_program.bind(texture_prog)
gh_object.render(quad)

 
Here is the demo:

 
Keep in mind that frame_texture is a regular 2D texture you can map on any object (torus, sphere, …):

Webcam output mapped on a sphere:

 
Webcam output mapped on a donut:

 
The demo shows how to save a file with OpenCV too. At the end of the FRAME script there is the following piece of code:

if (gh_imgui.button("Screenshot", 100, 20) == 1):
  demo_dir = gh_utils.get_demo_dir() 	
  screenshot_counter = screenshot_counter + 1	
  cv_img_filename = "%s/webcam_frame_%.2d.jpg" % (demo_dir, screenshot_counter)
  cv2.imwrite(filename=cv_img_filename, img=frame)

You can easily save an OpenCV image on disk using cv2.imwrite().

• Python 3 Demopack Download
Demo: geexlab-python3-demopack/opencv/02-image/main.xml
• GeeXLab Downloads
• Forum thread (EN)

 
This second tutorial will show you how to load an image with OpenCV and how to convert it to a GeeXLab texture for rendering.

An OpenCV image is a NumPy array (numpy.ndarray). NumPy is one of the most important libraries for scientific computing in Python. It provides all you need to work with multidimensional arrays. This article is not a tutorial about NumPy. I added some links about Numpy at the end of the article.

To load an image with OpenCV, we use the imread() function:

import cv2

demo_dir = gh_utils.get_demo_dir() 		
cv_image = cv2.imread(demo_dir + 'data/kool-image.jpg', cv2.IMREAD_COLOR)

The size of the image can be retrieved with:

im_height, img_width = cv_image.shape[0:2]

Now let’s see how to create a regular GeeXLab texture from this OpenCV image.

PF_U8_RGB = 1
PF_U8_BGR = 2
PF_U8_RGBA = 3

# BGR U8 is the default pixel format of OpenCV images.
# We create an empty texture
tex_image = gh_texture.create_2d(img_width, img_height, PF_U8_BGR)

# and we update the texture pixmap with the OpenCV image.
gh_texture.update_gpu_memory_from_numpy_img(tex_image, cv_image)

It’s done! We have now an GeeXLab texture filled with the data of an OpenCV image. It’s a detail but it can save you some time to know that OpenCV stores image pixels in BGR format…

The demo displays two images: the normal image and a smaller version of the same image. The smaller version has been created with the OpenCV resize() function:

img2_width = int(round(img_width/3))
img2_height = int(round(img_height/3))
cv_image2 = cv2.resize(cv_image,(img2_width, img2_height),interpolation=cv2.INTER_LINEAR) 

Another useful function is copyMakeBorder(). This function adds a border to an image. Here is the way to add a red border of 10 pixels:

cv_image2 = cv2.copyMakeBorder(cv_image2,
  top=10,
  bottom=10,
  left=10,
  right=10,
  borderType=cv2.BORDER_CONSTANT,
  value=(0, 0, 255))

For the color value, keep in mind that the pixel format is BGR…

Here is the demo:

 
Links:

• OpenCV 4.2.0 documentation
• OpenCV 4.2.0 image processing functions
• OpenCV 4.2.0 geometric image transformations
• OpenCV 4.2.0 image file reading and writing
• OpenCV copyMakeBorder function
• The Basics of NumPy Arrays
• Python Numpy Tutorial

• Python 3 Demopack Download
Demo: geexlab-demopack-python3/opencv/01-version/main.xml
• GeeXLab Downloads
• Forum thread (EN)

 
Here is the first article of a series a four small tutorials about OpenCV and Python 3 with GeeXlab.

What is OpenCV ?

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. Being a BSD-licensed product, OpenCV makes it easy for businesses to utilize and modify the code.

The library has more than 2500 optimized algorithms, which includes a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3D models of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a high resolution image of an entire scene, find similar images from an image database, remove red eyes from images taken using flash, follow eye movements, recognize scenery and establish markers to overlay it with augmented reality, etc.

– source –

 
The nice thing with OpenCV is that it comes with a complete Python 3 library.

The latest GeeXlab 0.29.17.0 for Windows 64-bit comes with Python 3.8.2 and OpenCV 4.2.0. So if you don’t have Python 3 on your system, you can still run Python 3 demos.

But if you want to use your own Python 3 installation, you can tell GeeXLab to use it. There are two ways:

1/ with a command line option: /python3_home="......". On Windows you should have something like:

GeeXLab /python3_home="C:/Users/YOUR_NAME/AppData/Local/Programs/Python/Python38/"

 
2/ with the python3_home attribute in the init0.xml (in GeeXLab root folder) file:

<glslhacker_init
    demofile=""
    python3_home="C:/Users/YOUR_NAME/AppData/Local/Programs/Python/Python38/"
/>

On Linux and Raspberry Pi, GeeXLab is not shipped with a Python 3 installation. It uses the Python 3 that comes with the system.

If you want to use your own Python 3 installation, let’s see how to install OpenCV for Python.

Before installing OpenCV for Python, be sure to have the pip utility. pip is the package installer for Python. If you don’t have pip, don’t worry, you can easily install it. I shipped the get-pip.py script in the Python 3 demopack. This script can be also downloaded from this page. To install pip, run the following command:

python get-pip.py

Now you should have pip installed. Now let’s install OpenCV for Python.

You can do this in command line with:

pip install opencv-python

OpenCV for Python should be now installed.

The boring task of installing OpenCV for Python is now behind us.

From now on, I supposed that Python 3 and OpenCV are ready to be used by GeeXLab.

The first demo is really simple. The goal is to check that OpenCV is installed and to display the version of OpenCV:

 
The first thing to do in a Python / OpenCV demo is to import OpenCV:

import cv2

The version of OpenCV can be read with cv2.__version__:

opencv_version = cv2.__version__
(major, minor, patch) = cv2.__version__.split(".")
opencv_version_major = int(major, base=10)
opencv_version_minor = int(minor, base=10)
opencv_version_patch = int(patch, base=10)

That’s all.

In the FRAME script, OpenCV and Python 3 versions are displayed like this (blue and yellow texts):

...

libfont2_print(font_b, 10, y_offset, 0.2, 0.7, 1.0, 1, "Python version: %d.%d.%d" %(sys.version_info.major, sys.version_info.minor, sys.version_info.micro))

y_offset = y_offset + 30
libfont2_print(font_b, 10, y_offset, 1.0, 1.0, 0.0, 1, "OpenCV version: %d.%d.%d" % (opencv_version_major, opencv_version_minor, opencv_version_patch))

...

Python and OpenCV versions are displayed in the ImGui window with:

...

gh_imgui.text_wrapped("Python " + str(sys.version))
gh_imgui.spacing()
gh_imgui.spacing()
gh_imgui.text_wrapped("OpenCV " + str(cv2.__version__))

...

 
Next article: how to load an image with OpenCV and how to display it with GeeXLab…

Downloads

• GeeXLab for Windows 64-bit
• GeeXLab for Linux 64-bit
• GeeXLab for Raspbian Buster+
• All GeeXLab Downloads
• Feedback thread / Forum (EN)

1 – Release Notes

GeeXLab 0.29.17 improves the support of Python and comes with a handy function that allows to use a NumPy array (np.array) as a texture. This function is like a bridge between NumPy and GeeXLab. Why NumPy? Because some popular Python libs like OpenCV, return NumPy images (a 2D numerical array).

In the following example, myTex is a regular GeeXLab texture and npImg is a Numpy image. We can update the texture with the new update_gpu_memory_from_numpy_img() function of the gh_texture lib:

gh_texture.update_gpu_memory_from_numpy_img(myTex, npImg)

I will start a series of Python + OpenCV articles and this function will be the core of some demos.

On Windows, the Python plugin has been updated with Python 3.8.2 that has been released few days ago. And GeeXLab for Windows comes with some popular Python libraries (NumPy, OpenCV, Scapy or feedreader) in the Python 3 site-packages folder.

On Linux, the Python 3 plugin (dylibs/plugin_gxl_python3_x64.so) is now linked to a shared object named libpython_3.so. libpython_3.so is a symlink (symbolic link) and points either to Python 3.5 shared object (on Linux Mint 18.3 for example) or to Python 3.6 (on Linux Mint 19.3). By default, libpython_3.so points to Python 3.5. The libpython_3.so should be deleted before the first launch of GeeXLab. Once deleted, it can be recreated to point to the correct version of Python using one of the following scripts (in GeeXLab folder):

– python3_plugin_update_symlink_for_python35.sh :

#!/bin/bash          
ln -s /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0 dylibs/libpython_3.so

– python3_plugin_update_symlink_for_python36.sh :

#!/bin/bash          
ln -s /usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0 dylibs/libpython_3.so

You can also link to Python 3.7 or 3.8. That should work. Sorry for that working but Python support on Linux is a bit messy.

Raspbian Buster (Raspberry Pi) is shipped with Python 3.7. So on Raspbian, GeeXLab uses Python 3.7.

2 – Changelog

This changelog is intended for all versions of GeeXLab.

Full changelog from beginning of time is available HERE.

Version 0.29.17.0 - 2020.03.05
! (2020.03.05) [Linux] the Python 3 plugin is linked with libpython_3.so, a smylink 
  towards the real Python shared object.
+ (2020.03.04) [Windows] Python 3 plugin updated plugin with Python 3.8.2.
! (2020.03.04) [Python 2/3] fixed a memory leak when a script was in live coding mode.
+ (2020.03.02) added get_num_strings(), get_string_num_chars(), set_string_char_position_offset()
  and set_string_char_color_factor() to gh_font lib.
+ (2020.02.27) [Python 3] added update_gpu_memory_from_numpy_img() to gh_texture lib.
+ (2020.02.27) [Windows] added feedreader, scapy, numpy and OpenCV in the Python 3 distribution 
  that comes with GeeXLab ({GeeXLab}/python3/Lib/site-packages/).
+ (2020.02.25) [Python] added  libfont2 in Python libs ({GeeXLab}/libs/python/libfont/).
! (2020.02.27) updated SMOL-V support with latest version.
+ (2020.02.21) added uniform_3x3f() to gh_gpu_program lib.
+ (2020.02.21) added math_from_to_rotation_matrix3x3() to gh_utils.
+ (2020.02.21) [PRO version] added check_for_new_version() to gh_utils.