eGrabber Recorder

Table of Contents

• Introduction
• Programming Interface
  • Basic usage
  • Open modes
  • Close modes
  • Recorder parameters
  • Writing
    • Alignment constraints
    • Writing data
    • Creating chapters
  • Reading
    • Getting UTC timestamps
    • Walking chapters
  • Exporting
    • Exporting chapters
• Glossary
  • eGrabber
  • Metadata

Introduction

eGrabber Recorder is a dynamic library that enables image acquisition applications, such as eGrabber-based applications, to write acquired buffers to disk efficiently.

For the Recorder library, a buffer is made of two parts: the metadata (grouped in the buffer info) and the actual image pixels (component values). When a buffer is written to disk, those two parts are saved together as one entity into a Recorder container. A Recorder container is located in a user-supplied directory, its capacity is defined upfront by the user but can be further increased or reduced according to application needs or disk constraints.

Prior to recording buffers, an application needs to open a container in write mode and set its capacity (known as the container size). When this is done the application can write one or more buffers to the container.

Prior to reading buffers from a container, an application needs to open the container in read mode and set the buffer cursor (known as the record index) to the desired position. Afterwards, the application can read one or more buffers from the container.

The Recorder library also handles chapters inside a container. A chapter groups a set of buffers belonging to the same recording session. A recording application can create new chapters at any moment before writing buffers. A chapter also provides a synchronization point between system and UTC timestamps. This allows a reading application to get record buffer timestamps in UTC.

Note: this product requires a license to work; if you consider using the product, please contact Euresys.

Programming Interface

The Recorder library exposes two programming interfaces:

• The C API, a low-level C API that defines the library symbols and types
• The C++ API, a few user-friendly C++ classes built on top of the C API

In this guide, we will focus on the C++ API. Here are a few of its key assets:

• The class RecorderLibrary loads/unloads the dynamic library in its constructor/destructor
• The function openRecorder returns a Recorder object that manages the recorder handle returned and used by the C API
• The error codes returned by the library functions are converted into specific exceptions

Basic usage

Here is a typical code for creating a container and recording a few buffers:

#include <EGrabberRecorder.h>
 
using namespace Euresys::EGrabberRecorder;
 
extern void *acquireBuffer(RECORDER_BUFFER_INFO *info);
 
void write() {
    RecorderLibrary recorderLib; // load Recorder library
 
    // create a new recorder
    Recorder recorder(recorderLib.openRecorder("path/to/container", RECORDER_OPEN_MODE_WRITE));
    const int64_t ONE_MIB = 1024 * 1024;
    recorder.setParameterInteger(RECORDER_PARAMETER_CONTAINER_SIZE, ONE_MIB);
 
    // acquire and record 10 buffers
    for (int i = 0; i < 10; ++i) {
        RECORDER_BUFFER_INFO info {};
        void *pixels = acquireBuffer(&info); // acquire a buffer somehow
        recorder.write(&info, pixels); // write buffer (pixels & metadata) to container
    }
}

Typical code for reading a few buffers from a container:

#include <EGrabberRecorder.h>
 
using namespace Euresys::EGrabberRecorder;
 
extern void processPixels(void *pixels, RECORDER_BUFFER_INFO *info);
 
void read() {
    RecorderLibrary recorderLib; // load Recorder library
 
    // open a recorder
    Recorder recorder(recorderLib.openRecorder("path/to/container", RECORDER_OPEN_MODE_READ));
 
    // read all recorded buffers from container
    int64_t count = recorder.getParameterInteger(RECORDER_PARAMETER_RECORD_COUNT);
    for (int64_t i = 0; i < count; ++i) {
        RECORDER_BUFFER_INFO info {};
        std::vector<char> buffer(recorder.read(&info)); // read buffer (pixels & metadata) from container
        processPixels(&buffer[0], &info); // process somehow pixels & metadata from container
    }
}

And code for exporting images from a container:

#include <EGrabberRecorder.h>
#include <PFNC.h>
 
using namespace Euresys::EGrabberRecorder;
 
void exportImages() {
    RecorderLibrary recorderLib; // load Recorder library
 
    // open a recorder
    Recorder recorder(recorderLib.openRecorder("path/to/container", RECORDER_OPEN_MODE_READ));
    int64_t count = recorder.getParameterInteger(RECORDER_PARAMETER_RECORD_COUNT);
 
    // convert all images to Mono8 and export them as TIFF files
    recorder.exportImages("path/to/output/@n.tiff", count, Mono8);
}

In the following sections, we will further explain the recorder functionalities.

Open modes

A recorder container can be opened in 3 exclusive open modes. In this document we will call session the time during which a container is opened.

The available open modes are:

• RECORDER_OPEN_MODE_WRITE: this mode is used to create a recording session on a newly created container or to overwrite the content of an already existing container
• RECORDER_OPEN_MODE_READ: this mode is used to create a playback session on an existing container
• RECORDER_OPEN_MODE_APPEND: this mode is used to create a recording session on an existing container starting past the last record of the container

Please note that the modes are exclusive: it is not possible to open more than one session on the same container path. Moreover when a container is opened in write or append mode, it's not possible to read data from the container until it is closed; and when a container is opened in read mode, it's not possible to write data to the container until it is closed. However the library can manage several recording or playback sessions on distinct containers concurrently.

When a recording session is opened and no chapter has been explicitly created, a chapter is automatically created when the first buffer is written so that each recording session has at least its own chapter (see Creating chapters for more information).

Close modes

When a recording session is closed (explicitly using Recorder::close or implicitly when a Recorder object goes out of scope), the user can decide how the container is closed by specifying one of the close modes. Such close mode may be defined upfront when the Recorder object is created with openRecorder.

The available close modes are:

• RECORDER_CLOSE_MODE_TRIM: the container size (RECORDER_PARAMETER_CONTAINER_SIZE) is reduced to the smallest size that fits the container contents
• RECORDER_CLOSE_MODE_KEEP: the container size (RECORDER_PARAMETER_CONTAINER_SIZE) is unchanged

The following flag can be optionally added to the chosen close mode:

• RECORDER_CLOSE_MODE_DONT_TRIM_CHAPTERS: when the flag is added to the close mode, the trailing chapters will be kept even if they are empty; by default, trailing empty chapters are removed

Recorder parameters

When a recording session is opened (whatever its open mode), information or parameters of the container can be queried or modified. Two types of parameters are supported: integer (int64_t) and string; depending on the parameter, one or both types are available.

The complete list of parameters is available here.

The most important parameters are:

• RECORDER_PARAMETER_CONTAINER_SIZE (integer):
  • in write or append mode: read-write parameter to get/set the reserved container size
  • in read mode: the parameter is read-only
• RECORDER_PARAMETER_BUFFER_OPTIMAL_ALIGNMENT (integer):
  • read-only parameter, see Alignment constraints for details
• RECORDER_PARAMETER_RECORD_INDEX (integer):
  • in write or append mode: read-only parameter to get the position of the next record to write to the container
  • in read mode: read-write parameter to get/set the position of the record to read from the container
• RECORDER_PARAMETER_RECORD_COUNT (integer):
  • read-only parameter to get the number of records in the container
• RECORDER_PARAMETER_REMAINING_SPACE_IN_CONTAINER (integer):
  • read-only parameter to get the remaining space (in bytes) on the reserved container space

The parameters can be queried or modified using these functions:

• for integer parameters:
  • Recorder::getParameterInteger
  • Recorder::setParameterInteger
• for string parameters:
  • Recorder::getParameterString
  • Recorder::setParameterString

Writing

Before using a recording session (either in write or append mode), the container size must be set to reserve the container space on disk upfront.

For example, to reserve one MiB (1024^2 bytes)

recorder.setParameterInteger(RECORDER_PARAMETER_CONTAINER_SIZE, 1024 * 1024);

Alignment constraints

To achieve high write throughput, the buffer data (pixels) to record in a container must meet some alignment constraints. Those constraints are not mandatory for the library to work properly but aligned buffers will offer better performance than unaligned ones.

The alignment constraint is given by the recorder parameter RECORDER_PARAMETER_BUFFER_OPTIMAL_ALIGNMENT. Please note that the alignment value depends on the characteristics of the disk where the container is stored therefore this value should not be assumed or reused between recorder sessions.

For better performance, both the address and the size of the buffers must be aligned on that value. Aligned memory can be allocated using

• _aligned_malloc on Windows
• aligned_alloc on Linux

For Coaxlink and Gigelink users, the data stream module can be configured from eGrabber so that allocated GenTL buffers meet those constraints. Prior to allocating and announcing buffers, the data stream module can be configured as follows:

int64_t alignment = recorder.getParameterInteger(RECORDER_PARAMETER_BUFFER_OPTIMAL_ALIGNMENT);
grabber.setString<StreamModule>("BufferAllocationAlignmentControl", "Enable");
grabber.setInteger<StreamModule>("BufferAllocationAlignment", alignment);
grabber.reallocBuffers(10); // alloc and announce 10 buffers aligned for optimal recorder performance

Writing data

Writing to the container is a synchronous operation that blocks the caller until the buffer is effectively written. For example, to write the metadata info and its buffer of pixel values:

recorder.write(&info, buffer);

When the write function is done, both the metadata and the buffer data are saved to disk and both the record index and record count are incremented.

Please note that the function blocks the current thread until the I/O operations are finished. The duration will depend on the size of the buffer to write as well as on the disk speed and its current load.

Creating chapters

During a recording session, an application may need to mark the beginning of a sequence of buffers. The application can do so as follows:

recorder.startChapter("Sequence01");

This will mark the beginning of a new chapter called "Sequence01" and all subsequent buffer writes will be associated to that chapter until another chapter is created or the container is closed. It is also possible to provide a description for the chapter (as the optional second parameter of the function).

Alternatively, the chapter name can be omitted and in that case, the name will be set automatically to Chapter<xx> where <xx> is the index of the new chapter.

As already explained in the introduction, a chapter also captures an accurate synchronization point between the system time (in nanosecond since the computer booted) and the current UTC time. This synchronization point will be used while reading back from the container to convert automatically record buffer system timestamps into UTC timestamps using the corresponding chapter synchronization point.

Reading

When a container is opened in read mode, the record index is automatically set to 0 so that buffers can be read immediately from the first record.

Reading from the container is a synchronous operation. It's possible to read

• only buffer metadata
• only buffer data (pixels)
• buffer metadata and data

In the following example we read the buffer metadata into info and we get the buffer data without explicitly allocating memory.

RECORDER_BUFFER_INFO info {};
std::vector<char> buffer(recorder.read(&info));

The Recorder class provides two read functions:

• a simple read(RECORDER_BUFFER_INFO *info) that returns a std::vector<char> with the buffer data, and optionally returns the buffer info in the provided structure (info can be set to nullptr if the info is not needed)
• a more advanced read(RECORDER_BUFFER_INFO *info, void *buffer, size_t size) that copies buffer data from the container into a user-supplied buffer, please refer to the documentation of RecorderRead of the C API for more details about this function.

When the read function is done, the record index is incremented only if buffer data was read successfully. Please note that reading buffer info only does not move the record index.

Getting UTC timestamps

When a buffer is written to the container, its timestamp is given by the application in the metadata. This timestamp must be expressed in nanoseconds since the computer booted. In the buffer metadata, there is also a field called utc where the application may provide the UTC timestamp of the buffer. The application can also set the utc field to 0; this is the recommended value.

While reading buffer metadata info from the container, all the metadata fields are retrieved as they were saved except the field utc, which may have a different value.

• if utc was different from 0 when written: its value is returned as is when reading
• if utc was 0 when written: the library computes the buffer UTC timestamp by converting the buffer system timestamp (expressed in nanoseconds since the computer booted) to UTC using the pair of synchronized timestamps of the buffer chapter

Walking chapters

An application can walk through the collection of chapters stored in a container. For each chapter, the application can get details such as the chapter name, description, creation time (in system time as well as in UTC) and the number of records. These operations are done by setting and getting the chapter-related recorder parameters.

The following code gets the list of available chapter names of a container opened in read mode:

#include <EGrabberRecorder.h>
 
using namespace Euresys::EGrabberRecorder;
 
void listChapters(std::vector<std::string> &chapterNames,
                  std::vector<std::string> &chapterUserInfos) {
    RecorderLibrary recorderLib; // load Recorder library
 
    // open a recorder
    Recorder recorder(recorderLib.openRecorder("path/to/container", RECORDER_OPEN_MODE_READ));
 
    // get name and user info of all chapters stored in the container
    int64_t chapterCount = recorder.getParameterInteger(RECORDER_PARAMETER_CHAPTER_COUNT);
    chapterNames.resize(chapterCount);
    chapterUserInfos.resize(chapterCount);
    for (int64_t i = 0; i < chapterCount; ++i) {
        recorder.setParameterInteger(RECORDER_PARAMETER_CHAPTER_INDEX, i);
        chapterNames[i] = recorder.getParameterString(RECORDER_PARAMETER_CHAPTER_NAME);
        chapterUserInfos[i] = recorder.getParameterString(RECORDER_PARAMETER_CHAPTER_USER_INFO);
    }
}

Whereas this sample shows how to get the UTC timestamps of all the buffers stored in the last chapter of a container:

#include <EGrabberRecorder.h>
 
using namespace Euresys::EGrabberRecorder;
 
void getUtcTimestampsOfLastSequence(std::vector<uint64_t> &utcTimestamps) {
    RecorderLibrary recorderLib; // load Recorder library
 
    // open a recorder
    Recorder recorder(recorderLib.openRecorder("path/to/container", RECORDER_OPEN_MODE_READ));
 
    // select the last chapter
    int64_t chapterCount = recorder.getParameterInteger(RECORDER_PARAMETER_CHAPTER_COUNT);
    recorder.setParameterInteger(RECORDER_PARAMETER_CHAPTER_INDEX, chapterCount - 1);
 
    // walk through the records of the selected chapter
    int64_t recordCount = recorder.getParameterInteger(RECORDER_PARAMETER_CHAPTER_RECORD_COUNT);
    utcTimestamps.resize(recordCount);
    for (int64_t i = 0; i < recordCount; ++i) {
        RECORDER_BUFFER_INFO info {};
        // the record index is set prior to reading the buffer metadata because when reading
        // metadata only, the record index is not incremented automatically
        recorder.setParameterInteger(RECORDER_PARAMETER_CHAPTER_RECORD_INDEX, i);
        recorder.read(&info, NULL, 0); // read buffer metadata only
        utcTimestamps[i] = info.utc; // the buffer timestamp in UTC (either converted or provided)
    }
}

Exporting

Images can also be exported from a container opened in read mode. For example, to export one image:

recorder.exportImages("path/to/output/image.tiff", 1);

The record index is incremented by exportImages. This is similar to the read function.

The first argument to exportImages (the output path) can include one or more @ patterns. For example, to export three images named image.0.tiff, image.1.tiff, and image.2.tiff:

recorder.exportImages("image.@n.tiff", 3);

Optionally, images can be converted by exportImages. For example:

recorder.exportImages("export.mkv", 1000, RGB8);

Please refer to the documentation of RecorderExport of the C API for more details.

Exporting chapters

Exporting a complete container into chapters as .mkv (Matroska video) files can be achieved in one operation (assuming the record index is set to the beginning of the container, i.e. 0):

recorder.exportImages("@c-@C.mkv", recorder.getParameterInteger(RECORDER_PARAMETER_RECORD_COUNT), RGB8);

This will produce one .mkv file for each chapter. Each file will be named @c-@C.mkv where

• @c will be replaced by the chapter index
• @C will be replaced by the chapter name

Glossary

eGrabber

eGrabber is a set of Euresys C++ classes that provide a high-level interface on top of GenTL. eGrabber classes expose a grabber-oriented interface that hides the complexity of the underlying hierarchical structure of GenTL modules; it's the recommended API to use Coaxlink frame grabbers.

Metadata

The buffer metadata are all the buffer information except the actual pixel component values; they include the buffer width, height and pixel format; please refer to RECORDER_BUFFER_INFO for an exhaustive list of metadata supported by the recorder library.