NIST Data Flow System II - User's Guide - Draft Version - Chapter 3

RETURN TO DOCUMENT TABLE OF CONTENTS

Chapter 3: Table of Contents

Supported architectures
Supported operating systems
Supported sensor hardware
Supported flows
Supported libraries

Supported architectures

Major hardware architectures are supported by the NDFS-II, including x86, PowerPC and AMD64 machines. By supported it means the core of the NDFS-II system will be usable and functional; some additional libraries used for signal processing may not be supported.

Customs flows are developed for the NDFS-II. Flows are usually under user scope, meaning that it is their responsibility to implement flows, and therefore to handle endianness differences.

For example, if the user creates a flow transporting audio data and acquires them on one architecture to process them on a different architecture, the user needs to make sure the data will be presented properly with the same byte order on the receiving host.

Some flows provided by the system are however endian free. Please look at the readme file or code documentation associated with each flow for more information about endianness support. Flows often have custom methods commented in the source files of the flow.

Supported operating systems

The NDFS-II has been tested on many flavors of operating systems. It runs on different Linux distributions including

• Fedora core 5,6,7,8
• Ubuntu 7.x
• Microsoft Windows XP, mobile 5
• Mac OS X (Jaguar, Tiger, Leopard)

The compatibility is mainly achieved by using common capabilities of the operating systems, the ACE, and the Qt library.

Note: If the ACE library is functional on a specific operating system, most likely the NDFS-II will be functional as well.

Supported sensor hardware

Research conducted on context sensitive environments fertilized development of basic audio and video processing clients and flows which are now made available to the community. Supported sensors include: include:

• UVC based cameras (usb) on linux, windows and osx systems. Basically all hardware supported by the OpenCV library should do well.
• HD Video Cameras (MPEG2 via firewire)
• MkIII NIST microphone array (UDP rawdata via Ethernet)
• Hamerfall ADAT multichannel audio system (used via ALSA (linux) or DirectSound (windows))

Capture is performed using NDFS-II capture clients (sources) which stream data using flows as described next.

Supported flows

Besides the example clients using 'Flow_BlockTest' we would like to mention four more complex flows which have been developed in order to facilitate our research. These flows might be a good entry point to design and develop new flows, because they address problems of transporting signal processing type data and make use of more sophisticated techniques (metadata, serialization, stream dynamics). These flows are provided with the audio/video processing library.

• The mentioned Flow_BlockTest is a simple buffer used in example clients.
• Flow_Audio_SFA transports audio data and is momentarily the most complex flow. The transported type is a class hierarchy based on the SFAudio class (smart flow audio), which can transport many types of single- to multichannel data of various sampling rate and bit-depth (32, 24, 16, 8 bit integer, channels interleaved or non-interleaved). Using metadata and typecasting, this flow demonstrates how to change type and parameters of the transported data on the fly. This means that it is possible to change bit-depth, amount of channels, sample size and sample-rate while a processing pipeline is running. A source separation processing node could for example stream 2 channels of separated audio data in case there were two sources detected, but switch later on to stream 3 channels because of the addition of a third source to the scene.
• Flow_Video_SFI transports video and image data. Only one type of data, called SFImage (smart flow image) is transported. Using metadata the flow can be used dynamically as well (change of image size on the fly). SFI wraps OpenCVs IplImage or uses a raw BGR representation on platforms where OpenCV is not available. Based on this type we have also developed a AVCodec compression flow, which encodes and decodes a image sequence on the fly, using ffmpeg. This flow will be added soon to the repository and is at the moment available only on request.
• Flow_Feature_SFM transports matrices and vectors (float). The transported data type is called SFMatrix (smart flow matrix). Handling of the matrix attributes (size, transpose) is done dynamically as well, so different types of matrices can be transported using the same stream.
• Flow_Feature_SSO transports a vector of SSO (simple scene object) data of unkown size. This flow demonstrates how to apply metadata to group single data-types (in this case the SSO class) using a standard container (vector).

Please keep in mind that the flows buffer size has to be known a priori. In order to use the dynamic features of the described flows, e.g. increase the data-size on the fly, please use buffers bigger than your current data to avoid problems.

Supported libraries

The signal processing example clients and flows make use of a moderate set of third party libraries, which demonstrates how easily system integration and cross platform interoperability can be performed. These libraries are:

• OpenGL on linux, osx and windows for real-time 3d data visualization.
• OpenCV on linux, osx and windows for image processing and pattern recognition.
• Torch3Vision on linux for image processing and pattern recognition.
• Sphinx using Java for speech recognition.
• FFMpeg on linux for video encoding and decoding.
• ACE, since NDFS-II is based on ACE the complete library functionality (threading, event management, network) is of course available as well cross-platform.
• Qt4 for cross-platform GUI development.