NIST Data Flow System II - User's Guide - Draft Version - Chapter 4

RETURN TO DOCUMENT TABLE OF CONTENTS

Chapter 4: Table of Contents

Dependencies
Installation 
 -- Linux/Unix or Mac OS X 
 -- Windows XP
Checking your firewall
Testing your installation
Web Control Center Installation

Dependencies

The NDFS-II has few dependencies in order to ease the portability. The two main dependencies are the ADAPTIVE Communication Environment (ACE) and Qt from Trolltech.

The ACE framework implements many useful core patterns such as shared memory management, event demultiplexing and event handler dispatching, service initialization, or explicit dynamic linking that function across a variety of operating system. Moreover ACE was designed to support applications having quality of service requirements, including high performance for bandwidth-intensive and real-time applications. All these reasons oriented our choice to use ACE to implement the core of NDFS-II.

ACE can be downloaded here: http://www.cs.wustl.edu/~schmidt/ACE.html

Qt4 provides a high level API and a set of tools to design GUI. So we chose it to implement all GUI used to perform control over the NDSF-II network. Qt4 can be downloaded here: http://www.trolltech.com/

If you intend to deploy the system on a cluster of computers, you most likely won't need any graphical user interface on some of the machine. Qt is not needed in that case.

Installation

The NDFS-II is available for download at http://www.nist.gov/smartspace

Linux/Unix or Mac OS X

First, you need to install the ACE (Adaptive Communication Environment) 5.* libraries and Qt4 libraries for graphical tools and client nodes on your system. If you don't intend to use Qt-based programs on a specific system, Qt is optional.

ACE can be compiled from sources or installed from packages. If you choose this last option, be sure to install the ace and ace-xml packages and the associated development packages as well.

Once ACE is installed you need to set up the $ACE_ROOT variable if it has been installed in a custom folder by modifying your .profile, .login, .tcshrc or .bashrc file. If you don't want to edit one of these scripts, you can customize the files env.csh (if you have csh or tcsh shell), or env.sh (if you have a bash shell) located in the NDFS-II root folder and load the environment variables specified in the script.

If you install ACE in /opt/ACE you can add in your .login, .profile or .tcshrc (in case your shell is csh or tcsh).

setenv ACE_ROOT /opt/ACE
#for linux/unix
setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:${ACE_ROOT}/lib
#for Mac OS X
setenv DYLD_LIBRARY_PATH ${DYLD_LIBRARY_PATH}:${ACE_ROOT}/lib

If your shell is bash, you need to add the following lines in your .login, .profile or .bashrc:

ACE_ROOT=/opt/ACE
#for linux/unix
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${ACE_ROOT}/lib
#for Mac OS X
DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:${ACE_ROOT}/lib

export ACE_ROOT

After you have installed ACE, you need to make sure Qt4 is installed on your system if you plan on using the Graphical User Interfaces provided with the framework. If Qt is not present, you need to install it. The development package of Qt is required. After its installation, make sure you properly set up your environment variables. If you install Qt in /opt/qt, you need to add the following lines in your .login, .profile or .bashrc (in case your shell is bash):

QTDIR=/opt/qt
PATH=$QTDIR/bin:$PATH

#for linux/unix
LD_LIBRARY_PATH=$QTDIR/lib:$LD_LIBRARY_PATH

#for Mac OS
DYLD_LIBRARY_PATH=$QTDIR/lib:$DYLD_LIBRARY_PATH

#for linux/unix
export QTDIR PATH MANPATH LD_LIBRARY_PATH

#for Mac OS
export QTDIR PATH MANPATH DYLD_LIBRARY_PATH

If your shell is csh or tcsh, you need to add the following lines in your .tcshrc, .login or .profile

setenv QTDIR /opt/qt
setenv PATH $QTDIR/bin:$PATH

#for linux/unix
setenv LD_LIBRARY_PATH $QTDIR/lib:$LD_LIBRARY_PATH

#for Mac OS
setenv DYLD_LIBRARY_PATH $QTDIR/lib:$DYLD_LIBRARY_PATH

After Qt and ACE are properly installed, you can take care of the NDFS-II. First you need to unpack the archive if you haven't already done it:

tar -xzf Smartflow2.tar.gz #uncompress and unpack the archive

Now you need to set up some environment variables for the NDFS-II such as the SF2_ROOT variable. Here the NDFS-II has been uncompressed in /opt. Add this following lines to your .login, .profile or .tcshrc file (if your shell is csh or tcsh):

setenv SF2_ROOT /opt/Smartflow2

#for linux/unix
setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:${SF2_ROOT}/lib:${SF2_ROOT}/lib/flows:
${SF2_ROOT}/lib/utility

#for Mac OS
setenv DYLD_LIBRARY_PATH ${DYLD_LIBRARY_PATH}:${SF2_ROOT}/lib:${SF2_ROOT}/lib/flows:
${SF2_ROOT}/lib/utility

setenv PATH ${PATH}:${SF2_ROOT}/bin/admin:${SF2_ROOT}/bin/clients:${SF2_ROOT}/bin/tools

If you use bash, add the following lines to your .login, .profile or .bashrc

SF2_ROOT=/opt/Smartflow2

#for linux/unix
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$SF2_ROOT/lib:$SF2_ROOT/lib/flows:$SF2_ROOT/lib/utility

#for Mac OS
DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$SF2_ROOT/lib:$SF2_ROOT/lib/flows:$SF2_ROOT/lib/utility

#for both
PATH=$PATH:$SF2_ROOT/bin:$SF2_ROOT/bin/admin:${SF2_ROOT}/bin/clients:${SF2_ROOT}/bin/tools

#for linux/unix
export SF2_ROOT LD_LIBRARY_PATH PATH

#for Mac OS
export SF2_ROOT DYLD_LIBRARY_PATH PATH 

You are now ready to build the system. There are presently two ways to do it, using the Qt method with qmake or using the configure; make; make install method. Having two different ways to build the project on the same platform can seem odd. Originally qmake was used, since it is a very convenient way to build the project. However, it requires Qt installed on the machine. If you intend to deploy the framework on machines without need for any graphical tool of the system, you don't need Qt so we don't want to force the installation of Qt only for the purpose of building the project.

Note: If you experienced some linking errors with ACE while building the data flow on Mac OS X, set the environment variable MACOSX_DEPLOYMENT_TARGET to 10.4.

Building the system using qmake

The NDFS-II comes with a variety of client nodes that can capture video data from a camera, or audio data from a NIST Mk-III microphone array, etc. Some of these client nodes have specific dependencies. By default, these client nodes won't be built but you can enable them by editing the file sf2.pro.inc located at the root folder of the NDFS-II folder (named Smartflow2 for historical reasons). To do this, un comment the following lines:

#If you have OpenCV installed
CONFIG += sf2_opencv

#If you have OpenGL installed
CONFIG += sf2_opengl

#To compile Audio Processing clients (microphone array/array signal processing)
CONFIG += sf2_audio_processing

#To compile Video Processing clients (capturing, compression, computer vision, sensor fusion)
CONFIG += sf2_video_processing

You are now ready to build NDFS-II. Qmake will generate the Makefiles. Run qmake in the root directory of the NDFS-II. Once the Makefiles are created, run make to build NDFS-II. It first compiles the NDFS-II library, then the server (and duplicator), the flows, the client nodes and finally the tools.

Building the system using the configure script

NDFS-II can also be built using the configure; make; make install method. A lot of parameters such as the destination directory can be overridden. To get the details, just invoke:

configure --help

The configure script automatically detects the libraries installed on your machine and generates the Makefiles for the client nodes accordingly. For example, if it finds opencv, it generates the Makefiles for the client nodes using opencv.

After you run the configure script, just type make and then make install to finish the installation.

If no prefix is specified when running the configure script, the destination folder will be /usr/local/bin for executables and /usr/local/lib for the libraries. When building the system this way, the sub folders used to sort the executable's programs of the system and the libraries are not created and every executable and library is saved in the bin and lib folders.

In that case, you need to update the PATH and LD_LIBRARY_PATH environment variables accordingly.

The configure script presently does not build the Control Center under Mac OS X. If you need it, build it using the qmake method provided by Qt.

Windows XP

There are two ways to install the NDFS-II on Windows XP.

An installer is provided including every binary of the system and the sources, the ACE and Qt libraries, and some libraries used in specific client nodes.

The source can also be downloaded and the system built using Visual Studio 2005. A Visual Studio solution file is provided and located in the root folder of the system.

If you download the project sources, you will need to download Qt and ACE and build them before building the NDFS-II.

Checking your firewall

The NDFS-II needs to have some ports opened for incoming connections to function properly. The data flow server needs the TCP ports 1713 to 1716 and the UPD port 20001 opened to operate. The TCP ports 25000 and some above need also to be opened for data transport.

Duplicators are the programs that transport the data. Each duplicator has a specific port; they take the first available port after 25000. Having the ports 25000 to 25050 opened allow you to handle 50 duplicators on a machine, i.e. 50 different instances of flows.

If you are running the NDFS-II under Windows XP, allow sf2d.exe and duplicator.exe to accept incoming connections

Testing your installation

Now that the NDFS-II is installed, you can test it. To do so you first need to launch a NDFS-II server. The server executable 'sf2d' is located in Smartflow/bin/admin if you built the system using the qmake method, or in /usr/local/bin by default using the configure; make; make install method.

The server will open all necessary ports for incoming connections.

Once the server has been launched, go to the folder Smartflow/bin/clients and launch the client nodes example_provider_memcpy and example_consumer_memcpy.

The first client node sends double precision numbers to the second client. If you can see the data on the consumer side, the system is working properly.

You can perform this test within a machine and between machines to make sure the transport across the network is functional. In this last case, you need to start a data flow server on each host used for testing.

Web Control Center Installation

A web version of the Control Center is also provided with the system. It allows building and controlling application graphs from a web browser. In that case, it is then possible to build and control applications from computers where no prior installation of NDFS-II is required.

The Web Control Center needs however to be installed on a computer. It has been developed in Java and requires a Tomcat server to run.

You first need to install the NDFS-II on the machine. The last version of the Java Development Kit (JDK) is also required and is available on the Sun Java website (http://java.sun.com).

The next step is to install a Tomcat Java server. You can download it on http://tomcat.apache.org In addition you need to setup, build and run the following two modules:

• The "Smartflow2JavaApplet":
  
  This module allows the Web Control Center module running on the Tomcat Java server to send requests to the data flow server using our Java wrapper.

1. 1. The environment variable TOMCAT_HOME must point to your Tomcat distribution directory.
   2. Compile and setup the module:
      
      Change your directory to java/tools/ControlCenterJavaVersion/Smartflow2JavaApplet and execute the following commands:
      
      javac dist/*.java
      
      javac *.java
      
      rmic -v1.2 ControlCenterServer
      
      cp -f *Stub* $TOMCAT_HOME/webapps/ROOT/ControlCenter/build/classes/
      javah -jni SmartFlowJava
      
      qmake
      
      make

If your are using Mac OS X please add this:

cp libSmartFlowJava.dylib libSmartFlowJava.jnilib

1. 1. Before trying to run the module make sure that you have the sf2d server running.
      1. Set your Library Path:

   #Linux: export
   LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
   #Mac OS X:
        export DYLD_LIBRARY_PATH=.:$DYLD_LIBRARY_PATH

1. 1. 1. Run the Java server:

   java ControlCenterServer

• The "Web Control Center" module:
  
  This module programmed in Java to run on the Tomcat Server provides the same functionalities as the Control Center and can be seen as a clone of it.

1. First, you need to start the Tomcat server.
   
   cd $TOMCAT_HOME/bin
   
   ./stratup.sh
2. One way to deploy the applet is to copy both of the Control Center folder and the "ndfs2_doc_index.html" file located at java/tools/ControlCenterJavaVersion to your $TOMCAT_HOME/webapps/ROOT folder.

Once this is done, you can try to access the Web Control Center using any Web browser using the address http://localhost from the computer you installed it on.