OpenKWS 2013 Evaluation Infrastructure Setup and Testing Instructions
The following is the OpenKWS Evaluation Infrastructure setup and testing instructions
- Download the Babel-Formated, Spoken Term Detection data (if requested) with the URL provided by NIST to the registered POC email.
- Download the latest IndusDB 20130402 URL: https://control.nist.gov/indus/OpenKWS13/IndusDB.20130402.OpenKWS.e61f427d714d2df990c3f8cca505cf38.tar.bz2
- Download and install SCTK Version 2.4.7 or later (from http://www.nist.gov/itl/iad/mig/tools.cfm) according to the INSTALL document in the distribution.
- Download JobRunner 1.2.1 or later (from http://www.nist.gov/itl/iad/mig/tools.cfm)
- Download and check the F4DE 2.5.1 or later (from http://www.nist.gov/itl/iad/mig/tools.cfm) according to the instructions under "INSTALLATION" section in the"README" file of the distribution.
- Note, F4DE requires many CPAN libraries and other utilities. It is vital that a you successfully run either the full "make check" or minimally the 'make mincheck' and 'make KWSEvalcheck' before proceeding to Step 7.
- Send NIST Information to set up your Indus Scoring Server account which uses SSH public/private keys for authentication. (Instructions summarized from the "KWSEval/BABEL/Participants/README" in the F4DE release).
- If not familiar with SSH, please see: http://www.wikihow.com/Use-SSH
- Decide on the user name of your team; it must follow the pattern "kws-<TEAM>[_<SITE>]" (<TEAM> or <SITE> can not contain - or _)
- Generate an SSH Protocol 2 Key: Either ensure you have a valid SSH Protocol 2 public/private keys or generate a pair using either '% ssh_keygen -t dsa' or '% ssh_keygen -t rsa' with a non empty passphrase of 12 or more characters.
- email 'indus at nist.gov' with the following:
- your user, site and team names
- the list of users expected to use the service with contact information (email + phone)
- Attach your public key (not inline text) to the email.
- Send a list of the IP subnets that will need access to the ssh Indus serve
- After completing step 5 and receiving your Indus Account, follow the Indus Scorer Setup instructions found in "KWSEval/BABEL/Participants/README" of the F4DE release.
- Run the test submissions through the scoring server by:
- Change directory to the KWSEval/BABEL/Participants/Checks directory
- Execute: "./SubmissionHelper_checks.sh ../KWS13-dbDir/Comp*"
- Unpacked password-protected zip file (TestData.zip) to make sure you can read the delivered data. The password is "abc123". The file listing should be: