edu.ucla.sspace.mains

Class RandomIndexingMain

java.lang.Object

edu.ucla.sspace.mains.GenericMain

edu.ucla.sspace.mains.RandomIndexingMain

public class RandomIndexingMain
extends GenericMain

An executable class for running RandomIndexing from the command line. This class provides several options:

• Required (at least one of):
  • -d, --docFile=FILE[,FILE...] a file where each line is a document
  • -f, --fileList=FILE[,FILE...] a list of document files
• Algorithm Options:
  • -l, --vectorLength=INT length of semantic vectors
  • -p, --usePermutations=BOOL whether to permute index vectors based on word order
  • -s, --windowSize=INT how many words to consider in each direction
  • -L, --loadVectors=FILE specifies a file containing word-to-index vector mappings that should be used by the RandomIndxing instance. This allows multiple invocations of this program to reuse the same semantic space.
  • -S, --saveVectors=FILE specifies a file in which the word-to-index vector mappings will be saved after the RandomIndxing instance has finished processing all the documents. When used in conjunction with the --loadVectors option, this allows later invocations of this program to reuse the this invocation's semantic space.
  • -F, --tokenFilter=FILE[include|exclude][,FILE...] specifies a list of one or more files to use for filtering the documents. An option flag may be added to each file to specify how the words in the filter filter should be used: include if only the words in the filter file should be retained in the document; exclude if only the words not in the filter file should be retained in the document.
• Advanced Algorithm Options:
  • -n, --permutationFunction=CLASSNAME the PermutationFunction class to use for permuting TernaryVectors, if permutation is enabled.
• Program Options:
  • -o, --outputFormat=text|binary} Specifies the output formatting to use when generating the semantic space (.sspace) file. See SemanticSpaceUtils for format details.
  • -t, --threads=INT the number of threads to use
  • -v, --verbose prints verbose output
  • -w, --overwrite=BOOL specifies whether to overwrite the existing output

An invocation will produce one file as output random-indexing.sspace. If overwrite was set to true, this file will be replaced for each new semantic space. Otherwise, a new output file of the format random-indexing<number>.sspace will be created, where <number> is a unique identifier for that program's invocation. The output file will be placed in the directory specified on the command line.

This class is desgined to run multi-threaded and performs well with one thread per core, which is the default setting.

Author:

David Jurgens

See Also:

RandomIndexing

Field Summary

Fields inherited from class edu.ucla.sspace.mains.GenericMain

argOptions, EXT, isMultiThreaded, verbose

Method Summary

Methods

Modifier and Type	Method and Description
protected void	addExtraOptions(ArgOptions options) Adds all of the options to the ArgOptions.
protected SemanticSpace	getSpace() Returns an instance of RandomIndexing.
protected SemanticSpaceIO.SSpaceFormat	getSpaceFormat() Returns the format as the default format of a RandomIndexing space.
static void	main(String[] args)
protected void	postProcessing() If --saveVectors was specified, write the accumulated word-to-index vector mapping to file.
protected Properties	setupProperties() Returns the Properties object that will be used when calling SemanticSpace.processSpace(Properties).

Methods inherited from class edu.ucla.sspace.mains.GenericMain

addCorpusReaderIterators, addDocIterators, addFileIterators, getAlgorithmSpecifics, getDocumentIterator, handleExtraOptions, loadValidTermSet, parseDocumentsMultiThreaded, parseDocumentsSingleThreaded, processDocumentsAndSpace, run, saveSSpace, setupOptions, usage, verbose, verbose

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

addExtraOptions

protected void addExtraOptions(ArgOptions options)

Adds all of the options to the ArgOptions.

Overrides:

addExtraOptions in class GenericMain

Parameters:

options - the ArgOptions object which more main specific options can be added to.

See Also:

GenericMain.handleExtraOptions()

main

public static void main(String[] args)

setupProperties

protected Properties setupProperties()

Returns the Properties object that will be used when calling SemanticSpace.processSpace(Properties). Subclasses should override this method if they need to specify additional properties for the space. This method will be called once before GenericMain.getSpace().

Overrides:

setupProperties in class GenericMain

Returns:

the Properties used for processing the semantic space.

getSpace

protected SemanticSpace getSpace()

Returns an instance of RandomIndexing. If loadVectors is specified in the command line options, this method will also initialize the word-to-TernaryVector mapping.

Specified by:

getSpace in class GenericMain

getSpaceFormat

protected SemanticSpaceIO.SSpaceFormat getSpaceFormat()

Returns the format as the default format of a RandomIndexing space.

Overrides:

getSpaceFormat in class GenericMain

Returns:

the format in which the semantic space will be saved

postProcessing

protected void postProcessing()

If --saveVectors was specified, write the accumulated word-to-index vector mapping to file.

Overrides:

postProcessing in class GenericMain