edu.ucla.sspace.vsm

Class VectorSpaceModel

java.lang.Object

edu.ucla.sspace.common.GenericTermDocumentVectorSpace

edu.ucla.sspace.vsm.VectorSpaceModel

All Implemented Interfaces:

SemanticSpace

public class VectorSpaceModel
extends GenericTermDocumentVectorSpace

An implementation of the Vector Space Model (VSM). This model was first based on the paper

• G. Salton, A. Wong, and C. S. Yang (1975), "A Vector Space Model for Automatic Indexing," Communications of the ACM, vol. 18, nr. 11, pages 613–620. Available here

The VSM first processes documents into a word-document matrix where each unique word is a assigned a row in the matrix, and each column represents a document. The values of ths matrix correspond to the number of times the row's word occurs in the column's document. Optionally, after the matrix has been completely, its values may be transformed. This is frequently done using the Tf-Idf Transform.

This class offers one configurable parameter.

Property: "edu.ucla.sspace.vsm.VectorSpaceModel.transform"
Default: none

This variable sets the preprocessing algorithm to use on the term-document matrix. The property value should be the fully qualified named of a class that implements Transform. The class should be public, not abstract, and should provide a public no-arg constructor.

This class is thread-safe for concurrent calls of processDocument. Once processSpace has been called, no further calls to processDocument should be made. This implementation does not support access to the semantic vectors until after processSpace has been called.

Author:

David Jurgens

See Also:

Transform

Field Summary

Fields

Modifier and Type	Field and Description
static String	MATRIX_TRANSFORM_PROPERTY The property to define the Transform class to be used when processing the space after all the documents have been seen.

Fields inherited from class edu.ucla.sspace.common.GenericTermDocumentVectorSpace

documentCounter, LOG, wordSpace

Constructor Summary

Constructors

Constructor and Description
VectorSpaceModel() Constructs the VectorSpaceModel using the system properties for configuration.
VectorSpaceModel(boolean readHeaderToken, BasisMapping<String,String> termToIndex, MatrixBuilder termDocumentMatrixBuilder) Constructs a new VectorSpaceModel using the provided objects for processing.

Method Summary

Methods

Modifier and Type	Method and Description
String	getSpaceName() Returns a unique string describing the name and configuration of this algorithm.
void	processSpace(Properties properties) Once all the documents have been processed, performs any post-processing steps on the data.

Methods inherited from class edu.ucla.sspace.common.GenericTermDocumentVectorSpace

getVector, getVectorLength, getWords, handleDocumentHeader, processDocument, processSpace

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MATRIX_TRANSFORM_PROPERTY

public static final String MATRIX_TRANSFORM_PROPERTY

The property to define the Transform class to be used when processing the space after all the documents have been seen.

See Also:

Constant Field Values

Constructor Detail

VectorSpaceModel

public VectorSpaceModel()
                 throws IOException

Constructs the VectorSpaceModel using the system properties for configuration.

Throws:

IOException - if this instance encounters any errors when creatng the backing array files required for processing

VectorSpaceModel

public VectorSpaceModel(boolean readHeaderToken,
                BasisMapping<String,String> termToIndex,
                MatrixBuilder termDocumentMatrixBuilder)
                 throws IOException

Constructs a new VectorSpaceModel using the provided objects for processing.

Parameters:

readHeaderToken - If true, the first token of each document will be read and passed to handleDocumentHeader, which discards the header.

termToIndex - The BasisMapping used to map strings to indices.

termDocumentMatrixBuilder - The MatrixBuilder used to write document vectors to disk which later get processed in processSpace.

Throws:

IOException - if this instance encounters any errors when creatng the backing array files required for processing

Method Detail

getSpaceName

public String getSpaceName()

Returns a unique string describing the name and configuration of this algorithm. Any configurable parameters that would affect the resulting semantic space should be expressed as a part of this name.

processSpace

public void processSpace(Properties properties)

Once all the documents have been processed, performs any post-processing steps on the data. An algorithm should treat this as a no-op if no post-processing is required. Callers may specify the values for any exposed parameters using the properties argument.

By general contract, once this method has been called, processDocument will not be called again.

Parameters:

properties - a set of properties and values that may be used to configure any exposed parameters of the algorithm. See this class's javadoc for the full list of supported properties.