edu.ucla.sspace.rri

Class ReflectiveRandomIndexing

java.lang.Object

edu.ucla.sspace.rri.ReflectiveRandomIndexing

All Implemented Interfaces:

Filterable, SemanticSpace

public class ReflectiveRandomIndexing
extends Object
implements SemanticSpace, Filterable

An implementation of Reflective Random Indexing, which uses a two passes through the corpus to build semantic vectors that better approximate indirect co-occurrence. This implementation is based on the paper:

This class defines the following configurable properties that may be set using either the System properties or using the ReflectiveRandomIndexing(Properties) constructor.

Property: "edu.ucla.sspace.ri.ReflectiveRandomIndexing.vectorLength"
Default: 4000

This property sets the number of dimensions to be used for the index and semantic vectors.

Property: "edu.ucla.sspace.ri.ReflectiveRandomIndexing.sparseSemantics"
Default: true

This property specifies whether to use a sparse encoding for each word's semantics. Using a sparse encoding can result in a large saving in memory, while requiring more time to process each document.

This class implements Filterable, which allows for fine-grained control of which semantics are retained. The setSemanticFilter(Set) method can be used to speficy which words should have their semantics retained. Note that the words that are filtered out will still be used in computing the semantics of other words. This behavior is intended for use with a large corpora where retaining the semantics of all words in memory is infeasible.

This class is thread-safe for concurrent calls of processDocument. The getVector method will only return valid reflective vectors after the call to processSpace.

Author:

David Jurgens

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_VECTOR_LENGTH The default number of dimensions to be used by the index and semantic vectors.
static String	USE_SPARSE_SEMANTICS_PROPERTY Specifies whether to use a sparse encoding for each word's semantics, which saves space but requires more computation.
static String	VECTOR_LENGTH_PROPERTY The property to specify the number of dimensions to be used by the index and semantic vectors.

Constructor Summary

Constructors

Constructor and Description
ReflectiveRandomIndexing() Creates a new ReflectiveRandomIndexing instance using the current System properties for configuration.
ReflectiveRandomIndexing(Properties properties) Creates a new ReflectiveRandomIndexing instance using the provided properites for configuration.

Method Summary

Methods

Modifier and Type	Method and Description
String	getSpaceName() Returns a unique string describing the name and configuration of this algorithm.
IntegerVector	getVector(String word) Returns the semantic vector for the provided word.
int	getVectorLength() Returns the length of vectors in this semantic space.
Set<String>	getWords() Returns the set of words that are represented in this semantic space.
void	processDocument(BufferedReader document) Updates the semantic vectors based on the words in the document.
void	processSpace(Properties properties) Computes the reflective semantic vectors for word meanings
void	setSemanticFilter(Set<String> semanticsToRetain) Specifies the set of words that should have their semantics retained, where all other words do not.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VECTOR_LENGTH_PROPERTY

public static final String VECTOR_LENGTH_PROPERTY

The property to specify the number of dimensions to be used by the index and semantic vectors.

See Also:

Constant Field Values

USE_SPARSE_SEMANTICS_PROPERTY

public static final String USE_SPARSE_SEMANTICS_PROPERTY

Specifies whether to use a sparse encoding for each word's semantics, which saves space but requires more computation.

See Also:

Constant Field Values

DEFAULT_VECTOR_LENGTH

public static final int DEFAULT_VECTOR_LENGTH

The default number of dimensions to be used by the index and semantic vectors.

See Also:

Constant Field Values

Constructor Detail

ReflectiveRandomIndexing

public ReflectiveRandomIndexing()

Creates a new ReflectiveRandomIndexing instance using the current System properties for configuration.

ReflectiveRandomIndexing

public ReflectiveRandomIndexing(Properties properties)

Creates a new ReflectiveRandomIndexing instance using the provided properites for configuration.

Method Detail

getVector

public IntegerVector getVector(String word)

Returns the semantic vector for the provided word.

Specified by:

getVector in interface SemanticSpace

Parameters:

word - a word that may be in the semantic space

Returns:

The Vector for the provided word or null if the word was not in the space.

getSpaceName

public String getSpaceName()

Returns a unique string describing the name and configuration of this algorithm. Any configurable parameters that would affect the resulting semantic space should be expressed as a part of this name.

Specified by:

getSpaceName in interface SemanticSpace

getVectorLength

public int getVectorLength()

Returns the length of vectors in this semantic space. Implementations are left free to define whether the returned value is valid before processSpace is called.

Specified by:

getVectorLength in interface SemanticSpace

getWords

public Set<String> getWords()

Returns the set of words that are represented in this semantic space.

Specified by:

getWords in interface SemanticSpace

Returns:

the set of words that are represented in this semantic space.

processDocument

public void processDocument(BufferedReader document)
                     throws IOException

Updates the semantic vectors based on the words in the document.

Specified by:

processDocument in interface SemanticSpace

Parameters:

document - a reader that allows access to the text of the document

Throws:

IOException - if any error occurs while reading the document

processSpace

public void processSpace(Properties properties)

Computes the reflective semantic vectors for word meanings

Specified by:

processSpace in interface SemanticSpace

Parameters:

properties - a set of properties and values that may be used to configure any exposed parameters of the algorithm.

setSemanticFilter

public void setSemanticFilter(Set<String> semanticsToRetain)

Specifies the set of words that should have their semantics retained, where all other words do not. Note that all words will still have an index vector assigned to them, which is necessary to properly compute the semantics.

Specified by:

setSemanticFilter in interface Filterable

Parameters:

semanticsToRetain - the set of words for which semantics should be computed.