edu.ucla.sspace.gws

Class GenericWordSpace

java.lang.Object

edu.ucla.sspace.gws.GenericWordSpace

All Implemented Interfaces:

DimensionallyInterpretableSemanticSpace<String>, Filterable, SemanticSpace, Serializable

public class GenericWordSpace
extends Object
implements DimensionallyInterpretableSemanticSpace<String>, Filterable, Serializable

The most basic co-occurrence model that counts word co-occurrence within a sliding window with no further processing. This class is meant as a generic model that can be used to measure the efficacy of other SemanticSpace instances.

This class also provides for a slight variation on the basic model by differentiating co-occurrences on the basis of their relative position to the focus word. In such a case, for example, an occurrence of "red" two before the focus word would be represented by a different position than "red" one position before. This is reminiscent of the RandomIndexing model with permutations. However, unlike Random Indexing, this model is not fixed in the number of dimensions it may use, with a possible numWords * windowSize * 2 dimensions. Such a large number of dimensions can negatively impact the further operations on the semantic space's vectors, e.g., finding the most similar vectors for a word.

The dimensions of this space are annotated with a description of what they represent. In the basic model, this will be the co-occurring word. In the model that takes into account word order, the description will include the relative position of the word.

This class defines the following configurable properties that may be set using either the System properties or using the GenericWordSpace(Properties) constructor.

Property: "edu.ucla.sspace.gws.GenericWordSpace.windowSize"
Default: 2

This property sets the number of words before and after that are counted as co-occurring. With the default value, 2 words are counted before and 2 words are counter after. This class always uses a symmetric window.

Property: "edu.ucla.sspace.gws.GenericWordSpace.useWordOrder"
Default: false

This property sets whether co-occurrences of the same word should be distinguished on the basis of their relative position to the focus word.

This class implements Filterable, which allows for fine-grained control of which semantics are retained. The setSemanticFilter(Set) method can be used to speficy which words should have their semantics retained. Note that the words that are filtered out will still be used in computing the semantics of other words. This behavior is intended for use with a large corpora where retaining the semantics of all words in memory is infeasible.

This class is thread-safe for concurrent calls of processDocument. At any given point in processing, the getVector method may be used to access the current semantics of a word. This allows callers to track incremental changes to the semantics as the corpus is processed.

The processSpace method does nothing for this class and calls to it will not affect the results of getVector.

Author:

David Jurgens

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_WINDOW_SIZE The default number of words to view before and after each word in focus.
static String	GWS_SSPACE_NAME
static String	USE_WORD_ORDER_PROPERTY The property to specify whether the relative positions of a word's co-occurrence should be use distinguished from each other.
static String	WINDOW_SIZE_PROPERTY The property to specify the number of words to view before and after each word in focus.

Constructor Summary

Constructors

Constructor and Description
GenericWordSpace() Creates a new GenericWordSpace instance using the current System properties for configuration.
GenericWordSpace(int windowSize) Creates a new GenericWordSpace with the provided window size that ignores word order.
GenericWordSpace(int windowSize, BasisMapping<Duple<String,Integer>,String> basis) Creates a new GenericWordSpace with the provided window size that uses the specified basis mapping to map each co-occurrence at a specified position to a dimension.
GenericWordSpace(int windowSize, boolean useWordOrder) Creates a new GenericWordSpace with the provided window size that optionally includes word order.
GenericWordSpace(Properties properties) Creates a new GenericWordSpace instance using the provided properites for configuration.

Method Summary

Methods

Modifier and Type	Method and Description
void	clearSemantics() Removes all associations between word and semantics while still retaining the words' basis mapping.
String	getDimensionDescription(int dimension) Returns a description of what features with which the specified dimension corresponds.
String	getSpaceName() Returns a unique string describing the name and configuration of this algorithm.
SparseIntegerVector	getVector(String word) Returns the semantic vector for the provided word.
int	getVectorLength() Returns the length of vectors in this semantic space.
Set<String>	getWords() Returns the set of words that are represented in this semantic space.
void	processDocument(BufferedReader document) Updates the semantic vectors based on the words in the document.
void	processSpace(Properties properties) Does nothing.
void	setSemanticFilter(Set<String> semanticsToRetain) Specifies the set of words that should have their semantics retained, where all other words do not.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

GWS_SSPACE_NAME

public static final String GWS_SSPACE_NAME

See Also:

Constant Field Values

WINDOW_SIZE_PROPERTY

public static final String WINDOW_SIZE_PROPERTY

The property to specify the number of words to view before and after each word in focus.

See Also:

Constant Field Values

USE_WORD_ORDER_PROPERTY

public static final String USE_WORD_ORDER_PROPERTY

The property to specify whether the relative positions of a word's co-occurrence should be use distinguished from each other.

See Also:

Constant Field Values

DEFAULT_WINDOW_SIZE

public static final int DEFAULT_WINDOW_SIZE

The default number of words to view before and after each word in focus.

See Also:

Constant Field Values

Constructor Detail

GenericWordSpace

public GenericWordSpace()

Creates a new GenericWordSpace instance using the current System properties for configuration.

GenericWordSpace

public GenericWordSpace(Properties properties)

Creates a new GenericWordSpace instance using the provided properites for configuration.

GenericWordSpace

public GenericWordSpace(int windowSize)

Creates a new GenericWordSpace with the provided window size that ignores word order.

GenericWordSpace

public GenericWordSpace(int windowSize,
                boolean useWordOrder)

Creates a new GenericWordSpace with the provided window size that optionally includes word order.

GenericWordSpace

public GenericWordSpace(int windowSize,
                BasisMapping<Duple<String,Integer>,String> basis)

Creates a new GenericWordSpace with the provided window size that uses the specified basis mapping to map each co-occurrence at a specified position to a dimension.

Parameters:

basis - a basis mapping from a duple that represents a word and its relative position to a dimension.

Method Detail

clearSemantics

public void clearSemantics()

Removes all associations between word and semantics while still retaining the words' basis mapping. This method can be used to re-use the same instance of a GenericWordSpace on multiple corpora while keeping the semantics of the dimensions identical.

getDimensionDescription

public String getDimensionDescription(int dimension)

Returns a description of what features with which the specified dimension corresponds.

Specified by:

getDimensionDescription in interface DimensionallyInterpretableSemanticSpace<String>

Parameters:

dimension - a dimension number

Returns:

a description of the features for the dimension

getVector

public SparseIntegerVector getVector(String word)

Returns the semantic vector for the provided word. Note that because the word space is potentially growing as new documents are processed, the length of the returned vector is equal to the number of dimensions at the time of this call and therefore may be less that the number of dimensions for the same word when obtained at a later time.

Specified by:

getVector in interface SemanticSpace

Parameters:

word - a word that may be in the semantic space

Returns:

The Vector for the provided word or null if the word was not in the space.

getSpaceName

public String getSpaceName()

Returns a unique string describing the name and configuration of this algorithm. Any configurable parameters that would affect the resulting semantic space should be expressed as a part of this name.

Specified by:

getSpaceName in interface SemanticSpace

getVectorLength

public int getVectorLength()

Returns the length of vectors in this semantic space. Implementations are left free to define whether the returned value is valid before processSpace is called.

Specified by:

getVectorLength in interface SemanticSpace

getWords

public Set<String> getWords()

Returns the set of words that are represented in this semantic space.

Specified by:

getWords in interface SemanticSpace

Returns:

the set of words that are represented in this semantic space.

processDocument

public void processDocument(BufferedReader document)
                     throws IOException

Updates the semantic vectors based on the words in the document.

Specified by:

processDocument in interface SemanticSpace

Parameters:

document - a reader that allows access to the text of the document

Throws:

IOException - if any error occurs while reading the document

processSpace

public void processSpace(Properties properties)

Does nothing.

Specified by:

processSpace in interface SemanticSpace

Parameters:

properties - a set of properties and values that may be used to configure any exposed parameters of the algorithm.

setSemanticFilter

public void setSemanticFilter(Set<String> semanticsToRetain)

Specifies the set of words that should have their semantics retained, where all other words do not. Note that all words will still have an index vector assigned to them, which is necessary to properly compute the semantics.

Specified by:

setSemanticFilter in interface Filterable

Parameters:

semanticsToRetain - the set of words for which semantics should be computed.