edu.ucla.sspace.hal

Class HyperspaceAnalogueToLanguage

java.lang.Object

edu.ucla.sspace.hal.HyperspaceAnalogueToLanguage

All Implemented Interfaces:

SemanticSpace

public class HyperspaceAnalogueToLanguage
extends Object
implements SemanticSpace

A SemanticSpace implementation of the Hyperspace Analogue to Language (HAL) algorithm described by Lund and Burgess. This implementation is based on the following paper:

• K. Lund and C. Burgess Producing high-dimensional semantic spaces from lexical Co-occurrence Behavior Research Methods, Instrumentation, and Computers, 28, pages 203-208, 1996. Available here

See here for additional papers that use HAL.

HAL is based on recording the co-occurrence of words in a sparse matrix. HAL also incorporates word order information by treating the co-occurrences of two words x y as being different than y x. Each word is assigned a unique index in the co-occurrence matrix. For some word x, when another word x co-occurs before, matrix entry x,y is update. Similarly, when y co-occurs after, the matrix entry y,x is updated. Therefore the full semantic vector for any words is its row vector concatenated with its column vector.

Typically, the full vectors are used (for an N x N matrix, these are 2*N in length). However, HAL also offers two posibilities for dimensionality reduction. Not all columns provide equal amount of information that can be used to distinguish the meanings of the words. Specifically, the information theoretic entropy of each column can be calculated as a way of ordering the columns by their importance. Using this ranking, either a fixed number of columns may be retained, or a threshold may be set to filter out low-entropy columns.

This class provides four parameters that may be set:

Property: "edu.ucla.sspace.hal.HyperspaceAnalogueToLanguage.windowSize"
Default: 5

This variable sets the number of words before and after that are counted as co-occurring. With the default value, 5 words are counted before and 5 words are counter after. This class always uses a symmetric window.

Property: "edu.ucla.sspace.hal.weighting"
Default: edu.ucla.sspace.hal.LinearWeighting

This property sets the fully-qualified class name of the WeightingFunction class that will be used to determine how to weigh co-occurrences. HAL traditionally uses a ramped, linear weighting where those words occurring closets receive more weight, with a linear decrease based on distance.

Property: "edu.ucla.sspace.hal.retainColumns"
Default: unset

This optional property enables dimensionality reduction by retaining only a fixed number of columns. The columns with the high entropy are retrained. The value should be an integer. This property may not be set concurrently with "edu.ucla.sspace.hal.HyperspaceAnalogueToLanguage.threshold", and will throw an exception if done so.

Property: "edu.ucla.sspace.hal.HyperspaceAnalogueToLanguage.threshold"
Default: unset

This optional property enables dimensionality reduction by retaining only those columns whose entropy is above the specified threshold. The value should be a double. This property may not be set concurrently with "edu.ucla.sspace.hal.retainColumns", and will throw an exception if done so.

Note that the weight function can also be used to create special cases of the HAL model, For example, an asymmetric window could be created by assigning a weight of 0 to all those co-occurrence on one side.

Author:

Alex Nau, David Jurgens

See Also:

SemanticSpace, WeightingFunction

Field Summary

Fields

Modifier and Type	Field and Description
static WeightingFunction	DEFAULT_WEIGHTING The default WeightingFunction to use.
static int	DEFAULT_WINDOW_SIZE The default number of words before and after the focus word to include
static String	ENTROPY_THRESHOLD_PROPERTY The property to specify the minimum entropy theshold a word should have to be included in the vector space after processing.
static String	RETAIN_PROPERTY The property to specify the number of words to view before and after each word in focus.
static String	WEIGHTING_FUNCTION_PROPERTY The property to set the WeightingFunction to be used with weighting the co-occurrence of neighboring words based on their distance.
static String	WINDOW_SIZE_PROPERTY The property to specify the number of words to view before and after each word in focus.

Constructor Summary

Constructors

Constructor and Description
HyperspaceAnalogueToLanguage() Constructs a new instance using the system properties for configuration.
HyperspaceAnalogueToLanguage(Properties properties) Constructs a new instance using the provided properties for configuration.

Method Summary

Methods

Modifier and Type	Method and Description
String	getSpaceName() Returns a unique string describing the name and configuration of this algorithm.
Vector	getVector(String word) Returns the semantic vector for the provided word.
int	getVectorLength() Returns the length of vectors in this semantic space.
Set<String>	getWords() Returns the set of words that are represented in this semantic space.
void	processDocument(BufferedReader document) Processes the contents of the provided file as a document.
void	processSpace(Properties properties) Once all the documents have been processed, performs any post-processing steps on the data.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ENTROPY_THRESHOLD_PROPERTY

public static final String ENTROPY_THRESHOLD_PROPERTY

The property to specify the minimum entropy theshold a word should have to be included in the vector space after processing. The specified value of this property should be a double

See Also:

Constant Field Values

WINDOW_SIZE_PROPERTY

public static final String WINDOW_SIZE_PROPERTY

The property to specify the number of words to view before and after each word in focus.

See Also:

Constant Field Values

RETAIN_PROPERTY

public static final String RETAIN_PROPERTY

The property to specify the number of words to view before and after each word in focus.

See Also:

Constant Field Values

WEIGHTING_FUNCTION_PROPERTY

public static final String WEIGHTING_FUNCTION_PROPERTY

The property to set the WeightingFunction to be used with weighting the co-occurrence of neighboring words based on their distance.

See Also:

Constant Field Values

DEFAULT_WINDOW_SIZE

public static final int DEFAULT_WINDOW_SIZE

The default number of words before and after the focus word to include

See Also:

Constant Field Values

DEFAULT_WEIGHTING

public static final WeightingFunction DEFAULT_WEIGHTING

The default WeightingFunction to use.

Constructor Detail

HyperspaceAnalogueToLanguage

public HyperspaceAnalogueToLanguage()

Constructs a new instance using the system properties for configuration.

HyperspaceAnalogueToLanguage

public HyperspaceAnalogueToLanguage(Properties properties)

Constructs a new instance using the provided properties for configuration.

Method Detail

processDocument

public void processDocument(BufferedReader document)
                     throws IOException

Processes the contents of the provided file as a document.

Specified by:

processDocument in interface SemanticSpace

Parameters:

document - a reader that allows access to the text of the document

Throws:

IOException - if any error occurs while reading the document

getWords

public Set<String> getWords()

Returns the set of words that are represented in this semantic space.

Specified by:

getWords in interface SemanticSpace

Returns:

the set of words that are represented in this semantic space.

getVector

public Vector getVector(String word)

Returns the semantic vector for the provided word.

Specified by:

getVector in interface SemanticSpace

Parameters:

word - a word that may be in the semantic space

Returns:

The Vector for the provided word or null if the word was not in the space.

getVectorLength

public int getVectorLength()

Returns the length of vectors in this semantic space. Implementations are left free to define whether the returned value is valid before processSpace is called.

Specified by:

getVectorLength in interface SemanticSpace

processSpace

public void processSpace(Properties properties)

Once all the documents have been processed, performs any post-processing steps on the data. An algorithm should treat this as a no-op if no post-processing is required. Callers may specify the values for any exposed parameters using the properties argument.

By general contract, once this method has been called, processDocument will not be called again.

Specified by:

processSpace in interface SemanticSpace

Parameters:

properties - a set of properties and values that may be used to configure any exposed parameters of the algorithm.

getSpaceName

public String getSpaceName()

Returns a unique string describing the name and configuration of this algorithm. Any configurable parameters that would affect the resulting semantic space should be expressed as a part of this name.

Specified by:

getSpaceName in interface SemanticSpace