edu.ucla.sspace.tools

Class BigramExtractor

java.lang.Object

edu.ucla.sspace.tools.BigramExtractor

public class BigramExtractor
extends Object

A utility class for computing bigram statistics from a corpus.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BigramExtractor.SignificanceTest The significance tests to use in determing how two tokens are statistically related in their occurrences.

Constructor Summary

Constructors

Constructor and Description
BigramExtractor() Creates a new bigram extractor
BigramExtractor(int expectedNumBigrams) Creates a new bigram extractor that expects approximately the specified number of bigrams

Method Summary

Methods

Modifier and Type	Method and Description
static void	main(String[] args)
void	printBigrams(PrintWriter output, BigramExtractor.SignificanceTest test, int minOccurrencePerToken) Prints all of the known bigrams, where each token in the bigram must occur at least the number of specified time.
void	process(BufferedReader text) Processes the tokens in the reader to gather statistics for any bigrams contained therein
void	process(Iterator<String> text) Processes the tokens in the iterator to gather statistics for any bigrams contained therein
void	process(String text) Processes the tokens in the text to gather statistics for any bigrams contained therein

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BigramExtractor

public BigramExtractor()

Creates a new bigram extractor

BigramExtractor

public BigramExtractor(int expectedNumBigrams)

Creates a new bigram extractor that expects approximately the specified number of bigrams

Method Detail

process

public void process(String text)

Processes the tokens in the text to gather statistics for any bigrams contained therein

process

public void process(BufferedReader text)

Processes the tokens in the reader to gather statistics for any bigrams contained therein

process

public void process(Iterator<String> text)

Processes the tokens in the iterator to gather statistics for any bigrams contained therein

printBigrams

public void printBigrams(PrintWriter output,
                BigramExtractor.SignificanceTest test,
                int minOccurrencePerToken)

Prints all of the known bigrams, where each token in the bigram must occur at least the number of specified time.

Parameters:

output - the writer where all the bigrams should be printed

test - the significant test to use in rating the statistical correlation of two tokens

minOccurrencePerToken - the minimum number of times each token in a bigram must occur for the bigram's score to be reported

main

public static void main(String[] args)