edu.ucla.sspace.text

Class DirectoryCorpusReader<D extends Document>

java.lang.Object

edu.ucla.sspace.text.DirectoryCorpusReader<D>

All Implemented Interfaces:

CorpusReader<D>

Direct Known Subclasses:

BloglinesCorpusReader, BloglinesCorpusReader, ChildesCorpusReader, ChildesCorpusReader, SenseEvalDependencyCorpusReader, TemporalBloglinesCorpusReader, TemporalUsenetCorpusReader, UsenetCorpusReader, UsenetCorpusReader

public abstract class DirectoryCorpusReader<D extends Document>
extends Object
implements CorpusReader<D>

An abstract base class for corpus reading iterators that need to traverse through a large nested directory structure to find files containing text. Sub classes will need to implement a method that will traverse a single file at a time and return strings holding the contents of a single document..

Note that initialize(String) uses the argument as a directory name, and not the name of a text file to be processed. This CorpusReader instead uses that directory name as the root directory of the nested directory structure. This class also does not implement initialize(Reader).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	DirectoryCorpusReader.BaseFileIterator

Constructor Summary

Constructors

Constructor and Description
DirectoryCorpusReader() Constructs a new DirectoryCoprusReader that uses no DocumentPreprocessor.
DirectoryCorpusReader(DocumentPreprocessor processor) Constructs a new DirectoryCoprusReader that uses processor to pre-process any raw text extracted from a corpus file.

Method Summary

Methods

Modifier and Type	Method and Description
protected abstract Iterator<D>	corpusIterator(Iterator<File> fileIter) Returns an Iterator over documents contained in the Files traversed by fileIter.
void	initialize(Reader baseReader) Unsupported.
Iterator<D>	read(File dir) Initializes the DirectoryCorpusReader to start processing all files accessbile under the directory specified by dirName.
Iterator<D>	read(Reader reader) Unsupported.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DirectoryCorpusReader

public DirectoryCorpusReader()

Constructs a new DirectoryCoprusReader that uses no DocumentPreprocessor.

DirectoryCorpusReader

public DirectoryCorpusReader(DocumentPreprocessor processor)

Constructs a new DirectoryCoprusReader that uses processor to pre-process any raw text extracted from a corpus file.

Method Detail

read

public Iterator<D> read(File dir)

Initializes the DirectoryCorpusReader to start processing all files accessbile under the directory specified by dirName.

Specified by:

read in interface CorpusReader<D extends Document>

Parameters:

dirName - A directory path containing a large directory structure that contains numerous text files that can be processed by a subclass of DirectoryCoprusReader.

read

public Iterator<D> read(Reader reader)

Unsupported.

Specified by:

read in interface CorpusReader<D extends Document>

Parameters:

reader - A Reader that will extract text from a data source, such as a URL, a File, a data stream, or any other source accesible via the Reader interface. Each CorpusReader should specify the expected text format, be it an XML schema or some other unique format.

corpusIterator

protected abstract Iterator<D> corpusIterator(Iterator<File> fileIter)

Returns an Iterator over documents contained in the Files traversed by fileIter. Sub-classes are encouraged to sub-class BaseFileIterator for the return value of this method.

initialize

public void initialize(Reader baseReader)

Unsupported.

Throws:

UnsupportedOperationException - when called