gov.llnl.ontology.text.tokenize
Class TokenizerAdaptor

java.lang.Object
  gov.llnl.ontology.text.tokenize.TokenizerAdaptor

All Implemented Interfaces:

opennlp.tools.tokenize.Tokenizer

Direct Known Subclasses:

OpenNlpMETokenizer, OpenNlpWhiteSpaceTokenizer, StemmingTokenizer

public class TokenizerAdaptor
extends Object
implements opennlp.tools.tokenize.Tokenizer

This Adaptor class simplifies extending functionality of an already existing Tokenizer. Subclasses can call the underlying Tokenizer and then reprocess the the tokenized results.

Author:

Keith Stevens

Field Summary

protected opennlp.tools.tokenize.Tokenizer

tokenizer The underlying Tokenizer

Constructor Summary

TokenizerAdaptor(opennlp.tools.tokenize.Tokenizer tokenizer)
Creates a new Tokenizer that decorates the functionality of the existing Tokenizer.

Method Summary

String[]	tokenize(String sentence)
opennlp.tools.util.Span[]	tokenizePos(String sentence)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

tokenizer

protected final opennlp.tools.tokenize.Tokenizer tokenizer

The underlying Tokenizer

Constructor Detail

TokenizerAdaptor

public TokenizerAdaptor(opennlp.tools.tokenize.Tokenizer tokenizer)

Creates a new Tokenizer that decorates the functionality of the existing Tokenizer.

Parameters:

tokenizer - The Tokenizer to decorate

Method Detail

tokenize

public String[] tokenize(String sentence)

Specified by:

tokenize in interface opennlp.tools.tokenize.Tokenizer

tokenizePos

public opennlp.tools.util.Span[] tokenizePos(String sentence)

Specified by:

tokenizePos in interface opennlp.tools.tokenize.Tokenizer