org.apache.lucene.analysis.ngram

Class EdgeNGramTokenizer

public class EdgeNGramTokenizer extends Tokenizer

Tokenizes the input from an edge into n-grams of given size(s).

Author: Otis Gospodnetic Adam Hiatt

Nested Class Summary
static classEdgeNGramTokenizer.Side
Specifies which side of the input the n-gram should be generated from
Field Summary
static intDEFAULT_MAX_GRAM_SIZE
static intDEFAULT_MIN_GRAM_SIZE
static EdgeNGramTokenizer.SideDEFAULT_SIDE
Constructor Summary
EdgeNGramTokenizer(Reader input, EdgeNGramTokenizer.Side side, int minGram, int maxGram)
Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range
EdgeNGramTokenizer(Reader input, String sideLabel, int minGram, int maxGram)
Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range
Method Summary
Tokennext()
Returns the next token in the stream, or null at EOS.

Field Detail

DEFAULT_MAX_GRAM_SIZE

public static final int DEFAULT_MAX_GRAM_SIZE

DEFAULT_MIN_GRAM_SIZE

public static final int DEFAULT_MIN_GRAM_SIZE

DEFAULT_SIDE

public static final EdgeNGramTokenizer.Side DEFAULT_SIDE

Constructor Detail

EdgeNGramTokenizer

public EdgeNGramTokenizer(Reader input, EdgeNGramTokenizer.Side side, int minGram, int maxGram)
Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Parameters: input Reader holding the input to be tokenized side the Side from which to chop off an n-gram minGram the smallest n-gram to generate maxGram the largest n-gram to generate

EdgeNGramTokenizer

public EdgeNGramTokenizer(Reader input, String sideLabel, int minGram, int maxGram)
Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Parameters: input Reader holding the input to be tokenized sideLabel the name of the Side from which to chop off an n-gram minGram the smallest n-gram to generate maxGram the largest n-gram to generate

Method Detail

next

public final Token next()
Returns the next token in the stream, or null at EOS.
Copyright © 2000-2007 Apache Software Foundation. All Rights Reserved.