A grammar-based tokenizer constructed with JavaCC.
See: Description
Interface Summary | |
---|---|
CharStream | This interface describes a character stream that maintains line and column number positions of the characters. |
StandardTokenizerConstants |
Class Summary | |
---|---|
FastCharStream | An efficient implementation of JavaCC's CharStream interface. |
ParseException | This exception is thrown when parse errors are encountered. |
StandardAnalyzer | Filters StandardTokenizer with StandardFilter, LowerCaseFilter and StopFilter, using a list of English stop words. |
StandardFilter | Normalizes tokens extracted with StandardTokenizer. |
StandardTokenizer | A grammar-based tokenizer constructed with JavaCC. |
StandardTokenizerTokenManager | |
Token | Describes the input token stream. |
TokenMgrError |
Note that JavaCC defines lots of public classes, methods and fields that do not need to be public. These clutter the documentation. Sorry.
Note that because JavaCC defines a class named Token, org.apache.lucene.analysis.Token must always be fully qualified in source code in this package.