Package org.pipservices3.expressions.tokenizers.generic

Class GenericTokenizer

java.lang.Object

org.pipservices3.expressions.tokenizers.AbstractTokenizer

org.pipservices3.expressions.tokenizers.generic.GenericTokenizer

All Implemented Interfaces:

ITokenizer

public class GenericTokenizer
extends AbstractTokenizer

Implements a default tokenizer class.

Field Summary

Fields inherited from class org.pipservices3.expressions.tokenizers.AbstractTokenizer

_lastTokenType, _nextToken, _scanner

Constructor Summary

Constructors

Constructor	Description
GenericTokenizer()

Method Summary

Methods inherited from class org.pipservices3.expressions.tokenizers.AbstractTokenizer

clearCharacterStates, getCharacterState, getCommentState, getDecodeStrings, getMergeWhitespaces, getNumberState, getQuoteState, getScanner, getSkipComments, getSkipEof, getSkipUnknown, getSkipWhitespaces, getSymbolState, getUnifyNumbers, getWhitespaceState, getWordState, hasNextToken, nextToken, readNextToken, setCharacterState, setCommentState, setDecodeStrings, setMergeWhitespaces, setNumberState, setQuoteState, setScanner, setSkipComments, setSkipEof, setSkipUnknown, setSkipWhitespaces, setSymbolState, setUnifyNumbers, setWhitespaceState, setWordState, tokenizeBuffer, tokenizeBufferToStrings, tokenizeStream, tokenizeStreamToStrings

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

GenericTokenizer

public GenericTokenizer()
                 throws Exception

Throws:

Exception