Package org.pipservices3.expressions.tokenizers.generic
package org.pipservices3.expressions.tokenizers.generic
-
ClassesClassDescriptionThis state will either delegate to a comment-handling state, or return a token with just a slash in it.This state will either delegate to a comment-handling state, or return a token with just a slash in it.A CommentState object returns a comment from a scanner.A NumberState object returns a number from a scanner.A quoteState returns a quoted string token from a scanner.The idea of a symbol is a character that stands on its own, such as an ampersand or a parenthesis.Implements a default tokenizer class.A whitespace state ignores whitespace (such as blanks and tabs), and returns the tokenizer's next token.A wordState returns a word from a scanner.This class is a special case of a
SymbolNode.