org.fife.ui.rsyntaxtextarea
public abstract class Token extends Object
A Token
is a piece of text representing some logical token in
source code for a programming language. For example, the line of C code:
int i = 0;would be broken into 8
Token
s: the first representing
int
, the second whitespace, the third i
, the fourth
whitespace, the fifth =
, etc.Modifier and Type | Field and Description |
---|---|
static int |
COMMENT_DOCUMENTATION |
static int |
COMMENT_EOL |
static int |
COMMENT_MULTILINE |
static int |
DATA_TYPE |
static int |
ERROR_CHAR |
static int |
ERROR_IDENTIFIER |
static int |
ERROR_NUMBER_FORMAT |
static int |
ERROR_STRING_DOUBLE |
static int |
FUNCTION |
static int |
IDENTIFIER |
static int |
LITERAL_BACKQUOTE |
static int |
LITERAL_BOOLEAN |
static int |
LITERAL_CHAR |
static int |
LITERAL_NUMBER_DECIMAL_INT |
static int |
LITERAL_NUMBER_FLOAT |
static int |
LITERAL_NUMBER_HEXADECIMAL |
static int |
LITERAL_STRING_DOUBLE_QUOTE |
static int |
MARKUP_TAG_ATTRIBUTE |
static int |
MARKUP_TAG_DELIMITER |
static int |
MARKUP_TAG_NAME |
static int |
NULL |
static int |
NUM_TOKEN_TYPES |
int |
offset
The offset into the document at which this token resides.
|
static int |
OPERATOR |
static int |
PREPROCESSOR |
static int |
RESERVED_WORD |
static int |
SEPARATOR |
char[] |
text
The text this token represents.
|
int |
textCount |
int |
textOffset |
int |
type
The type of token this is; for example,
FUNCTION . |
static int |
VARIABLE |
static int |
WHITESPACE |
Constructor and Description |
---|
Token()
Creates a "null token."
|
Token(char[] line,
int beg,
int end,
int startOffset,
int type)
Constructor.
|
Token(Token t2)
Creates this token as a deep copy of the passed-in token.
|
Modifier and Type | Method and Description |
---|---|
StringBuffer |
appendHTMLRepresentation(StringBuffer sb,
RSyntaxTextArea textArea,
boolean fontFamily)
Appends HTML code for painting this token, using the given text area's
color scheme.
|
boolean |
containsPosition(int pos)
Returns whether the token straddles the specified position in the
document.
|
void |
copyFrom(Token t2)
Makes one token point to the same text segment, and have the same value
as another token.
|
int |
documentToToken(int pos)
Returns the position in the token's internal char array corresponding
to the specified document position.
|
String |
getHTMLRepresentation(RSyntaxTextArea textArea)
Returns a
String containing HTML code for painting this
token, using the given text area's color scheme. |
Token |
getLastNonCommentNonWhitespaceToken()
Returns the last token in this list that is not whitespace or a
comment.
|
Token |
getLastPaintableToken()
Returns the last paintable token in this token list, or
null
if there is no paintable token. |
String |
getLexeme()
Returns the text of this token, as a string.
|
abstract int |
getListOffset(RSyntaxTextArea textArea,
TabExpander e,
float x0,
float x)
Determines the offset into this token list (i.e., into the
document) that covers pixel location
x if the token list
starts at pixel location x0 . |
Token |
getNextToken()
Returns the token after this one in the linked list.
|
int |
getOffsetBeforeX(RSyntaxTextArea textArea,
TabExpander e,
float startX,
float endBeforeX)
Returns the position in the document that represents the last character
in the token that will fit into
endBeforeX-startX pixels. |
float |
getWidth(RSyntaxTextArea textArea,
TabExpander e,
float x0)
Returns the width of this token given the specified parameters.
|
abstract float |
getWidthUpTo(int numChars,
RSyntaxTextArea textArea,
TabExpander e,
float x0)
Returns the width of a specified number of characters in this token.
|
boolean |
is(int type,
String lexeme)
Returns whether this token is of the specified type, with the specified
lexeme.
|
boolean |
isComment()
Returns whether this token is a comment.
|
boolean |
isHyperlink()
Returns whether this token is a hyperlink.
|
boolean |
isLeftCurly()
Returns whether this token is a
SEPARATOR representing a single
left curly brace. |
boolean |
isPaintable()
Returns whether or not this token is "paintable;" i.e., whether or not
the type of this token is one such that it has an associated syntax
style.
|
boolean |
isRightCurly()
Returns whether this token is a
SEPARATOR representing a single
right curly brace. |
boolean |
isSingleChar(char ch)
Returns whether this token is the specified single character.
|
boolean |
isWhitespace()
Returns whether or not this token is whitespace.
|
abstract Rectangle |
listOffsetToView(RSyntaxTextArea textArea,
TabExpander e,
int pos,
int x0,
Rectangle rect)
Returns the bounding box for the specified document location.
|
void |
makeStartAt(int pos)
Makes this token start at the specified offset into the document.
|
void |
moveOffset(int amt)
Moves the starting offset of this token.
|
float |
paint(Graphics2D g,
float x,
float y,
RSyntaxTextArea host,
TabExpander e)
Paints this token.
|
abstract float |
paint(Graphics2D g,
float x,
float y,
RSyntaxTextArea host,
TabExpander e,
float clipStart)
Paints this token.
|
protected void |
paintBackground(float x,
float y,
float width,
float height,
Graphics2D g,
int fontAscent,
RSyntaxTextArea host,
Color color)
Paints the background of a token.
|
void |
set(char[] line,
int beg,
int end,
int offset,
int type)
Sets the value of this token to a particular segment of a document.
|
void |
setHyperlink(boolean hyperlink)
Sets whether this token is a hyperlink.
|
void |
setNextToken(Token nextToken)
Sets the "next token" pointer of this token to point to the specified
token.
|
int |
tokenToDocument(int pos)
Returns the position in the document corresponding to the specified
position in this token's internal char array (
textOffset -
textOffset+textCount-1 ). |
String |
toString()
Returns this token as a
String , which is useful for
debugging. |
public char[] text
public int textOffset
public int textCount
public int offset
public int type
FUNCTION
.public static final int NULL
public static final int COMMENT_EOL
public static final int COMMENT_MULTILINE
public static final int COMMENT_DOCUMENTATION
public static final int RESERVED_WORD
public static final int FUNCTION
public static final int LITERAL_BOOLEAN
public static final int LITERAL_NUMBER_DECIMAL_INT
public static final int LITERAL_NUMBER_FLOAT
public static final int LITERAL_NUMBER_HEXADECIMAL
public static final int LITERAL_STRING_DOUBLE_QUOTE
public static final int LITERAL_CHAR
public static final int LITERAL_BACKQUOTE
public static final int DATA_TYPE
public static final int VARIABLE
public static final int IDENTIFIER
public static final int WHITESPACE
public static final int SEPARATOR
public static final int OPERATOR
public static final int PREPROCESSOR
public static final int MARKUP_TAG_DELIMITER
public static final int MARKUP_TAG_NAME
public static final int MARKUP_TAG_ATTRIBUTE
public static final int ERROR_IDENTIFIER
public static final int ERROR_NUMBER_FORMAT
public static final int ERROR_STRING_DOUBLE
public static final int ERROR_CHAR
public static final int NUM_TOKEN_TYPES
public Token()
public Token(char[] line, int beg, int end, int startOffset, int type)
line
- The segment from which to get the token.beg
- The first character's position in line
.end
- The last character's position in line
.startOffset
- The offset into the document at which this
token begins.type
- A token type listed as "generic" above.public Token(Token t2)
t2
- The token from which to make a copy.public StringBuffer appendHTMLRepresentation(StringBuffer sb, RSyntaxTextArea textArea, boolean fontFamily)
sb
- The buffer to append to.textArea
- The text area whose color scheme to use.fontFamily
- Whether to include the font family in the HTML for
this token. You can pass false
for this parameter
if, for example, you are making all your HTML be monospaced,
and don't want any crazy fonts being used in the editor to be
reflected in your HTML.getHTMLRepresentation(RSyntaxTextArea)
public boolean containsPosition(int pos)
pos
- The position in the document to check.public void copyFrom(Token t2)
t2
- The token from which to copy.public int documentToToken(int pos)
Note that this method does NOT do any bounds checking; you can pass in a document position that does not correspond to a position in this token, and you will not receive an Exception or any other notification; it is up to the caller to ensure valid input.
pos
- A position in the document that is represented by this token.textOffset
and
< textOffset+textCount
.tokenToDocument(int)
public String getHTMLRepresentation(RSyntaxTextArea textArea)
String
containing HTML code for painting this
token, using the given text area's color scheme.textArea
- The text area whose color scheme to use.appendHTMLRepresentation(StringBuffer, RSyntaxTextArea, boolean)
public Token getLastNonCommentNonWhitespaceToken()
null
if there isn't one.public Token getLastPaintableToken()
null
if there is no paintable token.public String getLexeme()
Note that this method isn't used much by the
rsyntaxtextarea
package internally, as it tries to limit
memory allocation.
public abstract int getListOffset(RSyntaxTextArea textArea, TabExpander e, float x0, float x)
x
if the token list
starts at pixel location x0
.
This method will return the document position "closest" to the
x-coordinate (i.e., if they click on the "right-half" of the
w
in awe
, the caret will be placed in
between the w
and e
; similarly, clicking on
the left-half places the caret between the a
and
w
). This makes it useful for methods such as
viewToModel
found in javax.swing.text.View
subclasses.
This method is abstract so subclasses who paint themselves differently
(i.e., VisibleWhitespaceToken
is painted a tad differently than
DefaultToken
when rendering hints are enabled) can still return
accurate results.
textArea
- The text area from which the token list was derived.e
- How to expand tabs.x0
- The pixel x-location that is the beginning of
tokenList
.x
- The pixel-position for which you want to get the corresponding
offset.tokenList
is
null
or has type Token.NULL
, then
-1
public Token getNextToken()
setNextToken(org.fife.ui.rsyntaxtextarea.Token)
public int getOffsetBeforeX(RSyntaxTextArea textArea, TabExpander e, float startX, float endBeforeX)
endBeforeX-startX
pixels.
For example, if you're using a monospaced 8-pixel-per-character font,
have the token "while" and startX
is 0
and
endBeforeX
is 30
, this method will return the
document position of the "i" in "while", because the "i" ends at pixel
24
, while the "l" ends at 32
. If not even the
first character fits in endBeforeX-startX
, the first
character's position is still returned so calling methods don't go into
infinite loops.textArea
- The text area in which this token is being painted.e
- How to expand tabs.startX
- The x-coordinate at which the token will be painted. This
is needed because of tabs.endBeforeX
- The x-coordinate for which you want to find the last
character of t
which comes before it.public float getWidth(RSyntaxTextArea textArea, TabExpander e, float x0)
textArea
- The text area in which the token is being painted.e
- Describes how to expand tabs. This parameter cannot be
null
.x0
- The pixel-location at which the token begins. This is needed
because of tabs.getWidthUpTo(int, org.fife.ui.rsyntaxtextarea.RSyntaxTextArea, javax.swing.text.TabExpander, float)
public abstract float getWidthUpTo(int numChars, RSyntaxTextArea textArea, TabExpander e, float x0)
3
here returns the width of the "whi" portion of the token.
This method is abstract so subclasses who paint themselves differently
(i.e., VisibleWhitespaceToken
is painted a tad differently than
DefaultToken
when rendering hints are enabled) can still return
accurate results.
numChars
- The number of characters for which to get the width.textArea
- The text area in which the token is being painted.e
- How to expand tabs. This value cannot be null
.x0
- The pixel-location at which this token begins. This is needed
because of tabs.getWidth(org.fife.ui.rsyntaxtextarea.RSyntaxTextArea, javax.swing.text.TabExpander, float)
public boolean is(int type, String lexeme)
type
- The type to check for.lexeme
- The lexeme to check for.public boolean isComment()
isWhitespace()
public boolean isHyperlink()
setHyperlink(boolean)
public boolean isLeftCurly()
SEPARATOR
representing a single
left curly brace.isRightCurly()
public boolean isRightCurly()
SEPARATOR
representing a single
right curly brace.isLeftCurly()
public boolean isPaintable()
Token.NULL
.public boolean isSingleChar(char ch)
ch
- The character to check for.public boolean isWhitespace()
true
iff this token is whitespace.isComment()
public abstract Rectangle listOffsetToView(RSyntaxTextArea textArea, TabExpander e, int pos, int x0, Rectangle rect)
null
is returned.textArea
- The text area from which the token list was derived.e
- How to expand tabs.pos
- The position in the document for which to get the bounding
box in the view.x0
- The pixel x-location that is the beginning of
tokenList
.rect
- The rectangle in which we'll be returning the results. This
object is reused to keep from frequent memory allocations.public void makeStartAt(int pos)
pos
- The offset into the document this token should start at.
Note that this token must already contain this position; if
it doesn't, an exception is thrown.IllegalArgumentException
- If pos is not already contained by
this token.moveOffset(int)
public void moveOffset(int amt)
amt
- The amount to move the starting offset. This should be
between 0
and textCount
, inclusive.IllegalArgumentException
- If amt
is an invalid value.makeStartAt(int)
public final float paint(Graphics2D g, float x, float y, RSyntaxTextArea host, TabExpander e)
g
- The graphics context in which to paint.x
- The x-coordinate at which to paint.y
- The y-coordinate at which to paint.host
- The text area this token is in.e
- How to expand tabs.public abstract float paint(Graphics2D g, float x, float y, RSyntaxTextArea host, TabExpander e, float clipStart)
g
- The graphics context in which to paint.x
- The x-coordinate at which to paint.y
- The y-coordinate at which to paint.host
- The text area this token is in.e
- How to expand tabs.clipStart
- The left boundary of the clip rectangle in which we're
painting. This optimizes painting by allowing us to not paint
paint when this token is "to the left" of the clip rectangle.protected void paintBackground(float x, float y, float width, float height, Graphics2D g, int fontAscent, RSyntaxTextArea host, Color color)
x
- The x-coordinate of the token.y
- The y-coordinate of the token.width
- The width of the token (actually, the width of the part of
the token to paint).height
- The height of the token.g
- The graphics context with which to paint.fontAscent
- The ascent of the token's font.host
- The text area.color
- The color with which to paint.public void set(char[] line, int beg, int end, int offset, int type)
line
- The segment from which to get the token.beg
- The first character's position in line
.end
- The last character's position in line
.offset
- The offset into the document at which this token begins.type
- A token type listed as "generic" above.public void setHyperlink(boolean hyperlink)
hyperlink
- Whether this token is a hyperlink.isHyperlink()
public void setNextToken(Token nextToken)
nextToken
- The new next token.getNextToken()
public int tokenToDocument(int pos)
textOffset
-
textOffset+textCount-1
).Note that this method does NOT do any bounds checking; you can pass in an invalid token position, and you will not receive an Exception or any other indication that the returned document position is invalid. It is up to the user to ensure valid input.
pos
- A position in the token's internal char array
(textOffset
- textOffset+textCount
).documentToToken(int)