TokenKind in rustc_ast::token - Rust
Description: API documentation for the Rust `TokenKind` enum in crate `rustc_ast`.

Skip to main content

## TokenKind

# Enum TokenKind

Source

    pub enum TokenKind {
    Show 62 variants    Eq,
        Lt,
        Le,
        EqEq,
        Ne,
        Ge,
        Gt,
        AndAnd,
        OrOr,
        Bang,
        Tilde,
        Plus,
        Minus,
        Star,
        Slash,
        Percent,
        Caret,
        And,
        Or,
        Shl,
        Shr,
        PlusEq,
        MinusEq,
        StarEq,
        SlashEq,
        PercentEq,
        CaretEq,
        AndEq,
        OrEq,
        ShlEq,
        ShrEq,
        At,
        Dot,
        DotDot,
        DotDotDot,
        DotDotEq,
        Comma,
        Semi,
        Colon,
        PathSep,
        RArrow,
        LArrow,
        FatArrow,
        Pound,
        Dollar,
        Question,
        SingleQuote,
        OpenParen,
        CloseParen,
        OpenBrace,
        CloseBrace,
        OpenBracket,
        CloseBracket,
        OpenInvisible(InvisibleOrigin),
        CloseInvisible(InvisibleOrigin),
        Literal(Lit),
        Ident(Symbol, IdentIsRaw),
        NtIdent(Ident, IdentIsRaw),
        Lifetime(Symbol, IdentIsRaw),
        NtLifetime(Ident, IdentIsRaw),
        DocComment(CommentKind, AttrStyle, Symbol),
        Eof,
    }

## Variants§

§

### Eq

`=`

§

### Lt

`<`

§

### Le

`<=`

§

### EqEq

`==`

§

### Ne

`!=`

§

### Ge

`>=`

§

### Gt

`>`

§

### AndAnd

`&&`

§

### OrOr

`||`

§

### Bang

`!`

§

### Tilde

`~`

§

### Plus

§

### Minus

§

### Star

§

### Slash

§

### Percent

§

### Caret

§

### And

§

### Or

§

### Shl

§

### Shr

§

### PlusEq

§

### MinusEq

§

### StarEq

§

### SlashEq

§

### PercentEq

§

### CaretEq

§

### AndEq

§

### OrEq

§

### ShlEq

§

### ShrEq

§

### At

`@`

§

### Dot

`.`

§

### DotDot

`..`

§

### DotDotDot

`...`

§

### DotDotEq

`..=`

§

### Comma

`,`

§

### Semi

`;`

§

### Colon

`:`

§

### PathSep

`::`

§

### RArrow

`->`

§

### LArrow

`<-`

§

### FatArrow

`=>`

§

### Pound

`#`

§

### Dollar

`$`

§

### Question

`?`

§

### SingleQuote

Used by proc macros for representing lifetimes, not generated by lexer right now.

§

### OpenParen

`(`

§

### CloseParen

`)`

§

### OpenBrace

`{`

§

### CloseBrace

`}`

§

### OpenBracket

`[`

§

### CloseBracket

`]`

§

### OpenInvisible(InvisibleOrigin)

Invisible opening delimiter, produced by a macro.

§

### CloseInvisible(InvisibleOrigin)

Invisible closing delimiter, produced by a macro.

§

### Literal(Lit)

§

### Ident(Symbol, IdentIsRaw)

Identifier token. Do not forget about `NtIdent` when you want to match on identifiers. It’s recommended to use `Token::{ident,uninterpolate}` and `Parser::token_uninterpolated_span` to treat regular and interpolated identifiers in the same way.

§

### NtIdent(Ident, IdentIsRaw)

This identifier (and its span) is the identifier passed to the declarative macro. The span in the surrounding `Token` is the span of the `ident` metavariable in the macro’s RHS.

§

### Lifetime(Symbol, IdentIsRaw)

Lifetime identifier token. Do not forget about `NtLifetime` when you want to match on lifetime identifiers. It’s recommended to use `Token::{ident,uninterpolate}` and `Parser::token_uninterpolated_span` to treat regular and interpolated identifiers in the same way.

§

### NtLifetime(Ident, IdentIsRaw)

This identifier (and its span) is the lifetime passed to the declarative macro. The span in the surrounding `Token` is the span of the `lifetime` metavariable in the macro’s RHS.

§

### DocComment(CommentKind, AttrStyle, Symbol)

A doc comment token. `Symbol` is the doc comment’s data excluding its “quotes” (`///`, `/**`, etc) similarly to symbols in string literal tokens.

§

### Eof

End Of File

## Implementations§

Source§

### impl TokenKind

Source

#### pub fn lit(kind: LitKind, symbol: Symbol, suffix: Option<Symbol>) -> TokenKind

Source

#### pub fn break_two_token_op(&self, n: u32) -> Option<(TokenKind, TokenKind)>

An approximation to proc-macro-style single-character operators used by rustc parser. If the operator token can be broken into two tokens, the first of which has `n` (1 or 2) chars, then this function performs that operation, otherwise it returns `None`.

Source

#### pub fn similar_tokens(&self) -> &[TokenKind]

Returns tokens that are likely to be typed accidentally instead of the current token. Enables better error recovery when the wrong token is found.

Source

#### pub fn should_end_const_arg(&self) -> bool

Source

#### pub fn is_delim(&self) -> bool

Source

#### pub fn open_delim(&self) -> Option<Delimiter>

Source

#### pub fn close_delim(&self) -> Option<Delimiter>

Source

#### pub fn is_close_delim_or_eof(&self) -> bool

## Trait Implementations§

Source§

### impl Clone for TokenKind

Source§

#### fn clone(&self) -> TokenKind

Returns a duplicate of the value. Read more
1.0.0 · Source§

#### fn clone_from(&mut self, source: &Self)

Performs copy-assignment from `source`. Read more
Source§

### impl Debug for TokenKind

Source§

#### fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

### impl<__D: SpanDecoder> Decodable<__D> for TokenKind

Source§

#### fn decode(__decoder: &mut __D) -> Self

Source§

### impl<__E: SpanEncoder> Encodable<__E> for TokenKind

Source§

#### fn encode(&self, __encoder: &mut __E)

Source§

### impl Hash for TokenKind

Source§

#### fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given `Hasher`. Read more
1.3.0 · Source§

#### fn hash_slice<H>(data: &[Self], state: &mut H)
where H: Hasher, Self: Sized,

Feeds a slice of this type into the given `Hasher`. Read more
Source§

### impl<__CTX> HashStable<__CTX> for TokenKind
where __CTX: HashStableContext,

Source§

#### fn hash_stable(&self, __hcx: &mut __CTX, __hasher: &mut StableHasher)

Source§

### impl PartialEq<TokenKind> for Token

Source§

#### fn eq(&self, rhs: &TokenKind) -> bool

Tests for `self` and `other` values to be equal, and is used by `==`.
1.0.0 · Source§

#### fn ne(&self, other: &Rhs) -> bool

Tests for `!=`. The default implementation is almost always sufficient, and should not be overridden without very good reason.
Source§

### impl PartialEq for TokenKind

Source§

#### fn eq(&self, other: &TokenKind) -> bool

Tests for `self` and `other` values to be equal, and is used by `==`.
1.0.0 · Source§

#### fn ne(&self, other: &Rhs) -> bool

Tests for `!=`. The default implementation is almost always sufficient, and should not be overridden without very good reason.
Source§

### impl Copy for TokenKind

Source§

### impl Eq for TokenKind

Source§

### impl StructuralPartialEq for TokenKind

## Auto Trait Implementations§

§

### impl DynSend for TokenKind

§

### impl DynSync for TokenKind

§

### impl Freeze for TokenKind

§

### impl RefUnwindSafe for TokenKind

§

### impl Send for TokenKind

§

### impl Sync for TokenKind

§

### impl Unpin for TokenKind

§

### impl UnsafeUnpin for TokenKind

§

### impl UnwindSafe for TokenKind

## Blanket Implementations§

Source§

### impl<T> Aligned for T

Source§

#### const ALIGN: Alignment

Alignment of `Self`.
Source§

### impl<T> Any for T
where T: 'static + ?Sized,

Source§

#### fn type_id(&self) -> TypeId

Gets the `TypeId` of `self`. Read more
Source§

### impl<T> Borrow<T> for T
where T: ?Sized,

Source§

#### fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

### impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

#### fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

### impl<T> CloneToUninit for T
where T: Clone,

Source§

#### unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (`clone_to_uninit`)
Performs copy-assignment from `self` to `dest`. Read more
Source§

### impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

Source§

#### fn equivalent(&self, key: &K) -> bool

Checks if this value is equivalent to the given key. Read more
Source§

### impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

Source§

#### fn equivalent(&self, key: &K) -> bool

Checks if this value is equivalent to the given key. Read more
Source§

### impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

Source§

#### fn equivalent(&self, key: &K) -> bool

Compare self to `key` and return `true` if they are equal.
Source§

### impl<T> From<T> for T

Source§

#### fn from(t: T) -> T

Returns the argument unchanged.

Source§

### impl<T> Instrument for T

Source§

#### fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided `Span`, returning an `Instrumented` wrapper. Read more
Source§

#### fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current `Span`, returning an `Instrumented` wrapper. Read more
Source§

### impl<T, U> Into<U> for T
where U: From<T>,

Source§

#### fn into(self) -> U

Calls `U::from(self)`.

That is, this conversion is whatever the implementation of `From<T> for U` chooses to do.

Source§

### impl<T> IntoEither for T

Source§

#### fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts `self` into a `Left` variant of `Either<Self, Self>` if `into_left` is `true`. Converts `self` into a `Right` variant of `Either<Self, Self>` otherwise. Read more
Source§

#### fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts `self` into a `Left` variant of `Either<Self, Self>` if `into_left(&self)` returns `true`. Converts `self` into a `Right` variant of `Either<Self, Self>` otherwise. Read more
Source§

### impl<T> MutVisitorResult for T

Source§

#### type Result = ()

Source§

### impl<T> Pointable for T

Source§

#### const ALIGN: usize

The alignment of pointer.
Source§

#### type Init = T

The type for initializers.
Source§

#### unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

#### unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

#### unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

#### unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

### impl<T> Same for T

Source§

#### type Output = T

Should always be `Self`
Source§

### impl<T> ToOwned for T
where T: Clone,

Source§

#### type Owned = T

The resulting type after obtaining ownership.
Source§

#### fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

#### fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

### impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

#### type Error = Infallible

The type returned in the event of a conversion error.
Source§

#### fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

### impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

#### type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

#### fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

### impl<T> WithSubscriber for T

Source§

#### fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided `Subscriber` to this type, returning a `WithDispatch` wrapper. Read more
Source§

#### fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default `Subscriber` to this type, returning a `WithDispatch` wrapper. Read more

## Layout§

Note: Most layout information is completely unstable and may even differ between compilations. The only exception is types with certain `repr(...)` attributes. Please see the Rust Reference's “Type Layout” chapter for details on type layout guarantees.

Size: 16 bytes

Size for each variant:

- `Eq`: 0 bytes
- `Lt`: 0 bytes
- `Le`: 0 bytes
- `EqEq`: 0 bytes
- `Ne`: 0 bytes
- `Ge`: 0 bytes
- `Gt`: 0 bytes
- `AndAnd`: 0 bytes
- `OrOr`: 0 bytes
- `Bang`: 0 bytes
- `Tilde`: 0 bytes
- `Plus`: 0 bytes
- `Minus`: 0 bytes
- `Star`: 0 bytes
- `Slash`: 0 bytes
- `Percent`: 0 bytes
- `Caret`: 0 bytes
- `And`: 0 bytes
- `Or`: 0 bytes
- `Shl`: 0 bytes
- `Shr`: 0 bytes
- `PlusEq`: 0 bytes
- `MinusEq`: 0 bytes
- `StarEq`: 0 bytes
- `SlashEq`: 0 bytes
- `PercentEq`: 0 bytes
- `CaretEq`: 0 bytes
- `AndEq`: 0 bytes
- `OrEq`: 0 bytes
- `ShlEq`: 0 bytes
- `ShrEq`: 0 bytes
- `At`: 0 bytes
- `Dot`: 0 bytes
- `DotDot`: 0 bytes
- `DotDotDot`: 0 bytes
- `DotDotEq`: 0 bytes
- `Comma`: 0 bytes
- `Semi`: 0 bytes
- `Colon`: 0 bytes
- `PathSep`: 0 bytes
- `RArrow`: 0 bytes
- `LArrow`: 0 bytes
- `FatArrow`: 0 bytes
- `Pound`: 0 bytes
- `Dollar`: 0 bytes
- `Question`: 0 bytes
- `SingleQuote`: 0 bytes
- `OpenParen`: 0 bytes
- `CloseParen`: 0 bytes
- `OpenBrace`: 0 bytes
- `CloseBrace`: 0 bytes
- `OpenBracket`: 0 bytes
- `CloseBracket`: 0 bytes
- `OpenInvisible`: 3 bytes
- `CloseInvisible`: 3 bytes
- `Literal`: 15 bytes
- `Ident`: 7 bytes
- `NtIdent`: 15 bytes
- `Lifetime`: 7 bytes
- `NtLifetime`: 15 bytes
- `DocComment`: 7 bytes
- `Eof`: 0 bytes