TokenType in rustc_parse::parser::token_type - Rust
Description: Used in “expected”/“expected one of” error messages. Tokens are added here as necessary. Tokens with values (e.g. literals, identifiers) are represented by a single variant (e.g. `Literal`, `Ident`).

Skip to main content

## TokenType

# Enum TokenType

Source

    pub enum TokenType {
    Show 112 variants    Eq,
        Lt,
        Le,
        EqEq,
        Gt,
        AndAnd,
        OrOr,
        Bang,
        Tilde,
        Plus,
        Minus,
        Star,
        And,
        Or,
        At,
        Dot,
        DotDot,
        DotDotDot,
        DotDotEq,
        Comma,
        Semi,
        Colon,
        PathSep,
        RArrow,
        FatArrow,
        Pound,
        Question,
        OpenParen,
        CloseParen,
        OpenBrace,
        CloseBrace,
        OpenBracket,
        CloseBracket,
        Eof,
        Operator,
        Ident,
        Lifetime,
        Path,
        Type,
        Const,
        KwAs,
        KwAsync,
        KwAuto,
        KwAwait,
        KwBecome,
        KwBox,
        KwBreak,
        KwCatch,
        KwConst,
        KwContinue,
        KwContractEnsures,
        KwContractRequires,
        KwCrate,
        KwDefault,
        KwDyn,
        KwElse,
        KwEnum,
        KwExtern,
        KwFinal,
        KwFn,
        KwFor,
        KwGen,
        KwIf,
        KwImpl,
        KwIn,
        KwLet,
        KwLoop,
        KwMacro,
        KwMacroRules,
        KwMatch,
        KwMod,
        KwMove,
        KwMut,
        KwPub,
        KwRaw,
        KwRef,
        KwReturn,
        KwReuse,
        KwSafe,
        KwSelfUpper,
        KwStatic,
        KwStruct,
        KwSuper,
        KwTrait,
        KwTry,
        KwType,
        KwUnderscore,
        KwUnsafe,
        KwUse,
        KwWhere,
        KwWhile,
        KwYield,
        SymAttSyntax,
        SymBikeshed,
        SymClobberAbi,
        SymInlateout,
        SymInout,
        SymIs,
        SymLabel,
        SymLateout,
        SymMayUnwind,
        SymNomem,
        SymNoreturn,
        SymNostack,
        SymNull,
        SymOptions,
        SymOut,
        SymPin,
        SymPreservesFlags,
        SymPure,
        SymReadonly,
        SymSym,
    }

Expand description

Used in “expected”/“expected one of” error messages. Tokens are added here as necessary. Tokens with values (e.g. literals, identifiers) are represented by a single variant (e.g. `Literal`, `Ident`).

It’s an awkward representation, but it’s important for performance. It’s a C-style parameterless enum so that `TokenTypeSet` can be a bitset. This is important because `Parser::expected_token_types` is very hot. `TokenType` used to have variants with parameters (e.g. all the keywords were in a single `Keyword` variant with a `Symbol` parameter) and `Parser::expected_token_types` was a `Vec<TokenType>` which was much slower to manipulate.

We really want to keep the number of variants to 128 or fewer, so that `TokenTypeSet` can be implemented with a `u128`.

## Variants§

§

### Eq

§

### Lt

§

### Le

§

### EqEq

§

### Gt

§

### AndAnd

§

### OrOr

§

### Bang

§

### Tilde

§

### Plus

§

### Minus

§

### Star

§

### And

§

### Or

§

### At

§

### Dot

§

### DotDot

§

### DotDotDot

§

### DotDotEq

§

### Comma

§

### Semi

§

### Colon

§

### PathSep

§

### RArrow

§

### FatArrow

§

### Pound

§

### Question

§

### OpenParen

§

### CloseParen

§

### OpenBrace

§

### CloseBrace

§

### OpenBracket

§

### CloseBracket

§

### Eof

§

### Operator

Any operator.

§

### Ident

Any identifier token.

§

### Lifetime

Any lifetime token.

§

### Path

Any token that can start a path.

§

### Type

Any token that can start a type.

§

### Const

Any token that can start a const expression.

§

### KwAs

§

### KwAsync

§

### KwAuto

§

### KwAwait

§

### KwBecome

§

### KwBox

§

### KwBreak

§

### KwCatch

§

### KwConst

§

### KwContinue

§

### KwContractEnsures

§

### KwContractRequires

§

### KwCrate

§

### KwDefault

§

### KwDyn

§

### KwElse

§

### KwEnum

§

### KwExtern

§

### KwFinal

§

### KwFn

§

### KwFor

§

### KwGen

§

### KwIf

§

### KwImpl

§

### KwIn

§

### KwLet

§

### KwLoop

§

### KwMacro

§

### KwMacroRules

§

### KwMatch

§

### KwMod

§

### KwMove

§

### KwMut

§

### KwPub

§

### KwRaw

§

### KwRef

§

### KwReturn

§

### KwReuse

§

### KwSafe

§

### KwSelfUpper

§

### KwStatic

§

### KwStruct

§

### KwSuper

§

### KwTrait

§

### KwTry

§

### KwType

§

### KwUnderscore

§

### KwUnsafe

§

### KwUse

§

### KwWhere

§

### KwWhile

§

### KwYield

§

### SymAttSyntax

§

### SymBikeshed

§

### SymClobberAbi

§

### SymInlateout

§

### SymInout

§

### SymIs

§

### SymLabel

§

### SymLateout

§

### SymMayUnwind

§

### SymNomem

§

### SymNoreturn

§

### SymNostack

§

### SymNull

§

### SymOptions

§

### SymOut

§

### SymPin

§

### SymPreservesFlags

§

### SymPure

§

### SymReadonly

§

### SymSym

## Implementations§

Source§

### impl TokenType

Source

#### fn from_u32(val: u32) -> TokenType

Source

#### pub(super) fn is_keyword(&self) -> Option<Symbol>

Source

#### pub(super) fn to_string(&self) -> String

## Trait Implementations§

Source§

### impl Clone for TokenType

Source§

#### fn clone(&self) -> TokenType

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

#### fn clone_from(&mut self, source: &Self)

Performs copy-assignment from `source`. Read more
Source§

### impl Debug for TokenType

Source§

#### fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

### impl PartialEq for TokenType

Source§

#### fn eq(&self, other: &TokenType) -> bool

Tests for `self` and `other` values to be equal, and is used by `==`.
1.0.0 (const: unstable) · Source§

#### fn ne(&self, other: &Rhs) -> bool

Tests for `!=`. The default implementation is almost always sufficient, and should not be overridden without very good reason.
Source§

### impl Copy for TokenType

Source§

### impl StructuralPartialEq for TokenType

## Auto Trait Implementations§

§

### impl DynSend for TokenType

§

### impl DynSync for TokenType

§

### impl Freeze for TokenType

§

### impl RefUnwindSafe for TokenType

§

### impl Send for TokenType

§

### impl Sync for TokenType

§

### impl Unpin for TokenType

§

### impl UnsafeUnpin for TokenType

§

### impl UnwindSafe for TokenType

## Blanket Implementations§

Source§

### impl<T> Aligned for T

Source§

#### const ALIGN: Alignment

Alignment of `Self`.
Source§

### impl<T> Any for T
where T: 'static + ?Sized,

Source§

#### fn type_id(&self) -> TypeId

Gets the `TypeId` of `self`. Read more
Source§

### impl<T> AnyEq for T
where T: Any + PartialEq,

Source§

#### fn equals(&self, other: &(dyn Any + 'static)) -> bool

Source§

#### fn as_any(&self) -> &(dyn Any + 'static)

Source§

### impl<'tcx, T> ArenaAllocatable<'tcx, IsCopy> for T
where T: Copy,

Source§

#### fn allocate_on(self, arena: &'tcx Arena<'tcx>) -> &'tcx mut T

Source§

#### fn allocate_from_iter( arena: &'tcx Arena<'tcx>, iter: impl IntoIterator<Item = T>, ) -> &'tcx mut [T]

Source§

### impl<T> Borrow<T> for T
where T: ?Sized,

Source§

#### fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

### impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

#### fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

### impl<T> CloneToUninit for T
where T: Clone,

Source§

#### unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (`clone_to_uninit`)
Performs copy-assignment from `self` to `dest`. Read more
Source§

### impl<T> DynClone for T
where T: Clone,

Source§

#### fn __clone_box(&self, _: Private) -> *mut ()

Source§

### impl<T> From<T> for T

Source§

#### fn from(t: T) -> T

Returns the argument unchanged.

Source§

### impl<T> Instrument for T

Source§

#### fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided `Span`, returning an `Instrumented` wrapper. Read more
Source§

#### fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current `Span`, returning an `Instrumented` wrapper. Read more
Source§

### impl<T, U> Into<U> for T
where U: From<T>,

Source§

#### fn into(self) -> U

Calls `U::from(self)`.

That is, this conversion is whatever the implementation of `From<T> for U` chooses to do.

Source§

### impl<T> IntoEither for T

Source§

#### fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts `self` into a `Left` variant of `Either<Self, Self>` if `into_left` is `true`. Converts `self` into a `Right` variant of `Either<Self, Self>` otherwise. Read more
Source§

#### fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts `self` into a `Left` variant of `Either<Self, Self>` if `into_left(&self)` returns `true`. Converts `self` into a `Right` variant of `Either<Self, Self>` otherwise. Read more
Source§

### impl<T> Pointable for T

Source§

#### const ALIGN: usize

The alignment of pointer.
Source§

#### type Init = T

The type for initializers.
Source§

#### unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

#### unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

#### unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

#### unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

### impl<T> Same for T

Source§

#### type Output = T

Should always be `Self`
Source§

### impl<T> ToOwned for T
where T: Clone,

Source§

#### type Owned = T

The resulting type after obtaining ownership.
Source§

#### fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

#### fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

### impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

#### type Error = Infallible

The type returned in the event of a conversion error.
Source§

#### fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

### impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

#### type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

#### fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

### impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

#### fn vzip(self) -> V

Source§

### impl<T> WithSubscriber for T

Source§

#### fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided `Subscriber` to this type, returning a `WithDispatch` wrapper. Read more
Source§

#### fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default `Subscriber` to this type, returning a `WithDispatch` wrapper. Read more
Source§

### impl<T> ErasedDestructor for T
where T: 'static,

## Layout§

Note: Most layout information is completely unstable and may even differ between compilations. The only exception is types with certain `repr(...)` attributes. Please see the Rust Reference's “Type Layout” chapter for details on type layout guarantees.

Size: 1 byte

Size for each variant:

- `Eq`: 0 bytes
- `Lt`: 0 bytes
- `Le`: 0 bytes
- `EqEq`: 0 bytes
- `Gt`: 0 bytes
- `AndAnd`: 0 bytes
- `OrOr`: 0 bytes
- `Bang`: 0 bytes
- `Tilde`: 0 bytes
- `Plus`: 0 bytes
- `Minus`: 0 bytes
- `Star`: 0 bytes
- `And`: 0 bytes
- `Or`: 0 bytes
- `At`: 0 bytes
- `Dot`: 0 bytes
- `DotDot`: 0 bytes
- `DotDotDot`: 0 bytes
- `DotDotEq`: 0 bytes
- `Comma`: 0 bytes
- `Semi`: 0 bytes
- `Colon`: 0 bytes
- `PathSep`: 0 bytes
- `RArrow`: 0 bytes
- `FatArrow`: 0 bytes
- `Pound`: 0 bytes
- `Question`: 0 bytes
- `OpenParen`: 0 bytes
- `CloseParen`: 0 bytes
- `OpenBrace`: 0 bytes
- `CloseBrace`: 0 bytes
- `OpenBracket`: 0 bytes
- `CloseBracket`: 0 bytes
- `Eof`: 0 bytes
- `Operator`: 0 bytes
- `Ident`: 0 bytes
- `Lifetime`: 0 bytes
- `Path`: 0 bytes
- `Type`: 0 bytes
- `Const`: 0 bytes
- `KwAs`: 0 bytes
- `KwAsync`: 0 bytes
- `KwAuto`: 0 bytes
- `KwAwait`: 0 bytes
- `KwBecome`: 0 bytes
- `KwBox`: 0 bytes
- `KwBreak`: 0 bytes
- `KwCatch`: 0 bytes
- `KwConst`: 0 bytes
- `KwContinue`: 0 bytes
- `KwContractEnsures`: 0 bytes
- `KwContractRequires`: 0 bytes
- `KwCrate`: 0 bytes
- `KwDefault`: 0 bytes
- `KwDyn`: 0 bytes
- `KwElse`: 0 bytes
- `KwEnum`: 0 bytes
- `KwExtern`: 0 bytes
- `KwFinal`: 0 bytes
- `KwFn`: 0 bytes
- `KwFor`: 0 bytes
- `KwGen`: 0 bytes
- `KwIf`: 0 bytes
- `KwImpl`: 0 bytes
- `KwIn`: 0 bytes
- `KwLet`: 0 bytes
- `KwLoop`: 0 bytes
- `KwMacro`: 0 bytes
- `KwMacroRules`: 0 bytes
- `KwMatch`: 0 bytes
- `KwMod`: 0 bytes
- `KwMove`: 0 bytes
- `KwMut`: 0 bytes
- `KwPub`: 0 bytes
- `KwRaw`: 0 bytes
- `KwRef`: 0 bytes
- `KwReturn`: 0 bytes
- `KwReuse`: 0 bytes
- `KwSafe`: 0 bytes
- `KwSelfUpper`: 0 bytes
- `KwStatic`: 0 bytes
- `KwStruct`: 0 bytes
- `KwSuper`: 0 bytes
- `KwTrait`: 0 bytes
- `KwTry`: 0 bytes
- `KwType`: 0 bytes
- `KwUnderscore`: 0 bytes
- `KwUnsafe`: 0 bytes
- `KwUse`: 0 bytes
- `KwWhere`: 0 bytes
- `KwWhile`: 0 bytes
- `KwYield`: 0 bytes
- `SymAttSyntax`: 0 bytes
- `SymBikeshed`: 0 bytes
- `SymClobberAbi`: 0 bytes
- `SymInlateout`: 0 bytes
- `SymInout`: 0 bytes
- `SymIs`: 0 bytes
- `SymLabel`: 0 bytes
- `SymLateout`: 0 bytes
- `SymMayUnwind`: 0 bytes
- `SymNomem`: 0 bytes
- `SymNoreturn`: 0 bytes
- `SymNostack`: 0 bytes
- `SymNull`: 0 bytes
- `SymOptions`: 0 bytes
- `SymOut`: 0 bytes
- `SymPin`: 0 bytes
- `SymPreservesFlags`: 0 bytes
- `SymPure`: 0 bytes
- `SymReadonly`: 0 bytes
- `SymSym`: 0 bytes