TokenTree in rustc_proc_macro - Rust
Description: A single token or a delimited sequence of token trees (e.g., `[1, (), ..]`).

Skip to main content

## TokenTree

# Enum TokenTree

1.29.0 · Source

    pub enum TokenTree {
        Group(Group),
        Ident(Ident),
        Punct(Punct),
        Literal(Literal),
    }

Expand description

A single token or a delimited sequence of token trees (e.g., `[1, (), ..]`).

## Variants§

§1.29.0

### Group(Group)

A token stream surrounded by bracket delimiters.

§1.29.0

### Ident(Ident)

An identifier.

§1.29.0

### Punct(Punct)

A single punctuation character (`+`, `,`, `$`, etc.).

§1.29.0

### Literal(Literal)

A literal character (`'a'`), string (`"hello"`), number (`2.3`), etc.

## Implementations§

Source§

### impl TokenTree

1.29.0 · Source

#### pub fn span(&self) -> Span

Returns the span of this tree, delegating to the `span` method of the contained token or a delimited stream.

1.29.0 · Source

#### pub fn set_span(&mut self, span: Span)

Configures the span for only this token.

Note that if this token is a `Group` then this method will not configure the span of each of the internal tokens, this will simply delegate to the `set_span` method of each variant.

## Trait Implementations§

1.29.0 · Source§

### impl !Send for TokenTree

1.29.0 · Source§

### impl !Sync for TokenTree

1.29.0 · Source§

### impl Clone for TokenTree

Source§

#### fn clone(&self) -> TokenTree

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

#### fn clone_from(&mut self, source: &Self)

Performs copy-assignment from `source`. Read more
1.29.0 · Source§

### impl Debug for TokenTree

Prints token tree in a form convenient for debugging.

Source§

#### fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
1.29.0 · Source§

### impl Display for TokenTree

Prints the token tree as a string that is supposed to be losslessly convertible back into the same token tree (modulo spans), except for possibly `TokenTree::Group`s with `Delimiter::None` delimiters and negative numeric literals.

Note: the exact form of the output is subject to change, e.g. there might be changes in the whitespace used between tokens. Therefore, you should not do any kind of simple substring matching on the output string (as produced by `to_string`) to implement a proc macro, because that matching might stop working if such changes happen. Instead, you should work at the `TokenTree` level, e.g. matching against `TokenTree::Ident`, `TokenTree::Punct`, or `TokenTree::Literal`.

Source§

#### fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
1.30.0 · Source§

### impl Extend<TokenTree> for TokenStream

Source§

#### fn extend<I: IntoIterator<Item = TokenTree>>(&mut self, trees: I)

Extends a collection with the contents of an iterator. Read more
Source§

#### fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Extends a collection with exactly one element.
Source§

#### fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Reserves capacity in a collection for the given number of additional elements. Read more
1.29.0 · Source§

### impl From<Group> for TokenTree

Source§

#### fn from(g: Group) -> TokenTree

Converts to this type from the input type.
1.29.0 · Source§

### impl From<Ident> for TokenTree

Source§

#### fn from(g: Ident) -> TokenTree

Converts to this type from the input type.
1.29.0 · Source§

### impl From<Literal> for TokenTree

Source§

#### fn from(g: Literal) -> TokenTree

Converts to this type from the input type.
1.29.0 · Source§

### impl From<Punct> for TokenTree

Source§

#### fn from(g: Punct) -> TokenTree

Converts to this type from the input type.
1.29.0 · Source§

### impl From<TokenTree> for TokenStream

Creates a token stream containing a single token tree.

Source§

#### fn from(tree: TokenTree) -> TokenStream

Converts to this type from the input type.
1.29.0 · Source§

### impl FromIterator<TokenTree> for TokenStream

Collects a number of token trees into a single stream.

Source§

#### fn from_iter<I: IntoIterator<Item = TokenTree>>(trees: I) -> Self

Creates a value from an iterator. Read more
Source§

### impl ToTokens for TokenTree

Source§

#### fn to_tokens(&self, tokens: &mut TokenStream)

🔬This is a nightly-only experimental API. (`proc_macro_totokens` #130977)
Write `self` to the given `TokenStream`. Read more
Source§

#### fn into_token_stream(self) -> TokenStream

🔬This is a nightly-only experimental API. (`proc_macro_totokens` #130977)
Convert `self` directly into a `TokenStream` object. Read more
Source§

#### fn to_token_stream(&self) -> TokenStream

🔬This is a nightly-only experimental API. (`proc_macro_totokens` #130977)
Convert `self` directly into a `TokenStream` object. Read more

## Auto Trait Implementations§

§

### impl Freeze for TokenTree

§

### impl RefUnwindSafe for TokenTree

§

### impl Unpin for TokenTree

§

### impl UnsafeUnpin for TokenTree

§

### impl UnwindSafe for TokenTree

## Blanket Implementations§

Source§

### impl<T> Any for T
where T: 'static + ?Sized,

Source§

#### fn type_id(&self) -> TypeId

Gets the `TypeId` of `self`. Read more
Source§

### impl<T> Borrow<T> for T
where T: ?Sized,

Source§

#### fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

### impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

#### fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

### impl<T> CloneToUninit for T
where T: Clone,

Source§

#### unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (`clone_to_uninit` #126799)
Performs copy-assignment from `self` to `dest`. Read more
Source§

### impl<T> From<T> for T

Source§

#### fn from(t: T) -> T

Returns the argument unchanged.

Source§

### impl<T, U> Into<U> for T
where U: From<T>,

Source§

#### fn into(self) -> U

Calls `U::from(self)`.

That is, this conversion is whatever the implementation of `From<T> for U` chooses to do.

Source§

### impl<T> ToOwned for T
where T: Clone,

Source§

#### type Owned = T

The resulting type after obtaining ownership.
Source§

#### fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

#### fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

### impl<T> ToString for T
where T: Display + ?Sized,

Source§

#### fn to_string(&self) -> String

Converts the given value to a `String`. Read more
Source§

### impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

#### type Error = Infallible

The type returned in the event of a conversion error.
Source§

#### fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

### impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

#### type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

#### fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.

## Layout§

Note: Most layout information is completely unstable and may even differ between compilations. The only exception is types with certain `repr(...)` attributes. Please see the Rust Reference's “Type Layout” chapter for details on type layout guarantees.

Size: 20 bytes

Size for each variant:

- `Group`: 20 bytes
- `Ident`: 12 bytes
- `Punct`: 8 bytes
- `Literal`: 16 bytes