# Third-Party Licenses

This crate embeds tokenizer vocabulary data from several sources.
Each is subject to its original license as described below.

## OpenAI tiktoken (cl100k_base, o200k_base, p50k_base, p50k_edit, r50k_base)

- Source: https://github.com/openai/tiktoken
- License: MIT
- Copyright (c) 2022 OpenAI

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions: The above copyright
notice and this permission notice shall be included in all copies or substantial
portions of the Software.

## Meta Llama 3 (llama3)

- Source: https://huggingface.co/meta-llama/Llama-3.1-8B
- License: Llama 3 Community License Agreement
- Copyright (c) Meta Platforms, Inc.

Built with Meta Llama 3.

The tokenizer data is distributed under the Llama 3 Community License Agreement.
A copy of the license is available at:
https://github.com/meta-llama/llama3/blob/main/LICENSE

Key requirements:
- Attribution: "Built with Meta Llama 3" must be prominently displayed
- A copy of the license agreement must accompany any distribution

## DeepSeek V3 (deepseek_v3)

- Source: https://huggingface.co/deepseek-ai/DeepSeek-V3
- License: DeepSeek Model License (code under MIT)
- Copyright (c) 2024 DeepSeek

The tokenizer data is distributed under the DeepSeek Model License.
A copy of the license is available at:
https://github.com/deepseek-ai/DeepSeek-V3/blob/main/LICENSE-MODEL

Key requirements:
- Use-based restrictions from Attachment A must be preserved
- A copy of the license must accompany any distribution
- Modified files must carry prominent modification notices

## Qwen 2.5 (qwen2)

- Source: https://huggingface.co/Qwen/Qwen2.5-7B
- License: Apache 2.0
- Copyright (c) Alibaba Cloud

Licensed under the Apache License, Version 2.0.
You may obtain a copy of the License at:
https://www.apache.org/licenses/LICENSE-2.0

## Mistral v3 Tekken (mistral_v3)

- Source: https://github.com/mistralai/mistral-common
- License: Apache 2.0
- Copyright (c) Mistral AI

Licensed under the Apache License, Version 2.0.
You may obtain a copy of the License at:
https://www.apache.org/licenses/LICENSE-2.0
