Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ sh INSTALL_MEGATRON.sh
| Server startup scripts | transformers/megatron | [Script](cookbook/client/server) |

## Changelog
- 🎉2026-08-12 The ModelScope training service has been deployed to [Qwen/Qwen3.8-27B](https://www.modelscope.cn/models/Qwen/Qwen3.8-27B).
- 🎉2026-08-04 Sandboxed multi-turn RL is now supported: run model-generated code in isolated [AgentENV](https://github.com/kvcache-ai/AgentENV) Firecracker microVMs, or in an OpenEnv server, with the same `train.py`. See the [cookbook](cookbook/rl/envs) and the [deployment guide](docs/source_en/Usage%20Guide/Agentic-RL-Deployment-and-Training.md).
- 🎉2026-05-20 Support DeepSeek-V4-Flash and DeepSeek-V4-Pro models.
- 🎉2026-05-20 Multi-turn rollout and tool calling in RL are now supported. The Cookbook is currently being written. You can use `from twinkle_agentic.rollout import MultiTurnRollout/APIMultiTurnRollout` directly for multi-turn rollout.
Expand Down Expand Up @@ -142,13 +143,16 @@ supported on Twinkle✨ framework.
> For serverless training service accessed via `base_url=https://www.modelscope.cn/twinkle`, it
> is currently provided via the Tinker-compatible APIs. We will be rolling out services that support
> both Tinker APIs, as well as the full-fledged Twinkle✨ native APIs. The serverless endpoint is backed
> by one training base at a time, and currently it is [Qwen3.6-27B](https://modelscope.cn/models/Qwen/Qwen3.6-27B).
> by one training base at a time, and currently it is [Qwen3.8-27B](https://modelscope.cn/models/Qwen/Qwen3.8-27B).

| Model Type | Model ID on [ModelScope](https://modelscope.cn) | Model Size | Requires | Support Megatron | HF Model ID |
|---------------------|-----------------------------------------------------------------------------------------------------------------|:---------------------------------------:|----------------------|:----------------:|:---------------------------------------------------------------------------------------------------------:|
| qwen3 series | [Qwen/Qwen3-14B-Base](https://modelscope.cn/models/Qwen/Qwen3-14B-Base) | 0.6B/1.7B/4B/8B/14B | transformers>=4.51 | ✔ | [Qwen/Qwen3-14B-Base](https://huggingface.co/Qwen/Qwen3-14B-Base) |
| | [Qwen/Qwen3-32B](https://modelscope.cn/models/Qwen/Qwen3-32B) | 0.6B/1.7B/4B/8B/14B/32B | transformers>=4.51 | ✔ | [Qwen/Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B) |
| qwen3_moe series | [Qwen/Qwen3-30B-A3B-Base](https://modelscope.cn/models/Qwen/Qwen3-30B-A3B-Base) | 30B-A3B/A3B-Base,235B-A22B | transformers>=4.51 | ✔ | [Qwen/Qwen3-30B-A3B-Base](https://huggingface.co/Qwen/Qwen3-30B-A3B-Base) |
| qwen3.8 series | [Qwen/Qwen3.8-27B](https://www.modelscope.cn/models/Qwen/Qwen3.8-27B) | 27B | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
| qwen3.6 moe series | [Qwen/Qwen3.6-35B-A3B](https://www.modelscope.cn/models/Qwen/Qwen3.6-35B-A3B) | 35B-A3B, etc. | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) |
| qwen3.6 series | [Qwen/Qwen3.6-27B](https://www.modelscope.cn/models/Qwen/Qwen3.6-27B) | 4B ~ 27B | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) |
| qwen3.5 moe series | [Qwen/Qwen3.5-35B-A3B](https://www.modelscope.cn/models/Qwen/Qwen3.5-35B-A3B) | 35B-A3B,122B-A10B, etc. | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) |
| qwen3.5 series | [Qwen/Qwen3.5-9B](https://www.modelscope.cn/models/Qwen/Qwen3.5-9B) | 2B ~ 27B | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) |
| qwen2 series | [Qwen/Qwen2-0.5B-Instruct](https://modelscope.cn/models/Qwen/Qwen2-0.5B-Instruct) | 0.5B/1.5B/7B/72B | transformers>=4.37 | ✔ | [Qwen/Qwen2-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct) |
Expand Down Expand Up @@ -254,7 +258,7 @@ from twinkle.dataset import Dataset, DatasetMeta
from twinkle.preprocessor import SelfCognitionProcessor
from twinkle.server.common import input_feature_to_datum

base_model = 'ms://Qwen/Qwen3.6-27B'
base_model = 'ms://Qwen/Qwen3.8-27B'
base_url='your-base-url'
api_key='your-api-key'

Expand Down
8 changes: 6 additions & 2 deletions README_ZH.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ sh INSTALL_MEGATRON.sh
Twinkle✨支持相同的算法接口运行在单GPU、torchrun多机、Ray、Client等各场景下。其算法过程是外露的,非常便于修改和调试。完整的框架介绍请查看[快速开始](https://modelscope.github.io/twinkle-web/zh/docs/usage-guide/quick-start/)

## 更新日志
- 🎉2026-08-12 ModelScope的训练服务部署为[Qwen/Qwen3.8-27B](https://www.modelscope.cn/models/Qwen/Qwen3.8-27B)。
- 🎉2026-08-04 支持沙箱环境下的多轮RL训练:模型生成的代码可在隔离的 [AgentENV](https://github.com/kvcache-ai/AgentENV) Firecracker microVM 或 OpenEnv 服务中执行,两个后端共用同一份 `train.py`。参考 [cookbook](cookbook/rl/envs) 和[部署文档](docs/source_zh/使用指引/Agentic%20RL部署与训练.md)。
- 🎉2026-05-20 支持DeepSeek-V4-Flash and DeepSeek-V4-Pro系列模型。
- 🎉2026-05-20 支持多轮rollout和RL中的工具调用,Cookbook正在编写中,可以直接使用`from twinkle_agentic.rollout import MultiTurnRollout/APIMultiTurnRollout`进行多轮rollout。
Expand Down Expand Up @@ -132,13 +133,16 @@ Twinkle✨支持相同的算法接口运行在单GPU、torchrun多机、Ray、Cl
随着新模型的发布,我们将添加对更多模型的支持。下表列出了 Twinkle✨ 框架当前支持的模型。

>[!Note]
> 通过 `base_url=https://www.modelscope.cn/twinkle` 访问的无服务器训练服务,目前是通过兼容Tinker的API提供的。我们将陆续推出同时支持Tinker API和完整Twinkle✨原生 API的服务。无服务器端点每次由一个训练基座支持,目前使用的是[Qwen3.6-27B](https://modelscope.cn/models/Qwen/Qwen3.6-27B)。
> 通过 `base_url=https://www.modelscope.cn/twinkle` 访问的无服务器训练服务,目前是通过兼容Tinker的API提供的。我们将陆续推出同时支持Tinker API和完整Twinkle✨原生 API的服务。无服务器端点每次由一个训练基座支持,目前使用的是[Qwen3.8-27B](https://modelscope.cn/models/Qwen/Qwen3.8-27B)。

| Model Type | Model ID 举例 | Model Size | Requires | Support Megatron | HF Model ID |
|---------------------|-----------------------------------------------------------------------------------------------------------------|:---------------------------------------:|----------------------|:----------------:|:---------------------------------------------------------------------------------------------------------:|
| qwen3 全系列 | [Qwen/Qwen3-14B-Base](https://modelscope.cn/models/Qwen/Qwen3-14B-Base) | 0.6B/1.7B/4B/8B/14B | transformers>=4.51 | ✔ | [Qwen/Qwen3-14B-Base](https://huggingface.co/Qwen/Qwen3-14B-Base) |
| | [Qwen/Qwen3-32B](https://modelscope.cn/models/Qwen/Qwen3-32B) | 0.6B/1.7B/4B/8B/14B/32B | transformers>=4.51 | ✔ | [Qwen/Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B) |
| qwen3_moe 全系列 | [Qwen/Qwen3-30B-A3B-Base](https://modelscope.cn/models/Qwen/Qwen3-30B-A3B-Base) | 30B-A3B/A3B-Base,235B-A22B | transformers>=4.51 | ✔ | [Qwen/Qwen3-30B-A3B-Base](https://huggingface.co/Qwen/Qwen3-30B-A3B-Base) |
| qwen3.8 全系列 | [Qwen/Qwen3.8-27B](https://www.modelscope.cn/models/Qwen/Qwen3.8-27B) | 27B | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
| qwen3.6 moe 全系列 | [Qwen/Qwen3.6-35B-A3B](https://www.modelscope.cn/models/Qwen/Qwen3.6-35B-A3B) | 35B-A3B, etc. | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) |
| qwen3.6 全系列 | [Qwen/Qwen3.6-27B](https://www.modelscope.cn/models/Qwen/Qwen3.6-27B) | 4B ~ 27B | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) |
| qwen3.5 moe 全系列 | [Qwen/Qwen3.5-35B-A3B](https://www.modelscope.cn/models/Qwen/Qwen3.5-35B-A3B) | 35B-A3B,122B-A10B, etc. | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) |
| qwen3.5 全系列 | [Qwen/Qwen3.5-9B](https://www.modelscope.cn/models/Qwen/Qwen3.5-9B) | 2B ~ 27B | transformers>=5.2.0 | ✔ | [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) |
| qwen2 全系列 | [Qwen/Qwen2-0.5B-Instruct](https://modelscope.cn/models/Qwen/Qwen2-0.5B-Instruct) | 0.5B/1.5B/7B/72B | transformers>=4.37 | ✔ | [Qwen/Qwen2-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct) |
Expand Down Expand Up @@ -243,7 +247,7 @@ from twinkle.dataset import Dataset, DatasetMeta
from twinkle.preprocessor import SelfCognitionProcessor
from twinkle.server.common import input_feature_to_datum

base_model = 'ms://Qwen/Qwen3.6-27B'
base_model = 'ms://Qwen/Qwen3.8-27B'
base_url='your-base-url'
api_key='your-api-key'

Expand Down
14 changes: 7 additions & 7 deletions cookbook/client/server/megatron/server_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ applications:
server_config:
per_token_model_limit: 3 # Maximum number of models (adapters) per token (server-globally enforced)
supported_models:
- Qwen/Qwen3.6-27B
- Qwen/Qwen3.8-27B

deployments:
- name: TinkerCompatServer
Expand All @@ -59,11 +59,11 @@ applications:
# 3. Sampler Service - Runs inference / sampling using vLLM engine
# Used for generating text from the model (e.g., evaluating LoRA results).
# Config: TP=2 x DP=2 on 4 GPUs, ~27GB weights/GPU, ~37GB for KV cache + LoRA
- name: sampler-Qwen3.6-27B
route_prefix: /api/v1/sampler/Qwen/Qwen3.6-27B
- name: sampler-Qwen3.8-27B
route_prefix: /api/v1/sampler/Qwen/Qwen3.8-27B
import_path: sampler
args:
model_id: "ms://Qwen/Qwen3.6-27B" # ModelScope model identifier
model_id: "ms://Qwen/Qwen3.8-27B" # ModelScope model identifier
nproc_per_node: 4 # Number of GPU processes per node
sampler_type: vllm # Inference engine: 'vllm' (fast) or 'torch' (TorchSampler)
engine_args: # vLLM engine-specific settings
Expand Down Expand Up @@ -102,12 +102,12 @@ applications:

# 2. Model Service - Hosts the base model for training.
# Config: PP=2 x DP=2 on 4 GPUs, ~27GB weights/GPU, comfortable for LoRA training
- name: models-Qwen3.6-27B
route_prefix: /api/v1/model/Qwen/Qwen3.6-27B
- name: models-Qwen3.8-27B
route_prefix: /api/v1/model/Qwen/Qwen3.8-27B
import_path: model
args:
backend: megatron # Use Megatron-LM backend
model_id: "ms://Qwen/Qwen3.6-27B" # ModelScope model identifier
model_id: "ms://Qwen/Qwen3.8-27B" # ModelScope model identifier
max_length: 32768 # model max length
max_loras: 3 # model max loras
nproc_per_node: 4 # Number of GPU processes per node
Expand Down
2 changes: 2 additions & 0 deletions docs/source_en/Components/Model/SupportedModels.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ Twinkle supports any model compatible with HuggingFace Transformers or Megatron-

| Model Family | Model IDs | Parameters | Features |
|:-------------|:----------|:-----------|:---------|
| Qwen 3.8 | `Qwen/Qwen3.8-27B` | 27B | Thinking mode |
| Qwen 3.6 | `Qwen/Qwen3.6-27B`, `Qwen/Qwen3.6-35B-A3B` | 27B, 35B-A3B | MoE, Thinking mode |
| Qwen 3.5 | `Qwen/Qwen3.5-0.6B` ~ `Qwen/Qwen3.5-235B-A22B` | 0.6B–235B | MoE, Thinking mode |
| Qwen 2.5 | `Qwen/Qwen2.5-0.5B` ~ `Qwen/Qwen2.5-72B` | 0.5B–72B | Dense |
| DeepSeek V4 | `deepseek-ai/DeepSeek-V4` | 685B MoE | Custom DSML encoding |
Expand Down
8 changes: 4 additions & 4 deletions docs/source_en/Usage Guide/Introduction-with-Qwen3.5.md
Original file line number Diff line number Diff line change
Expand Up @@ -465,9 +465,9 @@ init_tinker_client()

from tinker import ServiceClient

# Base model
base_model = 'Qwen/Qwen3.5-4B'
base_url = 'http://www.modelscope.cn/twinkle'
# Base model (currently deployed in the ModelScope official environment)
base_model = 'Qwen/Qwen3.8-27B'
base_url = 'https://www.modelscope.cn/twinkle'


def train():
Expand Down Expand Up @@ -530,7 +530,7 @@ Alongside the open-source release of Twinkle, ModelScope provides a hosted model

```python
base_url = 'https://www.modelscope.cn/twinkle'
base_model = 'Qwen/Qwen3.5-4B' # Model currently deployed in the official environment
base_model = 'Qwen/Qwen3.8-27B' # Model currently deployed in the official environment
```

---
Expand Down
2 changes: 1 addition & 1 deletion docs/source_en/Usage Guide/Server and Client/Overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,5 +106,5 @@ The same examples work with the ModelScope managed service by changing only the
```bash
export TWINKLE_SERVER_URL=https://www.modelscope.cn/twinkle
export TWINKLE_SERVER_TOKEN="$MODELSCOPE_TOKEN"
export TWINKLE_MODEL_ID=Qwen/Qwen3.6-27B
export TWINKLE_MODEL_ID=Qwen/Qwen3.8-27B
```
10 changes: 5 additions & 5 deletions docs/source_en/Usage Guide/Train-as-a-Service.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Alongside the open-source release of the Twinkle framework, we also provide a hosted model training service (Training as a Service) powered by ModelScope's backend infrastructure. Developers can use this service to experience Twinkle's training API for free.

The model currently running on the cluster is [Qwen/Qwen3.6-27B](https://www.modelscope.cn/models/Qwen/Qwen3.6-27B). Below are the detailed usage instructions:
The model currently running on the cluster is [Qwen/Qwen3.8-27B](https://www.modelscope.cn/models/Qwen/Qwen3.8-27B). Below are the detailed usage instructions:

## Step 1. Register a ModelScope Account and Obtain Your API Key

Expand Down Expand Up @@ -30,7 +30,7 @@ from twinkle.dataset import Dataset, DatasetMeta
from twinkle.preprocessor import SelfCognitionProcessor
from twinkle.server.common import input_feature_to_datum

base_model = 'ms://Qwen/Qwen3.6-27B'
base_model = 'ms://Qwen/Qwen3.8-27B'
base_url='https://www.modelscope.cn/twinkle'
api_key=os.environ.get('MODELSCOPE_TOKEN')

Expand Down Expand Up @@ -64,7 +64,7 @@ for epoch in range(2):
print(f'Saved checkpoint for epoch {epoch} to {result.path}')
```

With the code above, you can train a self-cognition LoRA based on `Qwen/Qwen3.6-27B`. This LoRA will change the model's name and creator to the names specified during training. To perform inference using this LoRA:
With the code above, you can train a self-cognition LoRA based on `Qwen/Qwen3.8-27B`. This LoRA will change the model's name and creator to the names specified during training. To perform inference using this LoRA:

```python
import os
Expand All @@ -79,7 +79,7 @@ init_tinker_client()

from tinker import ServiceClient

base_model = 'Qwen/Qwen3.6-27B'
base_model = 'Qwen/Qwen3.8-27B'
base_url = 'https://www.modelscope.cn/twinkle'

# Step 2: Define the base model and connect to the server
Expand All @@ -92,7 +92,7 @@ service_client = ServiceClient(
# The model_path is a twinkle:// URI pointing to a previously saved LoRA checkpoint.
# The server will load the base model and apply the LoRA adapter weights.
sampling_client = service_client.create_sampling_client(
model_path='twinkle://xxx-Qwen_Qwen3.6-35B-A3B-xxx/weights/twinkle-lora-1',
model_path='twinkle://xxx-Qwen_Qwen3.8-27B-xxx/weights/twinkle-lora-1',
base_model=base_model
)

Expand Down
8 changes: 4 additions & 4 deletions docs/source_zh/使用指引/Qwen3.5最佳实践.md
Original file line number Diff line number Diff line change
Expand Up @@ -465,9 +465,9 @@ init_tinker_client()

from tinker import ServiceClient

# 基座模型
base_model = 'Qwen/Qwen3.5-4B'
base_url = 'http://www.modelscope.cn/twinkle'
# 基座模型(魔搭官方环境当前部署的模型)
base_model = 'Qwen/Qwen3.8-27B'
base_url = 'https://www.modelscope.cn/twinkle'


def train():
Expand Down Expand Up @@ -530,7 +530,7 @@ Twinkle 框架开源的同时,魔搭社区依托自身算力基础设施,提

```python
base_url = 'https://www.modelscope.cn/twinkle'
base_model = 'Qwen/Qwen3.5-4B' # 官方环境当前部署的模型
base_model = 'Qwen/Qwen3.8-27B' # 官方环境当前部署的模型
```

---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,5 +106,5 @@ python cookbook/client/twinkle/self_cognition.py
```bash
export TWINKLE_SERVER_URL=https://www.modelscope.cn/twinkle
export TWINKLE_SERVER_TOKEN="$MODELSCOPE_TOKEN"
export TWINKLE_MODEL_ID=Qwen/Qwen3.6-27B
export TWINKLE_MODEL_ID=Qwen/Qwen3.8-27B
```
10 changes: 5 additions & 5 deletions docs/source_zh/使用指引/训练服务.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
在 Twinkle 框架开源的同时,我们依托ModelScope的后台服务,也提供了托管的模型训练服务(Training as a Service),开发者可以通过这一服务,
免费体验Twinkle的训练API。

目前在集群中运行的模型是[Qwen/Qwen3.6-27B](https://www.modelscope.cn/models/Qwen/Qwen3.6-27B)。下面介绍具体的使用方法:
目前在集群中运行的模型是[Qwen/Qwen3.8-27B](https://www.modelscope.cn/models/Qwen/Qwen3.8-27B)。下面介绍具体的使用方法:

## Step 1. 注册ModelScope用户并获取 API Key

Expand Down Expand Up @@ -31,7 +31,7 @@ from twinkle.dataset import Dataset, DatasetMeta
from twinkle.preprocessor import SelfCognitionProcessor
from twinkle.server.common import input_feature_to_datum

base_model = 'ms://Qwen/Qwen3.6-27B'
base_model = 'ms://Qwen/Qwen3.8-27B'
base_url='https://www.modelscope.cn/twinkle'
api_key=os.environ.get('MODELSCOPE_TOKEN')

Expand Down Expand Up @@ -65,7 +65,7 @@ for epoch in range(2):
print(f'Saved checkpoint for epoch {epoch} to {result.path}')
```

通过上述代码,你可以训练一个原模型为`Qwen/Qwen3.6-27B`的自我认知lora。这个lora会改变模型的名称和制造者为训练时指定的名称。使用这个lora进行推理:
通过上述代码,你可以训练一个原模型为`Qwen/Qwen3.8-27B`的自我认知lora。这个lora会改变模型的名称和制造者为训练时指定的名称。使用这个lora进行推理:

```python
import os
Expand All @@ -80,7 +80,7 @@ init_tinker_client()

from tinker import ServiceClient

base_model = 'Qwen/Qwen3.6-27B'
base_model = 'Qwen/Qwen3.8-27B'
base_url = 'https://www.modelscope.cn/twinkle'

# Step 2: Define the base model and connect to the server
Expand All @@ -93,7 +93,7 @@ service_client = ServiceClient(
# The model_path is a twinkle:// URI pointing to a previously saved LoRA checkpoint.
# The server will load the base model and apply the LoRA adapter weights.
sampling_client = service_client.create_sampling_client(
model_path='twinkle://xxx-Qwen_Qwen3.6-35B-A3B-xxx/weights/twinkle-lora-1',
model_path='twinkle://xxx-Qwen_Qwen3.8-27B-xxx/weights/twinkle-lora-1',
base_model=base_model
)

Expand Down
Loading
Loading