Persistent rollout with straw#
straw provides filesystem-based queues and packed tensor storage for vime. It persists prompt tasks, partial rollouts, completed samples and training batches. Generation and training processes read and write payloads in parallel across machines; Ray carries control messages and references.
The supported shared-filesystem target is JuiceFS. All nodes must mount the storage pool at the same absolute path. Other network filesystems, including NFS, require separate verification of locking, visibility and durability.
Enable it#
The standard vime installation and Docker build include straw-queue. To
install it in an existing environment, run this on every rollout and training
node, using the same version throughout the job:
pip install 'straw-queue>=0.1.3'
Add these arguments to your training command:
--rollout-data-transport straw \
--rollout-data-dir /shared/juicefs/jobs/my-run/rollout_data \
--rollout-queue-run-id my-run \
--rollout-storage-profile juicefs \
--rollout-storage-declaration /shared/juicefs/deployment.json
The deployment declaration describes verified mount settings; it does not
configure the mount. It includes direct_mount: true, writeback: false,
open_cache: 0, readdir_cache: false, client_version and
durability_description. See straw’s
filesystem contract.
Use the default local profile for local POSIX development and testing.
If --rollout-data-dir is omitted, a fresh run uses <save>/rollout_data; without
--save, an explicit shared directory is required. Checkpoint recovery can infer
the pool from the saved checkpoint.
Storage and execution mode are independent. The defaults are Ray object-store
transport and synchronous rollout. For distributed fully async execution, also add:
--rollout-function-path vime.rollout.fully_async_rollout.generate_rollout_fully_async
Fully async runs one generation process per Ray node with CPU resources and shares the configured concurrency across them. Faster workers can supply a training batch without waiting for every worker. Producers pause admission during weight synchronization and persist in-flight results before resuming.
The default object-store transport works without straw installed. Selecting straw without the package fails at startup with the installation command.
Data flow and scheduling#
Workers request prompt groups. The data source reads the dataset and saves both the groups and its updated cursor to the queue.
Workers generate and score samples. Returned partial groups are persisted for continuation; completed, accepted groups become available for training.
The batch builder applies global reward/conversion hooks and accepts the result as a queue control task. It writes the converted batch once; training ranks receive indices and read their samples directly from that shared batch. After training finishes, vime advances the consumer cursor and drops the unfinished-batch reference. The model checkpoint separately controls how long recovery retains the batch.
Rollout and training data share the same storage pool. Large R3 and SC tensors can be reused across these stages without duplicating their bytes. Queue tasks use renewable leases to assign work to workers; unfinished work from a lost worker becomes available again after lease expiry.
Scheduling prioritizes completed groups → partial groups → fresh prompts. Within each stage, groups with older generated-token weight versions come first, with FIFO ordering for ties. Groups without a numeric version come last in that stage. Staleness is the current serving version minus the oldest generated-token version, so fresh samples have staleness 0. Scheduling does not automatically discard stale samples.
--buffer-filter-path is unsupported with straw. --buffer-sort-by-staleness
applies to the in-memory data source; straw uses the ordering above. Reward and
sample-selection hooks are available; distributed fully async does not support
--rollout-all-samples-process-path. See customization
for custom rollout functions and queue readers.
Packed storage and supported data#
Packing is enabled whenever straw transport is selected. Each writer appends
multiple samples and tensors to the same file (a pack), reducing small-file
metadata overhead on shared storage. vime uses straw’s default target size,
currently 1 GiB. Use --rollout-queue-segment-mib to override it in MiB.
The target size controls file rotation; readers can access each publication without waiting for the pack to fill. A publication larger than the target stays intact, and explicit sealing may leave smaller packs.
Immutable tensor references let rollouts, training batches, and checkpoints share data. Updates write new records while retained references continue to identify their original contents.
The sample codec supports nested lists, tuples and dictionaries; scalar and byte values; NumPy arrays; PyTorch tensors; PIL images; and Sample fields, including supported custom fields. R3 routes and SC top-k/ragged top-p data are stored as typed tensors. Unsupported Python objects and cyclic structures raise an error when published. Stored records and tensor reads are checksummed.
Use --use-rollout-routing-replay for R3 and --use-score-centering for SC.
With straw transport, their tensors are persisted with the sample group. R3
training reads only the rows assigned to the current CP/TP rank. Large replay
tensors stay lazy during batch conversion, although selected Sample metadata
and ordinary fields still occupy manager memory.
Option |
Default |
Purpose |
|---|---|---|
|
Unset; uses straw’s default (currently |
Override the target pack rotation size in MiB; explicit values must be positive. |
|
|
Bound concurrent serialization and filesystem I/O submissions |
|
|
Worker lease duration; active readers renew it |
Online GC#
Online GC is disabled by default. Enable it with --rollout-queue-online-gc.
vime reports when tasks and training batches are finished or discarded. straw
reclaims a sealed pack only after all tasks, readers, checkpoints and archives
have released their references. One live record keeps the whole pack alive;
active writers and journal history also consume space.
Retained checkpoints and archives protect their payloads even with GC enabled, so rollback does not require disabling GC. Retention must also be released when data is no longer needed; deleting an index file alone does not release its storage ownership. Do not manually delete pack files from an active pool. For offline removal, stop all coordinators, writers and readers first.
A GC failure is reported by subsequent queue operations and at shutdown. Disk exhaustion requires freeing unneeded retention or expanding storage; it does not reset the queue automatically. Live-pack and journal compaction are not implemented.
Recovery and checkpoints#
Save model and rollout state together through normal training checkpoints. They contain the dataset cursor, sample/group counters, pending and partial inputs, ready groups, and training progress. Payloads already in straw are referenced rather than copied into each checkpoint. Retaining a checkpoint keeps its referenced data available.
This section covers restarting with a new serving cluster. Stop the entire previous job first, including remote workers. The save-directory lock rejects concurrent coordinators, but does not stop orphaned readers. There is no automatic coordinator failover.
If only Megatron training failed and Ray and serving are still running, follow the fault-tolerance guide to resubmit training while retaining the serving cluster and queue controller.
Resume or select a step#
To resume the latest completed checkpoint, use the same directories:
--rollout-data-transport straw \
--load /shared/checkpoints/run \
--save /shared/checkpoints/run \
--save-interval 1
Whenever --save is set, Megatron requires a positive --save-interval.
The example saves after each rollout; choose the interval for your workload.
For example, load step 16 and resume training at step 17:
--load /shared/checkpoints/run \
--save /shared/checkpoints/run \
--ckpt-step 16
Recovery uses the same output directory and queue. It removes later Straw queue records, source/builder checkpoints, and debug archives matching the configured output path. Rollout starts again at step 17. Prefetched ready groups and partial responses saved in the checkpoint are regenerated from their inputs; their old outputs are discarded. Prompt identities and the dataset cursor are preserved, and old leases cannot submit results to the restored queue.
Checkpoints through step 16 remain available. Later steps can be saved again at the original paths. With GC enabled, cleanup releases storage ownership and removes packs that no retained checkpoint or archive references. Straw cleanup does not delete model weight files.
You can also edit latest_checkpointed_iteration.txt to select an earlier step,
or use --load /shared/checkpoints/run/rollout/committed_16.json. An explicit
--ckpt-step or commit file takes precedence. Automatic selection uses the
latest completed model-and-queue checkpoint, ignoring newer incomplete saves.
Keep the dataset, model/tokenizer configuration, Straw run ID and storage profile consistent with the checkpoint. Training recovery requires optimizer and training RNG state. Fully async workers restart with newly generated results.
Missing state and recovery limits#
Missing, corrupt or unsupported queue snapshots and missing payloads produce a
warning and start an empty queue. Recovery uses a readable dataset cursor from
the snapshot, then rollout/global_dataset_state_dict_<step>.pt, then offset 0
if neither cursor is usable. Each queue checkpoint also saves this independent
cursor file. The model still resumes from the selected step; outputs from other
steps are not reused. Invalid model checkpoints, missing required optimizer/RNG
state and configuration mismatches still raise errors.
Before any model checkpoint exists, restarting the original run with the same
--save and model/input configuration can recover persisted rollout work, but
only before the first training batch has been planned. After batch planning,
recovery requires a matching model/optimizer and rollout checkpoint.
Full queue recovery uses the original straw pool; copying a checkpoint directory alone does not copy its payloads. Unavailable payloads trigger the dataset-cursor fallback above. The recovery boundary is a completed rollout training batch, not an optimizer microstep. GPU KV caches and generation RNG state are not restored, so newly generated tokens and random hooks may differ.
When extending --num-rollout, use Megatron’s
--use-checkpoint-opt-param-scheduler if you want to retain the saved optimizer
schedule. Queue restoration does not override optimizer settings.
Debug archives and sample lookup#
Both .pt and .straw.json work with the debug save/load flags:
--save-debug-rollout-data '/shared/debug/rollout_{rollout_id}.straw.json'
# In a separate train-only job, without vLLM:
--load-debug-rollout-data '/shared/debug/rollout_{rollout_id}.straw.json'
Format |
Contents |
Storage requirement |
|---|---|---|
|
Sample data and materialized tensors |
Self-contained file |
|
Immutable index with sample/task keys |
Keep the referenced straw pool available |
An indexed archive retains its data independently of queue consumption and GC.
It shares existing tensors when saved with straw transport; otherwise it creates
a straw-data pool alongside the index. There is one index per rollout, with
samples packed together. Evaluation uses eval_<id> in place of the rollout ID.
Copying only the JSON index does not copy the payloads. For train-only replay
with straw transport automatically reuses the archive’s storage pool and run,
overriding --rollout-data-dir and --rollout-queue-run-id. Each replay creates
an isolated queue in that writable pool; the source queue is left untouched.
Full replay (without subsampling) reuses the archived Sample and tensor records
without republishing them.
All rollout archives in one replay must belong to the same pool and run.
--load-debug-rollout-data-subsample also applies to archives.
from vime.data.archive import RolloutArchive
from vime.observability.rollout_data_utils import load_debug_rollout_data
with RolloutArchive('/shared/debug/rollout_7.straw.json') as archive:
print(archive.keys()) # (sample key, optional task key), in archive order
samples = archive.load_samples(sample_key='sample:42')
group = archive.load_samples(task_key='prompt:21')
archive.export_pt('/shared/debug/rollout_7.pt')
# Convert a .pt dump to an indexed archive with an adjacent straw-data pool.
samples = load_debug_rollout_data('/shared/debug/rollout_7.pt', rollout_id=7)
RolloutArchive.save('/shared/debug/imported_7.straw.json', samples, rollout_id=7)
Keys select data as saved in that archive. Missing keys raise KeyError.
Lookups return lists because compact trajectories can share a sample index.
Samples without an index use position:<ordinal>; samples without queue
provenance have no task key.
Closing an archive closes its reader while retaining its data. After all readers
finish, open the archive and call archive.release() to release its retention.
Other checkpoints, queues and archives retain their own references. An exported
.pt remains readable after straw reclaims the payloads.