Running AV Reasoning Models Within Onboard Compute and Latency Limits

AI-Generated Summary NVIDIA Alpamayo 2 Edge is a 2B-parameter adaptive reasoning vision-language-action model designed for real-time operation on NVIDIA DRIVE AGX Thor. The model generates Chain-of-Causation reasoning only when needed and uses a diffusion action expert to produce driving trajectories from surround-camera observations and navigation inputs. Closed-loop reinforcement learning in AlpaGym improves driving performance by 48.9% over the best checkpoint without closed-loop RL. AlpaJudge provides human-calibrated evaluation of reasoning quality through reference-based and reference-free variants for different evidence types. Three workflows connect the edge model with repeatable improvement: running AlpaJudge on PhysicalAI-AV samples, selecting higher-value training examples, and using reasoning scores as rewards for post-training. Deployment recipes in the Alpamayo Developer Hub enable inference optimization with TensorRT and DFlash2, achieving 160-190 ms pixel-to-trajectory latency on DRIVE AGX Thor. Next Steps Explore the Alpamayo 2 Edge model on Hugging Face to evaluate its capabilities. Explore the AlpaJudge model on Hugging Face for reasoning evaluation. Visit the Alpamayo developer forum for discussions and developer questions. Autonomous vehicle (AV) developers need reasoning models that can operate within the compute and latency constraints of an onboard system. Larger models provide a foundation for developing these capabilities, but moving them into a vehicle requires additional work to reduce model size, preserve useful reasoning, and validate performance on the target hardware. NVIDIA Alpamayo 2 Edge addresses this need with an adaptive reasoning vision-language-action (VLA) model designed for real-time operation on NVIDIA DRIVE AGX Thor. Built on the 2B Nemotron-based reasoning backbone of NVIDIA Cosmos 3 Edge and paired with a diffusion action expert, it leverages surround-camera observations, temporal context, and navigation inputs to generate Chain-of-Causation (CoC) reasoning and future driving trajectories. It reasons adaptively, generating reasoning only when needed, reducing unnecessary token generation and conserving inference compute during routine driving. Alpamayo 2 Edge is post-trained with closed-loop reinforcement learning (RL) in NVIDIA AlpaGym, improving closed-loop driving performance by 48.9% on top of the best-performing checkpoint without closed-loop RL. Together, these capabilities give developers a strong, edge-ready foundation for bringing reasoning-based driving into an onboard stack. Making a reasoning model compact enough for onboard deployment introduces a key challenge: preserving the quality of its reasoning and driving behavior as model size is reduced. This requires a way to measure reasoning quality and use that feedback to improve the model. NVIDIA AlpaJudge is the human-calibrated evaluation and reward model used in Alpamayo 2 Edge development to evaluate reasoning traces, select higher-value training data, and provide RL rewards. This post introduces Alpamayo 2 Edge, explains its inference optimization on DRIVE AGX Thor, and explores three ways to use AlpaJudge in AV development workflows: Run AlpaJudge-RB and AlpaJudge-RF on a PhysicalAI-AV sample, inspect their scores, and adapt the evaluation to policy-generated reasoning Select training examples that expose useful learning opportunities Use human-aligned reasoning scores as rewards for post-training Together, these workflows connect an edge-oriented driving model with a repeatable process for measuring and improving it. Bring reasoning-based driving to the edge An onboard driving model must interpret a changing scene and produce a useful trajectory within the system’s inference budget. The key question is how much driving and reasoning capability a model retains under that constraint. To balance model capacity and efficiency, Alpamayo 2 Edge leverages multiple new features: A 2B-parameter Nemotron-based reasoning backbone, sized for onboard inference Adaptive reasoning, reducing unnecessary token generation and conserving inference compute during routine driving Offline distillation, training with CoC reasoning autolabels generated by Alpamayo 2 Super Closed-loop reinforcement learning post-training in AlpaGym, in which the model learns from the consequences of its actions end-to-end in simulation Multi-resolution inputs to efficiently encode surround-view observations online Speculative decoding with DFlash2 to accelerate online reasoning without sacrificing quality Recipes for quantization and inference acceleration with NVIDIA TensorRT and NVIDIA TensorRT-Edge-LLM Figure 2, below, shows how these choices translate into capability, comparing Alpamayo 2 Edge with Qwen-Drive-1.0 across closed-loop driving, open-loop trajectory prediction, and scene understanding and reasoning. Early partner feedback also highlights Alpamayo 2 Edge’s driving performance and inference speed: “Within days of getting early access to Alpamayo 2 Edge, we had it running closed-loop in NuRec reconstructions of our own London street recordings. In our early simulation tests, Edge kept its lane in more than twice as many runs as Alpamayo 1.5, recovered from lane offsets about 40% more often, and out of the box planned roughly twice as fast as our hand-tuned Alpamayo 1.5 server on the same GPU.” —Metas Simonavicius, CTO, PORT “Alpamayo 2 Edge demonstrated impressive generalization capabilities, reducing displacement error (minADE) by 13.6% and lateral prediction error (RMSE) at 3 seconds by 46.1% compared to Alpamayo 2 Super in our zero-shot evaluation, despite substantial differences in sensor configurations. Fine-tuning on our own driving data delivered a further 44.2% reduction in displacement error compared to the Alpamayo 2 Edge baseline. Within just a few days, we went from receiving the model to running our first tests on a real vehicle.” — Luca Crotti, co-founder and motion planning lead, Niulinx “In less than 48 hours, we made NVIDIA’s Alpamayo 2 Edge run at 10 Hz on AGX Thor for real-time planning.”— Hans Salomonsson, CEO and co-founder, Embedl The Alpamayo 2 Edge repository’s inference script shows how the model operates: It takes in surround-view camera observations over time, navigation text, and historical ego-motion to produce CoC reasoning traces and future trajectories. import torch import transformers from alpamayo2_edge.checkpoint import load_model from alpamayo2_edge.load_physical_aiavdataset import load_physical_aiavdataset from alpamayo2_edge.helper import prepare_model_inputs, to_device from alpamayo2_edge.models.utils import fuse_traj_tokens data = load_physical_aiavdataset(**clip) model, loading_info = load_model("nvidia/Alpamayo2-Edge") settings = dict( seed=43, top_p=0.98, top_k=None, temperature=0.6, num_traj_sets=1, num_traj_samples=1, max_generation_length=256, diffusion_kwargs={"inference_step": 10}, ) prepared = prepare_model_inputs(data, model.config, model.tokenizer) prepared = to_device(prepared, device="cuda") """Seed text and diffusion sampling, then run the model on prepared inputs.""" torch.manual_seed(settings["seed"]) generator = torch.Generator(device="cuda").manual_seed(settings["seed"]) with ( torch.inference_mode(), torch.autocast( device_type=torch.device("cuda").type, dtype=model.dtype, enabled=model.dtype != torch.float32, ), ): xyz, rot, _, extra = model.sample_trajectories_from_data( prepared, num_traj_sets=settings["num_traj_sets"], num_traj_samples=settings["num_traj_samples"], top_p=settings["top_p"], top_k=settings["top_k"], temperature=settings["temperature"], max_generation_length=settings["max_generation_length"], diffusion_kwargs={**settings["diffusion_kwargs"], "generator": generator}, return_extra=True, ) In particular, note that the model will now produce special tokens instead of reasoning traces sometimes, indicating routine driving and conserving inference compute. Retaining quality during deployment When downloading the model from Hugging Face, by default, Alpamayo 2 Edge’s weights are represented in BF16 and it is directly runnable as a PyTorch model on any supported NVIDIA GPU. To run at real-time on DRIVE AGX Thor, however, Alpamayo 2 Edge must be further inference-optimized. The Alpamayo Developer Hub includes the recipe for the DRIVE AGX Thor deployment workflow: stage the Alpamayo 2 Edge checkpoint and DFlash2 drafter, calibrate FP8 scales, export ONNX graphs, build TensorRT engines with TensorRT Edge-LLM plugins, and run the Python harness. The recipe’s reference configuration was measured on DRIVE AGX Thor-X with DRIVE OS 7.2.7.0, CUDA 13.4, and TensorRT 11.3.1.9. After following the Alpamayo 2 Edge deployment recipe to stage the model, evaluation inputs, and engines, run the bundle with python -m alpamayo2edge_edgellm.runtime.run \ --engines "$WORK/engines" \ --model "$MODEL" \ --reference "$INPUTS" \ --dflash2 \ --rolling \ --warmup 1 \ --repeat 5 \ --output "$WORK/results.json" The recipe documents export, build, calibration provenance, board requirements, and per-stage measurements so developers can reproduce the latency and paired-quality check on their DRIVE AGX Thor configuration. For reference, the resulting optimized model has a pixel-to-trajectory inference latency of 160 ms to 190 ms depending on the amount of reasoning. The same recipe check compares optimized model outputs with a BF16 PyTorch reference built from identical inputs and seeded action noise. Evaluate reasoning with AlpaJudge Trajectory metrics describe how a model plans or drives. Reasoning evaluation adds another dimension: whether the model’s explanation identifies the relevant scene context and supports the decision it describes. AlpaJudge provides the human-calibrated critic for this aspect of Alpamayo 2 Edge development (as well as prior models). The following workflows describe how to conduct that evaluation and use it for model improvement. Choose the judge for your available inputs AlpaJudge provides two checkpoint variants. Choose the variant according to the evidence available for the example you want to evaluate. RB measures agreement with the supplied reference, rather than independently checking the scene. RF does not require a reference rationale, but it does require synchronized visual evidence and formatted meta-action context. Workflow 1: Run AlpaJudge on a PhysicalAI-AV sample The example notebook provides a useful starting point, visualizing the data, meta-action context, candidate rationale, and reference rationale tied to a PhysicalAI-AV sample. The same record can then be rescored with a policy-generated CoC trace. Preserve clip ID, timestamp, camera names, and action context in the output so a low score can be traced back to the scene. In open-loop evaluation, the candidate explanation and trajectory can be compared with recorded annotations. In closed-loop evaluation, the explanation at each policy decision can be scored using the observation and action context from that rollout. A high reasoning score is evidence that a rationale matches the judge’s rubric, while collision, road-departure, close-encounter, and trajectory measures describe different aspects of driving behavior. Report them together. Step 1: Set up the package and open the notebook From the AlpaJudge repository root, install the locked environment, authenticate for the model and PhysicalAI-AV dataset, and open the example notebook: uv sync --locked --dev hf auth login uv run jupyter lab examples/alpajudge_pai.ipynb The notebook uses clip 030c760c-ae38-49aa-9ad8-f5650a545d26 at 5.1 s. It decodes front-wide and front-tele frames at −1.0, −0.5, and 0.0 s, oldest first. The example rationale and meta-action strings are manually authored; the dataset supplies the camera frames. Replace all text fields together when using a different moment. Step 2: Download the two judge checkpoints The notebook downloads nvidia/alpajudge once and selects the reference-based/ and reference-free/ subfolders. These are separate checkpoints with separate loaders. Its snapshot call filters to those two folders; set ALPAJUDGE_MODEL_REVISION to an immutable revision for a reproducible run. A model load needs a CUDA GPU and roughly 5 GB for weights before activations. checkpoint_root = snapshot_download( repo_id="nvidia/alpajudge", revision=MODEL_REVISION, allow_patterns=["reference-based/*", "reference-free/*"], ) Step 3: Score the sample and inspect both outputs Run the notebook’s fixed construction-zone sample first. With its recorded checkpoint and inputs, the smoke test reports RB alignment 0.999 and RF overall 0.842. The five RF subscores assess action/reference agreement, action safety, component grounding, component completeness, and causal consistency. The RF model defines action = p_match × (1 + p_safe)/2 and component = mean(p_grounded, p_complete, p_causal), then combines them overall = 0.6 × action + 0.4 × component. Core notebook calls: rb_model, rb_tokenizer = load_reference_based( str(checkpoint_root), subfolder="reference-based", device="cuda", ) rb = score_reference_based( rb_model, rb_tokenizer, [{ "question": QUESTION, "answer": REFERENCE_RATIONALE, "student": CANDIDATE_RATIONALE, }], )[0].item() rf_model, rf_processor = load_reference_free( str(checkpoint_root), subfolder="reference-free", device="cuda", ) rf = score_reference_free( rf_model, rf_processor, front_wide_frames=front_wide_frames, front_tele_frames=front_tele_frames, meta_action_history=META_ACTION_HISTORY, meta_action_future=META_ACTION_FUTURE, rationale=CANDIDATE_RATIONALE, ) print(rb, rf.overall, rf.subscores) To evaluate Alpamayo 2 Edge itself, run its inference script on the selected clip and inspect the generated rationale in output.json under text.cot. Use that same clip and decision time when loading AlpaJudge-RF’s six frames, and replace the notebook’s hand-authored rationale with the generated string. Provide meta-action history and future that match this decision rather than copying the notebook’s construction-scene example. If an expert reference CoC trace is available, pass it to RB as answer and the Edge CoC as student. Workflow 2: Select higher-value training examples AlpaJudge’s score is useful for training data selection because the highest and lowest scoring examples often serve different purposes. A very low reference-based score can indicate a mismatched label, a failed rollout, or a genuinely difficult case. Review these cases before discarding them; a blanket removal of low-scoring clips would also remove rare failures the policy needs to learn. Very high scores can identify scenes the current model already explains consistently and is not learning much from anymore. The middle range often contains data that is highly informative for further training. Applying this to your own dataset begins with candidate CoC rollouts per clip and a fixed AlpaJudge checkpoint. Score each candidate against the paired reference (if using AlpaJudge-RB), record the score, clip ID, and any scenario metadata, then review low-scoring and safety-critical examples manually. Set selection thresholds on a development split, stratify the retained set by scene type and maneuver, and keep the held-out evaluation split untouched. Resample informative clips for further training and compare the new checkpoint with the same baseline set. Workflow 3: Use reasoning scores for post-training The public Alpamayo RL recipe offers a concrete pattern for adding a reasoning reward to open-loop GRPO training. It generates groups of policy rollouts, decodes each CoC and trajectory, computes trajectory error and comfort, then calls a pluggable reasoning grader before combining the terms into one scalar reward. AlpaJudge-RB can be that grader when each sample has a reference reasoning trace: pass the fixed driving question, reference as answer, and sampled CoC as student, then use the returned alignment probability as the reasoning signal. An adapter can implement the recipe’s BaseReasoningGrader.score(predictions, references) -> Tensor[N] contract by batching the text pairs through score_reference_based. Register that adapter in GRADER_REGISTRY and point the reasoning reward config to the corresponding checkpoint. The reward function should keep its trajectory gate and log both component scores, so the policy is less likely to earn a high total reward from a fluent but wrong driving explanation. Get started with Alpamayo 2 Edge and AlpaJudge These workflows form a repeatable development loop: run the edge model, score its reasoning next to trajectory and safety metrics, select informative clips, and compare the next candidate against the initial baseline. For teams that also train policies with RL, AlpaJudge can provide a reasoning reward through an adapter such as the one described above. Explore Alpamayo 2 Edge and AlpaJudge on Hugging Face. For discussions and developer questions, visit the Alpamayo developer forum. Check out the following for more information on: Alpamayo Open Platform and its latest releases. State-of-the-art research informing Alpamayo’s development. About the Authors Comments