foxAIAI is happening. The future is updating.
English
中文English日本語한국어

Jev-alpha-26B-A4B

A standalone decision model derived from Google Gemma 4 26B A4B IT. It reads a state and typed questions, then exposes probabilities over the allowed options. This repository contains full merged BF16 weights. It does not require a LoRA adapter at inference. The model retains the upstream Gemma4ForConditionalGeneration architecture and tokenizer. Only text decision behavior is fine-tuned and evaluated; other modalities and general chat are not validated here. This is seed 42, KL weight 0.1, checkpoint 2100. It is not an average of three adapters. The development screen selects the KL weight. The final test does not select this seed. Training uses 8,400 questions, two epochs, LoRA rank 8, alpha 16, and dropout zero. The loss is option-token cross-entropy plus 0.1 times conditional reference-to-model KL. The reference top 64 tokens exclude the question's valid option tokens at the answer position. Base revision: `4d7ae4984b7db7de8f8457170b3f1a419ee76d52`. The 26B/A4B name follows the upstream size class; all base weights remain required in memory. These measurements reload the saved merged model with Transformers, without PEFT or an adapter. Equal-type final accuracy: **79.50%** on 1,100 questions. Each source contributes 100 final questions. The aggregate gives equal weight to noul, choice, and score. The merged model has its own fitted temperature: **1.736507**, from 300 separate calibration questions. BF16 merging changes numerical behavior. Use these release results instead of the earlier adapter or three-seed mean. Calibration error uses ten equal-width confidence bins. Lower Brier and calibration error are better.

A standalone decision model derived from Google Gemma 4 26B A4B IT. It reads a state and typed questions, then exposes probabilities over the allowed options. This repository contains full merged BF16 weights. It does not require a LoRA adapter at inference. The model retains the upstream Gemma4ForConditionalGeneration architecture and tokenizer. Only text decision behavior is fine-tuned and evaluated; other modalities and general chat are not validated here. This is seed 42, KL weight 0.1, checkpoint 2100. It is not an average of three adapters. The development screen selects the KL weight. The final test does not select this seed. Training uses 8,400 questions, two epochs, LoRA rank 8, alpha 16, and dropout zero. The loss is option-token cross-entropy plus 0.1 times conditional reference-to-model KL. The reference top 64 tokens exclude the question's valid option tokens at the answer position. Base revision: `4d7ae4984b7db7de8f8457170b3f1a419ee76d52`. The 26B/A4B name follows the upstream size class; all base weights remain required in memory. These measurements reload the saved merged model with Transformers, without PEFT or an adapter. Equal-type final accuracy: **79.50%** on 1,100 questions. Each source contributes 100 final questions. The aggregate gives equal weight to noul, choice, and score. The merged model has its own fitted temperature: **1.736507**, from 300 separate calibration questions. BF16 merging changes numerical behavior. Use these release results instead of the earlier adapter or three-seed mean. Calibration error uses ten equal-width confidence bins. Lower Brier and calibration error are better.

Sources

huggingface.co ↗