pythia-1b

モデル紹介

The *Pythia Scaling Suite* is a collection of models developed to facilitate interpretability research (see paper). It contains two sets of eight models of sizes 70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, and 12B. For each size, there are two models: one trained on the Pile, and one trained on the Pile after the dataset has been globally deduplicated. All 8 model sizes are trained on the exact same data, in the exact same order. We also provide 154 intermediate checkpoints per model, hosted on Hugging Face as branches. The Pythia model suite was deliberately designed to promote scientific research on large language models, especially interpretability research. Despite not centering downstream performance as a design goal, we find the models match or exceed the performance of similar and same-sized models, such as those in the OPT and GPT-Neo suites. Previously, we released an early version of the Pythia suite to the public. However, we decided to retrain the model suite to address a few hyperparameter discrepancies. This model card lists the changes ; see appendix B in the Pythia paper for further discussion. We found no difference in benchmark performance between the two Pythia versions. The old models are still available, but we suggest the retrained suite if you are just starting to use Pythia. **This is the current release.**

主な特徴

特点摘要待依据来源补齐，不凭模型名称推断能力。

制約と注意点

具体的な制約を判断する出典が不足しています。実際の性能、遅延、データ方針、ライセンス、提供期間を確認してください。

出典別料金と費用分析

料金はチャネル・地域・設定によって異なります。各チャネルの料金は個別に表示し、提供元の統一公式料金とは見なしません。

公開評価と実行設定

暂无已采集的可核验公开评分。模型未上榜不代表能力较低。

よくある質問

出典と追跡記録

许可证：apache-2.0。模型介绍与第三方评测均以原来源的具体版本和方法为准。