AI is happening. The future is updating.StableBeluga2
This repository contains the model from the stabilityai/StableBeluga2 repository with the following changes: 1. **Storing weights in `bfloat16` instead of `float32`.** This leads to 2x smaller files and a small quality loss, which is not significant compared to the loss caused by NF4 quantization used in Petals by default. 1. **Storing weights in small shards.** Each transformer block is stored in its own shard (1.71 GB each). The input and output embeddings and adjacent layernorms are in a separate shard (1.05 GB) too. This way, Petals clients and servers don't have to download any excess data besides the layers they actually use. 1. **Using Safetensors instead of Pickle.** This allows faster loading with smaller RAM requirements. We provide the original README below. Please refer there for model details and licensing information.
This repository contains the model from the stabilityai/StableBeluga2 repository with the following changes: 1. **Storing weights in `bfloat16` instead of `float32`.** This leads to 2x smaller files and a small quality loss, which is not significant compared to the loss caused by NF4 quantization used in Petals by default. 1. **Storing weights in small shards.** Each transformer block is stored in its own shard (1.71 GB each). The input and output embeddings and adjacent layernorms are in a separate shard (1.05 GB) too. This way, Petals clients and servers don't have to download any excess data besides the layers they actually use. 1. **Using Safetensors instead of Pickle.** This allows faster loading with smaller RAM requirements. We provide the original README below. Please refer there for model details and licensing information.