blt-1b-hf

**BLT (Byte Latent Transformer)** is a tokenizer-free transformer architecture that operates directly on raw byte sequences. Instead of processing text token by token, BLT dynamically groups bytes into **entropy-based patches**, enabling more efficient and scalable processing for byte-level tasks. Key components: - **Local Encoder → Latent Transformer → Local Decoder** architecture. - **Entropy-based patcher (BltPatcher)**: scans byte streams and creates patches when entropy thresholds are met. - **Hash n-gram embeddings**: maintain contextual information over neighboring bytes. BLT achieves competitive performance compared to traditional token-based transformers, supporting multilingual, noisy, or mixed-script input.

번역 준비 중입니다. 원문을 표시합니다.