robeczech-base

The tokenizer in the initial release (a) contained a hole (51959 did not correspond to any token), and (b) mapped several tokens (unseen during training but required by the BBPE tokenizer) to the same ID as the `[UNK]` token (3). That sometimes caused problems, as in https://huggingface.co/ufal/robeczech-base/discussions/4. See https://huggingface.co/ufal/robeczech-base/discussions/4#64b8f6a7f1f8e6ea5860b314 for more information. In version 1.1, the tokenizer was modified by (a) removing the hole, (b) mapping all tokens to a unique ID. That also required increasing the vocabulary size and embeddings weights (by replicating the embedding of the `[UNK]` token). Without finetuning, version 1.1 and version 1.0 gives exactly the same embeddings on any input (apart from the pooler missing in v1.1), and the tokens in version 1.0 that mapped to a different ID than the `[UNK]` token map to the same ID in version 1.1. However, the sizes of the embeddings (and LM head weights and biases) are different, so the weights of the version 1.1 are not compatible with the configuration of version 1.0 and vice versa.

번역 준비 중입니다. 원문을 표시합니다.