AI is happening. The future is updating.llm-jp-3-1.8b
This repository provides large language models developed by the Research and Development Center for Large Language Models at the National Institute of Informatics. The tokenizer of this model is based on huggingface/tokenizers Unigram byte-fallback model. The vocabulary entries were converted from `llm-jp-tokenizer v3.0`. Please refer to README.md of `llm-jp-tokenizer` for details on the vocabulary construction procedure (the pure SentencePiece training does not reproduce our vocabulary). The models released here are in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.
This repository provides large language models developed by the Research and Development Center for Large Language Models at the National Institute of Informatics. The tokenizer of this model is based on huggingface/tokenizers Unigram byte-fallback model. The vocabulary entries were converted from `llm-jp-tokenizer v3.0`. Please refer to README.md of `llm-jp-tokenizer` for details on the vocabulary construction procedure (the pure SentencePiece training does not reproduce our vocabulary). The models released here are in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.