gena-lm-bigbird-base-t2t

Differences between GENA-LM (`gena-lm-bigbird-base-t2t`) and DNABERT: - BPE tokenization instead of k-mers; - input sequence size is about 36000 nucleotides (4096 BPE tokens) compared to 512 nucleotides of DNABERT; - pre-training on T2T vs. GRCh38.p13 human genome assembly. This repository also contains models that are finetuned on downstream tasks and models that are used in our GENA-Web web tool for genomic sequence annotation: - splice sites prediction (branch gena_web_spliceai) GENA-LM (`gena-lm-bigbird-base-t2t`) model is trained in a masked language model (MLM) fashion, following the methods proposed in the BigBird paper by masking 15% of tokens. Model config for `gena-lm-bigbird-base-t2t` is similar to the `google/bigbird-roberta-base`:

번역 준비 중입니다. 원문을 표시합니다.