AI is happening. The future is updating.caduceus-ph_seqlen-131k_d_model-256_n_layer-16
This is the Caduceus-Ph model with hidden dimension 256 and 16 MambaDNA layers. This model is not inherently reverse complement (RC) equivariant. Rather, it was pre-trained using RC data augmentation. Its intended usage is as follows: for downstream tasks, the model should be trained with RC data augmentation. At downstream task inference, the model should be run twice: once on a sequence and once on its RC. The output of these two applications should be combined (averaged) to form the downstream task prediction. This model was pre-trained on the human reference genome with sequence length 131,072 for 50k steps (each step contained ~1M base pairs / tokens). For more details, please see our paper: Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling.
This is the Caduceus-Ph model with hidden dimension 256 and 16 MambaDNA layers. This model is not inherently reverse complement (RC) equivariant. Rather, it was pre-trained using RC data augmentation. Its intended usage is as follows: for downstream tasks, the model should be trained with RC data augmentation. At downstream task inference, the model should be run twice: once on a sequence and once on its RC. The output of these two applications should be combined (averaged) to form the downstream task prediction. This model was pre-trained on the human reference genome with sequence length 131,072 for 50k steps (each step contained ~1M base pairs / tokens). For more details, please see our paper: Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling.