AI正在发生 未来正在更新caduceus-ps_seqlen-131k_d_model-256_n_layer-16
To use the pre-trained model for masked language modeling, use the following snippet: Alternatively, you can instantiate a model from scratch to train on your own data as follows: This is the Caduceus-PS model with hidden dimension 256 and 16 MambaDNA layers. This model is reverse complement (RC) equivariant and thus no RC data augmentation is required when training this model, either during pre-training or for downstream fine-tuning. Note that the model hidden state will be **twice** that of a non-RC equivariant counterpart. For downstream task training and inference, and to ensure RC **invariant** outputs at downstream time, one can either run the downstream model on the hidden state and its RC or one can take the hidden state and its RC and average them before passing to the downstream model. To RC the hidden states, one can use: `hidden_states.flip(dim=(-2, -1))` which will flip along the sequence lenght and channel dimensions.
译文尚未就绪，显示来源原文
To use the pre-trained model for masked language modeling, use the following snippet: Alternatively, you can instantiate a model from scratch to train on your own data as follows: This is the Caduceus-PS model with hidden dimension 256 and 16 MambaDNA layers. This model is reverse complement (RC) equivariant and thus no RC data augmentation is required when training this model, either during pre-training or for downstream fine-tuning. Note that the model hidden state will be **twice** that of a non-RC equivariant counterpart. For downstream task training and inference, and to ensure RC **invariant** outputs at downstream time, one can either run the downstream model on the hidden state and its RC or one can take the hidden state and its RC and average them before passing to the downstream model. To RC the hidden states, one can use: `hidden_states.flip(dim=(-2, -1))` which will flip along the sequence lenght and channel dimensions.