MiniLM-L12-H384-uncased

MiniLM is a distilled model from the paper "MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers". Please find the information about preprocessing, training and full details of the MiniLM in the original MiniLM repository. Please note: This checkpoint can be an inplace substitution for BERT and it needs to be fine-tuned before use!