サイト内検索
AI モデル 76
- TinyCLIP-ViT-39M-16-Text-19M-YFCC15M↗
wkcn
**TinyCLIP** is a novel **cross-modal distillation** method for large-scale language-image pre-trained models. The method introduces two core techniques: **affinity mimicking** and **weight inheritance**. This work unleashes the capacity of small CLIP models, fully leveraging large-scale models as well as pre-training data and striking the best trade-off between speed and accuracy. Our code is based on CLIP, OpenCLIP, CoFi and PyTorch. Thank contributors for their awesome contribution!
- PE-Core-L-14-336↗
timm
This is an OpenCLIP (image + text) remaped version of the the original Perception Encoder (PE) is a state-of-the-art encoder for image and video understanding trained via simple vision-language learning. It was introduced in "Perception Encoder: The best visual embeddings are not at the output of the network". **Model Overview**: Perception Encoder (PE) is a family of large-scale vision encoder models with state-of-the-art performance on a large variety of vision tasks. By using a robust contrastive pretraining recipe and finetuning on synthetically aligned videos, PE not only outperforms all existing models on classification and retrieval, but it also internally produces strong, general features that scale for downstream tasks. PE unlocks the ability for large-scale contrastive pretraining to transfer to downstream tasks with alignment tuning to capitalize on those general features.
- MobileCLIP-S2-OpenCLIP↗
apple
MobileCLIP was introduced in MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training (CVPR 2024), by Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Raviteja Vemulapalli, Oncel Tuzel. * Our smallest variant `MobileCLIP-S0` obtains similar zero-shot performance as OpenAI's ViT-B/16 model while being 4.8x faster and 2.8x smaller. * `MobileCLIP-S2` obtains better avg zero-shot performance than SigLIP's ViT-B/16 model while being 2.3x faster and 2.1x smaller, and trained with 3x less seen samples. * `MobileCLIP-B`(LT) attains zero-shot ImageNet performance of **77.2%** which is significantly better than recent works like DFN and SigLIP with similar architectures or even OpenAI's ViT-L/14@336.
- CLIP-convnext_xxlarge-laion2B-s34B-b82K-augreg-soup↗
laion
1. Model Details 2. Uses 3. Training Details 4. Evaluation 5. Acknowledgements 6. Citation A series of CLIP ConvNeXt-XXLarge (a custom `timm` ConvNeXt size) models trained on LAION-2B (english), a subset of LAION-5B, using OpenCLIP. The core training run was performed in pieces over a period of ~ 2 months. The global batch size for the core run was 81920. The last ~10% of training was re-done at a 95744 global batch size w/ higher LR and aug than original finish. The two were averaged together in a 'soup'. See more details in Training Details.
- vit_large_patch14_clip_224.openai↗
timm
The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within. This instance of the CLIP model is intended for loading in * `timm` (https://github.com/rwightman/pytorch-image-models) and * `OpenCLIP` (https://github.com/mlfoundations/open_clip) libraries. Please see https://huggingface.co/openai/clip-vit-large-patch14 for use in Hugging Face Transformers.
- DFN2B-CLIP-ViT-L-14-39B↗
apple
A CLIP (Contrastive Language-Image Pre-training) model trained on DFN-2B. Data Filtering Networks (DFNs) are small networks used to automatically filter large pools of uncurated data. This model was trained on 2B images that were filtered from a pool of 12.8B uncurated image-text pairs (12.8B image-text pairs from CommonPool-12.8B). This model has been converted to PyTorch from the original JAX checkpoints from Axlearn (https://github.com/apple/axlearn). These weights are directly usable in OpenCLIP (image + text).
- DFN5B-CLIP-ViT-H-14-378↗
apple
A CLIP (Contrastive Language-Image Pre-training) model trained on DFN-5B. Data Filtering Networks (DFNs) are small networks used to automatically filter large pools of uncurated data. This model was trained on 5B images that were filtered from a pool of 43B uncurated image-text pairs (12.8B image-text pairs from CommonPool-12.8B + 30B additional public image-text pairs). This model has been converted to PyTorch from the original JAX checkpoints from Axlearn (https://github.com/apple/axlearn). These weights are directly usable in OpenCLIP (image + text). - **Model Type:** Contrastive Image-Text, Zero-Shot Image Classification. - **Dataset:** DFN-5b - **Papers:** - Data Filtering Networks: https://arxiv.org/abs/2309.17425 - **Samples Seen:** 39B (224 x 224) + 5B (384 x 384) | dataset | metric | |:-----------------------|---------:| | ImageNet 1k | 0.84218 | | Caltech-101 | 0.954479 | | CIFAR-10 | 0.9879 | | CIFAR-100 | 0.9041 | | CLEVR Counts | 0.362467 | | CLEVR Distance | 0.206067 | | Country211 | 0.37673 | | Describable Textures | 0.71383 | | EuroSAT | 0.608333 | | FGVC Aircraft | 0.719938 | | Food-101 | 0.963129 | | GTSRB | 0.679018 | | ImageNet Sketch | 0.73338 | | ImageNet v2 | 0.7837 | | ImageNet-A | 0.7992 | | ImageNet-O | 0.3785 | | ImageNet-R | 0.937633 | | KITTI Vehicle Distance | 0.38256 | | MNIST | 0.8372 | | ObjectNet 1 | 0.796867 | | Oxford Flowers-102 | 0.896834 | | Oxford-IIIT Pet | 0.966841 | | Pascal VOC 2007 | 0.826255 | | PatchCamelyon | 0.695953 | | Rendered SST2 | 0.566722 | | RESISC45 | 0.755079 | | Stanford Cars | 0.959955 | | STL-10 | 0.991125 | | SUN397 | 0.772799 | | SVHN | 0.671251 | | Flickr | 0.8808 | | MSCOCO | 0.636889 | | WinoGAViL | 0.571813 | | iWildCam | 0.224911 | | Camelyon17 | 0.711536 | | FMoW | 0.209024 | | Dollar Street | 0.71729 | | GeoDE | 0.935699 | | **Average** | **0.709421** |
- ViT-SO400M-14-SigLIP↗
timm
This model has been converted to PyTorch from the original JAX checkpoints in Big Vision. These weights are usable in both OpenCLIP (image + text) and timm (image only). - **Model Type:** Contrastive Image-Text, Zero-Shot Image Classification. - **Original:** https://github.com/google-research/big_vision - **Dataset:** WebLI - **Papers:** - Sigmoid loss for language image pre-training: https://arxiv.org/abs/2303.15343
- CLIP-ViT-B-32-laion2B-s34B-b79K↗
laion
1. Model Details 2. Uses 3. Training Details 4. Evaluation 5. Acknowledgements 6. Citation 7. How To Get Started With the Model A CLIP ViT-B/32 model trained with the LAION-2B English subset of LAION-5B (https://laion.ai/blog/laion-5b/) using OpenCLIP (https://github.com/mlfoundations/open_clip). As per the original OpenAI CLIP model card, this model is intended as a research output for research communities. We hope that this model will enable researchers to better understand and explore zero-shot, arbitrary image classification. We also hope it can be used for interdisciplinary studies of the potential impact of such model.
- ViT-B-16-SigLIP-i18n-256↗
timm
This model has been converted to PyTorch from the original JAX checkpoints in Big Vision. These weights are usable in both OpenCLIP (image + text) and timm (image only). - **Model Type:** Contrastive Image-Text, Zero-Shot Image Classification. - **Original:** https://github.com/google-research/big_vision - **Dataset:** WebLI - **Papers:** - Sigmoid loss for language image pre-training: https://arxiv.org/abs/2303.15343