How CLIP Smart Tagging Works
One of the most exciting features in MeshSeen is AI Smart Tagging — the ability to automatically categorize your 3D models without manual work. Here’s how it works under the hood.
What is CLIP?
CLIP (Contrastive Language-Image Pre-training) is an AI model developed by OpenAI that understands the relationship between images and text. Unlike traditional image classifiers that can only recognize a fixed set of categories, CLIP can match images to any text description.
Our Pipeline
When you index a new model, MeshSeen runs this pipeline:
- Render — We generate a thumbnail image of the 3D model from multiple angles
- Embed — The CLIP model creates a vector embedding of the image
- Match — We compare the embedding against our category descriptions
- Tag — The top matching categories are assigned as tags
Privacy First
The entire process runs locally on your machine. No data is ever sent to external servers. The CLIP model is bundled with MeshSeen and runs on your CPU (or GPU if available).
Performance
On a modern MacBook Pro, tagging takes approximately:
- 50ms per model on Apple Silicon
- 200ms per model on Intel
- 25ms per model with GPU acceleration
What’s Next
We’re working on letting you define custom categories for specialized collections. Imagine automatically tagging your Warhammer miniatures by faction, or sorting mechanical parts by function.
Stay tuned for updates!