CM3leon by Meta
UnknownMeta research foundation model for efficient multimodal text-to-image and image-to-text generation.
Tool Information
What is CM3leon by Meta?
CM3leon by Meta is a single causal masked mixed-modal foundation model that performs both text-to-image and image-to-text generation. It is trained with a retrieval-augmented pre-training and multitask supervised fine-tuning recipe adapted from text-only language models, enabling generation of sequences of text and images conditioned on arbitrary prior image and text content.
Core Features
Text-to-image generation
Generates coherent images from text prompts, including highly compositional structures and complex objects, achieving state-of-the-art FID of 4.88 on zero-shot MS-COCO.
Image-to-text generation
Produces text outputs such as captions and answers from image inputs as part of the same multimodal model.
Text-guided image editing
Edits images according to textual instructions while understanding both text and visual content, such as changing sky color.
Visual question answering
Answers questions about image content through multitask instruction tuning.
Image caption generation and long-form captioning
Generates image captions and long-form descriptions, with zero-shot performance competitive against larger models.
Conditional and multitask image generation
Supports conditional image generation and super-resolution within a single model via large-scale multitask supervised fine-tuning.
Retrieval-augmented pre-training and efficient training
Uses large-scale retrieval-augmented pre-training and multitask SFT; trained with five times less compute than previous transformer-based methods while remaining inference-efficient.
Use Cases
- #1
Complex compositional image synthesis
Creating detailed images from multi-constraint prompts, such as a potted cactus with sunglasses and a hat or a raccoon anime character preparing for battle.
- #2
Text-based image editing
Modifying existing images via natural language instructions while preserving global shapes and local details.
- #3
Vision-language understanding tasks
Performing image captioning, long-form captioning, and visual question answering with a single model.
- #4
Multimodal sequence generation
Generating mixed sequences of text and images conditioned on arbitrary prior image and text content.
Pricing Plans
Research model announcement only; no pricing, commercial product, public access, API, or licensing details provided.
Alternatives
Frequently Asked Questions
What is CM3leon by Meta?
What features does CM3leon offer?
Is CM3leon available as a public product or API?
Who is the target audience for CM3leon?
How efficient is CM3leon’s training?
What pricing information is available for CM3leon?
CM3leon by Meta Contact & Social Media
- CompanyMeta
- Website:
Reviews
Share your experience with this tool...