Amazon is entering the AI foundation model game with its new family of multimodal models: Amazon Nova. Obviously, this isn’t the first time Amazon (AWS in particular) has invested in artificial intelligence. For years, the company has offered companies access to various generative AI solutions, intelligent analytics tools, bots, and beyond.
But the launch of Amazon Nova marks a new avenue for Amazon’s AI journey. Just as Microsoft has begun developing its own proprietary AI models and Google has the Gemini family, Amazon will now offer its own suite of foundation models.
Announced at AWS Re: Invent 2024, these models are designed to deliver incredible generative AI capabilities, catering to countless applications, from multimodal reasoning to text generation.
Here’s everything you need to know about the Amazon Nova portfolio.
What is Amazon Nova? New Foundation Models
Amazon Nova is a new collection of innovative AI foundation models promising industry-leading price performance and cutting-edge capabilities. Available exclusively through Amazon Bedrock, these models are powered by Amazon’s own infrastructure and proprietary Trainium and Inferentia chips, which Amazon says gives them exceptional scalability and efficiency.
Aiming to outpace competitors like OpenAI GPT-4o, Meta’s Llama, and of course, Google Gemini, Amazon’s Nova models promise faster, more cost-effective performance than their competitors.
According to Amazon, companies can use Nova models to lower costs and latency for virtually any generative AI task. The models can be used to build AI agents, generate video content, analyze documents and charts, and more.
Notably, just like Google’s Gemini models, these solutions are designed to be multimodal from the ground up. That means they can understand image, text, and video inputs. However, there isn’t much information about whether they’ll support audio input yet.
The Amazon Nova Models: Pro, Lite, Micro and Premium
Amazon’s model strategy seems to have taken a similar approach to Google (and a few other AI businesses). Instead of just building one super-effective model, Amazon is giving developers access to a range of different models, all with their own unique capabilities.
While each of these models promises excellent speed and cost efficiency, as well as support for agentic abilities and UI actuation, they target specific use cases. For instance:
Amazon Nova Micro
Arguably the smallest Amazon Nova model, Micro is a text-only model that promises the lowest latency responses in the family of tools. It has a context length of 128k tokens. It’s also one of the most affordable, speedy models available. This model is potentially intended for smaller companies that want to tap into AI development opportunities on a limited budget.
Micro excels at tasks like translation, text summarization, interactive chat, content classification, and brainstorming. It can also handle simple coding and mathematical reasoning and supports customization with proprietary data fine-tuning.
Amazon Nova Lite
Another ultra-low-cost model in the Amazon Nova family, Lite is the company’s smallest multimodal solution. This model delivers lightning-fast image, video, and text processing capabilities and can generate text output at speed. According to the tech giant, Lite can handle customer interactions in real-time, visual question answering, and document analysis.
It can process inputs of up to 300k tokens, and can even analyze multiple images at once, or up to thirty minutes of video in a single request. Lite also supports multimodal and text fine-tuning, as well as model distillation for customization.
Amazon Nova Pro
Currently the most capable Amazon Nova model (for now), Pro combines accuracy, speed, and low-cost performance for various tasks. It can handle up to 300k input tokens and sets new standards for agentic workflows and multimodal intelligence.
This model has achieved state-of-the-art performance on key benchmarks such as visual question answering and video understanding. Plus, Amazon says the Pro model excels at analyzing financial documents and can process code bases with more than 15,000 lines of code. On top of that, Pro serves as a “teacher” model for custom versions of Lite and Micro.
Amazon Nova Premier
Set to be released in 2025, Amazon Nova Premier is the ultimate next-generation model from AWS and will replace Pro as the most capable model in the family. Amazon hasn’t revealed much information about what this solution will be able to do, as it’s still in training.
However, it should introduce multimodal interpretation and a hefty data library, which organizations can use to train other models.
The Features of the Amazon Nova Model Line-Up
So, what makes the Amazon Nova models so special? Amazon says they excel in Retrieval-Augmented Generation (RAG), agentic applications, and function calling. The model’s performance in various tests, such as the Comprehensive RAG Benchmark (CRAG) evaluation, Mind2Web, and VisualWebBench, verifies these claims.
Beyond that, Amazon’s large language models are highly customizable. You basically start with a foundation model, and then fine-tune them with text, images, and videos, adjusting them to suit the specific needs of your business or industry.
Some of the key features of the Amazon Nova Models include:
Multi-Lingual, Multimodal Support, with Long Context Windows
Amazon says that Nova models are designed to speak in your language. Unlike many of their competitors, all three initial models support over 200 languages. Plus, even the smallest model (Micro) supports up to 128k input tokens, while the more advanced options support up to 300k tokens.
Microsoft is planning to extend these context windows even further, potentially offering models with a two million input token limit. These ultra-long context windows will allow companies to use the systems for a wide range of different use cases, from analyzing corporate documentation to creating complex workflows.
Fast, Cost-Effective and Customizable Performance
Amazon seems to focus on making generative AI development more accessible to the masses with the Nova collection. All of the models are extremely fast and cost-effective. In fact, the three initial models are at least 75% less expensive to run than the next best-performing models in their respective classes across Amazon Bedrock. All of the models work seamlessly with a user’s data and systems and support a wide selection of tasks.
Beyond that, Amazon Nova models support custom fine-tuning. You can adjust the models with your data to boost their performance. There’s even the option to implement “distillation”, which enables the transfer of specific knowledge from a larger teacher model (Nova Pro) to a smaller, cheaper model that’s easier to run.
Seamless Amazon Bedrock Integration
Another standout feature of the Amazon Nova models is they’re fully connected with Amazon Bedrock. This is the fully managed service that makes high-performing foundational models available to companies through a single API.
This integration makes deploying and scaling models incredibly simple. Users can tap into knowledge bases on Bedrock to enhance models with unique information. They can also create Amazon Bedrock Agents for complex workflows. Plus, dedicated Guardrails are available to promote and support responsible AI usage. The platform even supports real-time streaming for interactive applications, detailed monitoring, and batch processing.
Excellent Accuracy and Versatility
As mentioned above, Amazon Nova models are linked to the knowledge bases on Amazon Bedrock and are fantastic for Retrieval Augmented Generation. This means they deliver exceptional accuracy by grounding responses in your own data. If you’re worried about issues like AI hallucinations, that could make the Nova models a great pick.
They’re also specifically optimized for agentic AI. This means you can create models capable of completing entire workflows rather than single tasks.
The Amazon Nova Content Creation Models
Beyond the base foundation models offered in the Nova collection, Amazon has released two initial “content generation” models. These are specifically designed for generating visual content at scale.
Amazon Nova Canvas, for instance, is the image-generation model designed to produce studio-quality images, giving users exceptional control over content and style. It’s similar to tools like Midjourney and supports rich editing features like outpainting, inpainting, and background removal. This model has already excelled in various evaluations and benchmarks. For instance, it scored excellently in the “TIFA” evaluation.
Alternatively, Amazon Nova Reel is a video generation model that allows users to produce short videos with images and text prompts. You can control visual styles and pacing and customize all kinds of videos for entertainment, marketing, and advertising.
How Companies are Using Amazon Nova Models
Though the Nova model collection is still brand new, many major companies have already begun experimenting with it. SAP, for instance, is integrating the Nova models into its AI Core generative AI hub of supported LLMs. This enables developers to develop new skills for SAP’s AI copilot and Joule and design context-driven apps.
Deloitte will leverage the Nova models to create innovative new solutions for businesses across every sector. Dentsu Digital is integrating the “Reel” model into its creative process. It hopes to help its marketing teams accelerate the creation of visual campaigns. Similarly, MusixMatch, the world’s largest lyrics platform, is including Nova Reel in the MusixMatch Pro service to give creators access to a tool for developing high-quality music videos.
Elsewhere, 123RF uses Amazon Nova Canvas and Reel to simplify its design process with easy-to-use tools for creators. Caylent, the cloud services company, is embedding Amazon Nova models into its system to bring video understanding capabilities to customers across industries.
Even Shutterstock has shared that it’s going to include Amazon’s Canvas model into its AI image generator tool. Plus, Palantir Technologies is using Nova Pro in the Ontology System within its platform to enhance decision-making workflows for companies across 40+ industries.
Is Amazon Nova Safe? Guardrails and Ethics
As companies continue to prioritize safe, explainable, and ethical AI, Amazon has noticed that all its models are built with a focus on customer safety, trust, and security. The model collection comes with built-in safety measures and content moderation capabilities.
In particular, AWS says that its guardrails will help to combat the spread of misinformation, child abuse content, and other dangerous content. However, it hasn’t been particularly transparent about how the guardrails will achieve those goals.
Notably, though, Amazon does say that the images and videos generated with tools like Reels and Canvas will include digital “watermarks.” This could be an important step in fighting back against the issue of AI-generated deepfakes in the current enterprise landscape.
One thing we’re not entirely certain about is how Amazon will address text-based copyright issues. The company doesn’t share which information the models are trained on. However, AWS does offer an indemnification policy to cover customers if one of its models “regurgitates” copyrighted content.
What’s Next for Amazon’s AI Models?
Next on the roadmap for Amazon’s Nova collection is Nova Premier (mentioned above). This model will appear sometime in the first quarter of 2025. Additionally, the company has also said that it’s going to introduce a ” speech-to-speech” model in 2025, designed to transform conversational AI applications by allowing them to understand speech input.
This model will allow bots to interpret verbal and nonverbal cues (such as cadence and tone) and deliver human-like responses with minimal latency. Amazon is also developing a novel model that should be able to take text, images, audio, and video as input and generate output in all of those modalities. This “multimodal-to-multimodal” solution is set to appear in mid-2025.
Theoretically, the new model can perform many tasks, such as translating content from one modality into another and powering multimodal AI agents.
For now, companies interested in experimenting with Amazon Nova can access all three initial models and the Canva and Reels tools in Amazon Bedrock. They’re available in various US regions and operate on a “pay-as-you-go” model (just like the other models on Amazon Bedrock). You can find the Bedrock pricing information here.
Plus, if you want to provide feedback to guide the next innovations in these models, you can share your thoughts on the AWS Community website.
