Chapter 8
Features
LocalAI provides a comprehensive set of features for running AI models locally. This section covers all the capabilities and functionalities available in LocalAI.
Core Features
- Text Generation - Generate text with GPT-compatible models using various backends
- Image Generation - Create images with Stable Diffusion and other diffusion models
- Audio Processing - Transcribe audio to text and generate speech from text
- Text to Audio - Generate speech from text with TTS models
- Sound Generation - Generate music and sound effects from text descriptions
- Voice Activity Detection - Detect speech segments in audio data
- Video Generation - Generate videos from text prompts and reference images
- Embeddings - Generate vector embeddings for semantic search and RAG applications
- GPT Vision - Analyze and understand images with vision-language models
Advanced Features
- OpenAI Functions - Use function calling and tools API with local models
- Realtime API - Low-latency multi-modal conversations (voice+text) over WebSocket
- Constrained Grammars - Control model output format with BNF grammars
- GPU Acceleration - Optimize performance with GPU support
- Distributed Inference - Scale inference across multiple nodes
- P2P API - Monitor and manage P2P worker and federated nodes
- Model Context Protocol (MCP) - Enable agentic capabilities with MCP integration
- Agents - Autonomous AI agents with tools, knowledge base, and skills
Specialized Features
- Object Detection - Detect and locate objects in images
- Reranker - Improve retrieval accuracy with cross-encoder models
- Stores - Vector similarity search for embeddings
- Model Gallery - Browse and install pre-configured models
- Backends - Learn about available backends and how to manage them
- Backend Monitor - Monitor backend status and resource usage
- Runtime Settings - Configure application settings via web UI without restarting
Getting Started
To start using these features, make sure you have LocalAI installed and have downloaded some models. Then explore the feature pages above to learn how to use each capability.