Skip to content

AI & LLM Tools

Curated open-source tools, frameworks, local model runtimes and learning resources for building with artificial intelligence and large language models (LLMs).

41 hand-picked resources

Every link on this page was checked in November 2026. Know a great project that is missing? Suggest it on GitHub or read the contribution guide.

Run models locally

  • Ollama - Get up and running with Llama, Gemma, Qwen and other open models on your own machine.
  • llama.cpp - LLM inference in C/C++ that runs on CPUs, Apple Silicon and GPUs.
  • vLLM - High-throughput, memory-efficient inference and serving engine for LLMs.
  • Open WebUI - Self-hosted, ChatGPT-style web interface for Ollama and OpenAI-compatible APIs.
  • LocalAI - Drop-in, OpenAI-compatible API that runs models on consumer hardware.
  • Jan - Open-source desktop app that runs LLMs 100% offline.

Frameworks & SDKs

  • LangChain - Framework for building context-aware LLM applications and agents.
  • LlamaIndex - Data framework to connect your own data to LLMs (RAG).
  • Hugging Face Transformers - Thousands of pretrained models for text, vision and audio.
  • LiteLLM - Call 100+ LLM APIs using the OpenAI format, with a proxy server and cost tracking.
  • Anthropic Python SDK - Official Python library for the Claude API.
  • OpenAI Python SDK - Official Python library for the OpenAI API.
  • Vercel AI SDK - TypeScript toolkit for building AI-powered web apps and agents.
  • DSPy - Programming, not prompting, language models.

Agents & MCP

Coding assistants

  • Aider - AI pair programming in your terminal.
  • Continue - Open-source AI code assistant for VS Code and JetBrains.
  • Cline - Autonomous coding agent inside your IDE.
  • Tabby - Self-hosted AI coding assistant.

Vector databases & RAG

  • Chroma - Open-source embedding database for AI apps.
  • Qdrant - High-performance vector search engine written in Rust.
  • Milvus - Cloud-native vector database built for scalable similarity search.
  • pgvector - Vector similarity search for PostgreSQL.
  • Weaviate - Open-source vector database with hybrid search.

Image, audio & demos

  • ComfyUI - Node-based interface for diffusion models.
  • Stable Diffusion web UI - Browser interface for Stable Diffusion.
  • Whisper - Robust speech recognition via large-scale weak supervision.
  • Gradio - Build and share ML demos and web apps in Python.
  • Streamlit - Turn data scripts into shareable web apps in minutes.

Learn AI