Self-hosted open-source AI platforms (& tools)



Flows

Langflow

GitHub - langflow-ai/langflow: Langflow is a low-code app builder for RAG and multi-agent AI applications. It’s Python-based and agnostic to any model, API, or database.
Langflow is a low-code app builder for RAG and multi-agent AI applications. It’s Python-based and agnostic to any model, API, or database. - langflow-ai/langflow

Flowise

GitHub - FlowiseAI/Flowise: Drag & drop UI to build your customized LLM flow
Drag & drop UI to build your customized LLM flow. Contribute to FlowiseAI/Flowise development by creating an account on GitHub.

Dify

GitHub - langgenius/dify: Dify is an open-source LLM app development platform. Dify’s intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, lettin…

ComfyUI

GitHub - comfyanonymous/ComfyUI: The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. - comfyanonymous/ComfyUI

Observability

OpenLIT

GitHub - openlit/openlit: Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers,…

OpenTelemetry (feat. OpenLIT)

A complete guide to LLM observability with OpenTelemetry and Grafana Cloud | Grafana Labs
Explore why monitoring LLM applications is so important and how you can do it more easily by using OpenTelemetry, Grafana Cloud, and OpenLIT.
LLM Observability with Elastic, OpenLIT and OpenTelemetry — Elastic Observability Labs
Langchain applications are growing in use. The ability to build out RAG-based applications, simple AI Assistants, and more is becoming the norm. Observing these applications is even harder. Given the various options that are out there, this blog shows how to use OpenTelemetry instrumentation with the OpenLIT instrumentation library to ingest traces into Elastic Observability APM.
An Introduction to Observability for LLM-based applications using OpenTelemetry
Large Language Models (LLMs) are really popular right now, especially considering the wide range of applications that they have from simple chatbots to Copilot bots that are helping software engineers write code. Seeing the growing use of LLMs in production, it’s important for users to learn how to understand and monitor how these models behave. In the following example, we’ll use Prometheus and Jaeger as the target backend for metrics and traces generated by an auto-instrumentation LLM monitoring library OpenLIT. We will use Grafana as the tool to visualize the LLM monitoring data. You can choose any backend of your choice to store OTel metrics and traces.

Langfuse

GitHub - langfuse/langfuse: 🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23 -…

Phoenix

GitHub - Arize-ai/phoenix: AI Observability & Evaluation
AI Observability & Evaluation. Contribute to Arize-ai/phoenix development by creating an account on GitHub.

Prompt Management

Langfuse

GitHub - langfuse/langfuse: 🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23 -…

Pezzo

GitHub - pezzolabs/pezzo: 🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.
🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more. - pezzolabs/pezzo

Agenta

GitHub - Agenta-AI/agenta: The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place.
The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place. - Agenta-AI/agenta

Latitude-LLM

GitHub - latitude-dev/latitude-llm: Latitude is the open-source prompt engineering platform to build, evaluate, and refine your prompts with AI
Latitude is the open-source prompt engineering platform to build, evaluate, and refine your prompts with AI - latitude-dev/latitude-llm

Coding Agent

Cline

GitHub - cline/cline: Autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, using the browser, and more with your permission every step of the way.
Autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, using the browser, and more with your permission every step of the way. - cline/cline

Screenshot to Code

GitHub - abi/screenshot-to-code: Drop in a screenshot and convert it to clean code (HTML/Tailwind/React/Vue)
Drop in a screenshot and convert it to clean code (HTML/Tailwind/React/Vue) - abi/screenshot-to-code

Aider

GitHub - Aider-AI/aider: aider is AI pair programming in your terminal
aider is AI pair programming in your terminal. Contribute to Aider-AI/aider development by creating an account on GitHub.

RAGs

AnythingLLM

GitHub - Mintplex-Labs/anything-llm: The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more. - Mintplex-Labs/anything-llm

Kotameon

GitHub - Cinnamon/kotaemon: An open-source RAG-based tool for chatting with your documents.
An open-source RAG-based tool for chatting with your documents. - Cinnamon/kotaemon

Chats

AnythingLLM

GitHub - Mintplex-Labs/anything-llm: The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more. - Mintplex-Labs/anything-llm

LobeChat

GitHub - lobehub/lobe-chat: 🤯 Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Qwen / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS/Plugins/Artifacts). One-click FREE deployment of your private ChatGPT/ Claude application.
🤯 Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Qwen / DeepSeek), Knowledge Base (file upload / knowledge manageme…

LibreChat

GitHub - danny-avila/LibreChat: Enhanced ChatGPT Clone: Features Agents, DeepSeek, Anthropic, AWS, OpenAI, Assistants API, Azure, Groq, o1, GPT-4o, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, Code Interpreter, langchain, DALL-E-3, OpenAPI Actions, Functions, Secure Multi-User Auth, Presets, open-source for self-hosting. Active project.
Enhanced ChatGPT Clone: Features Agents, DeepSeek, Anthropic, AWS, OpenAI, Assistants API, Azure, Groq, o1, GPT-4o, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message se…

OpenWebUI

GitHub - open-webui/open-webui: User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
User-friendly AI Interface (Supports Ollama, OpenAI API, ...) - open-webui/open-webui

Khoj

GitHub - khoj-ai/khoj: Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous…

Document Processing

Unstract

GitHub - Zipstack/unstract: No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents
No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents - Zipstack/unstract

Chats in CLI

Mods

GitHub - charmbracelet/mods: AI on the command line
AI on the command line. Contribute to charmbracelet/mods development by creating an account on GitHub.

Research

Storm

GitHub - stanford-oval/storm: An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.
An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations. - stanford-oval/storm

Proxies

LiteLLM

GitHub - BerriAI/litellm: Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq] - BerriAI/litellm

vLLM

GitHub - vllm-project/vllm: A high-throughput and memory-efficient inference and serving engine for LLMs
A high-throughput and memory-efficient inference and serving engine for LLMs - vllm-project/vllm

Bedrock Access Gateway

GitHub - aws-samples/bedrock-access-gateway: OpenAI-Compatible RESTful APIs for Amazon Bedrock
OpenAI-Compatible RESTful APIs for Amazon Bedrock. Contribute to aws-samples/bedrock-access-gateway development by creating an account on GitHub.

Extras

OpenHealth

GitHub - OpenHealthForAll/open-health: OpenHealth, AI Health Assistant | Powered by Your Data, Running Locally
OpenHealth, AI Health Assistant | Powered by Your Data, Running Locally - OpenHealthForAll/open-health
How I Built an Open Source AI Tool to Find My Autoimmune Disease (After $100k and 30+ Hospital Visits) - Now Available for Anyone to Use
by u/Dry_Steak30 in selfhosted