Maki is a Python framework for building multi-agent LLM applications. It supports multiple LLM backends (Ollama, OpenAI, Anthropic, HuggingFace), a plugin system with 16 built-in tools, a workflow engine with dependency resolution and parallel execution, and a distributed layer for serving agents over HTTP.
The framework is organized into four layers:
- Public API — maki/__init__.pylazy-loads all exports on first access
- LLM Backends — MakiLLama,MakiOpenAI,MakiAnthropic, andHFBackendall implement the abstractLLMBackendcontract
- Agent System — AgentcomposesPluginHandlerandReasoningEnginemixins;AgentManagerorchestrates agents viaWorkflowTaskandWorkflowState
- Distributed Layer — AgentServer(FastAPI) exposes agents over HTTP;AgentProxyprovides a remote-agent client with circuit-breaking;DistributedAgentManagermixes local and remote agents
- Infrastructure — Connector(SSRF-protected HTTP with connect-time IP pinning), shared data classes, typed exceptions, runtime config, and structured logging
The Plugin System sits alongside the Agent layer: plugins are loaded on demand and invoked automatically when the LLM emits a TOOL: directive or via native tool-calling APIs (Ollama, OpenAI, Anthropic).
- MakiLLama— Ollama chat API with synchronous, streaming, async, and vision-capable workflows
- MakiOpenAI— OpenAI chat completions, including reasoning models (o3/o4)
- MakiAnthropic— Anthropic messages API (Claude Sonnet, Haiku, Opus)
- HFBackend— direct HuggingFace Transformers integration with quantization and device selection
- Agent— role-based agents with task execution, memory, reasoning, and plugin support; per-agent execution lock for concurrent safety
- AgentManager— multi-agent orchestration: sequential pipelines, collaborative tasks, and dependency-aware workflows with parallel batching and checkpoint/resume
- ConversationMemory— token-budgeted, pair-based conversation history shared by- Agent(stateful mode) and- ChatSession
- Native tool-calling for all backends (Ollama tools=, OpenAI, Anthropic tool use) with multi-round execution and self-correction
- 16 built-in plugins covering files, web content, search, trading, and memory
- Distributed agent serving: maki serveexposes any agent over HTTP;AgentProxyconsumes remote agents transparently
- SSRF-protected HTTP connector with DNS pinning, error classification, and configurable timeouts
- Fail-closed plugin security: every plugin declares ALLOWED_METHODS; destructive methods require explicit opt-in
- Explicit logging setup and a typed exception hierarchy
curl -fsSL https://raw.githubusercontent.com/BowlOfData/maki/main/install.sh | bashThis clones the repo into ~/.maki, installs it into an isolated virtual environment, and symlinks the maki command into ~/.local/bin (override with the MAKI_INSTALL_DIR / MAKI_BIN_DIR env vars). Re-run it any time to update to the latest main.
pip install -e .For development tools:
pip install -e ".[dev]"Some built-in plugins and backends rely on optional extras (defined in pyproject.toml):
- maki-framework[web]—- feedparser,- readability-lxml,- html2text(web search / web-to-Markdown)
- maki-framework[trends]—- pytrends(Google Trends)
- maki-framework[alpaca]—- alpaca-py(market data, news, trading, streaming)
- maki-framework[ftp]—- paramiko(FTP/SFTP)
- maki-framework[gui]—- PySide6(desktop GUI)
- maki-framework[openai]—- openai(OpenAI backend)
- maki-framework[anthropic]—- anthropic(Anthropic backend)
- maki-framework[distributed]—- fastapi,- uvicorn,- pyyaml(agent server and proxies)
- maki-framework[distributed-redis]—- redis(Redis workflow checkpoints)
Install everything with pip install -e ".[all]".
Shared runtime defaults live in maki/config.py. All values are overridable via environment variables or a .env file (python-dotenv is supported).
from maki import MakiLLama
llm = MakiLLama(model="gemma4:26b")
response = llm.chat("Explain recursion in one sentence.")
print(response.content)import asyncio
from maki import MakiLLama
from maki.objects import GenerationConfig
config = GenerationConfig(temperature=0.7, max_tokens=512)
llm = MakiLLama(model="gemma4:26b", config=config)
reply = llm.chat("Give me three project naming ideas.")
print(reply.content)
for chunk in llm.stream("Write a short haiku about testing"):
print(chunk, end="", flush=True)
async def main():
response = await llm.async_chat("Summarize the benefits of type hints.")
print(response.content)
asyncio.run(main())from maki import MakiLLama
llm = MakiLLama(model="gemma4:26b")
session = llm.session(system="You are a concise engineering assistant.")
session.say("We are building a release checklist.")
response = session.say("What should we verify before publishing a Python package?")
print(response.content)from maki import MakiOpenAI, MakiAnthropic
# OpenAI
llm = MakiOpenAI(model="gpt-4o")
response = llm.chat("What is the capital of France?")
# Anthropic
llm = MakiAnthropic(model="claude-sonnet-4-5")
response = llm.chat("Summarize this code in one sentence.")from maki import MakiLLama
from maki.agents import Agent
llm = MakiLLama(model="gemma4:26b")
agent = Agent(
name="Reviewer",
maki_instance=llm,
role="code reviewer",
instructions="Focus on bugs, regressions, and missing validation.",
stateful=True,
)
result = agent.execute_task("Review this design: a plugin system with file access.")
print(result)agent.remember("repo", "maki")
print(agent.recall("repo"))
steps = agent.think_step_by_step("How should we structure plugin validation?")
subtasks = agent.decompose_task("Prepare this repository for a public release")for chunk in agent.stream_task("Draft a short changelog entry."):
print(chunk, end="", flush=True)By default, execute_task sends one blocking HTTP request. For tasks that exceed the configured timeout (default 120 s), set use_streaming=True — the timeout then applies per chunk rather than to the whole response.
agent = Agent(
name="Ranker",
maki_instance=llm,
role="content ranker",
use_streaming=True,
)
result = agent.execute_task("Rank these 50 articles by relevance: ...")
print(result)AgentManager coordinates multiple agents and can run collaborative or dependency-aware workflows.
from maki import MakiLLama
from maki.agents import AgentManager, WorkflowTask
llm = MakiLLama(model="gemma4:26b")
manager = AgentManager(llm)
manager.add_agent("Researcher", role="researcher")
manager.add_agent("Writer", role="writer")
workflow = [
WorkflowTask(
name="research",
agent="Researcher",
task="Find the main public-release risks for this repository.",
),
WorkflowTask(
name="summary",
agent="Writer",
task="Summarize the research into a release checklist.",
dependencies=["research"],
),
]
results = manager.run_workflow(workflow)
print(results["summary"]["result"])Supported manager patterns:
Serve any agent over HTTP with maki serve:
maki serve --config agent.yaml --host 127.0.0.1 --port 8100# agent.yaml
name: MyAgent
model: gemma4:27b
role: assistant
plugins:
- web_search
- file_readerConnect to a remote agent from another process:
from maki.distributed.proxy import AgentProxy
agent = AgentProxy(name="MyAgent", base_url="http://127.0.0.1:8100")
result = agent.execute_task("Summarize the latest AI news.")DistributedAgentManager lets you mix local and remote agents in the same workflow.
Built-in plugins are registered in maki/plugins/__init__.py:
from maki import MakiLLama
from maki.agents import Agent
llm = MakiLLama(model="gemma4:26b")
agent = Agent(name="ToolUser", maki_instance=llm, role="assistant")
agent.load_plugin("file_reader")
result = agent.execute_task(
"Read the first lines of README.md and summarize them.",
use_plugins=True,
)
print(result)When use_plugins=True (or the backend supports native tool-calling), available plugin methods are advertised to the model and executed automatically. Destructive methods (file writes, trades, FTP deletes) require Agent(allow_dangerous_tools=True).
HFBackend runs models directly via HuggingFace Transformers — no Ollama required.
from maki import HFBackend
llm = HFBackend(model="mistralai/Mistral-7B-Instruct-v0.2", device="cuda")
response = llm.chat("Explain attention mechanisms.")
print(response.content)Supports quantization and device selection (cpu, cuda, mps).
Top-level imports exposed by maki:
- MakiLLama,- MakiOpenAI,- MakiAnthropic,- HFBackend
- LLMBackend,- BackendType
- Agent,- AgentManager
- GenerationConfig,- LLMResponse,- Message,- ToolCall
- ConversationMemory,- RateLimiter
- Connector,- Utils
- config
All exports are lazy-loaded on first access.
The repository includes a PySide6/QML desktop shell (requires maki-framework[gui]):
maki-guipytest888 tests covering backends, agents, workflows, plugins, connectors, distributed layer, and security-related behaviour. Tests marked @pytest.mark.network (requiring live external services) are excluded by default; run them explicitly with pytest -m network.
Contributions are welcome: bug fixes, documentation improvements, new plugins, and feature suggestions all help move the project forward. Open an issue or submit a pull request on GitHub.
If you are interested in this line of research, consider joining Bowl of Data, an open-source AI research community.