Loading repository data…
Loading repository data…
SomeOddCodeGuy / repository
WilmerAI is one of the oldest LLM semantic routers. It uses multi-layer prompt routing and complex workflows to allow you to not only create practical chatbots, but to extend any kind of application that connects to an LLM via REST API. Wilmer sits between your app and your many LLM APIs, so that you can manipulate prompts as needed.
"What If Language Models Expertly Routed All Inference?"
This project is still under development. The software is provided as-is, without warranty of any kind.
This project and any expressed views, methodologies, etc., found within are the result of contributions by the maintainer and any contributors in their free time and on their personal hardware, and should not reflect upon any of their employers.
WilmerAI is an application designed for advanced semantic prompt routing and complex task orchestration. It originated from the need for a router that could understand the full context of a conversation, rather than just the most recent message.
Unlike simple routers that might categorize a prompt based on a single keyword, WilmerAI's routing system can analyze the entire conversation history. This allows it to understand the true intent behind a query like "What do you think it means?", recognizing it as historical query if that statement was preceded by a discussion about the Rosetta Stone, rather than merely conversational.
This contextual understanding is made possible by its core: a node-based workflow engine. Like the rest of Wilmer, the routing is a workflow, categorizing through a sequence of steps, or "nodes", defined in a JSON file. The route chosen kicks off another specialized workflow, which can call more workflows from there. Each node can orchestrate different LLMs, call external tools, run custom scripts, call other workflows, and many other things.
To the client application, this entire multi-step process appears as a standard API call, enabling advanced backend logic without requiring changes to your existing front-end tools.
The below shows Open WebUI connected to 2 instances of Wilmer (recorded before multi-user support was added; a single instance can now serve multiple users). The first instance just hits Mistral Small 3 24b directly, and then the second instance makes a call to the Offline Wikipedia API before making the call to the same model.
Click the image to play gif if it doesn't start automatically
A zero-shot to an LLM may not give great results, but follow-up questions will often improve them. If you regularly perform the same follow-up questions when doing tasks like software development, creating a workflow to automate those steps can have great results.
With workflows, you can have as many LLMs available to work together in a single call as you have computers to support. For example, if you have old machines lying around that can run 3-8b models? You can put them to use as worker LLMs in various nodes. The more LLM APIs that you have available to you, either on your own home hardware or via proprietary APIs, the more your workflow network can do. A single prompt to Wilmer could reach out to 5+ computers, including proprietary APIs, depending on how you build your workflow.
Advanced Contextual Routing The primary function of WilmerAI. It directs user requests using context-aware logic. This is handled by two mechanisms:
Crucially, these routing decisions can be based on the entire conversation history, not just the user's last messages, allowing for a much deeper understanding of intent.
For my own edification, to ensure I didn't accidentally add something that would negatively impact Wilmer's privacy posture, I'll sometimes ask Claude Code to do an end-to-end check to look for any outbound calls or other data leakage. It's not as good as a formal code audit, but it gives me peace of mind. I've included the results of the check here.
I've been doing this check whenever I make a really big set of changes, just to make sure that I didn't introduce something I didn't intend to via a library or sloppy coding.
On 2026-06-21, Claude Code (Claude Opus 4.8) was asked to search the codebase and report any outbound network calls, telemetry, or other privacy-relevant behavior it could find. The results listed below were generated for my own personal use and were shared for transparency; they are not a guarantee.
If privacy matters to your deployment, please run your own analysis before using WilmerAI.
What Was Checked
----------------
The Middleware/ and Public/ source trees, the top-level entry points (server.py, run_eventlet.py,
run_waitress.py), and all shell/batch launcher scripts (run_macos.sh, run_windows.bat,
Scripts/rekey_encrypted_files.sh, Scripts/rekey_encrypted_files.bat) were searched for outbound
HTTP calls (requests.get, requests.post, requests.Session, requests.request), raw socket usage,
subprocess invocations, dynamic imports, hardcoded external URLs, telemetry-related keywords
(analytics, telemetry, phone-home, tracking, metrics), and environment variable reads. The entry
points and launcher scripts contained no outbound network calls; run_eventlet.py sets TCP_NODELAY
on the local listening socket but makes no external connections.
Outbound Network Calls
----------------------
The main outbound network call sites in Wilmer's own code, each targeting a destination the user
configures (this reflects the current source and is representative rather than a guaranteed
exhaustive registry; run your own analysis if it matters to you):
1. Middleware/llmapis/handlers/base/base_llm_api_handler.py
- session.post() to user-configured LLM endpoint (self.base_url from endpoint config)
2. Middleware/workflows/tools/offline_wikipedia_api_tool.py
- requests.get() to user-configured host; defaults to 127.0.0.1:5728
- Disabled unless activateWikiApi is set
3. Public/workflow_python_scripts/_isevendays_mcp_scripts/mcp_service_discoverer.py
- requests.get() to user-configured or env-var MCPO server; defaults to localhost:8889
4. Public/workflow_python_scripts/_isevendays_mcp_scripts/mcp_tool_executor.py
- requests.request() to same MCPO server as above
5. Middleware/workflows/handlers/impl/web_fetch_handler.py
- requests.request() to the URL set on a WebFetch node (user-authored); optional HTTP/SOCKS
proxy. TLS verification is on by default.
6. Middleware/workflows/handlers/impl/curl_command_handler.py
- spawns the system `curl` binary via subprocess.Popen (shell=False) against the URL(s) in a
CurlCommand node's args (user-authored)
7. Middleware/workflows/tools/mcp_client_tool.py
- connects to an MCP server defined under Public/Configs/MCPServers/ (user-configured) over
stdio (local process), SSE, or streamable HTTP, for MCPToolCall nodes
No telemetry, analytics, phone-home, auto-update, or hardcoded external URLs were found. Every
outbound call Wilmer is known to make goes to a destination the user configures -- there are no
connections to hosts that Wilmer picks on its own. Any optional tool or node not individually
listed above follows the same rule: it stays off unless the user enables it, and it targets only
the host the user points it at (typically defaulting to a local address). The WebFetch and
CurlCommand nodes additionally support an opt-in SSRF guard (blockPrivateAddresses / allowedHosts)
that can restrict which hosts they are allowed to reach.
Data Storage
------------
- JSON conversation/memory files: Optionally encrypted at rest using Fernet (AES-128-CBC with
HMAC, PBKDF2 with 100k iterations) when encryptUsingApiKey is enabled.
- SQLite databases: Used for vector memory and workflow locks. These are NOT encrypted, even
when the encryption feature is enabled.
- Log file