Quick start

One line on Linux, macOS, or WSL:

curl -fsSL https://raw.githubusercontent.com/Krish-1507/Voice-Automated-Desktop-Agent-J.A.M.E.S/main/install.sh | bash

One line on Windows (PowerShell):

irm https://raw.githubusercontent.com/Krish-1507/Voice-Automated-Desktop-Agent-J.A.M.E.S/main/install.ps1 | iex

The one-line installer clones the repository if you are not already inside it, creates a virtual environment, and installs JAMES with the orb GUI and the MCP client by default. Add flags to install more: --with-browser, --with-voice, --with-desktop, --with-docs, --with-all, or --minimal for core only.

Manual setup:

git clone https://github.com/Krish-1507/Voice-Automated-Desktop-Agent-J.A.M.E.S.git
cd "Voice-Automated-Desktop-Agent-J.A.M.E.S"
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[ui,mcp]"   # add extras you want: docs, browser, voice, desktop, all
cp .env.example .env          # edit: set LLM_PROVIDER + your API key, or custom for Ollama
python -m james --check
python -m james doctor        # optional self-diagnostics
python -m james --text        # or: python -m james --ui

Minimal .env for a cloud provider:

LLM_PROVIDER=openai
LLM_MODEL=gpt-4o-mini
OPENAI_API_KEY=sk-...
VOICE_ENABLED=false

Minimal .env for a fully local setup (no API keys at all):

LLM_PROVIDER=custom
LLM_MODEL=llama3.1
CUSTOM_BASE_URL=http://localhost:11434/v1   # Ollama
CUSTOM_API_KEY=ollama
VOICE_ENABLED=false

And for a privacy-certified, zero-egress run (everything on your machine):

OFFLINE_MODE=true
LLM_PROVIDER=custom
LLM_MODEL=llava                 # a local vision model for computer-use
VISION_MODEL=llava
CUSTOM_BASE_URL=http://localhost:11434/v1
CUSTOM_API_KEY=ollama
VOICE_ENABLED=false
pip install "james-assistant[desktop,memory]"

Providers and model selection

Pick a provider with LLM_PROVIDER and the exact model with LLM_MODEL. Because the model id is fully editable, you choose what fits best for cost, speed, and capability.

Voice providers are flexible:

• STT: whisper_local (offline), whisper_api, google, none (typed).
• TTS: pyttsx3 (offline), edge (free, high quality), openai, elevenlabs, none.

Extensibility: MCP and Skill Forge

MCP — inherit the whole ecosystem

Install the MCP extra, then point JAMES at any Model Context Protocol server (stdio or HTTP/SSE) through MCP_SERVERS or an mcp.json file. Its tools show up in the registry like native ones, with no glue code per integration.

[{"name":"fs","command":"npx","args":["-y","@modelcontextprotocol/server-filesystem","./workspace"]}]

Skill Forge — JAMES teaches itself

After a complex task, you can say "save this as a skill called " and JAMES writes a real, typed @tool plugin to ./plugins/, validates it, and loads it immediately. Unlike generic "skills" (free-text recipes), the result is a directly executable native tool — no re-implementation and no re-prompting next time. Manage them with list_skills / forget_skill.

Self-improving Skill Forge (automatic): with AUTO_SKILL=true (the default), after a successful multi-tool task JAMES generates that capability as a native @tool on its own. It inspects the tool chain it just ran, asks the model to write a clean @tool plugin that encapsulates the workflow, validates it, persists it to ./plugins/, and hot-loads it. Next time, the model can call the saved tool directly.

Computer-use (vision desktop control)

computer_use runs a tight local loop: it screenshots the screen, asks a vision model what to do next, acts with pyautogui (click, type, scroll), and repeats until the instruction is done. No cloud browser is required; pair it with a local vision model such as Ollama's llava in offline mode. Granular control tools click_at, type_text, press_key, and screenshot_save are also available.

Delegation

delegate spins up isolated sub-agents for parallel subtasks and combines their results, so you can split a big job across threads in a single request. Sub-agent tool calls stream into the same live task canvas as the parent.

Research and self-learning

JAMES is built to be self-learning: it looks things up, teaches itself new skills, and grows more capable the more you use it.

• research(query) searches the web, reads the top sources, and returns a concise cited answer. Use it whenever the task needs current facts or how-to context.
• learn_skill(goal) researches the goal, asks the model to write a native @tool plugin that implements it, and saves it to ./plugins/ where it is hot-loaded and available immediately. This is the self-improving loop: research, understand, and turn it into executable code — no re-prompting next time.

The system prompt is explicit: whenever JAMES lacks a capability it must research it and learn it, then use the skill it just created. Combined with the auto Skill Forge (above), JAMES becomes more capable after every real task rather than forgetting what it did.

Background execution

background_task(task) runs a request in an isolated sub-agent on a daemon thread and returns a task id right away, so JAMES can keep helping you while the work continues. Later, get_background_result(id) returns the outcome and list_background_tasks() shows everything in flight. Background tasks are persisted, so they survive restarts.

Full file-explorer control

Beyond read/write/search/delete, JAMES can create_directory, copy_file, move_file, rename_file, and render a directory_tree. It manages your files proactively as part of completing a task. Destructive file operations (delete, move, rename) ask for confirmation unless disabled.

Autonomous File Explorer Manager

manage_files(scope, goal) lets JAMES take full, 100% agentic control of a part of the filesystem and run the job in the background. It explores the directory tree, organises files by type or date, removes or quarantines duplicates, tidies names, and reports back — without interrupting you. The scope can be desktop, documents, downloads, workspace, home, whole, or any absolute path. It is fully autonomous: no confirmation prompts, no stop-and-ask, it works until the location is tidy. Track it with list_file_manager_tasks / stop_file_manager.

Enable AUTO_FILE_MANAGER=true to start a daemon that keeps your main folders (Desktop, Documents, Downloads, ...) organised on a schedule, so the file explorer manages itself. Set FILE_MANAGER_INTERVAL (seconds) and FILE_MANAGER_SCOPES to tune it.

Optional GUI

Prefer a visual shell? Install the UI extra, then run python -m james --ui to launch the orb interface: a live status orb, a task canvas that lists every tool the agent calls in real time, streaming replies revealed word by word, a full log, and a system tray icon to hide, show, and quit. The assistant runs in a worker thread.

Privacy-certified offline mode

Run JAMES as a fully local assistant with OFFLINE_MODE=true (or python -m james --offline). A socket-level firewall blocks every non-loopback network attempt — both DNS resolution and TCP connections — so the LLM SDKs, the browser, and any future plugin cannot send data elsewhere. Only 127.0.0.1 / localhost is permitted, which is exactly where a local model like Ollama listens. Every allowed and blocked connection is written to EGRESS_AUDIT_LOG, so you (or an auditor) can prove that nothing left the machine. Web tools refuse explicitly in this mode.

Doctor and resilience

• python -m james doctor runs a one-shot self-diagnostic that reports PASS/WARN/FAIL for the Python version, core and optional dependencies, API keys, microphone, .env, writable workspace, browser, computer-use, semantic memory, and offline mode. It is useful for first-run checks and bug reports.
• Model failover: set LLM_FAILOVER="anthropic:claude-3-5-sonnet-latest,groq:llama-3.3-70b-versatile" and JAMES automatically retries the next provider if the primary errors (rate limits, outages) so a task never dies mid-flight. Permanent errors (bad API key, auth failures, HTTP 401–403) are not retried, since they will not succeed on another provider — JAMES fails fast instead of wasting requests, and logs which provider actually served each request.

Architecture

james/
├── __main__.py