Andromeda GitHub Details, Stars and Alternatives | OpenRepoFinder
5chm33 / repository
Andromeda
A production-grade recursive self-improvement engine — autonomously proposes, validates, and commits improvements to its own codebase via multi-agent debate and RLAIF.
A transparent discovery signal based on current public GitHub metadata.
Recent activity35% weight
100
Community adoption25% weight
0
Maintenance state20% weight
100
License clarity10% weight
100
Project information10% weight
100
This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
Development started March 3, 2026. A 6-month journey from a local desktop launcher to a production-grade autonomous agent.
What It Does
Andromeda is a production Node.js/TypeScript application that autonomously resolves real GitHub issues from open-source Python repositories. Given a problem statement and a failing test suite, it:
Localizes the relevant source files inside the SWE-bench Docker image using LLM-guided keyword search
Builds structured context using keyword-aware skeleton assembly with ±10-line padding around relevant functions, with full line numbers
Resolves cross-file symbols by scanning imports and call chains to pull in dependent files (up to 200k char budget)
Generates candidate patches using a 4-agent parallel consensus engine (conservative, creative, defensive, refactor styles)
Validates each candidate by running the actual test suite inside the Docker container
Iterates with traceback feedback for up to 5 revision attempts, with 3-tier model escalation
Applies patches robustly using fixHunkCounts pre-processing + git apply --fuzz=15 + --unidiff-zero fallback to handle LLM formatting drift
Detects Python version in the testbed container to ensure probe scripts use compatible syntax (Python 3.5 vs 3.6+)
The system is evaluated on SWE-bench Verified — the standard benchmark for autonomous software engineering agents. Note: The 66% headline score is based on a standard 50-instance subset run, which is a common cost-constrained evaluation methodology. A previous full 500-instance run scored 19.2% but was affected by a fallback contamination issue that has since been resolved.
Language-Agnostic Core
A common misconception is that Andromeda is a "Python-only" agent because of the SWE-bench results. This is incorrect. Andromeda's core architecture — the RSI engine, MAD protocol, RLAIF loop, and ReAct engine — is entirely language-agnostic and written in TypeScript. The SWE-bench harness targets Python solely because the benchmark itself is built on Python repositories. The agent is fully capable of resolving issues in JavaScript, TypeScript, Go, Rust, or any other language with a test runner.
Cost-Efficiency as a First Principle
Andromeda is designed for B2B/enterprise cost efficiency, not just raw benchmark scores. While running a frontier model (like Claude Fable 5) on every instance yields high scores, it costs ~$2.00+ per instance. Andromeda's 3-tier model escalation resolves ~60% of issues using fast, cheap models (like Sonnet 5 at ~$0.05/instance), escalating to expensive frontier models only when the traceback loop detects a hard failure. This achieves near-frontier quality at ~10% of the cost.
Benchmarks
SWE-bench Verified — Run History (50-instance validation set)
All runs use the same 50-instance slice: 22 astropy + 28 django instances.
Run
Date
Score
Astropy
Django
Key Change
v3 baseline
Jun 30 2026
26.0% (13/50)
9.1% (2/22)
39.3% (11/28)
Skeleton context + 4-agent consensus
Run 7
Jun 2026
52.0% (26/50)
59.1% (13/22)
46.4% (13/28)
Pipeline stabilization
Run 8
Jul 2026
66.0% (33/50)
77.3% (17/22)
57.1% (16/28)
2-tier escalation + 21 pipeline fixes
Run 9
Jul 2026
~70% (partial — credits exhausted at inst. 24/50)
72.7% (16/22)
—
3-tier escalation + Fixes 22–32
Run 8 is a +14 percentage point improvement over Run 7 in a single session — and a +40 point improvement over the v3 baseline.
Run 9 reached 72.7% on the astropy subset before API credits were exhausted. The 3-tier escalation (Sonnet 5 as mid-tier) resolved instances that previously timed out, including astropy-13977 which had failed in run 8 after 770 seconds.
v3 Agent — Claude Sonnet 4.5 Exclusive (50-instance, Jun 30 2026)
Metric
Result
Resolved (Official Score)
26.0% (13 / 50 instances)
Django resolve rate
39.3% (11 / 28)
Astropy resolve rate
9.1% (2 / 22)
Model
Claude Sonnet 4.5 exclusively via OpenRouter
This was the baseline before the 21-fix improvement sprint. Astropy at 9.1% was the primary weakness — the skeleton context and symbol resolution fixes brought it to 77.3%.
v3 Agent — Mixed Model (500 instances, Jun 28 2026)
Metric
Result
Resolved (Official Score)
19.20% (96 / 500 instances)
Model
Claude Sonnet 4.5 + DeepSeek Coder fallback
The 19.20% score was degraded by DeepSeek fallback contamination — 101/500 instances used patches that failed to apply. The current pipeline eliminates this.
Andromeda has two distinct code paths that share core components:
1. SWE-bench Evaluation Pipeline (scripts/run_swebench.ts + server/sweBench*.ts)
A specialized, purpose-built loop designed for the SWE-bench benchmark format. Takes a problem statement + Docker image + failing tests, and produces a git diff patch. This is what the benchmark scores measure.
2. Main Agent (server/reactEngine.ts + server/externalRepoFixer.ts)
A general-purpose ReAct (Reason + Act) loop that handles user requests via chat. Uses the same LLM infrastructure and buildSmartContext for code editing tasks. Improvements to the SWE-bench pipeline are ported back to this path.
Shared components (improvements to either path benefit both):
server/sweBenchModelConfig.ts — all LLM presets and escalation logic
server/llmRouter.ts — model routing (code tasks → Claude Sonnet 4.5 → Sonnet 5 → Fable 5)
server/tools/webSearch.ts — Tavily search (now exported for pipeline use)
The SWE-bench benchmark is the highest-stress test available for a code agent — real GitHub issues, real test suites, no hints. A strong SWE-bench score is a credible, verifiable measure of the underlying intelligence that also powers the main agent.
scripts/run_swebench.ts — Main runner. Loads the SWE-bench dataset from HuggingFace cache, extracts file content from Docker images, runs localization, and orchestrates the full pipeline. Implements 3-tier model escalation via environment variables. Includes Fix 26 (proper Promise.race cleanup to prevent process crashes on instance timeout).
server/sweBenchConsensus.ts — 4-agent parallel patch generation. Each agent uses a different temperature and reasoning style (conservative, creative, defensive, refactor). The best-passing candidate wins.
server/sweBenchTracebackLoop.ts — Iterative test-feedback loop. Applies the candidate patch inside the Docker container, runs the actual test suite, captures the traceback, and feeds it back to the LLM for up to 5 revision attempts. Implements fixHunkCounts for robust patch application. Hard cap of 120k characters on revision prompts (Fix 22). Detects Python version in container for probe script compatibility (Fix 32).
server/sweBenchContextBuilder.ts — Smart context assembly. Keyword-aware truncation with ±10-line padding anchored to the first keyword match. Builds skeleton context for large files. Handles the maxChars=80000 budget for revision prompts. Exports runDebugProbe and buildDebugProbePrompt with Python version awareness.
server/sweBenchModelConfig.ts — Model configuration. All LLM presets (Sonnet 4.5, Sonnet 5, Fable 5, Kimi, DeepSeek). Implements createEscalatingLLMProvider for 2-tier and 3-tier escalation.
server/sweBenchInfra.ts — Docker infrastructure. Handles image pulling, disk space management, and container lifecycle.
server/externalRepoFixer.ts — Main agent's GitHub repo fixing path. Now uses buildSmartContext for intelligent context selection (Fix 30) and multi-attempt revision with model escalation (Fix 31).
server/tools/webSearch.ts — Web search integration. Tavily as primary provider (AI-optimized, high relevance). Falls back to Brave → SearXNG → DuckDuckGo. Exports searchTavilyDirect for pipeline use (Fix 25a). Wired into sweBenchSearchFallback.ts as primary provider (Fix 25b).
Context Assembly (Current)
For files larger than 12,000 characters, the pipeline builds a smart context instead of blindly truncating:
Extracts all class and function signatures (the skeleton) — the LLM sees the full structural map of the file with line numbers
Fully expands any function whose name appears in the issue description or failing test names
Uses ±10-line padding around the first keyword match for precision anchoring
Caps the total context at 40,000 characters for initial patches; 80,000 characters for revision prompts
Resolves cross-file symbols by scanning imports and call chains, adding dependent fil
-brightgreen){kind=icon}
