Table of Contents

• News and Updates
• Overview
• Installation and Usage
• Contributing
• Documentation
• Roadmap

News and Updates

2025-06-15

⚠️ This repository is no longer maintained. Please fork it if you wish to continue development or use it in your own projects.

2024-12-04:

New mode - Run Qodo Cover Pro in your GitHub CI workflow. Currently in preview and available for free for a limited time for Python projects, leveraging your own LLM API key from your favorite LLM provider. It's a practical way to improve code quality and reliability. For more details, reach out to the Qodo team.

2024-11-05:

New mode - scan an entire repo, auto identify the test files, auto collect context for each test file, and extend the test suite with new tests. See more details here.

Qodo-Cover

Welcome to Qodo-Cover. This focused project utilizes Generative AI to automate and enhance the generation of tests (currently mostly unit tests), aiming to streamline development workflows. Qodo-Cover can run via a terminal, and is planned to be integrated into popular CI platforms.

We invite the community to collaborate and help extend the capabilities of Qodo Cover, continuing its development as a cutting-edge solution in the automated unit test generation domain. We also wish to inspire researchers to leverage this open-source tool to explore new test-generation techniques.

Overview

This tool is part of a broader suite of utilities designed to automate the creation of unit tests for software projects. Utilizing advanced Generative AI models, it aims to simplify and expedite the testing process, ensuring high-quality software development. The system comprises several components:

1. Test Runner: Executes the command or scripts to run the test suite and generate code coverage reports.
2. Coverage Parser: Validates that code coverage increases as tests are added, ensuring that new tests contribute to the overall test effectiveness.
3. Prompt Builder: Gathers necessary data from the codebase and constructs the prompt to be passed to the Large Language Model (LLM).
4. AI Caller: Interacts with the LLM to generate tests based on the prompt provided.

Installation and Usage

Requirements

Before you begin, make sure you have the following:

• OPENAI_API_KEY set in your environment variables, which is required for calling the OpenAI API.
• Code Coverage tool: A Cobertura XML code coverage report is required for the tool to function correctly.
  • For example, in Python one could use pytest-cov. Add the --cov-report=xml option when running Pytest.
  • Note: We are actively working on adding more coverage types but please feel free to open a PR and contribute to cover_agent/CoverageProcessor.py

If running directly from the repository you will also need:

• Python installed on your system.
• Poetry installed for managing Python package dependencies. Installation instructions for Poetry can be found at https://python-poetry.org/docs/.

Standalone Runtime

Qodo Cover can be installed as a Python Pip package or run as a standalone executable.

Python Pip

To install the Python Pip package directly via GitHub run the following command:

pip install git+https://github.com/qodo-ai/qodo-cover.git

Binary

The binary can be run without any Python environment installed on your system (e.g. within a Docker container that does not contain Python). You can download the release for your system by navigating to the project's release page.

Repository Setup

Run the following command to install all the dependencies and run the project from source:

poetry install

Running the Code

After downloading the executable or installing the Pip package you can run the Cover Agent to generate and validate unit tests. Execute it from the command line by using the following command:

cover-agent \
  --source-file-path "<path_to_source_file>" \
  --test-file-path "<path_to_test_file>" \
  --project-root "<path_to_project_root>" \
  --code-coverage-report-path "<path_to_coverage_report>" \
  --test-command "<test_command_to_run>" \
  --test-command-dir "<directory_to_run_test_command>" \
  --coverage-type "<type_of_coverage_report>" \
  --desired-coverage <desired_coverage_between_0_and_100> \
  --max-iterations <max_number_of_llm_iterations> \
  --included-files "<optional_list_of_files_to_include>"

You can use the example code below to try out the Cover Agent. (Note that the usage_examples file provides more elaborate examples of how to use the Cover Agent)

Python

Follow the steps in the README.md file located in the templated_tests/python_fastapi/ directory to setup an environment, then return to the root of the repository, and run the following command to add tests to the python fastapi example:

cover-agent \
  --source-file-path "templated_tests/python_fastapi/app.py" \
  --test-file-path "templated_tests/python_fastapi/test_app.py" \
  --project-root "templated_tests/python_fastapi" \
  --code-coverage-report-path "templated_tests/python_fastapi/coverage.xml" \
  --test-command "pytest --cov=. --cov-report=xml --cov-report=term" \
  --test-command-dir "templated_tests/python_fastapi" \
  --coverage-type "cobertura" \
  --desired-coverage 70 \
  --max-iterations 10

Go

For an example using go cd into templated_tests/go_webservice, set up the project following the README.md. To work with coverage reporting, you need to install gocov and gocov-xml. Run the following commands to install these tools:

go install github.com/axw/gocov/gocov@v1.1.0
go install github.com/AlekSi/gocov-xml@v1.1.0

and then run the following command:

cover-agent \
  --source-file-path "app.go" \
  --test-file-path "app_test.go" \
  --code-coverage-report-path "coverage.xml" \
  --test-command "go test -coverprofile=coverage.out && gocov convert coverage.out | gocov-xml > coverage.xml" \
  --test-command-dir $(pwd) \
  --coverage-type "cobertura" \
  --desired-coverage 70 \
  --max-iterations 1

Java

For an example using java cd into templated_tests/java_gradle, set up the project following the README.md. To work with jacoco coverage reporting, follow the README.md Requirements section: and then run the following command:

cover-agent \
  --source-file-path="src/main/java/com/davidparry/cover/SimpleMathOperations.java" \
  --test-file-path="src/test/groovy/com/davidparry/cover/SimpleMathOperationsSpec.groovy" \
  --code-coverage-report-path="build/reports/jacoco/test/jacocoTestReport.csv" \
  --test-command="./gradlew clean test jacocoTestReport" \
  --test-command-dir=$(pwd) \
  --coverage-type="jacoco" \
  --desired-coverage=70 \
  --max-iterations=1

Record & Replay Feature

To save LLM service credits, a response recording mode is available. The starting point is a group hash, generated from the hashes of the source and test files used in each test run. If either file changes, the corresponding LLM responses should be re-recorded. Run the following command to execute all tests with LLM response recording enabled:

poetry run python tests_integration/run_test_all.py --record-mode

If you run the same command without the --record-mode flag:

poetry run python tests_integration/run_test_all.py

it will use the recorded responses to generate tests without calling the LLM if recordings are available. Otherwise, it will call the LLM to run the tests.

You may also record LLM responses from a separate test run. Run a test as you normally would, and add the --record-mode flag to the command:

poetry run python tests_integration/run_test_with_docker.py \
  --docker-image "embeddeddevops/python_fastapi:latest" \
  --source-file-path "app.py" \
  --test-file-path "test_app.py" \
  --code-coverage-report-path "coverage.xml" \
  --test-command "pytest --cov=. --cov-report=xml --cov-report=term" \
  --coverage-type "cobertura" \
  --model "gpt-4o-mini" \
  --record-mode

The table below explains the behavior of the test runner depending on whether the --record-mode flag is set and whether a recorded file already exists:

Flag	Record File	Result
❌	❌	Regular test run (file not recorded)
✅	❌	Records a new file
✅	✅	Overwrites an existing file
❌	✅	Replays a recorded file

Recorded responses are stored in the stored_responses folder. Files are named based on the test name and a hash value that depends on the contents of the source and test files.

<test_name>_responses_<hash_value>.yml

# i.e.
python_fastapi_responses_a9d9de927a82a7d776889738d2880bec7166c5f69d3518837183a20ef48b2a37.yml

A response file corresponding to the same source and test files group hash in a file name is updated during each recording session with new prompt hash entries. To regenerate it from scratch, you can delete the existing response file and run a new recording session.

Outputs

A few debug files will be outputted locally within the repository (that are part of the .gitignore)

• run.log: A copy of the logger that gets dumped to your stdout
• test_results.html: A results table that contains the following for each generated test:
  • Test status
  • Failure reason (if applicable)
  • Exit code,
  • stderr
  • stdout
  • Generated test

You can suppress logs using the --suppress-log-files flag. This prevents the creation of the run.log, test_results.html, and the test results db files.

Additional logging

If you set an environment variable WANDB_API_KEY, the prompts, responses, and additional information will be logged to Weights and Biases.

Using other LLMs

This project uses LiteLLM to communicate with OpenAI and other hosted LLMs (supporting 100+ LLMs to date). To use a different model ot