DeepExtract - PE Context Extraction Framework

Overview

DeepExtract is an IDA Pro 9.x plugin that bridges the gap between compiled PE binaries and AI coding agents. It extracts the full structural context of a binary (PE metadata, function signatures, disassembly, decompiled C/C++ code, cross-references, control flow, and more), and writes it in formats that agents like Claude Code, Codex, and Cursor can directly ingest and reason over. The goal is to let these agents read, navigate, and understand PE files the same way they understand source code repositories, enabling AI-assisted vulnerability research and reverse engineering at scale.

Results are written to a per-binary SQLite database, generated as C++ source files organized by module, and optionally as assembly (.asm) files, with rich per-function metadata headers designed for AI agent consumption. DeepExtract operates in two modes: headless for automated batch processing of large binary datasets, and interactive for targeted single-binary analysis within the IDA GUI.

For a deep dive into the motivation and design journey behind DeepExtract, from the initial experiments with raw decompiled C++ dumps through the challenges of making agents reason over disconnected binary data, to the structured extraction pipeline and agent runtime architecture, read the full writeup: Making Compiled Binaries Accessible to AI Coding Agents. The post covers why traditional agent code-navigation tools break on decompiler output, what led to the SQLite-backed structured approach, and how the DeepExtractRuntime extends coding agents with vulnerability research capabilities.

How It Works

The extraction pipeline has three stages:

PE Binary (.exe/.dll/.sys)
        |
        v
  IDA Pro 9.x + DeepExtract Plugin
  (disassembly, decompilation, analysis)
        |
        v
  Structured Output:
    - SQLite database (per-binary)
    - C++ source files (optional, --generate-cpp)
    - Assembly files with AI metadata headers (optional, --generate-asm)
    - JSON metadata (module_profile, function_index, file_info)

Stage 1 - Binary Loading. IDA Pro loads the PE file, performs auto-analysis, and builds its internal database (IDB). If PDB symbols are available, IDA resolves function names and type information.

Stage 2 - Extraction. DeepExtract iterates over every function in the IDB and extracts:

• Disassembly and Hex-Rays decompiled code (when available)
• Inbound and outbound cross-references
• Stack frame metrics, string literals, global variable accesses
• Dangerous API calls, loop structures, vtable analysis (experimental), indirect call targets

At the file level, it extracts PE headers, imports, exports, sections, security features, Rich header data, TLS callbacks, and .NET CLR metadata.

Stage 3 - Output. All extracted data is written to a SQLite database. When C++ generation is enabled (--generate-cpp), the plugin groups decompiled functions into source files organized by class and module, and generates JSON metadata files (function_index.json, module_profile.json, file_info.json) alongside a human-readable report (file_info.md). When assembly generation is enabled (--generate-asm), the plugin writes .asm files containing raw disassembly grouped by class membership and address order, with structured per-function headers (callers, callees, strings, dangerous APIs) for AI agent analysis. Library/CRT functions are separated into dedicated files. Assembly output is independent of decompiler availability.

Key Concepts

• PE (Portable Executable): The binary format used by Windows for executables (.exe), dynamic libraries (.dll), and drivers (.sys). PE files contain code sections, import/export tables, resource data, and metadata headers.

• IDA Pro: A disassembler and reverse engineering platform. IDA loads a binary, identifies functions, resolves cross-references, and builds a navigable representation of the code. The analysis results are stored in an IDA database (.idb / .i64).

• Hex-Rays Decompiler: An IDA Pro component that converts disassembled machine code into a C-like pseudo-code representation. DeepExtract stores this output as decompiled_code for each function.

• Cross-references (xrefs): Records of which functions call which other functions. Inbound xrefs list all callers of a function; outbound xrefs list all functions it calls. These form the binary's call graph.

• Headless mode: Running IDA Pro from the command line without a GUI, using idat.exe / idat64.exe with the -A flag. DeepExtract detects this mode and runs the full extraction pipeline automatically on startup.

• Structured output: The SQLite database and optional C++ files produced by DeepExtract. The database contains three tables (file_info, functions, function_xrefs) and is designed for programmatic queries by SQL, Python, or AI agent frameworks.

Quick Start

Install the plugin:

hcli plugin install DeepExtract

Extract a single binary (headless):

"C:\Program Files\IDA Professional 9.2\idat.exe" -A -L"C:\output\log.txt" -S"main.py --sqlite-db C:\output\kernel32.db --generate-cpp" "C:\Windows\System32\kernel32.dll"

Batch-extract a directory:

.\headless_batch_extractor.ps1 -ExtractDirRecursive "C:\Windows\System32" -StorageDir "C:\Analysis"

Interactive mode: Open a binary in IDA Pro, then Edit > Plugins > DeepExtract (or Ctrl-Shift-E).

Extraction Capabilities

The extractor operates at two levels: file-level metadata and per-function analysis.

File-Level Metadata

The plugin extracts 30+ metadata points per binary:

• Identification: MD5, SHA256, file size, extension
• PE headers: Sections, entry points, Rich header (linker toolchain data), TLS callbacks
• Version information: Product name, company name, copyright, original filename, PDB path
• Security features: ASLR, DEP/NX, CFG, SafeSEH status, DLL characteristics
• Runtime environment: .NET assembly detection, CLR metadata, delay-load DLL imports

Function-Level Analysis

For every identified function:

• Signatures: Base and extended function signatures, including demangled and mangled names
• Code: Full disassembly and Hex-Rays decompiled C/C++ output
• Cross-references: Inbound and outbound xrefs in full and simplified formats, plus a deduplicated relational table for SQL-based call graph queries
• Dangerous API detection: Matches outbound calls against 480+ security-critical APIs (e.g., CreateRemoteThread, LoadLibrary, CreateProcessW)
• String literals: Per-function string references
• Global variable accesses: Read/write references to global data
• Stack frame analysis: Aggregate frame sizes (locals, arguments, saved registers), frame pointer and exception handler flags, and stack canary detection via multi-heuristic analysis (variable names, security cookie calls, XOR patterns)
• Loop analysis: Natural loop detection via dominator-based back edges with SCC fallback for irreducible control flow, per-loop cyclomatic complexity, and infinite loop heuristic (zero exit edges)
• VTable analysis (experimental): Virtual call resolution for [reg+offset] call patterns, vtable slot inspection, and per-class method grouping from demangled names. Limited to single vtable at object offset 0; no multiple/virtual inheritance, thunk handling, or RTTI-based class hierarchy inference
• Indirect call resolution (experimental): Backward pattern matching to resolve common indirect call and jump patterns (register loads, memory dereferences, function pointer arrays), jump table detection via IDA's switch analysis with manual fallback, and basic obfuscation handling (XOR/ADD/SUB transforms). Coverage is limited to intra-procedural heuristics within a short instruction window

Output Architecture

SQLite Database

Each binary produces a SQLite database containing three tables.

file_info stores binary-level metadata:

• file_path, file_name, file_extension, file_size_bytes
• md5_hash, sha256_hash
• imports, exports, entry_point
• file_version, product_version, company_name, pdb_path
• rich_header, tls_callbacks, is_net_assembly, clr_metadata
• dll_characteristics, security_features, exception_info

functions stores per-function analysis data:

• function_signature, mangled_name, function_name
• assembly_code, decompiled_code
• inbound_xrefs, outbound_xrefs (full and simple JSON)
• vtable_contexts, global_var_accesses, dangerous_api_calls
• string_literals, stack_frame, loop_analysis
• analysis_errors, created_at

function_xrefs stores deduplicated cross-references for SQL-based call graph queries:

• source_id, target_id (foreign keys into functions)
• target_name, target_module
• function_type (generic, library, API, vtable, etc.)
• xref_type, direction (inbound/outbound)
• Unique constraint on (source_id, target_id, target_name, target_module, xref_type, direction)

C++ Output (Optional)

When --generate-cpp is enabled, the plugin writes decompiled functions as grouped C++ source files:

• Class methods: Grouped by class into files of approximately 450-500 lines, named {module}_{class}_group_N.cpp. Methods are ordered alphabetically; each is preceded by a comment block with its name and signature.
• Standalone functions: Grouped into files of approximately 450-500 lines, named {module}_standalone_group_N.cpp, with the same ordering and comment conventions.
• function_index.json: Maps every function name to its .cpp file and library tag (WIL, STL, WRL, CRT, ETW/TraceLogging, or null for application code). See the Function Index Format Reference.
• module_profile.json: Pre-computed module fingerprint covering identity, scale, library composition, API surface, complexity metrics, and security posture. See the Module Profile Format Reference.
• file_info.md / file_info.json: Human-readable and machine-readable analysis reports. See the Analysis Metadata and Reports Reference.

Only functions with real Hex-Rays pseudocode are included in C++ output. Functions where decompilation failed (license unavailable, timeout, empty output, or decompiler returned None) are excluded from .cpp files and correctly reported as failed in module_profile.json and function_index.json.

Assembly Output (Optional)

When --generate-asm is enabled, the plugin writes raw disassembly into grouped .asm files. This output is independent of the Hex-Rays decompiler; it works with any IDA license that supports disassembly. Assembly files are written alongside C++ files in the same extracted_code/<module>/ directory by default, or to a custom directory via --asm-output-dir.

• Class methods: Grouped by class name (sorted by address within each class), named {module}_{class}_group_N.asm.
• Standalone functions: Sorted by address order and split at approximately 2000-2500 lines per file, named {module}_standalone_group_N.asm.
• Library/CRT functions: Separated into dedicated {module}_library.asm file(s) to avoid polluting application code analysis.

Each function is preceded by a structured metadata header:

; ============================================================
; Function: sub_401000
; Address: 0x401000
; Signature: int __cdecl sub_401000(int a1, int a2)
; Callers: WinMain (id:1), sub_402300 (id:45)
; Callees: WSAStartup [WS2_32.dll], socket [WS2_32.dll], sub_4015A0 (id:12)
; Strings: "Failed to connect", "192.168.1.1"
; Dangerous APIs: connect, WSAStar