Rafx

Rafx is a multi-backend renderer that prioritizes performance, flexibility, and productivity. It optionally integrates with the distill asset pipeline to provide workflows and tools suitable for real-world projects with multidisciplinary teams.

This crate contains several layers:

• rafx-api: Low-level graphics API abstraction
• rafx-visibility: Low-level retained API for visibility determination
• rafx-framework: Mid-level framework that eases resource management, lifetime handling, and draw call dispatching
• rafx-assets: Asset layer that integrates with distill
• rafx-renderer: A pipelined renderer with a plugin system for customizing it
• rafx-plugins: Implementations of many basic and advanced rendering techniques and features

Rafx also provides tools for building shaders and assets.

Documentation

Amazon Lumberyard Bistro scene (Original assets: https://developer.nvidia.com/orca/amazon-lumberyard-bistro)

^ Video of this renderer running on iOS hardware

(Tilemap screenshot is an imported ldtk project file)

Current Support:

Rafx supports most mainstream platforms. Proprietary platforms can be supported by adding an additional backend.

Platform	Vulkan	Metal	DX12	GL ES 3.0 (4,5,6)
Windows	✅		✅	✅
macOS	✅	✅		✅
Linux	✅ (1)			✅ (1)
iOS	✅	✅		🟥 (3)
Android	🟨 (2)			🟥 (3)
WebGL (6)				✅

Caveats:

1. Some backend/driver/window manager combinations might not work. Most recently tested with Ubuntu 20.04 LTS.
2. Android might work with vulkan but has not been tested. (It likely can be made to work)
3. Mobile OpenGL devices might work but may need improvements to create the initial OpenGL graphics context.
4. Limitations in the OpenGL ES 3.0 API prevent full implementation. The main demo does not fully work in GL ES 2.0/3.0.
5. OpenGL ES 3.0 backend still uses 2.0 paths for some functionality. There is room for improvement!
6. OpenGL/WebGL will not receive much improvement/support going forward.

Please keep in mind, this crate is still in pre-0.1.0 status!

• Documentation
• For rustdoc: run cargo doc --no-deps --open in the root of the crate (docs.rs is incomplete because it does not build with features)

Roadmap

• The API of rafx-api is unlikely to change significantly
• Other higher level crates are being refactored/improved based on demo improvements and dogfooding in other projects
• For the near-term future, the focus will be on:
  • Improvements to the framework (visibility, multithreading, API ergonomics)
  • Exploring options that would allow adding rendering features in a modular way, letting end-users more easily share their work.
  • Maturing the existing backends and adding new backends
  • Extending the demo with more rendering techniques

This crate will use semantic versioning rules after 0.1.0 of this project has been published. The 0.0.x versions do not follow semantic versioning rules.

Safety

Rafx is unsafe because hardware-accelerated rendering is fundamentally unsafe. However, we reserve usage of the unsafe keyword for when an operation can produce undefined behavior for reasons other than interacting with the GPU. Please see these documents for more detail:

• https://github.com/aclysma/rafx/blob/master/docs/api/api_design.md#safety
• https://github.com/aclysma/rafx/blob/master/docs/api/safety.md

Prerequisites

CMake must be available on your path. It is used by SDL2 in the demo and by upstream crates used within the shader pipeline.

Running the Demo

The demo should be run from within the demo directory of the repository. These commands should clone the repo and run the demo.

git clone https://github.com/aclysma/rafx.git
cd rafx
cd demo
cargo run --bin demo --release --features "[BACKEND_FEATURE]"

BACKEND_FEATURE should either be rafx-vulkan, rafx-metal, or some other backend.

Running in release reduces logging and disables GPU validation. The first time it will load more slowly because it has to import the assets, including a GLTF mesh with large textures. Using profile overrides to optimize upstream crates is highly recommeneded. Asset processing is extremely slow in debug mode. (i.e. 30s instead of 2s)

Demo Features

• Render thread decoupled from main thread (diagram)
• Shader build pipeline (diagram)
• Offline texture compression and mipmap generations (via basis-universal format)
• Asynchronous asset loading
• Assets can be streamed to remote hardware (i.e. a phone)
• OR assets can be cooked into a binary blob for shipping
• Hot-reloading assets (needs more work, some asset types do not work reliably)
• Render graph can be used for efficient and flexible definition of a render pipeline, including rendering to textures, multiple cameras, and compute shaders.
• Auto-generated shader bindings make working with descriptor sets convenient and less error prone.
• Material System supporting multiple passes
• Multi-camera support (to produce shadow maps, for example)
• Demo game state stored in ECS (NOTE: demo uses legion but the renderer is ECS-agnostic)
• PBR Meshes
• Sprites
• Debug Draw
• imgui/egui
• Clustered Forward Lighting with shadow atlas cache, supporting 100+ shadow-casting lights
• HDR Pipeline with Bloom/Auto-exposure/TAA/SSAO/FidelityFX CAS
• HDR display support (only tested on macOS, might work with vulkan)
• Point, Spot, and Directional Lights
• Multiple Spot/Directional/Point light soft shadows
• Font Rendering
• LDTK tilemap import
• Frustum culling for multiple cameras in 2D or 3D, in both orthogonal or perspective projections

Tools

The renderer includes a few tools for processing shaders and packing data in a binary blob.

There is also a blender add-on that works, but is still being prototyped and is not yet documented.

Shader Processor

This tool parses GLSL and produces matching rust code. This makes working with descriptor sets easier and safer!

• The tool is located at /shader-processor
• The demo includes a shaders crate to compile the generated rust code. It's located at /demo/shaders. Just the rust code is auto-generated, not the Cargo.toml.
• The easiest way to "refresh shaders" in the demo is to hit compile.bat or compile.sh in that folder
• You can cargo install rafx-shader-processor. Be sure to install the same version as you include in your project. Also, don't forget to update it if you update your project's dependencies!

The shader processor produces the following assets artifacts

• API-specific such as compiled SPIR-V or metal source code
• Metadata used by rafx at runtime
• Rust code that makes working with the shader easier
• Intermediate formats for debugging/inspection

The shader package can be loaded as an asset and contains everything needed to load a compiled shader.

Packaging Assets

This tool currently is only useful for packing assets.

• Pack files like this: run --package cli -- --pack out.pack
• Run the demo like this: run --package demo -- --packfile out.pack
• You'll likely want to create a binary from your own main.rs for this tool. That way you can add your own resource types.

Blender Add-On

There is also a blender addon that implements workflows for larger levels and exports a custom format that avoids duplicating shared asset data. While it functions, it is still early in implementation and not yet documented.

• Exporting: It automatically exports textures, materials, meshes, models (set of meshes with LOD levels), and scenes in easily-consumed formats, avoiding duplicating shared data, with pre-defined directory layout and export settings. This makes the export workflow easy and reliable.
• Level Editing: Supports a workflow that can handle massive levels in a team environment. Levels can be sliced by location or function and loaded individually. This allows creating levels larger than a DCC software can support, and allows more granular file checkouts, leading to fewer merge conflicts. In contrast to most DCC file formats, level and slice files are stored as text, making them diffable and mergeable.

^ Video of slice-based level editing in Blender

This video shows a scene with >5k objects, 1M+ verts being loaded at once in blender and sliced into multiple smaller pieces on an M1 mac mini.

Features

• rafx-vulkan: Use the vulkan backend
• rafx-metal: Use the metal backend
• framework: Include and re-export rafx-framework from the rafx crate
• assets: Include and re-export rafx-assets from the rafx crate
• renderer: Include and re-export rafx-renderer from the rafx crate

Crates

• rafx-base - Shared helpers/data structures. Nothing exciting
• rafx-api - Rendering API abstraction layer.
  • Vulkan backend for windows/linux
  • Metal backend for macOS/iOS
• rafx-visibility - Retained visibility API.
  • Simple handle-based abstraction.
  • Fast, thread-safe visibility queries.
• rafx-framework - Resource management for images, buffers, descriptor sets, etc.
  • Most things are hashed and reference counted
  • Provides a render graph
  • Nearly all assets are data-driven from serializable and hashable structures rather than hard-coded.
  • Buffers and images are asynchronously uploaded on dedicated transfer queue when available
  • Multi-pass material abstraction with bindable parameters
  • Render Features and Jobs system - Inspired by the 2015 GDC talk "Destiny's Multithreaded Rendering Architecture."
    • A job system with extract, prepare, and write phases
    • Rendering is pipelined with simulation thread, and the job structure is parallelizable by the application
    • Handles multiple views and phases allowing advanced features like shadow mapping
    • Flexible sorting mechanism for interleaving and batching write commands from multiple rendering features
  • Visibility Region - Built on top of rafx-visibility
    • Higher-level visibility API, integrated with framework render jobs
• rafx-assets - An asset loading and management system.
  • Assets can hot reload from files (but see #14)
  • Because distill pre-processes and stores cached assets as they change, custom processing/packing can be implemented while maintaining extrem