ORLA

Usage

1. Enter a text prompt describing the 3D model you want to generate (e.g., "a futuristic sports car")
2. Click "Generate" and wait for the AI to process your request through Stability AI's image generation API
3. Review the generated image optimized for 3D conversion (with white background and clear object definition)
4. Select the image to initiate 3D model creation through Hunyuan3D-2
5. Interact with your 3D model using mouse controls (orbit to rotate, scroll to zoom, right-click to pan)

Image Generation Process

ORLA uses advanced AI models to generate 3D assets through a multi-step process:

1. Text Prompt Processing: Your text prompt is enhanced with specific parameters optimized for 3D modeling
2. AI Image Generation: The backend uses Stability AI's Stable Diffusion 3.5 Large Turbo model to generate high-quality 2D renders
3. Image Optimization: Generated images are processed with specific attributes (isolated objects, solid background, uniform lighting, etc.)
4. 3D Model Creation: Selected 2D images are transformed into 3D models using Tencent's Hunyuan3D-2 neural network model
5. Real-time Visualization: The 3D model is rendered in the browser using Three.js for interactive viewing

The technical workflow includes:

• Text prompts sent to FastAPI endpoints
• Enhanced prompts include specifications for optimal 3D conversion (low-poly style, sharp edges, white background, etc.)
• Celery workers process the requests asynchronously with Stable Diffusion API
• Generated images are saved and served from the server's static directory
• Selected images are processed by Hunyuan3D-2 to create detailed 3D models
• Redis manages the task queue for efficient processing of multiple requests

Project Structure

ORLA/
├── client/                # Frontend React application
│   ├── public/            # Static assets
│   └── src/
│       ├── assets/        # Images and other assets
│       ├── components/    # React components
│       ├── layout/        # Layout components
│       └── pages/         # Page components
└── server/                # Backend FastAPI application
    ├── app/
    │   ├── main.py        # FastAPI application
    │   ├── tasks.py       # Celery tasks
    │   └── model_generator.py # 3D model generation
    ├── Dockerfile         # Docker configuration
    └── docker-compose.yml # Docker Compose configuration

Development

Running the Frontend Locally

cd client
npm run dev

Running the Backend Locally

cd server
pip install -r requirements.txt
uvicorn app.main:app --reload

In a separate terminal:

# Start Redis
docker run -p 6379:6379 redis
# Start Celery worker
cd server
celery -A app.tasks worker --loglevel=info

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgements

• Stable Diffusion 3.5 by Stability AI for image generation
• Hunyuan3D-2 by Tencent for the 3D model generation
• React Three Fiber for 3D rendering in React
• FastAPI for the backend API