Key Features

• User Recognition - Identifies users through facial recognition
• Wake Word Activation - Activates using configurable wake words
• Vision-based Q&A - Answers questions about what the camera sees
• Natural Speech Interaction - Processes spoken questions and provides spoken responses
• User Registration - Adds new users to the system for future recognition
• Session Logging - Maintains history of interactions in current session for reference and an all time history with all interactions with ADA.
• Real-time Visual Feedback - Displays system status and responses on screen

Getting Started

Required:

• OpenAI API key (Get one at OpenAI Platform)

Run the Bash script

The setup script will guide you through installation and prompt for your OpenAI API key.

From the root directory, run:

./setup_and_run.sh

The script will:

1. Set up a Python virtual environment
2. Install all required dependencies
3. Prompt you for your OpenAI API key
4. Launch the ADA application

[!IMPORTANT] ADA maintains a session history that records all Q&A interactions during your current session. This history is used to provide context to the model, allowing for more coherent multi-turn conversations. There is also a All time history log that includes all conversations with ADA up-to date. Additionally the User Detection module keeps images of all user faces it has regognized so far under the 'user_faces' subfolder, there is also a db that has the face encodings which are linked with the face image and aditional info. All face data, that is the face images and encodings that are used to identify the user are only stored locally and are not tracked or transmitted to anyone. Additionally all session history and all time history with the model is also kept locally and is not tracked. See the gitignore for all the files that are ingored under the (sensitive data) section for more info.

Using ADA with Wake Words

Once ADA is running, activate it using any of these wake words:

• "Hey ADA"
• "Hey A.D.A"
• "OK ADA"
• "Hello ADA"
• "ADA"
• "Hi"

After saying a wake word, ADA will detect your presence and begin listening for your questions. You can ask questions about what the camera sees, and ADA will analyze the image and provide spoken responses. See the terminal for all logs related to the system

[!IMPORTANT] Each interaction with ADA uses your OpenAI API credits. No conversation data is transmitted except to the OpenAI API during direct requests.

Install Python (if you don't have Python)

macOS:

brew install python3

Windows:

1. Download the installer from python.org
2. Run the installer and make sure to check "Add Python to PATH"
3. Verify installation by opening command prompt and typing python --version

Models and API's

• Vosk small, for voice recognition (see the Models folders readme for more info)
• GPT-4o: OpenAI's multimodal model that powers ADA's vision capabilities
  • Functionality: Processes both images and text to answer questions about visual content
  • Capabilities: Object recognition, text reading (OCR), scene analysis, counting, and logical reasoning
  • Cost Estimate:
    • Approximately $0.01-0.05 per question (varies with image size and complexity)
    • $5-10 for ~100-500 typical interactions
    • Based on OpenAI's pricing of $5 per 1M input tokens and $15 per 1M output tokens
    • Each image counts as approximately 1,000 tokens depending on resolution

System Requirements

• Python 3.10 or higher
• 4GB RAM minimum (8GB recommended)
• Webcam for user detection and visual analysis
• Microphone for voice input
• Internet connection for OpenAI API access
• OpenAI API key with access to GPT-4o

License

This project is licensed under the MIT License - see the LICENSE file for details.

• LICENCE