python math_solver.py

4. Interact with the Application

   • Live Feed
     • The application will display the live feed from your webcam.
   • Perform Hand Gestures
     • Draw: Use one finger to draw on the canvas.
     • Clear: Use one finger to clear the canvas.
     • Solve Math Problem: Use all fingers except the thumb to submit a math problem for solving.

Code Explanation

1. getHandInfo(img)

   • Purpose: Detects and tracks the user's hand in each frame of the webcam feed.
   • Implementation: Uses the HandTrackingModule to identify and monitor hand movements.

2. draw(info, prev_pos, canvas)

   • Purpose: Draws on the canvas based on the detected hand gestures.
   • Parameters:
     • info: Information about the detected hand gestures.
     • prev_pos: Previous position of the drawing.
     • canvas: Canvas to draw on.

3. sendToAI(model, canvas, fingers)

   • Purpose: Sends the drawn content to Google's Generative AI for math problem solving based on recognized gestures.
   • Parameters:
     • model: Instance of the Generative AI model.
     • canvas: Canvas containing the drawn content.
     • fingers: Array indicating the state of fingers (e.g., for gesture recognition).

Hand Detection Concept

Left-Right Hand Detection

Hand Gesture Detection

Results

Math Expression Detection & Canvas

License

• This project is licensed under the MIT License. See the LICENSE file for details.