What is a Skill?

A Skill is a reusable operation for robotics, computer vision, and Physical AI. Skills span 2D/3D perception (6D pose estimation, 2D/3D detection, segmentation, image processing), motion planning (RRT*, motion generators, trajectory optimization), and motion control (model predictive control, reinforcement learning policies). Skills can be chained into pipelines to build real-world robotics applications.

Example 1: Segment image using SAM — docs

from telekinesis import cornea                                # Import Cornea - Image segmentation module

# Executing a 2D image segmentation Skill                     
result = cornea.segment_image_using_sam(                      # Executing Skill - `segment_image_using_sam`
    image=image,
    bboxes=[[400, 150, 1200, 450]]
)
# Access results
annotations = result.to_list()                

Example 2: Detect objects using RF-DETR — docs

from telekinesis import retina                                   # Import Retina - Object detection module

# Executing a 2D object detection Skill
annotations, categories = retina.detect_objects_using_rfdetr(    # Executing Skill - `detect_objects_using_rfdetr`
    image=image,
    score_threshold=0.5,
)
# Access results
annotations = annotations.to_list()
categories = categories.to_list()  

Skills are organized in Skill Groups. Each can be imported from the telekinesis library:

from telekinesis import cornea     # image segmentation skills
from telekinesis import retina     # object detection skills
from telekinesis import pupil      # image processing skills
from telekinesis import vitreous   # point cloud processing skills
from telekinesis import synapse  # robotics skills
from telekinesis import medulla    # hardware communication skills
from telekinesis import illusion   # synthetic data generation skills
from telekinesis import iris       # AI model training skills

Cornea: Image segmentation

from telekinesis import cornea

• Color-based segmentation: RGB, HSV, LAB, YCrCb
• Region-based segmentation: Focus region, Watershed, Flood fill
• Deep learning segmentation: BiRefNet (foreground), SAM
• Graph-based segmentation: GrabCut
• Superpixel segmentation: Felzenszwalb, SLIC
• Filtering: Filter by area, color, mask
• Thresholding: Global threshold, Otsu, Local, Yen, Adaptive, Laplacian-based

Retina: Object detection

from telekinesis import retina

• Classical shape detection - Hough Transform, Contours
• 2D Object detection - YOLOX, RF-DETR
• Open-Vocabulary detection - Qwen-VL, Grounding DINO

Pupil: 2D image processing

from telekinesis import pupil

• Morphology: erode, dilate, open/close, gradient, top-hat
• Structure: Frangi, Hessian, Sato, Meijering
• Edges: Sobel, Scharr, Laplacian, Gabor
• Denoising: Gaussian, median, bilateral, box filters
• Enhancement: CLAHE, gamma correction, white balance
• Transforms: pyramids, mask thinning

Vitreous: 3D point cloud processing

from telekinesis import vitreous

• Point cloud: centroids, normals, bounding boxes, principal axes
• Filtering: masks, outliers, downsampling, plane & cylinder removal
• Segmentation: DBSCAN, density, color, plane-based clustering
• Transforms: rigid transforms, scaling, projection
• Registration: ICP (P2P, P2Plane), global registration, cuboid sampling
• Meshes: shapes, mesh to point cloud, convex hull, Poisson reconstruction

Illusion: Synthetic data generation

from telekinesis import illusion

• Synthetic image data generation for AI model training

Iris: AI model training and deployment

from telekinesis import iris

• AI model training pipelines • Fine-tuning and evaluation of foundation models

Synapse: Robotics

from telekinesis import synapse

• Kinematics: forward & inverse kinematics, collision-free IK, IK with seed/motion profile, solver setup, default joint configuration
• Motion: Cartesian & joint moves, joint-space Cartesian targets, freedrive, teach mode, jog, move-until-contact, motion stop, protective stop
• Servo control: Cartesian, joint, and circular servoing
• Force control: tool-contact detection
• State reading: TCP pose/force/speed, joint positions/velocities/torques, target values, timestamp, connection status
• Robot statuses: robot mode, safety mode, runtime state, controller frequency
• Diagnostics: speed scaling, target speed fraction
• Tools (grippers): open/close, move, set force/speed/position range/unit, read current position
• Visualization & model: visual & collision meshes, link transforms
• Supported robots: ABB, Fanuc, Franka Emika, KUKA, Motoman, Neura, Universal Robots (no-hardware quickstarts included)
• Supported tools: parallel grippers — Robotiq 2F-85, OnRobot RG6

What is a Physical AI Agent?

Recent advances in LLMs and VLMs have shown the potential of learned models to perform semantic reasoning, task decomposition, and high-level planning from vision and language inputs.

In the Telekinesis library, a Physical AI Agent—typically a Vision Language Model (VLM) or Large Language Model (LLM)—autonomously interprets natural language instructions and generates high-level Skill plans. In autonomous Physical AI systems, Agents continuously produce and execute Skill plans, allowing the system to operate with minimal human intervention.

To learn more, explore Tzara.

How do Physical AI Agents use Skills?

Telekinesis Agentic Skill Library Architecture

Flow Overview

1. A user prompts the robot with a task: "Hey, can you do a CNC machine loading task?"
2. The Agent interprets the intent and reasons over the given task and sequencing.
3. The Agent constructs a high-level plan and orchestrates the Skills—selecting order and parameters.
4. Skills are executed on the robot using production-grade perception, motion, and control algorithms, with continuous feedback from the environment.

What Can You Build?

Telekinesis Agentic Skill Library helps you build real-world robotics and Physical AI applications for industries such as manufacturing, automotive, aerospace, and others. Below are use cases the Telekinesis team has deployed using the skill library.