Hand Gesture Datasets for HCI and XR AI
Diverse hand gesture and sign language corpora with keypoint annotation, demographic variation, and device-specific capture conditions. Built for production hand tracking, gesture control, and XR interaction AI.
20+
Years in AI training data
1,000+
Language locales
1M+
Hours of video annotated
ISO
27001 certified
The Challenge
Beyond Public Gesture Benchmarks
Jester and EgoGesture advanced gesture recognition research. Production hand tracking for XR, automotive HCI, and sign language recognition requires fine-grained gesture taxonomy, demographic diversity, and device-matched capture.
XR gesture control requires precise hand keypoint tracking across diverse skin tones, hand sizes, and lighting conditions. Sign language recognition requires signer diversity and linguistic accuracy. Public gesture datasets cover neither use case adequately.
Off-the-shelf gesture datasets suffer from demographic bias (limited skin tone and hand morphology diversity) and gesture taxonomy mismatches for product-specific interaction vocabularies.
LXT builds custom hand gesture datasets with your gesture vocabulary, demographic targets, and capture conditions. We deliver keypoint-annotated gesture corpora that generalize across the hand diversity your production system encounters.
Why Teams Upgrade
Limitations of Public Hand Gesture AI Datasets
Standard benchmarks serve research well. Production deployments need more.
| Dataset | Primary Limitation | Impact |
|---|---|---|
| Jester | 27 general gestures only; limited skin tone diversity; consumer webcam capture; no egocentric or device-specific perspectives | 27 gestures |
| EgoGesture | Egocentric only; 83 gestures but single camera rig; limited demographic diversity; RGB-D capture not always available | Ego-only |
| NVGesture | 25 gestures; controlled lab environment; limited background and lighting variation; NVIDIA sensor requirement | Lab conditions |
| Chalearn IsoGD | Isolated gestures from controlled collection; limited continuous gesture and natural transition coverage | Isolated gestures |
| HGR2B | Polish Sign Language focus; limited signer diversity; not applicable to other sign languages or HCI gestures | Single language |
Not sure which specs you need?
Our data specialists help you scope the right dataset for your model architecture.
Configurable Specifications
Specs Built Around Your Model
Public datasets come fixed. Yours is configured for your architecture, environment, and use case.
Gesture Vocabulary
Interaction Design
- Custom Gestures: Your product's gesture set with natural transition variants
- Signer Diversity: Multiple performers per gesture to capture morphological variation
- Continuous Sequences: Natural gesture transitions and co-articulation sequences
Capture Configuration
Device Matching
- Sensors: RGB, depth, IR, and stereo camera configurations
- Perspective: Egocentric, frontal, overhead, and device-matched viewing angles
- Environment: Controlled, semi-controlled, and real deployment backgrounds
Annotation Depth
Label Types
- Gesture Class: Per-frame or per-clip gesture label with start and end timestamps
- Hand Keypoints: 21-point hand skeleton for pose and motion-based gesture models
- Quality Flags: Occlusion, truncation, and ambiguity flags per annotated clip
Need a custom configuration?
We've built datasets across dozens of domains and use cases. Let's scope yours.
Capturing Complexity
Edge Cases in Gesture Recognition
High-accuracy models handle rare attributes that public datasets miss.
Skin Tone and Lighting Interaction
Dark skin tones in high-contrast or low-light environments cause depth sensor failures and tracking loss. Targeted collection across Fitzpatrick scale skin tones in varied lighting prevents these failures.
Hand Morphology Variation
Hand size, finger length ratio, and joint mobility vary across demographics. Multi-participant collection per gesture ensures training coverage across natural hand morphology variation.
Gesture Ambiguity and Co-Articulation
Natural gesture streams contain ambiguous transitions between gestures. Continuous sequence annotation with explicit transition labeling trains models to handle real-world gesture boundaries.
Occlusion from Object Manipulation
Hands holding objects or partially occluded by devices produce partial visibility. Explicitly collected object-in-hand and partial occlusion examples improve real-world gesture tracking robustness.
Ground Truth Quality
Human-in-the-Loop Annotation
Precise annotation bridges raw data and learnable signal. Expert annotators deliver precision automated tools can't match.
Gesture Class Annotation
Expert annotators label gesture class, start and end timestamps, and quality flags. Multi-annotator agreement on gesture boundary decisions verified before delivery.
Hand Keypoint Annotation
21-point hand skeleton annotation per frame with visibility and confidence labels. Supports both 2D keypoint and 3D hand pose model training.
Continuous Sequence Labeling
Frame-level annotations in continuous gesture streams with co-articulation and transition labeling for sequence model and real-time gesture streaming training.
Industry Applications
Hand Gesture Datasets for Your Domain
Custom taxonomies and collection protocols for specific deployment contexts.
XR and AR
Hand interaction, object manipulation, XR navigation
Automotive HCI
In-car gesture control, infotainment navigation
Sign Language
ASL, BSL, and custom sign language recognition
Touchless Computing
Air typing, desktop control, accessibility input
Gaming
Full-hand gaming interaction, motion control
Robotics
Human-robot handover, teleoperation gestures
Medical Devices
Sterile touchless control, surgical gesture
EdTech
Sign language learning, gesture-based interaction
Compliance & Ethics
Secure and Ethical Data Collection
Data collection involving people and sensitive content requires robust security, compliance, and ethical protocols at every stage.
Global Demographic Reach
Collection across 1,000+ locales and diverse demographics to prevent algorithmic bias in your deployed models.
ISO 27001 Certified
Sensitive projects processed in certified secure facilities meeting the highest information security standards.
GDPR & Privacy Compliance
All collection and annotation protocols vetted for consent and privacy. Legally robust for global deployment.
Frequently Asked Questions
Hand Gesture Dataset FAQs
Can you collect for our custom gesture vocabulary?+
Yes. We design collection protocols around your gesture set, including natural variation, transition sequences, and performer diversity targets. Gesture script design is included in project scoping.
How do you ensure skin tone diversity?+
We apply Fitzpatrick scale skin tone quotas to participant recruitment. Collection is tracked against quotas and blocked until all skin tone groups meet minimum per-class instance counts.
What sensors and perspectives can you support?+
RGB, RGB-D, IR, and stereo camera configurations. Egocentric, frontal, overhead, and device-matched perspectives. We can match your production sensor and viewpoint for minimal deployment shift.
Do you provide hand keypoint annotations?+
Yes. 21-point MediaPipe-compatible hand skeleton annotations per frame with visibility and confidence scores. 3D keypoint annotation available with depth sensor capture.
Can you collect sign language data?+
Yes. We collect sign language datasets with certified deaf signers for ASL, BSL, and other sign languages. Linguistic accuracy review by native signers is included.
What does a custom hand gesture dataset cost?+
Projects range from $15K for focused HCI gesture sets (500-2,000 sequences) to $100K+ for large sign language or multi-gesture XR corpora.
How do you handle participant consent for gesture and hand data?+
All participants provide informed consent under GDPR-compliant protocols. Consent covers use in AI model training and is documented in the delivery package.
Get Started
Scope Your Custom Hand Gesture Dataset
Share your gesture vocabulary, sensor configuration, and demographic requirements. A computer vision data specialist will provide a detailed proposal within 48 hours.
Please provide us with the details of your inquiry and one of our team members will be in touch.
Join our global team of contributors today
Apply here to be considered for future projects including data collection, annotation and transcription
Start application