CLERC Épée v0.2

The first AI-grade sign language data layer.

A multi-signer ASL keypoint corpus designed for AI training, research benchmarking, and inter-signer variability studies. v0.2 expands the signer pool from 3 to 4 and the corpus from 300 to 600 clips, in a fully parallel structure — every one of 150 phrases is signed by all four Deaf signers.

CLERC builds the data layer underneath sign language AI — not a translation tool, not an accessibility app. Infrastructure.

---

Dataset Summary

• 600 ASL clips — 150 unique phrases × 4 Deaf signers (fully parallel structure)
• Inter-signer parallel structure — identical phrases across all four signers for direct variability analysis
• Multimodal keypoints — hands, body, eyes, mouth, head silhouette (MediaPipe-extracted)
• Linguistically validated — ASL gloss annotations with temporal segmentation

This release ships extracted keypoints and annotations only — no raw video. Source clips remain proprietary; access is reserved for commercial licensing (contact florian@clerc.io).

This is v0.2, a pilot release representing a portion of the full CLERC catalog. Full corpus access available via commercial license.

---

Benchmark — why multi-signer data matters

A small BiLSTM trained on the four release signers and tested on signers held entirely outside the training set shows the core result: one signer does not generalize to a stranger, four do.

• Train on 1 → 2 → 3 → 4 signers, tested on a brand-new signer: 22% → 40% → 50% → 59% accuracy (macro-F1 0.13 → 0.38).
• More data keeps lifting it: 23% → 61% as training examples grow, and the curve is not saturated.
• Tested on two held-out signers (no leakage), 24 shared glosses, 8 seeds.

Full method, numbers, and honest caveats: BENCHMARK.md.

---

Dataset Statistics

Top 10 glosses (cumulative coverage of corpus):

---

Languages

• American Sign Language (ASL) — ISO 639-3: ase
• Written translations in English

---

Dataset Structure

epee/
├── keypoints/           # 600 .npy arrays, shape (n_frames, 128, 3)
├── annotations/         # 600 .json files
└── metadata.csv         # master index (1 row per clip)

Keypoint layout (128 landmarks per frame)

The 36 head-silhouette landmarks come from MediaPipe FaceMesh FACE_OVAL indices 10, 338, 297, 332, 284, 251, 389, 356, 454, 323, 361, 288, 397, 365, 379, 378, 400, 377, 152, 148, 176, 149, 150, 136, 172, 58, 132, 93, 234, 127, 162, 21, 54, 103, 67, 109 (in that traversal order). The points form a closed polygon outlining the head — no internal facial features are included, so the privacy stance is preserved.

Coordinate space

Coordinates are MediaPipe's image-normalized space, NOT clipped to [0, 1]:

• x is in [0, 1] (frame width); a landmark extrapolated just off-frame can fall slightly outside [0, 1]
• y is in [0, 1] for points visible in frame, but can exceed 1.0 for body landmarks extrapolated below the visible frame
• z is depth relative to the hips, roughly in MediaPipe Pose's world-scale units

Source clips are framed waist-up. Lower-body landmarks (dataset indices 54–63) come from MediaPipe Pose's full-body prediction. For hand/face-only SLR pipelines, they can be dropped:

kp_slr = np.concatenate([kp[:, :54], kp[:, 64:]], axis=1)  # → (n_frames, 118, 3)

Zero values (0, 0, 0) indicate a landmark was not detected for that frame (e.g. an off-screen hand).

Gloss conventions

Glosses (uppercase ASL labels) follow a few conventions worth knowing before training:

Base gloss — WHAT, YOU, BATHROOM. The standard form of a sign.

Variants — BASE_N (e.g. SIGN_2, WHAT_3). Alternative ways to sign the same English concept (different handshape, location, or movement). The number N is an internal disambiguator, not an intensity marker. Treat WHAT, WHAT_2, WHAT_3 as siblings sharing the same English target.

Directional / movement suffixes — POINTER_RIGHT, GO_LEFT, HOW_RIGHT_MOVE. These mark spatial/movement components inherent to the sign and should not be collapsed with their base form.

Phrase repetitions — Some clips contain the target phrase signed more than once (emphasis, demonstration, self-correction). Each occurrence is a separate gloss segment. This is natural signer behavior, not a labeling error.

Recommended preprocessing

import re
def base_gloss(g):
    return re.sub(r"_\d+$", "", g)   # SIGN_2 → SIGN

from collections import Counter
def has_repeat(segments):
    return any(c >= 2 for c in Counter(s["gloss"] for s in segments).values())

Annotation schema (per clip)

{
  "clip_id": "clerc_v02_001",
  "signer_id": "ALPHA",
  "sign_language": "ASL",
  "text_en": "What's up?",
  "fps": 30.0,
  "n_frames": 139,
  "segments": [
    { "gloss": "WHAT'S UP", "start": 0.9, "end": 1.4 },
    { "gloss": "QUESTION",  "start": 2.0, "end": 2.8 }
  ]
}

---

Signers

Demographic distribution: 2 female / 2 male, all between 30–40 years old. All native ASL signers (Deaf, ASL as first language). Signer identities are pseudonymized.

Signers participated under written informed consent. The signing space, framing, lighting, and recording protocol were standardized across signers.

Parallel structure: all four signers sign the same 150 phrases. The clip blocks are phrase-aligned: clerc_v02_001, _151, _301, _451 are the four signers' renderings of phrase #1, and so on — enabling direct inter-signer comparison.

Stylistic note: Phrase repetition rates vary by signer — natural inter-signer stylistic variation, annotated as separate gloss segments. See gloss conventions for filtering.

---

Intended Use

Designed for

• Inter-signer variability analysis (style, rhythm, signing space)
• Research on sign language linguistics, gesture recognition, multimodal AI
• Educational use in academic settings
• Prototyping sign language recognition (SLR) pipelines on a parallel multi-signer corpus

Not designed for

• Speaker identification or biometric applications
• Surveillance or evaluation of individual signers

For production-grade systems or sign language generation models trained at scale, see commercial licensing for access to the full multi-signer corpus.

---

Loading the Dataset

This release ships as plain .npy + .json files for transparency and zero-dependency loading.

import json
import numpy as np
import pandas as pd
from pathlib import Path
from huggingface_hub import snapshot_download

ROOT = Path(snapshot_download(repo_id="CLERC-DATA/epee", repo_type="dataset"))

metadata = pd.read_csv(ROOT / "metadata.csv")

clip_id = "clerc_v02_001"
with open(ROOT / "annotations" / f"{clip_id}.json") as f:
    annotation = json.load(f)
keypoints = np.load(ROOT / "keypoints" / f"{clip_id}.npy")

hands       = keypoints[:, :42]
upper_body  = keypoints[:, 42:54]
face_inner  = keypoints[:, 64:92]
head_oval   = keypoints[:, 92:128]

---

License

CC BY-NC-SA 4.0 — creativecommons.org/licenses/by-nc-sa/4.0

Commercial licensing: for enterprise use, training of commercial models, or integration into commercial products, contact florian@clerc.io.

---

Ethical Considerations

CLERC is Deaf-led infrastructure. This release adheres to:

• Informed consent — all signers have provided written consent for public release under this license
• Privacy protection — face landmarks restricted to non-identifying features (eyes + mouth + head outline); full biometric data excluded
• Community benefit — released to advance sign language technology research; commercial revenue supports continued Deaf-led data infrastructure
• No surveillance use — must not be used for individual identification, behavioral profiling, or signer surveillance

---

Limitations

• Pilot release — 600 clips is a baseline pilot, not a production-scale corpus
• 4 signers — limited inter-signer diversity; full catalog includes a broader signer pool
• Phrase domain — conversational/social phrases; not domain-specific (medical, legal, technical)
• Reduced face landmarks — full facial grammar (brow, cheeks, head tilt) not included
• Gloss only — no morphological, prosodic, or spatial annotation layers in v0.2

---

Versioning & Roadmap

---

How to Cite

@dataset{clerc_epee_v02_2026,
  author       = {M{\'e}loux, Florian and {CLERC}},
  title        = {{CLERC} {\'E}p{\'e}e v0.2: Sign Language Data Layer},
  year         = {2026},
  publisher    = {Hugging Face},
  version      = {0.2},
  doi          = {10.5281/zenodo.20268565},
  url          = {https://huggingface.co/datasets/CLERC-DATA/epee}
}

---

About CLERC

CLERC builds the data infrastructure that lets AI understand sign language as a first-class language — not an accessibility afterthought.

Sign language is not to be translated. It is to be inscribed.

Website: clerc.io · Contact: florian@clerc.io · LinkedIn: clerc-io

---

Changelog

v0.2 — June 2026

• Added 4th signer (DELTA) and expanded to 600 clips
• Restructured to 150 fully-parallel phrases × 4 signers (phrase-aligned clip blocks)
• Same 128 multimodal keypoints/frame and gloss schema as v0.1

v0.1 — May 2026

• Initial public release — 300 clips, 3 signers, parallel structure