





















Abstract:The reliability of clinical artificial intelligence (AI) depends on high-quality data, yet Electronic Health Records are often inconsistent with existing scientific knowledge. Current quality assessments are limited: they either focus on syntax or rely on labor-intensive manual rules to capture semantic nuances. To overcome these scalability barriers, we propose Medical Data Pecking, a methodology that adopts software unit testing principles for medical data validation. It introduces Semantic Data Coverage, employing Large Language Models to generate context-aware tests that "peck" for inconsistencies between observed data and epidemiological evidence. To demonstrate this methodology, we implemented a reference tool using a Retrieval-Augmented Generation architecture that synthesizes medical literature into executable code. When applied to three datasets, this implementation generated dozens of tests per cohort, identifying discrepancies between observed distributions and epidemiological priors. These discrepancies encompass both genuine data inconsistencies and expected cohort-selection effects. This work provides an initial framework for scalable semantic auditing, shifting assurance from manual rules to the generative and context-sensitive verification required for trustworthy AI.
| Comments: | 23 pages, 5 figures (+ appendix) |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2507.02628 [cs.LG] |
| (or arXiv:2507.02628v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2507.02628 arXiv-issued DOI via DataCite |
From: Irena Girshovitz [view email]
[v1]
Thu, 3 Jul 2025 13:54:50 UTC (360 KB)
[v2]
Mon, 25 May 2026 15:05:54 UTC (551 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。