Building gold-standard AI systems requires gold-standard AI measurement science. The Applied Systems team at the Center for AI Standards and Innovation (CAISI) is seeking an AI Research Scientist (IT Specialist) with expertise in evaluation methods to join our small, high-powered group within the National Institute of Standards and Technology.

The Applied Systems team leverages multidisciplinary methodologies to identify, assess, and measure AI systems in application and real-world settings to accelerate trustworthy innovation, support the adoption of reliable AI, and protect national security. Our projects run the gamut of deliverables, including guidelines, tooling, innovative AI measurement, research papers for publication, workshops and convenings, and reports, several of which are in furtherance of the AI Action Plan. We work with the wider community of practitioners, industry (small and large), and government agencies, among others. Join us!

Position Title: IT Specialist (AI), ZP-2210-IV/V

Job and Organization Description: Department of Commerce, National Institute of Standards and Technology, Center for AI Standards and Innovation

In this position, you will:

• Evaluate and assess the functionality and trustworthiness of AI models in furtherance of the AI Action Plan.
• Design methods for scientifically sound evaluations of AI systems.
• Evaluate AI systems in application, post-deployment, and in real world settings.
• Develop methods, tooling, and guidelines for rigorous evaluation of end-to-end AI systems in use (e.g., in the workplace and/or in sector-specific tasks) to identify failure modes and areas of un-reliability.
• Develop analyses of evaluations and AI measurement instruments, themselves, for example evaluation and improvement of red teaming, benchmarks, and automated scoring techniques.
• Adapt and improve evaluation techniques, and develop new techniques where needed.
• Collaborate with colleagues and across teams to develop innovative solutions to AI evaluation problems.
• Communicate complex AI concepts to both technical and non-technical audiences.
• Inform the development of technical guidelines and best practices for AI evaluation.

Geographic Region: Duty station will be in Washington, D.C. or San Francisco

Salary Range: $120,579 to - $195,200 per year

Vacancy Open Period: 12/04/2025 to 12/29/2025

Application Process: Only electronic applications accepted through USA Jobs. 

Please note:  CAISI has a separate form/posting for expressing general interest in working at CAISI. That form is not an application for a position. To be considered for this AI Research Scientist (IT Specialist) position, you must apply directly to USA Jobs as described above. If you would like to indicate interest in future roles at CAISI, you may also fill out the expression of interest form.

US Citizenship required and the Department of Commerce is an Equal Opportunity Employer.

Note, USA Jobs has strict submission requirements for resumés. All applicants must submit a resumé by creating one in USAJOBS or uploading one of their choosing. The resumé MUST NOT exceed two pages on standard-size paper. If the resume exceeds two pages, only the first two pages will be reviewed for qualifications or assessment determinations. Cover letters are optional. To receive any credit for relevant experience, please list the month/year and the number of hours worked for experience listed on your resume. We suggest you review the qualification requirements for this position as you may need to customize your resume to ensure that it supports the position qualifications. Please refer to the Resume Guidance on how to build an effective resume.