Introduction

"It's not enough to give an Agent skills; we must understand how those skills are actually 'absorbed' by the model."

This is the 84th article in the "One Open Source Project per Day" series. Today, we are introducing SkillLens from Microsoft.

If the previously featured SkillOpt is an execution strategy for boosting AI skills, then SkillLens is the "microscope" for studying the evolutionary process of those skills. It provides a scientific analysis framework to help researchers and developers understand how a skill summarized by an AI actually impacts the execution efficiency of another AI.

What You Will Learn

• The full lifecycle of an Agent Skill: Experience → Extraction → Consumption.
• Core Metrics: Extraction Efficacy and Target Evolvability.
• How to validate skill effectiveness across five major Agent benchmarks.

---

Project Background

Overview

SkillLens is an open-source framework from Microsoft Research dedicated to the systematic study of "model-generated agent skills." It provides a complete pipeline covering everything from trajectory loading and skill extraction to inference validation.

Released alongside the paper From Raw Experience to Skill Consumption, this project is one of the most authoritative tools in the field of AI Agent skill research.

Core Value

1. Full Lifecycle Coverage: Focuses not just on the final skill structure (extraction) but also on where skills come from (experience) and how they are utilized (consumption).
2. Method Comparison: Features built-in support for multiple extraction methods, including the single-pass sequential baseline and the sophisticated parallel method (per-trajectory extraction with hierarchical merging).
3. authoritative Benchmarking: Integrated support for five industry-standard benchmarks, including SWE-bench, ALFWorld, and SpreadsheetBench.

---

Main Features

1. Unified Schema Normalization

Converts raw trajectories from diverse sources (e.g., complex SWE-bench debugging logs or simple ALFWorld game traces) into a unified JSON Schema, enabling large-scale batch skill extraction.

2. Hierarchical Merge Extraction

A key technology within SkillLens is its parallel extraction approach. It analyzes individual trajectories to distilled specific "modes" and uses hierarchical merging algorithms to generate high-level, generalized skill_set.json files.

3. All-in-One Inference CLI

Using the simple skilllens infer command, developers can easily compare Agent success rates between "skill-injected" and "base" runs.

---

Technical Deep Dive

The 4-Stage Research Pipeline

SkillLens standardizes every experiment into four distinct stages:

1. Raw Experience Generation: Running the Agent on a benchmark to collect raw trajectories.
2. Schema Normalization: Standardizing raw outputs into a unified format.
3. Skill Extraction: Distilling the experience pool into actionable skill sets.
4. Skill Consumption: Injecting the extracted skills back into a target model for performance evaluation.

This rigorous scientific process serves as an excellent reference for developers looking to integrate "self-evolving" capabilities into their own AI products.

---

Links and Resources

Official Resources

• 🌟 GitHub: microsoft/SkillLens
• 📄 Research Paper: arXiv:2605.23899
• 🌍 Project Homepage: microsoft.github.io/SkillLens

---

Conclusion

While SkillOpt focuses on the "how," SkillLens explains the "why." As a vital component of Microsoft's Agent research ecosystem, SkillLens reveals the deep underlying mechanisms by which AI learns from its own experience and translates it into executable knowledge.

For developers seeking peak performance in Agent systems, the empirical evaluation methods provided by SkillLens are an indispensable navigation beacon.

---

Find more useful knowledge and interesting products on my Homepage