knowledge-distillation agent skill by Orchestra-Research
Compresses large language models into smaller student models using knowledge distillation, covering temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training.
At a glance
| Publisher | Orchestra-Research |
|---|---|
| Repository | Orchestra-Research/AI-Research-SKILLs (81 listed skills) |
| Category | AI Agents and LLM Engineering |
| License | MIT |
| Last commit | June 15, 2026 (repository) |
| SKILL.md | 448 lines, 1,338 words |
| Bundled files | 2 files, including references |
| Quality score | 77/100 · C Passes all seven graded checks |
| Security scan | Not scanned |
| README badge | Get the listing badge and raise the score |
Before you install
Read from the skill's files at the directory's October 19, 2026 snapshot: what ships with it, what its code reads, and the terms it is published under.
- Bundled scripts: none.
- Commands in SKILL.md: its code blocks run
pip install, which fetches or runs software from outside the skill folder. - License: MIT, an open-source or open-content license.
Install knowledge-distillation
The skills CLI works with Claude Code, Codex, Cursor and other agents; the other routes copy the folder by hand.
npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/19-emerging-techniques/knowledge-distillationtmp="$(mktemp -d)"
git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git "$tmp/repo"
mkdir -p .claude/skills
cp -R "$tmp/repo/"19-emerging-techniques/knowledge-distillation .claude/skills/
rm -rf "$tmp"python3 ~/.codex/skills/.system/skill-installer/scripts/install-skill-from-github.py \
--repo Orchestra-Research/AI-Research-SKILLs \
--path 19-emerging-techniques/knowledge-distillationOrchestra-Research/AI-Research-SKILLs is third-party code. Read SKILL.md and any bundled scripts before you install it.
How the author describes it
Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.
From the description field of SKILL.md. Agents read this line to decide when to load the skill.
Sections in SKILL.md
- When to Use This Skill
- Installation
- Quick Start
- Basic Knowledge Distillation
- MiniLLM (Reverse KLD)
- Response Distillation
- Core Concepts
- 1. Temperature Scaling
- 2. Loss Function Components
- 3. Forward vs Reverse KLD
- Training Strategies
- Strategy 1: Logit Distillation
- Strategy 2: Two-Stage Distillation
- Strategy 3: Multi-Teacher Distillation
- Production Deployment
- Complete Training Script
Files in the skill folder
SKILL.mdreferences/minillm.md
Suede Skill Check: 77/100 · C
Report snapshot · · graded-v2
The rubric totals 100 points; published scores are capped at 99. Security scan: not scanned.
Source revision: 773a52944ba4. Report version 87859dfdef6d. Badge URLs follow the latest listing; save the report to keep this version.
Download report · Request a fresh review
What to review next
- Recent maintenance: 3 of 9 points. 9 for a repository commit within 30 days of the snapshot; 6 within 90; 3 within 180; 1 within 365.
- Boundary clause: 0 of 4 points. 4 when the description says when not to use the skill (“Not for …”, “Do not use …”, “Use … instead”).
- Example request: 0 of 3 points. 3 when the description quotes an example request (3–60 characters in quotation marks).
Add declarations and bundled files when they describe the skill’s actual needs. Update the source, then request a fresh review with the changed commit or PR.
knowledge-distillation passes the required frontmatter check and all seven graded checks, which cover the SKILL.md format, a declared license and a commit in the last year. Those checks earn 56 of the score’s points; the rest come from description quality, instruction depth, bundled files, maintenance and curated-list citations. How the score works.
See the checks and points
| Check | Result |
|---|---|
| Frontmatter parses and declares a name and a description | Pass |
| Name uses lowercase letters, digits and hyphens, 64 characters or fewer | Pass |
| Name matches the skill's folder name | Pass |
| Description is between 50 and 1,024 characters | Pass |
| Description says when an agent should use the skill | Pass |
| SKILL.md body is 500 lines or fewer | Pass |
| A license is declared in the frontmatter or the repository | Pass |
| Repository has a commit in the last 12 months | Pass |
| Score component | Points |
|---|---|
| Structural checks | 56 of 56 |
| Description length | 5 of 5 |
| Boundary clause | 0 of 4 |
| Example request | 0 of 3 |
| Instruction depth | 7 of 7 |
| Reference files | 3 of 3 |
| Bundled scripts | 0 of 2 |
| Recent maintenance | 3 of 9 |
| Curated-list citations | 3 of 7 |
| Tool permissions declared | 0 of 2 |
| Compatibility declared | 0 of 2 |
Show the listing and raise the score
knowledge-distillation is listed with 77/100 · C. This badge links to its Suede listing. Skills scoring 80–99 add a numeric and a letter quality shield; the largest open points for knowledge-distillation are recent maintenance (6), boundary clause (4) and example request (3). Send your repository and the points you want to raise, then request a fresh review after updating the skill.
[](https://skills.suedeai.ai/directory/orchestra-research/ai-research-skills/knowledge-distillation/)<a href="https://skills.suedeai.ai/directory/orchestra-research/ai-research-skills/knowledge-distillation/"><img src="https://skills.suedeai.ai/directory/badge.svg" alt="Suede Skills Directory: listed" height="20"></a>Where this skill is cited
Orchestra-Research/AI-Research-SKILLs appears in 2 curated lists: VoltAgent/awesome-agent-skills, skillmatic-ai/awesome-agent-skills.