huggingface-community-evals agent skill by huggingface
Runs evaluations for Hugging Face Hub models on local hardware using inspect-ai and lighteval, including backend selection and choosing between vLLM, Transformers, and accelerate.
At a glance
| Publisher | Hugging Face Official |
|---|---|
| Repository | huggingface/skills (24 listed skills) |
| Category | AI Agents and LLM Engineering |
| License | Apache-2.0 |
| Last commit | October 8, 2026 (repository) |
| SKILL.md | 202 lines, 883 words |
| Bundled files | 5 files, including scripts |
| Quality score | 86/100 · B Passes all seven graded checks |
| Security scan | Not scanned |
| README badge | Get the badges for this listing |
Before you install
Read from the skill's files at the directory's October 19, 2026 snapshot: what ships with it, what its code reads, and the terms it is published under.
- Bundled scripts: 3 script files (3 Python). Your agent may run them, so read them first.
- Environment variables its scripts read:
HF_TOKEN. Check SKILL.md for which ones it needs. - License: Apache-2.0, an open-source or open-content license.
Install huggingface-community-evals
The skills CLI works with Claude Code, Codex, Cursor and other agents; the other routes copy the folder by hand.
npx skills add https://github.com/huggingface/skills/tree/main/skills/huggingface-community-evalstmp="$(mktemp -d)"
git clone --depth 1 https://github.com/huggingface/skills.git "$tmp/repo"
mkdir -p .claude/skills
cp -R "$tmp/repo/"skills/huggingface-community-evals .claude/skills/
rm -rf "$tmp"python3 ~/.codex/skills/.system/skill-installer/scripts/install-skill-from-github.py \
--repo huggingface/skills \
--path skills/huggingface-community-evalshuggingface/skills is third-party code. Read SKILL.md and any bundled scripts before you install it.
How the author describes it
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.
From the description field of SKILL.md. Agents read this line to decide when to load the skill.
Sections in SKILL.md
- Option A: inspect-ai with local inference providers path
- Option B: inspect-ai on Local GPU
- Option C: lighteval on Local GPU
Files in the skill folder
SKILL.mdexamples/USAGE_EXAMPLES.mdscripts/inspect_eval_uv.pyscripts/inspect_vllm_uv.pyscripts/lighteval_vllm_uv.py
Suede Skill Check: 86/100 · B
Report snapshot · · graded-v2
The rubric totals 100 points; published scores are capped at 99. Security scan: not scanned.
Source revision: c3ff942be3e0. Report version c7529d3c83c5. Badge URLs follow the latest listing; save the report to keep this version.
Download report · Request a fresh review
What to review next
- Example request: 0 of 3 points. 3 when the description quotes an example request (3–60 characters in quotation marks).
- Reference files: 0 of 3 points. 3 when the skill folder ships a references/, reference/ or docs/ folder.
- Tool permissions declared: 0 of 2 points. 2 when the frontmatter declares allowed-tools.
Add declarations and bundled files when they describe the skill’s actual needs. Update the source, then request a fresh review with the changed commit or PR.
huggingface-community-evals passes the required frontmatter check and all seven graded checks, which cover the SKILL.md format, a declared license and a commit in the last year. Those checks earn 56 of the score’s points; the rest come from description quality, instruction depth, bundled files, maintenance and curated-list citations. How the score works.
See the checks and points
| Check | Result |
|---|---|
| Frontmatter parses and declares a name and a description | Pass |
| Name uses lowercase letters, digits and hyphens, 64 characters or fewer | Pass |
| Name matches the skill's folder name | Pass |
| Description is between 50 and 1,024 characters | Pass |
| Description says when an agent should use the skill | Pass |
| SKILL.md body is 500 lines or fewer | Pass |
| A license is declared in the frontmatter or the repository | Pass |
| Repository has a commit in the last 12 months | Pass |
| Score component | Points |
|---|---|
| Structural checks | 56 of 56 |
| Description length | 5 of 5 |
| Boundary clause | 4 of 4 |
| Example request | 0 of 3 |
| Instruction depth | 7 of 7 |
| Reference files | 0 of 3 |
| Bundled scripts | 2 of 2 |
| Recent maintenance | 9 of 9 |
| Curated-list citations | 3 of 7 |
| Tool permissions declared | 0 of 2 |
| Compatibility declared | 0 of 2 |
Two badges for huggingface-community-evals
Choose the numeric badge, the letter badge, or both. Each links to this skill’s Suede listing; either valid linked badge can qualify for the feature offer.
[](https://skills.suedeai.ai/directory/huggingface/skills/huggingface-community-evals/)
[](https://skills.suedeai.ai/directory/huggingface/skills/huggingface-community-evals/)<a href="https://skills.suedeai.ai/directory/huggingface/skills/huggingface-community-evals/"><img src="https://skills.suedeai.ai/directory/huggingface/skills/huggingface-community-evals/score.svg" alt="Suede Skill Check: 86/100" width="160" height="175"></a>
<a href="https://skills.suedeai.ai/directory/huggingface/skills/huggingface-community-evals/"><img src="https://skills.suedeai.ai/directory/huggingface/skills/huggingface-community-evals/badge.svg" alt="Suede Skill Check: Grade B" width="160" height="175"></a>Suede Certified on the shield refers to this published structural quality check. Security scan: not scanned. Read the methodology.
Limited-time feature offer: display either the numeric or letter badge in your repository or skill README (both also qualify), then request badge verification. Eligible skills scoring 80–99 can receive a featured placement after Suede verifies at least one valid badge linked to the skill’s listing. Suede has not announced an end date.
Where this skill is cited
huggingface/skills appears in 2 curated lists: heilcheng/awesome-agent-skills, skillmatic-ai/awesome-agent-skills.