---
title: 'Claude skills: what we found in the 53 our marketing agent can load'
description: >-
  We audited the 53 Claude skills our marketing agent can load. 456 files, 3
  runnable scripts and a lock file that pins nothing. The checklist we use now.
author: elroot_labs
date: '2026-11-18'
categories:
  - artificial-intelligence
---
Claude skills are folders of instructions, and sometimes code, that Claude loads when a task matches the skill's description. Before you install one, treat it like a dependency: read every file, list anything that is not Markdown, and search for install commands and for lines that run on load or skip permission prompts. Then pin the version you read. At Elroot Labs we did exactly that on 18 November 2026, for the 53 third-party skills our marketing agent can load. This is what we found, and the checklist we run now.

## What a Claude skill is, and why Markdown can still act

A skill is a directory with a `SKILL.md` file at its root. Anthropic's documentation describes [three levels of loading](https://docs.anthropic.com/en/docs/agents-and-tools/agent-skills/overview): the name and description are loaded at startup and sit in the system prompt, the body of `SKILL.md` is read when a request matches, and any bundled scripts are run through bash, with only their output entering the context. The same page is direct about trust. Use skills only from sources you created yourself or obtained from Anthropic, and audit everything else, because "a malicious Skill can direct Claude to call tools or run code in ways that don't match the Skill's stated purpose."

In Claude Code, the place most teams meet skills, two details matter more than the scripts. A skill's frontmatter can set `allowed-tools`, which lets it [use the listed tools without asking](https://docs.anthropic.com/en/docs/claude-code/skills) during the turn that invokes it. And a line written as an exclamation mark followed by a command in backticks runs that shell command before the skill's content is sent to Claude, without a permission prompt while the skill renders. Both live in plain Markdown. A review that only asks "does this skill ship code?" misses them.

Claude Code also gives skills the same network access as any other program on your machine. A skill that tells the agent to fetch a URL brings whatever that URL serves into the session.

## What we found in the 53 skills our agent can load

Our repository holds 54 skill folders. 53 come from one public marketing pack, installed on 31 October with the `npx skills` installer; the 54th is our own release procedure. We counted every file by type, then searched the Markdown for the patterns above.

| File type   | Files | Where                                                      |
| ----------- | ----- | ---------------------------------------------------------- |
| Markdown    | 338   | all 54 skills                                              |
| JSON        | 86    | 53 of them are evals, test prompts for the skill itself    |
| JavaScript  | 26    | one skill, as tracking snippets meant for a website        |
| Python      | 2     | the same skill, scripts that build a tag manager container |
| HTML        | 2     |                                                            |
| Node (.mjs) | 1     | one skill, a teardown script                               |
| CSV         | 1     |                                                            |

![Bar chart of the files in the 54 Claude skills folders our marketing agent can load, by type: 338 Markdown, 86 JSON, 26 JavaScript, 2 Python, 2 HTML, 1 Node and 1 CSV.](/images/blog/elroot/figures/claude-skills-files-by-type.png)

Most of a skill pack is text, as you would expect. The rest is worth naming:

- Three runnable scripts, in two skills. None runs unless the agent decides to run it, which is why the permission prompt matters.
- 26 JavaScript snippets in one skill, written to be pasted into a website. They would not run on our machine, but they would run on our visitors' browsers if an agent copied them into the site.
- 17 Markdown files in 10 skills tell the agent to run an install or download command, such as `npx`, `curl` or `pip install`. Each of those is a fresh download from somewhere else at the moment the agent follows the instruction.
- No skill sets `allowed-tools`, and no Markdown file contains the run-on-load form.

The descriptions surprised us more than the code. Anthropic's documentation puts the always-loaded cost at about 100 tokens per skill. Our 54 names and descriptions add up to roughly 42,900 characters, about 790 per skill, and the longest is 1,023 characters, one short of the limit. In a Claude Code session in this repository on 18 November, the skill list showed full descriptions for some skills and only the names of others. A description is a prompt the model reads before any skill is chosen, and a pack update can change it.

## The lock file records an install; it does not pin one

The installer writes `skills-lock.json` with the source and a computed hash for each skill. We first took it for a pin that would hold the reviewed version in place. What it holds is a record of the last install: running the update command downloads whatever the upstream repository has that day and rewrites the lock to match, even when a skill has changed.

So an update is reviewed as a diff, the way you would review a dependency bump. That has a catch we had not noticed: our `.gitignore` excludes the skills we have not committed, so `git diff` shows changes to 7 of the 54 folders. For the other 47, the only trace of an update is a changed hash in the lock file. We now read the lock diff first, then open each skill whose hash moved.

## Only six skills reach the unattended writer

The agent that drafts our blog runs on a schedule, unattended, in its own clean clone of the repository. A clone has only what git tracks, so it sees 7 skill folders: the 6 the writing pipeline uses and our release procedure. The other 47 exist only on the owner's laptop, for interactive sessions where a person watches each tool call.

Three rules hold that line:

1. The [playbook](/blog/ai-blog-pipeline) forbids installing, updating or downloading skills during a run, because the run follows them as instructions; they change only through a reviewed commit.
1. Since 31 October a test fails if any of the 6 pipeline skills gains a file that is not Markdown.
1. The post gate refuses citations from any host outside an approved list, so a skill that steers the agent toward an unknown site cannot get that site into a published post.

## The check we were missing, found while writing this

Halfway through the audit we asked whether our own test would catch the two Markdown-only risks. It would not. A `SKILL.md` with an `allowed-tools` line, or a line that runs a command on load, contains nothing but Markdown and passes a file-type check.

We added a second test the same evening. It reads every Markdown file in the 6 pipeline skills and fails on either pattern. To prove it works, we planted each pattern in a copy of one skill, watched the test fail with the file named, removed it and watched it pass. The other 47 skills are still unwatched by that test, because the unattended run never sees them.

Claude Code also offers a setting, `disableSkillShellExecution`, that replaces every run-on-load command with a notice instead of running it. We have not turned it on, because nothing here uses the feature and our test now watches the skills that run unattended. If your team installs skills from many sources, turning it on costs little.

## Are Claude skills safe to install? The checklist we use

A skill is as safe as the files in it and the access your agent has when it runs them. OWASP lists [supply chain weaknesses](https://genai.owasp.org/llmrisk/llm032025-supply-chain/) among the top risks for LLM applications and asks teams to vet suppliers and keep an inventory of components. For skills, that becomes:

1. Know the source. Record the repository and the commit you reviewed, and prefer skills you wrote or got from Anthropic.
1. List every file that is not Markdown. Open each script and each snippet, and decide where it would run.
1. Search the frontmatter for `allowed-tools` and the Markdown for run-on-load commands. Either one needs a reason.
1. Search for install and download commands such as `npx`, `curl`, `pip install` and `npm install`. Each one is a new supply chain at run time.
1. Read the description as a prompt. The model reads it before choosing any skill, so it should say what the skill does and when, and nothing else.
1. Choose where it lives. A personal folder affects every project on your machine; a project folder affects everyone who clones the repository.
1. Give unattended agents only reviewed skills, committed to the repository, and add a test that fails when they change shape.
1. Review every update as a diff, including the lock file's hashes for anything git ignores.

## Where this fits with the rest of an agent's controls

A skill is one more way to hand an agent instructions, so the same limits apply as for any tool it calls. Our checklist for [least-privilege agent permissions](/blog/ai-agent-permissions) decides what a skill could reach if it misbehaved, and the review in [MCP server security](/blog/mcp-server-security) covers the other common way third-party code reaches an agent. Both sit under the eight safeguards in our [production AI agent safety](/blog/production-ai-agent-safety) guide.

If you are setting up agents for a team and want the skills, permissions and review steps decided before anything runs unattended, [tell us what you are building](/contact).
