ai.txt generator

Choose which file types AI dataset tools may use. Get an ai.txt in the format Spawning introduced in 2023, plus what it can and can't do.

Read this first: ai.txt is not a standard

Several unrelated proposals share the name. The one you’ll find on real sites is Spawning’s 2023 format, which dataset tools that use Spawning’s service can check when they download your files. The major AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot) document robots.txt controls, not ai.txt. So publish ai.txt if you like, but put your real crawler rules in robots.txt: build them with the robots.txt generator and test them with the AI crawler checker.

Format

Spawning ai.txt (/ai.txt)IETF individual draft (/.well-known/ai.txt)

AI training on your files

Choose by file typeBlock everythingAllow everything

Allow training on

Images.bmp .gif .ico .jpeg .jpg .png .svg .tif .tiff + 12 more
Text and documents.txt .pdf .doc .docx .odt .rtf .tex .wks .wpd + 17 more
Audio.aac .aiff .amr .flac .m4a .mp3 .oga .opus .wav + 12 more
Video.mp4 .webm .ogg .avi .mov .wmv .flv .mkv .3gp + 16 more
Code.py .js .java .c .cpp .cs .h .css .php + 26 more

Unchecked types get Disallow lines. Web pages follow the text setting (Allow: / or Disallow: /).

Also block these extensions (optional)

Blocked: Images, Text and documents, Audio, Video, Code. Allowed: nothing.

ai.txt

# Spawning AI
# Prevent datasets from using the following file types

User-Agent: *
Disallow: *.bmp
Disallow: *.gif
Disallow: *.ico
Disallow: *.jpeg
Disallow: *.jpg
Disallow: *.png
Disallow: *.svg
Disallow: *.tif
Disallow: *.tiff
Disallow: *.webp
Disallow: *.avif
Disallow: *.eps
Disallow: *.ai
Disallow: *.indd
Disallow: *.heif
Disallow: *.raw
Disallow: *.psd
Disallow: *.cr2
Disallow: *.nef
Disallow: *.orf
Disallow: *.sr2
Disallow: *.txt
Disallow: *.pdf
Disallow: *.doc
Disallow: *.docx
Disallow: *.odt
Disallow: *.rtf
Disallow: *.tex
Disallow: *.wks
Disallow: *.wpd
Disallow: *.wps
Disallow: *.html
Disallow: *.htm
Disallow: *.md
Disallow: *.odf
Disallow: *.xls
Disallow: *.xlsx
Disallow: *.ppt
Disallow: *.pptx
Disallow: *.csv
Disallow: *.xml
Disallow: *.ods
Disallow: *.xlr
Disallow: *.pages
Disallow: *.log
Disallow: *.key
Disallow: *.odp
Disallow: *.aac
Disallow: *.aiff
Disallow: *.amr
Disallow: *.flac
Disallow: *.m4a
Disallow: *.mp3
Disallow: *.oga
Disallow: *.opus
Disallow: *.wav
Disallow: *.wma
Disallow: *.alac
Disallow: *.dss
Disallow: *.dvf
Disallow: *.m4p
Disallow: *.mmf
Disallow: *.mpc
Disallow: *.msv
Disallow: *.ra
Disallow: *.tta
Disallow: *.vox
Disallow: *.weba
Disallow: *.mp4
Disallow: *.webm
Disallow: *.ogg
Disallow: *.avi
Disallow: *.mov
Disallow: *.wmv
Disallow: *.flv
Disallow: *.mkv
Disallow: *.3gp
Disallow: *.3g2
Disallow: *.h264
Disallow: *.m4v
Disallow: *.mpg
Disallow: *.mpeg
Disallow: *.rm
Disallow: *.swf
Disallow: *.vob
Disallow: *.mts
Disallow: *.m2ts
Disallow: *.qt
Disallow: *.yuv
Disallow: *.rmvb
Disallow: *.asf
Disallow: *.amv
Disallow: *.mpg2
Disallow: *.py
Disallow: *.js
Disallow: *.java
Disallow: *.c
Disallow: *.cpp
Disallow: *.cs
Disallow: *.h
Disallow: *.css
Disallow: *.php
Disallow: *.swift
Disallow: *.go
Disallow: *.rb
Disallow: *.pl
Disallow: *.sh
Disallow: *.sql
Disallow: *.json
Disallow: *.ts
Disallow: *.jsx
Disallow: *.vue
Disallow: *.r
Disallow: *.kt
Disallow: *.dart
Disallow: *.rs
Disallow: *.lua
Disallow: *.asm
Disallow: *.bash
Disallow: *.erl
Disallow: *.hs
Disallow: *.vbs
Disallow: *.bat
Disallow: *.f
Disallow: *.lisp
Disallow: *.scala
Disallow: *.groovy
Disallow: *.ps1
Disallow: /

Upload to the root of your site so it loads at https://yourdomain.com/ai.txt. Same pattern syntax Spawning’s own generator writes.

Five files share the name

There is no single ai.txt standard. A survey of the proposals counts five: Spawning’s 2023 training opt-out at /ai.txt, a 2023 GitHub proposal, an arXiv paper (2505.07834, May 2025) describing a DSL, aitxt.ing (2026), and an individual IETF Internet-Draft published June 12, 2026 for /.well-known/ai.txt. Individual drafts are work in progress with no standing as standards; that one expires in December 2026 unless it’s updated.

What Spawning’s ai.txt does

Spawning introduced ai.txt on May 30, 2023. It looks like robots.txt but works by file type: Disallow: *.jpg means “don’t train on my JPEGs”. Spawning said its API would pass these permissions to partners including Hugging Face and Stability AI, and that the file is checked when links in the LAION-5B dataset are used to download your images, which robots.txt can’t do after the fact. Our output matches the lines Spawning’s own open-source WordPress plugin writes. One honest caveat: spawning.ai now leads with a data marketplace, Source.Plus, so check who still reads the file before you rely on it.

What actually controls AI crawlers

robots.txt. The big AI crawlers publish user-agent tokens and say they follow robots.txt rules for them. A practical setup:

Decide per crawler in robots.txt, for example block GPTBot and Google-Extended for training but keep search crawlers allowed. The robots.txt generator has presets.
Check the live result with the AI crawler checker.
Add ai.txt as a belt-and-braces signal for dataset tools. It costs nothing and doesn’t affect search.

Our guide to robots.txt, llms.txt and AI crawlers walks through the full setup, including where llms.txt fits.

Questions

Will ai.txt stop ChatGPT, Claude or Gemini from training on my site?

Don't count on it. OpenAI (GPTBot), Anthropic (ClaudeBot), Google (Google-Extended) and Perplexity (PerplexityBot) document robots.txt as the way to control their crawlers, not ai.txt. Use robots.txt for those, and treat ai.txt as an extra signal for dataset tools that check it.

Is ai.txt the same as llms.txt?

No. llms.txt is a Markdown summary of your site meant to help AI tools read it. ai.txt, in Spawning's format, states which of your files may be used for AI training. They answer different questions and you can have both.

Which ai.txt format should I publish?

If you publish one, use Spawning's format at /ai.txt: it's the one found on real sites and the one Spawning's API reads. The IETF-style file at /.well-known/ai.txt follows an individual Internet-Draft from June 2026 that has no standing as a standard; publish it only if you want to follow that proposal.

Does blocking AI training hurt my SEO?

ai.txt has no effect on search crawling. In robots.txt, blocking Google-Extended doesn't affect Google Search ranking according to Google's crawler docs, but blocking Googlebot would. Keep search crawlers allowed.

Autopilot · early access

Want this fixed automatically?

Autopilot checks your site, publishes researched articles at a safe pace, and tracks whether ChatGPT, Perplexity, Gemini and Claude cite you. $49/mo.

See Autopilot Launch your product free

Related free tools

See all free tools