Free generators
Create a correct robots.txt, sitemap.xml, llms.txt and schema markup in minutes, and learn what each file does and where it goes.
Four files every site should have
Search engines and AI tools look for a few standard files on every site. Getting them right takes minutes, and getting them wrong can hide your site. The generators are free and need no account.
| File | What it does | Generator |
|---|---|---|
| robots.txt | Tells crawlers which parts of your site they may visit | robots.txt generator |
| sitemap.xml | Lists the pages you want search engines to find | sitemap.xml generator |
| llms.txt | Summarises your site for AI tools and links your key pages | llms.txt generator |
| Schema markup | Describes a page to machines: a product, an article, a business | Schema generator |
robots.txt
A plain text file at https://yoursite.com/robots.txt. It has groups of rules: which crawler they apply to (User-agent), and which paths it may not visit (Disallow). It also points to your sitemap.
User-agent: *
Disallow: /cart/
Disallow: /account/
Sitemap: https://brewlab.coffee/sitemap.xmlThings to know:
- robots.txt is a request, not a lock. Well behaved crawlers follow it. Anything truly private needs a login.
- Blocking a page does not remove it from Google if other sites link to it. To keep a page out of results, let it be crawled and add a noindex tag.
- Decide about AI crawlers on purpose. The generator lets you allow or block them one by one. See AI visibility.
Where it goes: the root of your domain. On WordPress, SEO plugins edit it for you. Shopify generates it and lets you customise it through a template.
sitemap.xml
A file listing the addresses of the pages you want indexed, with optional last-modified dates. It helps search engines find new and deep pages faster.
- Include only pages you want in search: no redirects, no errors, no noindex pages.
- Large sites split it into several sitemaps listed in a sitemap index.
- Add a
Sitemap:line to robots.txt and submit the sitemap in Google Search Console.
Most platforms create one automatically. Use the generator for hand-built sites or when the automatic one is broken.
llms.txt
A short Markdown file at https://yoursite.com/llms.txt that tells AI tools what your site is and where the important pages are. A simple one looks like this:
# Brewlab
> Brewlab roasts single origin coffee in Portland and teaches home brewing.
## Guides
- [Cold brew recipe](https://brewlab.coffee/guides/cold-brew): 12-hour method and ratios
- [Pour over ratio](https://brewlab.coffee/guides/pour-over-ratio): the 1:16 rule explained
## Shop
- [Coffee beans](https://brewlab.coffee/shop): current roasts and pricesIt is a young, voluntary convention: helpful for AI tools that read it, with no downside, but not a ranking factor and not a guarantee of citations.
Schema markup
Structured data, usually written as JSON-LD, describes what a page is in a vocabulary search engines share, from schema.org. It can make you eligible for rich results, such as review stars, prices or FAQs, and it helps AI engines understand entities.
Common types:
| Page | Schema type |
|---|---|
| Homepage | Organization or LocalBusiness |
| Article or guide | Article or BlogPosting |
| Product page | Product, with Offer and AggregateRating when you have them |
| Genuine FAQ section | FAQPage |
| Step by step guide | HowTo |
Paste the generated block into the page's HTML, then run the site audit to validate it. Only mark up what is visible on the page: marking up reviews or FAQs that visitors cannot see breaks Google's guidelines.