Free generators
Create a correct robots.txt, sitemap.xml, llms.txt and schema markup in minutes, learn what each file does and where it goes, then check it.
Four files every site should have
Search engines and AI tools look for a few standard files on every site. Getting them right takes minutes, and getting them wrong can hide your site. The generators are free, need no account and run in your browser. Each file has a free checker too, so you can confirm it works once it is live.
| File | What it does | Build it | Check it |
|---|---|---|---|
| robots.txt | Tells crawlers which parts of your site they may visit | robots.txt generator | robots.txt tester |
| sitemap.xml | Lists the pages you want search engines to find | sitemap.xml generator | Sitemap checker |
| llms.txt | Summarises your site for AI tools and links your key pages | llms.txt generator | llms.txt checker |
| Schema markup | Describes a page to machines: a product, an article, a business | Schema generator | Schema validator |
robots.txt
A plain text file at https://yoursite.com/robots.txt. It has groups of rules: which crawler they apply to (User-agent), and which paths it may not visit (Disallow). It also points to your sitemap.
User-agent: *
Disallow: /cart/
Disallow: /account/
Sitemap: https://brewlab.coffee/sitemap.xmlThings to know:
- robots.txt is a request, not a lock. Well behaved crawlers follow it. Anything truly private needs a login.
- Blocking a page does not remove it from Google if other sites link to it. To keep a page out of results, let it be crawled and add a noindex tag.
- Decide about AI crawlers on purpose. The generator lets you allow or block them one by one. See AI visibility.
Where it goes: the root of your domain. On WordPress, SEO plugins edit it for you. Shopify generates it and lets you customise it through a template. Once it is live, paste one of your page addresses into the robots.txt tester to confirm Googlebot and the AI crawlers you want can still reach it.
sitemap.xml
A file listing the addresses of the pages you want indexed, with optional last-modified dates. It helps search engines find new and deep pages faster.
- Include only pages you want in search: no redirects, no errors, no noindex pages.
- Large sites split it into several sitemaps listed in a sitemap index.
- Add a
Sitemap:line to robots.txt and submit the sitemap in Google Search Console.
Most platforms create one automatically. Use the generator for hand-built sites or when the automatic one is broken, and the sitemap checker to confirm either kind is valid and lists pages that actually load.
llms.txt
A short Markdown file at https://yoursite.com/llms.txt that tells AI tools what your site is and where the important pages are. A simple one looks like this:
# Brewlab
> Brewlab roasts single origin coffee in Portland and teaches home brewing.
## Guides
- [Cold brew recipe](https://brewlab.coffee/guides/cold-brew): 12-hour method and ratios
- [Pour over ratio](https://brewlab.coffee/guides/pour-over-ratio): the 1:16 rule explained
## Shop
- [Coffee beans](https://brewlab.coffee/shop): current roasts and pricesIt is a young, voluntary convention: helpful for AI tools that read it, with no downside, but not a ranking factor and not a guarantee of citations. After you upload it, the llms.txt checker confirms it follows the format and that every link in it works.
Schema markup
Structured data, usually written as JSON-LD, describes what a page is in a vocabulary search engines share, from schema.org. It can make you eligible for rich results, such as review stars, prices or FAQs, and it helps AI engines understand entities.
Common types:
| Page | Schema type |
|---|---|
| Homepage | Organization or LocalBusiness |
| Article or guide | Article or BlogPosting |
| Product page | Product, with Offer and AggregateRating when you have them |
| Genuine FAQ section | FAQPage |
| Step by step guide | HowTo |
For a page that is already live, the JSON-LD generator drafts the block from the page's own tags instead. Paste the generated block into the page's HTML, then check it with the schema validator, which also accepts code you paste before publishing. The site audit checks it too, and flags markup that does not match what the page shows. Only mark up what is visible on the page: marking up reviews or FAQs that visitors cannot see breaks Google's guidelines.