bridgy-fed/static/robots.txt

User-agent: *
Disallow: /*?*
Disallow: /ap/*
Disallow: /atp/*
Disallow: /bluesky/*
Disallow: /convert/*
Disallow: /nostr/*
Disallow: /recent*
Disallow: /r/*
Disallow: /user/*
Disallow: /web/*
start on home page html, style, and assets 2017-08-18 14:31:31 +00:00			`User-agent: *`
			`Disallow: /?`
AP users: update robots.txt for new URL prefixes for #512 2023-05-30 23:41:55 +00:00			`Disallow: /ap/*`
			`Disallow: /atp/*`
			`Disallow: /bluesky/*`
			`Disallow: /convert/*`
			`Disallow: /nostr/*`
add /recent to robots.txt 2022-11-18 14:54:57 +00:00			`Disallow: /recent*`
add /r/* redirects to robots.txt 2023-01-11 16:55:31 +00:00			`Disallow: /r/*`
AP users: update robots.txt for new URL prefixes for #512 2023-05-30 23:41:55 +00:00			`Disallow: /user/*`
			`Disallow: /web/*`