bridgy-fed/static/robots.txt

User-agent: *
Disallow: /*?*
Disallow: /ap/
Disallow: /atp/
Disallow: /bsky/
Disallow: /bluesky/
Disallow: /bridge-user
Disallow: /convert/
Disallow: /log*
Disallow: /nostr/
Disallow: /recent
Disallow: /r/
Disallow: /render
Disallow: /user/
Disallow: /web/
start on home page html, style, and assets 2017-08-18 14:31:31 +00:00			`User-agent: *`
			`Disallow: /?`
robots.txt bug fix hopefully this fixes /r/ pages showing up in google search 2023-11-02 21:07:08 +00:00			`Disallow: /ap/`
			`Disallow: /atp/`
add /bsky to robots.txt 2024-04-26 16:57:26 +00:00			`Disallow: /bsky/`
robots.txt bug fix hopefully this fixes /r/ pages showing up in google search 2023-11-02 21:07:08 +00:00			`Disallow: /bluesky/`
disable /bridge-user until Bluesky launch 2024-03-13 20:19:34 +00:00			`Disallow: /bridge-user`
robots.txt bug fix hopefully this fixes /r/ pages showing up in google search 2023-11-02 21:07:08 +00:00			`Disallow: /convert/`
robots.txt: add /log 2024-05-16 17:28:32 +00:00			`Disallow: /log*`
robots.txt bug fix hopefully this fixes /r/ pages showing up in google search 2023-11-02 21:07:08 +00:00			`Disallow: /nostr/`
			`Disallow: /recent`
			`Disallow: /r/`
robots.txt: add /render 2023-11-02 21:11:10 +00:00			`Disallow: /render`
robots.txt bug fix hopefully this fixes /r/ pages showing up in google search 2023-11-02 21:07:08 +00:00			`Disallow: /user/`
			`Disallow: /web/`