fix: syntax for robots.txt disallow rules (#1925)

pull/1930/head
Baptiste Girardeau 2023-03-26 17:11:22 +02:00 zatwierdzone przez GitHub
rodzic e062fb5e52
commit 881a49e0d9
Nie znaleziono w bazie danych klucza dla tego podpisu
ID klucza GPG: 4AEE18F83AFDEB23
1 zmienionych plików z 15 dodań i 15 usunięć

Wyświetl plik

@ -3,21 +3,21 @@ Allow: /
# Disallow authenticated pages # Disallow authenticated pages
Disallow /intent Disallow: /intent
Disallow /settings Disallow: /settings
Disallow /blocks Disallow: /blocks
Disallow /bookmarks Disallow: /bookmarks
Disallow /compose Disallow: /compose
Disallow /conversations Disallow: /conversations
Disallow /domain_blocks Disallow: /domain_blocks
Disallow /favourites Disallow: /favourites
Disallow /home Disallow: /home
Disallow /mutes Disallow: /mutes
Disallow /notifications Disallow: /notifications
Disallow /pinned Disallow: /pinned
Disallow /search Disallow: /search
Disallow /settings Disallow: /settings
Disallow /share-target Disallow: /share-target
# Wait 1 second between successive requests. # Wait 1 second between successive requests.
Crawl-delay: 1 Crawl-delay: 1