fix: syntax for robots.txt disallow rules (#1925)

pull/1930/head
Baptiste Girardeau 2023-03-26 17:11:22 +02:00 zatwierdzone przez GitHub
rodzic e062fb5e52
commit 881a49e0d9
Nie znaleziono w bazie danych klucza dla tego podpisu
ID klucza GPG: 4AEE18F83AFDEB23
1 zmienionych plików z 15 dodań i 15 usunięć

Wyświetl plik

@ -3,21 +3,21 @@ Allow: /
# Disallow authenticated pages
Disallow /intent
Disallow /settings
Disallow /blocks
Disallow /bookmarks
Disallow /compose
Disallow /conversations
Disallow /domain_blocks
Disallow /favourites
Disallow /home
Disallow /mutes
Disallow /notifications
Disallow /pinned
Disallow /search
Disallow /settings
Disallow /share-target
Disallow: /intent
Disallow: /settings
Disallow: /blocks
Disallow: /bookmarks
Disallow: /compose
Disallow: /conversations
Disallow: /domain_blocks
Disallow: /favourites
Disallow: /home
Disallow: /mutes
Disallow: /notifications
Disallow: /pinned
Disallow: /search
Disallow: /settings
Disallow: /share-target
# Wait 1 second between successive requests.
Crawl-delay: 1