8 Commity (1cbd5df5aea8f8755a2a0b75998abaaf0813b05d)

Autor	SHA1	Wiadomość	Data
Štěpán Škorpil	1cbd5df5ae	Added config for seed domain timeout	2022-11-29 14:11:27 +01:00
Štěpán Škorpil	4113d78a17	It respects noindex option in mastodon	2022-11-22 17:01:25 +01:00
Štěpán Škorpil	704c7c066e	Added robots.txt handling	2022-11-22 16:37:11 +01:00
Štěpán Škorpil	45cafbe94b	Set mastodon page limit to 40	2022-11-22 10:57:34 +01:00
Štěpán Škorpil	29acce3906	Properly applied code quality tools	2022-09-18 13:32:25 +02:00
Štěpán Škorpil	3ed37c04ce	Not existing feeds are deleted from databse	2022-04-17 14:03:53 +02:00
Štěpán Škorpil	0c2ade0b52	Added more env configs	2022-01-03 13:26:17 +01:00
Štěpán Škorpil	74e7dca8bd	Initial crawler commit	2021-12-23 15:14:06 +01:00