mirrors/nitter
1
0
Fork 0
mirror of https://github.com/zedeus/nitter.git synced 2025-01-05 17:35:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
997 commits 4 branches 0 tags 7.6 MiB
Commit graph

3 commits

Author SHA1 Message Date
Zed 138826fb4f Fix Twitterbot rule in robots.txt 2022-06-04 17:55:35 +02:00
Zed dfb2519870 Explicitly allow Twitterbot to generate previews 2022-06-04 17:52:47 +02:00
minus c543a1df8c
Block search engines via robots.txt (#631) 
Prevents instances from being rate limited due to being senselessly
crawled by search engines. Since there is no reason to index Nitter
instances, simply block all robots. Notably, this does *not* affect link
previews (e.g. in various chat software).
 2022-06-04 17:48:25 +02:00