GoFestivalBot
This crawler visits the home page of film festival websites to collect publicly available dates, submission conditions and news.
- It respects each site's robots.txt.
- It waits several seconds between two requests to the same site.
- It revisits a given page at most once a week.
To opt out, add “User-agent: GoFestivalBot” and “Disallow: /” to your robots.txt.