User-agent: * # Ignore pages with sub-session id in stepbased # The "&" based directive is for crawlers that don't handle the "?" listing correctly, even though "?" is supposed to encompass both Disallow: /*?v=* Disallow: /*&v=* Disallow: /er/General/ Disallow: /er/Matchmake/ Disallow: /er/gdpr/ Disallow: /er/static/ Disallow: /er/images/ Disallow: /er/jQuery/ Crawl-delay: 10 User-agent: SemrushBot Disallow: / User-agent: Yandex Disallow: /