Google Updates Robots.txt Policy: Unsupported Fields Are Ignored via @sejournal, @MattGSouthern

Google limits robots.txt support to four fields, clarifying its stance on unsupported directives.

  • Google only supports four specific robots.txt fields.
  • Unsupported directives in robots.txt will be ignored.
  • Consider auditing your robots.txt files in light of this update.

Leave a Reply

Your email address will not be published. Required fields are marked *