Google‘s main business has been search, and now it wants to make a core part of it an internet standard.

The internet giant has categorical plans to turn robots exclusion agreement (REP) — better known as robots.txt — into an internet accepted after 25 years. To that effect, it has also made its C robots.txt parser that underpins the Googlebot web crawler accessible on GitHub for anyone to access.

“We wanted to help website owners and developers create amazing adventures on the internet instead of annoying about how to ascendancy crawlers,” Google said. “Together with the aboriginal author of the protocol, webmasters, and other search engines, we’ve accurate how the REP is used on the modern web, and submitted it to the IETF.”

The REP is one of the cornerstones of web search engines, and it helps website owners manage their server assets more easily. Web crawlers — like Googlebot — are how Google and other search engines commonly scan the internet to ascertain new web pages and add them to their list of known pages.

Crawlers are also used by sites like the Wayback Machine to periodically aggregate and annal web pages, and can be advised with an intent to scrape data from specific websites for analytics purposes.

A website’s robots.txt file accurately informs automatic crawlers about what agreeable to scan and what to exclude, thereby aspersing abortive pages from being indexed and served. It can also forbid crawlers from visiting arcane advice stored in assertive folders and anticipate those files being indexed by other search engines.

By open-sourcing the parser used to analyze the robots.txt file, Google is aiming to annihilate all confusion by creating a connected syntax to create and parse rules.

“This is a arduous botheration for website owners because the cryptic de-facto accepted made it difficult to write the rules correctly,” Google wrote in a blog post.

It said the library will help developers build their own parsers that “better reflect Google‘s robots.txt parsing and matching.”

Read next: Apple's Tim Cook denies his aloofness in design led to Ive's abandonment