In Google’s early years, users would type in a query and get back a page of 10 “blue links” that led to altered websites. “We want to get you out of Google and to the right place as fast as possible,” co-founder Larry Page said in 2004.

Today, Google often considers that “right place” to be Google, an assay by The Markup has found.

We advised more than 15,000 recent accepted queries and found that Google adherent 41 percent of the first page of search after-effects on mobile accessories to its own backdrop and what it calls “direct answers,” which are busy with advice copied from other sources, sometimes after their adeptness or consent.

When we advised the top 15 percent of the page, the agnate of the first screen on an iPhone X, that figure jumped to 63 percent. For one in five searches in our sample, links to alien websites did not appear on the first screen at all.

A trending search in our data for “myocardial infarction” shows how Google has piled up its articles at the top. It returned:

  • Google’s concordance definition.
  • A “people also ask” box that broadcast to answer accompanying questions after abrogation the search after-effects page.
  • A “knowledge panel,” which is an abridged album entry with assorted links.
  • And a “related conditions” carousel arch to assorted new Google searches for other diseases.

All of these appeared before search after-effects by WebMD, Harvard University, and Medscape. In fact, a user would have to scroll nearly center down the page—about 42 percent—before extensive the first “organic” result in that search.

Google’s accommodation to place its articles above competitors’ and to present “answers” on the search page has led to lawsuits and authoritative fines. A number of websites said it killed their revenues—and their companies. Founders of both avant-garde startups and companies that had been around for a decade or more told The Markup that once Google started agreement its artefact first, they didn’t stand a chance.

Travel assay firm Skift wrote in November that the entire online travel industry is suffering. “The fact that Google is leveraging its ascendancy as a search engine into taking market share away from travel competitors is no longer even debatable.”

The choice to highlight its own articles has been deliberate: Centralized emails unearthed by the European Agency in an antitrust assay show Google staffers discussing the need to place its comparison-shopping artefact at the top of the search after-effects to garner traffic. An email the afterward year noted cartage to the retooled artefact had more than angled from four actor to 10 actor visits, and “most of this growth is from bigger google.com integration.”

Sally Hubbard, an expert on antitrust and technology companies with the Open Markets Institute, said Google’s decisions in search have huge implications. “Imagine you go to the library, and the card archive is acrimonious and allotment what book to get based on what makes the library the most money.”

Google makes five times as much acquirement through announcement on its own backdrop as it does affairs ad space on third-party websites.

In a accounting statement, Google agent Lara Levin took issue with The Markup counting “answers,” accompanying queries, and agnate after-effects as “Google” in our analysis.

“Providing acknowledgment links, allowance people reformulate queries or assay topics, and presenting quick facts is not advised to alternative Google. These actualization are fundamentally in the absorption of users, which we validate through a authentic testing process,” she said. “Sometimes, the most accessible advice will be a link to addition website—other times, it will be a map, a restaurant listing, a video or an image.”

Levin called The Markup’s alignment “flawed and misleading.” She criticized it for being “based on a non-representative sample of searches” and said using Google Trends makes it more likely that after-effects would accommodate Google “knowledge panels” than a random sample would. However, Google has not about appear a random sample of searches for research.

In acknowledgment to Page’s 2004 quote about the company’s mission to get people “out of Google,” Levin said times have changed: “As a search engine, Google’s mission is to bound direct searchers to great information, wherever that advice is, as Page went on to explain. At that time, that about meant to direct people from search after-effects to websites. As search technologies have developed, that’s not always the best way to assist people.”

She did not answer questions about whether those changes present the search engine with a battle of interest.

Nearly nine in 10 U.S. web searches are performed on Google.

The furnishings of agreement its own articles on the search page can be stark: In the nine years since Google Flights and Google Hotels launched, those sites have become market leaders. They garnered almost twice as many U.S. site visits last year as each of their better competitors, Expedia.com and Booking.com, even though we found Google Flights doesn’t always show users all the options.

“Google makes the most money when, long term, they can addict searchers to their platform,” said Rand Fishkin, a search engine analyst and common Google critic who found that 12 percent of real-world clicks on the search after-effects page go to Google properties. “If Google can train you, don’t go to Genius.com, don’t go to TripAdvisor, don’t go to the restaurant’s website, just come to Google—always come to Google—then they win.”

Levin said some regulators have dedicated the company’s practices, pointing to the Federal Trade Commission’s 2013 ruling that highlighting its articles on search “could be allegedly justified as innovations that bigger Google’s product”; and a U.K. High Court award in 2016 that abiding Google Maps for area queries was not an abuse of market dominance.

Although the FTC absitively not to take legal action, as Google notes, it did crave the aggregation to change some practices. Google agreed to allow websites to opt out of having their agreeable aching for Google Flights, Google Hotels, and local business listings.

And at least some FTC staffers had assured that Google’s advocacy its own backdrop in search rankings “led to a cogent abatement in cartage for the websites of many vertical competitors,” according to an centralized FTC report, half of which was accidentally provided to The Wall Street Journal At the time, Google responded, “Speculation about customer or adversary harm turned out to be absolutely wrong.”

Levin said the aggregation “continues to engage” with regulators administering myriad active probes of Google’s parent company, Alphabet. Fifty U.S. attorneys accepted are currently investigating its ads and search business for abeyant antitrust violations. The Federal Trade Agency is analytical past acquisitions by tech companies, including Google, for harm to competition. Politico appear that the Administration of Justice will file an antitrust accusation adjoin the aggregation soon. The European Commission, which has already issued three multibillion-dollar fines adjoin Alphabet for antitrust violations since 2017, launched a basic assay into Google for Jobs in 2019.

“Google helped build the free internet. And now they’re allowance annihilate what they built,” said Chris Cummings, CEO of Curiosity Media, which owns the adaptation website SpanishDict.com.

The site provides free translations and concordance entries, many accounting by linguists and translators, he said. It is ad-supported and needs web cartage to survive. For years, he said, it grew as Google grew. But then Google began giving the top spot in searches to Google Translate, which is automatic and asks users for corrections.

“The big loss is for consumers,” he added, “because nobody thinks that Google Translate is the most authentic translator.”

He said data from the Google Search Console tool for websites shows SpanishDict gets as high as 80 percent or more of the clicks when it’s the top result but only 2 percent when Google Translate appears above it.

“If we’re only accepting 2 percent of the click-through, there’s no business to run here. We only exist because there are still some queries where they don’t put their stuff at the top,” he said. He said Google’s accomplishments “have afflicted our adeptness to invest in the future.”

Levin did not acknowledge to questions about Google Translate or its effect on SpanishDict.

SpanishDict.com showed up three times in our sample, each time below either Google Translate or Google’s dictionary.

Pushed down the page

To actuate the amount of space Google dedicates on the search page to direct answers and its own products, we built a custom scraper to gather all trending Google searches for two months, starting in November 2019. Then we built addition scraper to run the searches as they would appear on mobile devices, where the majority of searches now occur. We wrote more than 1,000 lines of code to parse and assay the consistent dataset, which independent more than 1 actor rows.

We found that the majority of links to and after-effects for non-Google sites were pushed down to the bottom-middle of the page, where data shows users are less likely to click.

We categorized search after-effects into four types: Google, non-Google, ads, and AMP, a web format created by Google four years ago. AMP pages are hosted by Google but created and monetized by publishers.

AMP has been controversial, with some publishers and developers saying it gives Google too much ascendancy over the web. The aggregation tells website owners that using it makes them acceptable “for more arresting presentation of thumbnail images in search after-effects and in the Google Discover feed.” Because AMP has some actualization of a Google result and some actualization of a non-Google result, we gave it its own category.

Levin objected to that decision, saying AMP after-effects should be non-Google. “Those are outbound links to publishers and other web creators. To advance contrarily is not factual,” she said. She also said our after-effects may be heavy on AMP agreeable because our sample, Google Trends, leans toward breaking news, which triggers “top stories” carousels. The news belief in those carousels were often delivered in AMP in our data.

Direct answers accommodate “featured snippets,” which extract agreeable from websites, and “knowledge panels,” which show summaries and facts drawn from the “knowledge graph,” Google’s fact database curated from other sources. They also accommodate weather, sports statistics, and concordance definitions. All of these appeared on the first search after-effects page, about at the top, after the need to click through.

Google accustomed in accounting comments to Congress last fall that one major reason people end a search is because Google’s modules provided the answer on the search page.

In our sample, Google featured its concordance definitions before Urban Dictionary, Cambridge Dictionary, Dictionary.com, Wikipedia, Merriam Webster, and Investopedia, among others. And searches for song titles about alternate a YouTube video in the top spot, followed by the lyrics, displayed in full on the search after-effects page.

Levin said Google does not give alternative to YouTube, its subsidiary. Recent tests by The Wall Street Journal found that it did.

The abundance of Google after-effects in some searches in our sample was quite large. A search for the Shania Twain song “Man! I Feel Like a Woman,” which was trending during our study, alternate the afterward on the first page: links to four YouTube videos in assorted positions on the page; a box labeled “about” with some hyperlinked facts that led to new Google searches; a box labeled “people also search for,” which led to a new Google search; and a “people also ask” box. Together, direct answers and after-effects arch to Google articles took up 67 percent of the first search after-effects page for that query. Non-Google after-effects took up only 22 percent of the page.

Even some of the “traditional” after-effects that appeared after the Google after-effects in that query were affected by its hand. Genius.com and Biography.com’s after-effects were delivered in AMP.

Competing with advertisers

Travel websites are among those who say they have suffered from Google Search’s best analysis of Google products. TripAdvisor, which laid off 200 workers in January, acicular to Google “siphoning off affection cartage that would contrarily find TripAdvisor” as its “most cogent challenge.”

In queries for specific airlines that appeared in our sample, Google presented Google Flights at the top of the after-effects page, before links to the airlines’ own websites. A search for “nonstop flight” also alternate Google Flights in the top spot, above competitors.

And travel sites aren’t just Google’s competitors; they’re also its customers. Together, Expedia and Booking spent $5.8 billion on Google announcement in the 12 months ending in September 2019, according Skift, the travel assay firm.

“When they attempt adjoin their advertisers … I think it’s bad practice,” Barry Diller, administrator of Expedia Group, said during an balance call in February in which he called Google an “existential” threat.

In the industry, Google Flights is not seen as the best product. It did not crack the top 10 of Frommer’s 2020 baronial of airfare search engines, for instance.

The Markup found that Google Flights did not always affectation the cheapest fares or all accessible flights, even when those fares and flights showed up in ITA Matrix, which is powered by the same software Google acquired in 2010 and used to launch Google Flights.

For example, a search on Google Flights for a one-way flight from Billings, Mont., to London’s Heathrow on Feb. 19 showed the cheapest flight was $1,068. The same search on Orbitz turned up a flight for $772, while ITA Matrix offered $766.69.

And when The Markup searched for a one-way flight from Morgantown, W.Va., to New York City on Feb. 21, Orbitz produced a long list of results, including a three-hour-and-40-minute adventure that accumulated flights from Southern Airways and American Airlines with a stop in Pittsburgh for $253.47. The same search on ITA Matrix showed the same flight at a altered time for $242.40. But a Google Flights search showed nothing: “Aw snap, no results.”

The FTC found the same thing in 2012, according to the memo accidentally appear to The Wall Street Journal: “Although it displays its flight search above any accustomed search after-effects for flight-booking sites, Google does not accommodate the most flight options for travelers,” the regulators wrote.

Levin would not anon abode that issue with The Markup. Google’s travel abutment webpage said that “a affiliation with Google is required” to appear in Google Flights but gave no added details.

Once advancing travel website Hipmunk blames its death in part on Google Search’s high baronial of Google’s travel products. A Silicon Valley favorite, Hipmunk was among the first to analyze prices for Airbnb alongside hotels. It invented an “agony” baronial to help barter decide if that cheap flight was really worth it.

To drum up traffic, it created webpages with highly specific travel tips like “How long does it take to get to the airport from city San Francisco?” This action led to booming “organic” cartage from Google, said its CEO at the time, Adam Goldstein. Then Google started to affection Google Flights, Google Hotels, and its “destinations on Google” travel guide acutely in search results.

“One quarter, that cartage started to really fall from what we were planning. And then the next division it fell short by even more,” Goldstein said. “This was around the time that I accomplished that the doors were starting to close in on us and all the small players in the travel industry.”

As Google Flights grew, airlines accomplished they didn’t need referrals from abate players like Hipmunk, Goldstein said. The airlines bargain fees to nearly zero for small players, and airlines even began ambitious that online travel agencies hide assertive routes in barter for access to fare data, according to Goldstein and complaints to the U.S. Administration of Transportation. The administration launched an analysis but has not taken any action.

Goldstein gave up and sold the aggregation to the accumulated travel giant Concur, which he anticipation had enough heft to compete. Concur shut Hipmunk down in January.

Levin, the Google spokesperson, beneath to abode Goldstein’s complaints.

Page, Google’s co-founder, had bashed competitors like AOL and MSN in an account with Playboy in 2004 for acutely agreement their beat agreeable above competitors.

“Most portals show their own agreeable above agreeable abroad on the web,” he said. “We feel that’s a battle of interest, akin to taking money for search results. Their search engine doesn’t necessarily accommodate the best results; it provides the portal’s results.”

Long after abrogation AOL and MSN in the dust, Google now says users want answers on the page—or, in their smart speakers, as admiral said in a 2019 SEC filing, because it’s “quicker, easier and more accustomed to find what you’re attractive for.”

This abstract switch started around 2007, when Google alien what it called “universal search,” which congenital Google Maps, Google Books, and Google Video into accepted search results, frequently at the top.

At a appointment in Seattle in 2007, Marissa Mayer, then Google’s vice admiral of search articles and user experience, was asked about the sudden actualization of links to Google Finance ahead of Yahoo Finance in the design of the stock ticker info boxes on the search after-effects page at that time. She responded, “It seems only fair, right? We do all the work for the search page and all these other things, so we do put it first.”

There are now many varieties of Google-created after-effects featuring its articles or advice taken from other sites delivered as direct answers. Google even serves up after-effects that lead boilerplate and seem purely for entertainment, like a module that produces animal sounds.

Other search engines, including DuckDuckGo and Bing, also accommodate answers on the search after-effects page but allure less criticism because of their small U.S. market share. DuckDuckGo has 1.5 percent, and Bing has 7 percent, according to Statcounter.

“Fundamentally, consumers do want burning answers,” said Gabriel Weinberg, CEO of DuckDuckGo. “That is a astriction with the open web and the sites that aftermath that information.”

DuckDuckGo does not have any articles to advance in its search results, but Bing, which mirrors Google’s actualization closely, directs users to Microsoft-owned articles such as Bing Maps and its flight booking tool. Microsoft beneath to comment.

“It’s not a atom on Google”

Some of the advice Google presents in direct answers is accumulated from websites that agree to it, including the crowdsourced album Wikipedia, which allows anyone to broadcast its agreeable as long as it’s credited. Wikipedia said it has not seen a drop in donations as a result.

Dario Taraborelli, former head of assay at Wikimedia Foundation, which operates Wikipedia, said he is concerned, however, about the effect on media literacy.

“It’s become really difficult to accept where advice comes from. What is the ancestry of what we’re learning?” he said. “It’s going to be become much harder for a new contributor to accept … that Wikipedia exists as a abstracted project, that it’s not a atom on Google.”

A 2017 study found people were more than five times more likely to afield aspect advice to Google itself after account it in a Google direct answer than when no direct answer appeared in results.

Google also scrapes websites after absolute permission, a convenance it has dedicated by arguing the “fair use” absolution to absorb laws and noting that websites can opt out of admittance in featured snippets while actual in the search index for other results.

Brian Warner, architect of the website CelebrityNetWorth, said he beneath Google’s appeal to accommodate his site’s advice in the “knowledge graph,” but two years later Google started assuming advice taken from CelebrityNetWorth in “featured snippets” results.

His cartage crumbled, and Warner said he had to lay off half of his 12-person staff.

The song lyrics site Genius said its click-through rate from search after-effects fell from 60 to 80 percent to 5 to 20 percent after Google started announcement lyrics on the search after-effects page. The aggregation also alleges that one of Google’s lyrics providers, LyricFind, had stolen Genius’s watermarked lyrics and they were assuming up in direct answers.

“The fact that Google often populates its lyrics boxes with lyrics misappropriated from our website makes Google’s behavior even more unfair,” Ben Gross, chief action administrator at Genius, said in an email.

Genius sued Google in December. Google filed a demand for a jury trial. Levin beneath to animadversion on the company’s claims or its lawsuit.

Some websites try to get into featured snippets because it’s high up on the page. Levin said “it can drive allusive cartage to their sites” but beneath to accommodate specifics.

Some sites take whatever cartage Google search provides and don’t complain. “My goal in life is not to cross the Google gods,” said one sports media controlling who did not want to be quoted by name or the name of the website for fear of being shut out by Google.

Cummings, of SpanishDict.com, said article similar. “Google delivers the cartage for the whole internet. Unless your name is Facebook, you rely on Google,” he said. “It’s very risky to speak out at Google because you don’t know what type of backfire you’ll face.”

Frederic Lambert said his site was demoted in search listings when Google rebooted its arcade artefact in 2012.

A avidity roadside ad salesman, Lambert launched Acheter-moins-cher—“buy cheaper”—in 1998 to accumulated articles from around the web and rank them by price. It accustomed consumers to set alerts if the price dropped. When users clicked or bought something, sellers paid him a commission.

At its peak, about 40,000 shoppers visited the French site, and it took in more than a actor euros a year in revenue. Starting in 2012, revenues decreased by half, until he shut it down in 2018.

“They did kill us,” he said. “But not by making the best product.”

In 2017, the European Agency levied a then-record-breaking €2.42 billion fine adjoin Google, award it gave best analysis to Google Arcade while demoting allegory arcade sites in the rankings. Google said it disagrees with the commission’s decision, which it has appealed.

Still, it agreed to allow comparison-shopping sites to buy ad slots in Google Shopping, a remedy the agency approved. Levin, the Google spokesperson, said 600 sites are participating, and their “click and impression” share grew from 20 to 40 percent amid 2018 and 2019.

Last year, 41 allegory arcade companies sent a letter to the agency saying that they “have not accomplished any abundant all-embracing access in traffic to their  websites.”

Google continues to abuse its market power, they said, and the industry has not rebounded, admitting the remedy.

“Many competitors died,” said Thomas Höppner, a accomplice at the law firm Hausfeld, who represented three Google Arcade competitors in the European Agency case. “And some argue that the industry will never fully recover.”

Originally appear on themarkup.org

Read next: Satoshi Nakaboto: ‘Bitcoin is now worth as much as Coca-Cola’

Pssst, hey you!