A handful of Internet service providers (ISPs) in the U.S. are redirecting search traffic around specific keywords to brands’ websites, presumably for affiliate marketing revenue.
A study released today by a UC Berkeley research group revealed that for some Internet users on some ISPs, using a search engine and typing in a word such as “apple” or “bloomingdales” would redirect the user to websites for Apple or Bloomingdale’s rather than to a page or search results about the keyword in question.
The Berkeley project, called Netalyzr, was created to measure DNS behavior. However, over the past few months, the Netalyzr team noticed some unexplained and unexpected redirections across at least 12 ISPs in the United States.
In a blog post on the findings, the team wrote, “The affected ISPs use services provided by a company called Paxfire to monetize certain web search requests. Paxfire’s main line of business is DNS-error traffic monetization, i.e., the practice of presenting advertisements and search results to users who mistyped a website’s address in their browser.
“In addition, some ISPs employ an optional, unadvertised Paxfire feature that redirects the entire stream of affected customers’ web search requests to Bing, Google and Yahoo via HTTP proxies operated by Paxfire.”
Following the money
The Electronic Frontier Foundation helped the Netalyzr team investigate the matter. As EFF senior staff technologist Peter Eckersley told VentureBeat, “They knew the general category of false DNS responses might be possible and worth checking for, while the details that emerged about Paxfire and what it was actually up to were a bit more surprising.”
“We knew that some forms of malware would change DNS results locally on a victim’s computer, so it made sense to look for such meddling,” said Vern Paxson, one of the Berkeley researchers.
The research team found that around 170 specific, brand-related keywords would trigger interference by the HTTP proxies, causing users to be redirected to affiliate marketing landing pages. “We don’t have a comprehensive list [of keywords], just a bunch of terms we tried, such as the names of popular web sites,” said Paxson. Although the team was testing only for single-word search terms, Paxson also said, “It’s possible that other searches are redirected too, but we haven’t tried that.”
Through the redirection process, the researchers wrote, “The ISPs and Paxfire presumably earn commission payments for the redirected flows.”
Some of the ISPs involved are, according to data presented by multiple organizations involved in the investigation, Cavalier, Cincinnati Bell, Cogent, DirecPC, Frontier, Fuse, Hughes, IBBS, Insight Broadband, Megapath, Paetec, RCN, Wide Open West and XO Communication. Charter and Iowa Telecom claim to have recently stopped doing DNS redirects.
While it’s likely that ISPs had at least some knowledge of at least some of the DNS redirection, if not search traffic redirection, it’s less likely that the brands themselves were involved in the scheme. “There is probably a chain of several intermediaries in these affiliate marketing programs between the brand itself and Paxfire,” said Eckersley.
“We would find it surprising that so many brandholders would agree to this sort of redirection, so we expect that they are not complicit,” said Paxson.
In other words, it’s difficult to say at the outset where the buck stops in this scheme and whose hands are in the cookie jar. What we do know is that many of the ISPs involved are claiming a lack of knowledge about the search redirects and pointing to third-party vendors as the real villains in the scenario.
A Charter representative told VentureBeat today that when search traffic redirects were occurring across that ISP, “We were not aware of it. It was a third party, and in a sit-down with the vendor, we said, ‘You need to be more careful about putting us into this mix… Charter doesn’t think this practice is acceptable.”
Steven Crosby of Frontier Communications Corportation told VentureBeat, “In terms of Frontier’s practices, we do not hijack any search traffic. We have clear business rules in our legal agreement with Paxfire that allows them to monetize URL address bar errors (e.g., ‘www.abc.cmo’ instead of ‘www.abc.com’ or typing an actual word like ‘PC’ into the address bar). Paxfire is not allowed to touch any search traffic that originates directly from toolbars or search bars.”
While the Charter rep was not able to name the exact vendor involved, Paxfire is just one of many Internet marketing companies that are using technical architectures for commercial and marketing purposes. These firms, which include companies like Barefruit and Golog, engage in murky practices such as search redirects, practices that violate our expectations of how the web should work and that rob us of any trust we might have in our ISPs.
If you use one of the affected ISPs, the EFF recommends running a Netalyzr test and installing a browser plugin such as HTTPS Everywhere to use HTTPS for all your web browsing “With HTTPS, attempts by the ISP or a company like Paxfire to alter the results would cause a certificate warning,” said Eckersley.
Google has also recommended using Google Public DNS and is beta-testing encrypted web search for users who want to better protect their search traffic.