April 2nd, 2002
Search Engines and Editorial Integrity; Is the jig up for honest search results?
By J.D. Lasica
Online Journalism Review
Many of us in the new media industry have watched in despair during the past
few months as several major search engines have abandoned all pretense at editorial
integrity by adopting deceptive, misleading advertising practices at the expense
of their users.
Finally, someone has stood up and said, Enough is enough. And now it’s time
for the rest of us to join the battle as well.
Commercial Alert, a 3-year-old consumer organization in Portland, Ore., founded
by Ralph Nader, filed a formal complaint with the Federal Trade Commission last
week, charging that eight of the major search engines were ‘inserting advertisements
in search engine results without clear and conspicuous disclosure that the ads
To which I say: Bravo! But also: It’s not enough. Better that the search engines
clean up their act on their own by bowing to their users’ wishes rather than
bend to government coercion. See below for how you can make your voice heard.
Why should this matter to journalists, researchers and other Net denizens?
Because search engines have become indispensable to our online existence as
we look for ways to sensibly navigate the Web’s 2 billion pages and 14 billion
links. Seven of the 10 most visited Web sites are search engines. A February
survey by the Pew Internet & American Life Project found that Internet users’
top two activities are e-mail and online searches.
The Net is becoming central to Americans’ everyday lives. A nationwide survey
by the Markle Foundation released this month found that by a huge margin, the
leading metaphor in the public’s mind for the Internet is ‘a library’ ? not
‘a shopping mall.’ But an increasing number of search engines don’t seem to
‘Crass commercialism over editorial integrity’
‘The problem has become acute lately,’ says Gary Ruskin, Commercial Alert’s
executive director. ‘Search engines have become essential to the quest for learning
and knowledge in the Internet Age, and we don’t want such an important platform
to be used to deceive the public and skew search results on behalf of hucksters.
They’ve chosen crass commercialism over editorial integrity.’
No one, including Commercial Alert, begrudges the search engines their right
to turn a profit and make an honest buck. It’s the honest part that’s at issue.
Here’s what has happened: As the dot-com meltdown and economic slowdown have
combined to pummel online ad revenues, Internet companies, including search
engines, have scrambled to come up with other ways to staunch the flow of red
Search engines have increasingly turned to two significant revenue streams:
? Paid placement: In addition to the main editorial-driven search results,
the search engines display a second ? and sometimes third ? listing that’s usually
commercial in nature. The more you pay, the higher you’ll appear in the search
? Paid inclusion: An advertiser or content partner pays the search engine to
spider its site and include the results in the main editorial listing. The result?
A site is more likely to see its pages appear in a search result, but there
are no guarantees.
There’s nothing intrinsically wrong with either of these business revenue streams.
A user who enters ‘star wars’ into a search engine may be doing research on
the Strategic Defense Initiative ? or she may want to buy plastic action figures.
The trouble is that many search engines have gone to great lengths to fuzz
the line between editorial and commercial listings. ‘This is a breach of the
editorial-advertising line,’ Ruskin says. ‘We don’t oppose advertising, of course.
We just support editorial integrity.’
Commercial Alert’s complaint names Lycos, HotBot, AltaVista, LookSmart, MSN.com,
Netscape, iWon and Direct Hit, owned by Ask Jeeves. All eight display paid listings
in addition to objective, algorithm-based results, but they call them ‘Featured
Links’ or ‘Partner Results’ rather than disclosing that they’re paid by sponsors.
In addition, MSN, Alta Vista and LookSmart accept payments for inclusion in
their editorial results.
So, for example, LookSmart returns three listings: Featured Listings (paid
listings), Directory Topics (spidered results) and Reviewed Web sites (including
paid listings). Clear as a bell, no?
Notably missing from the list are Google, Yahoo and Excite, three of the industry’s
leaders, which clearly are wearing the white hats in this contest.
Google, especially, has consistently staked out the high ground, telling Interactive
Week: ‘We think if we’re the only ones out there with a reputation for integrity
in our search results, that can only be good for us.’ A second Google spokesperson
told Searchenginewatch.com: ‘Our search engine results represent our editorial
integrity, and we have no plans to alter our automated processes.’
Google publishes paid listings, but clearly marks them as ‘Sponsored Links.’
And it has bucked the commercial tidal wave by not using paid inclusions. Chaste
almost to a fault, Google won’t display search results for tens of thousands
of pages that it has already crawled and indexed for internal use by corporate
partners such as Cisco and MarthaStewart.com ? because it doesn’t want to show
favoritism to its business clients. A rare virtue in these times.
Fittingly, last Wednesday Google Inc. won the Webby Award for best practices,
a new category that honors a single site that serves as a model of overall excellence.
An industry observer’s take
Danny Sullivan, editor of Searchenginewatch, has been one of the more seasoned
observers of the portal business in recent years, keeping tabs on paid placements
and paid inclusions by all the major search engines.
Sullivan, speaking by phone from Shrewton, England, is no anti-business zealot.
He observes, smartly, that search engines are still underutilized by advertisers
seeking highly targeted audiences. ‘It amazes me that there’s been so little
written about this aspect of search engines,’ he says. ‘There are very few places
where people tell you exactly what they want, in real time. Banner ads are a
poor tool to respond to those requests. The ability to interact with the users
right within a search results area ? that’s gold to the advertisers.’
And it should be gold to users, too ? if it’s done on the up and up. ‘We’re
seeing sites all over the Web displaying skyscraper ads and popup ads that are
untargeted, intrusive, in-your-face,’ Sullivan says. ‘We should be clamoring
for more targeted commercial search results, because they just might be relevant
to my needs. People need to understand that if you don’t support the sponsored
areas, the alternative is an all-commercial zone.’
That’s the big fear that Sullivan often hears from webmasters: that search
engines will eventually offer nothing but paid listings. ‘That’s not why most
of us are on the Internet,’ he says. ‘It’s more than just an online Yellow Pages.’
Writing a feature article on Aimee Mann? A business backgrounder on Dell? A
medical story on Propecia? A travel piece on Maui? Get ready to wade through
a torrent of commercial come-ons first when researching your story.
‘This is like one day opening your newspaper and finding it filled with nothing
but ads,’ says Sullivan, who has covered the subject in essays such as Buying
Your Way Into Search Engines, Can Portals Resist the Dark Side? and The Evolution
of Paid Inclusion.
Both Sullivan and Ruskin point out that many of the smaller search engines
engage in similar practices, or worse. Dogpile used to be a fairly useful search
engine, but search the Web on Dogpile today and you’ll find page after page
of paid commercial links (from GoTo, FindWhat, Sprinks, ePilot, ah-ha, BrainFox
and Kanoodle) before it displays anything remotely relevant or objective. (Studies
show that the vast majority of users don’t bother looking past the top 30 results.)
Also falling from grace into the pitfires of hell is Disney’s Go, which once
operated one of the slickest search engines on the Web in Infoseek but now merely
coughs up results from GoTo’s commercial engine.
The empire strikes back
What’s the reaction of search engines to all this? LookSmart and Alta Vista
issued statements dismissing the charges as groundless. A few weeks earlier,
LookSmart’s CEO, Evan Thornley, told the San Francisco Chronicle his managers
did not even raise the subject of ethics when the company adopted a pay-for-placement
program last month. ‘We can’t afford to have ideological debates anymore,’ he
Sarah Lefko, a product manager for MSN.com, added that surveys by Microsoft
show that consumers already assume that all search results are for sale. (How
conveniently self-serving ? and what rubbish.)
Matt Stoever, a vice president at Lycos, told the New York Times: ‘We thought
long and hard and decided it doesn’t matter if we are paid for a link, so long
as the results are what the user wants. ... The industry has trained users to
avoid anything that looks commercial. By calling them paid listings, it hurts
Such is the arrogant, Alice-in-wonderland, upside-down world of Internet executives
these days. Black is white, profit is all, and the entire Internet is filled
with callow users who are either (a) so cynical that they assume all search
results are bought and paid for, or (b) clueless droids who have somehow been
goaded into ignoring wonderfully useful commercial listings.
Says Ruskin: ‘Their arguments aren’t credible. If what they’re claiming is
true, why wouldn’t they be straight with the users and reveal that their ads
are ads? The fact that they’re hiding and obfuscating that fact proves that
Have your say
Ruskin says it’s too early to tell whether the FTC will move aggressively to
act on the organization’s complaint, noting that chairman Timothy J. Muris took
over the agency’s reins only last month. But he notes there are extensive precedents
for the FTC to intervene. ‘For example, they’ve repeatedly told makers of infomercials
that they have to disclose to the public that they’re paid ads rather than independent
programming,’ he says.
Personally, I’m not counting on the Bush appointees on the FTC to defend my
interests as a consumer. I’m voting with my mouse clicks by doing my searches
on Google and Yahoo.
But I also think the out-of-touch search engines could use some interaction
with their audience. Ruskin agrees: ‘Yes, we’re hoping that users will switch
to sites that have more editorial integrity, and that they’ll make their voices
heard. It’s a very simple request: We don’t want trickery.’
Here’s the message I hope the search engines will hear and begin to heed:
We want our search results clean and unsullied. Display your paid listings,
too, in a separate area, but be honest and upfront about them. Tell us your
practices in a straightforward disclosure statement, and make it easy to find.
Don’t deceive us, and don’t belittle us by saying we’re too shallow to care
about editorial integrity.
If you’d like to give the search engines a piece of your mind, here’s where
to contact them:
* Direct Hit
And while you’re at it, toss a little praise to Google, Yahoo and Excite as
well. Editorial integrity needs all the friends it can find these days.