Keeping the Search Engines Happy

Charlie Morris

February 28, 2000

By now, even the smallest child knows that having your site listed in the major search engines is the key to Earthly happiness. Human nature being what it is, there are those (quite a lot, actually) who would "beat the system," and try to trick the search engines into giving them more coverage than is rightfully theirs. The searchies wage an ongoing battle against these scofflaws, and if you aren't careful, you can be caught in the middle. Even if all you're trying to do is honestly promote your site, you may inadvertently violate one or more search engines' guidelines, and be unjustly barred from their listings. Here's an overview of what's acceptable and what's not, with tips on how to stay on the right side of the line.

While search sites may be thought of as a sort of public resource, in fact they are for-profit services, and fiercely competitive with one another. Thus, the goal of every search site is to have the very best database of sites that they can. This means two things: including as many legitimate entries as possible, and excluding as many of the spurious kind as possible. By this I don't mean that they pass judgement on the quality of the actual sites submitted (although some directories do to a certain extent), but rather on the appropriateness of a submission of a particular site for a particular category. If a directory accepted every entry that is submitted to them, then every category would soon become filled with listings for get-rich-quick schemes, weight-loss products and pornography, which would crowd out the more appropriate listings and make the directory useless. Of course, on the less reputable search sites, including those entities known as "free for all link pages", this is exactly what happens, which is why it's a waste of time submitting to such sites. But reputable directories expend a great deal of effort making sure that sites do not get listed in inappropriate categories. Likewise, search engines try to make sure that all their listings include only appropriate keywords.

Of course, a search site dares not be too strict about accepting submissions, because they want to get as many legitimate submissions as they can. Make no mistake, search engines value your submissions, and the better ones make it as easy as possible for you to submit. But they have to walk a fine line between welcoming valid submissions and keeping the bad apples out. And bad apples there are in plenty. The same hordes of sleazy characters and snake-oil salesmen that fill our e-mail inboxes with spam constantly attempt to do the same to the search sites, and like the evil bulk e-mailers, they have a large repertoire of tricks with which they attempt to deceive the gatekeepers.

So, you certainly don't want to offend the search engine gods, but at the same time, you want to get as much coverage as you can, in order to drive as much traffic as possible to your site. Getting multiple listings, and trying to make your pages appear as high in search results as possible, are quite desirable. They are also perfectly allowable, as long as you do things the right way.

A lot of people spend a lot of time trying to devise dishonest ways to increase their coverage in search engines and directories. The majority of these outlaws are of course purveyors of fraudulent products, or operators of one dishonest scam or another. However, sometimes perfectly legitimate businesspeople can, either inadvertently or through a misguided excess of zeal, fall foul of the search engines, and be branded spamdexers, especially if they have employed an unscrupulous or incompetent paid submitter. The irony of the situation is that these attempts at deception are almost always unsuccessful. The search sites' livelihood depends on foiling such attempts, and they usually manage to stay one step ahead of the scamsters.

Search-related fraud falls into the following categories:

Spamdexing: Submitting the same Web page again and again to the same search sites, submitting to directories in inappropriate categories, or submitting a bunch of different URLs that all point to the same page.

Keyword Stuffing: Loading Web pages with an excessive number of keywords, hidden keywords, or keywords that are inappropriate for the site's subject matter.

Bait and Switch: Creating a Web page specially designed to rank high in search engines, which redirects the user to another unrelated page.

Now, you don't want to indulge in any of these nefarious practices, but you do want to get your pages listed in as many different places as possible, and you do want to use keywords to optimize your rankings. To understand how far you can go with these techniques, let's look at search engines and directories separately.

Keeping the Search Engines Happy

Search-related fraud falls into the following categories:

Contents: