Search engines gather data from sites all around the web and allow people to search that data using words and expressions. How they do this differs from one to another, and many want to keep their policies secret. The key to being found is selecting the right engines to be listed on and making sure your site is well designed and has the correct content.
There is a huge amount of information on the internet about search engines; below is a brief overview, for more detailed reading we suggest you try searchenginewatch.com where there is a plethora of information. See link at bottom of page.
What do the stats show?
The total global internet universe is, at the time of writing, just under 463 million people (source: Nielsen Netratings). The most visited sites with search facilities, are AOL, Yahoo, Google, (Terra) Lycos, Ask Jeeves and Excite. Of these Google is potentially the most important as it is the biggest pure search engine on the web (the others are online communities as well) provides data to many other sites.
Why Not Submit to All Engines?
There are thousands of search engines out there and most get very little useful traffic and simply duplicate information found on the bigger engines. In fact many smaller engines simply buy data from the bigger ones. Submitting to smaller enignes can take up a lot more time (and therefore money) and may well open you up for serious amounts of spam, junk email, email advertising and so on. This can range from mildly annoying to having your mailbox flooded with junk. The only time we would recommend submitting your site to a smaller engine is if you know it is particularly useful for a niche product/market.
With regard to smaller engines, it is worth thinking about who uses them and why are they out there. They are there to make money, mainly by selling advertising space and trading the information they gather, your information eg email address and other details. The thing to remember is they have exactly the same problem as you do, getting known. If you have not heard of these smaller search engines who else has, and is it really worth getting listed on them? It is difficult to imagine anyone who would exclusively use these engines without touching Google, Yahoo etc.
Which Engines Do You Recommend?
There are two main types of engine we recommend concentrating on. The first are the large independent engines and the second are the ISP run engines.
Independent engines include Yahoo, Google, Excite and Lycos.
ISP engines include AOL and MSN.
As well as containing their own data, ISP engines generally use data supplied by other independent engines, such as Google. This demonstrates the importance of being well registered with these engines.
This is the most important part. Firstly work out what you expect other people to use to find your site. Make a list of these words (or short phrases) and work them into your content. These words can also be placed in 'meta tags' on each page of your site. Meta tags are not visible on screen but are read by most search engine spiders when they visit you site.
Design
The first thing to do is make sure the site construction is suitable. Most search engines send out small programs called spiders to collect information from your site that will then be used to rank it. If the spiders are hampered by poor design then your position will suffer. Frames and graphical index, or gateway pages with no text are a no-no, as are Flash and other multimedia pages. This does not mean you cannot use them, just don't use them for your whole site or exclusively on key pages. Intelligent naming of pages and directories can help.
Robots and Spiders
Spiders, also known as robots, can be guided to the appropriate parts of your site and kept away from private or unimportant areas by using a robot file. This is a text file that contains information on where spiders should go. You can set global rules that apply to all spiders or have rules spefically targetted at selected spiders.
Links
Another way to improve rankings is through links. An engine will search its own database to see how many sites it already lists have references to your site. The more it finds the more important it will judge your site to be and therefore the higher it will rank it. It is also good to have your site linked to by as many other web sites as possible as this will almost always drive more traffic your way.
There are many more subtle changes that can be made to improve your ranking but these go beyond the scope of this document. Ultimately it comes down to good design, good content and how much competition you have on the web.
For general web trends and statistics try the following websites:
For further reading please try the following sites. There is everything from the official W3 body to search engine information, and the importance of cross platform/browser compatibility and good design...
