Downloaded from http://www.ndu.edu/ndu/library/search/search1.html
Click HERE to return to AWC Internet Searching Page.
Searching Home Page Search Tools Searching Hints Tutorials & Training Evaluating Web Sites Other Resources
Library Home Page Boolean Search Citing Electronic Resources Class Searches Updates Class Examples

Searching the Internet

Resources Overview


Search Engine: The software that searches an index and returns records which match the specified criteria. There are many types of search engines but most use a database of keywords automatically retrieved from web pages.




Subject Directories

Hierarchical subject classification schemes are the defining characteristic of most subject directories. You 'drill down' to the level of entries which best meets your needs. Directories of web sites typically have a large amount of human effort involved in the selection, evaluation and indexing of web pages and therefore are considerably smaller than the machine built indexes. Most often these databases are built of URLs submitted by individuals to the directory but humans or web spiders may selectively search out new sites to be included.

Subject directories are good starting points for general topics or topics where you expect to find large quantities of information. Because of their selectivity a potentially large body of literature is reduced to a few 'best' sites. When you know little about a subject area these sites can provide the keywords and concepts needed for use in other search engines. Since these sites have been evaluated they are often quite good, as are the links they contain

Typical subject directories include
LookSmart
Lycos Top 5%
Yahoo
Master List


Search Engines (general database)

General database search engines are the most common search tool encountered. These search engines create and search only their own database not relying on external resources. Their databases are built by web spiders or robots which search the internet for sites to be indexed; reported databases of 25 - 100 million citations are being claimed by their owners.

Excellent starting points under almost any circumstance, these search engines are preferred for controlling the breadth and depth of searches and your best results may come from them. They are rapidly developing sophisticated search capabilities that may soon rival the pay-for-use databases in results achieved.

Typical search engines include
Google
Teoma
Alta Vista
HotBot
Lycos
Master List


Meta Search Sites

Meta Search sites allow you to input 1 search statement which they use to search many single search engines simultaneously; they rely wholly on external resources and have no database of their own. Often, they will consolidate the resulting citations in order to eliminate any duplication.

Meta search engines are useful when you expect to find few materials on your topic; their ability to search multiple databases concurrently will quickly identify which database has materials meeting your criteria. The disadvantages are less apparent; search depth is shallow, the ability to refine searches is quite limited, they may overwhelm you with seemingly relevant materials, and, often, you have no choice in what search engines are used.

Typical meta search engines are
Dogpile
MetaCrawler
Master List


Collected Search Engines

Web pages exist which collect links to general and specialized search engines; collections of more than 100 different search engines are not uncommon. Some permit you to enter your search directly from their page; these pages are convenient when you want to conduct a survey of the information available on the internet since they allow you to repeat searches in many search engines from one location. This is a very successful strategy to locate search engines strong in the topic you are researching.

Examples of these sites include
Beaucoup
Search.Com
Master List


Directory Services

These are the white and yellow pages of the internet. Businesses may also be found by searching through the normal search engines.

Representative are
Yahoo People Search
WhoWhere
Master List


Intelligent Agents (Personalized Software)

A diversity of software, commercial and non-commercial, is available to assist you in managing or retrieving information on the internet. Typically this software is called a 'bot (for robot), spider, intelligent agent or agent. Some of the more popular commercial products work as a meta search engine which you install on your computer; these products can be personalized and customized in many ways.

BotSpot
A source for keeping up on all types of intelligent agents
http://www.botspot.com/

Web Ferret Pro  (not free)
A meta search package customizatable to meet your needs.
http://www.ferretsoft.com/netferret/

Copernic (not free)
Another meta search agent fully customizable to your needs.
http://www.copernic.com/