Search Tools - Advanced Features Search engines often have many advanced features which can focus your search results. Using these advanced features is often as simple as reading the help for each search tool that you use. It amazes me that users don't have time to the read the help, but they do have time to wade through 100,000 worthless hits.

There are hundreds of search engines which vary in size, methodology and features. To evaluate the search engines, you may want to do your own controlled experiment where you use the various search engines using the same query term. You should also recognize the difference between a subject directory (Yahoo) and a search engine (Alta Vista). Subject directories are best used during the early stages of your search, if you do not know much about a topic. The search engines are most useful when you are searching for a very specific term or concept.

When using any particular search engine, you should take the time to familiarize yourself with the default settings for that search engine and learn what the advanced options are. Most users simply type-in a couple of key words and get thousands of query results. Next a user may add several additional words to the query string, thing that this will help specify the search. What most users do not realize, is that the default logical operator in most search engines is an "or" (not "and"). Adding more words to a query string can often dilute your search results.
 

Alta Vista

Alta Vista has two different query modes: Simple and advanced. Each mode also has its own "help page" which contains a wealth of information. I encourage you to pull-up the help screen for each query mode and print it. This way you can refer to it while composing a query string. You will see that Alta Vista supports a full set of logical operators and lets you control the prioritization of the query results

Many of the more interesting search options in Alta Vista stem from the way Alta Vista builds its index. Alta Vista not only indexes the full text displayed on every web page it harvests, but is also indexes the HTML of each page. Alta Vista will let you search for terms in conjunction with various HTML attributes. Consider the following examples:
 

search string	returns web pages which :
gigabit	contain the word"gigabit"
gigabit testbed	contain the word "gigabit" or "testbed"
gigabit host.gov	contain the word "gigabit" and are hosted in US Government web sites
gigabit host.uk	contain the word "gigabit" and are hosted in United Kingdom web sites
link:*.whitehouse.gov	contain a hyperlink to the whitehouse web site
link:*.whitehouse.gov -host.whitehouse.gov	contain a hyperlink to the whitehouse web site, but are not themselves located at the whitehouse web site

These options are also helpful is searching a particular web site which does not offer its own web site search capability. Just go to Alta Vista and search for "keyword host:*.website.com"

To see the kind of "poor" search terms that other users try, look at Metacrawler's Metaspy. Magellan Search Voyeur, or the WebCrawler Search Voyeur.

Popular Search terms: searchterms.com, lycos50. Google Zeitgeist,
 
 
 

Dejanews

Dejanews is a searchable index of the full text of all messages in all the Usenet newsgroups. Once again, I encourage you to go beyond just the default query interface. Your first stop will be Dejanew's Power Search Form. The Power Search Form contains a first level of options where you can control the search. You should note that each option name is a hyperlink to the power search help page which guides you in setting the parameters for your search (take the time to look at the power search help page) One of the most useful options to set, is the "usenet database" option. By default, Dejanews searches its "current database" which only covers the past month or so of messages. To go back over a year into the newsgroups, you need to select the "old database"

One of the most powerful features of Dejanews, is its ability for you to define a Query Filter. With a query filter, you can perform your keyword search against a very detailed subset of the entire dejanews database. As shown in the course handouts, you can construct a query which shows all messages from government employees posted into the "comp" newsgroups during a 2 month window.

Another interesting feature of Dejanews is called the "Author Profile" When reading a particular posting via Dejanews, The "Author Profile" will search Dejanews for that author's email address, and show you all the messages that person has posted into any newsgroups. This can be an invaluable tool to get a sense of what that person is all about. Organizations can also use Dejanews to see if their employees have been conforming to the organization's Access policy with regards to the Newsgroups. All you have to do, is limit you search to authors from *@company.com
 
 
 

Search engine limitations

AltaVista and Hotbot may seem to cover a vast majority of the Internet's web pages, but they do have significant blind spots:

• They obey The Robot Exclusion Standard. The web site being visited by a search engine's spider (robot) the web site can limit which directories and files the search engine is allowed to "harvest" (copy and index) A web site may contain a file called "robots.txt" which spells out where a robot is allowed to go, and even which robots are allowed to copy a site. For example, read these "robot.txt" files located at www.disney.com/robots.txt and www.cisco.com/robots.txt. You can even tell which search engines Yahoo keeps out in their robots.txt file. See The Web Robot Pages ,  Robot Exclusion Standard Revisited , and robotstxt.org for details.
• Some sites require a password. If you have to be a registered user to access a site - then that site's content will be missed by the search engine. For example, the vast majority of the Wall Street Journal is not searchable through the search engines. (although the search engine can help you discover the existence of The Wall Street Journal)
• Many web pages are generated by database queries. For example, at Amazon Books most of the information about 1 million books exists in their database. The web pages describing a particular book, author, publisher, etc are only created in response to User queries. The search engines are very good at discovering pre-defined web pages by following hyperlinks - but they can't interact with web site databases by filling-in an online query form.

If search engines are giving you thousands of "hits", it is time to start using the advanced features of the search engine. take the time to learn how your favorite search engine works and how it interprets your search query

Contact me at 703-729-1757 or  Russ 'at' navigators.com  
If you use email, put "internet training" in the subject of the email.
Copyright ©  Information Navigators