What is the Invisible Web?
"The "visible web" is what you see in the results pages from general web search engines. It's also what you see in almost all subject directories. The "invisible web" is what you cannot retrieve ("see") in the search results and other links contained in these types of tools.
- Searchable Databases. [more]
- Most of the invisible web is made up of the contents of thousands of specialized searchable databases that you can search via the Web. The search results from many of these databases are delivered to you in web pages that are just for your search. Such pages very often are not stored anywhere: it is easier and cheaper to dynamically generate the answer page for each query than to store all the possible pages containing all the possible answers to all the possible queries people could make to the database. Search engines cannot find or create these pages. More explanation.
- Excluded Pages. [more]
- There are some types of pages that search engine companies exclude by policy. There is no technical reason they could not include them if they wanted. It's a matter of selecting what and what not to include in databases that are already huge, expensive to operate, and low revenue producers. More explanation.
and search engines cannot find or recreate them. You have to go to the page with a search box for each specialized database and search it. Additional invisible web pages are ones that search engines choose for various reasons to exclude.
Excerpt from:
Barker, Joe. Invisible Web: What it is, Why it exists, How to find it, and Its inherent ambiguity. Finding Information on the Internet: A Tutorial. 28 August 2003. UC Berkeley - Teaching Library Internet Workshops. 17 Nov 2003. http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/InivisibleWeb.html
Posted by iachan on Monday, 17 November 2003 @ 9:28 PM