Deep Web Search Tools (Strumenti per navigare nel web profondo)

The deep web is the newest phenomenon in the Internet world.  The World Wide Web, which can now also be known as the surface web, has another side, vastly larger and mostly unknown until recently. This is the deep web.  The deep web has been defined as web content that is found in searchable databases.  This web content is of the type that can only be found by some type of direct query.  The deep web is also known as the invisible web.  The deep web is not really invisible, but because searchable databases are not indexable or queryable by today’s search engines, they appear invisible to the average Internet user as they search the Internet. 
Search engines are web sites, whose primary purpose is to enable people to find information on the web.  These devices and their related software have the formidable task of indexing or attempting to index the entire World Wide Web.  All search engines create and maintain their own enormous searchable databases.  Currently, the best and biggest search engines only index from one third to one half of the publicly available documents on the Internet. 
Search engines are designed to read flat web pages.  Flat web pages are like this one or one that you have created yourself. As web sites evolve and grow, it becomes more and more difficult to create individual pages because of the sheer size of many sites. Many web sites have turned to databases to create web pages on the fly when requested by a user.  The database contains the information, which is inserted into a web page template on demand.  Therefore, there is no flat pages ever created, hence the problem stated above.  If there are no flat web pages created, then there are no pages for the spider or crawler, or bot to index and thus no listing in the search engine database.

Recent calculated estimates by estimate the surface web at 1-2 billion documents, while the deep web is estimated to be a mind boggling 550 billion documents. So, is there a way to access the deep web?  Is there some new type of web searching technology that can access the deep web?  Yes, there is. The following are some of the sites that have been created to access the vast number of online databases or directly search them.

Deep Web Facts

  • 400-500 times larger than the surface Web
  • 7,500 terabytes of info vs 19 terabytes of info
  • 550 billion document vs 1-2 billion documents
  • Over 200,000 deep Web sites presently exist
  • Deep Web sites receive 50% more monthly traffic than surface sites (average)
  • More than ½ of deep Web content resides in topic-specific databases
  • 95% of the deep Web is publicly accessible information and not subject to subscription fees

LINKS TheBigHub  The Big Hub maintains an index of over 3,000 subject specific searchable databases in over 300 categories.   -  This search site has links to over 2500 databases and directories.  This site is also know as the WebSearch Alliance Directory.  It is a selective collection of over 3,000 specialist search engines and topical guides. There are thousands of search engines on the Internet. aims to help users locate the best search tools for their search needs, resulting in faster and more accurate search results.  Invisible Web Directory  This selective directory of more than 1,000 specialized databases is the online companion to The Invisible Web: Finding Hidden Internet Resources Search Engines Can't See by Chris Sherman and Gary Price. Good descriptions, and direct links to search form pages. The directory lacks a search function. This site contains links to more than 2500 libraries around the world.

Search Engine Directory Sites

The following listing may also be helpful to search the deep web as many of the specialized search engines may include sites that have dynamic databases as part of their content Freeality Internet Search  This is a top-notch search engine guide that focuses on popular and general subjects. Internets  This site has the largest filtered collection of useful search engines and newswires anywhere on the World Wide Web. SearchBug  This site has over 500 search engines organized into 15 categories. Search Engine Colossus This comprehensive site has access to search engines from over 100 countries and a small selection of specialized search engines in 15 broad subject categories and has a few search engines covering specific cities. Search Engine Guide  This site has over 1000 search engines organized into 25 categories.  This site has exceptional good collections from Singapore, Japan, and China. It also has collections from many of the smaller Eastern countries in Europe. Search Engine Watch  This site contains great information about search engines and metacrawlers and helpful tutorials.