Deep Web Search Tools (Strumenti per navigare nel web profondo)

The deep web is the newest phenomenon in the Internet world.  The World Wide Web, which can now also be known as the surface web, has another side, vastly larger and mostly unknown until recently. This is the deep web.  The deep web has been defined as web content that is found in searchable databases.  This web content is of the type that can only be found by some type of direct query.  The deep web is also known as the invisible web.  The deep web is not really invisible, but because searchable databases are not indexable or queryable by today’s search engines, they appear invisible to the average Internet user as they search the Internet. 
Search engines are web sites, whose primary purpose is to enable people to find information on the web.  These devices and their related software have the formidable task of indexing or attempting to index the entire World Wide Web.  All search engines create and maintain their own enormous searchable databases.  Currently, the best and biggest search engines only index from one third to one half of the publicly available documents on the Internet. 
Search engines are designed to read flat web pages.  Flat web pages are like this one or one that you have created yourself. As web sites evolve and grow, it becomes more and more difficult to create individual pages because of the sheer size of many sites. Many web sites have turned to databases to create web pages on the fly when requested by a user.  The database contains the information, which is inserted into a web page template on demand.  Therefore, there is no flat pages ever created, hence the problem stated above.  If there are no flat web pages created, then there are no pages for the spider or crawler, or bot to index and thus no listing in the search engine database.

Recent calculated estimates by Completeplanet.com estimate the surface web at 1-2 billion documents, while the deep web is estimated to be a mind boggling 550 billion documents. So, is there a way to access the deep web?  Is there some new type of web searching technology that can access the deep web?  Yes, there is. The following are some of the sites that have been created to access the vast number of online databases or directly search them.

Deep Web Facts

LINKS

http://www.thebighub.com/ TheBigHub  The Big Hub maintains an index of over 3,000 subject specific searchable databases in over 300 categories.

http://www.beaucoup.com/   -  Beaucoup.com  This search site has links to over 2500 databases and directories.

http://www.completeplanet.com/ CompletePlanet.com.  This site has a free deep web search site that will search 100,000 of the 200,000 deep web database sites.  BrightPlanet also created a deep web search tool, called Lexibot.  Lexibot is a directed query engine.  It has the ability to query multiple search sites simultaneously.  Lexibot is downloadable from completeplanet.com.  Users can try it out for 30 days and then purchase it for a reasonable fee.   Lexibot is a true deep web search tool.

http://www.docdel.com/ Document Delivery Service.  Search for services that retrieve publically available documents, including patents, military specs, government reports, journal articles, etc.

http://gwis2.circ.gwu.edu/~gprice/direct.htm   Direct Search.com.  This is a site that has a growing compilation of links to the search interfaces of resources that contain data not easily or entirely searchable/accessible from general search tools.   This a very comprehensive site, well worth the time to view.  See also a related site - Price’s List of Lists at http://gwis2.circ.gwu.edu/~gprice/list.htm

www.findarticles.com FindArticles.  FindArticles.com is a vast archive of published articles that you can search for free. Constantly updated, it contains articles dating back to 1998 from more than 300 magazines and journals. You will find articles on a range of topics, including business, health, society, entertainment, sports and more. Unlike other online collections, each of the hundreds of thousands of articles in FindArticles can be read in its entirety and printed at no cost. For detailed information on how to use FindArticles, consult our Help tutorial.

http://www.fossick.com/ Fossick.com.  This site is also know as the WebSearch Alliance Directory.  It is a selective collection of over 3,000 specialist search engines and topical guides. There are thousands of search engines on the Internet.  Fossick.com aims to help users locate the best search tools for their search needs, resulting in faster and more accurate search results.

http://infomine.ucr.edu/search.phtml.  INFOMINE Multiple Database Search.  This site has a multiple database search option that allows users to search for subject-specific databases and full-text journal in academic disciplines, K-12, Internet and other useful categories.

http://www.invisible-web.net/  Invisible Web Directory  This selective directory of more than 1,000 specialized databases is the online companion to The Invisible Web: Finding Hidden Internet Resources Search Engines Can't See by Chris Sherman and Gary Price. Good descriptions, and direct links to search form pages. The directory lacks a search function.

http://dir.lycos.com/Reference/Searchable_Databases.  Invisible Web at Lycos.  This site has a comprehensive guide to searchable databases on the Web.

 http://www.invisibleweb.com/ Invisible Web.com.  This comprehensive site allows users to locate catalogs, archives, government, business and scholarly databases and much more.  This site has access to over 10,000 databases.

http://www.lii.org/  Librarians' Index To The Internet  This searchable annotated directory of Web resources, maintained by Carole Leita, is organized into "best of," "directories," "databases," and "specific resources." Each entry also includes linked cross-references, making it a browser's delight.

http://www.libraryspot.com/ Libraryspot.com. This site contains links to more than 2500 libraries around the world.

www.apple.com/sherlock Apple’ Sherlock   This site has information about Apple Computer’s Sherlock.  This program is an integrated part of the OS, offers the ability to search virtually any database through the use of plug-ins.

http://www.metasearchguide.com/ MetaSearchGuide.  The Meta Search Guide provides information about the ins and outs of meta search engines. There are a few search engines that enable you to search the deep sea on the web. The most intelligent deep web meta search engines are those that can deliver web page results straight from a keyword query. The majority of deep web meta search engines provide a two tiered approach whereby the searcher first selects a category of deep space search engines.  This site gives a detailed listing of those types of meta search engines.

http://www.webdata.com/  Web Data.com.  This is site is an Internet portal specializing in the cataloging, searching and comparison of online content from web sites with databases and is the first portal to uncover this formerly "invisible" part of the web. According to a recent study by Jupiter Communications, the driving force behind Internet use is the search for quality content. WebData.com is designed to provide quality content from databases and facts.

Search Engine Directory Sites

The following listing may also be helpful to search the deep web as many of the specialized search engines may include sites that have dynamic databases as part of their content

http://www.allsearchengines.com/ AllSearchEngines This site has general information about search engines and has 500 search engines organized into 27 subject categories.

http://www.leidenuniv.nl/ub/biv/specials Collection of Special Search Engines This site has a varied collection of search engines including engines that you might not find elsewhere.  Including medicine, graphical images, astrophysics, Celebrities, Classical Antiquity, Middle Ages, and Medieval.

http://www.finderseeker.com/ FinderSeeker  This site contains hundreds of search engines organized into 27 categories. This site also has the ability to use the search engines listed to search for information on 160 different countries.

http://www.freeality.com/ Freeality Internet Search  This is a top-notch search engine guide that focuses on popular and general subjects.

http://www.internets.com/ Internets  This site has the largest filtered collection of useful search engines and newswires anywhere on the World Wide Web.

http://www.lookoff.com/ Lookoff  This site is devoted to helping you to navigate the Internet using advanced tools and techniques that experts use. This site lists thousands of search engine sites. Remember that specific topics will be found more efficiently with specific search tools.

http://www.searchbug.com/ SearchBug  This site has over 500 search engines organized into 15 categories.

http://www.searchenginecolossus.com/ Search Engine Colossus This comprehensive site has access to search engines from over 100 countries and a small selection of specialized search engines in 15 broad subject categories and has a few search engines covering specific cities.

http://www.searchengineguide.com/ Search Engine Guide  This site has over 1000 search engines organized into 25 categories.  This site has exceptional good collections from Singapore, Japan, and China. It also has collections from many of the smaller Eastern countries in Europe.

http://www.searchenginewatch.com/ Search Engine Watch  This site contains great information about search engines and metacrawlers and helpful tutorials.

http://www.twics.com/~takakuwa/search/search Search Engines Worldwide This site has a large collection of search engines.  It has 1000 search engines from 138 countries.

http://www.zdnet.com/searchiq/subjects SearchIQ  This site has thousands of specialized search engines organized into 25 categories.

http://www.virtualfreesites.com/search Virtual Search Engines This site has a listing of thousands of search engines organized into 40 categories

http://www.webdata.com/ WebData.com  WebData is a database portal, specializing in finding, categorizing and organizing online databases, and providing annotated links with quality rankings.