Saikat Basu has a useful article over on MakeUseOf.com that looks at 10 search engines that you’ve probably never heard of but, as a researcher, might be useful. These sites don’t just explore the regular indexed pages that Google or Yahoo or Bing search. Instead, they provide access to what is called “the invisible web,” or the “deep web.” As Basu puts it:
Search engine technology has progressed by leaps and bounds. Today, we have real time search and the capability to index Flash based and PDF content. Even then, there remain large swathes of the web which a general search engine cannot penetrate. The term, Deep Net, Deep Web or Invisible Web lingers on.
To get a more precise idea of the nature of this ‘Dark Continent’ involving the invisible and web search engines, read what Wikipedia has to say about the Deep Web. The figures are attention grabbers – the size of the open web is 167 terabytes. The Invisible Web is estimated at 91,000 terabytes. Check this out – the Library of Congress, in 1997, was figured to have close to 3,000 terabytes! Link
The links he provides are interesting, with some of them being more applicable for certain research areas than others. I haven’t used any of them, but will probably play around to see how they work and if they might supplement the research tools I will have to as a grad student. (Personally, and this is probably something for another post, I think that more non-university affiliated people should have more access to some of the search and information resources that are currently limited to academic institutions and big libraries.)
Have you had cause to use any of these, or other “deep web” search engines for your research? If so, I hope you’ll leave a comment with some of your thoughts.
Powered by ScribeFire.