Damn good idea, if sketchily implemented at first. Yahoo! has released a new beta—Yahoo! Search Subscriptions, a preliminary attempt to uncover some commercial sources of the "hidden Web" (or what Yahoo! in its search blog, calls the "deep Web"). The well-known problem with Google Scholar is that it uncovers far more citations than actual scholarly text; you need subscription access to academic databases for Scholar to be ultimately useful. Yahoo! doesn't go near the academic database, but has developed a crawler for a small handful of walled consumer content destinations such as the Wall Street Journal, Consumer Reports, Forrester Research, TheStreet.com, and others. You must bring your own subscriptions, the value of the engine being a one-stop metasearcher for those sources.

With the model in place, Yahoo! needs only to expand its range of agreements with sources. But what I'd really like, as I was recently discussing with Naba Barkakati in his blog, is for Google or Yahoo! to act as a subscription broker, much as a university library does on behalf of its students and faculty. Then a premium search service could be offered by subscription: we pay Yahoo! or Google, not the individual sources, for access to large chunks of the hidden Web.