Bus. Std: Searching the Net -- New Technologies

My latest column in Business Standard:

One of the defining battles in the mid-1990s was between Netscape and Microsoft over control of the desktop. Netscape threatened Microsoft’s Windows lock with its web browser. Microsoft fought back with a vengeance and finally won, as a marginalized Netscape was bought by AOL. Now, there is another battle that’s shaping up which could be equally defining for the future of computing.

This time, the attacker is Google. Over the past few years, Google has become the search engine of choice. As its dominance has soared, so have its ambitions. Over the past couple years, Google has extended itself beyond search to other areas organically and via acquisitions. In recent times, the wheel has come a full circle with speculation rife that Google may be planning to launch its own browser.

There are two parts to the story as we see unfolding as Google, Yahoo and Microsoft, along with a host of others, work to define tomorrow’s interface to the information web. The two parallel threads consist of building better search engines and creating richer interfaces. The search engines are the backend to solve the information overload problem, while the interfaces are the doorways to the world of content and applications.

We will first discuss advances in search technologies. Later, we will look at how we will access this emerging world of “service-based computing.”

The problem of search is one of plenty. There is a lot of data on the web that needs to be converted into useful information. Search is one the solutions to the proliferation of data that has taken place with the growth of the Internet. As John Battelle of Searchblog put it recently: “Search is our response to the extraordinary info-abundance in which we're all awash.”

Google’s PageRank technology helped it separate the wheat from the chaff. In a recent article on Google’s history, the Economist (Technology Quarterly, Sep 16, 2004) explained how the algorithm works: “PageRank works by analysing the structure of the web itself. Each of its billions of pages can link to other pages, and can also, in turn, be linked to. [Google’s founders] Mr Brin and Mr Page reasoned that if a page was linked to many other pages, it was likely to be important. Furthermore, if the pages that linked to a page were important, then that page was even more likely to be important. There is, of course, an inherent circularity to this formula—the importance of one page depends on the importance of pages that link to it, the importance of which depends in turn on the importance of pages that link to them. But using some mathematical tricks, this circularity can be resolved, and each page can be given a score that reflects its importance.”

The search of today can be considered in the C-prompt era, and needs an upgrade. So, what will be the Windows of the search era? In an interview with ACM Ubiquity, Ramesh Jain, professor of computer science at Georgia Institute of Technology, explains what needs to be done: “Current search engines like Google do not give me a ‘steering wheel’ for searching the Internet. The search engines get faster and faster, but they're not giving me any control mechanism. The only control mechanism, which is also a stateless control mechanism, asks the searcher to put in keywords, and if I put in keywords I get this huge monstrous list. I have no idea how to refine this list. The only way is to come up with a completely new keyword list. I also don't know what to do with the 8 million results that Google threw at me. So when I am trying to come up with those keywords, I don't know really where I am. That means I cannot control that list very easily because I don't have a holistic picture of that list. That's very important. When I get these results, how do I get some kind of holistic representation of what these results are, how they are distributed among different dimensions…Two common dimensions that I find very useful in many general applications are time and space. If I can be shown how the items are distributed in time and space, I can start controlling what I want to see over this time period or what I want to see in that space.”

One glimpse of search innovation comes from Amazon with its A9 search engine, which is built around Google’s search results, and also integrates Amazon’s own book search results. John Battelle explained A9’s approach in a column for Business2.0: “A9 has broken search into its two most basic parts. Recovery is everywhere you've been before (and might want to go again); discovery is all that you may wish to find but have yet to encounter. A9 attacks recovery through its original Search History feature and its integrated toolbar, which tracks every site you visit. But new to this version of the site is a feature A9 calls ‘Discover,’ which finds sites you might be interested in based on your click stream and -- here's the neat part -- the click streams of others…A9 is more of a Web information management interface, with search as its principal navigational tool. [It is] betting that over time, Web users will come to recognize, then demand, that their search service not only find sites based on queries but also remember where they have been and what they have clicked on.”

A few years ago, it seemed the search game was over. Results were inaccurate and portals were the thing to do. Google’s cutting-edge technology of linking resurrected an industry. Yet, the innovation in search is far from over. The game has just begun. In the next column, we will look at some key ideas which will define tomorrow’s search.

Softbank's Son on Broadband

AlwaysOn Network has excerpts form an interview with Softbank's Masayoshi Son (from the WSJ conference 'D: all things digital'):

I think the TV or the PC or the handheld device all become just a subset terminal output device for the internet broadband service. So in broadband service you have lots of content or services in the server, in the network. You don't need that much on the local thing. Everything would be network computing, and innovation would happen along the network. I think it would be total set that makes it very powerful.

TV advertising is 40 billion dollars in Japan. There is no reason why broadband advertising will be less, because every TV will become robust. The new television will be all broadband. Every television set will be connected to broadband, so ten years from now you won't call it a television set if it is not connected to broadband. Broadband will provide all the advertisements for every television.

[Users] are buying the highest-speed internet service, 45 megabits, and the wireless [mobile services] and Voice Over IP and security, and so on. We are adding the television service on top of that. I think that average revenue per user would become $60 or $70...Today we are charging $20 per month for television. It's just the beginning, so I think we will have a much bigger base of users for the television, the average revenue will become $60 or $80.

Thin Client News

Linux Devices has a round-up: "Thin client terminals and systems have been around for decades, and Linux terminals have been around almost as long as Linux. Thin clients were even the focus of a brief computing craze in the late 90s, when IBM, Oracle, and Sun all launched ambitious "network computer" initiatives. The fad was shortlived, however, as Sun employees reportedly began bringing their laptops to work, and using their company-issue surplus SunRay thin clients (that no one was buying) as doorstops, while Oracle's New Internet Computer (NIC) spinoff died a quiet death in June of 2003. A year later, though, thin is back in again, however. IBM, Novell, HP, and Red Hat have all been talking up Linux-based thin clients. Thin client marketshare leader Wyse has redoubled its Linux efforts. And, the Linux thin-client pioneers like SmartFlex and Neoware haven't been idle, either. Even Microsoft has joined in; see this recent whitepaper for an outline of its thin client strategy."

Jotspot and Wikis

WSJ writes:

Wikis are Web pages that users can write on as well as read. The concept, developed in the mid-1990s by programmer Ward Cunningham, has evolved as a way for work groups to create documents that can be updated continuously.

The popularity of wikis attracted Joe Kraus and Graham Spencer. The two men in 1993 helped to start Excite -- which was sold in 1999 to @Home Corp. -- and have now formed JotSpot, a start-up that hopes to make wikis easier to set up and to program. That way, companies can more quickly create simple Web-based software to handle chores such as tracking job applicants or customer calls, Mr. Kraus said.

At the moment, many companies find it too difficult or expensive to tailor business applications themselves. In many cases, they resort to managing common tasks using a combination of Microsoft Corp.'s Excel spreadsheets and e-mail, Mr. Kraus noted. That poses problems, since it can be hard to keep track of information that can be stored in many mailboxes and isn't automatically updated.

JotSpot, a closely held company in Silicon Valley, has raised $5.2 million from venture-capital firms Mayfield and Redpoint Ventures. It plans to offer easier ways to edit wiki pages. It also will offer templates for simple business applications as well as easier ways to send e-mail to and from wikis. In addition, instead of making customers run wiki software on their own servers, it plans to offer wikis as a service managed on its own computers, though it hasn't announced pricing plans.

Peter O'Kelly, an analyst with market researcher Burton Group, says JotSpot made a good decision in adding database technology to help users structure and sort through information. Existing wikis can fill up with information that is hard to manage, he said.

Jeremy Zawodny adds from Web2.0:

Joe is doing a short demo of JotSpot, a new generation Wiki that makes it easier to add structured data to the system in addition to pulling in external data via RSS, Web Services, and so on. Very cool stuff. Their Wiki is easily scripted, so I can imagine using it to protoype loosely coupled web applications.

He's showing SalesForces.com integration via SOAP right now. Nice. He's pulling in Yahoo News RSS feeds for topics on his pages.

So it's a Wiki with structure that you can evolve into a full-blown application. This feels like Lotus Notes re-invented for the web or something.

Insider Pages

News.com writes about the newest idea from Idealab's Bill Gross:

Called Insider Pages, the Web site lets people sign up to connect with friends and mine their recommendations for local shops and services. The free product, still in experimental form for Los Angeles residents only, puts a new spin on social-networking services like Friendster by infusing it with the local insider feel of Craigslist.

Insider Pages is entering a market rife with promise, at least as investors and big-league Internet companies see it. In the last year, Yahoo, Google and InterActiveCorp's Citysearch.com have invested heavily in improving local-search services and mapping in order to lure new visitors and to begin to attract regional advertisers online. The prize could be big; small and midsize businesses spend about $22 billion on local advertising annually, according to The Kelsey Group.

Insider Pages could also bring a more focused business model to online social networking, in which people sign up to meet new friends or date. Although services such as Friendster and Google's Orkut have been popular, investors and analysts wonder if they can keep people's interest and prove to be a viable businesses. Insider Pages plans to make social networking valuable by helping consumers find trusted advice on services. That, in turn, will be valuable to advertisers, MacFarlane says.

"The most important driver for people when selecting local business providers is personal recommendations," said MacFarlane, who has worked at Idealab for two and half years and spun off Insider Pages into a new company. "It's the way the real world works."

He added: "We're aggressively going after local businesses to advertise online."

Insider Pages will sell advertising space at the top of search pages, charging marketers only when people use their service, or with a "pay per lead" model.

Some good ideas in there for PIN-News...

TECH TALK: The Network Computer: The Internet OS

Tim O’Reilly discussed the idea of the Internet Operating System (built around open-source software):

The first generation of any new technology is typically seen as an extension to the previous generations. And so, through the 1990s, most people experienced the Internet as an extension or add-on to the personal computer. Email and web browsing were powerful add-ons, to be sure, and they gave added impetus to a personal computer industry that was running out of steam.

(Open source advocates can take ironic note of the fact that many of the most important features of Microsoft's new operating system releases since Windows 95 have been designed to emulate Internet functionality originally created by open source developers.)

But now, we're starting to see the shape of a very different future. Napster brought us peer-to-peer file sharing, Seti@home introduced millions of people to the idea of distributed computation, and now web services are starting to make even huge database-backed sites like Amazon or Google appear to act like components of an even larger system. Vendors such as IBM and HP bandy about terms like "computing on demand" and "pervasive computing".

The boundaries between cell phones, wirelessly connected laptops, and even consumer devices like the iPod or TiVO, are all blurring. Each now gets a large part of its value from software that resides elsewhere. Dave Stutz characterizes this as software above the level of a single device.

I like to say that we're entering the stage where we are going to treat the Internet as if it were a single virtual computer. To do that, we'll need to create an Internet operating system.

The large question before us is this: What kind of operating system is it going to be? The lesson of Microsoft is that if you leverage insight into a new paradigm, you will find the secret that will give you control over the industry, the "one ring to rule them all", so to speak. Contender after contender has set out to dethrone Microsoft and take that ring from them, only to fail. But the lesson of open source and the Internet is that we can build an operating system that is designed from the ground up as "small pieces loosely joined", with an architecture that makes it easy for anyone to participate in building the value of the system.

Tim added recently: “I'm talking about the emergence of what I've started to call Web 2.0, the internet as platform. We heard about that idea back in the late 90s, at the height of the browser wars, but that turned out to be a false alarm. But I believe we're now starting the third age of the internet -- the first being the telnet-era command line internet, the second the web -- and the third, well, that tale grows in the telling. It's about the way that open source and the open standards of the web are commoditizing many categories of infrastructure software, driving value instead to the data and business processes layered on top of (or within) that software; it's about the way that web sites like eBay, Amazon, and Google are becoming platforms with rich add-on developer communities; it's about the way that network effects and data, rather than software APIs, are the new tools of customer lock-in; it's about the way that to be successful, software today needs to work above the level of a single device; it's about the way that the Microsofts and Intels of tomorrow are once again going to blindside established players because all the rules of business are changing.”

If there is one candidate to build the Internet OS, it is Google.

Tomorrow: Google OS/PC/Browser

Related Entries:  [All]
TECH TALK: Computing for the Next Billion: Network Computers [June 22, 2006]
TECH TALK: The Network Computer: Making It Happen [October 15, 2004]
TECH TALK: The Network Computer: Business Model [October 14, 2004]
TECH TALK: The Network Computer: The Fifth Option [October 13, 2004]
TECH TALK: The Network Computer: The Four Devices [October 12, 2004]