Bus. Std: Constructing the Memex – Part 2

My latest column from Business Standard:

[Part 1 of the "Constructing the Memex" series]

In the past, the information available to us was limited by what we could remember, file on our own, ask others in a close circle, or access in a library. The Internet changed it all by making available a vast library of inter-linked digital information. Finding the information of value to us when we need it is still a challenge. In the beginning, we had Yahoo’s editorial selection of the best sites on the Web. Then, its editors updated site listings via incoming requests for additions as the web expanded. Next came the first generation of search engines which moved beyond the directory approach and crawled millions of pages, indexing the text on these pages.

Google, representing the second generation of search engines, used the information about links between pages as part of its PageRank technology, to come out with a superior way of searching. And yet, the problem of finding the right information in billions of web pages remains. Whether it is information in our personal or work life, there seems to be no end to the massive amount of information that we have to sift through. It is time to look at a new approach.

Imagine if we could bridge directories and search engines, making them much more customised based on our likes and the trails that we leave as we surf the Internet, and also taking into account all that we write in emails, blogs and elsewhere. This system would use our memory and knowledge as the starting point. We would start by outlining our interest areas - the topics that form the knowledge-sphere of our lives. This is akin to the directory of topics – only much more personally relevant to us. For example, in my case the main categories of this list would be something like this: Affordable Computing, ICT for Development, Emerging Markets, Enterprise Software, Information Management, Small and Medium Enterprises, New Technologies and India.

Each topic needs to be drilled down further. What is needed is a hierarchy of topics, which helps in further defining our interests in greater detail. For example, my outline for Affordable Computing could look like this: Hardware (Thin Clients, Refurbished PCs, Set-Top Boxes, PDAs), Software (Terminal Services, Linux, Applications, Language Computing), and Communications (Ethernet, Wireless, Satellite, Broadband).

This hierarchy of topics serves as the basis for our interests. It gives a unique lens and context to the information that we browse on the Web, write in emails and receive as attachments. These topics will evolve as our interests change and as we come across experts who may have done a better job in building out a certain part of the information ecosystem.

This is an evolving information base – built not by a centralised organization, but in a distributed manner by each of us. Each of us would have a microcosm of the information space, created and updated continuously by what we did. It would ensure that our ideas would have a context, that we would never forget something, and that we could leverage on similar work done by millions of others like us. This is the real two-way web – linking not just documents, but people, ideas and information.

In the previous column, we discussed Vannevar Bush’s vision of a memex (memory extender), which envisioned just such a machine that would allow us to deal with information overload by leveraging aspects of the human mind and its capability to deal with associations. The platform exists today to construct the memex – built around millions of personal directories and welogs.

The weblog becomes a personal journal which allows us to store links and abstracts from articles we find of interest on the web, along with our own ruminations. A personal directory provides a table of contents, a glimpse into the way our mind catalogues information. Two existing XML-based standards would help share directories and syndicate the blog posts that we create – OPML (Outline Processor Markup Language) and RSS (Rich Site Summary).

In a sense, the memex is more about assimilation than aggregation, more value-added integration than scanning silos, more amplification than just presentation. The building blocks already exist in the form of the millions of blogs created by people like us. What is missing is the outline, a map of each of our minds. Once we start creating this in a distributed manner, we will see the emergent effects of a whole which is much bigger than the sum of the parts.

In the past few years, various technologies like weblogs, RSS, OPML, web services, Google’s API, visualisation tools and social software, combined with a deeper understanding of the science of networks have laid the foundation for leveraging individual intellect to put together a information framework that is far beyond the capacity of any person or organisation. The time for constructing the Memex has now come.

Lasica's Darknet Book Wiki

JD Lasica has put up a Wiki of his forthcoming book "Darknet: Remixing the Future of Movies, Music and Television." Its an interesting experiment - anyone can edit it! From the introduction:

Darknet tells the story of the digital media revolution. The future of movies, music, television, and the Internet are all on the line in this clash between the irresistible force of technological innovation and the immovable object of entertainment media.

I wrote this book for two reasons: to tell the stories of the strong personalities and colorful characters on both sides of the digital culture war, and to spotlight the ways in which entertainment companies are locking down the digital technologies we want to use.

The stories in this book speak to how technology is shifting the balance of power between big media and regular people. The rise of “personal media” is throwing the old rules into disarray. Media companies are losing their traditional control over audiences as the Internet propels a cultural shift from push to pull, from passive consumer to empowered user.

We no longer consider ourselves couch potatoes who absorb whatever mass media may funnel our way. We produce, publish, reinvent, and share personal media. We make our own movies. We create digital photos, animation, niche news sites, hyper-fiction, online picture albums. We program our personal video recorders so that we watch programming not on the networks’ schedule but on our terms. We capture TV shows and stream them from one room to another on our home networks. We record music with cheap tools that not long ago cost more than a house, and perhaps distribute our works on the Internet. We listen to satellite or Web radio stations that cater to our individual tastes rather than homogenized commercial radio. And we download music from the Net to our MP3 players and burn music to our own CDs.

We make our own media. We are our own media.

But traditional culture fights back. Under the banner of fighting piracy and protecting copyright, influential companies are threatening to turn back the clock. Our personal devices are becoming handcuffed, our televisions dumbed-down, and our computers hamstrung. This is not some far-off threat; it is happening today.

This book will show that the digital revolution encompasses much more than piracy or file sharing. I hope to enlarge the debate by bringing into focus some of the important new developments in digital culture: personal media, participatory culture, space shifting, edge TV, open media, digital rights, and darknets.

Now, about the title. Darknets refer to underground or private networks where people trade files and communicate anonymously. But there's a deeper meaning as well. Darknet serves as a warning about a world where digital media become locked down, a future where the network serves not the user but the interests of Hollywood and the music industry. The Darknet is where many of us may wind up if current trends continue.

The next few years will be pivotal in the war on creative expression. As Joe Kraus of the public interest group DigitalConsumer.org warns, “This battle will affect consumers’ rights for the next fifty years.”

Japan Benefits from China

WSJ writes:

Just as U.S. workers are now worrying that their jobs will be outsourced to China and India, many Japanese have long feared their industries would "hollow out" and move to China. For much of the 1990s, their fears were realized as China sucked manufacturing jobs out of Japan. But recently, China's boom is contributing to Japan's economic recovery and actually creating manufacturing work here.

Chinese demand is also combining with years of cost-cutting to allow Japanese companies to keep more manufacturing at home. Big manufacturers such as Hitachi Construction and consumer-electronics giant Matsushita Electric Industrial Co. have boosted efficiency and can now make highly complex goods as cheaply in Japan as they can overseas.

While the U.S. trade deficit with China surged 20% to $124 billion in 2003, Japan's deficit with China shrank 24% to the equivalent of $19 billion. This February, Japan ran a monthly trade surplus with China for the first time in 10 years. Exports of Japanese machinery and electronic gadgets -- exactly what China was supposedly grabbing from Japan -- are among the biggest drivers of Japan's recent expansion. Gross domestic product in Japan, the world's second-largest national economy after the U.S., grew by a surprise 2.7% in 2003, and economists expect similar growth this year.

With growth averaging 7% a year for the past 10 years, China is becoming a powerful consumer on the global stage. It is also becoming a giant builder, especially as the 2008 Beijing Olympics approach. That generates Japanese exports to China of chemicals, cement and steel girders.

Even more important is the increasing production by Japanese companies in China, as they set up factories there to make everything from cellphones to clothes irons. Of the goods Japanese companies make in China, some 40% stay within China itself, says Koji Hiiragi, an economist at UFJ Institute, a Tokyo think tank. The equipment and components used in these China-based factories mostly come from Japan, creating -- for the time being -- a net gain in production inside Japan. Indeed, the main worry for Japanese economists these days is that a slowdown in China -- which accounted for 79% of Japanese export growth last year -- will threaten Japan's recovery.

Like most advanced economies, Japan over the long term is gaining service jobs and losing those in manufacturing.

Red Hat Desktop

News.com writes about Red Hat's launch of a desktop Linux product aimed at governments, academic institutions and enterprises, especially in Europe and Asia.

Red Hat initially won't tackle the entire desktop software market, aiming instead for corporations whose employees need only basic computing features such as word processing and Web access. As with its existing server products, Red Hat will sell the desktop version as an annual subscription that includes support and software updates through the Red Hat Network. But it won't sell them individually, instead offering 50-computer subscriptions for $3,500 annually--about $70 per PC per year.

Price comparisons are awkward, given Red Hat's subscription model, but a copy of Microsoft Windows XP Professional retails for $299. Red Hat's top Linux competitor, Novell, sells one desktop product for $90, and a five-pack of a more business-oriented edition for $598. Perhaps closest to Red Hat's approach is that of Sun Microsystems, whose SuSE Linux-based Java Desktop System costs $100 per employee per year (though discounted to $50 through June 2).

"I would say that in 5 to 7 years, the strength of revenue with this relationship with the customer will be on the scale of the server business," Red Hat CEO Matthew Szulik said--but he cautioned that he believes that world also will arrive when applications run on a tightly coupled combination of desktops and servers.

Szulik said the next desktop operating system transition at Microsoft will be an opportunity to convince customers to make the change to Linux. "A lot of customers are faced with heartburn (from) thinking of that transition from one Microsoft environment to Longhorn, whenever it does make its debut," Szulik said.

Liberation and Development

Atanu Dey writes:

Power, credit, and knowledge are the basic ingredients for the recipe that liberates. The utility of liberation is expressed in the Zeroth Law of Development which is that liberation is a pre-condition for development. Without freedom of thought and action, nothing of value can be accomplished. At its core, development is about freedom -- economic, political, religious, ... ad infinitum. Casual empiricism bears out that law: where these freedoms are missing, development is absent. If you really insist on it, check out the human development indicies of countries and you would notice that countries that are in economic, political, and religious shackles are not developed.

I would like to now outline my argument here:

1. Energy, credit, and knowledge are the basic ingredients for liberation.
2. Liberation is a precondition for development.
3. So if one wishes to bring about development, one has to assure the availability of energy, credit, and knowledge.

I will argue that underdeveloped countries have to struggle so hard to become developed because they are deficient in some or all of the three essential ingredients of liberation. I will further argue that it is possible to bootstrap the process of development but only if resources are used efficiently and if problems are solved by addressing causes rather than by alleviating superficial effects. Finally, I would address the question of the use of information and communications technologies for development. The point that I would discuss is that knowledge is the active agent of transformation. ICT, as the name implies, is technology that is concerned with information, and not knowledge directly. Not keeping the distinction between knowledge and information leads to confused thinking and ultimately immense waste of resources.

Home PCs

Could this be the year the digital home starts to become a reality? News.com writes:

Entertainment PCs will be designed to emulate a home stereo or video component, such as a VCR, and will serve to handle consumer's audio and video needs by playing music and DVDs, recording TV programs and even showing picture slide shows on a TV. Most entertainment PCs will be purchased without monitors and will be operated almost exclusively via a remote control.

Lifestyle PCs, designed to inhabit bedrooms and kitchens, will look more like the familiar desktop. The high-end machines will be more stylish, however, and will also be operable via remote control, allowing consumers to use them to write an e-mail and then later play a video or some music. Most lifestyle PCs will also come with multimedia software such as Microsoft's Windows XP Media Center operating system, according to Intel.

From another News.com report: "Microsoft and Hewlett-Packard have come up with a concept PC that builds on the company's current Media Center entertainment idea and goes a few steps further. The Media Center PC of the future has a remote control with a built-in LCD screen for programming recordings. The PC, known as the Windows Home Concept, also supports Internet telephony, dual high-definition TV tuners, biometric security, and a built-in cable modem."

TECH TALK: The Company: Amar Bhide

The Origin and Evolution of New Businesses by Amar Bhide is a look at how entrepreneurs start and grow their businesses. From the book’s description:

What is this mysterious activity we call entrepreneurship? Does success require special traits and skills or just luck? Can large companies follow their example? What role does venture capital play?

In a field dominated by anecdote and folklore, this landmark study integrates more than ten years of intensive research and modern theories of business and economics. The result is a comprehensive framework for understanding entrepreneurship that provides new and penetrating insights. Examining hundreds of successful ventures, the author finds that the typical business has humble, improvised origins. Well-planned start-ups, backed by substantial venture capital, are exceptional. Entrepreneurs like Bill Gates and Sam Walton initially pursue small, uncertain opportunities, without much capital, market research, or breakthrough technologies. Coping with ambiguity and surprises, face-to-face selling, and making do with second-tier employees is more important than foresight, deal-making, or recruiting top-notch teams. Transforming improvised start-ups into noteworthy enterprises requires a radical shift, from "opportunistic adaptation" in niche markets to the pursuit of ambitious strategies. This requires traits such as ambition and risk-taking that are initially unimportant. Mature corporations have to pursue entrepreneurial activity in a much more disciplined way. Companies like Intel and Merck focus their resources on large-scale initiatives that scrappy entrepreneurs cannot undertake. Their success requires carefully chosen bets, meticulous planning, and the smooth coordination of many employees rather than the talents of a driven few.

Amar Bhide looks at the entrepreneur-driven Company and its early stage:

Most promising businesses start-out with meager funds – only a very select subset of high potential start-ups can raise funds from professional intermediaries such as venture capitalists. Most founders of promising businesses cannot raise much outside capital because they don’t have much verifiable human capital or proprietary technologies – they tend to have limited experience and often start their businesses by copying or slightly modifying someone else’s idea.

Promising start-ups have low “most likely” profit potential; but because of the nature of the opportunities they pursue, they have at least a chance of earning significant returns. Promising start-ups cluster in market niches characterized by high uncertainty generated by technological, regulatory or other such exogenous changes or by the amorphous nature of customer wants. High uncertainty and low capital and opportunity costs create a “heads I win, tails I don’t lose much” proposition for entrepreneurs.

The founders of promising ventures find their business ideas in the course of their previous employment or by chance, rather than through a systematic search. They devote little effort to prior market research or planning. Entrepreneurs rely on opportunistic adaptation to unexpected problems and opportunities.

To add value to their free option, entrepreneurs have to get customers and other “resource providers” to take a chance on their business. The lack of a track record and capital makes resource providers reluctant to do business with a start-up. Entrepreneurs overcome this reluctance by providing special benefits, exploiting others’ cognitive biases and reflexive tendencies, and by locating resource providers with unusual needs or willing to bear risk.

Unforeseen events play a significant role in promising start-ups because of the high uncertainty and lack of planning. Adapting to unforeseen circumstances requires an unusual tolerance for ambiguity.

Amar Bhide’s book provides a detailed look into the minds of entrepreneurs. The Company that every entrepreneur seeks to build is the One to change the world. And yet few survive over the long-term. It is this optimism and belief on which our world is built.

Tomorrow: Arie de Geus

Related Entries:  [All]