Engines of Creation: The Coming Era of Nanotechnology

THE NETWORK OF KNOWLEDGE

(Chapter 14)

TO PREPARE for the assembler breakthrough, society must learn to learn faster. Fact forums will help, but new technologies may help even more. With them, we will be able to spread, refine, and combine our information far faster than ever before.

Information overload has become a well-known problem: pieces of knowledge pile up too fast for people to sort and make sense of them. Thousands of technical journals cover thousands of subjects. Published articles mount by over a million per year. Fact forums help us to clear away falsehoods, which will smooth our efforts to make sense of the world. But any such formal institution will be overwhelmed by the modern flood of information: fact forums will be able to deal with only a fraction - though an important fraction - of the facts, and they will inevitably be somewhat sluggish. Formal institutions can tap only a tiny fraction of the mental energies of our society.

Today, our information systems hamper our progress. To see the problem, imagine handling a piece of information: You uncovered it - how do you spread it? Someone else published it - how do you find it? You found it - where do you file it? You see an error - how do you correct it? Your file grows - how do you organize it?

We now handle information clumsily. Our traditional electronic media are vivid and entertaining, but they are ill suited to handling complex, long-term debates; how could you, as a viewer, file, organize, or correct the information in a television documentary? In short, how could you make it a well-integrated part of an evolving body of knowledge? We can handle complex debates better using paper media, yet the weeks (or years) of delay in a typical publication process slow debate to a crawl. And even paper publications are difficult to file, organize, and correct. Printers produce bundles of inked paper; through heroic efforts, librarians and scholars manage to link and organize them in a loose fashion. Yet indexes, references, and corrections simply add more pages or more editions, and tracing the links they represent remains tedious.

Books and other paper bundles work, after a fashion. They hold many of our cultural treasures, and we now have no better way to publish most things. Still, they leave great room for improvement.

Our trouble in spreading, correcting, and organizing information leaves our shared knowledge relatively scarce, incorrect, and disorganized. Because established knowledge is often hard to find, we often do without it, making us seem more ignorant than we need be. Can new technologies help us?

They have before. The invention of the printing press brought great advances; computer-based text services promise yet more. To see how our information systems could be better, though, it may help to see how they could be worse. Consider, then, an imaginary mess and an imaginary solution:

The Tale of the Temple

Once upon a time, there lived a people with an information problem. Though they had replaced their bulky clay tablets with paper, they used it oddly. In the heart of their land stood a stately dome. Beneath the dome lay their great Chamber of Writings. Within this chamber lay a broad mound of paper scraps, each the size of a child's hand.

From time to time, a scholar would journey to this temple of learning to offer knowledge. A council of scribes would judge its worth. If it proved worthy, they would inscribe it on a scrap of paper and ceremoniously fling it upon the heap.

From time to time, some industrious scholar would come to seek knowledge - to rummage through the heap in search of the needed scrap. Some, skilled in such research, could find a particular scrap in as little as a month. The scribes always welcomed researchers: they were so rare.

We moderns can see their problem: in a disorderly heap, each added scrap buries the rest (as on so many desks). Every scrap is separate, unrelated to the others, and adding references would provide little help when finding a scrap takes months. If we used such a heap to store information, our massive, detailed writings on science and technology would become almost useless. Searches would take years, or lifetimes.

We moderns have a simple solution: we place pages in order. We place page after page to make a book, book after book to fill a shelf, then fill a building with shelves to make a library. With pages in order, we can find them and follow references more rapidly. If the scribes employed scholars to stack scraps by subject, their research would grow easier.

Yet, when faced with stacks on history, geography, and medicine, where should the scholars put scraps on historical geography, geographical epidemiology, and medical history? Where should they put scraps on "The History of the Spread of the Great Plague"?

But in our imaginary land, the scribes choose another solution: they send for a magician. But first they turn scholars loose in the chamber with needles and thread to run strands from scrap to scrap. Thread of one color links a scrap to the next in a series, another color leads to a reference, another to a critical note, and so forth. The scholars weave a network of relationships, represented by a network of threads. At last, the magician (with flashing eyes and floating hair) chants a spell, and the whole mess heaves slowly into the air to float like a cloud in the dome. Ever after, a scholar holding a scrap need only tap a thread knotted to its edge to make the linked scrap leap to hand. And the threads, magically, never tangle.

Now the scholars can link scraps on "The History of the Spread of the Great Plague" to related scraps on history, geography, and medicine. They can add all the notes and texts they please, linking them to best advantage. They can add special index scraps, able to bring instantly to hand whatever they list. They can place links wherever they wish, weaving a network of knowledge to match the connections of the real world.

We, with our inert stacks of paper, could only envy them - if we didn't have computers.

Magic Paper Made Real

In 1945, Vannevar Bush proposed a system he called a "memex." It was to be a desk-sized device, crammed with microfilm and mechanisms, able to display stored pages and let the user note relationships among them. A microfilm memex was never built, but the dream lived on.

Today, computers and screens are becoming cheap enough to use for ordinary reading and writing. Some paper publishers have become electronic publishers, making magazines, newspapers, and journals available through computer networks. And with the right programs, text-handling computers will let us link this information in ways even better than magic thread.

Theodor Nelson, the originator of the idea, has dubbed the result "hypertext": text linked in many directions, not just in a one-dimensional sequence. Readers, authors, and editors using a hypertext system will generally ignore the workings of its computers and screens just as they have generally ignored the mechanics of photocomposition and offset lithography in the past. A hypertext system will simply act like magic paper; anyone who fiddles with it will soon become familiar with its basic abilities. Still, a description of one system's structure will help in explaining how hypertext will work.

In the approach followed by the Xanadu hypertext group (in San Jose, California), the core of the system - the back end - is a computer network able to store both documents and links between documents. An initial system might be a single-user desktop machine; eventually a growing network of machines will be able to serve as an electronic library. Stored documents will be able to represent almost anything, whether novels, diagrams, textbooks, or programs - eventually, even music or movies.

Users will be able to link any part of any document to any other. When a reader points to one end of a link (whether it is shown on the screen by underlining, an asterisk, or a picture of colored thread), the system will fetch and display the material at the other end. Further, it will record new versions of a large document without storing additional copies; it need store only the parts that are changed. This will let it inexpensively store the earlier versions of any document published and modified on the system. It will do all this rapidly, even when the total amount of information stored becomes immense. A network of such machines could eventually mature into a world electronic library.

To locate material in most computer-based text systems, the user must supply key words or obscure codes. Hypertext, too, will be able to link text to codes, to key words, or even to a simulated card catalog, but most readers will probably prefer just to read and point to links. As Theodor Nelson has remarked, hypertext will be "a new form of reading and writing, in a way just like the old, with quotations and marginalia and citations. Yet it will also be socially self-constructing into a vast new traversable framework, a new literature.

What the reader will see when browsing through this framework will depend partly on the reader's own part of the system, the "frontend" machine, perhaps a personal computer. The back end will just file and fetch documents; the front end will order them fetched at the reader's request and will display them to suit the reader's taste.

To imagine how this will appear to a user, picture a screen the size of this open book, covered with print the size you are reading now-clear print, on a good screen. Today the screen would resemble a television set, but within a few years it could be a booklike, lap-sized object with a cord to an information outlet. (With nanotechnology, we can eliminate the cord: a book-sized object will be able to hold a hypertext system containing images of every page in every hook in the world, stored in fast, molecular-tape memory.)

In this book - the one now in your hands - I could describe Theodor Nelson's books about hypertext, Literary Machines and Computer Lib, but you couldn't see them on these pages. Their pages are elsewhere, leaving you trapped for the moment in one author's writings. But if this were a hypertext system and I, or someone else, had added the obvious link, you could point to the words "Literary Machines" here, and a moment later the text on the opposite page would clear, to be replaced by Ted Nelson's table of contents or by my selection of his quotes. From there, you could step into his book and roam through it, perhaps while having your front-end system display any notes that I had linked to his text. You could then return here (perhaps now displaying his notes on my text) or move on to yet other documents linked to his. Without leaving your chair, you could survey all the major writings on hypertext, moving from link to link through any number of documents.

By keeping track of links (say, between outlines, drafts, and reference materials), hypertext will help people write and edit more ambitious works. Using hypertext links, we can weave our knowledge into coherent wholes. John Muir observed that "When we try to pick out anything by itself, we find it hitched to everything else in the universe." Hypertext will help us keep ideas hitched together in ways that better represent reality.

With hypertext, we will be better able to gather and organize knowledge, increasing our effective intelligence. But for information gathering to be effective, it must be decentralized; information scattered among many minds cannot easily be put into the system by a few specialists. The Xanadu group proposes a simple solution: let everyone write, and have the system automatically pay royalties to the authors whenever readers use their material. Publishing will be easy and people will be rewarded for providing what other people want.

Imagine what you yourself have wanted to say about ideas and events. Imagine the insightful comments that are even now fading from the memories of both speakers and listeners all over the world. On a hypertext system, comments will be easy to publish and easy to find. Imagine the questions that have bothered you. You could publish them, too; someone finding an answer could then publish a reply.

Since everyone on the system will be able to write text and links, the hypertext network will accumulate great stores of knowledge and wisdom and even greater heaps of utter garbage. Hypertext will include old news, advertising, graffiti, ranting, and lies - so how will a reader be able to avoid the bad and focus on the good? We could appoint a central editorial committee, but this would destroy the openness of the system. Sorting information is itself an information problem for which hypertext fortunately will help us evolve good solutions.

Since hypertext will be able to do almost everything that paper systems can, we can at least use the solutions we already have. Publishers have established reputations in the paper-text media, and many of them have begun to move into electronic publishing. On a hypertext system they will be able to publish on-line documents that meet their established standards. Readers so inclined will be able to set their front-end systems to display only these documents, automatically ignoring the new garbage. To them, the hypertext system will seem to contain only material by established publishers, but material made more available by electronic distribution and by hypertext links and indexes. True garbage will still be there (so long as its authors have paid a small storage fee for their material), yet garbage need not intrude on any reader's screen.

But we will be able to do better than this. Approval of a document (shown by links and recommendations) can come from anyone; readers will pay attention to material recommended by whomever they respect. Conversely, readers who find documents they like will be able to see who has recommended them; this will lead readers to discover people who share their interests and concerns. Indirectly, hypertext will link people and speed the growth of communities.

When publishing becomes so fast and easy, writers will produce more material. Since hypertext will encourage free-lance editing, editors will find themselves with more work to do. Documents that quote, list, and link other documents will serve as anthologies, journals, or instant-access indexes. The incentive of royalties will encourage people to help readers find what they want. Competing guides to the literature will swiftly appear - and guides to the guides.

Hypertext links will be better than paper references, and not merely in speed. Paper references let an industrious reader follow links from one document to another - but try finding which documents refer to one you are reading! Today, finding such references requires the cumbersome apparatus of a citation index, available only in research libraries, covering limited topics, and months out of date. Hypertext links will work in both directions, letting readers find comments on what they are reading. This means a breakthrough: it will subject ideas to more thorough criticism, making them evolve faster.

The evolution of knowledge - whether in philosophy, politics, science, or engineering - requires the generation, spread, and testing of memes. Hypertext will speed this process. Paper media handle the process of generation and spread fairly well, but they handle testing clumsily.

Once a bad idea reaches print, it takes on a life of its own, and even its author can seldom drive a stake through its heart. A devastating refutation of the bad idea becomes just another publication, another scrap of paper. Days or years later, readers who encounter the bogus idea will still be unlikely to have chanced upon its refutation. Thus, nonsense lives on and on. Only with the advent of hypertext will critics be able to plant their barbs firmly in the meat of their targets. Only with hypertext will authors be able to retract their errors, not by burning all the libraries or by mounting a massive publicity campaign, but by revising their text and labeling the old version "retracted." Authors will be able to eat their words quietly; this will give them some compensation for the fiercer criticism.

Critics will use clear refutations to skewer nonsense (such as false limits to growth), clearing it from the intellectual arena - though not from the record - almost as soon as it pops into sight. Guides to good criticism will help readers see whether an idea has survived the worst objections yet raised. Today, the absence of known criticism doesn't mean much, because brief critical comments are hard to publish and hard to find. In an established hypertext system, though, ideas that have survived all known criticism will have survived a real challenge. They will gain real and growing credibility.

Linking Our Knowledge

The advantages of hypertext run deep; this is why they will be great. Hypertext will let us represent knowledge in a more natural way. Human knowledge forms an unbroken web, and human problems sprawl across the fuzzy boundaries between fields. Neat rows of books do a poor job of representing the structure of our knowledge. Librarians have labored to make these rows more like nets by inventing better ways to index, reference, and arrange pieces of paper. Yet despite the noble efforts and victories of librarians, library research still daunts all but a dedicated minority of the reading public. Libraries have evolved toward hypertext, yet the mechanics of paper still hobbles them. Hypertext systems will let us take a giant step in a direction we have been moving since the invention of writing.

Our very memories work through associations, through links that make recollections recallable. AI workers also find associations essential to making knowledge useful; they program what they call "semantic nets" to build knowledge representation systems. On paper, associations among words make a thesaurus useful; in the mind, one's working vocabulary relies on fast, flexible associations among words. Indeed, relationships in memory supply the context that gives meaning to our ideas. Using hypertext, people will associate ideas through published links, enriching their meaning and making them more available - indeed, making them more like parts of our own minds.

When we change our minds about what the world is like and where it is going, we change our internal networks of knowledge. Reasoned change often requires that we compare competing patterns of ideas.

To judge a worldview presented in a book, a reader must often remember or reread explanations from earlier pages - or from a conflicting article seen last year. But human memory is faulty, and digging around in old paper often seems like too much work. Knowing of this problem, authors vacillate between putting too much in (thus boring their readers) and leaving too much out (thus leaving weak spots in their discussions). Inevitably, they do both at once.

Hypertext readers will be able to see whether linked sources support an idea or linked criticisms explode it. Authors will write pithy, exciting summaries of ideas and link them to the lengthy, boring explanations. As authors expound and critics argue, they will lay out their competing worldview networks in parallel, point by point. Readers still won't be able to judge ideas instantly or perfectly, but they will be able to judge them faster and better. In this way, hypertext will help us with a great task of our time: judging what lies ahead, and adjusting our thinking to prospects that shake the foundations of established worldviews. Hypertext will strengthen our foresight.

By now, many useful applications of a mature hypertext system will be obvious - or as obvious as they can be today, before we have experienced them directly. Carrying news is one such obvious application.

News shapes our view of the world, but modern media sharply limit what reporters can portray. Often, stories about technology and world events only make sense in a broader context, but the limited space and onrushing deadlines of publishing strip needed context from stories. This weakens our grasp of events. Using hypertext, reporters will find it easy to link today's news to broader background discussions. What is more, the people in the stories and casual observers will be able to have their say, linking their comments to the reporter's story.

Advertising greases the wheels of the economy, leading (and misleading) us to available products. Well-informed consumers can avoid shoddy, overpriced goods, but the needed research and comparison shopping gobbles time. On a hypertext system, though, consumer service companies will assemble comparative catalogs, linking descriptions of competing products to one another, to test results, and to reports from consumers.

In education, we learn best when we are interested in what we read. But most books present ideas in just one sequence, at just one level of difficulty, regardless of a learner's background or interests. Again, popular demand will favor the growth of useful networks in hypertext. People will make links between similar presentations written at different levels. Students will be able to read at a comfortable level, peeking at parallel discussions that reach a bit deeper. Hard material will grow easier to handle, because links to primers and basic definitions will let readers pause for review - instantly, privately, and without embarrassment. Other links will lead in all directions to related material; links in a description of a coral reef will lead to both texts on reef ecology and tales of hungry sharks. When we can gratify momentary interests almost instantly, learning will become more fun. More people might then find it addictive.

Due process will thrive in hypertext. Because it will be open to all sides, and will allow questioning, response, and so forth, hypertext debate will have an inherent due-process quality. Indeed, hypertext will be an ideal medium for conducting fact forums. Forum procedures, in turn, will complement hypertext by distilling its wide-ranging debates into a clear (though tentative) statement of key technical facts.

In a final obvious effect, hypertext will reduce the problem of out-of-context quotes: readers will be able to make the original context reappear around any quote in the system at the touch of a button. This will be valuable, and not just to prevent misrepresentation of an author's position; indirect benefits may matter more. Reasonable statements torn loose from their background can seem absurd, but hypertext authors will know that "absurd" quotations will lead readers directly back to the author's original context. This will encourage bolder writing, giving memes based on evidence and reason an advantage over those based on mere convention and timidity.

Perhaps the most important (yet least vivid) benefit of hypertext will be a new ability to see absences. To survive the coming years, we must evaluate complex ideas correctly, and this requires judging whether an argument is full of holes. But today we have trouble seeing holes.

Still more difficult is recognizing the absence of fatal holes, yet this is the key to recognizing a sound argument. Hypertext will help us. Readers will scrutinize important arguments, attaching conspicuous objections where they find holes. These objections will make holes so consistently visible that an absence of good objections will clearly indicate an absence of known holes. It may be hard to appreciate how important this will he: the human mind tends not to recognize the problems caused by our inability to see the absence of holes, to say nothing of the opportunities this inability makes us miss.

For example, imagine that you have an idea and are trying to decide whether it is sound and worth publishing. If the idea isn't obvious, you might doubt its truth and not publish it. But if it does seem obvious, you might well assume that it has already been published, but that you just can't find out where. Hypertext, by making things much easier to find, will make it easier to see that something has not been published. By making holes in our knowledge more visible, hypertext will encourage hole-filling.

To understand and guide technology, we need to find the errors-including omissions - in complex technological proposals. Because we do this poorly, we make many mistakes, and the visibility of these mistakes makes our incompetence a vivid and menacing fact. This encourages prudence, yet it can also encourage paralysis: because we have difficulty seeing holes, we fear them everywhere, even where they do not exist. Hypertext will help build confidence, where confidence is justified, by exposing problems more reliably.

Dangers of Hypertext

Like most useful tools, hypertext could be used to do harm. Though it will help us keep track of facts, it could also help governments keep track of us. Yet, on balance, it may serve liberty. Designed for decentralization - with many machines, many writers, many editors - hypertext may help citizens more than it helps those who would rule them. Government data banks are growing anyway. Hypertext systems might even help us keep an eye on them.

Relying on electronic publishing holds another danger. Governments in the United States and elsewhere have often interpreted the ideal once expressed as "freedom of speech" and "of the press" to mean only freedom to talk and to sell inked paper. Governments have regulated the use of radio and television, requiring them to serve a bureaucracy's shifting notion of the public interest. Practical limitations on the number of broadcasting channels once gave some excuse for this, but those excuses must stop here. We must extend the principles of free speech to new media.

We would be horrified if the government ordered agents into libraries to burn books. We should be equally horrified when the government seeks to erase public documents from electronic libraries. If hypertext is to carry our traditions, then what is published must remain so. An electronic library will be no less a library for its lack of shelves and paper. Erasure will make no flame and smoke, but the stench of book-burning will remain.

From Desktop to World Library

Some of the benefits I have described will only result from a large, highly evolved hypertext system - one already serving as a forum for broad debates and on its way to becoming a world electronic library. Such a system may not have time to mature before the assembler and AI breakthroughs arrive. For hypertext to get off the ground, small systems must have practical applications, and for hypertext to help us handle the technology race, small systems must have an effect beyond their size. Fortunately, we can expect substantial benefits almost from the outset.

Individual hypertext machines will be able to serve several users at once. Even without linking to anything in the outside world, they will help companies, associations, and research groups handle complex information.

Outside links will be easy, though. The number of publicly available data bases has grown from a few dozen in the mid-1960s to a few hundred in the mid-1970s to a few thousand in the mid-1980s. Companies have made these available through computer networks. Hypertext systems will be able to fetch material from these data bases, storing the access codes instead of the actual text. This information will only seem to be in the hypertext system, but that will be good enough for many purposes.

People will use early systems to provide a community with a dialup service like that of existing computer bulletin boards but better. Special-interest discussion groups have already emerged on computer networks; they will find hypertext a better medium for exchanging information and views.

Early hypertext systems will also help us build and run organizations. Ordinary computer conferencing (simply sending short messages back and forth) already helps groups communicate. Advantages over face-to-face conferences include lower costs (no need to travel), smoother interactions (no need to wait or to interrupt people), and a better meeting of minds (through clearer messages and fewer personality clashes). Hypertext communications will extend these benefits by giving participants better tools for referencing, comparing, and summarizing. Because hypertext debates will need no single editor, they will allow organizations to become more open.

Using a hypertext service by telephone from a home computer during off-peak hours will probably cost several dollars per hour at first. This cost will fall over time. For several decades now, the real cost of computers has dropped by about a factor of ten every ten years; the cost of communications has also declined. Hypertext systems will be affordable to a substantial number of people almost as soon as they become available. Within ten years, the costs seem likely to fall low enough for mass-market use.

Electronic publishing is already catching on. The Academic American Encyclopedia, structured as a simple hypertext, has been made available to 90,000 subscribers, including 200 libraries and eight schools. Time magazine reports that children use it eagerly. Terminals in libraries already can access the text of scores of newspapers, magazines, and professional journals.

We won't need to wait for a universal system to enjoy universal benefits, because hypertext will begin to make a difference very early. We need hypertext in the hands of students, writers, researchers, and managers for the same reason that we need textbooks in schools, instruments in laboratories, and tools in workshops. Some books have made a great difference, even when read by fewer persons than one in a thousand, because they have sent new ideas rippling across society. Hypertext will do likewise, helping to refine ideas that will then spread more widely through the established print and broadcast media.

Hypertext and Printing Press

How will the hypertext revolution compare to the Gutenberg revolution? Some numbers suggest the answer.

Printing with movable type cut the cost of books dramatically. In the fourteenth century, the King's Advocate of France had only seventy-six books, yet this was considered a large library. Books embodied weeks of skilled labor - copyists were literate. The peasant masses could neither afford books nor read them.

Today, a year's labor can pay for thousands of books. Many homes hold hundreds; large libraries hold millions. Printing cut the cost of books by a hundredfold or more, setting the stage for mass literacy, mass education, and the ongoing world revolution of technology and democracy.

And hypertext? Gutenberg showed Europe how to arrange metal type to print pages; hypertext will let us rearrange stored text and send it cross-country at the speed of light. Printing put stacks of books in the home and mountains of books in libraries; hypertext will in effect bring these mountains of books to every terminal. Hypertext will extend the Gutenberg revolution by increasing the quantity of information available.

Yet its other advantages seem greater. Today, following a reference in a library typically takes minutes; with luck, a few hundred seconds - but it can take days or more, if the material is unpopular and hence absent, or too popular and hence missing. Hypertext will cut this delay from hundreds of seconds to about one second. Thus where the Gutenberg revolution reduced the labor cost of producing text by several hundredfold, the hypertext revolution will reduce the labor cost of finding text by several hundredfold. This will be a revolution indeed.

As I have discussed, making links more convenient will change the texture of text, bringing a revolution not merely in quantity but in quality. This increase in quality will take many forms. Better indexes will make information easy to find. Better critical discussion will weed out nonsense and help sound ideas thrive. Better presentation of wholes will highlight the holes in our knowledge.

With abundant, available, high-quality information, we will seem more intelligent. And this will increase our chances of handling the coming breakthroughs right. What could be more useful? Next time you see a lie being spread or a bad decision being made out of sheer ignorance, pause, and think of hypertext.