The Coming Technological Singularity | Page 7

From one angle, the vision fits many of our happiest dreams: a time
unending, where we can truly know one another and understand the
deepest mysteries. From another angle, it's a lot like the worst- case
scenario I imagined earlier in this paper.
Which is the valid viewpoint? In fact, I think the new era is simply too
different to fit into the classical frame of good and evil. That frame is
based on the idea of isolated, immutable minds connected by tenuous,
low-bandwith links. But the post-Singularity world does fit with the
larger tradition of change and cooperation that started long ago
(perhaps even before the rise of biological life). I think there are
notions of ethics that would apply in such an era. Research into IA and
high-bandwidth communications should improve this understanding. I

see just the glimmerings of this now [32]. There is Good's Meta-Golden
Rule; perhaps there are rules for distinguishing self from others on the
basis of bandwidth of connection. And while mind and self will be
vastly more labile than in the past, much of what we value (knowledge,
memory, thought) need never be lost. I think Freeman Dyson has it
right when he says [9]: "God is what mind becomes when it has passed
beyond the scale of our comprehension."
[I wish to thank John Carroll of San Diego State University and
Howard Davidson of Sun Microsystems for discussing the draft version
of this paper with me.]
Annotated Sources [and an occasional plea for bibliographical help]
[1] Alfve'n, Hannes, writing as Olof Johanneson, The End of Man?,
Award Books, 1969 earlier published as "The Tale of the Big
Computer", Coward-McCann, translated from a book copyright 1966
Albert Bonniers Forlag AB with English translation copyright 1966 by
Victor Gollanz, Ltd.
[2] Anderson, Poul, "Kings Who Die", If, March 1962, p8-36.
Reprinted in Seven Conquests, Poul Anderson, MacMillan Co., 1969.
[3] Asimov, Isaac, "Runaround", Astounding Science Fiction, March
1942, p94. Reprinted in Robot Visions, Isaac Asimov, ROC, 1990.
Asimov describes the development of his robotics stories in this book.
[4] Barrow, John D. and Frank J. Tipler, _The Anthropic Cosmological
Principle_, Oxford University Press, 1986.
[5] Bear, Greg, "Blood Music", Analog Science Fiction-Science Fact,
June, 1983. Expanded into the novel Blood Music, Morrow, 1985.
[6] Cairns-Smith, A. G., Seven Clues to the Origin of Life, Cambridge
University Press, 1985.
[7] Conrad, Michael et al., "Towards an Artificial Brain", BioSystems,
vol 23, pp175-218, 1989.

[8] Drexler, K. Eric, Engines of Creation, Anchor Press/Doubleday,
1986.
[9] Dyson, Freeman, Infinite in All Directions, Harper && Row, 1988.
[10] Dyson, Freeman, "Physics and Biology in an Open Universe",
_Review of Modern Physics_, vol 51, pp447-460, 1979.
[11] Good, I. J., "Speculations Concerning the First Ultraintelligent
Machine", in Advances in Computers, vol 6, Franz L. Alt and Morris
Rubinoff, eds, pp31-88, 1965, Academic Press.
[12] Good, I. J., [Help! I can't find the source of Good's Meta-Golden
Rule, though I have the clear recollection of hearing about it sometime
in the 1960s. Through the help of the net, I have found pointers to a
number of related items. G. Harry Stine and Andrew Haley have
written about metalaw as it might relate to extraterrestrials: G. Harry
Stine, "How to Get along with Extraterrestrials ... or Your Neighbor",
_Analog Science Fact- Science Fiction_, February, 1980, p39-47.]
[13] Herbert, Frank, Dune, Berkley Books, 1985. However, this novel
was serialized in Analog Science Fiction-Science Fact in the 1960s.
[14] Kovacs, G. T. A. et al., "Regeneration Microelectrode Array for
Peripheral Nerve Recording and Stimulation", _IEEE Transactions on
Biomedical Engineering_, v 39, n 9, pp 893-902.
[15] Margulis, Lynn and Dorion Sagan, _Microcosmos, Four Billion
Years of Evolution from Our Microbial Ancestors_, Summit Books,
1986.
[16] Minsky, Marvin, Society of Mind, Simon and Schuster, 1985.
[17] Moravec, Hans, Mind Children, Harvard University Press, 1988.
[18] Niven, Larry, "The Ethics of Madness", If, April 1967, pp82-108.
Reprinted in Neutron Star, Larry Niven, Ballantine Books, 1968.
[19] Penrose, Roger, The Emperor's New Mind, Oxford University

Press, 1989.
[20] Platt, Charles, Private Communication.
[21] Rasmussen, S. et al., "Computational Connectionism within
Neurons: a Model of Cytoskeletal Automata Subserving Neural
Networks", in Emergent Computation, Stephanie Forrest, ed.,
pp428-449, MIT Press, 1991.
[22] Searle, John R., "Minds, Brains, and Programs", in _The
Behavioral and Brain Sciences_, vol 3, Cambridge University Press,
1980. The essay is reprinted in The Mind's I, edited by Douglas R.
Hofstadter and Daniel C. Dennett, Basic Books, 1981 (my source for
this reference). This reprinting contains an excellent critique of the
Searle essay.
[23] Sims, Karl, "Interactive Evolution of Dynamical Systems",
Thinking Machines Corporation, Technical Report Series (published in
_Toward a Practice of Autonomous Systems: Proceedings of the First
European Conference on Artificial Life_, Paris, MIT Press, December
1991.
[24] Stapledon, Olaf, The Starmaker, Berkley Books, 1961 (but