Scientific discovery and topological transitions in collaboration networks

We analyze the advent and development of eight scientific fields from their inception to maturity and map the evolution of their networks of collaboration over time, measured in terms of co-authorship of scientific papers. We show that as a field develops it undergoes a topological transition in its collaboration structure between a small disconnected graph to a much larger network where a giant connected component of collaboration appears. As a result, the number of edges and nodes in the largest component undergoes a transition between a small fraction of the total to a majority of all occurrences. These results relate to many qualitative observations of the evolution of technology and discussions of the "structure of scientific revolutions". We analyze this qualitative change in network topology in terms of several quantitative graph theoretical measures, such as density, diameter, and relative size of the network's largest component. To analyze examples of scientific discovery we built databases of scientific publications based on keyword and citation searches, for eight fields, spanning experimental and theoretical science, across areas as diverse as physics, biomedical sciences, and materials science. Each of the databases was vetted by field experts and is the result of a bibliometric search constructed to maximize coverage, while minimizing the occurrence of spurious records. In this way we built databases of publications and authors for superstring theory, cosmic strings and other topological defects, cosmological inflation, carbon nanotubes, quantum computing and computation, prions and scrapie, and H5N1 influenza. We also built a database for a classical example of "pathological" science, namely cold fusion. All these fields also vary in size and in their temporal patterns of development, with some showing explosive growth from an original identifiable discovery (e.g. carbon nanotubes) while others are characterized by a slow process of development (e.g. quantum computers and computation). We show that regardless of the detailed nature of their developmental paths, the process of scientific discovery and the rearrangement of the collaboration structure of emergent fields is characterized by a number of universal features, suggesting that the process of discovery and initial formation of a scientific field, characterized by the moments of discovery, invention and subsequent transition into "normal science" may be understood in general terms, as a process of cognitive and social unification out of many initially separate efforts. Pathological fields, seemingly, never undergo this transition, despite hundreds of publications and the involvement of many authors. (C) 2009 Elsevier Ltd. All rights reserved.