Digital Libraries: State of the Art and Future Directions

 

David Robins

University of Pittsburgh

School of Information Science

Dept. of Library and Information Science

135 North Bellefield Avenue

644 Information Sciences Building

Pittsburgh, Pennsylvania 15260

Phone: 412.624.9441

Fax: 412.648.7001

Email: drobins@pitt.edu

http://robins.sis.pitt.edu/

Ken Sochats

University of Pittsburgh

School of Information Science

Visual Information Systems Center

135 North Bellefield Avenue

707 Information Sciences Building

Pittsburgh, Pennsylvania 15260

Phone: 412.624.9416

Fax: 412.624.3006

Email: sochats@sis.pitt.edu

http://ltl13.exp.sis.pitt.edu/

 

Abstract

 

            The aim of this paper is to characterize digital libraries as they are currently conceived, and to suggest future opportunities for digital libraries. We offer a definition of digital libraries and a comparison of digital and traditional libraries, and we then show examples of digital libraries that will demonstrate different uses for digital libraries. In the final section, we will discuss the opportunities for digital library development in the coming years. These sections are summarized below.

What are Digital Libraries?

            A digital library is a new type of library: one that may not necessarily obliterate traditional library activities, but one that certainly creates new services for its clientele. Particularly, digital libraries have the potential, at the very minimum, to:

·        Provide an environment in which a community of practice may discover information through means such as knowledge management, data mining, and intelligent agents

·        Support dynamic collections, dynamic documents, and collaboration (that is, to allow and foster contribution of materials by community members)

·        Support learning by individuals and groups

·        Demonstrate an architecture that supports appropriate organization of resources of various types of information in various locations.

·        Demonstrate an architecture that supports effective use of the resource.

            There remains little question that the dominant term referring to organized, accessible collections of information in digital format is “digital library.” The term “digital library” was not in common use before 1993, when the first Digital Library Initiative was announced by the National Science Foundation (NSF). It does, however, appear in the literature as early as 1988 in a document by the Corporation for National Research Initiatives (cited in Lyons, 1989). Fox (1993) provides a document summarizing possible agendas for digital library research, and this document served as a foundation for NSF initiatives directed at providing funding for such efforts. Griffin (1998) conceives digital libraries to be information in digital formats.

            There are broad and narrow conceptions of what digital libraries are. Broad conceptions of digital libraries are based on assumptions that a digital library is part of a system, in which the digital library is a subsystem. The system might be a University, a business, or any other community (to be defined in the next section) to which the development/acquisition/selection, storage/preservation, and exchange, etc., of information is necessary or desirable. The implication of this broader view of digital libraries is that, in order to make the most effective use of a digital library, its design and maintenance is subordinate to its mission within the larger system. For example, if one were creating a digital library to house objects of cultural heritage from a particular geographic region, a broad view of this library’s creation would necessitate data gathering regarding potential users of, and contributors to, its collection. It may require the assembly of a team of experts, not unlike those employed in traditional archives, who are able to analyze and classify objects in the collection. Historians, sociologists, etc. may be called upon to provide a context for objects in the collection. A framework for handling legal and political issues such as copyright may be considered. Finally, the contributors of the collection may be interviewed in order to collect first-hand information about objects—information that will be lost when the contributor dies. This type of holistic approach to digital libraries may improve their design and usability.

            At the other extreme of our continuum, the narrow approach to digital library design and maintenance is one that focuses more closely on the applications associated with the storage and retrieval of objects in the collection. This approach is not to be discounted. Technical problems in the storage and access of digital objects are immense, and not likely to go away. Problems such as narrow bandwidth, and hence, compression, remain. Algorithms to process natural language queries continue to be refined, although the ambiguity of human language, coupled with the rigidity of digital information systems, remains a major obstacle. Therefore, the somewhat narrower, more technical, focus on digital library research remains fertile.

            However, collaboration between broad and narrow research foci should yield productive research on digital libraries. This notion of collaboration among various types of researchers created an impetus for funding from various Federal agencies, most notably the National Science Foundation (NSF). The NSF has funded two phases of Digital Library Initiatives (DLI-1 and DLI-2) during the 1990’s, both with the goal of a broader approach to digital library research.

            The Digital library initiatives sponsored by the NSF represent two objectives: (1) to foster research and development among various research disciplines; and (2) to act as a catalyst in understanding and defining the nature of digital libraries. Research and development on the technical and social issues involved with digital library creation and maintenance characterize the first objective. These issues consist of, but are not limited to, test bed creation, engineering of information search and information storage capabilities, and user studies. The second objective, which may appear to be of secondary importance to the first, is a necessary step in any new field of research: definition. That is, it may be necessary to understand the “who, what, when, where, and why” of what we are doing in order to evaluate our progress toward doing it. This second point is critical in understanding why we currently use the term “digital libraries” to describe research which seeks to describe and build distributed information systems.

            Now that we have explored the notion of digital libraries, we may ask what digital libraries have in common traditional, or physical, libraries.

How Digital Libraries Compare to “Traditional” Libraries

            Generally, we can say that libraries, traditional or digital, exist to serve a more or less well-defined community. Libraries select, organize, store, disseminate, and preserve information within the physical limitations of an institution. Before the convergence of computing and telecommunications technologies, the physical location of a community was extremely important to a library’s ability to provide service in a timely and efficient way. It was necessary for libraries to select, organize, store, disseminate, and preserve information within the physical limitations of the institution. The whole notion of Digital Libraries is based on the fact that information no longer must be physically located where library users are in order for it to be useful. One central theme of this paper, therefore, is the impact of information technology on the distribution and use of information. Examples of digital libraries can help illustrate how people are using these institutions, and they are provided in the full paper.

            For our purposes, a library is an entity in which information containers are selected, acquired, organized, disseminated, and preserved. (Information containers are traditional formats in which information is stored, e.g., magazines, journals, monographs, indexes, reference materials, etc.). In order to accomplish these tasks, some administrative infrastructure, however complex, must control the workings of the entity.

            This definition is intentionally broad so that the various purposes for which libraries exist will not be excluded. For example, if we include the educational function that is sometimes associated with public libraries, we may exclude certain corporate libraries whose mission does not include such a function.

            At a more fundamental level, our definition may not be universally accepted. Crawford and Gorman (1995), argue that information is not the medium of exchange offered by libraries, but rather, it is knowledge. They define information as data that have been “processed and rendered useful,” and knowledge as “information transformed into meaning.” However, it is not clear from their analysis how libraries have actually transformed information into meaning. That is, in what ways have libraries transformed the information found in collected objects into meaning? Meaning in terms of what? Have librarians spent time interpreting and relating information? Only to a small degree, and on a very general level, have librarians attempted to cross-reference materials in their catalogs. Rather, it is a particular user or a community with similar interests that constructs meaning from what others might call disjointed information.

            Our definition of a traditional library is more akin to what Miksa (1996) calls the “modern library.” Miksa conceives libraries to be products of eras, that is, they are products of cultural/social milieu that determine the mission, infrastructure, etc. of a library. The construct, “library,” is not a thing that looks and feels the same regardless of the era in which it is manifested. Rather, “libraries” are, of necessity, dynamic entities that serve the needs of specific communities, at specific times and specific places. Therefore, the modern library is the product of the era beginning in the late half of the 19th century and continuing to the present time. This era of libraries may be characterized by attributes including:

·        Generalized classification schemes such as the Dewey Decimal System

·        Systematic means of collection development and cataloging

·        Formalized education for, and the emergence of, the “library professional”

·        Diffusion of the concept of a “public” library (that is, a library supported by, and intended for use by, the local population

            This list of characteristics is by no means complete, but it represents unique aspects of library culture that were not present prior to the late 19th century. These characteristics, in short, represent a major shift in services from the types of libraries that existed before (mainly private collections with idiosyncratic collection and organization methods). What caused this shift? Miksa (1996) contends that the period beginning in the 1850’s until the present time represents the culmination of development of the printing industry. Essentially, he maintains that the “modern library” is one solution to the problems of high production and distribution costs that makes diffusion of printed information costly. Libraries, therefore, became places of a “public” nature rather than private collections as before. In other words, after four hundred years during which a technology for the mass production of information existed, an efficient means of distribution was finally found.

            It is possible to list other causes for the emergence of the “traditional” or “modern” library. For example, the industrial revolution, beginning as early as the mid 18th century in England, and in the late 19th century in the United States, produced the need for an educated middle class. Public libraries served such a need by providing access to materials supporting such learning. Moreover, library work modeled itself after the management and labor practices shown to be effective in the management of other goods and services of the time. Specifically, the specialization of workers doing routine jobs such as cataloging, reference, circulation, and management streamlined the processes of library operations, thereby further reducing costs associated with the “mass delivery” of information (Ward and Robins, 1998). Therefore, a combination of technological and economic innovations during the late 19th century triggered what libraries have been for most of the 20th century. In short, centralized points of information delivery (public libraries), and the routinization of work (mass production technology) made a “public” orientation of information delivery a technological and economic possibility.

            Figure 1 shows the internal workings of traditional libraries. Inputs are seen as materials such as printed books and serials purchased on contract from vendors known as "jobbers." Trained librarians perform management, and clerical tasks associated with library work are to be performed by labor. The processing of materials is, for libraries, the defining segment of the model. Standard descriptions of materials, as found in shelf lists, card catalogs, and electronic catalogs accomplish material throughput. Labor, whose jobs it is to perform clerical tasks such as filing, trained professionals who report to funding agencies manage typing, and data entry. Service to an information community is accomplished when a library hires professionals with subject expertise matching that of a targeted population. The professionals are responsible for constant improvement of the system by way of feedback. The system described here is a closed system. A closed system is one that reacts only in small ways as a result of interaction with its environment. That is, such a system is not as responsive to environmental input as it is from internal input. Such a system engages mainly in activities that it has set for its own maintenance and survival.

            In order to benefit from the services of this traditional library, patrons must visit the physical plant. Figure 1 shows that a given library patron may be part of an information community, but must approach the library as one whose request is singular. The fact that most libraries cannot deliver information before a user approaches it with a request is due to many factors. First of all, there is typically not enough professional staff to deliver that sort of personalized service. Second, the amount of information available is too great for accurate matching with ongoing user needs. Therefore, the responsibility is on the user, in most cases, to seek out needed information. In Figure 1, a user who is an information community member approaches the library. The library is a going concern with its own classification system, and service points. By necessity, a typical library cannot customize its services. In order to provide maximum service to the most users, it must minimize personalized service. Our user may seek information through less formal means, such as communication with other information community members, but to approach the library is go to a different world.

Figure 1. Traditional libraries as more or less closed systems.

 

 

            But in the late 20th century, we have another fundamental technological innovation creating the necessity for rethinking our modes of information delivery. The convergence of computing technology and network technology has created possibilities for information delivery that may bring us full circle to the privatization and individualization of information collections. At this juncture, it may be helpful to provide illustrations of selected digital libraries to demonstrate the state of the art, and to set the stage for what digital libraries can be.

Examples of Digital Libraries: The State of the Art

            There are probably hundreds, if not thousands of “digital libraries” on the Web today, regardless of whether they are called such. Some make their collections and services available for the public to view and use. Others are not for public consumption, but serve specific, private clientele. In some cases, digital libraries offer different views and different levels of service to different audiences. Therefore, one of the caveats of evaluating the state of the art of digital libraries is that, as one goes in search of digital libraries, one may not see what the primary users see. That is, for example, if I view the NASA site on remote sensing data (NASA, 2002), I might be viewing only a small part of a “collection” that NASA is making available to the public. They may have many advanced collaborative or analytic features for in-house use not available to the public.

            Nevertheless, based on what we have seen of digital libraries on the Web, we provide as examples only a selected few for analysis of the state of the art. These were chosen on the basis of what they offer in terms of advanced features. What we have found is that most digital libraries offer, as a front end for public viewing on the Web, a very limited set of features that take advantage of the capabilities of computing and network technologies. In fact, most digital libraries are modeled so heavily on their traditional counterparts that they are, in some cases, little more than hypertext catalogs with search engines.

            For example, JSTOR is a project whose mission is to provide digital access to scholarly journal articles. By using JSTOR, libraries might reduce their operating costs by making available individual journal articles instead of subscribing to, and storing back volumes of, journal titles. When articles are requested, JSTOR charges libraries the necessary copyright fees, and any additional charges necessary to continue operations. The site is set up to allow people to browse and select desired articles.

            However, it may not be necessary for JSTOR (2002) to provide much in the way of, for instance, collaborative services or other advanced services on their site. If their main purpose is to provide access to journal articles in digital format, then no such facilities may be necessary. On the other hand, a broader vision of their services might include an “authors’ common” in which authors might “meet” to search collectively, or to share versions of co-authored papers.

            One project that proposes such collaborative components is DEBORA (Digital Access to Books Of the Renaissance), (Nichols, Pemberton, Dalhoumi, Larouk, Belisle & Twidale, 2000) part of the Ariadne project in the United Kingdom. DEBORA is still in the design phase, but represents an attempt to replicate what is missing from the digital environment: collaboration. These researchers used ethnographic methods to assess how people interacted with physical (traditional) libraries, and found that there was significant interaction among library users that contributed to learning and finding.

            With these examples in mind, we will now look into what digital libraries might become.

Opportunities for Digital Libraries: Future Directions

            Some challenges must be overcome before digital libraries can realize their true potential. Among such challenges are those listed below. That is, we might:

·        Exploit various resources, distributed and in multiple formats, for data mining to support communities of practice

·        Expect digital libraries to support collaboration and interaction

·        Design digital libraries as integral parts of communities of practice

·        Design digital libraries as open, even dissipative systems that are sensitive to their environments.

This list is not exhaustive. For example, we might have included Copyright/Intellectual property issues, which will continue to plague the free flow of information for years to come. Similarly, preservation of digital materials remains a frontier that challenges researchers and practitioners. But for the sake of brevity, we have chosen to concentrate on only a few possibilities. How might these opportunities be realized? In the sections below we explore some possibilities.

Knowledge Discovery in Databases, Data Warehousing, Data Mining

            In traditional library situations, discovery of information has occurred by way of direct searching, by browsing, or by some hybrid of the two. In order to get information from a library, one had to go downstream to get it. The convergence of technologies such as computing, telecommunications, and artificial intelligence, however, can significantly enhance traditional library services. Presently available technology can: (1) use computers to sift through large quantities of data; (2) recognize patterns useful to a given individual, community, or context; and (3) communicate this information to users without the need for action on the user’s part. The use of technology for such purposes is referred to as knowledge discovery in databases (KDD). In order to facilitate KDD, information system designers use one, or a combination of, related techniques such as (but not limited to) data mining/data warehousing.

            Vast quantities of data, sometimes in the terabyte range, reside on computers in various locations. These data are potentially useful to a community that thinks of a way to make use of them. At any given time, however, a different context exists for which a community might need bits and pieces of data. A digital librarian might design systems that provide information in a contextually sensitive fashion. The context in which information is needed must be described in such a way that a system can act on a one-time request or on an ongoing scenario. A community member may express context as two types of questions: (1) one that for which the answer is known, or thought, to exist; or (2) one that calls for a prediction under certain conditions. Both types of questions are posed in order to increase the effectiveness of decision-making. The former is a classic information retrieval problem, i.e. a data warehousing problem; the latter, a data mining problem.

            Data Warehousing “is a subject-oriented, integrated, time-variant, and nonvolatile collection of data in support of management’s decisions” (Cabena, Hadjinian, Stadler, Verhees & Zanasi, 1997, p. 19). Subject-oriented refers to the organization of information under subject headings. Integration occurs when data are consistent, reliable, and accessible to an entire interested community. The data contained in a data warehouse are somehow time-stamped so that a particular collection is maintained chronologically. Finally, for the purpose of stability of a collection, data in data warehouses are not removed once acquired. Traditional libraries, with few exceptions, have acted as data warehouses. Some libraries actively “weed” collections, and therefore violate the nonvolatility principle just mentioned. However, most library collections are stable and are organized for access by subject, author, title, etc. By way of example, questions posed to libraries or data warehouses are of the type, “What is the population density of the 15201 zip code?” or “What information may I find on deforestation in the Amazon region?” This simple query can be answered with a straightforward, factual response. The response would then be used as an aid in decision-making—presumably to help decide whether zip code 15201 has the population density that has been a good market for privacy fences in the past.

            Data mining takes the next step. It is an activity designed to use technology to extract data (from data warehouses) that could be for prediction. Prediction in data mining comes from finding patterns in massive data warehouses. Businesses such as banks and grocery chains have used data mining techniques to improve their processes and to maximize narrow profit margins. For example, one grocery chain found that men who shopped on Thursdays, and bought disposable diapers, also tended to buy beer. These shoppers did large shopping trips on Saturdays, but on Thursdays, they made a small trip, which included the purchase of beer for the upcoming weekend. The chain then encouraged such buying behavior by placing beer displays close to diaper displays (Palace, 1996).

            Data mining is accomplished through any number of techniques such as user modeling, artificial neural networks, Markov models, Bayesian Networks, etc. Essentially, these techniques are employed in an ongoing manner in an attempt to find statistical patterns based on certain criteria. In the example of the grocery chain, programmed “agents” might use neural networks to build models of co-occurrences of various criteria expressed as “What do men buy at the express lane?” These agents can gather information about any purchases, and other related data such as time of purchase, age of customer, etc., and then cluster the data according to any criteria requested by a user.

            It is no small leap in thinking to extend this scenario into digital libraries. Suppose for example that a digital library was organized around a group of medical researchers specializing in the rather broad field of heart disease. Data could be collected into a data warehouse from any number of sources, and constantly mined for patterns on any criteria the researchers deemed worthy. This method could directly help to solve ongoing research problems, or it could even produce unexpected new research directions. That is, the digital library would be an “active partner” in research.

Support of Collaboration and Interaction

            One of the most important variables in the success of information systems and information seeking is the human beings that use information systems and who engage in information behaviors. Research by Twidale et al (1997) Spink (1997), Saracevic, Spink and Wu (1997), Belkin (1993), Ingwersen (1996), Kuhlthau (1991), and others has begun to uncover some of the patterns unique to information seeking behavior. For example, Twidale et al., in conjunction with the Ariadne Project in the United Kingdom, describe three types of collaboration in digital and physical libraries: patron--patron collaboration; patron--staff collaboration; and staff--staff collaboration. The bulk of their research focused on patron—staff collaboration, but it is reasonable to suggest that the other two areas are promising areas of research, as well. We can summarize what researchers in this area have found in the following points.

·        Collaboration is a fact of life in information seeking and use, and information system designers are only beginning to address the problems associated with supporting collaborative behavior.

·        In some instances, disintermediation is a fact of life (although in others, information intermediaries are overwhelmed with work). There is movement toward designing systems that will reduce the need for staff that assist end-users of information systems. Increasingly, users expect to be able to access information quickly and easily, without the need of human assistance. Therefore, systems that support novice end-user behaviors and collaborative work are needed.

·        Searching for information on complex projects requires multiple search iterations (Spink, Wilson, Ellis & Ford, 1998). That is, contact with information systems is not something that happens once during a project. During the life of a project, information is sought many times (both formally from an information system, and informally from colleagues). As information is gathered, and more is learned about the problems associated with a project, the need for information may increase.

·        Information seeking and use is inextricably bound with the social environment of the information seekers.

·        Patterns in interactive information retrieval are difficult to identify

Clearly, research on interaction in information retrieval is in its infancy. More research is necessary to: (1) identify variables in information seeking and retrieval behavior; and (2) establish means by which collaborative behaviors may be facilitated for information communities.

Open Systems and Networked Organizations

In order to garner the maximum benefits from the networked environment within which digital libraries operate, it is necessary to conceive digital libraries as open systems. Open systems are characterized by a high degree of interaction with their environment, and hence the ability to adapt to changes in environmental conditions (Malhotra, 1993). This ability to adapt is especially important when an organization’s environment is characterized by turbulence, such as rapid technological change. In other words, digital libraries as open systems are characterized as organizations that see themselves less as a collection of materials than they see themselves as opportunists. That is, a digital library’s collection may be different from minute to minute depending on the needs of the information community. The role of a digital library in this sense is to be active in its relationships outside of the organization to ensure that users are constantly supplied with needed information. Ward and Robins (1998) explain:

In essence, these types of … structures are the basis for the newly emerging information society. No longer “bounded” by self-identity, these newly developed information network organizations focus exclusively on resource exchange, and output. The very technology and structure of information networks now fosters this type of reconfiguration and production. By linking across networks, formats, and organizations, information in any format can be transformed into new formats without the necessity of waiting for organizational change. No longer dependent on "format" for data, the process of production becomes dynamic, and spins away from formal organizational or professional structure. Thus, what an organization "is" is no longer relevant, and instead what an organization can "become" is the only environmental imperative. (p. 380)

            The only framework for these new, open systems is that they will grow “around” an information community and its evolving needs.

A New Model for Digital Libraries and Information Communities

            From the discussion to this point, we can see that digital libraries have the potential to enhance the activities of information communities in various ways. To summarize, digital libraries should play an active part in the activities of an information community, rather than being a passive entity that “waits” for users to seek information. In addition, digital libraries should foster communication, collaboration, and interactive searching among information community members. Finally, digital libraries should be “organisms” that grow with the information communities they serve. This growth should be a natural outcome of a partnership with the information community, cultivated through active participation in the information community’s activities.

            A digital library in concert with an information community, is an unbounded entity, in that, there are no limits on information sources. The convergence of telecommunications and computing technology has made it possible for libraries to find sources of information without entering into restrictive contract agreements. In other words, digital libraries are open systems that allow themselves to freely exchange resources in an environment charged with information. By fostering collaboration among information community members, and by participating in the research of community members, digital libraries become involved in the invisible colleges associated with their clientele. This model of the role of digital libraries in information communities is shown in Figure 2.

Figure 2. A digital library as a network organization.

            In this model, digital libraries become almost indistinguishable from their information community clientele. There is still a system component to the model, but inputs are now in the form of user models, and user contributions to the “collection.” Processing and outputs are of primary importance. A digital library seeks a constant state of “pulling” information from its environment to be converted into suitable formats for data warehousing. From a data warehouse, it is possible to mine data for use by information community members. Output in such form is fed back to the community members, and to the processing system as needed. In fact, the information community itself is a major portion of the input to the digital library system, and therefore, a major portion of the feedback loop cycles there. In addition, a digital library, rather than being tied to inter-library loan agreements, or to resource supplier contracts, will be able to open lines of communications with related information communities who may already be in communication with those served by said digital library.

            The most important aspect of this model is that boundaries between library, community, researchers, information communities, and information resources, become fuzzy. The institution of “library” becomes less relevant as the speed and access to varieties of information resources increases. However, we are not quite there yet!

Some Caveats Regarding the Role of Digital Libraries in Information Communities

            Some barriers must be overcome before such a utopia may exist. Among such barriers are:

·        The web is not a data warehouse. Information on the World Wide Web is not created and maintained in a consistent format. This makes the work necessary to structure such data in a form usable in data warehousing extremely involved and labor intensive.

·        Copyright/Intellectual property issues will haunt the free flow of information for years to come.

·        Many of the automatic systems necessary to carry out automated services for digital library clientele require natural language processing. Human language is known to be a rich, but ambiguous, form of communication. This leads to unintended results from information retrieval systems.

·        Human beings are notoriously inconsistent in their judgments and behaviors, and are therefore difficult to model.

Despite these barriers, it is worthwhile to pursue the promise of the digital information age outlined in Figure 2.

Conclusion

            A digital library is less about its collection than it is about its ability to be opportunistic. In a time when information is much cheaper to produce and transport, every effort will be made to take advantage of increased access to information. Similarly, the increased computing power that can be brought to bear on information stores such as data mining will be employed for a variety of purposes, subject to the needs of a digital library and an information community. Given the model of the role of digital libraries in information communities presented in this document, a digital library may achieve what Ranganathan called a growing organism. But through its activities, a digital library may also become a “learning organism.” This ability to learn comes from the technological advancements of the last 40 years, particularly from the coupling of computing and telecommunications technology. Digital libraries are a natural extension of the evolution in which libraries have been involved for centuries. They represent a fundamental leap forward in the provision of services for, and the partnership with, information communities.

References

            Belkin, N. J. (1993). Interaction with texts: Information retrieval as information seeking behavior. In Information Retrieval ‘93: von der Modellierung zur Anwendung. Proceedings of the First Conference of the Gesselschaft für Informatik Fachgruppe Information Retrieval (pp. 55-66). Regensburg, 1993.

 

            Cabena, P., Hadjinian, P., Stadler, R., Verhees J. & Zanasi, A. (1997). Discovering data mining: From concept to implementation. Upper Saddle River, NJ: Prentice Hall.

 

            Crawford, W. & Gorman, M. (1995). Future libraries: Dreams, madness, and reality. Chicago: American Library Association.

 

            Fox, E. (Ed.) (1993). Source book on digital libraries (version 1.0). Prepared for and sponsored by the National Science Foundation.

 

            Griffin, S. M. (1998). Taking the initiative for digital libraries. The Electronic Library, 16(1), 24-27.

 

            Ingwersen, P. (1996). Cognitive perspectives of information retrieval. Journal of Documentation, 52(1), 3-50.

 

            Kuhlthau, C. C. (1991). Inside the search process: Information seeking from the user’s perspective. Journal of the American Society for Information Science, 42(5), 361-371.

 

            Lyons, P. (1989). Policy issues in computer networks: Multi-access information systems. Paper presented at the Annual Telecommunications Policy Research Conference. ED 318 415.

 

            Malhotra, Y. (1993). "Role of Information Technology in Managing Organizational Change and Organizational Interdependence," Working Paper, Katz Graduate School of Business, University of Pittsburgh, 1994. (Available http://www.brint.com/papers/change/).

 

            Miksa, F. (1996). The cultural legacy of the “modern library” for the future. Journal of Education for Library and Information Science, 37(2), 100-119.

 

            NASA (National Aeronautics and Space Administration). (2002). Visualization of remote sensing data. Available http://rsd.gsfc.nasa.gov/rsd/. 

 

            Nichols D. M, Pemberton, D., Dalhoumi, S., Larouk, O., Belisle, C. & Twidale, M. B. (2000). DEBORA: Developing an interface to support collaboration in a digital library. Proceedings of the Fourth European Conference on Research and Advanced Technology for Digital Libraries (ECDL2000) (pp. 239-248). Heidelberg: Springer-Verlag.

 

            Palace, B. (1996). Data mining. Unpublished paper, Anderson Graduate School of Management at University of California at Los Angeles. Available http://www.anderson.ucla.edu/faculty/jason.frand/teacher/technologies/palace/index.htm

 

            Saracevic, T., Spink, A. & Wu, M. (1997). Users and intermediaries in interactive information retrieval (IR): What are they talking about? (pp. 43-54). Proceedings of the sixth international conference on user modeling. New York: Springer Wien.

 

            Spink, A. (1997). Study of interactive feedback during mediated information retrieval. Journal of the American Society for Information Science, 48(5), 382-394.

 

            Spink, A., Wilson, T., Ellis, D. & Ford, N. (1998). Modeling users' successive searches in digital environments. D-Lib Magazine, April. (Available http://www.dlib.org/dlib/april98/04spink.html).

 

            Twidale, M. Chaplain, D, Crabtree, A., Nichols, D., O’Brien, J. & Rouncefield, M. (1997). Collaboration in physical and digital libraries. (British Library Research and Innovation Report 63). (Available WWW: http://www.comp.lancs.ac.uk/computing/research/cseg/projects/ariadne/bl/report/).

 

            Ward, R. C. & Robins, D. (1998). The emergence of dissipative structures within information provider organizations. ASIS '98: Proceedings of the 61st ASIS Annual Conference, 35, 373-388.