Committee Description

The Evaluation Workgroup considers evaluation broadly, and sees it as an activity that impacts most if not all of the functions of the NSDL. In particular, formative evaluation addresses the planning and implementation aspects of the NSDL, from assessing the needs of potential users, to designing the user interface and testing it with users. Additionally, evaluation addresses the impact of the project more broadly from the specific user, to a system of users, to its impact on SMET education at all levels of education: secondary and post secondary, formal and informal. Driven by stakeholders and participants interests, evaluation activities also relate to selection of the library?s holdings and evaluation of their quality. Finally, evaluation must examine the impact of the library on student learning.

Evaluation interests and needs will be as broad as the NSDL user community. Thus evaluation must be flexible and nimble enough to answer the questions posed by K-12 SMET teachers, instructors in higher education, students at all levels, contributors, publishers, librarians, professional developers, public policy leaders, planners, and educational researchers as well as the NSDL developers and designers themselves. Evaluation results must ?speak to? this broad audience in clear, easy to translate language. The results must be articulated in such a way as to encourage their use in decision making by all participants.

We anticipate that evaluation of the NSDL will by necessity be multi-method in nature. Methods (e.g., focus groups, survey, interview, and observation) will all be employed in order to respond to various evaluation needs and questions. Evaluation should build on the existing, though nascent evaluation efforts already undertaken under the auspices of NSF DLI ? 2 funding and other funding supporting digital library efforts.

The goals for the Evaluation Workgroup center around disseminating information to others interested in implementing evaluation, as well as identifying effective evaluation designs, methods, tools, and processes for effective evaluations related to specific purposes such as informing site design, impact on student learning, use and adaptability of holdings, or user satisfaction. (Please note, this list is meant to be illustrative, not exhaustive.) Specific outcomes for the group include:

• compilation, and dissemination of existing studies conducted by NSDL participants
• compilation, and dissemination of studies related to digital library efforts
• identification of ?best practices? in evaluating digital library programs, services and collections
• coordination of evaluation efforts, including collaborative identification of evaluation metrics, indicators, tools, methods, etc. for the NSDL