NSDL Metrics Working Group & Report
In January 2009, in discussion with Pathways, a new working group was formed—the NSDL Metrics Working Group. The group is charged with exploring metrics-related issues and making recommendations to the NSDL community on best practices for metrics gathering and analysis, over multiple aspects of an NSDL project's activities and lifecycle. The working group is composed of NSDL projects' staff, and representatives from the NSDL Resource Center and NSDL Technical Network Services, and meets monthly via teleconference.
- Kim Lightle, Chair - Middle School Portal: Math and Science Pathways (MSP2)
- Ed Almasy, Applied Math and Science Education Repository (AMSER)
- Yolanda George, Biosciences EdNet (BEN)
- Lyle Barbato, ComPADRE Physics & Astronomy Pathway
- Paul Mackinney, Engineering Pathway (EP)
- Cathy Lowe, Materials Digital Library (MatDL) Pathway
- Sherry Hsi, SMILE Pathway
- Eileen McIlvain, NSDL Resource Center
- Sharon Clark, NSDL Technical Network Services
The group will report on their recommendations via this NSDL Community site, and at the 2009 NSDL Annual Meeting in November 2009. Stay tuned for more info - as it becomes available, notice will be posted here.
This post includes comments from the Web Portal discussion group at the Metrics session at the NSDL Annual Meeting.
Usage
- Usage is a broad topic and needs to be defined and subdivided further. Move beyond 'hits' so the data can be used in an informative way.
- What kind of usage - frequency or repeatability of visits.
- Is geographic data useful? Just knowing that a bunch of people from a city/state/region may not necessarily, but knowing why would be - is it a result of professional development or marketing activities? Are people sharing links?
- Alternatively, collecting Zip code data to determine geographic information is useful for funding and partners. Collecting the zip code of the school can be reported back to funders and be used to determine if users in the targeted areas are increasing. This is especially important to K-12 - they want to know # of users in a state (depts of ed). In these cases, numbers are just as important as trends.
- Are the users from beyond your target audience? Different grade ranges (undergraduates vs high school teachers, international audiences)?
Engagement
- on-site vs off-site (comments on the site vs link sharing through emails, twitter, etc)
Brand Reputation
- You can't get this from raw data, you need to ask them via surveys, etc. First determine if they aware of you, then determine what is their opinion of your resources.
- There's tensions between the different brands - NSF/NSDL/Pathway/Subcollections. What group is a user more interested in being the 'source' of a resource.
- Brand Recognition may eventually make a difference for sustainability.
Additions to Activities Matrix (Web portal)
Web Metrics
user demographics (institution type, student population served, teaching context)
Digital Library collections & collection building (diversity of resource types and subjects, number of resources rejected before accessioning)
Highlighted resource feature - this needs to be clarified - is it special collections or editor's choice?
Foldering/bookmarking systems - is there a way to track use or how many times a user returns to saved items?
Recommender services - this seems to fit better under social media.
Number of recommendations received - include # cataloged
Hi, all -
The Metrics Working Group Draft report will be available on the Annual Meeting homepage and this Forum by Monday morning, November 16. FYI - there is also a Metrics Interest Group, open to anyone in the NSDL community who wants to contribute ideas, discussion, suggestions, examples, etc. of metrics collection, use, and reporting. The Metrics Working Group Report Draft is an ongoing project, and the Working Group is eager to get your response and contributions to help us make the final report as comprehensive as possible.
best,
Eileen
Hi, everyone -
The Metrics Working Group is very pleased to make available the Draft Report: Metrics Recommendations and Resources for NSDL Projects - check it out! We hope you find its recommendations and suggestions helpful in your metrics collection and reporting efforts. That DRAFT status is important - we'd love to get your feedback, so we can ultimately finalize the report with your valuable input. So respond here with comments, questions, suggestion, critiques. We want to hear from you!
regards,
Eileen