- So is the idea that if I have a bunch of stuff that had a system map onto the concept space [then your system would be useful]?
Tammy Sumner [TS]: Basically there are a couple ways our system can be used. In addition to be able to search and browse and understand. We�ve been working with teachers, and one of the issues they have is that they don�t usually have field or domain knowledge. A lot of them are teaching out of field. We hope by providing conceptual overviews it will help teachers evaluate the resources that are returned through other discovery interfaces. We anticipate that most of the resources will actually be retrieved through NSDL interfaces. We are modeling alignment with national and state standards to try to help with this. Another project is at San Diego State trying to do this alignment in different ways and to do it automatically.
- I know in the case of the AAAS benchmarks, there would be a big mismatch in the granularity of activities. Benchmarks are about things that are such large units of instruction they may show up on your report card. They�re not about individual lessons. Do you think that may be part of your problem, with having chosen lesson plan?
Anne Diekema [AD]: So what would you suggest? That�s basically the only data we have.
Questioner: I think it�s something we should talk about.
- We have local systems that are taking the national standards and benchmarks and are driving down very deep to the lesson plan level. You might want to take one of those, since that�s what you�re doing is expanding the standards.
AD: Is that data available?
Questioner: We haven�t started using it at the Math Forum, but they are available.
- Did you run into any issues � we ran into issues in mathematics where terms at one level mean something different than terms at another level.
Darin Burleigh [DB]: We discussed this issue at some depth. Because we had this hierarchy, people thought that the lineage of the term qualified the term. For example, if the term reactions appeared under alcohol as well as under something else, people thought it was clear that reactions meant alcohol reactions as opposed to some other kind of reaction. In the end, we decided that didn�t work. We thought terms had to stand on their own.
- Were you aware of and informed by any of the thesaurus standards that are available?
DB:Yes, that actually wound up being more useful than the hierarchy we started with. We put in a layer of relationships instead.
Questioner: It doesn�t look like those were exposed.
DB: They�re not, we�re just working with them for our use now.
Questioner: So did you discover these standards part way through your process?
DB: No, we knew about them from early on. The question is how to best expose them to the world. We don�t want to just make it available when it�s not a fully formed idea. We want to come up with a more streamlined online submission process including having the authors select keywords. So we want them to be able to find terms in this list of 300-some words in a way that makes sense, so maybe these relationships will help that. We�re really, in terms of assigning things, we�re looking for a qualified Dublin Core for chemistry.
- I�m interested in knowing if you have decided what really constituted educational value.
Kristen Catlin [KC]:Just for microbiology. The microbiology educators got together and since they were the ones on the front lines, they decided what they would find of value.
Questioner: Is that documented?
KC: Yes, it�s on the ASM website and the Microbe Library website, the whole process is documented there.
- So are you using keywords as your term for controlled vocabulary? Is that a controlled vocab, or is it just words extracted from the resource?
KC: It�s not controlled. They keywords are decided upon by the author. We have a list of terms, but if someone submits a resource and they come up with a keyword that they think is important to searching for that resource, then they have the ability to add it. So it�s not controlled at all.
- I wanted to follow up on that question about waiting for something from qualified DC.
DB: We have all of our subject terms in one big list. To really navigate and understand that, we need to take all of chemistry and divide it up, break it down somehow. We have a categorization we use internally, but we don�t know that it�s the best way to do it. Now is the time to start thinking about what we�re going to use for qualified Dublin Core. Does everyone get to use their own, or are we going to make some kind of universal?
Questioner: DC already supports the sort of controlled syntax you�re waiting for. It already exists. What needs to happen to use those things interoperably depends on the support that your organization wants to give them. I don�t think there�s anything that you�re waiting for from Dublin Core.
DB: We are sort of groping around trying to figure out how do we take all of chemistry and break it up somehow? Can we take some chemistry education terms and then not worry about stepping on anyone else�s toes? That�s the kind of thing we�re working with.
- For the economics project, are you building in any options where the user can disable the server queries so that they can find software that will work on other platforms?
DZ:That asks what is available from the connection itself. The metadata doesn�t solve the problem of at the back end whether you have the software that can run on different platforms. If you have a Java application, then metadata will be able to help, but others programs may not be able to switch platforms.
Follow up from another questioner: You�re letting people write in any environment?
DZ: Yes; we have two classes of software available. One class is stuff we have built with Java so it will run everywhere, and some server-centric stuff that will run on any platform. The second class is that we allow people to write their own software and bring it together and let the user play with it.
Questioner: It seems like the parameter space could get rather large: even Java has different versions. Do you have a testing lab for bringing in software from others?
Currently, even for the second class, it is not something that they can fill out a form and automatically things will run. We are going to do some modification so that it will become compatible with our infrastructure. What we�re trying to do here is create an additional layer of software, an API, that provides some basic functions that are common to all experiments. Stuff like how to manage students or save parameter files. So then we could do those sort of intermediary things and hopefully developers will use that API.
- I had a question about the Microbiology Library. You said in generating your metadata you had educators and used the introductory class as your basic idea. Do you have problems with higher level classes not fitting into the schemes?
KC: The library is aimed at undergraduate education. The core themes are broad enough that even though the topics are aimed at an undergradute level even someone at a graduate level could find something appropriate.
- Have any of you done user studies, following what do users do with your vocabulary? What have you learned? For example, our users have varying abilities to use our vocabulary.
DB: We haven�t taken on the next task to see how people are actually searching, whether they�re using controlled vocabulary or whether they�d do that if we had a different interface. What we would like to do is use the thesaurus idea and building in from our controlled vocabulary to other related terms (possibly narrower terms), so they can find everything related to our controlled vocabulary. You can lead a horse to water but you can�t make him choose the right vocabulary.
TS: This is why we�re trying to lay out the concept space so that people can navigate through different terms graphically, we assume people don�t know the vocabulary.
Follow-up from another questioner: We�ve found that that�s effective but we haven�t found that it�s used by the users. You (chemistry project) have combined several terms into one vocabulary. Users look at the learning context/audience and browse the topics but don�t use that for searching frequently.
DB: We can�t even get our authors to choose from our controlled vocabulary list. We also let authors use other terms and consider adding them to our list eventually.
KC: We�ve done user surveys but they haven�t addressed that point. We do know that 70% of the people using it are undergraduate microbiology educators. We�re hoping to track that after we do some changes.