Presenter: Brent Cook, Product Manager, Summon (ProQuest)
Cook started off providing a clear example of the challenge of relevance, asking us to consider our expectation from the following search terms :
- apple (the food, the company?)
- Saturn (the planet, the car?)
- to be or not to be (all stop words)
Relevance improvements have been made to known-item search, which boosts record matching for combination fields such as title, author, publication data. A recently released new search algorithm addresses the imbalance between dynamic ranking (what’s happening at time of search) and static ranking (the type of content). This had been resulting in an over emphasis on records with short titles. Improvements lead to short and general topic search terms returning more books, whereas longer and specific queries would return articles. Even if it wasn’t exact, these results were more likely to produce records that would guide the user to better results.
Cook noted an important additional consideration in any usability study: that just collecting search terms is not effective as combining that with good user feedback on expectations from those search terms. Another challenge to relevance is the whack-a-mole problem of fixing one problem which causes another, and this requires constant monitoring and the understanding that the work is never done. To avoid whack-a-mole, the process for improving relevance involves, assessment, development, evaluating internally, testing externally and review candidate before either revise or release. Could be two or three releases in the process of revision and improvement.
Moving forward, plans are to continue to incorporate feedback on the current algorithm and adding version 2 to English and German searches, including better handling of spelling variations and verb stemming (e.g. searching, search, searched). Cut & pasted citation query is also a priority consideration. Finally, subject term boost, is a new test that they are currently seeking volunteers to help. Email summon.relevance.feedback@PRoquest.com with subject line: Subject Term Boost.
Questions from the audience:
Q: Is Summon considering more Gooogle-like following of data to recommend different searches or results?
A: “Did you mean” and results-based recommendations, definitely. But otherwise, requires ability to identify user which is both a privacy and multiple login issue.
Q: What about relevancy by institutional IP?
A: Could do something by institution type like med or law schools. Challenges involved around inadvertently hindering performance or diverse relevancy of result.