Assessment in Action

Virginia Polytechnic Institute and State University: Project Description

Page 1


Primary Outcome Examined (select one or more)

(No) Student Learning: Assignment

(No) Student Learning: Course

(No) Student Learning: Major

(No) Student Learning: Degree

(No) Student Engagement

Student Success

(No) Academic Intimacy/Rapport

(No) Enrollment

(No) Retention

(No) Completion

(No) Graduation

(No) Articulation

(No) Graduates' Career Success

(No) Testing (e.g., GRE, MCAT, LSAT, CAAP, CLA, MAPP)

(No) Other (please describe)

Primary Library Factor Examined (select one or more)

(No) Instruction

(No) Instruction: Games

(No) Instruction: One Shot

(No) Instruction: Course Embedded

(No) Instruction: Self-Paced Tutorials

(No) Reference

(No) Educational Role (other than reference or instruction)

(No) Space, Physical

Discovery (library resources integrated in institutional web and other information portals)

(No) Discovery (library resource guides)

(No) Discovery (from preferred user starting points)

(No) Collections (quality, depth, diversity, format or currency)

(No) Personnel (number and quality)

(No) Other (please describe)

Student Population (select one or more)

(No) Undergraduate


(No) Incoming

(No) Graduating

(No) Pre-College/Developmental/Basic Skills

(No) Other (please describe)

Discipline (select one or more)

(No) Arts

(No) Humanities

(No) Social Sciences

(No) Natural Sciences (i.e., space, earth, life, chemistry or physics)

(No) Formal Sciences (i.e., computer science, logic, mathematics, statistics or systems science)

(No) Professions/Applied Sciences

(No) English Composition

General Education

(No) Information Literacy Credit Course

(No) Other (please describe)

AiA Team Members (select one or more)

Assessment Office

(No) Institutional Research

(No) Teaching Faculty

(No) Writing Center

(No) Information/Academic Technology

(No) Student Affairs

(No) Campus Administrator

(No) Library Administrator

Other Librarian

(No) Other (please describe)

Page 2


Methods and Tools (select one or more)

(No) Survey

(No) Interviews

(No) Focus Group(s)


(No) Pre/Post Test

(No) Rubric

(No) Other (please describe)

Direct Data Type (select one or more)

(No) Student Portfolio

(No) Research Paper/Project

(No) Class Assignment (other than research paper/project)

Other (please describe)

Usability Test

Indirect Data Type (select one or more)

(No) Test Scores

(No) GPA

(No) Degree Completion Rate

(No) Retention Rate

Other (please describe)


Page 3

Inquiry Question (150 words open)

What was the project's primary inquiry question?

  1. The question the Virginia Tech AiA Team sought to answer was how effectively do graduate students use Summon? At Virginia Tech Summon is taught extensively in First Year Experience courses. It provides a simple interface conducive to interdisciplinary, undergraduate research. At the graduate level, students are expected to already have well developed, sophisticated library research skills tailored closely to their discipline of study. Therefore, graduate level information literacy courses do not focus on Summon search skills but instead emphasize research skills that are targeted toward discipline-specific databases. Interdisciplinary research is increasingly more prominent at the graduate level and this type of research requires skillfully refined searches that are well suited for a discovery layer. Our AiA team sought to examine graduate students use of Summon by comparing their skill sets with undergraduate students and faculty.

Executive Summary (150 words open)

  • How does the project align with your institution’s priorities and needs?
  • Why did you choose the outcome and library factor as areas to examine?
  • Why was the team composition appropriate?

  1. Providing transformative graduate education is an initiative that helps graduate students build an inclusive interdisciplinary academic community. And assessing our graduate student’s use of Summon aligns with one of our strategic goals; to collect and analyze the learning and research needs of Virginia Tech students and faculty. We know that our graduate students’ academic and research needs are quite different from the needs of their undergraduate student counterparts. Therefore, a thorough assessment of Summon from the perspective of our graduate students’ should yield practical data that can inform our actions in better aligning library services, tools and training programs that meet the expectations and needs of these students’ in order to increase their level of success.

  • What are the significant contributions of your project?
  • What was learned about assessing the library’s impact on student learning and success?
  • What was learned about creating or contributing to a culture of assessment on campus?
  • What, if any, are the significant findings of your project?

  1. The project was developed to study our Summon discovery tool and to analyze its alignment with our graduate students needs as researchers. The assessment indicated that overall graduate students were adept at using Summon more effectively than faculty or undergraduate students. However, many graduate students demonstrated some difficulty with finding relevant full-text articles and e-books, which requires the appropriate use of filters and subject terms. Additionally, Graduate students only used the Advance Search features 8% of the time, the least among the three groups tested. This may be significant because using appropriate subject terms in the Advance Search fields provides the most targeted, relevant results. An area for future research would be to study whether the effective use of Summon correlates to the type of library instruction that graduate students receive.

  • What will you change as a result of what you learned (– e.g., institutional activities, library functions or practices, personal/professional practice, other)?
  • How does this project contribute to current, past, or future assessment activities on your campus?

  1. Data-driven decision making is a vital part of the Virginia Tech culture of continuous improvement. We are heavily invested in the uses of assessment for our continued growth and excellence in University Libraries. As a result of participation with AiA, we will continue to assess our students' success with the Summon discovery layer. This will enable us to further engage students in scholarly inquiry and provide students with the tools needed in conducting research using innovative technologies in research, scholarship, and learning. This assessment of our Summon service has challenged our assumptions of graduate student research methods and the tools needed for success. As a commitment to our student’s success, we will continue the assessment of our discovery layer in order to transform the ways our patrons discover, access, and experience information.

Please list any articles published, presentations given, URL of project website, and team leader contact details.

  1. Tracy Gilmore is the Collections Assessment Librarian at Virginia Tech, University Libraries and can be reached at

How effectively do graduate students use Summon? Twenty-three graduate and undergraduate students participated in a usability study to measure skill factors that impact research. The study indicates that graduate students were adept at using Summon more effectively than faculty or undergraduate students. Future research would be necessary to know if this correlates to frequency and type of library instruction.
Gilmore_2016_AiA-Poster.pdf Assessing graduate student use and skill with the Summon discovery service