apply.ala.org
Assessment in Action

University of Massachusetts-Boston: Project Description

Page 1

Top

Primary Outcome Examined (select one or more)

(No) Student Learning: Assignment

Student Learning: Course

(No) Student Learning: Major

(No) Student Learning: Degree

(No) Student Engagement

(No) Student Success

(No) Academic Intimacy/Rapport

(No) Enrollment

(No) Retention

(No) Completion

(No) Graduation

(No) Articulation

(No) Graduates' Career Success

(No) Testing (e.g., GRE, MCAT, LSAT, CAAP, CLA, MAPP)

(No) Other (please describe)

Primary Library Factor Examined (select one or more)

Instruction

(No) Instruction: Games

Instruction: One Shot

(No) Instruction: Course Embedded

(No) Instruction: Self-Paced Tutorials

(No) Reference

(No) Educational Role (other than reference or instruction)

(No) Space, Physical

(No) Discovery (library resources integrated in institutional web and other information portals)

(No) Discovery (library resource guides)

(No) Discovery (from preferred user starting points)

(No) Collections (quality, depth, diversity, format or currency)

(No) Personnel (number and quality)

(No) Other (please describe)

Student Population (select one or more)

Undergraduate

(No) Graduate

(No) Incoming

(No) Graduating

(No) Pre-College/Developmental/Basic Skills

(No) Other (please describe)

Discipline (select one or more)

(No) Arts

(No) Humanities

(No) Social Sciences

(No) Natural Sciences (i.e., space, earth, life, chemistry or physics)

(No) Formal Sciences (i.e., computer science, logic, mathematics, statistics or systems science)

(No) Professions/Applied Sciences

English Composition

(No) General Education

(No) Information Literacy Credit Course

(No) Other (please describe)

AiA Team Members (select one or more)

Assessment Office

Institutional Research

Teaching Faculty

(No) Writing Center

(No) Information/Academic Technology

(No) Student Affairs

(No) Campus Administrator

Library Administrator

Other Librarian

(No) Other (please describe)

Page 2

Top

Methods and Tools (select one or more)

Survey

(No) Interviews

(No) Focus Group(s)

(No) Observation

Pre/Post Test

Rubric

(No) Other (please describe)

Direct Data Type (select one or more)

(No) Student Portfolio

(No) Research Paper/Project

(No) Class Assignment (other than research paper/project)

Other (please describe)

Booklet (ungraded) that students completed during library research instruction session(s)

Indirect Data Type (select one or more)

(No) Test Scores

(No) GPA

(No) Degree Completion Rate

(No) Retention Rate

(No) Other (please describe)

Page 3

Top
Inquiry Question (150 words open)

What was the project's primary inquiry question?

  1. Primary inquiry question: How does research instruction impact students’ ability to transfer research skills from one project to the next?

    Research skills:
    • Understand the difference between Googling information and conducting academic research
    • Use keywords, subject language, and controlled vocabulary to search for and access information on your paper topic
    • Understand how to handle conflicts between what you find and what you want to research/avoiding “cherry-picking”

    Additional exploratory query: Is there an interest in co-taught or librarian-supported, faculty-led research instruction classes?
    • 7 professors (14 sections): librarian-led research instruction
    • 3 professors (10 sections): hybrid/solo-faculty led research instruction

    Preliminary data suggest that, overall, librarian-led and hybrid/solo-faculty led research instruction yielded comparable post-survey results. Interest was expressed by ENGL 102 faculty in assuming responsibility for varying amounts of research instruction in future semesters.

Executive Summary (150 words open)

  • How does the project align with your institution’s priorities and needs?
  • Why did you choose the outcome and library factor as areas to examine?
  • Why was the team composition appropriate?

  1. Departmental learning goals are part of UMass Boston’s strategic plan to examine student progress and assess academic disciplines’ learning outcomes. ENGL 102 is an integral part of the freshman writing curriculum and addresses skills needed to satisfy the institution’s Writing Proficiency Requirement, which undergraduate students must pass in order to graduate. Students learn to write research-based essays and connect research to the writing process in ENGL 102, which makes it an ideal candidate for a course-embedded collaborative assessment of the impact of library research instruction on student learning. Through this project, we gained a clearer picture of the impact of library research instruction on student success. This is important because instruction librarians taught over 580 classes this academic year; given this extensive workload, it is important to measure whether our efforts have any impact on student learning and success so we can improve our instruction program and individual efforts.

  • What are the significant contributions of your project?
  • What was learned about assessing the library’s impact on student learning and success?
  • What was learned about creating or contributing to a culture of assessment on campus?
  • What, if any, are the significant findings of your project?

  1. Our project demonstrates some of the benefits of integrating writing and research instruction while indicating librarians and English faculty are natural partners in accomplishing this. It also highlights the library’s contributions to student learning, establishing the library as a potential collaborator in future learning assessment efforts.

    Findings include:
    • Students understood and appreciated the ways they could use library resources to research their paper topics and access credible sources
    • Students who did not identify the best answer in the post-survey often picked a good "wrong" answer
    • Students brainstormed more keywords in the post-survey than they did during research instruction sessions, and increased the number of synonyms and related terms in the post-survey (instead of simply mirroring the language describing their paper topics)
    • The value in collaborating with academic departments to assess student learning outcomes extends beyond one project
    • Library instruction doesn’t necessarily require a librarian’s presence in the classroom

  • What will you change as a result of what you learned (– e.g., institutional activities, library functions or practices, personal/professional practice, other)?
  • How does this project contribute to current, past, or future assessment activities on your campus?

  1. We are now more confident in designing library instruction sessions with built-in assessment and are continuing to collaborate with the English department by expanding our focus to all English courses and by piloting a new instructional model in which library research instruction is broken into smaller chunks and faculty will be encouraged to assume greater responsibility for teaching more of the components themselves. That way, more students enrolled in English courses benefit from library instruction, the amount of time a librarian spends in the classroom is reduced, and research instruction becomes embedded throughout the course.

    Our project built on work the library is doing with the sociology department and serves as a successful model for future collaborative assessment efforts. We will continue building upon the work that we began with the English department, exploring and assessing different ways we can work together to facilitate student learning and success.

Please list any articles published, presentations given, URL of project website, and team leader contact details.

  1. Team Leader:

    Iris Jahng
    Pedagogy and Learning Design Librarian
    Healey Library
    University of Massachusetts Boston
    100 Morrissey Boulevard
    Boston, MA 02125-3393
    617.287.5754
    iris.jahng@umb.edu

In Spring 2016, Healey Library and the English department at UMass Boston assessed the impact of library research instruction (LRI) on student learning in ENGL 102. 320 students in 24 sections received LRI. To measure this, booklets the students completed in class were evaluated with a rubric and a post-test was administered at the end of the semester. LRI has a measurable impact on student learning, and students retained and transferred what they learned.
Filename
AiA_Poster_-_UMass_Boston_-_Jahng.pdf Collaborative Assessment of Student Learning: A Tale of Two Departments