Assessment in Action

CUNY Borough of Manhattan Community College: Project Description

Page 1


Primary Outcome Examined (select one or more)

(No) Student Learning: Assignment

(No) Student Learning: Course

(No) Student Learning: Major

(No) Student Learning: Degree

(No) Student Engagement

Student Success

(No) Academic Intimacy/Rapport

(No) Enrollment

(No) Retention

(No) Completion

(No) Graduation

(No) Articulation

(No) Graduates' Career Success

(No) Testing (e.g., GRE, MCAT, LSAT, CAAP, CLA, MAPP)

(No) Other (please describe)

Primary Library Factor Examined (select one or more)

(No) Instruction

(No) Instruction: Games

(No) Instruction: One Shot

(No) Instruction: Course Embedded

(No) Instruction: Self-Paced Tutorials


(No) Educational Role (other than reference or instruction)

(No) Space, Physical

(No) Discovery (library resources integrated in institutional web and other information portals)

(No) Discovery (library resource guides)

(No) Discovery (from preferred user starting points)

(No) Collections (quality, depth, diversity, format or currency)

(No) Personnel (number and quality)

(No) Other (please describe)

Student Population (select one or more)


(No) Graduate

(No) Incoming

(No) Graduating

(No) Pre-College/Developmental/Basic Skills

(No) Other (please describe)

Discipline (select one or more)

(No) Arts

(No) Humanities

(No) Social Sciences

(No) Natural Sciences (i.e., space, earth, life, chemistry or physics)

(No) Formal Sciences (i.e., computer science, logic, mathematics, statistics or systems science)

(No) Professions/Applied Sciences

(No) English Composition

General Education

(No) Information Literacy Credit Course

(No) Other (please describe)

AiA Team Members (select one or more)

(No) Assessment Office

Institutional Research

(No) Teaching Faculty

(No) Writing Center

(No) Information/Academic Technology

(No) Student Affairs

Campus Administrator

(No) Library Administrator

(No) Other Librarian

Other (please describe)

Page 2


Methods and Tools (select one or more)

(No) Survey

(No) Interviews

(No) Focus Group(s)

(No) Observation

(No) Pre/Post Test

(No) Rubric

Other (please describe)

Direct Data Type (select one or more)

(No) Student Portfolio

(No) Research Paper/Project

(No) Class Assignment (other than research paper/project)

Other (please describe)

Indirect Data Type (select one or more)

(No) Test Scores

(No) GPA

(No) Degree Completion Rate

(No) Retention Rate

(No) Other (please describe)

Page 3

Executive Summary (150 words open)

  • How does the project align with your institution’s priorities and needs?
  • Why did you choose the outcome and library factor as areas to examine?
  • Why was the team composition appropriate?

  1. Borough of Manhattan Community College is a large, diverse, urban community college. The Library has provided 24/7 Chat Reference since 2007, using the QuestionPoint service from OCLC. In this project we undertook to analyze the transcripts for the year 2013.

    We now know that about 20% of the interactions took place outside of library hours (validating the use of a 24/7 service.) We have observed that the service is often used by students with English fluency issues. We now know that students are persistent. Evidence of student learning certainly exists within the contents of these anonymous transcripts, but it difficult to pin down. There continues to be evidence of technological barriers.

    There is a negative correlation between the announced best practices of the service, and the reality. We need to, within the QuestionPoint community to continue to discuss the difference between providing information, and providing instruction.

  • What are the significant contributions of your project?
  • What was learned about assessing the library’s impact on student learning and success?
  • What was learned about creating or contributing to a culture of assessment on campus?
  • What, if any, are the significant findings of your project?

  1. What makes this project unique is the amount of data that was analyzed. It is a significant contribution to the research on virtual reference. The major takeaways are the persistence of the students, the importance of virtual reference for a commuter campus, and the need to continue to talk about best practices. It is difficult it is to discover evidence of student learning in a way that can be documented, using anonymous transcripts. It is also important to continue to talk about the technological challenges to our students face, particularly when dealing with a community college population.

  • What will you change as a result of what you learned (– e.g., institutional activities, library functions or practices, personal/professional practice, other)?
  • How does this project contribute to current, past, or future assessment activities on your campus?

  1. I believe it is my responsibility to open additional conversations with the question point community. We need to discuss best practices, and the possibility of additional assessment measures.

    I have taken on additional responsibility for the assessment of my library. We will be undertaking this summer to set up a structure for assessing the whole library, not just the instruction program. I'm also now a member of the college wide Assessment Committee.

    I am also seriously considering going back to graduate school, particularly giving thought to a hybrid program (thanks to my positive experience with AIA).

Please list any articles published, presentations given, URL of project website, and team leader contact details.


    Robin Brown, MLS, MA
    Information Literacy Librarian
    Assistant Professor
    Borough of Manhattan Community College
    New York, NY 10007

The Library has provided 24/7 Chat Reference since 2007, using the QuestionPoint service from OCLC. We undertook to analyze the transcripts for the year 2013. About 810 transactions were documented. We have used both quantitative and qualitative measures to try and understand how the service is being used, and to open up conversations about best practices. Major outcomes include: English fluency issues, users are persistent (logging in more than once), and there are still technological barriers.
publication.pdf Lifting the Veil: Analyzing Collaborative Virtual Reference Transcripts to demonstrate value and Make Recommendations for Practice