Judith Herschman, Arts Librarian at University of California, Los Angeles, discussed the survey instrument and preliminary results of an assessment project conducted at UCLA 1998-2000. She is a member of the Instructional Services Advisory Committee (Patty Caravello, Chair) that designed and conducted the survey. This report will only touch on some of the highlights of her presentation. [Note: The UCLA Instructional Competencies Survey Project was first mentioned at the CCLI-South Spring Program, "What's all this about Information Literacy?", held at Claremont Colleges, June 12, 1998.]

Formal presentation of the results and findings will be at the ACRL National Conference to be held in Denver in March 2001. This data has not been presented yet to the UCLA faculty, as it is still in discussion within the library since the results were just compiled in November 2000.

Background
The survey had the following objectives:
Gain understanding of UCLA undergraduate information competencies.
Improve effectiveness of library instruction.
Provide UCLA with data for faculty discussion.

A literature review revealed very little research or findings to work with, mostly in the form of pre- and post-tests.

Measures
The committee developed their own measures of information literacy as this survey was designed before the acceptance and publication of the Information Literacy Competency Standards for Higher Education (January 2000). The UCLA measures coincide with most of the Standards.

The most difficult competency to measure, according to Ms. Herschman, was to be able to organize and synthesize the information; critically use and integrate information from a variety of sources; and understand the need to compile a bibliography and create footnotes.

Study Design
The goal was not to get a random sample, but systematic sample of the undergraduates. The Registrar's Office identified the sample group. Participants received ten dollars as an incentive and given a choice of four dates to take the survey. The statistical analysis was done by the survey center.

Survey Instrument
11 demographic questions (including level of library use and online experience.)
14 questions with 47 variables designed to test information competencies were administered. The questions were keyed into the committee's identified competencies.
The instrument was reviewed by library staff; circulation services students; and large lecture classes, with changes being made after each trial administration.
The survey was administered to library school students afterwards to compare average score results, with the expectation the library students would score better.

Findings
The average score was 61.7%, compared to 86.8% for the library school students. This score is similar to St. Olaf College's findings of 61% for a similar survey.
Those who used the library more frequently had a statistically significant higher mean score.
There was no statistical difference between classes except seniors.
No relationship was found between amount or kind of library instruction and test score.
No consistency of skill or understanding within any set of measured competencies.
Those who used the library more frequently did better.
Orion (UCLA online catalog) users scored higher as did users of Melvyl and journals databases, by at least 5 percentage points.
Seniors have a large jump in mean score (66% compared to 59% for combined other classes.)
Locating and retrieving information are the most problematic.

Highlights from the Discussion of Individual Questions
Ms. Herschman shared some striking statistics, many of which caused gasps or nodding in agreement in the audience.

82% know how to narrow their results.
45.5% did not understand Boolean OR.
68% know how to change terms if zero results are returned.
62% could not identify a correct and complete journal article citation.
33% would not include a footnote if they copied a whole paragraph; 56% would not include one if they read an article and rephrased it; and 76% know to footnote a quoted sentence from an article.
72% did not think that the subject was discernable from the call number.
78% said searching the web would be best for authoritative information.

Recommendations
The library must share with the faculty these findings and initiate a systematic program of Information Literacy education. This includes establishing basic competencies, adopting the Information Literacy Competency Standards for Higher Education.

For a future survey, the committee hopes to test incoming freshmen as a group with a re-designed survey instrument as well as test students at opposite ends of independent library research spectrum.

Submitted 2/7/01 by Judith Downie, jdownie@csusm.edu , CSU San Marcos