- Welcome and Announcements
Mr. Edward L. Flippen, Commission Chairman, welcomed members and introduced Ms. Brenda Atkins, Administrative and Finance Special Assistant to the President.
Ms. Atkins welcomed Commission members and guests to Longwood College.
- The Potential of Student Outcomes
Dr. Karl Schilling, Deputy Director, Policy, State Council of Higher Education for Virginia, presented his views on student outcome assessment including the following points:
- A standardized assessment plan cannot apply to all institutions unless it uses multiple approaches.
- Quantitative approaches do not allow for misinterpretation of the question.
- Standardized tests often do not connect to curriculum.
- The goal of descriptive, qualitative approaches is to understand what is already being done.
- Two reasons for doing assessment in higher education:
- To provide information that will enable campuses to improve the quality of educational experiences being provided to students.
- To create evidence that will allow the public to see the value-added to individuals and society by higher education.
- Assessment can be used to create a culture of evidence
- Need exists to value "evidence" over opinion, anecdote, or beliefs.
- Campus and system decision-making needs to demonstrate that assessment results are having a strong influence on resource allocation.
- Need exists to move beyond collecting assessment results to using them for educational improvement.
- How some institutions have addressed assessment:
- assessment embedded into the grading process, Grand Valley State, English Department
- assessment as part of the institutional fabric, Alverno College
- assessment as a collective effort, Truman State University
- assessment as internal accountability, SUNY, Fredonia
- assessment as scholarship, Harvard University
- assessment as student skill development, Hope College and Union College
- assessment as external accountability, Tennessee Higher Education System
- assessment as a faculty role, Draft Virginia Plan for Higher Education
- assessment as a stimulus to reflective practice, Miami University
- Characteristics of effective assessment efforts:
- Begin with a question or a statement of desired student learning outcomes.
- Utilize professional judgement - quantitative and qualitative.
- Look at students' entry and exit characteristics/abilities.
- Use existing data.
- Make assessment continuous and ongoing.
- Create a culture of evidence.
- Use multiple approaches.
- Engage faculty in all aspects - problem identification, design, data collection.
- Dilemmas in assessing teaching and learning:
- I can't know what you don't know.
- The educator's existential dilemma - I can teach you but I can't learn you.
- I can only get proxy measures of learning.
- A National Perspective
Dr. Thomas Dary Erwin, Director of Student Assessment and Professor of Psychology, James Madison University, presented remarks on the National Picture of Assessment in Higher Education and James Madison University's Assessment Program and included the following points:
- Demands for accountability are increasing because of:
- spiraling costs of attendance and operation.
- accrediting agency requirements.
- alarm among corporate and government leaders about global economic competitiveness.
- employers' dissatisfaction with new employee skills of recent graduates of baccalaureate programs.
- shifting financial priorities.
- criticism from inside and outside of the academy.
- 21 states have assessment policies including Virginia.
- Two questions frame the process:
- What do you want to describe?
- What group is to be assessed? (individual class; institutional program; system, i.e. Virginia)
- Performance indicators: outcomes (i.e. learning) must be differentiated from outputs (i.e. graduation rates)
- Steps for student outcome assessment
- establishing objectives
- selecting/designing instruments
- collecting information
- analyzing information
- maintaining information
- using information
- Some uses of assessment results:
- changes in course content
- changes in degree requirements
- changes in delivery of instruction
- points to areas needed in faculty hiring
- used in allocating faculty positions
- used in JMU program reviews
- used by external accreditation groups
- Dr. Erwin engaged in discussion with Commissioners:
Commissioner - Could there be an outside assessor? Now the university assesses by conferring the degree, especially in general education.
Dr. Erwin - The accrediting system provides the assessment.
Commissioner - You are in a better position to be the assessor, but can't you work with the end-product user?
Dr. Erwin - Let's look at the integrity of the degree.
4. An Inspirational Model
Dr. Georgine Loacker, Director of Assessment and Professor of English, Alverno College, Milwaukee, Wisconsin, refuted the "Inspirational Model" titling her presentation and said the model is more "down-to-earth" than inspirational. She also emphasized that Alverno's model works for Alverno, and other institutions should create models useful to them for the greatest benefit.
Dr. Loacker presented the Alverno College model for assessment including the following points:
- Two questions framed Alverno's process:
- What principles inform assessment?
- What structures support assessment?
- Alverno identified 8 institutional outcomes or abilities and developmental levels of competence, demonstration of which became graduation requirements:
- Develop communication abilities by connecting with everything involved in communication: people, ideas, texts, media, and technology.
- Develop analytical abilities.
- Develop facility in using problem solving processes.
- Develop facility in making value judgements and independent decisions.
- Develop facility for social interaction.
- Develop global perspectives.
- Develop effective citizenship.
- Develop aesthetic responsiveness: involvement with the arts.
- Content is integrated with abilities.
- Important basic assumptions:
- Education goes beyond knowing to doing what one knows.
- Assessment is an integral part of learning.
- Educators are RESPONSIBLE for making learning more available by ARTICULATING OUTCOMES and making them PUBLIC.
- Multiple performances are assessed in different contexts.
- Outside, community-based, assessors are involved.
- Alverno assessment definitions:
- student assessment-as-learning
- institutional or program assessment
- continuous improvement
- Distinctions:
- Leadership can come from a state agency.
- Development must come out of the faculty.
- LUNCH - speaker, Dr. Patricia Cormier, President, Longwood College
- Grades: For What They're Worth, A Presentation on Grade Inflation
Dr. Brad Wilson, Executive Director, National Association of Scholars, presented
his views on grading including the following points:
- There is substantial and credible evidence that grades have been inflating over a 30-year period.
- The percentage of Cs and As received reversed from 1969 to 1993.
- Evidence suggests that grade inflation is a phenomenon of the majority of colleges and universities across the array of American higher education.
- Grade inflation has proceeded more rapidly in the humanities than in the natural sciences, perhaps serving as an incentive for students to avoid the sciences.
- Over the period when grades were inflating most rapidly (1965 - 1980), average SAT, ACT, and GRE scores were in decline.
- Factors contributing to the upward movement of college grades:
- Erosion of commitment among humanistic disciplines to objective scholarship.
- Replacement of the authority of student wants for the authority of professional judgement.
- What can be done about grade inflation?
- List the average grade for all students enrolled in each class on student transcripts (Eastern Kentucky University).
- Require schools and departments to review their grading practices to bring consistency and rigor to the process (Princeton University).
- Suggestions to administrators:
- Tighten the curriculum to eliminate courses that survive because of their low standards.
- Eliminate lax drop policies that allow students to shop for courses with low demands.
- Revise the system of student evaluation of faculty to focus exclusively on academic content of the course and academic seriousness of the teacher.
- Give no quarter to student complaints about low grades, and decline to put tough graders in the dock.
- Dispense academic honors to the very top students (10%).
- Open discussion with the board of visitors, state legislature, and SCHEV about the possibility of over-enrollment. Given grade inflation, one can reasonably estimate that at least 15% of students graduating from American colleges would not have met minimal retention standards in the mid-1960s.
- Eliminate extra-curricular programs that compete with the academic mission of the university.
Comments from Commissioners:
Dr. Mikalson: Consider the victims of grade inflation; it hurts very good students and very bad students. The University of Virginia addressed this topic in a day-long symposium; students were the most outspoken against grade inflation.
Mr. Rainey: Employers are dramatically opposed to grade inflation.
- The History of Assessment in Virginia
Dr. Donna R. Brodd, Associate Director for Academic Affairs, State Council of Higher Education for Virginia,
presented an overview of A Chronology of Assessment Initiatives in Virginia, including the following points:
- The earliest assessment formalized by funding in Virginia was a 1985 award of $125,000 to James Madison University.
- 1984-1985 Senate Joint Resolution 125 asked SCHEV to study measurement of student achievement.
- 1986 Senate Joint Resolution 86 asked public institutions to initiate assessment programs.
- 1986 incentive funding, Funds for Excellence, focused on assessment.
- 1987 institutional assessment plans approved for implementation were published in the Virginia Plan for Higher Education.
- 1987 - the purpose of assessment was defined as improving student learning and performance. Additionally, nine guidelines were delineated:
- Student evaluation should be appropriate to the institution and allow for diversity of program goals.
- Some existing data may be appropriate for assessment purposes.
- Assessment should be cost effective in terms of student time.
- Assessment should be conducted at intervals during and after the college experience and include general education and majors.
- Institutions should identify minimal verbal and quantitative skills and procedures to identify students requiring remediation.
- Institutions should identify plans to measure impact of remediation.
- Progress of graduates of Virginia high schools will be reported to SCHEV for distribution to school divisions.
- Progress of transfer students from Virginia community colleges will be reported to SCHEV for distribution to appropriate institutions.
- Institutions should self-evaluate assessment procedures regularly.
- 1988-1989 - assessment was added to SCHEV duties.
- 1990-1991 - assessment plans were required for new program proposals and guidelines were adopted for off-campus instruction. Prior to this, the focus of assessment had been undergraduate programs.
- 1994 - JLARC study reported the positive impact of assessment on institutional effectiveness.
- 1996 - assessment reporting guidelines were revised allowing institutions greater latitude in selecting the focus for reporting.
- 1999 - SCHEV is considering a proposal to decentralize program approval for academic degrees, holding institutions accountable for enhancement of assessment programs and emphasizing continuous improvement.
- Panel on Institutional Approaches to Measuring Student Outcomes
Dr. Jon D. Mikalson, Professor of Classics, University of Virginia, moderated the panel discussion.
Dr. Gary A. Kreps, Associate Provost for Academic Affairs and Graduate Education, College of William and Mary, addressed the topic: Assessment as Part of the Institutional Fabric. His remarks included the following points:
- Assessment requires structure and the establishment of goals.
- Results of such efforts are something between the former lockstep and more recent area sequence system.
- The process became dynamic with faculty seriously and intellectually engaged.
- A new curriculum was passed but then it had to be assessed.
- Working groups were established for each general education requirement.
- An assessment steering committee studied portfolio options and survey analyses from faculty and students.
Dr. Karen Gentemann, Director of Institutional Assessment, George Mason University, addressed the topic: Assessment as an Accountability Tool. Her remarks about Academic Program Review at George Mason University included the following points:
- Accountability to eternal agencies:
- Southern Association for Colleges and Schools
- Specialized accrediting agencies for programs like nursing, business, engineering
- SCHEV
- Academic Program Review based on a culture of evidence and measurable goals:
- All units more outcome based; new president; new academic leadership; quality improvement.
- Five-year reviews characterized by movement beyond program audit with its emphasis on inputs; use of outcomes-oriented data; reflection on curricular and pedagogical issues (instructional practices, learning outcomes, program integration).
- Looking for evidence of explicit goals for student learning, goals evident in the curriculum, coherence in the curriculum, connections with scholarly inquiry, high quality teaching, effective advising, relationships to other disciplines and fields and to the Institutional Mission.
- Sources of evidence for students are quantitative and qualitative from multiple sources, direct and indirect measures of learning like portfolios, juried exhibits, student publishing records, external awards, and student satisfaction.
- Sources of evidence for faculty include research, presentations, publications relevant to teaching/learning; advising guidelines and training; and informal gatherings of faculty and majors.
- Feedback for improvement should be based on the review process and include a review of the review.
- Assessment is: a valuable tool for both internal improvement and external accountability, a sound educational practice, reflects educational values with emphasis on teaching and learning, is labor intensive and time consuming. "What gets measured gets better."
Dr. Edward D. Smith, Director of Assessment and Institutional Research, Longwood College, presented Assessment to Improve Learning including the following points:
- Using the Major Field of Achievement Test for Business and Economics majors, Longwood moved from the 35th percentile in 1989 to the 96th percentile in 1998.
- This improvement was accomplished through intentional continuous improvement including:
- Hired Dean Berkwood Farmer in 1991.
- Established a clear goal for AACSB accreditation.
- Engaged the participation of faculty.
- Used the mission as the driving force.
- Assessed progress toward measurable objectives.
- Changed the culture.
Dr. Sharon N. Robertson, Associate Dean, Curriculum and Enrollment Services, Northern Virginia Community College, presented Assessment as an Opportunity for Teaching including the following points:
- Student outcomes assessment is primarily used to improve teaching methods, curriculum, student support services, facilities and equipment.
- Elements assessed by community colleges: placement, developmental studies, computer competencies, general education, programs, transfer preparation to four-year institutions, retention, graduation rates, dual credit, distance learning, off-campus instruction, instructional technology (easy to show usage but difficult to show impact), and special topics like remediation.
- Tracking studies show proper placement, effect of cut-off score adjustments, success in developmental studies, effectiveness of remedial education, appropriateness of prerequisites, and graduation rates.
- Community colleges need clarification of who is to be assessed. Flexibility is required because of uniqueness of the colleges and the students.
Dr. W. E. (Ned) Moomaw, Institutional Assessment and Studies, University of Virginia, presented Assessment to Measure Success including the following points:
- Dr. Moomaw suggested a model for institutional assessment moving from ultimate outcomes and working back to adjust the educational environment.
- Outputs should serve as indicators of basic skills, i.e. 50% of UVA graduates go on to graduate school.
- Student inputs are impacted by the educational environment and by student involvement and effort, resulting in student outcomes.
- Examples of ultimate outcomes include: preparation for first and current jobs, satisfaction with first and current jobs, relationship of major to first and current job.
Dr. Mikalson moderated discussion:
- Adjournment
Mr. Edward L. Flippen adjourned the meeting at 4:40 p.m.
Back to the Main Page