CP Logo

The University of Texas at Austin College Portrait

Home Compare At A Glance Contact

EXPLORE THIS COLLEGE PORTRAIT

The University of Texas at Austin Learning Outcomes

 

In addition to administering the Collegiate Learning Assessment, The University of Texas at Austin has several structures and processes in place to support outcomes assessment. The institution requires that every academic program and many student support services units conduct outcomes assessment. For the 2011-2012 academic year, UT Austin lists 322 undergraduate and graduate programs. Across those programs, there were 1,128 learning outcomes that articulated what students were able to do or demonstrate as a result of their studies. Collectively, programs identified 3,479 methods to gather data on student achievement, resulting in 3,293 results, and 3,101 actions. Results are shared with deans and vice-presidents, who then distribute the report to associate deans, faculty, and staff. The data are housed on an on-line knowledge management system, and the majority of programs assess all of their outcomes each academic year. The Office of Institutional Accreditation and Program Assessment (IAPA)  supports assessment activities. More information on institutional initiatives and services can be found at:   http://www.utexas.edu/provost/planning/assessment/iapa/.

IAPA provides feedback on every assessment plan and has initiated use of an Institutional Effectiveness rubric. In addition to scoring the mechanics of each assessment plan (whether minimum implementation criteria are met), the rubric also focuses on assessment maturity, that is, gauging refinement of communication, feedback, and change processes. An example reflection question:  What processes do you have in place to ensure that annual assessment information is used to make curricular decisions? The feedback and rubric establish two-way communication between the programs and IAPA staff, helping to ensure continual improvement.

Here’s an example of improvement based on assessment even when students are meeting standards. In one program, students are expected to be able to effectively communicate concepts and ideas related to urban processes, through written and graphical media, to a broad and diverse group of stakeholders. Students’ skills are assessed through a semester-long, independent research project, completed during their senior year. Students are asked to produce a written document, as well as maps, diagrams, and other visual communications appropriate to their study and developed through original data acquisition and analysis. The project is completed individually, in consultation with a select faculty member who has expertise in an area related to the project. The goal or criterion for success is to have 90% of students demonstrate fair, good or excellent written and graphic communication skills on their senior projects. The research projects are measured on a scale of: excellent, good, fair, below average, and poor. During the 2009-2010 and 2011-2012 academic years, 44 senior projects were completed and assessed. Of those, 90% achieved the standard for written communication skills. Of those who used graphics, the percentage who demonstrated fair, good or excellent graphic skills was 100%. According to these results, a majority of students are meeting the standard for written and graphic communication. Nonetheless, we would like for students to exceed rather than merely meet the benchmark. In order to continue to improve students’ communication skills, a new component will be added to the senior project in the fall of 2013. Students will participate in a writing workshop covering the process of writing a major research paper, as well as common issues with style and organization. They also will be given opportunities during the workshop for peer review and group discussion of their projects.

And here’s an example of program change stemming from students not meeting standards. One of the learning outcomes for a B.S. degree plan is the ability to apply mathematics in the context of engineering. In order to assess this outcome, the program faculty and department chair host a yearly focus group with 10-15 seniors. During the 2011 – 2012 focus group, the faculty focused on training in mathematics, which is critical to success in two classes that have relatively large failure rates. During the discussion, students said that preparation across preliminary courses taken in another department was uneven. They reported that some instructors in that department did a great job of preparing students and some did not. Therefore, the program created a plan to improve the preparation of students in calculus and differential equation mathematics courses. The program also is working to revise two classes to give students more opportunities to apply mathematics in the context of engineering to reinforce what they are learning in co-registered math classes.

Outcomes assessment also helps support the continual improvement of the University’s general education curriculum. In 2012, a new set of Basic Education Requirements went into effect, aligning with the Texas Higher Education Coordinating Board’s Core Objectives. In 2014, the state is implementing new Core Objectives. Both the BERs and the Core Objectives are assessed annually. A campus-wide committee comprised of 20 faculty and staff—the General Education Liaisons’ Council (GELC)—reviews BER and Core Objective assessment results, as well as standardized test scores like CLA, NSSE, and SERU, and makes recommendations for improvement. GELC works closely with IAPA and the School of Undergraduate Studies, which also contributes Signature Course Program and Flags Program assessment data, to align assessment and curricular change for general education.

Outcomes assessment supports curricular change in other efforts, as well. Within graduate studies, all programs report on their outcomes assessment as part of their required 7-year program reviews. The reports are circulated with select internal stakeholders and external peer reviewers before being submitted to the state. Additionally, 14 academic programs undergo professional accreditation or certification (e.g., pharmacy, law, and social work), and most share outcomes assessment results with professional accreditation agencies or associations.

UT Austin continues to refine and develop the structures and processes intended to support institutional effectiveness. We value strategies that advance educational, programmatic, and operational excellence in a way that is evidence based, locally defined, and efficiency driven.




The University of Texas at Austin administered the CLA in 2011 - 2012.

The University of Texas at Austin conducted a Value-added administration of the CLA in 2011 - 2012. The results are displayed below in the SLO Results tab.

For additional information on UT Austin’s process for administering CLA, please click on the Assessment Process Tab below. For information on the students included in the administration, please click the Students Tested Tab.

Why did you choose the CLA for your institutional assessment?

In 2004-05, The University of Texas System, at the iniative of the Regents, contracted with the Council for Aid to Education to conduct a Collegiate Learning Assessment (CLA) of each of the nine academic campuses with the UT System.  The System's intent was to institutionalize an assessment program that was based on the broad cognitive skills such as problem solving , analytical reasoning, critical thinking, and analytical writing and to promote quality, comparability, and information that support policy development on all of the System campuses.  The University of Texas at Austin began using the CLA in 2004, and students have been tested each year since.  At the institutional level, the purpose of the CLA is to give faculty and administrators information that they can use to improve student learning.  


Which The University of Texas at Austin students are assessed? When?

The Center for Teaching and Learning (CTL), specifically Student Testing Services (STS) manages the administration and enrollment of students into the CLA Exam, and has processes and criteria used to schedule these students.  The CLA is administered at the University of Texas to freshmen in the fall semester and to graduating seniors in the spring semester.  STS administers the exam to 200 students who meet the criteria for each of these sessions.  Criteria for a student to take the CLA include class (freshman standing or senior graduating within the calendar year), transfer status (examinees must start at UT-Austin as freshmen and not as transfer students), college in which the student is enrolled, and GPA.  


How are assessment data collected?

Information forthcoming.  


How are data reported within The University of Texas at Austin?

CLA data are used for UT Austin's general education assessment.  Every year, a sample of general education courses are assessed, as well as several institutional level instruments. CLA results pertain to three of UT Austin's general education objectives: communicate clearly & accurately, defend an idea on the basis of evidence, draw conclusions, and evaluate the arguments of others; have a critical understanding of how human cultures are expressed in literature, philosophy, or language; participate in the process of inquiry through research, creative endeavors, or related activities.   Results are reviewed annually by the Vice-Provost for Faculty Affairs and Institutional Accreditation, as well as the General Education Liaison's Council.  


How are assessment data at UT Austin used to guide program improvements?

As part of the general education assessment, CLA results are used in conjuction with other direct and indirect data sources, such as coursework, extra-curricular activities, and surveys. We use the multiple data sources to gain understanding of how students perform in competencies related to critical thinking and inquiry.  The Genera Education Liaison's Council analyzes results and writes recommendations for program improvement, which are shared with our School of Undergraduate Studies.  They, in turn, make recommendations to faculty involved in general education. 


Of 7754 freshmen students eligible to be tested, 200 (3%) were included in the tested sample at The University of Texas at Austin.


Of 8122 senior students eligible to be tested, 199 (2%) were included in the tested sample at The University of Texas at Austin.


Probability sampling, where a small randomly selected sample of a larger population can be used to estimate the learning gains in the entire population with statistical confidence, provides the foundation for campus-level student learning outcomes assessment at many institutions. It's important, however, to review the demographics of the tested sample of students to ensure that the proportion of students within a given group in the tested sample is close to the proportion of students in that group in the total population. Differences in proportions don't mean the results aren't valid, but they do mean that institutions need to use caution in interpreting the results for the groups that are under-represented in the tested sample.

Undergraduate Student Demographic Breakdown

  Freshmen Seniors
Eligible Students Tested Students Eligible Students Tested Students
Gender Female 54% 56% 49% 53%
Male 46% 45% 51% 47%
Other or Unknown <1% <1% <1% <1%
Race/
Ethnicity
US Underrepresented Minority 50% 56% 46% 53%
White / Caucasian 46% 37% 51% 45%
International 4% 7% 3% 3%
Unknown <1% 1% <1% <1%
Low-income (Eligible to receive a Federal Pell Grant) 26% 24% 24% 20%

The testing population of 200 examinees per administration reflects the populations of UT Austin's individual colleges proportunately; the mean GPA for each college is determined and reflected in the examinee population as well.  Total freshmen tested in the 2011-12 academic year numbered 198, with seniors examined numbering 198 as well.  

The VSA advises institutions to follow assessment publisher guidelines for determining the appropriate number of students to test. In the absence of publisher guidelines, the VSA provides sample size guidelines for institutions based on a 95% confidence interval and 5% margin of error. So long as the tested sample demographics represent the student body, this means we can be 95% certain that the "true" population learning outcomes are with +/- 5% of the reported results. For more information on Sampling, please refer to the Research Methods Knowledge Base

The increase in learning on the performance task is at an institution testing students of similar academic abilities.

The increase in learning on the analytic writing task is at an institution testing students of similar academic abilities.

Seniors Detail

The charts below show the proportion of tested seniors who scored at each level of the nine subscales that make up the CLA. The subscale scores range from 1 to 6 with 6 representing a higher or better score. Due to rounding, subscores may not total 100%.


Performance Task Make-an-Argument Critique-an-Argument
Analytic Reasoning and Evaluation
Writing Effectiveness
Writing Mechanics
Problem Solving

Freshmen Detail

The charts below show the proportion of tested freshmen who scored at each level of the nine subscales that make up the CLA. The subscale scores range from 1 to 6 with 6 representing a higher or better score. Due to rounding, subscores may not total 100%.


Performance Task Make-an-Argument Critique-an-Argument
Analytic Reasoning and Evaluation
Writing Effectiveness
Writing Mechanics
Problem Solving