TCNJ uses student learning outcomes assessment to make continuous improvements in all academic programs. Programs with national accreditation follow comprehensive yearly assessment plans to measure all accreditation standards. All other programs participate in the college’s One Question/One Answer assessment program (initiated in 2005) to engage in yearly assessment of at least one program-defined student learning outcome. All programs share the results of their assessment efforts in order to facilitate the development of a culture of assessment across campus. Our campus-wide Liberal Learning program has revised its comprehensive assessment plan which includes the first administration of the ETS Proficiency Profile in academic year 2010-2011.
The College of New Jersey administered the ETS Proficiency Profile in 2010 - 2012.
The College of New Jersey conducted a Value-added administration of the ETS Proficiency Profile in 2010 - 2012. The results are displayed below in the SLO Results tab.
For additional information on TCNJ’s process for administering ETS Proficiency Profile, please click on the Assessment Process Tab below. For information on the students included in the administration, please click the Students Tested Tab.
The College of New Jersey selected the ETS Proficiency Profile because of the ability to measure all general education skills using one test. The ETS Proficiency Profile provides useful data to the College for accredidation, strategic planning, benchmarking to external peers and national groups, and for addressing strengths and weaknesses in curricula. The use of the ETS Proficiency Profile gives the institution an opportunity to make external comparisons and internal longitudunal comparisions for assessment purposes. The instrument also has high reliability and validity.
The College of New Jersey plans to administer the ETS Proficiency Profile to a representative sample of freshmen and seniors on a 3-year cycle. Freshmen are assessed during the fall term and seniors complete the instrument during the spring term of the same academic year.
The ETS Proficiency Profile was administered to a sample of 201 incoming freshmen students enrolled in First Year Seminar courses in fall 2010. Instruments were administered to entire classes of students in 16 different course sections with enrollment ranging from 8 to 18 (modal enrollment = 18). The courses selected were representative of the range of disciplines offered in First Year Seminar courses. Of the 201 instruments administered, 194 were complete and included for analysis. The average combined SAT score for this sample was 1235 (vs. 1243 for all incoming freshmen). The instrument was also administered to sample of 222 senior students enrolled in senior capstone courses in spring 2011. Instruments were administered to entire classes of students in 14 different course sections with enrollment ranging from 1 to 22 for all schools with the exception of Nursing and Engineering. The courses selected were proportionally representative of the enrollment by school. In Nursing and Engineering, all graduating seniors were given the instrument and a random sample in proportion to enrollment was included in the overall senior sample. Of the 222 instruments included in the senior sample, 215 were complete and included for analysis. The average combined SAT score for this sample was 1219 (vs. 1213 for all seniors).
At TCNJ the mean scores of the incoming freshmen exceeded the scores of 100% of the institutions who have taken the ETS Proficiency Profile in both critical thinking and writing from January 2006 through June 2010. The mean scores of the senior students exceeded 93% of all institutions in critical thinking and 97% of all institutions in writing. The percent of students classified as “Proficient” in critical thinking increased from 6% to 10% from the freshmen sample to the senior sample. The percent of students classified as “Proficient” in writing increased from 38% to 50% at level 2 and from 23% to 24% at level 3. Increases were found despite the fact that the average SAT score for the senior sample (1219) was significantly lower than the average SAT for the freshman sample (1235).
Results from the ETS Proficiency Profile are used in strategic planning for the College and for program accredidation purposes. After administering the ETS Proficiency Profile for a second time, longitudinal data will be used for curriculum analysis and discussion to support assessment and strategic planning.
Of 1421 freshmen students eligible to be tested, 203 (14%) were included in the tested sample at The College of New Jersey.
Of 909 senior students eligible to be tested, 292 (32%) were included in the tested sample at The College of New Jersey.
Probability sampling, where a small randomly selected sample of a larger population can be used to estimate the learning gains in the entire population with statistical confidence, provides the foundation for campus-level student learning outcomes assessment at many institutions. It's important, however, to review the demographics of the tested sample of students to ensure that the proportion of students within a given group in the tested sample is close to the proportion of students in that group in the total population. Differences in proportions don't mean the results aren't valid, but they do mean that institutions need to use caution in interpreting the results for the groups that are under-represented in the tested sample.
Undergraduate Student Demographic Breakdown
|Eligible Students||Tested Students||Eligible Students||Tested Students|
|Other or Unknown||<1%||<1%||<1%||<1%|
|US Underrepresented Minority||29%||32%||22%||22%|
|White / Caucasian||66%||62%||69%||69%|
|Low-income (Eligible to receive a Federal Pell Grant)||17%||2%||14%||12%|
Freshmen participated as a result of their enrollment in a First-Year seminar course at the College. The courses selected for participation were representative of the range of disciplines offered in the First-Year Seminar program. Seniors participated as a result of their enollment in a senior capstone course. The capstone courses selected were proportionally representative of senior enrollment in the College's seven schools.
The VSA advises institutions to follow assessment publisher guidelines for determining the appropriate number of students to test. In the absence of publisher guidelines, the VSA provides sample size guidelines for institutions based on a 95% confidence interval and 5% margin of error. So long as the tested sample demographics represent the student body, this means we can be 95% certain that the "true" population learning outcomes are with +/- 5% of the reported results. For more information on Sampling, please refer to the Research Methods Knowledge Base
The increase in learning on the performance task is at or near what would be expected at an institution testing students of similar academic abilities.
The increase in learning on the analytic writing task is at or near what would be expected at an institution testing students of similar academic abilities.
The chart below shows the distribution of student scores on the ETS Proficiency Profile Critical Thinking test. Students are scored as Not Proficient, Marginal, or Proficient.
The charts below show the distribution of student scores on the three levels of the ETS Proficiency Profile Writing Test. Students are scored as Not Proficient, Marginal, or Proficient on each level. Writing 3 represents more advanced writing skill than Writing 2, which represents more advanced writing skill than Writing 1.