Accreditation

Glossary of Assessment Terms

A

Alignment

The connection between learning objectives, learning activities, and assessment. An aligned course means that the learning objectives, activities, and assessments match up so ÅÝܽÊÓƵapps learn what is intended and that what they're learning is accurately assessed (Maki, 2004; O'Reilly; Suskie, 2010)

Assessment

The systematic, reported evaluation of ÅÝܽÊÓƵapp learning outcomes for demonstrating effectiveness and improving offerings (Harvey, 2004; McTighe & Ferrera, 1998; Morante, 2003; Suskie, 2010).

Academic program assessment consists of a systematic, collection, review, and use of information about educational programs undertaken for the purpose of improving ÅÝܽÊÓƵapp learning and development. It is a continuous process focused on understanding and improving ÅÝܽÊÓƵapp learning. It involves making our expectations explicit and public; setting appropriate criteria and standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance (Angelo, 1995; Palomba & Banta, 1999).

Also see:  

 

Assessment for Accountability

The assessment of some units, such as a department, program or entire institution, which is used to satisfy some group of external stakeholders. Stakeholders may include accreditation agencies, state government, or trustees. Results are often compared across similar units, such as other similar programs and are always summative (Leskes, 2002; Maki, 2004; Suskie, 2010).

An example of assessment for accountability would be ABET accreditation in engineering schools, whereby ABET creates a set of standards that must be met in order for an engineering school to receive ABET accreditation status.

Assessment for Improvement

Assessment activities designed to feed the results directly, and ideally, immediately, back into revising the course, program, or institution with the goal of improving ÅÝܽÊÓƵapp learning. Both formative and summative assessment may be used to guide improvements (Leskes, 2002; Suskie, 2010).

Assessment Instrument

Specific tool used to collect information about ÅÝܽÊÓƵapps' knowledge or performance. For example standardized exam, performance rubric, portfolio of papers or projects, etc. (Rice ÅÝܽÊÓƵapp).

Also see:

Assessment Method

Comprehensive description of how a learning outcome is going to be assessed. It includes descriptions of the assessment instrument, the population to be assessed, and at what point in time, as well as how reliability and validity will be addressed (Rice ÅÝܽÊÓƵapp).

Also see:

Assessment Plan

Document that includes the following sections: Mission Statement, Learning Outcome for academic program (or unit outcome for support units), Assessment Method, and Performance Targets (Morante, 2003; Rice ÅÝܽÊÓƵapp).

Also see:

Assessment Report

A document that includes the assessment plan with the addition of the assessment results and the use of assessment results. The results are the actual data stated in terms of the performance criteria. Example: 80% of ÅÝܽÊÓƵapps scored a 4.5 or higher (N=12); 13% of ÅÝܽÊÓƵapps scored a 3.4 to 4.4 (N=2); and 7% of ÅÝܽÊÓƵapps scored 2.5 to 3.4 (N=1) (USF, 2018).

The use of assessment results section is where the interpretation and analysis of the results is stated. It should include specific, actionable "next steps" the program will take to develop or improve on a programmatic level (e.g., curriculum mapping, revisit or revise the assessment method/rubric, revisions to plan of study/curricular offerings, development of new modules/courses, faculty development, etc.) (Maki, 2004; Morante, 2003; Suskie, 2010; USF, 2018).

Also see:

Benchmarking

An actual measurement of group performance against an established standard at defined points along the path toward the standard. Subsequent measurements of group performance use the benchmarks to measure progress toward achievement (Morante, 2003; New Horizons for Learning; Suskie, 2010)

Calibration

The process of establishing inter-rater reliability. Developed over successive applications of a scoring rubric to ÅÝܽÊÓƵapp work over an undefined period (i.e. semesters, years) (Maki, 2004).

Capstone Experience/Student Teaching

An activity for graduating seniors that is designed to demonstrate comprehensive learning in the major through some type of product or performance (Morante, 2003; Palomba & Banta, 1999; Suskie, 2010).

Case Studies

Detailed analyses of projects or problems that result in exemplary models. Counseling and School Psychology programs ask ÅÝܽÊÓƵapps to generate a course of action for dealing with a client based upon a comprehensive profile of that client (Maki, 2004; Morante, 2003; Rice ÅÝܽÊÓƵapp).

Classroom Assessment

The systematic and on-going study of what and how ÅÝܽÊÓƵapps are learning in a particular classroom; often designed for individual faculty who wish to improve their teaching of a specific course. Classroom assessment differs from tests and other forms of ÅÝܽÊÓƵapp assessment in that it is targeted at course improvement, rather than at assigning grades (Morante, 2003; National Teaching and Learning Forum).

Competence

The individual's demonstrated capacity to perform, i.e., the possession of knowledge, skills, and personal characteristics needed to satisfy the special demands or requirements of a particular situation (Clark College; DeMars et al., 2002).

Competency

A combination of skills, ability and knowledge needed to perform a specific task at a specified criterion. Usually a near-term target of a larger expected outcome. Students must normally learn and/or demonstrate several competencies for each ÅÝܽÊÓƵapp learning outcome (Clark College; Morante, 2003; Suskie, 2010).

Concept Map

Graphical representation used to reveal how ÅÝܽÊÓƵapps organize their knowledge about a concept or process. May include concepts, usually represented in enclosed circles or boxes, and relationships between concepts, indicated by a line connecting two concepts (Novak & Cañas, 2006).

Curriculum Mapping

Process of evaluating curriculum in relation to intended outcomes to ensure that ÅÝܽÊÓƵapps are receiving appropriate instruction and to enable the program/department to identify gaps in the curriculum and provide an overview of program accomplishments (Rice ÅÝܽÊÓƵapp; Suskie, 2010).

Demonstrations

Performances that show skills ÅÝܽÊÓƵapps have mastered. Theater ÅÝܽÊÓƵapps may be assessed on their acting skills demonstrated in theatrical productions (Palomba & Banta, 1999).

Direct Measures of Assessment

Direct Measures of Assessment: (see also Indirect Measures of Assessment): Direct measures of assessment gather evidence about ÅÝܽÊÓƵapp learning based on ÅÝܽÊÓƵapp performance that demonstrates the learning itself. They can be value added, related to standards, qualitative or quantitative, embedded or not, using local or external criteria (Eder, 2004; Leskes, 2002; Maki, 2004; Morante, 2003; Suskie, 2010).

Examples of direct measures of ÅÝܽÊÓƵapp learning include:

  • Written assignments
  • Presentations
  • Portfolios
  • Comparisons between pre and post-test scores (this does not have to be the exact same test as long as one is testing the same area of knowledge or skills)
  • Class assignments or tests
  • Capstone projects, including theses, field studies and papers
  • Case studies
  • Team/group projects and presentations
  • Field supervisor evaluations from internships and field experience
  • Service-learning projects
  • Ratings by employers of the level of skills and knowledge possessed by our graduates
  • Student performance on standardized tests, such as licensure exams or the ETS content tests

E

Embedded Assessment/Course-embedded Assessment

A means of gathering information about ÅÝܽÊÓƵapp learning that is built into and a natural part of the teaching-learning process. Classroom assignments that are evaluated to assign ÅÝܽÊÓƵapps a grade are often used for assessment purposes. Embedded assessment can assess individual ÅÝܽÊÓƵapp performance or aggregate the information to provide information about the course or program; can be formative or summative, quantitative or qualitative (Maki, 2004; Morante, 2003; Suskie, 2010).

Example: As part of a course, expecting each senior to complete a research paper that is graded for content and style, but is also assessed for advanced ability to locate and evaluate web-based information (as part of a college-wide outcome to demonstrate information literacy) (Leskes, 2002).

Evaluation

The use of assessment findings (evidence/data) to judge program effectiveness; used as a basis for making decisions about program changes or improvement (Allen, Noel, Rienzi, & McMillin, 2002).

Exams

Standardized tests that may include multiple-choice, true-false, response selection, essays and/or problems (ÅÝܽÊÓƵapp of Nebraska at Kearney).

  • National Exams are usually produced by a professional organization, such as the American Chemical Society, or by a commercial company, such as ETS (Educational Testing Service). National exams are usually nationally normed so there is feedback about the relative accomplishment of participants.
  • Local exams are produced by the department or institution using them. Usually careful attention is given to reliability and validity and data are kept to compare current participants with past groups of participants.

F

Formative Assessment

A planned classroom practice to elicit evidence of learning minute to minute, day by day in the classroom. Purpose: inform teachers of what ÅÝܽÊÓƵapps know or do not know, help ÅÝܽÊÓƵapps understand what it is they are ready to learn next, so teachers can adjust their instruction accordingly for each of their ÅÝܽÊÓƵapps (Goodrich, 2012; Harvey, 2004; Maki, 2004; Suskie, 2010).

Formative assessment refers to the gathering of information or data about ÅÝܽÊÓƵapp learning during a course or program that is used to guide improvements in teaching and learning. Formative assessment activities are usually low stakes or no-stakes; they do not contribute substantially to the final evaluation or grade of the ÅÝܽÊÓƵapp or may not even be assessed at the individual ÅÝܽÊÓƵapp level (Leskes, 2002; Maki, 2004; McTighe & Ferrera, 1998).

For example, posing a question in class and asking for a show of hands in support of different response options would be a formative assessment at the class level. Observing how many ÅÝܽÊÓƵapps responded incorrectly would be used to guide further teaching (Leskes, 2002).

Also see:

I

Indirect Measures of Assessment

Indirect Measures of Assessment: (see also Direct Measures of Assessment): Uses perceptions, reflections, or secondary evidence to make inferences about ÅÝܽÊÓƵapp learning (Eder, 2004; Leskes, 2002; Maki, 2004; Morante, 2003; Suskie, 2010).

Examples of indirect measures of ÅÝܽÊÓƵapp learning include:
Course grades
Surveys of employers
Student self-assessments
Number of alumni admitted to graduate school
Number of ÅÝܽÊÓƵapps presenting or publishing research
Honors, awards, scholarships, and other forms of public recognition earned by ÅÝܽÊÓƵapps and alumni

Institutional Effectiveness

The extent to which an institution achieves its mission and goals. Institutional Effectiveness is monitored through ongoing, integrated, institution-wide research-based planning and evaluation processes that 1) incorporate a systematic review of institutional mission, goals, and outcomes; 2) result in continuing improvement of institutional quality; and 3) demonstrate the institution is effectively accomplishing its mission (Morante, 2003; SACSCOC, 2006).

Inter-rater reliability

Inter-rater reliability: "the degree to which different individual observers or graders agree in their scoring" (Maki, 2004, p. 93). It serves to reach common calibrations among scorers. The process ensures that scorers' ratings are reliable across different samples and samples from representative ÅÝܽÊÓƵapp populations (Maki, 2004).

L

Learning_Activities

The specific design and implementation of learning opportunities (inquiry, exploration, discovery, listening, observation, reading, writing, planning, discussion, practice, experimentation) that lead to the desired learning outcomes (Clark College).

Learning Outcomes Assessment

The measurement of learning outcomes. Learning Outcomes Assessment examines ÅÝܽÊÓƵapp demonstrations of the results of learning. The process includes four cyclical steps: 1) teaching and learning, 2) developing questions and gathering information about ÅÝܽÊÓƵapp learning, 3) analyzing the information and drawing conclusions, and 4) reflecting and planning.

It documents the alignment (or dissonance) between the intended learning (as stated in the outcomes) and the actual learning (as demonstrated by the ÅÝܽÊÓƵapp). The practice of Learning Outcomes Assessment is collaborative and is intended to inform. Its goal is to continually improve ÅÝܽÊÓƵapp learning (Clark College).

Also see: 

 

Learning Outcomes/Student Learning Outcomes/Objectives

Learning outcomes are operational statements describing specific ÅÝܽÊÓƵapp behaviors that evidence the acquisition of desired knowledge, skills, abilities, capacities, attitudes, or dispositions. Learning outcomes can be usefully thought of as behavioral criteria for determining whether ÅÝܽÊÓƵapps are achieving the educational objectives of a program, and, ultimately, whether overall program goals are being successfully met Harvey, 2004; Maki, 2004; Morante, 2003, Suskie, 2010).

Outcomes are sometimes treated as synonymous with objectives, though objectives are usually more general statements of what ÅÝܽÊÓƵapps are expected to achieve in an academic program (Allen, Noel, Rienzi, & McMillin, 2002).

A learning outcome is a specific, observable, and measurable knowledge or skill that the ÅÝܽÊÓƵapp gains/develops as a result of a specific course. Thus, learning outcomes are clearly stated in the course syllabus.

There are three categories of ÅÝܽÊÓƵapp learning outcomes.

  1. Cognitive outcome/declarative knowledge: What ÅÝܽÊÓƵapps KNOW; knowledge, comprehension, application, analysis, synthesis, and evaluation.
  2. Affective outcome/attitudes, values, or habits of mind: What ÅÝܽÊÓƵapps CARE about; ÅÝܽÊÓƵapps' feelings, attitudes, interests, and preferences.
  3. Performance outcome/procedural knowledge: What ÅÝܽÊÓƵapps CAN DO; skilled performance, production of something new (e.g., a paper, project, piece of artwork), critical thinking skills (e.g., analysis and evaluation). (Allen, Noel, Rienzi, & McMillin, 2002; McTighe & Ferrera, 1998).

M

Mission Statement

The assertion of an institution's or unit's fundamental aspirations. Mission statements generally describe an organization's purpose, role, and scope (Gardiner, Anderson, & Cambridge, 1997).

N

Norm-Referenced Assessment

An assessment where ÅÝܽÊÓƵapp performance or performances are compared to a larger group. Usually the larger group or "norm group" is a national sample representing a wide and diverse cross-section of ÅÝܽÊÓƵapps. Students, schools, districts, and even states are compared or rank-ordered in relation to the norm group. The purpose of a norm-referenced assessment is usually to sort ÅÝܽÊÓƵapps and not to measure achievement towards some criterion of performance (Harvey, 2004; Maki, 2004; McTighe & Ferrera, 1998; American Public ÅÝܽÊÓƵapp System).

O

Operationalization

Defining an outcome or objective in a way that facilitates measurements (Rice ÅÝܽÊÓƵapp).

P

Performance Criteria/Performance Targets

Quantitative measures by which ÅÝܽÊÓƵapp performance is evaluated. Performance criteria help assessors maintain objectivity and provide ÅÝܽÊÓƵapps with important information about expectations, giving them a target or goal to which they may strive (New Horizons for Learning; Suskie, 2010).

Portfolio

A systematic and organized collection of a ÅÝܽÊÓƵapp's work that exhibits to others the direct evidence of a ÅÝܽÊÓƵapp's efforts, achievements, and progress over a period of time. The collection should involve the ÅÝܽÊÓƵapp in selection of its contents, and should include information about the performance criteria, the rubric or criteria for judging merit, and evidence of ÅÝܽÊÓƵapp self-reflection or evaluation (Maki, 2004; McTighe & Ferrera, 1998; Morante, 2003; Suskie, 2010).

It should include representative work, providing documentation of the learner's performance and a basis for evaluation of the ÅÝܽÊÓƵapp's progress. Portfolios may include a variety of demonstrations of learning and have been gathered in the form of a physical collection of materials, videos, CD-ROMs, reflective journals, etc. (New Horizons for Learning; Smith & Tillema, 2003).

Program Assessment

Uses the department or program as the level of analysis. Can be quantitative or qualitative, formative or summative, standards-based or value added, and used for improvement or for accountability. Ideally, program goals and objectives would serve as a basis for the assessment. Example: How well can senior engineering ÅÝܽÊÓƵapps apply engineering concepts and skills to solve an engineering problem? This may be assessed through a capstone project, by combining performance data from multiple senior level courses, collecting ratings from internship employers, etc. If a goal is to assess value added, some comparison of the performance to newly declared majors would be included (Leskes, 2002).

Also see:

Program outcome/program goal/program objective

The knowledge, skills, and abilities ÅÝܽÊÓƵapps should possess when they complete a program. Educational or degree programs are more than a collection of random courses. Educational programs prepare ÅÝܽÊÓƵapps for a range of particular outcomes that can be stated in measurable terms. Program assessment seeks to determine the extent to which ÅÝܽÊÓƵapps in the program can demonstrate these outcomes (George Mason ÅÝܽÊÓƵapp; Suskie, 2010).

Also see:  

Q

Qualitative Assessment

Collects data that does not lend itself to quantitative methods, but rather to interpretive criteria (Leskes, 2002; Maki, 2004; Suskie, 2010).

Quantitative Assessment

Collects data that can be analyzed using quantitative methods (see "assessment for accountability" for an example) (Leskes, 2002; Maki, 2004; Suskie, 2010).

R

Reliability

Capacity of an assessment method to produce consistent and repeatable results. A reliable assessment instrument or technique performs in a stable way over many uses at different times (Gardiner, Anderson, & Cambridge, 1997; Maki, 2004; McTighe & Ferrera, 1998; Morante, 2003).

Rubric

A rubric is a scoring tool that explicitly represents the performance expectations for an assignment or piece of work. A rubric divides the assigned work into component parts and provides clear descriptions of the characteristics of the work associated with each component, at varying levels of mastery. Rubrics can be used for a wide array of assignments: papers, projects, oral presentations, artistic performances, group projects, etc. Rubrics can be used as scoring or grading guides, to provide formative feedback to support and guide ongoing learning efforts, or both (Carnegie Mellon ÅÝܽÊÓƵapp; McTighe & Ferrera, 1998; Morante, 2003; Suskie, 2010).

Standards

Established level of accomplishment that all ÅÝܽÊÓƵapps are expected to meet or exceed. Standards do not imply standardization of a program or of testing. Performance or learning standards may be met through multiple pathways and demonstrated in various ways. For example, instruction designed to meet a standard for verbal foreign language competence may include classroom conversations, one-on-one interactions with a TA, of the use of computer software. Assessing competence may be done by carrying on a conversation about daily activities or a common scenario, such as eating in a restaurant, or using a standardized test, using a rubric or a grading key to score correct grammar and comprehensible pronunciation (Leskes, 2002; McTighe & Ferrera, 1998; Suskie, 2010).

Also see:

Strategic Planning

Long-term (i.e., often 3-5 year periods) planning at the level of the whole institution or unit that focuses on adaptation to the organization's external environment and the future. Guided by a vision of the organization in the future, strategic planning attempts to position the organization favorably with respect to needed resources (Gardiner, Anderson, & Cambridge, 1997)

Summative Assessment

The gathering of information at the conclusion of a course, program, or undergraduate career to improve learning or to meet accountability demands. When used for improvement, it impacts the next cohort of ÅÝܽÊÓƵapps taking the course or program. Example: examining ÅÝܽÊÓƵapp final exams in a course to see if certain specific areas of the curriculum were understood less than others (Leskes, 2002; Maki, 2004; McTighe & Ferrera, 1998; Suskie, 2010).

T

Triangulate/-tion

The use of a combination of assessment methods, such as using surveys, interviews, and observations to measure a unit outcome (Rice ÅÝܽÊÓƵapp).

U

Unit Outcome

Intended outcomes that reflect the area or service that can be improved using current resources and personnel and are assessable within one assessment cycle. Unit outcomes should be under the direct control of the unit and align with a university strategic plan goal, objective, and strategy. For administrative units, unit outcomes are primarily process-oriented, describing the support process/service the unit intends to address. For academic ÅÝܽÊÓƵapp support units, unit outcomes may include both process and ÅÝܽÊÓƵapp learning outcomes (Morante, 2003; Rice ÅÝܽÊÓƵapp).

Also see:

V

Validity

The extent to which an assessment method measures or assesses what it claims to measure or assess. A valid assessment instrument or technique produces results that can lead to valid inferences (Maki, 2004; McTighe & Ferrera, 1998; Morante, 2003).

Value Added

The increase in learning that occurs during a course, program, or undergraduate education. May either focus on the individual ÅÝܽÊÓƵapp (how much better a ÅÝܽÊÓƵapp can write, for example, at the end than at the beginning) or on a cohort of ÅÝܽÊÓƵapps (whether senior papers demonstrate more sophisticated writing skills-in the aggregate-than freshman papers). Requires a baseline measurement for comparison (Leskes, 2002; Morante, 2003; Suskie, 2010).

 References

References

References:

Allen, M., Noel, R.C., Rienzi, B.M., & McMillin, D.J. (2002). Outcomes Assessment Handbook. Long Beach, CA: California State ÅÝܽÊÓƵapp, Institute for Teaching and Learning.

American Public ÅÝܽÊÓƵapp System. (n.d.). Learning Outcomes Glossary. Retrieved from

Angelo, T. (1995). Reassessing (and defining) assessment. The AAHE Bulletin, 48(2), 7-9.

Bloom, B.S. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive Domain. White Plains. NY: Longman.

Carnegie Mellon ÅÝܽÊÓƵapp. (n.d.). Rubrics - Eberly Center - Carnegie Mellon ÅÝܽÊÓƵapp. Retrieved from

Clark College. (n.d.). Glossary of Assessment Terms. Retrieved from

DeMars, C.E., Cameron, L., & Erwin, T.D. (2002). Information literacy as foundational: determining competence. JGE: The Journal of General Education, 52(4), 253.

Eder, D.J. (2004). General education assessment within the disciplines. JGE: The Journal of General Education, 53(4), 135.

Gardiner, L. F.; Anderson, C.; & Cambridge, B. L. (1997). Learning through Assessment: A Resource Guide for Higher Education. Washington, DC: American Association for Higher Education.

George Mason ÅÝܽÊÓƵapp. (n.d.). Glossary of Assessment Terms. Retrieved from

Goodrich, K. (2012, July 23). What is formative assessment? Retrieved from

Harvey, L. (2004). Analytic Quality Glossary. Quality Research International. Retrieved from

Leskes, A. (2002). Beyond confusion: an assessment glossary. (Realty Check). Peer Review, 42.

Maki, P.L. (2004). Assessing for learning: building a sustainable commitment across the institution. Sterling, VA: AAHE.

McTighe, J., & Ferrara, S. (1998). Assessing learning in the classroom. Washington D.C.: National Education Association.

Morante, E. A. (2003). A Handbook on Outcomes Assessment for Two Year Colleges. Palm Springs, CA: College of the Desert.

New Horizons for Learning. (2002). Glossary of Assessment Terms.

Novak, J. D., & Cañas, A. J. (2006). The Theory of Underlying Concept Maps and How to Construct and Use Them. Pensacola,FL: Institute for Human and Machine Cognition.

O'Reilly, L. (n.d.). Assessment & Instructional Alignment: An Online Tutorial for Faculty. Retrieved July 5, 2018, from:

Palomba, C., & Banta, T. (1999). Assessment essentials: planning, implementing, and improving assessment in higher education. San Francisco, CA: Jossey Bass.

Rice ÅÝܽÊÓƵapp. (n.d.). Glossary of Assessment Terms. Retrieved from

SACSCOC (2006). Principles of Accreditation: Foundations for Quality Enchantment (2nd ed.). Decatur, GA.

Smith, K., & Tillema, H. (2003). Clarifying different types of portfolios. Assessment & Evaluation in Higher Education, 28(6), 625.

Suskie, L. (2010). Assessing ÅÝܽÊÓƵapp learning: A common sense guide. San Francisco, CA: Anker Publishing Company, Inc.

ÅÝܽÊÓƵapp of Nebraska at Kearney (n.d.) Glossary of Assessment Terms. Retrieved from

ÅÝܽÊÓƵapp (USF) (2018). Institutional Effectiveness System for Assessment Management User's Guide. Unpublished handbook, Office of Decision Support.

In addition to the primary sources above, this glossary draws from the collective expertise of the following secondary sources:

1. Assessment Glossary compiled by the American Public ÅÝܽÊÓƵapp System:

2. Glossary of Assessment Terms compiled by the Office of Institutional Effectiveness at Rice ÅÝܽÊÓƵapp:    

3. Common Assessment Terms compiled by the Eberly Center at Carnegie Mellon ÅÝܽÊÓƵapp:

4. Glossary of Assessment Terms compiled by the ÅÝܽÊÓƵapp of Nebraska at Kearney:

5. Glossary of Assessment terms from Clark College:

6. Glossary of Assessment Terms from George Mason ÅÝܽÊÓƵapp: