Facilitators: Allan H. Wheaton, Chief, Curriculum Division, U.S. Coast Guard Maritime Law Enforcement Agency, and James R. Parry, Test Development Manager, U.S. Coast Guard Training Center, Yorktown, VA

Title:
Test Defensibility in the US Coast Guard - Using the Angoff Method to Set Cut Scores

Background:

In order to distinguish between those who are competent to do the job and those who are not, trainers need to define the acceptable levels of competency as they relate to performance. The Angoff method uses a systematic and documented approach to establish defensible pass/fail scores. Used for over 30 years, this focus-group approach to standard setting has been widely accepted by testing professionals and courts.

The Angoff method is easy to implement and can be perfected by novice users with only minimal training. Attendees will be provided a handout that includes a seven step flowchart that will guide them in the design, development and implementation of the Angoff method for their organization. Attendees will leave the workshop with a newfound appreciation of the Angoff method’s value in the measurement of true competency.

During this session, the audience will be presented a case study from the Coast Guard Maritime Law Enforcement Academy’s (MLEA) Basic Boarding Officer Course. The MLEA will compare their traditional (arbitrary) method of establishing written test cut scores to that of a criteria-referenced technique called the Angoff method. The facilitators will discuss how the USCG is moving their enlisted advancement testing to a more defensible stance by using the Angoff method for establishing cut/pass scores.

Kick-off questions:

• How do you know your test takers are really minimally competent?
  
• What is a passing score on a test?
• Can you defend your cut or passing score in a court of law?