Transition to Computer-based Testing

SOA fellowship exams now take place at Prometric testing centers Jill K. Carpenter April 2021

Photo: iStock.com/damircudic

At the direction of the Society of Actuaries (SOA) Board of Directors, fellowship exams moved to computer-based testing (CBT) beginning with the fall 2020 exams. This meant candidates went to Prometric testing centers to take their exams on Prometric computers, where they provided their answers in Microsoft Word or Excel as directed. Candidates then uploaded their completed files at the conclusion of the exam to a secure Prometric server. From there, the SOA securely collected all candidate files from a portal accessible to SOA volunteer graders.

Background

The SOA and Prometric have a long history of working together. The SOA has used CBT for the multiple-choice associateship exams since September 2005. Thus, candidates are familiar with the process of booking an appointment, checking in at the center and the basic Prometric testing environment.

In 2018, the SOA introduced the Predictive Analytics exam. Candidates take this exam at a Prometric test center using a computer equipped with Microsoft Word, Excel and RStudio. They submit their final project as a Word file and upload their R file as well.

Over the last three years, SOA staff have worked closely with Prometric to fine-tune the logistics for its exams. The Education Committee has been working toward CBT for the fellowship exams for several years and had previously asked the exam committees to consider how Excel could be integrated into their questions.

Demo Video of Prometric Testing Environment

When the COVID-19 pandemic hit in early 2020, the SOA postponed the spring exams until July and worked diligently to secure testing space and proctors for small groups for all exams around the world. This created a situation where, to provide a safe environment for all, the SOA was subject to ever-changing global regulations. With a significant amount of diligence, the examinations took place in secure settings. It became clear that the current means of in-person proctored examinations were not viable. This prompted the Board of Directors’ decision to move the written-answer exams to CBT beginning with the fall 2020 exam sitting.

Advantages of Prometric Test Centers and CBT

There are several advantages to offering exams via CBT at Prometric testing centers. Prometric operates worldwide and keeps current with the regulations and requirements in all jurisdictions. It can administer tests in situations where the SOA cannot. In addition, the CBT format allows for the use of programs such as Microsoft Word, Excel and RStudio, which better aligns with the working tools of actuaries.

Reactions from Candidates

After the fall 2020 fellowship exams, the SOA administered a candidate survey to solicit feedback. The following themes emerged:

Writing in Word was a welcome change.
Candidates liked the functionality of Excel and not needing a calculator.
Getting into and out of the testing center was faster than the SOA-proctored exam centers (i.e., in an SOA-proctored environment, no one may leave the exam room until all exams are collected and accounted for).
Word was used for some calculation questions, so typing in formulas was clunky.
Some candidates were unsure how to show their work in an Excel file.
The fact that the upload had to be completed within the exam time period was stressful.

Reactions from Exam Committees

Additionally, the exam committee members offered the following thoughts after the fall 2020 fellowship exams:

Exam committees can construct more robust questions with the additional computing power of Excel.
Handwriting legibility was not an issue, which made grading easier.
Having formulas in Excel was often easier to follow and grade.
Some candidates used very long or unfamiliar formulas, which made grading harder.

Conclusion

After a first successful administration, it is clear the transition to CBT for the SOA’s fellowship exams will be a learning process for everyone involved—Prometric leaders and centers, SOA Education staff, exam committees and candidates.

Several issues already have been addressed. For instance, SOA tutorials and sample solutions now provide revised instructions and examples of how to show work in Excel. Beginning with the spring 2021 exam sitting, there will be a separate upload window at the conclusion of the exam. This upload window will appear immediately after the allotted exam time has expired, removing the stress of having to reserve time at the end of the exam to upload files. The volunteers writing the questions are refining their use of the additional functionality. Candidates are becoming more familiar with the setup and expectations. And, safety and reliability within the testing environment are improved. CBT for written-answer exams is here to stay!

Jill K. Carpenter, FSA, is the past general chair of the SOA Education Committee.