In this month’s letter, ADEA President and CEO Dr. Rick Valachovic shares the latest news on dental school application trends and other indicators confirming that the profession remains an attractive one.
In 2010, ADEA surveyed advanced education program directors and asked them to list the top qualities that are important to them in selecting candidates for their programs. Here are the attributes that ranked in the top five:
- Team player
- Assumes responsibility
- Integrity
- Interpersonal/communication skills
- Reliability
When the responses of each dental specialty were looked at individually, critical thinking and clinical skills, or both, typically made their way into the top five as well. All but one specialty (more on that later) ranked academic talent dead last.
This finding may sound counterintuitive, but it shouldn’t be surprising. Academic talent is essentially a given among applicants to advanced dental education programs. Only high achievers make it into dental school to begin with, and no one graduates by being an academic slouch. But two developments—the decision in 2008 by the Joint Commission on National Dental Examinations (JCNDE) to move to pass/fail licensure exams in 2012 and the increased number of program applicants coming from dental schools that employ pass/fail grading—have raised difficult questions for advanced education programs accustomed to using National Board scores, grade point averages and class rank to help them screen and select candidates.
In anticipation of these challenges, ADEA created the ADEA Future of Advanced Dental Education Admissions (ADEA/FADEA) Project in 2009 to identify the best ways to review, compare and select candidates for advanced dental education programs. The project’s first activity was to conduct the 2010 survey and the results revealed a clear disconnect between the qualities program directors said they most valued, and the tools they were using to identify them. When the survey asked the directors to name the top factors they found most useful in evaluating candidates, letters of evaluation topped the list; but the three factors that followed—class rank, NBDE Part I scores and dental school GPA—indicated the difficulties that lay ahead. Come 2012, these numerical scores would no longer be available for some or all candidates, and perhaps more importantly, many in our community openly questioned their value in predicting professional success.
I wrote about the reasoning behind pass/fail grading in last month’s Charting Progress. In a nutshell, more and more students are choosing to go to dental schools that do not provide a GPA or class rank because they find the pass/fail grading environment more conducive to learning. And when it comes to scores on National Board exams, there is no evidence to support the notion that differences within the range of passing scores reflect meaningful differences in knowledge and ability among test takers, despite the common use of the exams to make these distinctions in the past. Furthermore, the inability of National Board scores to predict success in advanced education programs is borne out by program directors who tell us that struggling students rarely lack academic skills. Rather, their problems stem from a lack of interpersonal or communication skills, or an inability to function well in teams.
These qualities are the focus of the ETS® Personal Potential Index (ETS PPI), a web-based system that evaluates applicants on personal characteristics in six areas: knowledge and creativity, communication skills, teamwork, resilience, planning and organization, and ethics and integrity. Considering these personal characteristics alongside traditional metrics such as scores and grades is at the crux of holistic review, an approach to admissions that ADEA member schools have widely embraced in selecting their predoctoral students. In fact, according to a recent report by Urban Universities for HEALTH, a project that aims to increase health equity by diversifying the health care workforce, dental schools lead among the health professions in their use of holistic admissions.
ADEA began piloting the ETS PPI as part of the ADEA Postdoctoral Application Support Service (ADEA PASS℠) application in 2012, and program directors report that they like having a standardized tool to assess many of the variables linked to success in graduate, business or professional school. They also say the ETS PPI is becoming more helpful with each passing year, in no small measure because we have been working to clarify the instructions for employing the tool and encouraging faculty to make full use of the comment fields to fill in any gaps that may exist in a student’s letters of reference. These letters remain a vital part of every student’s application package, but they can be as varied as the individuals who write them. Moving forward, we will be working with deans and others to make sure that letters of evaluation are comprehensive and demonstrate a genuine knowledge of each student’s strengths and weaknesses to give candidates and program directors their best possible shot at finding matches that will work for them both.
Since the creation of ADEA/FADEA, the project has collaborated with the ADEA Council of Hospitals and Advanced Education Programs (ADEA COHAEP) to host two collaborative summits on advanced education admissions. The first summit allowed ADEA members to discuss the findings of the 2010 survey. The second, held in 2013, brought program representatives together to learn about new tools within the ADEA PASS application and gain a better understanding of how pass/fail dental schools evaluate their students. Seven of the eight schools that currently use pass/fail grading attended and shed light on the systems they have in place to distinguish outstanding students from good students in their academic environments.
The 2013 summit also touched on the use of multiple mini-interviews (MMI), the practice of having applicants work through short, structured problems with a series of faculty members. I first wrote about this technique in a 2008 issue of Charting Progress, which looked at ways of assessing student professionalism. Today, more schools and programs are using MMIs to reveal many of the characteristics that faculty most value. (For more on MMIs and other topics related to admissions, see the April 2014 Journal of Dental Education (JDE).)
Last fall I learned that some of our colleagues in medicine are using a proprietary online personality assessment tool to gather similar information in a slightly different way. Residency Select purports to reveal candidates’ behavioral attributes in three areas: usual tendencies, stress tendencies and drivers. Programs customize the instrument so that each candidate assessment profile reflects the program’s particular screening criteria. According to the company behind Residency Select, no dental residency programs are currently using the tool, but it is available to them. It will be interesting to see how this assessment instrument works for medical residency programs on some of our campuses and to consider whether it might be useful for some of our programs as well.
One group of programs that might welcome a tool of this type is oral and maxillofacial surgery (OMS) residencies. If you haven’t already guessed, OMS is the specialty that ranked academic talent highly in the 2010 ADEA/FADEA survey. In fact, this group of specialists placed academic talent at the top of their list and chose “National Board score above 90” as the fifth most desirable quality they sought in applicants. Why did the results for this specialty diverge so much from the findings for other groups of advanced education programs?
To find out, I called David Shafer, D.M.D., Associate Professor and Chair of the Division of Oral and Maxillofacial Surgery at the University of Connecticut School of Dental Medicine. Dave has been a member of COHAEP for more than a decade and is now the ADEA Board Director for Hospitals and Advanced Education Programs so he is well versed in the concerns of his peers. He is also in an unusual position. Situated in a school that uses pass/fail grading and in a specialty that values high-stakes standardized exams, he is willing to defend both.
“There’s no reason to suspect that at pass/fail schools, the students aren’t being taught to a very high level,” he told me when I asked about the reluctance of some programs to consider candidates from these institutions. “I’d argue that some of the pass/fail schools are some of the hardest schools in the country to get into.”
He also shared praise for the ETS PPI and its role in bringing more standardization to the appraisal of candidates. “We’ve had the PPI for three cycles now,” David observed, “and people are getting better at writing comments that are useful.”
But for some of the more competitive programs, Dave says that determining high academic achievement remains an important part of the mix. For his specialty in particular, strong exam scores are essential for entering the profession.
“The problem we run into in oral and maxillofacial surgery is that we need to know students can perform well on a high-stakes exam because they’re going to have to take other ones. Many of our students go on to get an M.D., which requires an ability to pass Step 1 and Step 2 of the Medical Boards, and then they have to take their oral maxillofacial surgery boards. I need that data. I need to know who can perform.”
To address this reality, OMS programs began requiring scores from the National Board of Medical Examiners Comprehensive Basic Science Examination (CBSE) when NBDE scores ceased to be available. The CBSE is considered a statistically valid, in-depth examination of basic medical science knowledge, and the test seems to be a good fit for these specialty programs.
As I wrote last month, the American Dental Association is developing an admissions test for advanced dental education programs in 2016. At a minimum, the exam will expand the admissions tools available to advanced education programs, but these programs are diverse, and one exam is unlikely to suit them all. Personally, I’m not concerned. Each year, dental schools manage to evaluate applicants from 1,800 different colleges and universities seeking admission to their predoctoral programs. If we can do that, I’m confident we will figure out graduate admissions in a pass/fail environment as well.
adeachartingprogress.wordpress.com 