Special to the Journal
Augusta, Ga. – Health profession educators can now quickly find tests that score well in measuring their students’ competency.Educational researchers at the Medical College of Georgia at Georgia Regents University have teamed up with the AAMC (Association of American Medical Colleges) to provide the Directory and Repository of Educational Assessment Measures or DREAM.
The steadily growing DREAM, https://www.mededportal.org/dream, now lives on the site of the AAMC’s MedEdPORTAL Publications, which was established in 2005 to provide educators and the public easy, free access to peer-reviewed teaching resources such as tutorials, virtual patients, simulation cases, and podcasts.
”The AAMC’s partnership with the American Dental Education Association filled the void of a centralized clearing house of high quality health-care educational materials,” said Dr. Christopher S. Candler, MedEdPORTAL’s Editor-in-Chief and Co-Founder.
DREAM provides the same type of home for assessments of health professions student competencies such as whether a student has truly learned the best way to communicate with patients or understands the anatomy of the human body.
“DREAM is a very important and logical development for MedEdPORTAL Publications that will help us meet the needs of health professions’ educators even better,” Candler said.
Dr. Lara M. Stepleman, a psychologist and Co-Director of GRU’s Educational Innovation Institute, and Dr. Christie L. Palladino, Educational Researcher and an Associate Editor of MedEdPORTAL Publications, co-developed DREAM as a way to bring greater scientific objectivity to health professions education assessment.
While untested tests seem like a non sequitur, their literature searches found that nearly half of the tests used to measure a wide variety of competencies in the health professions were only used at a single institution and had no published data about their efficacy. That meant a lot of duplication of effort in terms of educators producing – but rarely validating – tests.
“Without validating tests, educators cannot be sure that a student passing a test has really mastered a given competency,” Palladino said.
Accurately demonstrating that students can master certain competencies is part of health professions universities’ obligation to students, patients, accrediting bodies, and the public,” Stepleman added. Also, there was no single source to find ‘tested’ tests, the researchers said.
“We test students all the time but how good are the tests we use?” Palladino said. “We want to put into educators’ hands the information they need to select the right assessment for what they are teaching so they can make good, evidence-based decisions on how well they are doing.”
Rather than a time-consuming scouring of the literature to find a test – that might not work well anyway – or making up their own test, educators can visit DREAM and find tests whose quality has been measured in published studies along with a 1,000-word peer-reviewed analysis of the test by a known expert in that particular area.
“The 1,000-word analyses serve as an evidence-based ‘review’ of each assessments strengths and weaknesses. “Social media has made us much more dependent on reviews,” Stepleman said. “What we are saying is if you read multiple reviews just to buy a camera shouldn’t we be able to apply that same level of criteria to making sure we have a doctor of high quality?”
“The goal as we talk about DREAM is to have measures that have been developed and tested so that if we say a student is good at something, they are good at it whether they are in Georgia or Washington or assessed by this faculty member or that one,” she said.
Like their educational colleagues around the country, they want evidence-based testing that not only tells an educator whether his curriculum is accomplishing what it should , but an objective means for comparing curriculums so that ultimately, the best ones get used.
“The facts are that while many medical educators, for example, don’t have ample time or training to develop tests, medical education is becoming more competency focused,” Stepleman said. Groups like the AAMC and the Accreditation Council on Graduate Medical Education, for example, set physician competencies like patient care, professionalism and self-directed learning.
The first DREAM visitors will find about 100 assessments available as a result of Stepleman and Palladino’s extensive perusals of the literature. Their systematic assessment of assessments also is identifying test gaps that they plan to help fill.
They note that not every test included in DREAM gets rave reviews, but it’s important that educators recognize the strengths and weaknesses of tests, Palladino said. “Think about scientific literature,” said Stepleman. “One of the biggest criticisms is that people don’t publish negative findings.”
“We want to become that place where someone goes when they want to measure a competency related to a health professions student,” Stepleman said.