Evaluation of the BioQuest Academy’s success
Although inquiry-related teaching has been shown to be effective in fostering science literacy, relatively limited research has been conducted to quantify the impact of inquiry-based science programs on student achievement. In response to this challenge, BioQuest has structured its evaluation plan to investigate the program’s impact on student achievement in science, at multiple moments in the participant’s programmatic and academic progression.
Educational researchers use a variety of methods to investigate student achievement, but experimental research is the only type of research that can truly test hypotheses concerning cause and effect relationships. Key problems with designing experiments include 1) controlling for extraneous variables; and, 2) establishing a suitable comparison group. Since Academy students are not chosen at random, the evaluation uses quasi-experimental methods of pre/post testing and comparison to a nationally administered standardized test. The specific evaluation methods include:
Student Demographics—To track participation of targeted teens, demographic questions pertaining to race, ethnicity, and gender will be included on the student Pre Test. Results will be compared to program targets and recruitment will be adjusted accordingly to maintain representation of underserved teens.
Student Pre/Post Tests—Quality of materials and experiences should be assessed through investigation of student learning outcomes, therefore, students will be asked to complete pre and post tests at the Academy. Questions will pertain to the direct benefits of the program (i.e., acquisition of a range of skills and knowledge; confidence in sharing new knowledge; and interest in a career in biomedical research or a related field). Analysis of variance will be used to check for comparable outcomes for diverse teens. Please contact us if you would like a copy of our evaluation tools!
Standardized Achievement Test —Because Academy students were not randomly selected, we have chosen to compare their scores on a standardized assessment given to students nationally. College Board SAT Subject Tests are designed to measure students' knowledge and skills in particular subject areas, as well as their ability to apply that knowledge. The College Board has made sure the tests are independent of any particular textbook or method of instruction. Therefore, they help to compare the academic achievement of students from different high schools in which course preparation and academic backgrounds can vary widely.
Student Interviews—Students will be interviewed mid-year of their first year of college. Using data from interviews, students can communicate descriptive information and narrative stories of their transformation as they move from high school juniors and seniors to college students. Students will be able to describe specific abilities acquired by participating in the Academy. Information gathered through student interviews will be useful for not only understanding program outcomes, but also as a means of improving Academy design.
Website tracking—Embedded tools will help us determine the size and location of the community of website users and the extent of dissemination. Embedded assessment of gains in content comprehension will be determined through the inclusion of embedded queries in the resources designed by the multimedia contractor.
