January 2014 School Reports for Biology 30, Chemistry 30 and Physics 30

I have downloaded the diploma exam results (School Reports) for Biology 30, Chemistry 30, and Physics 30.  These were just released last Friday on Extranet.  If you’d like to see the results for a particular science course, see my staff folder at:  Michael Gaschnitz\SCHOOL REPORTS\2014 January .

The School Reports provide student results but also permit deeper insights into the construction of the diploma exams than the Information Bulletins allow.  We can see which outcomes are emphasized and which are largely neglected.  This helps us begin with the end in mind so we are better able to design courses, exams, and labs that emphasize the key skills and abilities our students are most likely to require when they are faced with a diploma or equivalency exam.

The School Reports blueprint the items and provide fairly detailed descriptions such as this one for MC17 on Biology 30:  “B2.2k, B2.2s, STS;  Given a diagram of the female reproductive system, identify two structures that are specifically implicated in a particular assisted reproduction procedure. (Also B1.3k, B3.5k, and B3.1sts).”  Normally, I would say that attempting to evaluate five outcomes in a single item is crazy, but perhaps this is justified given that they are trying to test the integration of higher skills.  These descriptions can help focus instruction on the topics that are critical.  They can also guide item development..

Furthermore, the frequency at which each specific outcome is tested can be collected and analyzed to guide exam development for much better content validity.   An analysis Physics 30 Unit 2. Curious trends become apparent.  Some outcomes are rarely tested perhaps because writing statistically valid multiple-choice or numerical-response items for those outcomes is difficult.  We usually wouldn't want to bother attempting to create items that even Alberta Ed itself with all its resources and expertise isn’t able to create:  this just wouldn’t be a sensible investment of our very limited resources.  A possible methodology for tablulating this data is here:  http://acfonthesamepage.blogspot.ca/2013/11/decoding-diploma-exams-part-i_6.html  .  (This presumes that the diploma exams are the model of assessment perfection, but this might not be the case).

For example, Physics 30 outcome B3.9k (“describe, qualitatively, the effects of moving a conductor in an external magnetic field, in terms of moving charges in a magnetic field“)  was not tested on the last six Physics 30 diploma exams.  Given how short unit exams need to be, this is an outcome we wouldn’t want to test very often or even at all.  It might be better to address this outcome as part of a lab.

Regards,

Michael

Previous blog postings