Form Return Answer Sheet Video

We began using the Form Return answer sheets during the Science 10 Pilot which started in August 2013. Since then, we have been using the FR answer sheets for many other courses. During the pilot, we noticed that students were having some difficulty filling out the answer sheet accurately. Although the students have been getting better at properly filling out the answer sheets as we added more courses, there is still some confusion.

Because the final exams are coming up, many of which will use the Form Return answer sheets, we have created a short video to show our students and to post on your D2L. The video simply explains all the areas of the answer sheet and shows the students how the bubbles need to be filled in. The video also gives examples of different number combinations (0.01, 1.50, and 25) that are often seen in the numerical response and tend to confuse the students.

Please use the video as you like: show in class, post to D2L, post on your YouTube channel, etc.. We are also more than happy to come to your classes to give a demonstration and answer any questions your students may have.

Below is the link to the video:

Answer Sheet Video

Thank you.

Tips for Teaching Math/Science Diploma Exam Courses

 The CRC recently hosted a presentation aimed towards providing tips to teachers that are teaching a Math or Science 30 level course for the first time.   Presenters Stan Bissell and Pearl Nijjar, representatives of Alberta Education -  Education Program Standards and Assessment Branch,  provided information, resources and additional contacts to help clarify questions and procedures relating to Diploma Exams. 

The three fold purpose of Diploma Exams, as intended by Alberta Education, was identified as follows:

1. ·         To certify the level of individual student achievement in selected Grade 12 courses
2. ·         To ensure that province-wide standards of achievement are maintained
3. ·         To report individual and group results

To meet the objectives outlined above, Diploma exams go through an extensive design and testing process, beginning with an approved blueprint on the Program of Studies that corresponds to a desired level of difficulty. Teachers are directly involved in the item writing process and if you are interested in being involved let your Coordinator know. Once test questions are designed they are field tested, again with the help of teachers and their students. Many teachers use these field tests as a means of providing diploma-like test practice to their students, without the high-stakes pressure. Currently math and science field tests are only available in an online format. Criteria around field testing requirements and procedures can be found at this link:  Bulletin Alberta Education Field Testing. Results from the field tests are used to approve, fine-tune, or discard test questions for future diploma exams. As each diploma exam administration involves a unique combination of questions, they are designed to include a set number of “anchor” questions that are consistent from one administration to the next. This allows exams to be compared for difficulty level and results can be “equated” if necessary so as to maintain consistent standards over time.

For teachers teaching diploma level courses, the presenters identified the following resources to inform their practice:   Program of Studies,  Information Bulletins,  Standard Documents (Math 30-2,  Math 30-1, Biology 30,  Chemistry 30, Physics 30 and  Science 30 ),  Science Data Pages and Booklets , and Released Items from previous diploma exam administrations. Currently, with the more recent implementation of the Mathematics Program at grade 12 there are not any Released Items from Mathematics 30-1, but there is significant overlap between Mathematics 30-1 and the former Pure Math 30 objectives so as to provide a sample of questions. Alberta Education has released the January 2013 test for Mathematics 30-2 – see Released Math 30-2 Items, as the course objectives here are quite different from the Applied Math 30 course.

Within the vision of Inspiring Education, Alberta Education has made several changes with the intent to increase opportunities for students to write diploma exam and simplify the process of providing student accommodation. As part of the transformation surrounding increased availability and digital delivery, Alberta Education is working to administer multiple forms of diploma exams so as to more quickly increase their exam question banks. Along with the new delivery format, Alberta Education is increasing security and now only provides electronic versions of diploma exams for perusal. Specific information regarding these changes is found in this General Information Bulletin. 

Alberta Education has established Quest A+  in their move towards providing digital delivery of diploma exams.  This site is currently used by teachers to administer digital field tests and has the added value of providing statistical reports from each field test summarizing each question’s reporting category and student success rate.  Quest A+ also includes open unit and final tests in digital format, thus providing an opportunity for students to increase their experience and comfort with the format of diploma exam tests and the digital delivery process. 

Overall, even seasoned teachers found the session to be filled with a list of helpful reminders and resources as well as a summary of important updates regarding recent changes and future intents.

 

Final Exams

As you all know, final exams are fast approaching. The exam development team will be responsible for printing, assembling, and delivering these final exams to you or Jenniel (whichever is stated on the form). To order your final exams, please use this form that resides on the Foundational Curriculum website. Please order your exams at least 2 weeks in advance so we can ensure that all of the orders are completed ahead of time.

you can order for your classes individually.

Below is the link to the form:

Exam Request Form

Thank you

Online Diploma Exams

I recently attended a workshop at the Calgary Regional Consortium on Quest A+ diploma exam administration. The session was delivered by two Alberta Education associates, the primary presenter being Jeremy Wiebe Manager, Online Assessment. The presentation was very well done, and the participants had a chance to work with Quest A+ in both facilitator and student roles.

Before we got hands-on experience with Quest A+, Jeremy and his co-presenter Dan, gave some background information on the program, its province-wide implementation, and the changes to Provincial Achievement Tests (now called Student Learning Assessments—SLA). Provincial assessments (diplomas and SLAs) will be increasingly offered in an online environment, and in April 2015, all diploma exams will be offered to students only in digital format (except for those with special accommodations). Some changes have already been made in regards to the online process.

·         Alberta Education is supplying digital format diplomas for teacher perusal.

·         digital format SLAs will be written in September 2014

·         Quest A+ is already being used in many areas for Part A diplomas

·         Math/Science field tests are now exclusively digital

Many Part A (written response) exams are already online. Alberta Ed sees three main benefits to this: Data Security, Ease of Administration, and In-Exam Security. 

Data Security:
Student responses are uploaded every 2 minutes, or every 100 characters as well as every time a student clicks save. If there is an interruption to the internet, student responses are still backed up locally, so no data is lost. These responses are then uploaded automatically so the student can continue writing the exam without interruption (and the student will not be notified that anything was amiss). If there is a hardware issue, the student can switch machines and the most that would be lost is 99 characters. 

Easier Administration: 
The program keeps track of the elapsed time on an individual student basis and will automatically shut off as soon as time runs out. An extra time accommodation, and any other accommodation, can be enabled prior to the exam by the facilitator on an individual student basis. If there is an interruption to the exam (such as a fire alarm), the facilitator can click "pause all" and all exams will be paused. The facilitator can also pause an individual exam if the situation calls for it.

In-Exam Security 
In order to use Quest A+ in an exam situation, Locked Browser must be enabled. Once Locked Browser for Quest A+ is enabled, it automatically shuts down any running programs, and closes all other web browsers, tabs, and windows. Students are not able to insert a CD or USB while the locked browser is activated, and Quest A+ exams will not run without the locked browser. The software also disables functions such as screen sharing and disallows access to all other files and software on the computer. Students can bring their own devices to write the exam as the security features are built into the Locked Browser software, not on individual machines. The exams can be written using Google Chromebooks (CBE has purchased these for student use), desktops, and laptops. To date, there is no mobile device capability. 

The facilitators encouraged teachers to use Quest A+ with their students to prepare them for the impending change. Teachers can help to prepare students by using practice exams. To date, there are practice exams for all diplomas except ELA 30-1/2. Alberta Ed is working with publishers to make the readings available online. They expect they will have practice exams in the next few months.

Alberta Education has codes for instructors to use with Quest A+ to simulate an actual exam writing experience (requires student log-in and teachers can set accommodations). An unlimited number of students can use one code for practice exams, and so an entire class could write the same practice exam. Students get immediate feedback on their scores with a list of their incorrect items. Students can then go back and redo any incorrect items. I will be contacting Jeremy shortly for codes and will share them with any interested instructors. Please let me know if you would like codes for your class(es).

For more information, please see the Prezi that Jeremy and Dan provided. I’m also happy to answer any questions you may have about the session.

Maureen Stewart

Curriculum Development Exam Lead

Big thanks and congratulations!

The Curriculum Development Centre would like to congratulate Karim Jaber and Karlie Wimble on their nominations for the Bow Valley College Rookie of the Year Award. As you saw last Wednesday at the College Wide PD Day, Karim was the winner of the award. Congratulations Karim! 

Thank you Karlie and Karim for all your work with exam development, collaboration in the department, and for your perpetual sunny dispositions. It is a pleasure to work with both of you. You're both Rookie of the Year in our eyes. 

Please join us in thanking both Karlie and Karim, and congratulating Karim on his big win. We are so proud!

January 2014 School Reports for Biology 30, Chemistry 30 and Physics 30

I have downloaded the diploma exam results (School Reports) for Biology 30, Chemistry 30, and Physics 30.  These were just released last Friday on Extranet.  If you’d like to see the results for a particular science course, see my staff folder at:  Michael Gaschnitz\SCHOOL REPORTS\2014 January .

The School Reports provide student results but also permit deeper insights into the construction of the diploma exams than the Information Bulletins allow.  We can see which outcomes are emphasized and which are largely neglected.  This helps us begin with the end in mind so we are better able to design courses, exams, and labs that emphasize the key skills and abilities our students are most likely to require when they are faced with a diploma or equivalency exam.

The School Reports blueprint the items and provide fairly detailed descriptions such as this one for MC17 on Biology 30:  “B2.2k, B2.2s, STS;  Given a diagram of the female reproductive system, identify two structures that are specifically implicated in a particular assisted reproduction procedure. (Also B1.3k, B3.5k, and B3.1sts).”  Normally, I would say that attempting to evaluate five outcomes in a single item is crazy, but perhaps this is justified given that they are trying to test the integration of higher skills.  These descriptions can help focus instruction on the topics that are critical.  They can also guide item development..

Furthermore, the frequency at which each specific outcome is tested can be collected and analyzed to guide exam development for much better content validity.   An analysis Physics 30 Unit 2. Curious trends become apparent.  Some outcomes are rarely tested perhaps because writing statistically valid multiple-choice or numerical-response items for those outcomes is difficult.  We usually wouldn't want to bother attempting to create items that even Alberta Ed itself with all its resources and expertise isn’t able to create:  this just wouldn’t be a sensible investment of our very limited resources.  A possible methodology for tablulating this data is here:  http://acfonthesamepage.blogspot.ca/2013/11/decoding-diploma-exams-part-i_6.html  .  (This presumes that the diploma exams are the model of assessment perfection, but this might not be the case).

For example, Physics 30 outcome B3.9k (“describe, qualitatively, the effects of moving a conductor in an external magnetic field, in terms of moving charges in a magnetic field“)  was not tested on the last six Physics 30 diploma exams.  Given how short unit exams need to be, this is an outcome we wouldn’t want to test very often or even at all.  It might be better to address this outcome as part of a lab.

Regards,

Michael

Previous blog postings

Form Return - update and next steps

Update

When FormReturn was implemented in the Fall of 2013, it was initially used in a pilot project to mark a few exams. In December 2013, it was used to mark final and equivalency exams.  Since then, the number of answer sheets created has increased to 83 and 939 exams have been processed to date.

Subject	# Exams Processed	Answer Sheets
Biology	226	10
Chemistry	107	11
General Science	383	15
Math	119	32
Physics	50	11
English	54	4
Total	939	83

Table 1. The number of answer sheets created by subject and exams processed to date.

Exam	# Exams Processed
Biology 20 Final	29
Biology 20 Unit 1	28
Biology 20 Unit 2	24
Biology 20 Unit 3	19
Biology 30 EQ	35
Biology 30 Unit 1	47
Biology 30 Unit 2	42
Chem 30 Unit 2	26
English 30-2 EQ	30
Math 30-1 Unit 1	26
Math 30-1 Unit 2	23
Science 10 Final	38
Science 14 Final	36
Science 14 Unit 1	38

Table 2.  A detailed exam and item analysis report will be created for all exams having reached the target number of exams processed of 40 or more. (100  exams or more is ideal.)

Next steps

We will continue to create answer sheets and collect data for all official exams, especially math and science.  As the number of answer sheets increases, the amount of data collected continues to increase.

One purpose of FormReturn is to mark multiple choice and numerical response questions, but it also allows us to provide useful information to teachers.

Currently, we are providing teachers with immediate feedback on the exams in the form of simple student reports, and graphical representations of student marks and responses. (Figure 1) Once the number of exams processed reaches 40 or greater (see Table 2), we will also provide an exam and item analysis report.

These cumulative reports and any aggregate exam data collected will be made available to teachers on Google drive. 

Figure 1. Sample immediate feedback provided to teachers.

Alberta Writes 3

Bow Valley College, the University of Lethbridge, and the University of Alberta invite you to join us on April 11, 2014 for Alberta Writes 3. We will facilitate a discussion on the role of writing in students’ transition from academic study to full-time employment.

Adults preparing for the workforce or transitioning into the workforce with new roles or challenges need to develop the writing strategies and skills necessary to write across a range of modalities (social media, email), genres (reports, letters) and contexts (home, work and community). 

Alberta Writes 3 is the third in a series of one-day symposia about writing. The first two symposia explored the gaps, barriers and supports students face as they transition from writing in high school to writing in post-secondary environments. 

We are inviting participation from post-secondary, industry, workforce, and government contexts. I hope you will consider joining this conversation.

April 11, 2014 | 8:00 a.m. to 4:00 p.m.
Bow Valley College, North Campus, N231
345 6^th Avenue SE, Calgary, Alberta, T2G 4V1

For general registration use Eventbrite: bowvalleycollege.ca/AlbertaWrites3

(Lunch is provided)

For more information please contact Candace Witkowskyj at cwitkowskyj@bowvalleycollege.ca or 403-355-4664.

Exam Development Update

Happy March everyone! I hope your semester is going well so far. 

The Exam Development Team would like to update you on the progress of the current exam development projects in CEFL. Thanks to the many instructors who have worked hard on exam development projects! Your knowledge and expertise has allowed us to create valid and reliable exams for a number of courses. Thank you to Karlie Wimble and Karim Jaber for the ongoing and invaluable work they have done on creating answer sheets, converting and distributing exams, and revising answer keys. Thank you also to our coordinator Carey Hilgartner for his ongoing support and guidance.

Current Exam Development projects:

Science 14 Final and Unit Exam Review

-nearly complete Form B for the final

-Final Exam Form A has been used since August and we are continuing to collect data using Form Return answer sheets. This exam has proven to be valid and reliable.

-Unit exams 1 and 2 complete with Form Return answer sheets. We are working on Units C and D

Science 14 Exam Review Collaborators:

Lindsay Bonenfant, Danielle Dore, Karen Harris, Julie Hill, Penny Marcotte, Alix Macdonald, Neil Summers

English Language Arts 30-2 Equivalency Exam Review

-final stages of Form B revisions and review

-Form A has been in use since August. We are collecting data using Form Return answer sheets. Form A has proven to be valid and reliable.

ELA 30-2 Equivalency Exam Review Collaborators:

Meghan Clayton, Susan Lemmer, Lorna Malanik, Tasha Nott, Patricia Pryce,  Murray Ronaghan, Jennefer Rousseau, Chris Taylor

English Language Arts 20-1 Unit Exam Review

-The review process is under way and items for four unit exams are coming together.

-At the end of this project, we will have 4 common unit exams for ELA 20-1 (poetry, non-fiction, Shakespeare, and short story).

ELA 20-1 Unit Exam Review Collaborators:

 Meghan Clayton, Marc Desilets, Danielle Dore, Tasha Nott, Murray Ronaghan, Jennefer Rousseau, Duff Waddell

Bio 30 and Bio 20 Unit Exam Review

-Units 1 and 2 have been reviewed, revised, and the new forms blueprinted. We are using Form Return to track data for these exams

-Units 3 and 4 are coming along with item review and revision, as well as item writing.

Bio 20/30 Unit Exam Review Collaborators:

Lindsay Bonenfant, Michelle Bourne, Celia Herodek, Penny Marcotte, Lee Murray, Laura Slade

Common Unit and Final Exams

-We are making our way through the exam conversion process.

-thanks to Karlie's excellent workshop on how to convert exams, we have had more staff participating in the conversion project (you can view the video on the Foundational Curriculum website)

Exam Conversion Team:

Danielle Dore, Penny Marcotte, Karlie Wimble

Math 30-1 Form B Unit Exams

-Thanks to Debbie Scott for taking on the initiative to write Form B unit exams for MAT3791

-These exams have been reviewed and blueprinted and are currently being used. The exams also have Form Return answer sheets.

Exam Reviewers:

Debbie Scott, Michael Gaschnitz, Bev Kosior, Melanie Macdonald, Brenda Thomas

There has also been some Exam Development Centre work taking place that is not necessarily a project, but is still valuable work to the CEFL. Work is currently being done on Science 10 unit and final exams (Neil Summers, Penny Marcotte, Karim Jaber), and Chemistry 20 unit exams (Debbie Scott, Lindsay Bonenfant, Penny Marcotte), and Math 30-3 exam and assignment development thanks to Brenda Thomas and Bev Kosior.

Thank you again to all instructors who have graciously given their time and expertise to exam development projects! It’s a pleasure to work with all of you!

If you would like to be involved in an exam development project please contact Maureen Stewart (3410, mstewart@bowvalleycollege.ca). If you would like to submit an exam development project idea, please visit the Foundational Curriculum website and click on Submit a Project Idea.

If you spot an error in an exam, please fill out the Report and Error Form on the Foundational Curriculum website. We will respond within one business day. Many thanks to those already using the form including Brenda Thomas, Penny Marcotte, Neil Summers, Debbie Scott, Michelle Bourne, Celia Herodek, Laura Slade, and Lindsay Bonenfant.

We look forward to working with you!

Maureen Stewart
Curriculum Development Exam Lead