Exam Question Management

Exam questions can be built through the user interface or can be imported. Elentra currently supports a variety of question types:

• multiple choice (with optional multiple responses)

• short answer

• essay

• matching

• text

• fill in the blank

Before creating exam questions, you may want to create Labels so that you can assign Labels to questions as needed.

Create Exam Questions

Users can add individual questions through the user interface, import questions in bulk through a text box, or migrate questions, images, and responses from another exam management tool (this option is only available to medtech:admin users). This section addresses how to create individual questions in the user interface; see here for additional information on importing questions from another exam tool.

Permissions

Currently only staff:admin can create questions by default. To grant a user the ability to view, edit or create questions, assign that user to a label or course.

The Question Management feature handles permissions via two methods: course association and label association.

Course Associations Any course contact will be able to access questions that are associated with that course.

• Note that Course Contact access to questions is controlled through the access control list (a database table in Elentra) so that it can be configured for an institution's need. For example, you could adjust things so that Associated Faculty have the ability to save drafts of questions/groups, but not publish them.

Label Associations Labels are built in Elentra and you can associate individuals with one or more labels. For each association you make, you can control whether the user can read or write questions, groups and exams (in this case groups refers to question groups).

To allow learners to create questions, you must also mark them as "Student Admin" in their user profile.

Add an Exam Question

When you build a question through the user interface, you will walk through several tabs to add the required information.

You will need to complete the first two tabs ("Setup" and "Details") before access to the other tabs becomes available.

Setup

The ability to Publish a Question is available on this tab.

• Assessment Type (Formative or Summative): These categories will be applied in future when exams you create must be flagged as Summative or Formative. You'll only be able to apply Formative questions to Formative Exams, and Summative questions to Summative Exams.

• Question Code: This optional code field can be completed according to the guidelines set by your institution.

• Associated Courses: If a question is assigned to a course, by default, course contacts for that course (e.g. Course Director, Curriculum Coordinator, Program Coordinator, Associated Faculty) will have access to all questions associated with the course.

• Associated Labels: If question is assigned to a label, any permissions set up in that label will be applied to the question.

  • More than one label can be applied to a question. That means that if you want to associate a question with more than one specialty, more than one body system, or more than one topic, labels lets you do that.

  • Note that this replaces Exam Folders in previous versions of Elentra.

• Question Version: Note, for new questions this will be blank.

A question must be associated with either a label or a course. It can be associated to both.

For organizations that used Exams before software upgrades in ME 1.23, all existing questions will be set to summative.

Details

The ability to Publish a Question is available on this tab.

The Details tab allows you to specify the type of question, the question stem, description and options if applicable.

Available fields for all question types:

Questions

• Question Type: Select the required question type.

  • Multiple Choice questions support single or multiple responses.

• Question Text: Provide the question stem or prompt.

  • Use the options in the rich text editor to add images or media to a question.

• Question Description: Add a descriptive title.

• Rationale: Add a question rationale. Rationale is shown to learners when they get feedback on their exam and can explain a question answer or provide additional detail.

Answers

• Answers get added after you tab to or select the next question in a list.

• Select the arrow up or down beside an answer, or drag and drop them into any order you desire.

• Set which answers have been marked as right and wrong via a toggle on the right side.

• Optionally provide a per-answer rationale from the three dot menu on the right side of the response card.

  • Note that this does not currently display to learners so is just for information storage.

• Optionally lock an answer in a specific position (if applicable) from the three dot menu on the right side of the response card.

For additional field details please see Additional Question Type Details below.

Ability to Preview a Question Select the "Preview Question" button at the bottom of the screen to see how the question will appear to Learners.

Ability to Save a Draft of a Question Save a Question as a draft, meaning users can work on that draft over time without worrying it may be accidentally associated to an exam.

Click Save Draft to unlock access to the additional steps in the Question wizard.

Tags

Changes are saved automatically on this tab after an interaction is performed.

Note that the Curriculum Tag Selector does not currently include an option to contextually link other tags.

Optionally assign Curriculum Tags to questions. Tags assigned to questions will be displayed in the Exam Data section of an exam post and can also be leveraged in some exam reporting.

Status

Changes are saved automatically on this tab after an interaction is performed.

The Status tab allows you to configure various statuses to indicate different conditions of a question or reflect workflows.

Question statuses are displayed on the Question Index.

Question Status This is a read-only select box to indicate the current status and other available statuses. To change this use the appropriate actions on Details or Setup or on the Question Index.

Review Status Allows you to flag the question to indicate a review is "Requested", "Approved", or "Rejected".

Translation Status If your institution supports multiple languages this status is made available. It will allow you to indicate a translation is "Requested" or "Completed".

• A comment field is available to track the history or provide context for the statuses set.

• System statuses like Draft/Publish can only be changed by the appropriate action, not on the Status tab. Selecting either Draft/Publish will always create a new version.

Groups

The Groups tab displays a list of any Question Groups this question currently belongs to.

You can read the description or jump to the index page for the group as well.

Exams

The Exams tab displays a list of any Exams this question has been associated with.

You can easily see all the other questions attached to that exam, any posts for that exam and preview the exam as a learner.

Additional Exam Question Type Details

Essay

• Correct Text: Serves as a guide for graders and will be displayed to them when they are grading the exam.

• Maximum number of characters for essay format is set to 65,535.

• Currently essay questions and short answer questions are treated slightly differently in the Learner Feedback Report (available after an exam is completed). Essay questions will show the complete response, while short answer questions will be cut off.

Fill in the Blank

• Create a blank in the Question Text by typing "_?_"

• An answer field will be created for each blank that is detected. Add as many answers as are possible, separating answers with a comma.

• Click Add after you have entered the answer(s).

• Note that a question cannot start with a blank.

• Fill in the blank answers are case sensitive. You may wish to include two answers (e.g. cardiac arrest and Cardiac Arrest) to ensure learners are properly credited for their answers.

  • Graders can manually grade fill in the blank questions if needed.

Matching

• Add one list of terms as items.

• Add the corresponding list of terms as answers.

• For each item, pick the correct answer from the dropdown selector.

• When the learner completes the question, they'll see a list of items and choose an answer from an dropdown selector.

Multiple Choice

• Multiple choice questions presented in horizontal view will not have the strikethrough feature available when learners take an exam.

• Optionally set the toggle to indicate whether a learner is expected to provide multiple responses to a prompt.

• If multiple responses is enabled, complete the additional fields:

  • Grading Scheme

    • Partial: This will give partial credit for every correct reaction (e.g. selecting a correct answer AND not selecting an incorrect answer). By default, Elentra divides the total point value across the number of response options and awards credit for each response option which is correctly responded to (selected if correct, not selected if not correct).

    • All or Nothing: A learner must ace the question to score points. If they miss an answer (whether they select an incorrect answer or miss selecting a correct answer), the learner will get 0 points.

    • Partial with additional penalty: This will give partial credit for every correct reaction and will penalize a learner for an incorrect answer. Elentra divides the total point value across the number of response options and awards credit for each response option which is correctly responded to (selected if correct, not selected if not correct) and deducts credit for an incorrect answer. You can optionally change the value of each answer using the answer weight options.

  • Maximum number of responses

    • Define how many responses a learner will be permitted to enter.

  • Answer Weight

    • This optional feature allows you to support weighting different response options in partial or partial with additional penalty grading schemes.

    • Answers will be equally weighted by default. If you adjust their weights you can tailor the points awarded to a learner when partial and partial with additional penalty grading schemes are used.

  • Sample Use Cases

    • Assume a MC question has 2 correct responses and 2 incorrect responses where the correct responses are worth 40% each and the incorrect responses are worth 10% each. If a learner picks 1 correct response and 1 incorrect response, their score will be the following under each grading scheme:

    • Partial Grading Scheme: Learner scores 0.5 (+0.4 for identifying 1 correct answer, +0 for not selecting a correct response, +0 for selecting an incorrect response, +0.1 for not selecting an incorrect response)

    • All of Nothing Grading Scheme: Learner scores 0

    • Partial with additional penalty Grading Scheme: Learner scores 0 (+0.4 for identifying 1 correct answer, +0.1 for not selecting an incorrect answer, -0.4 for missing 1 correct answer, -0.1 for selecting an incorrect answer)

Short Answer

• Correct Text: Serves as a guide for graders and will be displayed to them when they are grading the exam.

• Currently essay questions and short answer questions are treated slightly differently in the Learner Feedback Report (available after an exam is completed). Essay questions will show the complete response, while short answer questions will be cut off.

Text

• Text questions are designed to allow an exam creator to insert information in an exam.

Copy an Exam Question

From the Setup or Details tab of an existing question you can optionally create a copy of that question.

When viewing the Question Text on the Details tab, the Question Text will be prefaced with "Copy of." The newly created copy will automatically be given draft status and will maintain the existing details and tags.

Batch Edit Exam Questions

When viewing the question index you can make changes to multiple questions at once to enhance productivity when using the Question Index. These actions include:

• Group questions

• Set status

• Set course

• Set curriculum tags

  • This process will open the curriculum tag selector and show you the tags already assigned, which question is assign with which tag.

  • Tags added will be applied to every question selected via checkbox

  • You are able to un-assign tags via this method as well

• Reactivate retired questions

Question Versions

Any time you make a change to a published question, a new version will be created. A new version will be created for any change—correcting a typo, linking the question to additional curriculum tags, altering a response option, etc.

When creating exams, you can add a question with multiple versions and choose to select an older version if desired. Do this by clicking on the forward/back arrow icon. If a question has changed since you created an exam, you will be given a notification for that and can update your exam to reflect the newest version of that question; however, you can still elect to use the old version of the question if you prefer. If the forward back arrow icon is blue, you’re not using the most recent version of a question on an exam.

You can view the versions on a question by selecting "Versions" from the "More" button menu.

The More menu, highlighting the Version option.

Versions listed in the modal will let you preview the previous version to be better able to compare what changed. Reverting to a previous version will create a new version based on that previous version, thereby preserving the complete history of a question.

Creating a new version of a question is as easy as making a change on a question and publishing or saving that version as a draft.

Remove a Question from the Bank

Retired Questions

Retiring a question is a new type of system status for questions. It allows you to prevent a question from being attached to new exams or being used in new exam posts when attached to existing exams. These are the features of a retired question:

• Not eligible to any new exam

• Excluded from new posts of existing exams

• Has a comment field when retiring as to why something is being retired

• Filtered out from the question index unless you opt to filter to view retired questions

• Can reactivate retired questions and then will become available for use on exams again

  • Filter question bank to show retired questions

  • Check off the questions you want to reactivate

  • Click Reactivate (at bottom right)

Deleted Questions

Only a question not attached to any existing exams can be deleted, otherwise we recommend Retiring that question. Deleting a question will remove it from a Question Group.