Question Index

When you click on the Questions tab you will land on the Question Index page. From this page you can search for questions by typing in a word, and you can apply multiple filters to refine your search.When viewing results, card view will show you the question text and responses along with an option to see more information at the bottom of each card.List view will let you see more questions in a smaller field and get a quick overview of their status, associated groups, assigned curriculum tags, application to exams, and more.

Search and Filter

Search query can target any visible field. Therefore, if you are showing additional columns via the "Columns" button, search queries will include those additional columns allowing for more granular ability to see only what you want to see.Filter options greatly expanded and now include Label, Date, Type, Author, Exam Usage (allows you to pick a specific exam and see the questions used), Course, Group, Retired, Status, Curriculum Tags, Assessment Type. Filters allow for more than one selection at a time allowing more granular ability to scope the returned question items to be exactly what you need.Each time you pick a category from Filter By (e.g. Question Types) you'll then be able to select your specific filters (e.g. Multiple Choice). One exception to this is the Retired option (it will just show you retired questions or all questions).Currently applied filters are displayed as badges, and can be cleared by selecting the x beside the individual badge, or via the Clear All Filters button.​

• A - Question Bank - select the Question Bank on the left-hand navigation bar to display Question Bank.

• B - Search - use keywords in the search box to refine your search.

• C - Filter - use the filter to refine the search by; author, course, group, label etc.

• D - Columns - add additional columns to the Question Bank to see more question information including; author, date, languages etc.

Enhanced Navigation Experience

The following enhancements have been made to both the table and card view of the index:

• ​The More button allows you to navigate to the various tabs of the Edit Question process

• ​The Edit button will start the Edit Question process

• Paginated screens have controls to display how many can be displayed, and the ability to jump to first and last pages

Table View

In Table View you can control which columns to show for your filtered questions via the "Columns" button.

Card View

The Card View will allow you a preview of the question as it might appear to learners, the ability to view more information about specific questions and the ability to show or hide the correct answer.

Status

Question Status is a new field. You can now create questions in draft mode and request that they be reviewed. This allows for careful oversight of all questions before they are published.

Currently you do not set a specific reviewer per question. Instead you can filter by status to find questions that require attention.

Additional icons will show a question's Review status:

You can Ability to set status on multiple questions by checking the question you'd like to edit and selecting: "Set Status".

Data Visibility

The Question Index user interface has been improved to include more data in a more visible state. This includes:

• Associated Groups

  • See groups this question belongs to and click edit to quickly go edit a group

• Associated Exams

  • You'll see the total question count, a count of any existing posts, and take the exam in a preview mode; each of these will link you to another tab.

• Associated Exam Types

• Associated Curriculum Tags

Click on the count for each entity to see more detail about a question.

Batch (aka Bulk) Index actions

When viewing the question index you can make changes to multiple questions at once to enhance productivity when using the Question Index. These actions include:

• Group questions

• Set status

• Set course

• Set curriculum tags

  • This process will open the curriculum tag selector and show you the tags already assigned, which question is assign with which tag.

  • Tags added will be applied to every question selected via checkbox

  • You are able to un-assign tags via this method as well

• Reactivate retired questions

Migrating responses from ExamSoft into Elentra allows you to record learner performance on past exams delivered through ExamSoft in Elentra.

Export from ExamSoft

• Export your learners results from ExamSoft with the following settings as a CSV file.

• If your exam does not have essay questions, you can uncheck the Essay Responses category.

Importing Exam Taker Results

• From the main page, click to update any questions then proceed to the Posts section.

• Create a post of this exam and associate it with a learning event for the cohort that wrote it. (Review the exam posting section for more information).

• Once your questions have been imported, return to your Exam Dashboard through the Exam menu located in the collapsible left-hand side bar.

• Click on the title of your imported exam to open it.

• Go back to the Questions section and select the Migrate Responses tab.

• Browse for and select the CSV of the learner results and then click Import Responses.

• This can take a few minutes to complete if there are many questions and learners.

• Once the results have been imported, the post will be updated so learner results can be reviewed and reports can be generated for the exam.

If exam questions are stored in Word or other word processing software, users can copy and paste the information and bulk import questions to Elentra. Specific formatting is required. Curriculum tags and labels can be included in the file import.

Import Questions from Text File

1. Navigate to Admin > Manage Exams.

2. Click on ‘Migrate ExamSoft’ from the left sidebar Manage Exams section.

3. Click on the Import sub-tab.

4. Optionally select a label.

5. Select a default course.

6. Select a default assessment type.

7. Paste in the question(s) text. Note that specific formatting is required.

8. Click Import.

Required Formatting

To format your questions, begin with “type:” followed by a space and then one of the following question type codes:

• mc_h for Multiple Choice Horizontal

• mc_v for Multiple Choice Vertical

• short for Short Answer

• essay for Essay

• match for Matching text Text

• mc_h_m for Multiple Choice Horizontal (multiple responses)

• mc_v_m for Multiple Choice Vertical (multiple responses)

• fnb for Fill in the Blank

You can also indicate the following information for any question, although it is optional:

• description (this is short description or title of the question)

• rationale

• correct_text (this can be used for short answer or essay questions)

• code 


• locked response(s) - this allows you to lock a response option on a question if you intend to randomize question response options when you post the exam

• curriculum tags - this allows you to import curriculum tags with your questions.

  • To import curriculum tags you must know the curriculum tag id. Ids can be found in Admin > Manage Curriculum. When you open a curriculum tag set, toggle to the table view of curriculum tags (use icons beside 'Add Tag'), and if ID is not a visible field, add a column by clicking the columns selector (under the Import from CSV button) and adding the ID column.

Example Question Format:

1. Which are vowels? a. a b. e c. i d. All of the above answer: d type: mc_v locked: d curriculum_tags: 3454, 3421

Formatting Requirements for Additional Question Types

Each question shows the minimal amount of information required. It is recommended that you also provide description, rationale, etc.

Multiple Choice Vertical

Multiple choice vertical questions have a type of "mc_v". They include one or more answer choices, and an attribute "answer" for the letter of the correct answer. An example is shown below.

1. What is your favorite color? a. red b. green c. blue

answer: a type: mc_v

Multiple Choice Vertical (multiple responses)

Multiple choice vertical questions with multiple responses have a type of "mc_v_m". They include one or more answer choices, and one or more "answer" attributes for the letter(s) of the correct answer(s). An example is shown below.

1. Which of these are blue? a. sky b. ocean c. trees

2. Where is your favorite color? type: short

answer: a answer: b or answer: a, b type: mc_v_m

Multiple Choice Horizontal

Multiple choice horizontal questions follow the same format as multiple choice vertical, but with a type of "mc_h". They can have 5 answer choices at most. An example is shown below. Note that questions presented in horizontal view to learners will not have the strikethrough feature available when learners take an exam.

1. What is your favorite color? a. red b. green c. blue

answer: a type: mc_h

Multiple Choice Horizontal (multiple responses)

Multiple choice horizontal questions with multiple responses have a type of "mc_h_m". They include 1-5 answer choices, and one or more "answer" attributes for the letter(s) of the correct answer(s). An example is shown below. Note that questions presented in horizontal view to learners will not have the strikethrough feature available when learners take an exam.

1. Which of these are blue? a. sky b. ocean c. trees

answer: a answer: b or answer: a, b type: mc_h_m

Short Answer

Short answer questions have a type of "short" and require only the question stem although it is recommended to provide correct text and rationale. An example is shown below.

1. Where is your favorite color? type: short

Essay

Essay questions have a type of "essay" and require only the question stem although it is recommended to provide correct text and rationale. An example is shown below.

1. What is your favorite color and why (3-5 sentences)? type: essay

Matching

Matching questions have a type of "match". They have one or more answer choices, one or more item stems, and the same number of correct answers as item stems. The answer choices are shown as options for each item stem. The item stems are a piece of text or a question that corresponds to one of the answer choices. The correct answer for an item stem is identified by the answer choice's letter. The correct answer attributes are assumed to be in the same order as the item stem attributes. An example is shown below. Note that when learners answer this type of question the system does not automatically eliminate responses (i.e., a learner can put answer c as a match for multiple stems). When scoring a matching question the system uses an all or none system (for questions where you can give partial scores use multiple choice, multiple responses). An example is shown below.

1. This is the matching question stem. a. This is the first answer choice. b. This is the second answer choice. c. This is the third answer choice. item: This is the first item stem, its correct answer is choice C. answer: c item: This is the second item stem, its correct answer is choice A. answer: a type: match

Text

Text questions are are generally used as instruction text for further questions. Text questions have a type of "text" and require no other special attributes. An example is shown below.

1. The next three questions cover the following scenario: A patient arrives at your family medicine clinic at 2:30 on a Friday afternoon... type: text

Fill in the Blank

Fill in the blank questions have a type of "fnb". The blanks are indicated by the string "?" in the question stem. Fill in the blank questions can have an "answer" attribute for each blank indicating possible correct answers for that blank; the possible correct answers are separated by pipe characters, "|". These "answer" attributes refer to the blanks in order. An example is shown below, in which the first blank has possible correct answers of "woodchuck", "beaver", or "marmot", and the second blank has possible correct answers of "woodchuck", "hamster", and "groundhog". An example is shown below.

1. How much wood could a ? chuck if a ? could chuck wood? answer: woodchuck|beaver|marmot answer: woodchuck|hamster|groundhog type: fnb

Exam questions can be built through the user interface or can be imported. Elentra currently supports a variety of question types:

• multiple choice (with optional multiple responses)

• short answer

• essay

• matching

• text

• fill in the blank

Create Exam Questions

Users can add individual questions through the user interface, import questions in bulk through a text box, or migrate questions, images, and responses from another exam management tool (this option is only available to medtech:admin users). This section addresses how to create individual questions in the user interface; see here for additional information on importing questions from another exam tool.

Permissions

Currently only staff:admin can create questions by default. To grant a user the ability to view, edit or create questions, assign that user to a label or course.

The Question Management feature handles permissions via two methods: course association and label association.

Course Associations Any course contact will be able to access questions that are associated with that course.

• Note that Course Contact access to questions is controlled through the access control list (a database table in Elentra) so that it can be configured for an institution's need. For example, you could adjust things so that Associated Faculty have the ability to save drafts of questions/groups, but not publish them.

Label Associations Labels are built in Elentra and you can associate individuals with one or more labels. For each association you make, you can control whether the user can read or write questions, groups and exams (in this case groups refers to question groups).

Add an Exam Question

When you build a question through the user interface, you will walk through several tabs to add the required information.

Additional Exam Question Type Details

Essay

• Correct Text: Serves as a guide for graders and will be displayed to them when they are grading the exam.

• Maximum number of characters for essay format is set to 65,535.

• Currently essay questions and short answer questions are treated slightly differently in the Learner Feedback Report (available after an exam is completed). Essay questions will show the complete response, while short answer questions will be cut off.

Fill in the Blank

• Create a blank in the Question Text by typing "_?_"

• An answer field will be created for each blank that is detected. Add as many answers as are possible, separating answers with a comma.

• Click Add after you have entered the answer(s).

• Note that a question cannot start with a blank.

• Fill in the blank answers are case sensitive. You may wish to include two answers (e.g. cardiac arrest and Cardiac Arrest) to ensure learners are properly credited for their answers.

  • Graders can manually grade fill in the blank questions if needed.

Matching

• Add one list of terms as items.

• Add the corresponding list of terms as answers.

• For each item, pick the correct answer from the dropdown selector.

• When the learner completes the question, they'll see a list of items and choose an answer from an dropdown selector.

Multiple Choice

• Multiple choice questions presented in horizontal view will not have the strikethrough feature available when learners take an exam.

• Optionally set the toggle to indicate whether a learner is expected to provide multiple responses to a prompt.

• If multiple responses is enabled, complete the additional fields:

  • Grading Scheme

    • Partial: This will give partial credit for every correct reaction (e.g. selecting a correct answer AND not selecting an incorrect answer). By default, Elentra divides the total point value across the number of response options and awards credit for each response option which is correctly responded to (selected if correct, not selected if not correct).

    • All or Nothing: A learner must ace the question to score points. If they miss an answer (whether they select an incorrect answer or miss selecting a correct answer), the learner will get 0 points.

    • Partial with additional penalty: This will give partial credit for every correct reaction and will penalize a learner for an incorrect answer. Elentra divides the total point value across the number of response options and awards credit for each response option which is correctly responded to (selected if correct, not selected if not correct) and deducts credit for an incorrect answer. You can optionally change the value of each answer using the answer weight options.

  • Maximum number of responses

    • Define how many responses a learner will be permitted to enter.

  • Answer Weight

    • This optional feature allows you to support weighting different response options in partial or partial with additional penalty grading schemes.

    • Answers will be equally weighted by default. If you adjust their weights you can tailor the points awarded to a learner when partial and partial with additional penalty grading schemes are used.

  • Sample Use Cases

    • Assume a MC question has 2 correct responses and 2 incorrect responses where the correct responses are worth 40% each and the incorrect responses are worth 10% each. If a learner picks 1 correct response and 1 incorrect response, their score will be the following under each grading scheme:

    • Partial Grading Scheme: Learner scores 0.5 (+0.4 for identifying 1 correct answer, +0 for not selecting a correct response, +0 for selecting an incorrect response, +0.1 for not selecting an incorrect response)

    • All of Nothing Grading Scheme: Learner scores 0

    • Partial with additional penalty Grading Scheme: Learner scores 0 (+0.4 for identifying 1 correct answer, +0.1 for not selecting an incorrect answer, -0.4 for missing 1 correct answer, -0.1 for selecting an incorrect answer)

Short Answer

• Correct Text: Serves as a guide for graders and will be displayed to them when they are grading the exam.

• Currently essay questions and short answer questions are treated slightly differently in the Learner Feedback Report (available after an exam is completed). Essay questions will show the complete response, while short answer questions will be cut off.

Text

• Text questions are designed to allow an exam creator to insert information in an exam.

Copy an Exam Question

From the Setup or Details tab of an existing question you can optionally create a copy of that question.

When viewing the Question Text on the Details tab, the Question Text will be prefaced with "Copy of." The newly created copy will automatically be given draft status and will maintain the existing details and tags.

Batch Edit Exam Questions

When viewing the question index you can make changes to multiple questions at once to enhance productivity when using the Question Index. These actions include:

• Group questions

• Set status

• Set course

• Set curriculum tags

  • This process will open the curriculum tag selector and show you the tags already assigned, which question is assign with which tag.

  • Tags added will be applied to every question selected via checkbox

  • You are able to un-assign tags via this method as well

• Reactivate retired questions

Question Versions

Any time you make a change to a published question, a new version will be created. A new version will be created for any change—correcting a typo, linking the question to additional curriculum tags, altering a response option, etc.

When creating exams, you can add a question with multiple versions and choose to select an older version if desired. Do this by clicking on the forward/back arrow icon. If a question has changed since you created an exam, you will be given a notification for that and can update your exam to reflect the newest version of that question; however, you can still elect to use the old version of the question if you prefer. If the forward back arrow icon is blue, you’re not using the most recent version of a question on an exam.

Versions listed in the modal will let you preview the previous version to be better able to compare what changed. Reverting to a previous version will create a new version based on that previous version, thereby preserving the complete history of a question.

Creating a new version of a question is as easy as making a change on a question and publishing or saving that version as a draft.

Remove a Question from the Bank

Retired Questions

Retiring a question is a new type of system status for questions. It allows you to prevent a question from being attached to new exams or being used in new exam posts when attached to existing exams. These are the features of a retired question:

• Not eligible to any new exam

• Excluded from new posts of existing exams

• Has a comment field when retiring as to why something is being retired

• Filtered out from the question index unless you opt to filter to view retired questions

• Can reactivate retired questions and then will become available for use on exams again

  • Filter question bank to show retired questions

  • Check off the questions you want to reactivate

  • Click Reactivate (at bottom right)

Deleted Questions

Only a question not attached to any existing exams can be deleted, otherwise we recommend Retiring that question. Deleting a question will remove it from a Question Group.

In Elentra ME 1.23 and higher, labels replace folders. Like folders, labels can be associated with specific users to give those users access to questions assigned that label. More than one label can be assigned to a question which gives you maximum flexibility in identifying when questions from your bank are used (e.g. associated with a course and a specialty topic and something else as needed).

Label Index

The Label Index allows you to easily search and filter labels attached to exams & questions. You can decide which columns are visible (searchable).

Deleting a label will remove it from any associated questions and remove any permissions associated with that label.

Create New Labels

To create a label:

1. Give the Label a name

2. Optionally give the Label a description

3. Specify the users that should be given permissions for anything associated with this label

4. For each user set if they should have Read Only or Read and Write permissions.

The use of labels extends any existing ACL rules for a question. If a question has both a course and label applied Elentra first uses the course contacts table, if fails to authorize at course level, it will check label and grant access as appropriate.

If labels have conflicting permissions and one question is tagged with both, the permissions are cumulative (so a user will get their highest permissable access).

Migrating Questions from ExamSoft allows you to move exam questions from ExamSoft into Elentra for use on future exams.

Exporting Exams from ExamSoft

• Export the exam you would like to import from ExamSoft with the settings below.

• Save the document with no spaces in the title.

• Using LibreOffice, convert the file into a text and html file.

  • Note: Currently, only files converted using a Mac can be successfully imported.

Importing Exam Questions

• Navigate to Admin > Manage Exams.

• From the Manage Exams navigation box, click Questions.

• Click on the Migrate Q's tab.

• Select the question folder you would like the questions uploaded to.

• Then, browse for and select your plain text file and click Import Questions.

• This will take you to a new page where you can review all the questions that will be imported from the text file

• Note: The system will look at the ExamSoft ID's to ensure no duplicate questions are created if questions are used on more than one exam.

• Scroll to the bottom of the page and click Import All Questions to complete the process. This will automatically create an exam in the system that is only accessible via the Exams Dashboard.

Importing Images

• Move to the Migrate Images tab.

• Browse for and select your html document and all the images for that exam.

• Click Import Question Images.

• Review that the correct images have imported, then click Confirm Exam Import.

Reviewing Flagged Questions

If you tried to import any questions from ExamSoft which were NOT multiple choice you'll need to review them in the Flagged Questions tab. Similarly, questions with images attached will also appear here. Non multiple choice questions are flagged because the import system can't tell if they were supposed to be matching, fill in the blank, or essay type questions. By default they are given a type of essay but this will need to be changed for matching or fill in the blank question types.

• Click on the Flagged Q’s tab. On this tab, you will see all of the questions with images and any non multiple choice questions.

• Click the pencil icon to edit the format of any questions.

• If there are no edits to be made, click the flag to approve the question.

• Images will automatically import to the top of the question.

• To edit the image position easily, click Source from the text editor box.

• Copy and paste the source code to the correct place in the question.

Groups Index

The Question Groups interface mimics the features and experience of the Question Index. This includes the ability to search and filter by assessment type, language, course, exam, updated by author and updated date. You can also choose which columns are visible, thus searchable.

• Select the "Show Tags" button to view all the tags associated with Questions within this group.

• Select the "Show Associated Exams" button to view all the exams associated with Questions within this group.

• Select "Remove Selected Question Groups" to remove the group from the Index. Questions within that group will not be deleted.

Add Question Group