The assessment and evaluation module provides a way for learners to be assessed, especially in clinical learning environments, and a way for all users to evaluate courses/programs, faculty, learning events, etc. Any forms used for assessment and evaluation require form items (e.g., questions, prompts, etc.).

How to create single items for use on assessment and evaluation forms

• Note that you can copy existing items which may save time. To copy an existing item, click on the item and click 'Copy Item' which is beside the Save button.

• Navigate to Admin>Assessment & Evaluation.

• Click 'Items'.

• Click 'Add A New Item'.

• Complete the required information, noting that different item types will require different information. Common item details are below: Item Type: This list shows the item types supported by Elentra. Item codes will display when you use a list view of items and A complete list of item types is provided below. Item Text: This is what will show up on a form this item is added to. When you view items in the detail view you'll also see the item text. Item Code: This is an optional field. Item codes do display when you view items in a list and they are searchable. Some organizations apply their own coding system to their items, but another use case might be if you are importing items from another tool or vendor and they have a coding system you want to match. Rating Scale: Rating scales can be configured through the Scales tab within Assessment & Evaluation. First select a scale type and then select the specific scale. Selecting a rating scale will prepopulate the response categories for this item. In some item types you will also be required to add response text (e.g. multiple choice items) and that text will show up on the actual form. In other question types you may rely on just the response categories. Mandatory: Click this checkbox if this item should be mandatory on any forms it is added to. Allow comments: Click this checkbox to enable comments to be added when a user responds to this item. If enabled, you have several options to control commenting.

  • Comments are optional will allow optional commenting for any response given on this item.

  • Require comments for any response will require a comment for any response given on this item.

  • Require comments for a prompted response means that for any response where you check off the box in the Prompt column, a user will be required to comment if they select that response.

  Allow for a default value: If you check this box you will be able to select a default response that will prepopulate a response when this item is used on any form. Set a default response by clicking on the appropriate response line in the Default column.

• Depending on the question type, add or remove answer response options using the plus and minus icons.

• Depending on the question type, reorder the answer response options by clicking on the crossed arrows and dragging the answer response option into the desired order.

• Add curriculum tags to this item as needed.

• If you have access to multiple courses/programs, first use the course/program selector to choose the appropriate course/program which will limited the available curriculum tags to those assigned to the course/program. Click the down arrow beside the course selector and search for the course by beginning to type the course name. Click the circle beside the course name.

• Click through the hierarchy of tags as needed until you can select the one(s) appropriate for the item.

• As you add curriculum tags, what you select will be listed under the Associated Curriculum Tags section.

• Scroll back up and click 'Save'.

Item Permissions

When users create items, they automatically have permission to access and use those items. Users can optionally give permission for other individuals, courses, or organisations to access items. As much as possible, we recommend permissioning items to courses and organisations to reduce future workload to reassign items to individuals as you experience staff changes.

Item types

• Horizontal Multiple Choice (single response): Answer options will display horizontally on the form and the user can select one answer. Response text required; response category optional. Response descriptors provide another data point so you can potentially report on them in the future. They are metadata in MC questions whereas in a rubric they are displayed. Horizontal MC will let you assign the same response descriptors to multiple responses.

• Vertical Multiple Choice (single response): Answer options will display in a vertical list on the form and the user can select one answer. Response text required; response category optional.

• Drop Down (single response): answer options will display in a dropdown menu. Response text required; response category optional.

• Horizontal Multiple Choice (multiple responses): Answer options will display horizontally on the form and the user can select two or more answers. Response text required; response category optional.

• Vertical Multiple Choice (multiple responses): Answer options will display in a vertical list on the form and the user can select two or more answers. Response text required; response category optional.

• Drop Down (multiple responses): Answer options will display in a dropdown list that remains open and allows users to select multiple responses using the control or command and enter/return keys.

• Free Text Comments: Use this item type to ask an open ended question requiring a written response. (In ME 1.11 and lower you can not map a free text comment to a curriculum tag set.)

• Date Selector: Use this item type to ask a question to which the response is a specific date (e.g. What was the date of this encounter?)

• Numeric Field: Use this item type to ask a question to which the response is a numeric value (e.g. How tall are you?)

• Rubric Attribute (single response): Use this to create an item that relies on response categories as answer options. If you enter text in the response text area it will not show up to the user unless you create a grouped item. If you create a grouped item remember you need to use the same scale across all items to be grouped together. If you want a rubric item to display response text, create a grouped item with just one item included.

• Scale Item (single response): Use this to create an item that relies on response categories as answer options. If you enter text in the response text area it will not show up to the user unless you create a grouped item. If you create a grouped item remember you need to use the same scale across all items to be grouped together.

• Autocomplete (multiple responses): Use this item to allow users to search the option list in an autocomplete fashion and then make multiple selections. The response options will display in the order they were added to the item and when a user begins to type a response, the list of options will be filtered. The user can select more than one response as needed.

How to create grouped items

Creating a grouped item allows you to group items and guarantee that they appear together on forms. If you use the rubric attribute or scale item item types, creating a grouped item will create a rubric with common response categories (e.g. developing, achieved) and specific response text for each field (e.g. performed a physical exam with 1-2 prompts from supervisor, independently performed a physical exam). There is also the ability to Copy a Grouped Item which is next to the Create & Attach a New Item and Attach Existing Item(s) buttons. You can choose to create a new item linkage to keep all items as grouped or to create new individual items from the original grouped item.

• Navigate to Admin>Assessment & Evaluation.

• Click on the Items tab.

• Click on the Grouped Items sub-tab.

• Click 'Add A New Grouped Item'.

• Provide a grouped item name and select a rating scale type and then a rating scale. All items in the group will have the same response categories assigned to them, as configured through the rating scale. (Rating scales can be set up through the Scales tab in Admin>Assessment and Evaluation.)

• Click 'Add Grouped Item'.

• Complete the required information, noting the following: Title: This will display when you view a list of grouped items. Description: This field is optional; note that the grouped item description will display below the grouped item title on forms produced for users (see image below). Permissions: Adding a group, course, or individual here will give those users access the the grouped item.

• To add items to a grouped item you can either create and attach a new item or add existing items (click the appropriate button).

• If attaching existing items, use the search bar and filters to find items. You will only be shown items that match the rating scale parameters you've selected. Click the checkbox beside a question (in list view) or beside the pencil icon (in detail view) and click 'Attach Selected'. Because an existing item may already be in use on another form, in some cases you will not be able to modify the response descriptors for that item.

• If creating and attaching items, follow the instructions above for creating items. The rating scale for your new items will be set to match the rating scale of the grouped item. After creating one item, you can repeat the steps to create and attach as many items as needed.

• Click 'Save'.

• To edit an item click on the pencil icon. Bear in mind that an existing item may already be in use on another form.

• To delete an item from a grouped item, click on the trashcan icon.

• To reorder the items in the grouped item, click on the crossed arrows and drag the item into the appropriate location.

• When you have added all required items to the grouped item, click 'Save'.

• Click 'Grouped Items' at the top of the screen to return to the list of grouped items.

How to add permissions to existing items

• Navigate to Admin>Assessment and Evaluation.

• Click 'Items'.

• From list view, click on any item to open it. From grid view, click the pencil icon to edit an item.

• Give permission to an individual, organisation or course by first selecting the appropriate title from the dropdown menu, and then beginning to type in the search bar. Click on the desired name from the list that appears below the search bar. Giving permission to an entire organisation will allow anyone affiliated with the organisation through their user profile, AND with access to the Assessment and Evaluation module to use the item. If you give permission to an entire course, anyone listed on as a course contact on the setup page AND with access to the Assessment and Evaluation module will have access to the item.

• After you've added all permissions, you can return to the list of all items by clicking 'Items'.

Tips for viewing existing items

• Toggle between list view and detail view using the icons beside the search bar.

• In detail view, see the details of an existing item by clicking on the eye icon.

• In detail view, edit an existing question by clicking on the pencil.

• To delete items, check off the tick box beside a question (list view) or beside the pencil icon (detail view) and click 'Delete Items'.

• From an Edit Item page you can click on a link to view the forms that use an item or the grouped item an item is included in.

• When viewing items in list view, the third column shows the number of answer options the item has. Clicking on it takes you to the item, and by clicking again you can see all the forms that use this item.

How to search for existing items

• From the Items tab type into the search box to begin to find questions.

• You can apply a variety of filters to refine your search for existing items.

• To select a filter, click on the down arrow beside the search box. Select the filter type you want to use, click on it, and then begin to type what you want to find or continue clicking to drill down and find the required filter field. Filter options will pop up based on your search terms or what you’ve clicked through and you can check off the filters you want to apply. Apply multiple filters to further refine your search.

• If you're working with a filter with multiple hierarchies, use the breadcrumbs in the left corner of the filter list to go back and add additional filters.

• When you’ve added sufficient filters, scroll down and click Apply Filters to see your results.

• To remove individual filters from your search, click on the down arrow beside the search field, click a filter type and click on the small x beside each filter you want to remove. Scroll down and click ‘Apply Filters’ to apply your revised selections.

• To remove all filters from your search, click on the down arrow beside the search field, click a filter type, scroll down, and click on ‘Clear All’ at the bottom of the filter search window.

Retiring Items

Retiring items allows a user to manage archived and future or currently active items effectively. When users create forms they will not see retire items, however retired items will still be included in all relevant reports. Retired items will continue show in the item bank, but will have their item type highlighted red.

• Click the tick box on an item card.

• Click the orange Retire Items button.

• Confirm your action by clicking 'Retire'.

• The recently retired item will have the item type bar highlighted in red.

Deleting Items

Deleting an item will remove it from the list of Assessment and Evaluation Items.

• Click the tick box on an item card.

• Click the red Delete Items button.

• Confirm your action by clicking 'Delete'.

• The deleted item will no longer show on the list of items.

Please note that there is no way through the user interface to recover an accidentally deleted item. If you have deleted something by mistake you will need help from a developer to correct the mistake.

Deleted items will remain on forms that already include that item.

To effectively use the various assessment tools in Elentra ME some initial setup is required.

For the Assessment & Evaluation module specifically, a staff:admin or medtech:admin will need to configure assessment response categories in System Settings. For information about configuring assessment response categories (required to build scales), please see the System Setup help section. A medtech:admin will have to configure scales through Admin > Assessment & Evaluation (see next section).

Note that assessment characteristics (in system settings), are used in the the Gradebook module to define assessment types like written exam, test, quiz, project, etc. You do not need to build a list of assessment characteristics to use the Assessment & Evaluation module.

If you are in an organisation that is not using CBME, the form templates tab is unlikely to be something you will ever use. Form templates allow users to make forms more consistent and in the case of the Competency-Based Medical Education module, allow users to configure the requirements for multiple forms through one user interface. Administrative staff specify EPAs and milestones, contextual variables, and rating scales to be used on assessment forms for clinical environments, and these forms get created after a form template is published. If you are using CBME please see here to learn more about using form templates.

Form templates are built off of something called a form blueprint and there is currently no way through the user interface to configure a form blueprint. If you want different form templates from those in a default installation of Elentra, you'll need a developer's help.

There are multiple tools available to facilitate assessment and evaluation through Elentra.

• To create items and forms to assess learner performance on tasks (e.g., clinical skills and workplace based assessment), use the Assessment and Evaluation Module.

• To create items and forms to have learners evaluate courses, faculty, themselves, and other activities, use the Assessment and Evaluation Module.

• To create assessment items to use in small, low-stakes assessments use the Exams Module. (Elentra does include a Quizzes Module but we recommend you use the much newer Exams Module instead.)

• To create assessment items to use for tests and exams use the Exams Module.

• To store learner grades and facilitate faculty grading of assignments use the Gradebook feature in each course. (To allow inline faculty grading using a rubric form or similar, please note that you will build the form in Assessment and Evaluation, then attach it to a Gradebook Assessment. Only specific item types are supported on forms attached to gradebook assessments.)

Elentra ME also includes a portfolio, which some organizations may choose to use for student assessment purposes. The portfolio can be connected to a course gradebook. For more detail about the portfolio please see the Portfolio help section.

Assessment & Evaluation Module

The Assessment & Evaluation Module in Elentra is predominantly used to assess learner performance in a clinical environment, and provide multiple user groups with the ability to evaluate faculty, courses/programs, and other activities within the context of your organization. Since the Assessment and Evaluation Module essentially allows you to create forms for people to complete you could even use it to do a pre-course survey, or as a way to collect information from a group about their work plans in a collaborative activity.

To use the Assessment & Evaluation Module users must create items (e.g., questions, prompts, etc.), and create forms (a collection of items). Organizations can optionally allow users to initiate specific forms based on a form workflow (e.g. to allow learners to initiate an assessment on themselves by a faculty member), or organizations can create tasks for individuals to complete using a distribution. A distribution defines who will complete a form, when the form will be completed, and who or what the form is about.

The Assessment and Evaluation Module users also have a quick way to view their assigned tasks and administrators can monitor the completion of tasks assigned to others.

Reporting exists to view and in some cases export the results of various assessments and evaluations.

The Competency-Based Medical Education module also includes a variety of form templates for use. There is no user interface to configure form templates at this point. For instructions specific to Competency-Based Medical Education, please see the CBME tab.

In this context, we use the term off-service to mean learners who are completing rotations outside their home program (e.g. a family medicine resident on a pediatrics rotation).

Clinical Experiences > Rotation Scheduler includes support for off-service rotations or slots. When an administrator builds rotations she can optionally create off-service slots and make them available to all other programs or specific programs. The following information assumes you are an organization using off-service slots and refers to rotation-based distributions. (This page does not discuss automated rotation evaluations nor rotation evaluations completed on demand via workflows.)

As the home program you can:

• Create assessment tasks targeting your learners while they are on an off-service rotation.

• Report on assessment tasks targeting your learners while they were on an off-service rotation.

  • For distributions you created, you can view results via the Learner Assessments report or the Learner Reports (Aggregated).

  • For distributions created by host programs, but targeting your learners, you can only view results via Learners Reports (Aggregated).

• Create evaluation tasks asking your learners to evaluate their experience on an off-service rotation.

• View individual responses and download a Weighted CSV of your learners' responses to an evaluation of their off-service experience (do this via the Completed Distribution Report); up to and including ME 1.17 a home program cannot use the Rotation Evaluation (Aggregated) Report to see an overview of their learners' experience on an off-service rotation (whether the distribution was set up by the home or host program).

As the host program you can:

• Create assessment tasks targeting visiting, off-service learners while they are on your rotation.

• Report on assessment tasks targeting visiting, off-service learners while they were on your rotation.

• Create evaluation tasks asking visiting, off-service learners to evaluate their experience on your rotation.

• Report on evaluation tasks completed by visiting learners to your rotation, as long as you created the distribution. If the home program created an evaluation asking their learners to evaluate your rotation or faculty, you will not be able to access the results.

A date range distribution allows you to send a form to the appropriate assessors/evaluators in a specific date range.

Add New Distribution

• Navigate to Admin>Assessment and Evaluation.

• Click 'Distributions' above the Assessment and Evaluation heading.

• Click 'Add New Distribution'.

Form

• Distribution Title: Provide a title. This will display on the list of Distributions that curriculum coordinators and program coordinators can view.

• Distribution Description: Description is optional.

• Task Type: Hover over the question mark for more detail about how Elentra qualifies assessments versus evaluations. If a distribution is to assess learners, it's an assessment. If it is to evaluate courses, faculty, learning events, etc. it is an evaluation. Notice that the language on Step 4 will change if you switch your task type as will other steps of the wizard.

• Disable Target Release: This will only display for Evaluations. If you do not want evaluators to be able to release the form to the target upon completion, check this box. If left unchecked, evaluators will be able to release the form to the target upon completion.

• Assessment/Evaluation Mandatory: This will be checked off be default.

• Disable Reminders: Check this box to exclude this distribution from reminder notification emails.

• Select Form: The form you want to distribute must already exist; pick the appropriate form.

• Select a Curriculum Period: The curriculum period you select will impact the list of available learners and associated faculty.

• Select a Course: The course you select will impact the list of available learners and associated faculty.

• Click 'Next Step'

Method

• Distribution Method: Select 'Date Range Distribution' from the dropdown menu.

• Start Date: This is the beginning of the period the form is meant to reflect.

• End Date: This is the end of the period the form is meant to reflect.

• Delivery Date: This is the date the task will be generated and delivered to the assessors/evaluators. (The delivery date will default to be the same as the start date.)

• Task Expiry: Optional. Set the date on which the tasks generated by the distribution will automatically expire (i.e. disappear from the assessor's task list and no longer be available to complete). Tasks will expire at 12:00 AM on the day that you select, so it is best to select the day after your intended expiry date.

• Warning Notification: If you choose to use the Task Expiry option, you'll also be able set an automatic warning notification if desired. This will send an email to assessors a specific number of days and hours before the task expires.

Targets

The target is who or what the form is about.

• Assessments delivered for: Use this area to specify the targets of the form.

• If you choose "Select learners," you'll see the following options:

  • Select Targets:

    • Use the dropdown menu to select the appropriate learners

  • CBME Options: This option applies only to schools using Elentra for CBME. Ignore it and leave it set to non CBME learners if you are not using CBME. If you are a CBME school, this allows you to apply the distribution to all learners, non CBME learners, or only CBME learners as required.

• Target Attempt Options: Specify how many times an assessor can assess each target, OR whether the assessor can select which targets to assess and complete a specific number (e.g. assessor will be sent a list of 20 targets, they have to complete at least 10 and no more than 15 assessments but can select which targets they assess).

  • If you select the latter, you can define whether the assessor can assess the same target multiple times. Check off the box if they can.

• Click 'Next Step'

Assessors

The assessors are the people who will complete the form.

• There are three options:

  • Select faculty members

    • Browse faculty and click on the required names to add them as assessors

    • Select Associated Faculty will add the names of all faculty listed as course contacts on the course setup page

    • Exclude self-assessments: If checked, this will prevent the assessor from completing a self-assessment

    • Feedback Options: This will add a default item to the distribution asking if the faculty member met with the trainee to discuss their assessment.

  • Select learners

    • Click Browse Assessors and click on the appropriate option to add learners as assessors

    • Exclude self-assessments: If checked, this will prevent the assessor from completing a self-assessment

  • Select individuals external to the installation of Elentra

    • This allows you to add external assessors to a distribution

    • Begin to type an email, if the user already exists you'll see them displayed in the dropdown menu. To create a new external assessor, scroll to the bottom of the list and click 'Add External Assessor'

    • Provide first and last name, and email address for the external assessor and click 'Add Assessor'

    • Feedback Options: This will add a default item to the distribution asking if the faculty member met with the trainee to discuss their assessment.

• Click 'Next Step'

Results

You can immediately save your distribution at this point and it will generate the required tasks, but there is additional setup you can configure if desired.

• Authorship: This allows you to add individual authors, or set the distribution to be accessible to everyone with A+E access in a course or organization. (This may be useful if you have multiple users who manage distributions or frequent staffing changes.)

• Target Release: These options allow you to specify whether the targets of the distribution can see the results of completed forms.

  • Task List Release:

    • "Targets can view tasks completed on them after meeting the following criteria" can be useful to promote completion of tasks and is often used in the context of peer assessments. Targets will only see tasks completed on them after they have completed the minimum percentage of their tasks set by you.

  • Target Self-Reporting Release: This controls whether targets can run reports for this distribution (i.e. to generate an aggregated report of all responses). When users access their own A+E they will see a My Reports button. This will allow them to access any reports available to them.

• Target Self-Reporting Options: This allows you to specify whether or not comments included in reports are anonymous or identifiable. (This will only be applied if you have set reports to be accessible to the targets.)

• Reviewers: This allows you to set up a reviewer to view completed tasks before they are released to the target (e.g. a staff person might review peer feedback before it is shared with the learner).

  • Check off the box to enable a reviewer.

  • Click Browse Reviewers and select a name from the list. Note that this list will be generated based on the course contacts (e.g. director, curriculum coordinator) stored on the course setup page.

• Prompted Response Notifications: This allows you to decide what action to take for any answers on the form that designated as prompted or flagged response options in items included on the form. (For example, if you have an item asking about student completion of a procedure and "I had to do it" was set as a prompted/flagged response.) You can optionally select to email the Assessment Reviewers, Program Coordinators, Program/Course Directors, or Distribution Authors.

• Click 'Save Distribution'.

A distribution based on a learning event schedule allows you to schedule forms to be sent out where the targets of the distribution are the attendees of the event OR faculty who taught the event OR a specific event within the selected event type.

Add New Distribution

• Navigate to Admin>Assessment and Evaluation.

• Click 'Distributions' above the Assessment and Evaluation heading.

• Click 'Add New Distribution'.

Form

• Distribution Title: Provide a title. This will display on the list of Distributions that curriculum coordinators and program coordinators can view.

• Distribution Description: Description is optional.

• Task Type: Hover over the question mark for more detail about how Elentra qualifies assessments versus evaluations. If a distribution is to assess learners, it's an assessment. If it is to evaluate courses, faculty, learning events, etc. it is an evaluation. Notice that the language on Step 4 will change if you switch your task type. The task type will also dictate other fields available throughout the wizard.

• Disable Target Release: This will only display for Evaluations. If you do not want evaluators to be able to release the form to the target upon completion, check this box. If left unchecked, evaluators will be able to release the form to the target upon completion.

  Assessment/Evaluation Mandatory: This will be checked off be default.

  Disable Reminders: Check this box to exclude this distribution from reminder notification emails.

• Select Form: The form you want to distribute must already exist; pick the appropriate form.

• Select a Curriculum Period: The curriculum period you select will impact the list of available learners and associated faculty.

• Select a Course: The course you select will impact the list of available learners and associated faculty.

• Click 'Next Step'

Method

• Distribution Method: Select 'Learning Event Schedule' from the dropdown menu.

• Select Event Type: Select the appropriate event type(s) from the dropdown menu. You can select multiple event types as needed.

• Select Events: Specify one or more events you would like to include. If you do not specify individual events the distribution will send for all events of the indicated event type.

• Release Date: This tells the system how far back on the calendar to go when creating tasks. Hover over the question mark for more detail.

• Task Expiry: If you check this box, the tasks generated by the distribution will automatically expire (i.e. disappear from a user's task list and no longer be available to complete). You can customize when the task will expire in terms of days and hours after the delivery.

  • Warning Notification: If you choose to use the Task Expiry option, you'll also be able to turn on a warning notification if desired. This can be set up to send an email a specific number of days and hours before the task expires.

Targets

The target is who or what the form is about.

If you are creating an Evaluation:

• Evaluations delivered for:

  • Faculty who taught events with the selected event types (this will generate an evaluation for any faculty member who taught the event type specified in the Method section, e.g. lecture, lab, etc.)

  • Events with the selected event types (this will generate an evaluation for any event of the event type specified in the Method section)

• Target Attempt Options: Specify how many times an evaluator can evaluate each target, OR whether the evaluator can select which targets to assess and complete a specific number (e.g. evaluator will be sent a list of 20 targets, they have to complete at least 10 and no more than 15 evaluations but can select which targets they evaluate).

  • If you select the latter, you can define whether the evaluator can evaluate the same target multiple times. Check off the box if they can.

  • Click 'Next Step'

If you are creating an Assessment:

• Assessments delivered for: Currently, the only option here is attendees who are enrolled in events with the specified event type.

• Target Attempt Options: Specify how many times an assessor can assess each target, OR whether the assessor can select which targets to assess and complete a specific number (e.g. assessor will be sent a list of 20 targets, they have to complete at least 10 and no more than 15 assessments but can select which targets they assess).

  • If you select the latter, you can define whether the assessor can assess the same target multiple times. Check off the box if they can.

• Click 'Next Step'

Assessors/Evaluators

The assessors/evaluators are the people who will complete the form.

• There are three options:

  • Assessors/evaluators are attendees enrolled in the event

    • Attendee Options (note that you must be using Elentra's attendance module within events to use this feature)

      • Send to all enrolled audience attendees that attended the event

      • Sent to all enrolled attendees, even if they did not attend

  • Assessors/evaluators are faculty members associated with the event

    • This will send the distribution to the faculty listed on the event (e.g. teacher, tutor, etc.)

  • Assessors/evaluators are external to the installation of Elentra

    • This allows you to add external assessors/evaluators to a distribution

    • Begin to type an name or email, if the user already exists you'll see them displayed in the dropdown menu. To create a new external user, scroll to the bottom of the list and click 'Add External Assessor/Evaluator'

    • Provide first and last name, and email address for the external assessor and click 'Add Assessor/Evaluator'

• Click 'Next Step'

Results

You can immediately save your distribution at this point and it will generate the required tasks, but there is additional setup you can configure if desired. (Not all options will display depending on the other parameters of the distribution.)

• Authorship: This allows you to add individual authors, or set the distribution to be accessible to everyone with A+E access in a course or organization. (This may be useful if you have multiple users who manage distributions or frequent staffing changes.)

• Target Release: These options allow you to specify whether the targets of the distribution can see the results of completed forms.

  • Task List Release:

    • "Targets can view tasks completed on them after meeting the following criteria" can be useful to promote completion of tasks and is often used in the context of peer assessments. Targets will only see tasks completed on them after they have completed the minimum percentage of their tasks set by you.

  • Target Self-Reporting Release: This controls whether targets can run reports for this distribution (i.e. to generate an aggregated report of all responses). When users access their own A+E they will see a My Reports button. This will allow them to access any reports available to them.

• Target Self-Reporting Options: This allows you to specify whether or not comments included in reports are anonymous or identifiable. (This will only be applied if you have set reports to be accessible to the targets.)

• Reviewers: This allows you to set up a reviewer to view completed tasks before they are released to the target (e.g. a staff person might review peer feedback before it is shared with the learner).

  • Check off the box to enable a reviewer.

  • Click Browse Reviewers and select a name from the list. Note that this list will be generated based on the course contacts (e.g. director, curriculum coordinator) stored on the course setup page.

• Prompted Response Notifications: This allows you to decide what action to take for any answers on the form that designated as prompted or flagged response options in items included on the form. (For example, if you have an item asking about student completion of a procedure and "I had to do it" was set as a prompted/flagged response.) You can optionally select to email the Assessment Reviewers, Program Coordinators, Program/Course Directors, or Distribution Authors.

• Click 'Save Distribution'.

When you create assessment and evaluation items you will have the option of applying rating scales to certain item types; creating rating scales promotes consistency across items and can be a time saver for the administrative staff creating items and forms.

You must be a medtech:admin user to manage rating scales. Scales can have permissions configured if needed (see end of page).

For additional detail about rating scales in CBME, please see here.

How to create new rating scales

• Navigate to Admin>Assessment and Evaluation.

• Click 'Scales' from the A&E tabs list. Any existing rating scales will be displayed.

• Click the green 'Add Rating Scale' button.

• Complete the required information, noting the following: Title: Title is required and is what users will see when they build items and add scales so make it clear. Description: This is optional and is not often seen though the platform. Rating Scale Type: This defines the type of rating scale you are creating. Later, if you add rating scales to items, or add standard scales to form templates, you will first have to select a scale type. There is no user interface to configure rating scale types. Sample scale types include generic, global rating scale, milestone scale, etc. In a default Elentra installation you'll likely just see a default scale type. (In installations with CBME enabled you'll see global rating and milestone/enabling competency scales.)

• Add or remove response categories by clicking the plus and minus icons.

• For each response category, select a descriptor (these are configured through the assessment response categories). Note that you can search for descriptors by beginning to type the descriptor in the search box.

• To edit an existing rating scale click on the scale title, make changes as needed, and click 'Save'.

• To delete a rating scale click the checkbox beside the rating scale and click 'Delete Rating Scale'.

Once scales are created, they will become visible options when creating items and using some form templates.

How to set permissions for a scale

Setting permissions for a scale dictates which users will be able to access a scale when they create assessment and evaluation items. For example, if you set a scale's permissions to Undergraduate Medicine, all users with access to Admin > A & E in the undergraduate organisation will be able to use the scale when creating items. If you set a scale's permissions to several individual users, only those users will be able to access the scale when creating items.

You must create a scale before you can edit the permissions for it. After a scale is created you will automatically be redirected to the edit page.

• In the Rating Scale Information section, look for the Scale Permissions heading.

• Select Individual, Organisation, or Course from the dropdown options.

• Type in the search bar to find the appropriate entity.

• Click on the entity name to add it to the permission list.

• Add as many permissions as required.

• Scroll down and click 'Save'.

The pdf below provides a sample view of all item types that can be included on an assessment and evaluation form (please note this does not include some items specific to forms built from CBME templates). Some items are designed to be used online (e.g. autocomplete multiselect). To view sample items in use in an online environment, check out your Elentra demo site.

A form is a collection of items used to assess a learner, or evaluate a faculty member, event, rotation, course or anything else in your organisation. Forms can be created for specific courses, or for use across an entire organization. When building a form administrators can optionally indicate a form workflow which allows Elentra users to initiate a form on-demand (e.g. for the purposes of clinical assessment).

This help section is about creating forms without CBME templates. See for information about creating CBME-specific forms.

Form Types

Elentra supports several types of forms.

• Generic Form

  • Users can add any items to this form and it will be immediately available for us.

• Standard Rotation Evaluation Form

  • Only use this form type if you are also using the Clinical Experience rotation scheduler.

  • Organizations can optionally enable this form type if they plan to use a templated rotation evaluation form to make available to learners. This form includes several templated questions and must be published before use.

  • You will require developer assistance to enable this form for your organization and to customize the templated components to include the items you want.

• Standard Faculty Evaluation Form

  • Organizations can optionally enable this form type if they plan to use a templated faculty evaluation form to make available to learners. This form includes several templated questions and must be published before use.

  • You will require developer assistance to enable this form for your organization and to customize the templated components to include the items you want.

Form Permissions

Any individual given permission to a form will be able to edit it until it is used in a distribution.

You can also grant permission on a form to a course. If you do this, any user who is a course contact for that course and who also has permission to access Admin > Assessment & Evaluation, will be able to access the form and include it in a distribution if needed.

How to create forms

• Navigate to Admin>Assessment & Evaluation.

• Click 'Forms'.

• Click 'Add Form'.

• Provide a form name and select a type (if applicable). See list of form types above.

• Depending on the form type selected, you may be required to identify a course/program.

• Click 'Add Form'.

• Provide a form description if desired. This will display to users when the form is accessed.

• Indicate the form workflow if your organization uses workflows (i.e. allows users to initiate specific forms on-demand).

• Set form permissions to give other users access to this form. Anyone given permission to the form will be able to edit it until it is used in a distribution.

  • Please note that forms using a workflow must be permissioned to the appropriate course.

  • Additionally, we strongly recommend permissioning forms to at least a course so that staffing changes are simplified. (If Jane Doe is the only person with access to a form and she retires, you'll need to manually reassign all her forms to a new user. If the form is permissioned to a course, any course contact with access to Assessment & Evaluation will be able to access the form.)

• Click 'Add Item(s)' to add existing items.

• Search for existing items and tick off the check boxes, then click 'Attach Selected' to apply your choices.

• You can also add grouped items, free text (e.g., to provide instructions), or a curriculum tag set to your form. To add any of these, click on the down arrow beside Add Items. If you choose to add Free Text or a Curriculum Tag Set, please note that you must save your choices within the item box using the small 'Save Required' button.

  • Note: Adding a Curriculum Tag Set is a very specific tool that supports field notes for use in family medicine. Most users should ignore this option.

• To create new items while creating your form, click Add Items and then click Create & Attach a New Item. When you complete your new item and save it, you will be returned to the form you in the process of building.

• Save the form when you have added all the relevant items.

• To preview your form, by clicking on the eye icon/Preview Form button.

• To download a copy of the form, use the Download PDF button.

How to manage forms (delete items, rearrange items, etc.)

• To delete items on a form, tick off the box on the item card and then click the red Delete button on the left.

• To rearrange items on a form, click the crossed arrow icon on the item card and drag the item to where you want it to be.

• To edit an item, click on the pencil icon on the item card. Note that an item already in use on a form that has been distributed will not be able to be edited. Instead you must copy and attach a new version of the item to edit and use it.

• To quickly view the details of an item, click on the eye icon on the item card.

How to copy an existing form

• Navigate to Admin>Assessment & Evaluation.

• Click 'Forms'.

• Use the search bar to look for the form you want to copy. Click the down arrow beside the search bar to apply filters to refine your search results.

• Click on the name of the form you want to copy.

• Click 'Copy Form' and provide a new name for the copied form.

• Click 'Copy Form'.

• Edit the form as needed (e.g., add additional items, change permission, etc.).

  • If you edit an item on a form and that item is in use on other forms, you will affect all of the associated forms. You can optionally view all forms that include the item.

    • For grouped items you can optionally copy and attach the grouped item to the form allowing you to change it as needed.

      • Create new item linkage

      • Create new items

    • For single items you can optionally copy the item to edit it. This will create a brand new item with no connection/link to the item it is copied from.

• Click 'Save'.

How to retire an existing form

Retiring a form means it will remain available in existing distributions, and reports, but will not be available for any new distribution.

• Navigate to Admin>Assessment & Evaluation.

• Click 'Forms'.

• Use the search bar to look for the form you want to retire. Click the down arrow beside the search bar to apply filters to refine your search results.

• Tick off the box beside the form name (you can select multiple forms to retire at once), and then click the orange Retire Form button.

• You will be prompted to confirm your action. Click 'Retire'.

• Retired forms will display with a red highlight around them.

How to delete an existing form

Deleting a form means that all pending and in-progress tasks that used that form will not have a form associated with them and will display an error message stating that the form has been deleted when an assessor/evaluator tries to access the form.

• Navigate to Admin>Assessment & Evaluation.

• Click 'Forms'.

• Use the search bar to look for the form you want to delete. Click the down arrow beside the search bar to apply filters to refine your search results.

• Tick off the box beside the form name (you can select multiple forms to delete at once), and then click the red Delete Form button.

Rotation Based Distributions allow you to set up a distribution based on a rotation schedule. This means you can easily send a form to all enrolled learners to be delivered when they are actively in the rotation. Note that you must have rotations built using the Clinical Experience Rotation Scheduler to use this distribution method.

Add New Distribution

• Navigate to Admin>Assessment and Evaluation.

• Click 'Distributions' above the Assessment and Evaluation heading.

• Click 'Add New Distribution'.

Form

• Distribution Title: Provide a title. This will display on the list of Distributions that curriculum coordinators and program coordinators can view.

• Distribution Description: Description is optional.

• Task Type: Hover over the question mark for more detail about how Elentra qualifies assessments versus evaluations. If a distribution is to assess learners, it's an assessment. If it is to evaluate courses, faculty, learning events, etc. it is an evaluation. Notice that the language on Step 4 will change if you switch your task type.

• Disable Target Release: If you do not want assessors/evaluators to be able to release the form to the target upon completion, check this box. If left unchecked, assessors/evaluators will be able to release the form to the target upon completion.

• Assessment/Evaluation Mandatory: This will be checked off be default.

• Disable Reminders: Check this box to exclude this distribution from reminder notification emails.

• Select Form: The form you want to distribute must already exist; pick the appropriate form.

• Select a Curriculum Period: The curriculum period you select will impact the list of available learners and associated faculty.

• Select a Course: The course you select will impact the list of available learners and associated faculty.

• Click 'Next Step'

Method

• Distribution Method: Select 'Rotation Distribution' from the dropdown menu.

• Rotation Schedule: Select the appropriate rotation schedule from the dropdown menu.

• Release Date: This tells the system how far back on the calendar to go when creating tasks. Hover over the question mark for more detail.

• Delivery Type:

  • Basic delivery (single): allows you to send a single evaluation task repeatedly, once per block, or once per rotation.

  • Dynamic delivery rules (multiple): allows you to deliver multiple evaluation tasks at specific intervals during a rotation. For example, if you have a 5-month rotation, and you’d like to deliver an interim evaluation at the 2-month mark and the 4-month mark, you can set a rule for the distribution to do so.

• If you select Basic delivery (single), then select your Delivery Period:

  • Choose between delivering tasks repeatedly, once per block, or once per rotation.

    • For each option additional customization allows you to control the timing of the distribution (e.g. 1 day after the start of the block, or 3 days before the end of the rotation).

    • Note: If a learner's schedule has two or more of the same rotation scheduled in a row, the system will treat them as a single rotation.

• If you select Dynamic deliver rules (multiple), then set your delivery rules.

  • Identify the specific delivery rules depending on the length of a rotation (you identify the length of the rotation in months, weeks, or days).

  • If you enter an integer larger than 1 for the number of tasks, you will be able to define when each task will be delivered (percent of the way through the rotation) and a visualization will be supplied by the Task bar below.

  • Clicking on Add new delivery will allow you to add additional delivery schedules that depend on the length of the rotation, which allows you to set different parameters for shorter rotations and longer rotations.

  • As noted in the interface, “larger” rules take precedence over smaller ones. So if you had a 0-1 month rule and a 0-2 month rule, the 0-2 month would supersede the other.

  • Remember: if a learner's schedule has two or more of the same rotation scheduled in a row, the system will treat them as a single rotation.

  • You can have as many rules as you want, but they must be contiguous (e.g., 0-2 months, 2-4 months, 4-6 months) to avoid leaving learners without an assessment.

Examples of Dynamic Delivery Scenarios:

• If you would like two interim assessment tasks delivered for any rotation 5 to 8 months in length, you could identify that for a 5 – 8-month rotation, task one should be delivered at 40% of the way through the rotation and task 2 should be delivered at 80% of the way through the rotation. Then, for a learner on a 5-month rotation, the interim task would be delivered at 2 months into the rotation and again at 4 months into the rotation, for example.

• If you would like three tasks delivered for any rotation 3 – 6 months in length, you could set task one at 25% through the rotation, task 2 at 50% through the rotation, and task 3 at 75% through the rotation, which would represent delivery of the task at ¼ of the way through the rotation, 1/2 the way through the rotation, and ¾ of the way through each 3-month, 4- month, 5-month, or 6-month rotation that a learner might be scheduled on.

• Task Expiry: If you check this box, the tasks generated by the distribution will automatically expire (i.e., disappear from the assessor's task list and no longer be available to complete). You can customize when the task will expire in terms of days and hours after the delivery.

• Warning Notification: If you choose to use the Task Expiry option, you'll also be able to turn on a warning notification if desired. This can be set up to send an email a specific number of days and hours before the task expires.

Targets

The target is who or what the form is about.

• Assessments delivered for: Use this area to specify the target of the form.

• If you choose "The targets for this Distribution are learners," you'll see the following options:

  • Assessments will be delivered for:

    • The targets for this Distribution are the assessors (self assessment)

    • The targets for this Distribution are learners

      • All learners in this rotation: Additionally specify whether to include learners from your program, and/or outside your program

      • Additional Learners: Check this off and then use the drop down selector to add the required learners. (Hover over a learner name to see their profile information.)

      • Specific learners in this rotation: Use the drop down selector to add the required learners. (Hover over a learner name to see their profile information.)

    • The targets for this Distribution are peers

      • This option allows for targets who have completed the rotation to assess all of their peers in the rotation block. Peer evaluation is one of the main types of resident feedback during residency.

  • CBME Options: This option applies only to schools using Elentra for CBME. Ignore it and leave it set to non CBME learners if you are not using CBME. If you are a CBME school, this allows you to apply the distribution to all learners, non CBME learners, or only CBME learners as required.

• Target Attempt Options: Specify how many times an assessor can assess each target, OR whether the assessor can select which targets to assess and complete a specific number (e.g. assessor will be sent a list of 20 targets, they have to complete at least 10 and no more than 15 assessments but can select which targets they assess).

  • If you select the latter, you can define whether the assessor can assess the same target multiple times. Check off the box if they can.

• Click 'Next Step'

Assessors

The assessors are the people who will complete the form.

• There are three options:

  • Assessors are learners

    • Learner Options: All Learners of Specific Learners

      • All Learners: Select all learners in the rotation or specific learners in the rotation

        • All learners: Select from My Program and/or Outside of my program, use the drop down selector to add additional learners

        • Specific learners in the rotation: Use the drop down selector to add required learners

  • Assessors are faculty members

    • Browse faculty and click on the required names to add them as assessors

    • Select Associated Faculty: This tool will pull the names of faculty listed on the course setup page as associated faculty

    • Feedback Options: This will add a default item to the distribution asking if the faculty member met with the trainee to discuss their assessment.

  • Assessors are external to the installation of Elentra

    • This allows you to add external assessors to a distribution

    • Begin to type an email, if the user already exists you'll see them displayed in the dropdown menu. To create a new external assessor, scroll to the bottom of the list and click 'Add External Assessor'

    • Provide first and last name, and email address for the external assessor and click 'Add Assessor'

    • Feedback Options: This will add a default item to the distribution asking if the faculty member met with the trainee to discuss their assessment.

  • Assessors are preceptors associated with a rotation schedule

    • This allows you to associated this distribution with all preceptors affiliated with the rotation based on the slot bookings made for learners. The distribution will dynamically update based on changes made to the rotation schedule.

    • Feedback Options: This will add a default item to the distribution asking if the faculty member met with the trainee to discuss their assessment.

• Click 'Next Step'

Results

You can immediately save your distribution at this point and it will generate the required tasks, but there is additional setup you can configure if desired.

• Authorship: This allows you to add individual authors, or set the distribution to be accessible to everyone with A+E access in a course or organization. (This may be useful if you have multiple users who manage distributions or frequent staffing changes.)

• Target Release: These options allow you to specify whether the targets of the distribution can see the results of completed forms.

  • Task List Release:

    • "Targets can view tasks completed on them after meeting the following criteria" can be useful to promote completion of tasks and is often used in the context of peer assessments. Targets will only see tasks completed on them after they have completed the minimum percentage of their tasks set by you.

  • Target Self-Reporting Release: This controls whether targets can run reports for this distribution (i.e. to generate an aggregated report of all responses). When users access their own A+E they will see a My Reports button. This will allow them to access any reports available to them.

• Target Self-Reporting Options: This allows you to specify whether or not comments included in reports are anonymous or identifiable. (This will only be applied if you have set reports to be accessible to the targets.)

• Reviewers: This allows you to set up a reviewer to view completed tasks before they are released to the target (e.g. a staff person might review peer feedback before it is shared with the learner).

  • Check off the box to enable a reviewer.

  • Click Browse Reviewers and select a name from the list. Note that this list will be generated based on the course contacts (e.g. director, curriculum coordinator) stored on the course setup page.

• Prompted Response Notifications: This allows you to decide what action to take for any answers on the form that designated as prompted or flagged response options in items included on the form. (For example, if you have an item asking about student completion of a procedure and "I had to do it" was set as a prompted/flagged response.) You can optionally select to email the Assessment Reviewers, Program Coordinators, Program/Course Directors, or Distribution Authors.

• Click 'Save Distribution'.

A distribution defines who will complete a form, who/what the form is about, and when the form will be completed. Usually, administrative staff manage distributions of forms to cohorts of learners, faculty, event participants, etc. Generally, the following language is used in the A&E module:

Assessor/Evaluator: The person completing a form/task Target: The person, course, experience or other that the form/task is about Distribution method: How the form/task is assigned to people (e.g. based on a date range, rotation schedule, event schedule, etc.)

The available distribution methods and some of their use cases are listed below:

Rotation Schedule: Use a rotation schedule distribution to send forms based on a rotation schedule. This could include assessments of learners participating in a rotation, evaluation by learners of a specific rotation site or experience, etc.

Delegation: Use a delegation to schedule a distribution to be sent to an intermediate person who will later forward the tasks to the appropriate assessor/evaluator. For example, if you want to set up distributions in September, but it is unknown who learners will work with in January because a clinic schedule hasn't been set yet, you could send the distribution to a delegator to forward once the clinic schedule is known.

Learning Event Schedule: Use a learning event schedule distribution when you want to have participants evaluate an event, when you want participants to evaluate the faculty teaching particular types of events, or when you want faculty to assess the learners participating in certain types of events.

Date Range: Use a date range to create the most generic type. You could use this for faculty or course evaluations, or for assessment of learners at any point.

A distribution wizard walks users through creating any distribution. Note that a distribution is only saved once all five steps of the wizard are complete. You can navigate forward and backward in a distribution using the Previous Step and Next Step buttons. You can cancel a distribution at any time by clicking 'Cancel'.

How to create a distribution

• Navigate to Admin>Assessment & Evaluation.

• Click 'Distributions'.

• Click 'Add New Distribution'.

• This will open a five-step wizard that walks you through creating a distribution. On the following pages you can read about the specifics of each distribution method.

How to copy a distribution

• Navigate to Admin>Assessment & Evaluation.

• Click 'Distributions'.

• Use the search tool to look for the distribution you want to copy. Click the down arrow beside the search bar to apply filters to refine your search results.

• Click on the cog icon to the right of the distribution you want to copy. Select 'Copy Distribution'.

• Edit the information in each step of the distribution wizard as needed and save your work.

How to delete a distribution

• Navigate to Admin>Assessment & Evaluation.

• Click 'Distributions'.

• Use the search bar to look for the distribution you want to delete. Click the down arrow beside the search bar to apply filters to refine your search results.

• Click the checkbox beside the name of the distributions you want to delete.

• Click the red Delete Distributions button.

How to manage distributions (monitor completion of tasks, remind people to complete tasks, etc.)

• Navigate to Admin>Assessment & Evaluation.

• Click 'Distributions'.

• Use the search bar to look for the distribution you want to manage. Click the down arrow beside the search bar to apply filters to refine your search results. You will only see distributions to which you have access.

• Click on the cog icon to the right of the distribution you want to copy. Select 'View Distribution Report'.

• Review progress. Click on any category (Pending, In Progress, Completed) to view more details about specific targets and assessors.

• Pending tasks have not yet been started. In Progess tasks have been started but not complete. Completed tasks are done and have been submitted.

• To delete tasks tick off the box below the garbage icon on each task card and then click the red Delete Task(s) button.

• To send reminders to those with incomplete forms, tick off the box below the bell icon on each task card and then click the blue Manage Distribution button and select Send Reminders from the dropdown list. To select all tasks for reminders click on the bell icon.

• Review your choices and, if correct, click Confirm Reminders. You will get a green success message.

• To add a task to a distribution, click the blue Manage Distribution button and select 'Add a Task' from the dropdown list. Complete the required information and click 'Confirm Task'.

Viewing progress results for learning event-based distributions: Program coordinators who set up such distributions should view progress from their My Assessments page.

How to manage tasks at the individual level

• Many of the functions described above can also be completed at an individual task level when logged into the system as a program coordinator or faculty director.

• Click the Assessment and Evaluation badge at the top right (beside the logout button).

• Click the My Learners tab.

• Click on the appropriate tab for a learner (e.g. CBME, Assessments, Logbook).

• When looking at a user's Assessment & Evaluation dashboard, some users will be able to send reminders, remove tasks, and download PDFs of selected tasks from their assigned learners or faculty.

Elentra provides several different distribution methods to allow users to easily collect feedback on courses, events, rotations and faculty. If you wish to reduce workload on your administrative staff and your organization has developed a standard rotation and faculty evaluation form, there are some additional options available.

1) On-demand workflows using standard rotation evaluation forms and standard faculty evaluation forms let learners initiate those forms when needed.

2) If your organization uses the rotation scheduler and a standard form to evaluate rotations within courses/programs, you could also use the automatic standard rotation evaluation option. This feature is enabled when you build rotations, and removes the need to have distributions or use workflows for rotation evaluations. Instead, each course/program can create a standard rotation evaluation form based on a preset list of items, and the form will be auto-distributed to learners based on their rotation schedules. This option requires developer assistance.

Please note that the automatic rotation evaluation option is specifically to support rotation evaluations, and that course evaluations should still be sent via distributions.

Implementation Notes

If your organization wants to use a standard faculty or rotation evaluation form a developer will need to add those two form types to the database and associate them with the correct organisation (relevant table: cbl_assessments_form_type_organisation).

Additionally, a developer will need to create the standard rotation evaluation items. These items that will be included on every standard rotation evaluation form (courses/programs will be able to add additional items as needed).

Once the form types and standard items are added, administrative users can build standard rotation evaluation forms and publish them as needed.

When you create a distribution, one of the options available via the Results tab is Target Release and Target Self-Reporting Options. This allows you to require that users complete a specific number of tasks before they can review their own results. This is typically used for peer assessments so that learners are required to provide feedback to others before they can see their own feedback. Please note that these settings are specific to a single distribution only. This tool does not require users to have completed a percentage of tasks across all distributions.

Elentra automatically sends email notifications to users when they have an Assessment and Evaluation task to complete or review.

Tasks scheduled via distribution

Assessors/evaluators will receive one email notification overnight with all new tasks linked in the body of the email.

For tasks scheduled via a distribution, the timing of when specific tasks will trigger an email will depend on how the distribution is configured (e.g., the task delivery date set, the timing of learning events for event-based distributions, dynamic delivery settings for rotation-based distributions).

Note that if a delegation-based distribution has been created, when a delegator forwards a task to a user, a distinct email will immediately be sent for that task.

User-initiated on-demand tasks

For user-initiated, on-demand tasks, email notifications will be created when the initiating user submits the form and the task moves to a new owner (i.e., assessor).

Additional Notifications

Other circumstances that can generate email notifications for a task include:

• A program administrator sends a reminder for a specific task;

• A task is forwarded from one user to another user.

Course directors, program administrators, and users set to receive prompted response notifications for specific distributions will also receive email notifications when they have tasks to review. This could include:

• In a CBME context, when any user completing a form has answered "Yes" to a concern item on a CBME form generated from a form template (i.e. Supervisor Form, Field Note Form, Procedure Form)

• When a user has indicated an answer configured as a prompted response on an item, when the form containing that item is included in a distribution configured to have prompted response notifications.

Database Configurations for email Options

There are some configurable options for email notification which can be controlled through the database (not through the user interface). This includes the ability to adjusts emails for tasks scheduled via distribution to send a nightly summary of all new tasks, and then a weekly summary of tasks that are still incomplete.

Program coordinators have quick access to review their assessment and evaluation tasks and those of their affiliated learners and faculty through the Assessment and Evaluation badge.

• Click the Assessment & Evaluation badge in the top right beside the logout button.

• A series of tabs will open under the Assessment & Evaluation header.

When viewing results on any of the tabs in A&E, use the search bar to look for a specific task by name and click the down arrow to limit your results by applying search filters like distribution method (date range, delegation, learning event, rotation schedule), curriculum period, course, and task status (pending, in progress, completed). Apply start and end dates to limit your results to a specific time frame. Remember to click 'Apply Filters' to apply your choices. Remember to click 'Remove Filters' to clear filters and view all results.

Assessment Tasks This tab shows users all pending tasks for which they are responsible. This is particularly important for program coordinators because of the delegation type distributions. If a distribution was created and set as a delegation to be sent to a program admin., s/he can access that delegation here and assign the task to the appropriate faculty (or other user). Users can view and complete a task by clicking on it. Users can remove a task by clicking 'Remove Task' and providing a reason for its removal. Users can download individual forms, or download multiple forms by clicking the download checkbox on each card and then clicking 'Download PDF(s)' at the top right. When users do this they will be able to choose whether to download all tasks as one file or as multiple files.

Tasks Completed on Me This tab displays all tasks completed on the user and which s/he has been given access to view. Whether a user can view tasks completed on them is controlled in the creation of a distribution.

My Reports: To view information from forms completed on oneself, the user can click My Reports on the right hand side.

• Set the date range.

• Remember that the creator of a distribution can set whether a user can view reports on a distribution, so not every distribution completed on a target is available for the target to view.

My Completed Tasks: * The user can view all completed tasks and again, download an individual file or download multiple files at once.

My Learners The My Learners tab will allow faculty to access a variety of information for learners associated with them. This list of available options will vary depending on which modules are in use in Elentra but could include CBME Dashboard, Assessments, and Log Book.

• Download a list of all learners associated with a faculty by clicking 'Download Enrolment'. A pdf title "learners-for-faculty-name" will download and list all learners, including their primary email and learner level.

• Search for an individual by beginning to type his/her name in the search learners area; the learner cards displayed will automatically display to reflect the searched name.

• Refine the list of learners by switching the curriculum period using the dropdown menu on the far right.

• Click on the appropriate tab to view the desired information.

My Learners - Assessments For each of the tabs described below you can search by a task name; apply advanced search filters like distribution method (date range, delegation, learning event, rotation schedule), curriculum period, course, and task status (pending, in progress, completed); set a date range; and download a pdf of an individual task or multiple tasks (note that you can select all if required).

Reports on the Learner:

• This tool allows you to view and download a pdf report (with or without comments) that aggregates learner performance on the selected form in one report. You can also use the Options dropdown menu to quickly view the form and the individual assessments.

• Click group by distribution as desired to sort the forms by distribution.

Tasks Completed on Learner: View tasks completed on the learner. Cards display the form title and type, task delivery and completion dates, form triggerer name (if applicable), and name and role of the assessor. Pending Tasks on Learner: These task cards will look similar to the completed tasks but will offer the ability to send a reminder about the task. To do so, click the checkbox beside the bell and then click 'Send Reminders' at the top of the page. You also have the option to remove a task on this screen. Be aware that is you remove a task you will be required to give a reason why. Upcoming Tasks on Learner: This displays scheduled tasks for the learner that are not yet active. Learner's Current Tasks: This displays tasks the learner is currently responsible for completing (e.g. faculty evaluation, service evaluation, etc.). Task cards may display a rotation or block name depending on how they were scheduled. Faculty can send a reminder or remove a task here as well. Learner's Upcoming Tasks: This displays a learner's upcoming tasks scheduled through a distribution.

Faculty: Program coordinators associated with a course will be able to access a tab listing faculty. Faculty will appear on this list if they have been the assessor in a distribution or the target or assessor of a triggered form tied to the same course as the program coordinator.

• Hide cards of external faculty by clicking Hide Card. Bring hidden cards back into view by clicking 'Show Hidden Faculty'.

• Update external faculty emails by clicking on Update Email and providing revised information.

• Program coordinators will be able to view the Current Tasks, Completed Tasks, Upcoming Tasks, and Tasks Completed on Faculty tabs for the available faculty.

• Program coordinators can send reminders, remove tasks or download tasks. (From this screen there is an option to select all and send reminders.)

• From the Tasks Completed on Faculty tab the program coordinator can also view Reports for this Faculty.

• This tool allows the program coordinator to view and download a pdf report (with or without comments) that aggregates evaluations of the selected faculty in one report. Program coordinators can also use the Options dropdown menu to quickly view the form and the individual evaluations.

• Responses and comments aggregated in these reports are de-identified.

• Click group by distribution as desired to sort the forms by distribution. If the same form has been used across multiple distributions this will tease apart each distribution and you can report on forms completed in a single distribution.

If you want to allow learners to initiate evaluations of rotations and faculty when needed you can use on-demand workflows.

Prerequisites:

• Have standard rotation and faculty evaluation forms built.

• Make sure all forms to be used are permissioned to the appropriate course and have the appropriate workflow associated with them.

• Make sure workflows are setup for your organization (requires developer assistance).

Notes:

• If a learner initiates a standard rotation evaluation, Elentra checks the learner's rotation schedule. The learner will be able to complete a rotation evaluation for any past or current rotation they are scheduled into. A learner will not be able to initiate an evaluation for a rotation they are scheduled into in the future.

• If a learner initiates a standard faculty evaluation, they will be able to access a complete list of all faculty in Elentra.

An adhoc distribution allows an administrator to set up a form that is available to a specific audience, during a specific time frame, and can be initiated on demand by learners or faculty. A potential use case is a clinical environment where it is unknown which learners will work with which assessors. Using an ad hoc distribution, administrative staff can set up the distribution then allow learners or faculty to trigger the assessment and provide details about who the target or assessor in the situation was.

One advantage of using an adhoc distribution over a workflow is that you will have a distribution report to review and monitor task completion for the specific distribution.

Please be aware of the following when using adhoc distributions:

• Users can only select 'complete and confirm via PIN' or 'send blank form' as form completion methods (you can't currently begin a form and send it via email or use the self-assess and send blank form options).

• When setting the targets of a distribution you can only use cohort and individuals (you can't currently set the distribution up with a course audience or course group as the target).

• The available assessors list will be based on course contacts from the course setup page.

• Use the ad hoc distribution for assessments only.

There are some database settings that can be used to tailor the use of the ad hoc distribution. If you want to change these you will need help from a developer.

• You can restrict the ability to initiate the form to only targets or only assessors (by default it is set to allow both to initiate forms).

• You can turn on or off the ability to allow forms to be completed and confirmed via PIN or to send a blank form (by default the system will allow both).

Creating an Ad Hoc Distribution

Permissions: Anyone with access to Admin > Assessment & Evaluation will be able to create an ad hoc distribution. For default installations this will include Medtech: Admin, Staff:Admin, Faculty:Admin, Staff:PCoordinator and Faculty:Directors.

Add New Distribution

• Navigate to Admin > Assessment & Evaluation.

• Click 'Distributions' above the Assessment and Evaluation heading.

• Click 'Add New Distribution'.

Form

• Distribution Title: Provide a title. This will display on the list of Distributions that curriculum coordinators and program coordinators can view. In addition, this distribution title will be used by learners and faculty to access the form. For this reason, a clear title including a course or use is recommended (e.g. Clinical Skills Week 1 History).

• Distribution Description: Description is optional.

• Task Type: Hover over the question mark for more detail about how Elentra qualifies assessments versus evaluations. If a distribution is to assess learners, it's an assessment. If it is to evaluate courses, faculty, learning events, etc. it is an evaluation. Notice that the language on Step 4 will change if you switch your task type as will other steps of the wizard.

• Assessment Mandatory: This will be checked off be default. Currently this information is recorded but does not impact how a task displays to a user.

• Select Form: The form you want to distribute must already exist and you must have permission to access the form; pick the appropriate form from the dropdown menu.

• Select a Curriculum Period: The curriculum period you select will impact the list of available learners and associated faculty.

• Select a Course: The course you select will impact the list of available learners and associated faculty.

• Click 'Next Step'.

Method

• Distribution Method: Select 'Adhoc' from the dropdown menu.

• Start Date: This is the beginning of the period the form is meant to reflect.

• End Date: This is the end of the period the form is meant to reflect.

• Task Expiry: Optional. Set the date on which the tasks generated by the distribution will automatically expire (i.e. disappear from the assessor's task list and no longer be available to complete). Tasks will expire at 12:00 AM on the day that you select, so it is best to select the day after your intended expiry date.

• Click 'Next Step'.

Targets

The target is who or what the form is about. You will only set specific targets if you are creating an assessment; if you are creating an evaluation you will not set a target list.

• Select Targets: Use this area to specify the targets of the form.

  • When setting the targets of an ad hoc distribution for an assessment you can only use cohort and individuals (you can't currently set the distribution up with a course audience or course group as the target).

• Target Attempt Options: Specify how many times an assessor can assess each target, OR whether the assessor can select which targets to assess and complete a specific number (e.g. assessor will be sent a list of 20 targets, they have to complete at least 10 and no more than 15 assessments but can select which targets they assess).

  • If you select the latter, you can define whether the assessor can assess the same target multiple times. Check off the box if they can.

• Click 'Next Step'.

Assessors

The assessors are the people who will complete the form. Currently, ad hoc distributions only support making faculty who are course contacts assessors.

Results

You can immediately save your distribution at this point and it will generate the required tasks, but there is additional setup you can configure if desired.

• Authorship: This allows you to add individual authors, or set the distribution to be accessible to everyone with A&E access in a course or organization. (This may be useful if you have multiple users who manage distributions or frequent staffing changes.)

• Target Release: These options allow you to specify whether the targets of the distribution can see the results of completed forms.

  • Task List Release:

    • "Targets can view tasks completed on them after meeting the following criteria" can be useful to promote completion of tasks and is often used in the context of peer assessments. Targets will only see tasks completed on them after they have completed the minimum percentage of their tasks set by you.

  • Target Self-Reporting Release: This controls whether targets can run reports for this distribution (i.e. to generate an aggregated report of all responses). When users access their own A+E they will see a My Reports button. This will allow them to access any reports available to them.

• Target Self-Reporting Options: This allows you to specify whether or not comments included in reports are anonymous or identifiable. (This will only be applied if you have set reports to be accessible to the targets.)

• Reviewers: This allows you to set up a reviewer to view completed tasks before they are released to the target (e.g. a staff person might review peer feedback before it is shared with the learner).

  • Check off the box to enable a reviewer.

  • Click Browse Reviewers and select a name from the list. Note that this list will be generated based on the course contacts (e.g. director, curriculum coordinator) stored on the course setup page.

• Prompted Response Notifications: This allows you to decide what action to take for any answers on the form that designated as prompted or flagged response options in items included on the form. (For example, if you have an item asking about student completion of a procedure and "I had to do it" was set as a prompted/flagged response.) You can optionally select to email the Assessment Reviewers, Program Coordinators, Program/Course Directors, or Distribution Authors.

• Click 'Save Distribution'.

Managing An Existing Ad Hoc Distribution

• Once an ad hoc distribution has been created you can monitor its progress from Admin > Assessment & Evaluation. Click on Distributions, and then the name of the distribution.

• The Show Progress screen you:

  • Not Started (forms that have been triggered but not started)

  • In Progress (forms that have been triggered, started, and saved in draft mode)

  • Completed (forms that have been triggered and completed).

• Click on any of the labels to view the names of the targets and assessors and delivery dates.

• Administrative staff can also send reminders, and add and delete tasks from here.

What Learners and Faculty Experience

After an ad hoc distribution has been set up (remember it may take up to a day for your distribution to become active depending on when behind the scenes cron jobs happen), learners and faculty who have a form available to them will see a "Trigger Assessment" button on their dashboard.

For Learners

• From the dashboard, click 'Trigger Assessment'.

• Select a distribution (begin to type a distribution name to quickly filter the list).

• Select an assessor. Hover over an assessor name to see their photo (if uploaded) and details about them including email and course affiliation.

• Select the assessment method. Email blank form will send a copy of the form to the selected assessor. Complete and confirm via PIN will allow the learner to immediately view and start the form, then have the assessor sign off on the form using his/her PIN. (For information on setting user PINs, please see here.)

  • If the assessor has a PIN set the system will automatically default to Complete and confirm via PIN as the assessment method.

• Select a date of encounter.

• Click 'Submit'.

• If the learner selected Email blank form, the assessor will receive an email alerting them to the task; additionally, they will see the task added to their Assessment Tasks list accessible from their A&E badge in the dashboard header.

• If the learner selected Complete and confirm via PIN, the form will display on the screen. The learner can begin to complete the form, then pass the device to the assessor. When the form is complete, the assessor can enter their PIN to confirm and complete the form.

  • Note that if a form triggered using Complete and confirm via PIN is saved as a draft, the learner will need to reopen the form when in the company of the assessor to have the form completed.

• After a form is completed, the learner's ability to view it will depend on the distribution settings. If the distribution allows the learners to view the tasks, they will be able to view them from their Tasks Completed on Me list accessed from their A&E badge in the dashboard header.

For Faculty

• From the dashboard, click 'Trigger Assessment'.

• Select a distribution (begin to type a distribution name to quickly filter the list).

• Select a target. Hover over a target name to see their photo (if uploaded) and details about them including email, group and role (e.g. student, 2022) and enrolled courses.

• Select a date of encounter.

• Click 'Begin Assessment'.

• The form will display on the screen.

• The faculty member can complete the form and Save as Draft or Submit. Note that if the user saves the form as draft, they will have to reopen and complete it at a later date.

  • The faculty member can delete the task if they triggered it in error.

  • The faculty member can forward the task to another faculty member if needed.

• Faculty can view their draft forms, or forms that leaners have triggered to them from their Assessment & Evaluation badge located in the dashboard header. The tasks to be completed will display in the Assessment Tasks list.

You can use a delegation distribution to send tasks to a delegator who can then forward the tasks to the appropriate assessor/evaluator. The use case for this might be setting up a distribution at the beginning of the year and not knowing exactly which preceptors will be working in a specific environment. You can send the distribution to a staff member who can forward the tasks once a clinic schedule is set or send the distribution to a user who is actually the target of the task and can then delegate the task to the appropriate assessor/evaluator.

Add New Distribution

• Navigate to Admin > Assessment and Evaluation.

• Click 'Distributions' above the Assessment and Evaluation heading.

• Click 'Add New Distribution'.

Form

• Distribution Title: Provide a title. This will display on the list of Distributions that curriculum coordinators and program coordinators can view.

• Distribution Description: Description is optional.

• Task Type: Hover over the question mark for more detail about how Elentra qualifies assessments versus evaluations. If a distribution is to assess learners, it's an assessment. If it is to evaluate courses, faculty, learning events, etc. it is an evaluation. Notice that the language on Step 4 will change if you switch your task type.

• Disable Target Release: This will only display for Evaluations. If you do not want evaluators to be able to release the form to the target upon completion, check this box. If left unchecked, evaluators will be able to release the form to the target upon completion.

• Assessment/Evaluation Mandatory: This will be checked off be default.

• Disable Reminders: Check this box to exclude this distribution from reminder notification emails.

• Select Form: The form you want to distribute must already exist; pick the appropriate form.

• Select a Curriculum Period: The curriculum period you select will impact the list of available learners and associated faculty.

• Select a Course: The course you select will impact the list of available learners and associated faculty.

• Click 'Next Step'

Method

• Distribution Method: Select 'Delegation' from the dropdown menu.

• Delegator: Click Browse Delegators and select the appropriate name from the dropdown menu. You can only select one delegator.

• Delegation Options:

  • Delegation based on date range

  • Delegation based on rotation schedule

• Delegator Type:

  • The delegator is an individual

  • The delegator is the target

• From this point forward the distribution wizard will be configured based on whether you selected a date range or rotation-based distribution. Please refer the the help pages for those two methods to continue through the wizard.

Completing a Delegation

Please see here for more information about how to complete a delegation task.

In ME 1.14 Elentra introduced a way for learner's to optionally release completed evaluation tasks to the targets of the task. In effect, a learner can complete an evaluation of a faculty member and choose to immediately release the completed task to said faculty member. The faculty can see who the feedback came from (in contrast, if you allow faculty to run reports on a distribution of which they are a target, they do not see the personal details of the evaluators).

While advantageous in some circumstances, allowing learners to optionally release completed evaluation tasks to facutly is not desirable to all organisations. There are database settings that can be used to turn this feature on or off at the course and organisation level. To use these you need a developers help. However, you can also control the learner's ability to release completed evaluation tasks at the distribution level on Step 1 of the distribution wizard.

The Disable Target Release option will only be visible on Evaluation tasks and it does not impact the faculty member's ability to create a report on a distribution (that is still controlled by the distribution in Step 5 of the distribution wizard).

To see what the learner experiences when allowed to optionally release completed evaluation tasks, please see here.

When completing a distribution you will notice a "Feedback Options" section in Step 4 if the assessor is set to faculty members (it will not appear when the assessor/evaluator is set to learners). If you check this off it will add an item to the form for this distribution asking both the assessor and the target whether or not they met to discuss the target's performance (see sample text below).

You might choose to use this option and add the item if you want to collect data on how often preceptors are meeting with learners.

In effect, this tool adds two items to a form. The first item will appear as above for the assessor to complete. Once the form is completed it will be stored in the learner's "Tasks Completed On Me" tab. The learner can access the form from there, and the learner will also receive an email notification that they can take action on the form.

The form is available for the learner to complete and answer the same question from their perspective. The comment box is available for learners to record any additional details.

As of Elentra ME 1.17, there is no reporting tool to compare learner and assessor responses, although they can be viewed on any completed forms. There is also no visual cue to an admin user that the response from the learner is pending; if there is no learner response displaying it means the learner hasn't answered the question.

Setting up on-demand workflows allows you to configure forms so that Elentra users can initiate the form when needed without administrative staff creating a distribution. You can use workflows to:

• allow users to initiate assessment tasks (EPA based and other), and

• allow learners to initiate rotation and faculty evaluations.

If there are forms you do not want users to be able to initiate on-demand, simply set the workflow to None.

You will need to decide as an organization how you want to use workflows and provide the relevant information to your developers for them to appropriately set up the database.

Sample uses include:

• Allow learners or faculty to initiate CBME assessments (e.g. Supervisor form) on demand

• Allow learners or faculty to initiate any generic assessment form on demand

• Allow learners to initiate faculty evaluations on demand

• Allow learners to initiate rotation evaluations on demand

There are some things to note about using on-demand workflows:

• Forms will only show on the Start Assessment screen if they have an affiliated workflow. If a form's workflow is set to None it will not be available on demand and should be used only in distributions.

• To use on-demand workflows, all forms must be permissioned to a course.

• Only learners should be set up to initiate faculty and rotation evaluations on demand. This is because Elentra uses learners' schedules to determine which forms they should be able to access. There is no user interface to support faculty completing standard course and faculty evaluation forms.

Types of Workflows

1. EPA - This is designed specifically for organizations using CBME. Use for forms tagged to EPAs that you want learners to be able to trigger. Forms using the EPA workflow contribute to CBME dashboards. Please note that forms generated from CBME form templates (e.g., supervisor form) will automatically have an EPA workflow added to them.

2. Other Assessment Form - Use for forms that you want learners to be able to trigger and complete on demand without tagging to EPAs, or forms that you don't want to appear in the EPA list when triggering. Can be applied to both forms tagged with EPAs/milestones and forms that are not tagged with EPAs/milestones. Only forms with EPAs/milestones tagged will contribute to CBME Learner and Program dashboards.

3. Faculty Evaluation - Use only for standard faculty evaluations (i.e. the same form is used across an organization). Learners can initiate on demand. A date range is included when triggered.

4. Rotation Evaluation - Use only for standard rotation evaluations. Learners can initiate on demand. Learners can only evaluate rotations that they have completed or are currently in. They cannot initiate an evaluation for a rotation scheduled in the future.

5. None - Use for forms that you do not want users to be able to initiate on demand. You can still attach these forms to distributions. You can still tag EPAs or milestones.

User Experience

When a user initiates an on-demand form they will have to select the type of workflow to use. In the example below a learner is initiating a form and has several workflow options available.

Developer Information for Workflows

To allow the use of form workflows in an organization, some setup is required. You need to associate form types with workflows, and, if necessary, restrict the user groups who can access specific workflows.

You can review the available form types in cbl_assessments_lu_form_types If your organization wants to use standard rotation and faculty evaluation forms, you will need to add those to the database and associate them with the correct audience.

Specific workflows are listed in cbl_assessments_lu_form_workflows

cbl_assessments_lu_form_type_workflow_link acts as a junction table to associate form workflows with form types.

The migration that runs will assign a workflow of None to all form types. How you configure things depends on how your organization will use workflows. For example, in a CBME setup you could associate Supervisor Forms with the EPA workflow. You could associate Generic Forms with Other Assessment Form workflow. How you set this up depends on how you want to allow users to initiate forms on demand.

Finally, cbl_assessments_lu_form_workflow_groups is responsible for restricting access to a specific workflow to specific user groups.

• The migration that runs will populate this table so that by default, only students can initiateFaculty Evaluation or Rotation Evaluation workflows on demand.

• Any form and user group combination added to this table creates exclusive rights for those user groups to access a form workflow.

• If no workflow is associated with a user group, then all user groups should be able to initiate that workflow on demand.

Program or Curriculum Coordinators have two methods to monitor completion of forms and tasks associated with their program and learners.

Coordinators can access their own tasks (including delegations) and monitor the progress of individual faculty and learners through the Assessment & Evaluation Badge located between the user name and the logout button.

Coordinators can monitor the progress of any distributions they have access to and view a complete inventory of outstanding, in progress, and deleted tasks for users associated with their course or program through the Admin > Assessment & Evaluation tab.

Users with access to Admin>Assessment and Evaluation will be able to view A&E reports. Generally this will include staff:admin, and staff:pcoordinator and faculty:director users assuming the staff:pcoordinators and faculty:directors are assigned to a specific course or program.

To view A&E Reports:

• Click Admin>Assessment & Evaluation.

• From the second tab menu, click on 'Reports'.

Reports are divided between evaluations, assessments (both based on the type of distribution selected), leave and distribution. As of ME1.13 these reports include data from distributed forms. If you also use triggered forms through the Competency Based Medical Education module note that these forms can be reported on through different tools including the Milestone Report and other aggregated reports available from the learner CBME dashboard.

Elentra includes a variety of reporting options through the Admin > Assessment & Evaluation tab. These reports reflect data collected through forms managed through distributions. To view reports for triggered forms when using the Competency Based Medical Education module, use the learner CBME dashboard. (More information about CBME reporting is available in the CBME section.) . One exception to this is if you are trying to report on form feedback from form built with form templates in CBME - that feedback is available through the Assessment Tools Feedback Report in the Assessments section.

A&E Reporting is an administrative reporting tool. Note the individual users may have access to their own reports via their Assessment & Evaluation button, but the availability of such reports depends on how distributions were set up.

Reporting options

When you create most reports you will have some additional options after selecting the appropriate course/faculty/learner/form, etc. These options allow you to customize the reports you run for different audiences.

Please note that not all these options will actually display on all reports, despite the fact that you have the option to select them through the user interface. Please see each specific report for additional detail about what will or will not be visible.

Include Comments: Enable this option if you'd like the report to include narrative comments made on the selected form. Unique Commenter ID: If you select to include comments you'll see this option. It allows you to apply a masked id number to each assessor/evaluator. This can be useful to identify patterns in comments (e.g., multiple negative comments that come from one person) while protecting the identity of those who completed the form. Include Commenter Name: If you would like to display the names of commenters click on the checkbox. Include Description: If you click this checkbox you can give a description to the report. The text you enter will be displayed at the top of the report generated. Include Average: Click this checkbox to include a column showing the average score. Include Aggregate Scoring: If you enable the average, you'll have the option to also include a column with aggregate positive and negative scoring in some reports. This gives a dichotomous overview of positive and negative ratings.

The Evaluation Reports section will allow you to generate reports based on evaluations completed via distributions. Evaluation reports will not include commenter names, even if you check off the commenter name option when setting the report options.

Rotation Evaluations (Aggregated)

This report is relevant only if your organization uses the Clinical Experiences rotation schedule. If you distribute rotation evaluations through a rotation based distribution, you can use this report to view results. Obviously the exact format of the report will depend on the form it is reporting on.

• Select a course, date range, rotation, curriculum period and form.

• Set the report parameters regarding displaying comments and averages.

• Click 'Generate Report'.

This report will not include learner names, even if you check off the commenter name option when setting the report options.

Learning Event Evaluations (Aggregated)

For use with distributions completed by event type.

• Set a date range.

• Select Individual Events: Check this off if you want the ability to select individual events (otherwise you will have to report on all events).

• Select the event type, distribution by a curriculum period, learning event and form.

This report will not include learner names, even if you check off the commenter name option when setting the report options.

Learning Event Feedback Forms (Aggregated)

Use this to report on feedback provided by participants when a feedback form is attached to an event as a resource.

• Select a course, date range, event type, form and learning event (optional).

You will notice some extra report options for this type of report.

• Separate Report for Each Event: This will provide separate files for each report if multiple events are selected to include.

• Include Event Info Subheader: This will provide a bit of detail about the event being evaluated (title, date, and teacher).

This report can include an average and an aggregate positive/negative score. This report will not include learner names, even if you check off the commenter name option when setting the report options.

Faculty Reports (Aggregated)

For use in viewing a summary report of learner evaluation of an instructor.

• Select a course and set a date range.

• Select a faculty from the dropdown menu by clicking on his/her name. Note that only faculty associated with the selected course in the given time period will show up on the list. Additionally, they must have been assigned as an assessor on at least one distribution in the organization. Please see additional information below.

• Select a form and distribution (optional).

• Set the report parameters regarding displaying comments and averages.

• Click 'Download PDF(s)'.

This report will not include an average, even if you check of Include Average when setting the report options.

The way faculty names become available to select for this report is when the faculty member is also an assessor on a distribution in the organization. This is designed in part to protect the confidential nature of faculty evaluations and prevent staff from being able to generate reports on any faculty at any time. If your organization does not use assessments or you require reports on faculty whose names aren't available, please reach out to us and we can help you put a work around in place.

You may also be able to report on faculty evaluations by accessing an aggregated report from a specific distribution. Please see more detail in the Weighted CSV Report section in Distribution.

Course Reports (Aggregated)

For use in viewing a summary report of learner evaluation of a course.

• Select a course and set a date range.

• Select a form and distribution (optional).

• Set the report parameters regarding displaying comments and averages.

• Click 'Download PDF(s)'.

This report will not include learner names, even if you check off the commenter name option when setting the report options. This report can include an average and an aggregate positive/negative score.

Individual Learning Event Evaluations

• Select a Course

• Set a date range for the report.

• Select the event types.

• Select the appropriate distributions.

• Select the relevant learning event (optional).

• Optionally select to download as one file.

Automatic Rotations Evaluations (Aggregated)

Although this report is included in Elentra ME 1.17, it will not be populated unless you are using automated rotation evaluations enabled via the Clinical Experiences Rotation Schedule.

• Select a course/program

• Set the date range

• Select a form (if you had multiple published forms you might be able to pick them)

• Select a rotation schedule (the options available will be based on the set date range)

  • If you select multiple rotations, results will be aggregated (this can function a bit like a program evaluation if desired)

• Choose whether to view results as a CSV or PDF

This report will not include commenter names because it is an evaluation.

When building a distribution a reviewer can be set in Step 5 of the Distribution Wizard. This person will need to review and approve, reopen or hide all completed tasks before the tasks as required. The use case for this might be providing an intermediary who checks the appropriateness of narrative comments on course and faculty evaluations or peer assessments.

For more information on setting up a reviewer on a distribution, please see the page for the distribution type you are working with.

Managing Tasks As a Reviewer

If you have been set as the reviewer for a distribution, click your Assessment and Evaluation badge and you'll see a tasks with a Reviewer label under your Assessment Tasks tab.

• Click View Task.

• Review the form contents.

• Choose one of three options:

  • Reopen the task: This will send the task back to the assessor/evaluator and it will remain in their pending/in progress task list until they complete it. If you reopen a task you will get a message indicating that the task has successfully been reopened.

  • Approve the task: This will essentially mark the task as finalized and release the task to the target (if allowed by the parameters of the distribution).

  • Hide the task: This will hide the task from being viewed by the target. The person who completed the task gets no feedback. If you hide a form, you will be prompted to enter a reason for why you hid the form. Note that hiding a form does not remove its data from reports generated about this form and distribution.

• Note that currently, there is no user interface to retrieve all hidden forms or to view the comments about why the form was hidden.

As a Medtech:Admin or Staff:Admin, when you navigate to Admin>Assessment & Evaluation, you will land on the Dashboard that provides a summary of Assessments and Evaluations, access to the Reports, and the tabs to access Distributions, Forms, Items and Scales.

Under the Assessments tab, you have the ability to Record Assessment if it was a paper based assessment, Trigger Assessment to send an assessment immediately to a target, and now Trigger Bulk Assessments. These same actions are available under the Evaluations tab. In addition, you can sort tasks based on whether they are Outstanding, Upcoming, Deleted or Prompted Responses.

Administrative users have the ability to trigger bulk assessments to send tasks to multiple assessors, for multiple targets, in the combinations required.

• Navigate to Admin > Assessment & Evaluation.

• Click 'Trigger Bulk Assessments'.

• Select a course/program.

• Select an assessor and target(s) as required. Note that to search for targets you will need to filter by curriculum period.

• For each target you can set a start and end date to reflect what the assessment period should be.

• To add additional assessors click the 'Add Assessor' button.

• Optionally set the assessments as mandatory.

• Optionally indicate if feedback is required for the assessments. If you check this off it will add an item to the form for this distribution asking the assessor whether or not they met with the trainee to discuss their performance (see sample text below).

• Optionally set prompted response notifications. This allows you to decide whom to contact if any answer on the form is designated as a prompted or flagged response option. (For example, if you have an item asking about student completion of a procedure and "I had to do it" was set as a prompted/flagged response.) You can optionally select to email the Program Coordinators, Program/Course Directors, or add a Custom Reviewer. If you select to add a Custom Reviewer you can select their name from a searchable list of users.

• Click 'Tools'. The tools returned will be those that are permissioned to the course/program you selected at the top of the page.

• Click 'Preview This Form' to confirm you have the form you want or click 'Send Assessment' to send the assessment to the selected assessors.

• You will be prompted to optionally provide an Assessment Cue. This can be a short note to explain the context of the assessment. The text entered will display at the top of a form when the assessor is completing it.

• Click 'Send Assessment.'

• A green success message will display at the top of the page and you will be returned to the beginning of the bulk trigger assessment process to repeat if needed.

One of the distribution options is to create a delegation. This sends tasks to a delegator to forward to the appropriate person at a later date. If there are multiple targets for a delegation, users will be able to forward some tasks to one assessor, and other tasks to another assessor.

How to complete a delegation

• If a distribution was created using a delegation, the designated user needs to complete the delegation by assigning assessors.

• Log in as the delegator.

• From the Assessment Tasks tab, click on the relevant assessment.

• You'll see a list of targets in a table.

• Click the checkbox beside a target name.

• At the bottom right click 'Select Assessors'.

• Search for as assessor and click the checkbox beside the assessor name.

• If you need to add an additional internal and/or external assessor you may do so by clicking the 'Add Additional Assessor' button. This will allow you to enter the name of an internal user and/or email address of an existing external assessor. You can even add a new external assessor if necessary.

• After adding the required assessors, click 'Proceed to Confirm Assessments'.

• If you wish to mark the delegation as complete, click the checkbox. This will move the task to the delegator's My Completed Tasks list. If you have targets without assessors/evaluators, you will receive a warning. If you'd rather that the task to stay on the Assessment Tasks tab, do not click the checkbox.

• Confirm your choice by clicking 'Create Assessment Tasks'.

• The list of targets and assessors/evaluators will be updated with the newly entered information.

In the date range based delegation above, the delegator has assigned 2 targets to Alex Adams and 2 to Bennett Adkins. The rest of the tasks will be assigned at a later date.

In addition to being able to access their personalized Assessment & Evaluation badge where they can view their own tasks and access My Learners and My Faculty lists, coordinators and some faculty will also have access to Admin>Assessment & Evaluation. From this tab, coordinators can filter results and easily view outstanding, upcoming, and deleted tasks, as long as they have been added as a Course Contact. Coordinators can send reminders and remove tasks from this screen as well.

Faculty who are the assessors in a distribution or the target or assessor on a triggered form should show up here. Learners enrolled in PA or Course Director's program will also display on this list.

From the A&E Dashboard users can search for individual tasks or users to view their progress. Hovering over the target number will reveal a list of targets. Reminders can be send by clicking the box below the bell icon and then clicking Send Reminders.

If you are at an institution using the CBME features of Elentra, note that you can filter by delivery method to view triggered assessment or assessments and evaluations sent out via delegation.

Note that the Owner of a task is the person who has responsibility for completing the next step of a task. As such, if a user has stored a task in draft mode, they may be the owner of the task, even if it will eventually be sent to a faculty/preceptor.

Leave by Block Report

For use when you have clinical learning courses with block schedules and want an overview of those learners who had an approved leave during a specific block.

• Select a curriculum period.

• Select a block.

• Click 'Download PDF'.

Rotation Leave Report

For use when you have clinical learning courses with rotation schedules and want an overview of those learners who had an approved leave during a specific rotation.

• Set a date range.

• Select one or more learners.

• Set the report parameters regarding displaying description and comments.

• Click 'Generate Report'.

• Note that once generated, this report is available to download by clicking 'Download PDF'.

Faculty have quick access to review their assessment and evaluation tasks and those of their affiliated learners through the Assessment and Evaluation badge.

• Click the Assessment & Evaluation badge in the top right beside the logout button.

• A series of tabs will open under the Assessment & Evaluation header.

When viewing results on any of the tabs in A&E, use the search bar to look for a specific task by name and click the down arrow to limit your results by applying search filters like distribution method (date range, delegation, learning event, rotation schedule), curriculum period, course, and task status (pending, in progress, completed). Apply start and end dates to limit your results to a specific time frame. Remember to click 'Apply Filters' to apply your choices. Remember to click 'Remove Filters' to clear filters and view all results. Note that your previous filter settings may still be applied so if you are seeing no results, or fewer results than you expected, try 'Remove Filters'.

Assessment Tasks This tab shows users all pending tasks for which they are responsible. Mandatory tasks are now indicated with a red highlight with a bold red "Mandatory" title on the card. Faculty can view and complete a task by clicking on it. Faculty can remove a task by clicking 'Remove Task' and providing a reason for its removal. Faculty can download individual forms, or download multiple forms by clicking the download checkbox on each card and then clicking 'Download PDF(s)' at the top right. When users do this they will be able to choose whether to download all tasks as one file or as multiple files.

Tasks Completed on Me This tab displays all tasks completed on the user and which s/he has been given access to view. This can be controlled in the creation of a distribution.

My Reports: To view information from forms completed on oneself, the user can click My Reports on the right hand side.

• Set the date range.

• Remember that the creator of a distribution can set whether a user can view reports on a distribution, so not every distribution completed on a target is available for the target to view.

My Completed Tasks:

• The user can view all completed tasks and again, download an individual file or download multiple files at once.

Faculty: Faculty who are listed as course/program directors have access to view the assessment and evaluation pages of faculty associated with their course/program (including external assessors). Faculty appear on this list if they have been the assessor or target in a distribution tied to the course.

• Hide cards of external faculty by clicking Hide Card. Bring hidden cards back into view by clicking 'Show Hidden Faculty'.

• Update external faculty emails by clicking on Update Email and providing revised information.

• Program directors will be able to view the Current Tasks, Completed Tasks, Upcoming Tasks, and Tasks Completed on Faculty tabs for their faculty.

• Send reminders, remove tasks or download tasks. (There is an option to select all and send reminders.)

• From the Tasks Completed on Faculty tab the course/program director can also view Reports for this Faculty.

My Learners The My Learners tab will allow faculty to access a variety of information for learners associated with them. This list of available options will vary depending on which modules are in use in Elentra but could include CBME Dashboard, Assessments, and Log Book.

• Download a list of all learners associated with a faculty by clicking 'Download Enrolment'. A pdf title "learners-for-faculty-name" will download and list all learners, including their primary email and learner level.

• Search for an individual by beginning to type his/her name in the search learners area; the learner cards displayed will automatically display to reflect the searched name.

• Refine the list of learners by switching the curriculum period using the dropdown menu on the far right.

• Click on the appropriate tab to view the desired information.

My Learners - Assessments For each of the tabs described below you can search by a task name; apply advanced search filters like distribution method (date range, delegation, learning event, rotation schedule), curriculum period, course, and task status (pending, in progress, completed); set a date range; and download a pdf of an individual task or multiple tasks (note that you can select all if required).

Reports on the Learner:

• This tool allows you to view and download a pdf report (with or without comments) that aggregates learner performance on the selected form in one report. You can also use the Options dropdown menu to quickly view the form and the individual assessments.

• Click group by distribution as desired to sort the forms by distribution. If the same form has been used across multiple distributions this will tease apart each distribution and you can report on forms completed in a single distribution.

Tasks Completed on Learner: View tasks completed on the learner. Cards display the form title and type, task delivery and completion dates, form triggerer name (if applicable), and name and role of the assessor. Pending Tasks on Learner: These task cards will look similar to the completed tasks but will offer the ability to send a reminder about the task. To do so, click the checkbox beside the bell and then click 'Send Reminders' at the top of the page. You also have the option to remove a task on this screen. Be aware that is you remove a task you will be required to give a reason why. Upcoming Tasks on Learner: This displays scheduled tasks for the learner that are not yet active. Learner's Current Tasks: This displays tasks the learner is currently responsible for completing (e.g. faculty evaluation, service evaluation, etc.). Task cards may display a rotation or block name depending on how they were scheduled. Faculty can send a reminder or remove a task here as well. Learner's Upcoming Tasks: This displays a learner's upcoming tasks scheduled through a distribution.

The Weighted CSV Report provides a csv file that includes the data collected through completed forms. It lists users who completed the form down the side and form items across the top. Inside each cell will be data from the form representing the scale ratings made by those who completed the forms. If your response descriptors include numbers (e.g. 1 - Overall, this instructor is an effective teacher.) note that those numbers will not necessarily be reflected in the csv.

It is important to note that the weighted CSV report was specifically designed to be used in conjunction with items using a rating scale (e.g. grouped items using a rubric), and allows you to create custom weights for scale response descriptors which get reflected in the report. There is currently no way to configure these weights through the user interface and you will need a developers help to assign weights to scale response descriptions in the database. (Developers, you'll need to use the cbl_assessment_rating_scale_responses table.)

If no weights are applied to the scale responses, the report defaults to assign a value of 0, 1, 2, 3, 4 to the responses in a left to right order. In effect, the Weighted CSV Report will work best if the rating scale you apply to the items mimics a 0-4 value (e.g. Not Applicable, Strongly Disagree, Disagree, Agree, Strongly Agree).

Accessing the Weighted CSV Report

You can access the Weighed CSV Report from two places: from the Assessment & Evaluation Reports tab or from an individual distribution.

To access the Weighted CSV Report from Admin>Assessment & Evaluation you must have general access to the Admin>A&E tools. Such access will usually apply to staff:admin, staff:pcoordinator, and faculty:director users when the staff:pcoordiantors and faculty:directors are affiliated with a course/program.

The weighted CSV report is accessible from the Admin>Assessment & Evaluation Reports tab.

• Click Admin>Assessment & Evaluation.

• From the second tab menu, click on 'Reports'.

• Scroll to the bottom of the list and in the Distribution section and click on 'Weighted CSV Report'.

To access the Weighted CSV Report from an individual distribution you must have access to that distribution.

• Click on Admin>Assessment & Evaluation.

• From the first tab menu, click on 'Distributions'.

• Search for or click on the title of the relevant distribution.

• Click on the Completed Assessments card (far right).

• Click on the Weighted CSV button under the Assessments Completed heading.

• A file should download to your computer.

Most reports are currently available as PDFs.

Some screen shots of sample reports are posted below but remember that your reports will contain the items relevant to the forms you've designed and used. In some cases information has been redacted to protect users' privacy.

The Assessments Reports section mostly allows you to generate reports based on assessments completed via distributions. There are some exceptions, however most reports are for distributed forms.

Learner Assessments

This report allows you to compile all assessments completed on a target in one or more courses into one or many files. It does not aggregate results, just compiles multiple forms.

• Select a course, set a date range and select a course group (optional).

• Select a learner.

• Select a form (optional).

• Click 'Download PDF(s)'.

• Choose whether to download as one file (all forms will be stored in one file) or not (you'll download a file for each form).

A file will download to your computer with the appropriate forms included. Each form will include the target and assessor, delivery and completion date, form responses and comments, etc. The file names will be: learnerfirstname-learnerlastname-assessment-datereportrun-#.pdf. For example: earnest-acosta-assessment-20181005-1.pdf

Learner Reports (Aggregated)

Use this report to create an aggregated report on learner performance on a single form that may have been used multiple times and completed by multiple assessors. For this report, the list of learners available will depend on someone's affiliation with a course/program if they are a staff:pcoordinator or faculty:director.

• Set the date range.

• Select a learner.

• Select a form.

• Set the report parameters regarding displaying comments and averages.

• Click 'Download PDF(s)'.

An average and aggregate positive and negative score are available with this report. This report will not include learner names beside comments, even if you check off the commenter name option when setting the report options.

Please note that if in the distribution for a form, you set comments to be identifiable, learners themselves will be able to view commenter names when they self-report the form results as seen below.

Timeliness of Completion Report

For use in reporting on tasks delivered to and completed by faculty. Report columns include the number of tasks delivered and completed as well as the average time to completion from delivery date and average time to completion from the end of the experience (e.g., a block) per user. It also provides an overall average across all users. Available as a PDF.

• Select a course.

• Set a date range.

• If there were external assessors used you will have the option to include externals in the report or not.

• Select one or more users by clicking the checkbox beside each required name. Please note that if you select all faculty it can take some time for all names to appear. Please be patient! To delete a user from the report click the 'x' beside the user's name.

• Include Average Delivery Date: Enable this if desired.

• Include Description: This allows you to enter a description to display at the top of the report.

• Click 'Download PDF(s)'.

• The report should download.

Assessment Tools Feedback Report

For use in collating responses provided by faculty completing forms produced through form templates in the competency-based medical education module. This report aggregates comments from forms that are sent out using a distribution. When logged in as an admin. you'll see a full list of all form feedback provided thus far display on the screen.

• Set a date range.

• Select a course from the dropdown options.

• Select a tool from the dropdown options.

• Click 'Apply Filters'.

• Results will display on the screen and you can click 'Download PDF(s)' if you need to download a copy.

• To begin a new search be sure to click 'Reset Filters' and select a new course and/or tool as appropriate.

You can click on the page icon to the right of the feedback column to view the form being referenced.

Triggered Assessments Report

An organization must be using the rotation scheduler to use this report. This report displays the number of completed assessments over the number of triggered assessments per block for all students in a course during the selected curriculum period. Data is grouped by block, based on the selected block type. The learner is considered the target of the assessments.

• Select a course (you can only select one).

• Select a curriculum period.

• Select a block template type (e.g. 1 week, 2 week). The block templates available will depend on the setup of the rotation schedule for the course.

• Click 'Download CSV(s)'

Completed Assessments Report

We plan to rename this the Triggered/Completed Assessments by Faculty Report.

For use in monitoring the progress of faculty in completing the tasks assigned to them.

• Select a course.

• Set a date range.

• Decide whether or not to include External assessors. (This is only relevant if you allow users to send tasks to external assessors, e.g., users without an Elentra account.)

• Select the relevant faculty (you can select more than one).

• Click 'Download CSV(s)'.

• A csv file will download to your computer.

External Assessors are members of a learning community who do not have Elentra accounts. Examples could include social workers, patients, allied health professionals, etc. External assessors can be included in distributions and learners can initiate tasks to external assessors.

When an external assessor is created their name and email are stored but they do not have a complete user record in Elentra. We avoid duplication of external assessor records by having the system check email addresses.

Existing external assessors will display at the end of a list of users when searching for a user when creating a distribution.

If a learner searches for an existing external assessor to initiate a task to, the name will display with other returned results and the assessor will be labelled as external on the user card.

Distribution Delivery Schedule

This report can be used by administrative staff to keep an inventory of distributions.

• Select a report type. You can see an overview of distributions or individual tasks.

• Select a course.

• Set a date range.

• Select a task type.

• Select a distribution or an individual task (your option will depend on the first selection you made on the page).

• Click 'Generate Report'.

• From here you can search within the results or click on any distribution to see its progress.

Distribution Reviewer

Use to see an overview of who is set as a reviewer for distributed tasks. (When you create a distribution you can assign a reviewer who serves as a gatekeeper of completed tasks before they are released to be seen by their target. This is completed during the final step of a distribution. For additional information please see the Assessment and Evaluation>Distributions help section.)

• More information coming soon.

Embedded Distribution Reviewer

This report functions like the report above but offers users a view of the report in the interface without requiring them to open a PDF.

Weighted CSV Report

Please see the next page for information on the Weighted CSV Report.

What is the Impact of Retiring vs. Deleting a Form?

Retiring a PPA or Rubric/Flex form that has been TRIGGERED:

• We recommend that if the form was in use at all, simply retire it instead of deleting it so that any Pending or In progress assessments/evaluations can still be completed by their assessor/evaluator.

Deleting a PPA or Rubric/Flex form that has been TRIGGERED:

• All pending and in-progress tasks that used that form will not have a form associated with them and will display an error message stating that the form has been deleted when assessor/evaluator tries to access them (retiring is the better choice!).

• If needed, a developer can restore the old form to allow assessors/evaluators to complete the assessment/evaluation task that uses the form.

• If you delete a form that has been used to trigger an assessment, then any pending assessments can no longer be completed.

Retiring a PPA or Rubric/Flex form used in a DISTRIBUTION:

• All pending and in-progress tasks that used the form will still be accessible to the assessors, but will be not be available for selection on any new distributions.

• You can go ahead and replace the form on the distribution whose form has been retired; all future tasks will use the new form.

Deleting a PPA or Rubric/Flex form used in a DISTRIBUTION:

• All pending and in-progress tasks that used that form will not have a form associated with them and will display an error message stating that the form has been deleted when assessor/evaluator tries to access them (retiring is the better choice!).

• If you replace that deleted form with a new form on the distribution, all future tasks will use the new form.

• If there are pending or in-progress tasks where users need to have the deleted form replaced with the new form, a developer can replace the old form with the new one for pending tasks.

  • EXCEPT: They cannot switch assessment forms for tasks where the assessor/evaluator has already completed the assessment/evaluation form for one or more targets, but not all of them. In this case, they would have to temporarily restore the old forms so that the assessor/evaluator can finish their tasks. They can only switch assessment/evaluation forms for completely pending assessments (meaning they don't have any progress).

Deleting a FORM TEMPLATE:

• Deleting a form template does NOT delete the forms that were previously generated by the deleted template. They remain to allow assessors/evaluators to complete them; however, the old forms will not be available going forward.

What is the Impact of Replacing a Form on an Active Distribution?

You can create your distribution using one form, and then switch the form on the distribution to the new form when it is ready. All future tasks created by the distribution will use the new form.

It is recommended that, if you replace a form that you no longer wish to use, you retire the form so that previously delivered tasks using that form can still be completed.

As long as you do not delete the old form altogether, any pending or in progress tasks that have been delivered using the old form can still be completed by the assessors/evaluators.

If you need to replace the old form with the new form for any pending tasks (pending tasks are tasks that have not yet been started by the assessor/evaluator) you will need to get help from a developer.

 Once all in progress and/or pending tasks using the old form have been completed, you can delete the old form if you wish. You can also choose to delete the old form if there are in progress and/or pending tasks that are using it, but the assessor/evaluator will receive an error message stating that the form no longer exists when they attempt to complete it. In this case, it is better to retire the form, so that these tasks can still be completed using the old form, but the old form will no longer be available to select when creating future distributions.

What is the Impact of Editing Other Aspects of an Active Distribution?

• Editing the Expiry Date: If an expiry date has been applied to the active distribution, you can edit the expiration date to 're-open' tasks to the targets who haven't completed them yet. Those tasks will be visible to the targets the day after the update is made to the expiry date (because the job that runs behind the scenes to deliver tasks runs at night). A developer can run the job immediately if it is time sensitive.

• Changing the Targets: If you add a new targets to a distribution, the system will reopen the task for the assessor/evaluator to allow them to complete the task on the new target.

• Editing the Rotation Schedule of Targets Associated with a Distribution: will cause new tasks to be delivered to the assessor.

• Changing the Reviewer of a Distribution: will cause the distribution to redeliver previously delivered tasks; be sure to apply a Release Date when making this change.

• Changing the Target Release Options in Step 5: Changes the visibility of tasks/reports retroactively for all completed tasks.

The Assessor/Evaluator Completed an Assessment/Evaluation on the Incorrect Target. What do I do?

• Navigate to Admin > Assessment & Evaluation.

• Click on the Distributions tab.

• Edit the distribution that had the incorrect target selected (remove the incorrect target, add the new target).

• Ask a developer to push the distribution through to reflect the changes OR the next day (after the system has a chance to run the changes to the distribution overnight), navigate to the distribution and click on its title to view its Progress page.

• Find the task in the Available assessments tab (the assessment is available because it is new).

• Click on the name of correct target.

• Use the Choose a Target dropdown menu to open the completed assessment, which was completed on the incorrect target.

• Click the Choose a Target dropdown menu and click the circular double arrow to the right of the correct target under Forms Not Started to assign the completed assessment to the correct target; you will see the page refresh with the correct target and the completed form.

• Now the incorrect target will be listed in the Forms Not Started list, and you can delete that target for the assessor/evaluator: click the Choose a Target dropdown menu, click on the incorrect target in the Forms Not Started list, click Delete Task, and complete the required fields to remove the task.

Please note that this feature may not be available in your installation of Elentra yet.

Prerequisites:

• Have a standard rotation evaluation form built for each course/program using it (note that the form must be permissioned to the course).

• Be using the Elentra rotation schedule to book learners into rotations.

Each program/course can only have one standard rotation evaluation form active at a time. If you create a second form of this type it will override the first form you made.

If you want to use the automated standard rotation evaluation feature, when you build a rotation, make sure to check off the "Automated standard rotation evaluation" button. Set the preferred delivery schedule for these automated tasks.

To report on forms completed this way, please use the "Automatic Rotation Evaluations (Aggregated)"

Learners have quick access to review their assessment and evaluation tasks through the Assessment and Evaluation badge.

• Click the Assessment & Evaluation badge in the top right beside the logout button.

• Three tabs will be visible to learners: Assessment Tasks, Tasks Completed on Me, and My Complete Tasks.

When viewing results on any of the tabs in A&E, use the search bar to look for a specific task by name and click the down arrow to limit your results by applying search filters like distribution method (date range, delegation, learning event, rotation schedule), curriculum period, course, and task status (pending, in progress, completed). Apply start and end dates to limit your results to a specific time frame. Remember to click 'Apply Filters' to apply your choices. Remember to click 'Remove Filters' to clear filters and view all results. Note that your previous filter settings may still be applied so if you are seeing no results, or fewer results than you expected, try 'Remove Filters'.

Assessment Tasks: This tab displays tasks the learner is currently responsible for completing (e.g. faculty evaluation, service evaluation, etc.). Mandatory tasks are now indicated with a red highlight with a bold red "Mandatory" title on the card. Task cards may display a rotation or block name depending on how they were scheduled. Learners can view a task, download individual tasks or multiple tasks at once, and can remove tasks. If the learner removes a task, they will have to provide a reason.

Tasks Completed on Me: This tab displays all the tasks that have been completed on the learner. Task cards display the form title and type, task delivery and completion dates, form triggerer name (if applicable), and name and role of the assessor. Learners can view a task and download individual tasks or multiple tasks at once. A Seen by Target icon now appears in the top right hand corner so that the learner, faculty and staff can confirm the learner has seen the completed form.

My Reports: From this button on the right hand side, learners can access reports on forms completed on them (if the distribution allowed for learners to view results). Provide the appropriate dates. Click 'Report'.

My Completed Tasks: This tab displays forms the learner has completed. Task cards display the form title and type, rotation/block name, task delivery and completion dates, and progress. Learners can view a task and download individual tasks or multiple tasks at once.

When you toggle to card view of a list of users (e.g. within a distribution) you'll be able to see their user photo (if they have provided one).

Learner release of complete task to faculty

When administrators create distributions they can optionally decide whether or not to release the tasks and associated reports to the targets of the forms. Choosing to release tasks allows the target to view forms completed on them. In the case of evaluations, administrators only have the option to release task reporting to targets, not release individual tasks. However, a distribution can be set up to allow learners to optionally release completed evaluation tasks to a target. This applies in the case of evaluations only.

If this optional target release feature is used, at the bottom of an assigned evaluation task, learners will see the option to release the evaluation to the selected target. If the learner picks 'Yes' from the dropdown selector, the task will become available for the target to view through their Assessment & Evaluation tab.

The faculty member can now view that completed form.

Note that learners opting to individually release tasks for faculty to view does not change the faculty member's ability to create a report on a distribution (that is still dictated by the distribution).