In this context, we use the term off-service to mean learners who are completing rotations outside their home program (e.g. a family medicine resident on a pediatrics rotation).
Clinical Experiences > Rotation Scheduler includes support for off-service rotations or slots. When an administrator builds rotations she can optionally create off-service slots and make them available to all other programs or specific programs. The following information assumes you are an organization using off-service slots and refers to rotation-based distributions. (This page does not discuss automated rotation evaluations nor rotation evaluations completed on demand via workflows.)
As the home program you can:
Create assessment tasks targeting your learners while they are on an off-service rotation.
Report on assessment tasks targeting your learners while they were on an off-service rotation.
For distributions you created, you can view results via the Learner Assessments report or the Learner Reports (Aggregated).
For distributions created by host programs, but targeting your learners, you can only view results via Learners Reports (Aggregated).
Create evaluation tasks asking your learners to evaluate their experience on an off-service rotation.
View individual responses and download a Weighted CSV of your learners' responses to an evaluation of their off-service experience (do this via the Completed Distribution Report); up to and including ME 1.17 a home program cannot use the Rotation Evaluation (Aggregated) Report to see an overview of their learners' experience on an off-service rotation (whether the distribution was set up by the home or host program).
As the host program you can:
Create assessment tasks targeting visiting, off-service learners while they are on your rotation.
Report on assessment tasks targeting visiting, off-service learners while they were on your rotation.
Create evaluation tasks asking visiting, off-service learners to evaluate their experience on your rotation.
Report on evaluation tasks completed by visiting learners to your rotation, as long as you created the distribution. If the home program created an evaluation asking their learners to evaluate your rotation or faculty, you will not be able to access the results.
In effect, up to and including ME 1.17, neither the home program nor the host program can create a Rotation Evaluation (Aggregated) Report to view responses of off-service learners to off-service rotation evaluations created by the home program.
Note that in a rotation based distribution you are able to toggle between a block view and date range view of the distribution report. When using Block view you can view completion per block and switch blocks as needed using the forward and back arrows.
Additionally, if the target option for a distribution was that the assessor/evaluator would define their target when completing a task, pending tasks will display the target as to be defined.
Rotation Based Distributions allow you to set up a distribution based on a rotation schedule. This means you can easily send a form to all enrolled learners to be delivered when they are actively in the rotation. Note that you must have rotations built using the Clinical Experience Rotation Scheduler to use this distribution method.
Navigate to Admin>Assessment and Evaluation.
Click 'Distributions' above the Assessment and Evaluation heading.
Click 'Add New Distribution'.
Distribution Title: Provide a title. This will display on the list of Distributions that curriculum coordinators and program coordinators can view.
Distribution Description: Description is optional.
Task Type: Hover over the question mark for more detail about how Elentra qualifies assessments versus evaluations. If a distribution is to assess learners, it's an assessment. If it is to evaluate courses, faculty, learning events, etc. it is an evaluation. Notice that the language on Step 4 will change if you switch your task type.
Disable Target Release: If you do not want assessors/evaluators to be able to release the form to the target upon completion, check this box. If left unchecked, assessors/evaluators will be able to release the form to the target upon completion.
Assessment/Evaluation Mandatory: This will be checked off be default.
Disable Reminders: Check this box to exclude this distribution from reminder notification emails.
Select Form: The form you want to distribute must already exist; pick the appropriate form.
Select a Curriculum Period: The curriculum period you select will impact the list of available learners and associated faculty.
Select a Course: The course you select will impact the list of available learners and associated faculty.
Click 'Next Step'
Distribution Method: Select 'Rotation Distribution' from the dropdown menu.
Rotation Schedule: Select the appropriate rotation schedule from the dropdown menu.
Specific Sites: Select a site if you wish to include only targets from a specific site on this distribution.
Release Date: This tells the system how far back on the calendar to go when creating tasks. Hover over the question mark for more detail.
Delivery Type:
Basic delivery (single): allows you to send a single evaluation task repeatedly, once per block, or once per rotation.
Dynamic delivery rules (multiple): allows you to deliver multiple evaluation tasks at specific intervals during a rotation. For example, if you have a 5-month rotation, and you’d like to deliver an interim evaluation at the 2-month mark and the 4-month mark, you can set a rule for the distribution to do so.
If you select Basic delivery (single), then select your Delivery Period:
Choose between delivering tasks repeatedly, once per block, or once per rotation.
Repeatedly means you set how often during the rotation the task sends (e.g. every 5 days)
Once per block means that for each booking the learner has in a rotation they will be sent a task
Once per rotation means that consecutive bookings (i.e., two or more back-to-back blocks) will be treated as one unit of time
For each delivery period, additional customization allows you to control the timing of the distribution (e.g. 1 day after the start of the block, or 3 days before the end of the rotation).
Options are before/after the start, before/after the middle, and before/after the end.
In this sample schedule, a 'Once per block' delivery for the IM rotation would send learner 012022 three tasks - one each for Blocks 1, 2, and 3. A 'Once per rotation' delivery for the IM rotation would send learner 012022 two tasks - one for Blocks 1+2, and one for Block 4.
Note that if learners have consecutive bookings at different sites they will not be treated as one rotation. For example a 'Once per rotation' delivery for the IM rotation for Tibor would result in two assessment tasks because he is at Hospital 1 and then Hospital 2.
If you select Dynamic deliver rules (multiple), then set your delivery rules.
Identify the specific delivery rules depending on the length of a rotation (you identify the length of the rotation in months, weeks, or days).
If you enter an integer larger than 1 for the number of tasks, you will be able to define when each task will be delivered (percent of the way through the rotation) and a visualization will be supplied by the Task bar below.
Clicking on Add new delivery will allow you to add additional delivery schedules that depend on the length of the rotation, which allows you to set different parameters for shorter rotations and longer rotations.
As noted in the interface, “larger” rules take precedence over smaller ones. So if you had a 0-1 month rule and a 0-2 month rule, the 0-2 month would supersede the other.
Remember: if a learner's schedule has two or more of the same rotation scheduled in a row, the system will treat them as a single rotation.
You can have as many rules as you want, but they must be contiguous (e.g., 0-2 months, 2-4 months, 4-6 months) to avoid leaving learners without an assessment.
Examples of Dynamic Delivery Scenarios:
If you would like two interim assessment tasks delivered for any rotation 5 to 8 months in length, you could identify that for a 5 – 8-month rotation, task one should be delivered at 40% of the way through the rotation and task 2 should be delivered at 80% of the way through the rotation. Then, for a learner on a 5-month rotation, the interim task would be delivered at 2 months into the rotation and again at 4 months into the rotation, for example.
If you would like three tasks delivered for any rotation 3 – 6 months in length, you could set task one at 25% through the rotation, task 2 at 50% through the rotation, and task 3 at 75% through the rotation, which would represent delivery of the task at ¼ of the way through the rotation, 1/2 the way through the rotation, and ¾ of the way through each 3-month, 4- month, 5-month, or 6-month rotation that a learner might be scheduled on.
Task Expiry: If you check this box, the tasks generated by the distribution will automatically expire (i.e., disappear from the assessor's task list and no longer be available to complete). You can customize when the task will expire in terms of days and hours after the delivery.
Warning Notification: If you choose to use the Task Expiry option, you'll also be able to turn on a warning notification if desired. This can be set up to send an email a specific number of days and hours before the task expires.
The target is who or what the form is about (e.g., learners, faculty members, a rotation). Note that you'll only see the option to set the rotation or faculty member as a target if you are creating an evaluation.
Assessments will be delivered for: Use this area to specify the target of the form.
Targets are the assessors (self assessment): use this for self-assessments. Learners will be delivered a task where they are both the target and assessor.
Targets are learners
Learner Options:
All learners in this rotation
Learners from My Program/Outside of my Program: This refers to organizations that use on and off-service rotation slots (usually PGME programs).
Checking My Program will target leaners who are scheduled in the rotation AND enrolled in the course.
Checking Outside of my program will target learners who are scheduled in the rotation but enrolled in a different course than that in which the rotation exists.
Learner Levels: This refers to learner levels usually used in CBME enabled organizations. Checking a learner level will restrict the distribution to target only learners at that learner level. Leaving all boxes unchecked will target all learners, regardless of their learner level.
Additional Learners
Click Browse Additional Learners. Search for a learner and check the box beside the learner name. To delete a learner, click the x beside their name.
Specific learners in this rotation: Use the drop down selector to add the required learners. (Hover over a learner name to see their profile information.)
CBME Options: This will display whether you select All Learners or Specific Learners. This option applies only to schools using Elentra for CBME. Ignore it and leave it set to non CBME learners if you are not using CBME. If you are a CBME school, this allows you to apply the distribution to all learners, non CBME learners, or only CBME learners as required.
Targets are peers
This option allows for targets who have completed the rotation to assess all of their peers in the rotation block.
CBME Options: This option applies only to schools using Elentra for CBME. Ignore it and leave it set to non CBME learners if you are not using CBME. If you are a CBME school, this allows you to apply the distribution to all learners, non CBME learners, or only CBME learners as required.
Targets for this Distribution will be determined by the Assessor
With this option, the assessor/evaluator selects the target based on their role and distribution assessor options when they are completing the task.
In the Distribution Progress Report, administrators will see the target listed as "Assessor will choose before completing task."
Target Attempt Options: Specify how many times an assessor can assess each target, OR whether the assessor can select which targets to assess and complete a specific number (e.g. assessor will be sent a list of 20 targets, they have to complete at least 10 and no more than 15 assessments but can select which targets they assess).
If you select the latter, you can define whether the assessor can assess the same target multiple times. Check off the box if they can.
Click 'Next Step'
The assessors are the people who will complete the form.
Assessor Options:
Assessors are learners
Learner Options: All Learners of Specific Learners
All Learners: Select all learners in the rotation or specific learners in the rotation
All learners: Select from My Program and/or Outside of my program, use the drop down selector to add additional learners
Specific learners in the rotation: Use the drop down selector to add required learners
Additional Learners: Check this option to add additional learners from outside the program.
Assessors are faculty members
Browse faculty and click on the required names to add them as assessors
Select Associated Faculty: This tool will pull the names of faculty listed on the course setup page as associated faculty
Assessors are preceptors associated with a rotation schedule
This allows you to associated this distribution with all preceptors affiliated with the rotation based on the slot bookings made for learners. The distribution will dynamically update based on changes made to the rotation schedule.
Select individuals regardless of role (Individuals or external assessors)
This allows you to add external assessors to a distribution
Begin to type an email, if the user already exists you'll see them displayed in the dropdown menu. To create a new external assessor, scroll to the bottom of the list and click 'Add External Assessor'
Provide first and last name, and email address for the external assessor and click 'Add Assessor'
Exclude Self Assessments
Feedback Options: This will only display when the assessors are faculty. Checking the box will add a default item to the distribution asking if the faculty member met with the trainee to discuss their assessment.
Give access to the results of previous assessments: This relates to Elentra's ability to provide a summary assessment task to users. If enabled, tasks generated by this distribution will link to tasks completed in the listed distributions. When users complete the summary assessment task they will be able to view tasks completed in the other distributions. For any items that are used on both forms, results from previously completed tasks will be aggregated for the assessor to view.
Click to add the relevant distribution(s).
Currently for rotation-based distributions there is no option to configure minimum tasks completed on linked distributions nor a fallback date for the summary assessment task.
Click 'Next Step'
You can immediately save your distribution at this point and it will generate the required tasks, but there is additional setup you can configure if desired.
Authorship: This allows you to add individual authors, or set the distribution to be accessible to everyone with A+E access in a course or organization. (This may be useful if you have multiple users who manage distributions or frequent staffing changes.)
Target Release: These options allow you to specify whether the targets of the distribution can see the results of completed forms.
Task List Release:
"Targets can view tasks completed on them after meeting the following criteria" can be useful to promote completion of tasks and is often used in the context of peer assessments. Targets will only see tasks completed on them after they have completed the minimum percentage of their tasks set by you.
Target Self-Reporting Release: This controls whether targets can run reports for this distribution (i.e. to generate an aggregated report of all responses). When users access their own A+E they will see a My Reports button. This will allow them to access any reports available to them.
Target Self-Reporting Options: This allows you to specify whether or not comments included in reports are anonymous or identifiable. (This will only be applied if you have set reports to be accessible to the targets.)
Reviewers: This allows you to set up a reviewer to view completed tasks before they are released to the target (e.g. a staff person might review peer feedback before it is shared with the learner).
Check off the box to enable a reviewer.
Click Browse Reviewers and select a name from the list. Note that this list will be generated based on the course contacts (e.g. director, curriculum coordinator) stored on the course setup page.
Notifications and Reminders:
Prompted Responses: This allows you to define whom to send an email to whenever a prompted response is selected on a form used in the distribution. For example, if you have an item asking about student completion of a procedure and "I had to do it" was set as a prompted/flagged response, any time "I had to do it" is picked as an answer an email notification will be sent.
You can optionally select to email Program Coordinators, Program/Course Directors, Academic Advisors, Curricular Coordinators, or you can add a Custom Reviewer. If you select to add a Custom Reviewer you can select their name from a searchable list of users.
Sender Details: Define the email address that notifications and reminders will be sent from for a distribution.
Options are the distribution author, an existing user, an existing generic email, or a new generic email.
To create a new generic email provide a name and email address. This will be stored in the system and available to other users to use as needed.
Click 'Save Distribution'.
After the distribution is created, the distribution progress page will show targets as "Assessor will choose before completing task."
When an assessor/evaluator accesses the task they will see that the target is to be defined by them.
When completing the task, the assessor/evaluator will define their target.
If an existing distribution needs to be changed we recommend retiring the existing distribution and creating a new one. Additional details here.
For rotation-based distributions users might also edit a rotation schedule and individual learner bookings and that can impact any existing distributions.
If you are using a rotation-based distribution that relies on the clinical experience rotation schedule to determine the assessor (based on scheduled preceptors), updating a booking by changing the preceptor AFTER tasks have already been generated and delivered will result in the existing task being removed from the distribution progress report and a new one generating for the newly assigned preceptor. However, the original task will not automatically be removed from the original assessor's A&E task list. If the task was optional they will be able to delete it themselves, however, if the task was mandatory, at this point the best option is for an administrator to find the task from the Admin > A & E Dashboard and delete it there.
