If you are building and testing supervisor forms, you'll want to set up the faculty PIN for some sample users so that your test forms can be completed and you can view progress on the CBME dashboard. When CBME is live for your organization, users will need to set their own pins in order to easily complete forms at the point of encounter. Faculty and learners can set PINs. This allows residents to complete peer assessments. Learners will only be able to trigger assessments on themselves and have a peer complete the form via PIN; residents cannot trigger forms on other learners.

Setting the PIN

To set a PIN, log in as the necessary user. Access the user's profile by clicking on the user's name at the top of the screen (to the right of the logout button). Once in the user's profile, click on the Account Information tab. Beside PIN enter a 4 to 6 digit PIN. Click 'Save Profile'. Done!

From the Dashboard, faculty have access to a green 'Trigger Assessment' button on the right side of the screen.

First faculty select a resident. They can begin to type a name to limit the list of options. When they mouse over a resident name, faculty can see the resident card including their name, photo (if uploaded) and program.

If the learner is enrolled in two programs the faculty will have to specify a program.

A date of encounter will also have to be set.

Next faculty select an EPA.

For a reminder on what is included in a specific EPA the black question mark provides a quick link to the EPA Encyclopedia (see above).

Users can easily navigate the list of EPAs by applying preset filters including Current Stage EPAs, Current Rotation EPAs, and Priority EPAs. Click on a filter title to apply it. In this example the Priority EPAs filter is being applied.

After an EPA is selected, the available assessment tools will be displayed.

Note the small clock in the top right of each tool. This is an estimate of how long the form will take to complete based on the experience of other users.

Faculty can click 'Begin Assessment' on any tool to begin.

From the Dashboard, learners have access to a green 'Trigger Assessment' button on the right side of the screen.

First learners select a faculty. They can begin to type a name to limit the list of options. When they mouse over a faculty name, learners can see the faculty card including their name, photo (if uploaded) and program.

Next, learners select an assessment method. Details about each assessment method are provided inline.

Note that faculty must have a PIN setup for learners to select the first option.

A date of encounter will also have to be set.

Next learners select an EPA.

For a reminder on what is included in a specific EPA the black question mark provides a quick link to the EPA Encyclopedia.

Users can easily navigate the list of EPAs by applying preset filters including Current Stage EPAs, Current Rotation EPAs, and Priority EPAs. Click on a filter title to apply it. In this example the Priority EPAs filter is being applied.

After an EPA is selected, the available assessment tools will be displayed.

Note the small clock in the top right of each tool. This is an estimate of how long the form will take to complete based on the experience of other users.

Learners can click 'Begin Assessment' on any tool to begin.

Depending on the method of assessment learners will either be directed to the assessment form to start it, or the form will be sent to the assessor.

Sample: Submit via PIN

In the first assessment method, learners will be able to save their form as draft or submit it to the assessor to confirm by PIN.

Different assessment methods will provide different options.

This feature allows residents to cognitively situate their assessors by adding an optional note to any assessment that will be emailed (complete and confirm via email; email blank form; self-assessment, then email blank form). It can be used to remind the assessor about specific case details, provide a focus for assessment, or anything else that the resident feels the assessor should know before completing the assessment. This note will stay attached to the assessment and will be visible from the CBME dashboard for reference.

This cue will stay attached to the assessment and will be visible from the CBME dashboard for reference.The cue is optional and if provided it will appear at the top of the assessment for the assessor. The cue can also be seen on the completed assessment for both the resident and the faculty member.

• For “Email blank form,” the cue modal will pop up from the Trigger Assessment page after “Send Assessment” has been clicked.

• For “Complete and Confirm via Email” and “Self-assessment, then email blank form”, the cue modal will pop up after clicking “Submit and notify attending by email” at the bottom of the form.

There is a tool to allow admin users to reopen a completed assessment and edit it as needed. This is a feature that can be turned on or off depending on how your organization wants to use it.

If enabled, staff>admin can access this feature by navigating to their Assessment & Evaluation badge (beside their name in the top left), clicking on My Learners and then the Assessments tab for a specific learner. From the list of Tasks Completed on Learner click on a task to view it. An orange "Reopen Task" button will be above the form. This will set the task to in-progress and the staff>admin or faculty will be able to adjust it. Note that once a task is reopened you can clear all existing form data and forward the task as needed to send an email to faculty alerting them to the new task status.

Use with caution!

This tool should be used judiciously to ensure that residents are not ”gaming” the system and bullying anyone into changing their assessments to be more favourable.

Faculty, program administrators and residents can easily review a learner's progress from the CBME dashboard.

Note that the CBME dashboard includes several tabs: Stages, Assessments, Assessments Items, Trends, Comments, and Pins. (Reports can be generated from the Stages page.)

There is another Assessments Dashboard that pulls in additional form information if your organization is using the Assessment and Evaluation module for other forms (e.g. program and faculty evaluations). This page discusses just the CBME Dashboard.

Accessing CBME Dashboards

When logged in as a faculty member or program coordinator, click the Assessment & Evaluation badge that appears between the user's name and the logout button in the top right. Click on 'My Learners' from the tab menu. Search for a learner as needed, and if you can't find someone, ensure that you are in the correct curriculum period using the dropdown menu on the right. After finding the correct learner, click on CBME Dashboard to view the learner's progress. Residents automatically land on their CBME Dashboard when they log into Elentra.

Reviewing Learner Progress

From a learner's CBME Dashboard, click through the different tabs to view a range of assessment information. On most tabs, you can apply filters in order to refine the list of assessments visible to others. To use this option, select all the appropriate filters (e.g. date, curriculum tag, contextual variable, assessment tool, etc.) and then click 'Apply Filters'. Note that the filters a PA or PD applies on one learner will be maintained as you move through different pages.

Stages

From the stages tab you can see a summary of a learner's progress across EPAs and stages.

EPAs are displayed in order of learner stage and completed stages can be collapsed. If you are using the rotation scheduler, EPAs specific to a learner's current rotation are outlined in the stage colour and the priority and likelihood of an EPA in the learner's specific rotation is shown through the exclamation mark and bar chart (unlikely, likely, very likely). Whether or not the rotation scheduler is in use, green checkmarks indicate that a stage or EPA is complete (this is set by the Competency Committee).

Click the down arrow on the right side of an EPA card to see a list of completed assessments for that EPA. Depending on the form there may be a count of the learner's global assessment rating, which you can click on to access an aggregated report of the learner's performance on this form.

When in a faculty or PA role you can also access a Milestone Report from the Stages tab. More details about the Milestone Report are in another lesson.

Stages Tab: Reporting Options

Users can produce an aggregated report for any form displaying on this page. Click the grey down arrow on the right side of a EPA card, and click on a form title. A new tab will open to display the aggregated data with a trend chart at the bottom. On the trend chart at the bottom of the report hover over any individual assessment to click 'View Assessment' and see the specific assessment form. Note that you can download a PDF of this form as needed (green button at top right). If you want to look at only the forms completed with a certain global rating click on a blue number on the right hand side.

Assessments

See a list of all completed assessments and filter as desired. Toggle between completed, in progress, pending and deleted assessments below the filter settings. On each individual assessment card, note that the form type and relevant EPA are displayed. You can also click the down arrow on the right to show some form information (e.g., global rating, comments, assessor and assessment method), and click 'View Details' on the left to see the assessment form. Users can mark an assessment as read by clicking on the eye icon. A greyed out eye means an assessment is unread. There is an option to mark all assessments as read on the right side above the assessment cards. You can also quickly view all unread assessments by clicking the small blue number beside 'Assessments' in the tab menu. Marking assessments as read or unread is user specific so learners and faculty will see their own results. Users can pin an assessment from this screen and learners can give an assessment a "thumbs up" to provide feedback to an assessor about their feedback.

Assessments Items

Quickly see a list of all completed assessment items and filter as desired. Click the down arrow on the right to see comments, response descriptor and the name of the assessor (as applicable). Click View Details at the bottom left of each card to access the completed assessment form.

Users can pin an assessment item from this screen.

Trends

View trends in learner performance across global assessment ratings. Note the overall tally of ratings in the top right corner of each card. Hover over on a node on the chart to view information about the form name, date and rating; you can also click through to access the actual assessment form.

Comments

Quickly access a list of all narrative comments provided to the learner on any complete CBME assessment tool. The tool type and relevant EPA will be displayed below the form name on each comment card.

Users can pin comments from this tab.

Pins

Quickly view all assessments, items, or comments that have been pinned. Apply filters as desired, or just scroll down to find the toggle to switch between assessments, items, and comments.

Program Administrators and Faculty Directors have the ability to get an overview of pending and in progress tasks from Admin>Assessment and Evaluation. When users access this page they can now view 'Outstanding' tasks as well as 'Upcoming' and 'Deleted' tasks.

Viewing Outstanding Tasks

1. Users can filter for distributed forms or triggered forms.

2. Users can search for specific forms, or owners.

3. The 'Owner' is the person who is responsible for completing the form.

4. Hover over Targets to view the targets of the form.

5. Check the appropriate boxes and then send reminders as needed.

6. Delete tasks as needed.

Click on any form to complete it on behalf of another user.

(Please note, not all users will currently see the Trigger Assessment button included in this image.)

The list of users included in the Outstanding task list are those faculty listed as "Associated Faculty" on the program setup tab, faculty who have been included in a distribution in the course, and learners in the audience of a program enrolment (also configured on the program setup tab).

PAs can enter complete assessment forms on behalf of assessors.

Navigate to Admin>Assessment & Evaluation. Click on the green 'Record Assessment' button below the orange Assessment Tasks heading. Select a resident and assessor from the dropdown menus and provide a date of encounter. Select an EPA as you would to trigger a form. Search for the appropriate form and click Select. Complete the form and click Submit. The PA will be submitting the assessment on behalf of the selected assessor. There is a reminder of the selected assessor displayed at the top of the form in a yellow information bar.