The Resident Metrics Dashboard focuses on individual residents and is designed to be used by Residents and Competency Committee members.
The resident dashboard has a wealth of information that is grouped into different categories for easier comprehension. First if you arrived at the resident dashboard by selecting a resident on the normative dashboard their data is automatically fetched for you. However, if you manually switched over to the resident dashboard by clicking on the navigation tabs above, you will need to select a resident from the drop-down in the filter panel situated at the top of the dashboard. The drop-down contains the list of all the residents in the program with their name and their corresponding progress rate. The names of residents are further grouped by their training stage and then sorted alphabetically for easier access.
The drop-down is also an editable text box and so you can type a resident’s name partially to automatically filter the available options in the drop-down. This makes it easier to search for a particular resident in a program with many residents.
After selecting a resident, users can then click on the “GET RECORDS’’ button to visualize their assessment data. You might notice the small button with the calendar icon on it. This is used to highlight assessment data gained by the resident in a particular time period. For now, ignore it but we will learn more about it further down. The resident dashboard consists of several main sub sections. Let us look at each one individually.
This section provides the following summarized metrics of the resident:
Total EPAs observed - This is a count of the total number of EPAs filled out by a resident.
Progress Rate - This is the number of EPAs a resident has achieved divided by the total number of EPAs they are required to achieve for all the valid EPA forms in a program across the different training phases.
Achievement Rate - This is the total number of EPAs a resident has achieved divided by the total number of EPAs completed by that resident. An achieved EPA is one where the EPA meets certain requirements set in the assessment plan, such as acquiring a rating of 4 or above on a 5-point scale, or satisfying specific contextual variable requirements, or meeting diverse assessor role requirements.
To the right of the acquisition metrics is a line chart that shows the weekly EPA acquisition rate for the last six months by the resident. This is meant to give a high level overview at a quick glance of the residents assessment gathering in the recent past.
This section is meant to quickly lookup a residents’ recent performance with the option to view records in the following ranges: last 10 days, last 25 days, last month and last 3 months. The chart does not visually distinguish the different EPA types (i.e., EPA-F1 vs EPA-C2), instead, it provides this and other additional information in a pop-up menu that can be invoked by hovering the mouse over a point.
The line chart provides a simple representation of the last “N’’ assessments filled by the resident where every EPA is represented as a point with the oldest record starting on the left. The points are arranged vertically using the O-Score Entrustability scale with 5 being the highest (I did not need to be there) and 1 being the lowest (I had to do). The better a resident performs in an EPA, the higher is the vertical position of the point in the chart.
If a resident has assessments that were filled on EPA forms across several different rating scales then each rating scale is provided with its own recent EPA chart as shown in the above image.
The final section provides a detailed overview of every single EPA completed by the resident. The entire list of EPAs that residents are required to complete are broken down into four groups based on the training phase during which a resident is supposed to complete them and are numbered accordingly. With the addition of support for dynamic CBE the number of training phases can be higher or lower than 4. Some programs such as surgical foundation for example only have two training phases.
Each training phase is presented as a block with the title of the training phase and a label indicating whether the resident has completed the training phase or not. If a training phase is in progress a completion rate is shown to indicate the number of assessments the resident has achieved in that training phase relative to the total number of required assessments for every EPA in that phase. Each training phase block acts as an accordion and can be expanded or collapsed to view the list of all EPAs in that block.
Although residents generally complete the EPAs of their current training phase before they pick up EPAs of later phases, there are exceptions. Due to various external factors such as their rotation schedules and the nature of medical cases of the patients they attend to, residents can occasionally end up completing EPAs which are not in their current training phase. This means residents can have a non-zero completion rate for training phases that they have not yet started officially.
When a training block is open, all the EPAs in that block are arranged sequentially based on the numbering order in a 3-column layout as shown above.
EPA ID and a textual description of the corresponding medical scenario that the EPA targets.
The residents acquisition metrics for each EPA are provided as 3 numbers along with two bullet charts that visualize how far along the resident is in completing that EPA. If an assessment plan is not available for an EPA the required and achieved numbers default to “N/A” (not available). The first bullet chart (blue) visualizes the observed EPA count relative to the required count while the second bullet chart visualizes the achieved EPA count relative to the required count. A green check mark icon indicates the completed status of the EPA. It can show up either because the resident has achieved the required number of EPAs or if the competence committee has marked the EPA as complete (even if the achieved count is not met). In the scenario shown in the image above the latter is true. However, if an EPA has not been marked complete and the resident has not met the required achieved EPA count then a “TO GO” metric is shown in place of the check mark icon as shown above.
This is a visualization of all assessments filled by the resident for that EPA. The information is visualized like the recent EPA chart discussed above. Assessments are first grouped by the EPA form type and version number. This ensures that assessments of a similar variety are visualized together. Within each chart, assessments are arranged chronologically on the X axis with the oldest to the left and are arranged vertically based on the EPA rating (5-point O-Score Entrustability scale) with 5 being the highest (resident managed the situation independently) and 1 being the lowest (Assessor had to completely take over the situation). However, the rating scale is not always a standard 5-point scale and can change depending on the type of scale used in the assessment plan for a given EPA. For example, in the image shown below the EPA form has a 2-point (yes/no) rating scale.
Further each point in this chart can be hovered upon to view additional information about that assessment such as narrative feedback, situational context, assessor name and their role in an onscreen popup window as shown above.
Finally, three buttons are provided as seen in the bottom left corner of each chart. The first button (sliders icon) brings up a set of drop-down filter lists that can be used to visually identify a particular record based on patient demographics or other contextual variables such as “Case Complexity’’ or “Clinical Presentation’’. For example, if a user wanted to see which of the records were for “respiratory distress”, they could select that option from the Clinical presentation drop-down list and the corresponding points (observation scores) would turn light red.
The second button (book icon) can be clicked to see all the records in a table, which can be sorted and filtered through. To filter the table start typing in the input box at the top of each column in the table. This will dynamically filter the table as you type. To sort the table by a column simply click on the column header. For example, in the image below the table is being sorted in an ascending order by the first column (date).
The third button brings up a popup screen that shows the achievement criteria breakdown for the EPA. This feature has been duplicated from the main CBME dashboard where hovering over the “i” icon in an EPA gives a breakdown of the assessment criteria as shown above.
If a school has enabled the ability to track expired assessments, an optional section is visible at the end of the dashboard which shows a tabular breakdown of all the expired assessments filled against a selected resident.
This is a common feature across all the sections of the resident dashboard that highlights all assessments that were filled in a particular period. To enable this, head over to the filter panel at the top of the dashboard and click on the small button with the calendar icon on it. This will open a panel where you can set the start date and end date for the period. You can either type in directly into the input box or use the date selector on the calendar above.
Once the start date and end date are set, all assessments that fall in that period are converted into diamonds across the dashboard. This provides a way to visually distinguish these EPAs while still viewing them relative to other EPAs filled outside of the selected period. This feature can be particularly useful during competence committee meetings which happen once every three months such that the period can be set to highlight only the EPAs filled by the resident since the last meeting.
The checkbox provided in the date filter panel when enabled hides all EPA levels which do not have any assessments filled in the selected training period. If an entire training phase does not have any EPAs filled in the training period, then the whole training phase block is also hidden. This can be useful to reduce the visual clutter on the dashboard and only focus on a small subset of EPAs.
