To use the CBME Module, nine required curriculum tag sets must be in place. An auto-setup tool will help users create these.

Navigate to Admin>Manage Course>select a program, and click CBME.

If the required curriculum tag sets aren't yet configured the program will prompt users to auto-setup the required curriculum tag sets and will populate the roles, stages, and contextual variables tags.

Users without permission to use the auto-setup feature will be directed to contact a system administrator with the message below.

After the required, standardized curriculum tags have been populated, all programs must also complete and upload multiple templates through the CBME tab for a program. Note that program directors and program coordinators will have access to the CBME tab for their affiliated programs. The templates to complete are:

• The EPA Template

• The Program Specific Key Competency Template (optional)

• The Program Specific Enabling Competency Template (optional)

• The Milestones Template

• The Contextual Variables Template

To access these templates, navigate to Admin>Manage Courses, and search for the required program. Using the gear icon beside the program name, click 'CBME' and then click 'Import CBME Data' from the tabs.

On the right you will see a dropdown menu titled 'Download Example CSV Templates' for your use.

Click on the required template and it will download to your computer. Look for it in your downloads file if it doesn't immediately open.

Why are there two enabling competency templates?

There are two similarly named templates because of two options available: the option to import program specific enabling competencies, and the option to map EPAs to enabling competencies instead of milestones. Most programs will map EPAs to milestones and will never use the 'Enabling Competency Map Template.' However, if you have a program piloting CBME before its national launch they may choose to map to enabling competencies if milestones aren't finalized yet and we have left that option available. Use the Enabling Competency Template to import program specific enabling competencies. Use the Enabling Competency Map Template only if a program chooses to map EPAs to enabling competencies instead of milestones.

A program needs to complete this template OR the Enabling Competencies Template, not both. Most programs will complete the Milestones Template. Users define which option they will use in in Step 2 of the import CBME process.

Navigate to Admin>Manage course and select a course. Click 'CBME' and 'Import CBME Data' to access and download the template.

Three Columns to Complete

1) EPA code:Codes should be recorded in uppercase letters. Codes should refer to the stage of training and the EPA number, e.g. D1, D2, D3, F1, C2. Using consistent codes across your program and school is important. The system is setup to accept the following codes for learner stage: D: Transition to Discipline F: Foundations of Discipline C: Core Discipline P: Transition to Practice

2) Milestone code: Codes should be recorded in uppercase letters. The format for the milestone code is: Leaner Stage letter, followed by a space, CanMEDS Role letters, Key Competency number, followed by a period, Enabling Competency number, followed by a period, Milestone number

Note that there should be no space between the CanMEDS Role letter and the Key Competency number. CanMEDS Role refers to the CanMEDS roles and should be coded as follows:

Professional PR Communicator CM Collaborator CL Scholar SC Leader LD Advocate HA Medical Expert ME

Creating and confirming your Milestones codes takes patience. You'll likely notice that in some RC documents there is only a two digit milestone code. For the purposes of mapping your curriculum in Entrada, you must have three digit milestone codes. We recommend you add the third digit in the order the milestones appear in the RC documents you're using. Make sure you check for duplication as you go so that unique milestones have their own codes but different codes aren't applied to repeating milestones (within one stage). Using the data organization tools to reorder the milestone code columns and title columns can help you identify unneeded duplication.

You may also notice that the RC allows programs to use milestones coded from one stage in another stage (so you may see an F milestone in a C stage EPA). How programs and organizations handle this is ultimately up to them but we recommend that you align the milestone code with the stage that it is actually being assessed in/mapped to. For example, if it’s a “D” milestone being mapped to an F EPA, rename it as an “F” milestone. This is because you are (presumably) having a different expectation of performance on the milestone in a different stage, even if it’s the same task.

3) Title: The text of the milestone should be copied verbatim from the national specialty committee EPAs.

Mapping multiple EPAs to one milestone

The system is designed to allow multiple EPAs to map to one milestone. To do so, simply insert a comma between the EPA code entries in column one. e.g. D1,D2,F1

Save as a .csv and upload

After saving your file as a .csv, scroll to step 3 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

The EPA Template needs to be completed by all programs. This template organizes the information about EPAs decided by each national specialty committee and allows the information to be uploaded to the system.

Navigate to Admin>Manage Course and select a course. Click CBME and Import CBME Data to access and download the required template.

Template Columns to Complete

Each program should complete an EPA template with the following information:

• Code: This is the EPA code. It should be recorded in uppercase letters and include the learner stage of training and the EPA number. Use the learner stage letters outlined below to ensure that the EPAs correctly map to other curriculum tags. D: Transition to Discipline F: Foundations of Discipline C: Core Discipline P: Transition to Practice Sample EPA Codes include D1, D2, D3, F1, F2, F3, C1, C2, etc. Note that there is no space between the letter and number in the EPA Code.

• Title: This should be the EPA text. This can be copied from the EPAs as decided by each national specialty committee.

• Detailed Description: In some cases, these may be the same as the EPA title; in other cases, an EPA has an additional description. Not every program will use the ‘Detailed Description’ column. Note that this information is displayed to users in the EPA Encyclopedia.

• Entrustment required (optional): This field is optional but will populate the EPA Encyclopedia which is visible to all users.

Remember to use these codes for the learner stage on all templates:

D: Transition to Discipline F: Foundations of Discipline C: Core Discipline P: Transition to Practice

Save as a .csv and upload

After saving your file as a .csv, scroll to Step 1 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

A program needs to complete this template OR the Milestones Template, not both. Most programs will map milestones and there is no need to complete this template. Users define which option they will use in in Step 2 of the import CBME process.

Navigate to Admin>Manage course and select a course. Click CBME and Import CBME Data to access and download the template.

Columns to Complete

• EPA code: Codes should be recorded in uppercase letters. Codes should refer to the stage of training and the EPA number. e.g. D1, D2, D3, F1, C2. Using consistent codes across your program and school is important.The system is setup to accept the following codes for learner stage: D: Transition to Discipline F: Foundations of Discipline C: Core Discipline P: Transition to Practice

• Enabling Competency Code: Note that the template already includes a list of Enabling Competency Codes using the CanMEDS role prefixes. In the first column, simply add the EPAs codes you wish to map to each enabling competency.

  If there are unmapped Enabling Competency Codes left over you must delete those rows before you upload your .csv file.

Mapping multiple EPAs to one enabling competency

The system is designed to allow multiple EPAs to map to one milestone. To do so, simply insert a comma between the EPA code entries in the first column.

Save as a .csv and upload

After saving your file as a .csv, scroll to step 3 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

Note that you'll only have this step available if you decided to map program specific key and enabling competencies in import Step 2.

Use the Enabling Competencies Template if a program wants to import program specific enabling competencies. This template organizes the information about Enabling Competencies decided by each national specialty committee and allows the information to be uploaded to the system.

Navigate to Admin>Manage Course and select a course. Click CBME and Import CBME Data to access and download the required template.

Template Columns to Complete

Note that the first column, Enabling Competency, is completed for you with alphanumeric codes based on the CanMEDS roles. Adjust this list as needed to match the information from the national specialty committee.

When coding the Enabling Competencies remember these required codes for the CanMEDS roles: Professional PR Communicator CM Collaborator CL Scholar SC Leader LD Advocate HA Medical Expert ME

Complete the second column, Title, by providing the enabling competency text.

The third column, Description, is optional. This will store information but will not be visible to users.

Save your file as a CSV.

Uploading Your File

After saving your file as a .csv, scroll to step 4 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

Note that you'll only have this step available if you decided to map program specific key and enabling competencies in import Step 2.

Use the Key Competencies Template if a program wants to import program specific key competencies. This template organizes the information about Key Competencies decided by each national specialty committee and allows the information to be uploaded to the system.

Navigate to Admin>Manage Course and select a course. Click CBME and Import CBME Data to access and download the required template.

Template Columns to Complete

Note that the first column, Key Competency, is completed for you with alphanumeric codes based on the CanMEDS roles. Adjust this list as needed to match the information from the national specialty committee.

When coding the Key Competencies remember these required codes for the CanMEDS roles: Professional PR Communicator CM Collaborator CL Scholar SC Leader LD Advocate HA Medical Expert ME

Complete the second column, Title, by providing the key competency text.

The third column, Description, is optional. This will store information but will not be visible to users.

Save your file as a CSV.

Uploading Your File

After saving your file as a .csv, scroll to step 3 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

When importing data to create EPA maps, programs are required to upload a contextual variable template to provide contextual variable response options (e.g. case complexity: low, medium, high). In addition to this information, programs will need to add specific details about any procedures included in their contextual variable response options. This additional information, called procedure attributes, plays an important part in building procedure assessment forms. Think of the procedure attributes as assessment criteria (e.g. obtained informed consent, appropriately directed assistants, etc.) for each procedure .

Required Information

For each procedure in your program, you must define headings, rubrics and list items that may be used to assess a learner completing the procedure. Headings appear at the top of a form, rubric becomes the subtitle of a rubric, and list items are the things that will actually be assessed on the rubric.

All this information must be stored and uploaded in a CSV spreadsheet that uses two columns: Type and Title.

Within ‘Type’ there are three things you can include: H to represent heading (e.g. Participate in Surgical Procedures, NG Insertion) R to represent rubric (e.g. Procedure, Pre-Procedure) L to represent list items (e.g. Apply knowledge of anatomy)

The three procedure criteria form a hierarchy. A heading can have multiple rubrics, and a rubric can have multiple list items. Arrange the criteria in your spreadsheet to reflect their nesting hierarchies.

In the 'Title' column enter the information to be uploaded.

When complete, save the file in a CSV format.

Sample Procedure Form On this sample form, the following information was uploaded as procedure attributes: H: Participate in Surgical Procedures R: Procedure L: (ME1.3.2) Apply knowledge... (ME3.4.2) Perform procedural tasks... Use common surgical instruments...

Tip: You can use the same file for multiple procedures if they share the same heading, rubric, and list information.

Uploading Procedure Attributes

• Ensure you are in a Program Coordinator or Program Director role.

• Navigate to Admin>Manage Courses/Programs.

• Scroll down or type in text and click on the relevant program name to select it.

• Click on ‘CBME’ on the tab menu.

• Click on ‘Contextual Variable Responses’ on the second tab menu.

• From the list of contextual variables, click on ‘Procedure’.

• Beside each procedure you will see a small black up arrow beside a green disk icon.

• Click on the up arrow to upload a .csv file of information for each procedure.

• Either choose your file name from a list or drag and drop a file into place.

• Use the dropdown menu to select the EPA to apply your procedure attributes to.

• Click ‘Save and upload criteria’.

Repeat this process for each procedure and each EPA. Remember, a program can use the same procedure attributes for multiple procedures and EPAs if appropriate.

Managing Existing Procedure Attributes

You can view what attributes were already uploaded to a procedure by clicking on the green check mark in the criteria column. You can review the criteria posted for a specific EPA and opt to replace the criteria for that specific EPA.

After a program has uploaded an initial set of contextual variable responses, they can be managed through the CBME tab.

Navigate to Admin>Manage course and select a course. Click CBME and then click the Manage Contextual Variable Responses tab below.

If contextual variable response information has been uploaded for the program, you will see a list of contextual response variables.

To modify the contextual variables within a category, click the title of the contextual variable or click 'Show' to the right of the category you want to edit.

To edit an existing response, make the required change and click the green disk icon.

To delete an existing response, click the red minus button beside the response.

To add a new contextual variable response, click the black plus button at the bottom of the category window. This will open a blank response space at the end of the list. Fill in the required content and save.

Troubleshooting

If you get a yellow bar across the screen when you try to modify contextual variables it means none have been uploaded for the program you are working in. Return to the Import CBME tab and complete Step 4.

Uploading Templates

All templates you upload to the system must be in .csv format.

If you are working in Excel or another spreadsheet manager use the “Save as” function to select and save your file as a .csv

To upload your files, either search for or drag and drop the file into place. You'll often see a green checkmark when your file is successfully uploaded.

If you receive an error message make sure that you have:

• Deleted any unused lines. This is especially relevant in templates with pre-populated columns including the contextual variables template, and enabling competencies mapping template.

• Completed all required columns. If a column is empty for a specific line the file may fail to upload.

Viewing EPA Maps

If you've imported all CBME data for a program but you are not able to see EPA maps in the EPA Encyclopedia or in the Map Curriculum Tags tab of CBME, double check that you've uploaded your files without spaces between the EPA Code letter and number, nor between the CanMEDS Role and Key Competency number. Having spaces in the incorrect places will prevent the system from retrieving the required information to produce maps.

Mistakes When Mapping Relationships

While you can correct minor typos and add additional mapping info. through the user interface you can not delete, or undo mapping between, EPAs and milestones or enabling competencies. There is an option to reset all CBME data for a program. This will remove all the uploaded information and delete all existing forms (complete forms will remain on the learner's CBME dashboard but you'll have to rebuild forms to allow them to be triggered moving forward). The Reset CBME Data option is only available to users with administrator role permissions. Find it on a program's Import CBME Data tab.

Although there is the ability to upload program specific key and enabling competencies in Elentra ME 1.12, organizations are still required to upload a standard list of key and enabling competencies at the organization level. For Canadian schools using CBME we recommend making the CanMEDS Key and Enabling Competencies as your standard list. These are available online or from another institution.

To download Elentra templates, complete them, and upload key and enabling competencies information, navigate to Admin>Manage Curriculum>Curriculum Tags.

Click on a curriculum tag set name and then click 'Import from CSV' in the top right to access and download the template to complete.

After you complete each required template, save it in a .csv format and upload it from the same screen you were on.

Accessing CanMEDS Key and Enabling Competencies

You can access the CanMEDS Key and Enabling Competencies at http://canmeds.royalcollege.ca/guide. The Key Competencies are preceded by one number like 1 or 2, and are stored under CanMEDS roles. The Enabling Competencies are preceded by a number like 1.1 or 2.2 and are also stored under CanMEDS roles.

Sample Key Competency: 1 Practise medicine within their defined scope of practice and expertise

Sample Enabling Competency: 1.1 Demonstrate a commitment to high-quality care of their patients

Both of these are from the Medical Expert role so would be coded with the prefix ME. Alphanumeric codes are provided in the templates, you will need to input the key and enabling competency text.

You can export the text from the CanMEDS guide.

CanMEDS Key Competencies

Download the template through Manage Curriculum>Curriculum Tags>Key Competencies.

Note that the first column, objective_code, is completed for you with alphanumeric codes based on the CanMEDS roles.

Complete the second column, objective_name, by providing the key competency text. You can access the key competencies at http://canmeds.royalcollege.ca/guide or request them from another institution.

Save your file as a CSV.

CanMEDS Enabling Competencies

Download the template through Manage Curriculum>Curriculum Tags>Enabling Competencies.

Note that the first column, objective_code, is completed for you with alphanumeric codes based on the CanMEDS roles.

Complete the second column, objective_name, by providing the enabling competency text. You can access the enabling competencies at http://canmeds.royalcollege.ca/guide or request them from another institution.

Save your file as a CSV.

Uploading Templates and Setting as Standardized Curriculum Tag Set

To upload completed templates, navigate to Manage Curriculum>Curriculum Tags, and select Enabling Competencies or Key Competencies. Use the 'Import from CSV' button in the top right to upload your templates. Search for or drag and drop a file in place, then click the green 'Import' button.

After you upload an EPA Template, you will be prompted to select how you'd like to continue mapping your EPAs. Each program has two decisions to make: 1. Competency Options: Decide whether to upload program specific key and enabling competencies. If you select to use the standard key and enabling competencies (e.g., CanMEDS key and enabling competencies) the system will copy the items previously uploaded via Manage Curriculum and make them the program's default key and enabling competencies. 2. Curriculum Tag Options: Decide whether to map EPAs to Milestones or Enabling Competencies. Most programs will map their EPAs to Milestones.

It is possible to configure the Elentra database to prevent users from having the choice between using course-specific or standard key and enabling competencies. If you get to an option that ONLY asks whether you want to map to milestones or enabling competencies consult with your CBME implementation lead as to whether your organization should have the option to provide program specific KCs and ECs.

Contextual Variable Responses are used to describe the context in which a learner completes something. Examples of contextual variables include diagnosis, clinical presentation, clinical setting, case complexity, patient demographics, etc. For each variable, programs must define a list of response options. For example, under clinical presentation, a program might include cough, dyspnea, hemoptysis, etc.

Elentra will autosetup a specific list of contextual variables in the Manage Curriculum interface. Institutions can also add contextual variables (e.g., assessor's role) to this list before uploading contextual variable responses (e.g., nurse, physician, senior resident) via each program's CBME tab.

Do not delete any contextual variables from the existing list within Manage Curriculum or the CBME auto-setup feature will prompt you to re-add them the next time you use CBME. If you wish to add to the contextual variables list you may do so manually through Manage Curriculum and then import your responses through the CBME tab. By default Elentra is set up to provide access to the same Contextual Variables for all programs. You don't necessarily have to provide responses for all CVs in all programs but by default each program will see the same list of CVs.

Adding Additional Contextual Variables

Elentra's auto-setup feature will pre-populate a specific set of contextual variables. Users will be able to upload response options for that list through the CBME tab. However, institutions may wish to add additional contextual variables to make available to their programs. You can do this by following two steps.

1) Through Admin>Manage Curriculum>Curriculum Tags>Contextual Variables you can add new contextual variable tags (e.g. observer role, consult type, domain) that will become visible and useable to all programs within the organization. When you add these new tags to the tag set, you'll be required to provide a code and title. It is recommended that you make the code and title the same, but separate the words in the code with underscores. Example: Title: Observer Role Code: observer_role Please note that there is by default a 24 character limit for curriculum tag codes and a 240 character limit for curriculum tag titles. If you enter more characters than the limit, the system will automatically cut off your entry at the maximum characters. 2) After you have added Contextual Variables to the existing tag set, you will be able to add the new response codes (and responses) to the contextual variable response template and successfully import them.

Importing Contextual Variable Responses

Once you have built out the list of all contextual variables you want included in the system you can import specific contextual variable responses for each program.

Teamplate Columns to Complete

Navigate to Admin>Manage course and select a course. Click CBME and Import CBME Data to access and download the contextual variable response template. Complete the following columns:

• Response Code: This column is pre-populated by fields including assessor’s role, basis of assessment, case complexity, case type, etc. You can add to this list if you have built additional contextual variables through Manage Curriculum (see above). Make sure to use the exact response code from the newly created contextual variable tag in the response_code column.

• Response: In this column, you can add the response variables required (e.g. Case complexity response variables: low, medium, high). To add more than one response variable per response code, simply insert another line and fill in both the response code and response columns.

• If there are items in the response code column that you don’t use, you must delete them before you upload the file.

• Description (optional): This field is not seen by users at this point and simply serves as metadata.

Save as a .csv and upload

After saving your file as a .csv, scroll to step 4 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

Staff>Admin. users have the ability to completely reset a program's CBME data. This is a dramatic step and should be used judiciously. Resetting CBME data will delete all EPAs, competencies and milestones for a program and they will have to be reloaded. Additionally, all existing forms built for the program will have to be rebuilt.

Resident Data

If you reset CBME data for a program, all existing, completed forms remain in existence. They will display on a learner's Assessment and Evaluation tab but will not display on the CBME dashboard Stages tab as the dashboard Stages tab will reflect data that relates to the newly loaded EPAs.

A major change for CBME in Entrada ME 1.12 is the ability to create assessment plans for a program. Currently these simply house information about the requirements for each program, but eventually reporting tools will be built to reflect the assessment plan information.

The assessment plan allows you to set minimum completion requirements for the different assessment tools built for a program. These minimums can apply to the number of assessments, the number of assessors, and specific contextual variable responses.

Please note that currently the assessment plan does not apply to procedure forms.

You need to have EPAs mapped and forms built before you can create an assessment plan.

Mapping Entrustable Professional Activities, key and enabling competencies, milestones, contextual variable responses and procedure attributes for each program is what allows users to quickly and efficiently create assessment tools and report of resident progress. There are two stages in mapping curriculum; in the first stage a central administrator (e.g. PGME office) needs to configure Elentra, then, in the second stage, the curriculum information specific to each program needs to be uploaded.

Steps taken once, at the organisational level, include:

• auto-setup CBME features

• Import central key and enabling competencies,

• Import custom list of contextual variables as needed

• Build assessment scales as needed

Then individual programs can upload Entrustable Professional Activities, key and enabling competencies, milestones, contextual variable responses and procedure attributes.

If you want to manually map a connection between an EPA and a milestone, you can do so through the Map Curriculum Tags tab. This allows you to map additional milestones to EPAs after you have uploaded your templates.

• Navigate to admin, manage course and select a course.

• Click CBME and then map curriculum tags.

• Scroll down to the Map Curriculum Tags area.

• Either type in and select the tags required to sort the displayed results OR click through the displayed options to create the mapping structure you require.

  • You can display the results for more than one role at a time by clicking on multiple roles. Note that any already mapped milestones will still show up in the milestone list.

• Click to highlight the appropriate tags and then click ‘Save Curriculum Map’.

Improve learner identification of priority EPAs

From the CBME dashboard, EPAs can show their priority and the likelihood of being encountered within a learner's rotation. For this feature to be active you must also be using the rotation scheduler in Elentra.

Adding Priority and Likelihood Levels

Navigate to Admin>Rotation Schedule Create a draft schedule or open a published schedule. If you are create a new rotation schedule, you will also need to add rotations to the schedule. Once rotations exist within a schedule a small Objectives badge will appear beside the rotation name. Click on the Objectives badge to open a list of EPAs tagged to the rotation.

Click the plus sign beside any EPA you'd like to label with priority or likelihood. Indicate the likelihood by clicking on the appropriate button. If you'd like to set an EPA as a priority, click the exclamation mark in the priority column. Repeat as necessary. Return to the rotations list using the Back to Rotations button.

To modify existing EPAs and milestones navigate to the CBME tab of a program page. In the appropriate step (e.g. Import EPAs or Import Milestones), look for "Click here if you wish to make changes to existing EPAs" and click on the blue text.

This will open a window where you can edit the text, description, and entrustment of EPAs. Note that this should only be used to make small modifications to this information and should not change the meaning of the text.

If you edit an EPA, it will be updated automatically across the system including on all completed forms. If you edit any milestones, you will have to republish forms on which they are used. Edits made to milestones will not be made on already published forms.

If you need to add new information to your EPAs, enabling competencies, or milestones templates, simply add new information to the existing template and upload it again in the appropriate spot. The system will identify any NEWLY ADDED EPAs/milestones/enabling competencies and append them to the existing EPA information.

Because of this feature, it can be valuable to keep a copy of the most recent file you’ve uploaded at all times.

If you need to correct a typo or minor error in an EPA, milestone, etc. you can now do so in through the user interface. Navigate to the Import CBME Data screen and follow the instructions on the screen to edit existing EPAs, milestones, etc.

After sufficient information has been added to the system, a user can generate a map showing the connections between EPAs, roles, key competencies, enabling competencies, and milestones. This information can be access through the EPA Encyclopedia or a program's CBME page.

EPA Encyclopedia

From the dashboard, access the EPA Encyclopedia from the left sidebar.

If you have access to multiple programs, set the program using the dropdown menu in the top left.

Search for an EPA by typing in its code, or scroll down to find an EPA.

After clicking on an EPA, click on Detailed Description, Entrustment or Program Map to view details about the EPA.

Viewing Maps from the CBME page

Navigate to Admin>Manage course and select a course. Click CBME and the Map Curriculum tab.

Scroll down to click on an EPA or type the beginning of the EPA code and title into the dialogue box and click on the relevant EPA when it appears below.