After the required, standardized curriculum tag sets have been populated, all programs must also complete and upload multiple templates through the CBME tab for a program. Note that program directors and program coordinators will have access to the CBME tab for their affiliated programs. The templates to complete are:

• The EPA Template

• The Program Specific Key Competency Template (optional)

• The Program Specific Enabling Competency Template (optional)

• The Milestones Template (or the Enabling Competency Map Template)

• The Contextual Variables Template

To access these templates:

• Navigate to Admin > Manage Courses.

• Search for the required program (you may only have access to one depending on your role in your organization).

• Using the gear icon beside the program name, click 'CBME'.

• Click 'Import CBME Data' from the tab menu below the Competency-Based Medical Education heading.

• With the introduction of EPA Versioning, you will now be prompted to indicate which EPAs you are changing; if you are uploading a program's CBME data for the first time, this does not apply and you can click 'Next' on Steps 1 and 2 to move forward.

• After skipping Steps 1 and 2, you'll arrive at a screen showing Steps 1 - 6 for importing CBME data.

• On the right you will see a dropdown menu titled 'Download Example CSV Templates' for your use.

• Click on the required template and it will download to your computer. Look for it in your downloads file if it doesn't immediately open.

Mapping Entrustable Professional Activities, key and enabling competencies, milestones, contextual variable responses and procedure attributes for each program is what allows administrators to use the CBME form templates to quickly create assessment tools. There are two stages in mapping curriculum; in the first stage a central administrator (e.g. PGME office) needs to configure Elentra, then, in the second stage, the curriculum information specific to each program needs to be uploaded.

Steps taken once, at the organisational level, include:

• Auto-setup CBME features,

• Import standard list of key and enabling competencies,

• Import custom list of contextual variables as needed, and

• Build assessment rating scales as needed.

Then individual programs can upload Entrustable Professional Activities, key and enabling competencies, milestones, contextual variable responses and procedure attributes.

What is considered a new EPA?

Think in terms of the EPA or Milestone code. A new version of an EPA is when:

1. The EPA title/description has changed sufficiently to change the context of the EPA, AND/OR

2. You are changing, adding, or retiring any milestones within the EPA, AND/OR

3. You are changing the EPA code association (e.g., you have retired an EPA in the middle of your numbering, such as D2, and now need to shift D3, D4, D5, into D2, D3, D4).

A new version of a Milestone is when:

1. The Milestone title has changed sufficiently to change the context of the Milestone.

What do the objective statuses mean?

Replaced

Select this option if you wish to replace the EPA entirely, including all milestones. This is the easiest option to work through the wizard. This option will require you to have a complete set of CSV files, rather than just the changes. Use this option if you have many changes to implement across EPAs and milestones.

Not changing

The objectives will remain unchanged. This exact version of the objective will be carried-forward into the new curriculum version. Note that selecting this option will stop any objectives with the same code from being uploaded. Any EPAs marked as not changing will have all existing tools carried over.

Changing

The objective will be changing. A new version of this EPA will be created, which may include objectives within it that you mark as changing.

For a milestone (or competency), if you select “changing” on one EPA, the system will update all other instances of that milestone to “changing”. This will ensure that you only have one version of that milestone.

Retire

The EPA (remember, think in terms of EPA Code, e.g. "C1") is being retired from the next version. This means that there will be no new version of that EPA Code in the next curriculum version. If you include an EPA with the retired code in the spreadsheet, it will not get uploaded. You are unlikely to use this status for EPAs unless there is an EPA at the end of your numbering that is being removed. For example, you have D1, D2, D3, D4 currently. You are retiring D4 from the next version and no new D4 will be uploaded.

For milestones, a “retired” status will remove that milestone from that EPA in the new version. This means that there will be no new version of that milestone for that EPA. It will not retire it from other EPAs unless you explicitly tell it to.

What do I need to do to prepare?

Please ensure that you have documented any changes that will need to be made, in addition to creating (or modifying) new CSV templates for upload. From the user interface, you will be able to indicate:

• Whether an EPA is being replaced, changing, not changing, or being retired

• Within each EPA that has been marked as changing, indicate if any KC, EC, or Milestones will be changing, not changing, or retired

• If you are adding milestones to an EPA, mark it as changing so that the uploader will add the new milestones

• Entirely new EPAs will be detected by the uploader without you needing to indicate anything. For example, if you currently have D1, D2, and D3, but will be adding a D4 and D5, the system will detect these in your spreadsheet and add them to the new curriculum.

• The new curriculum version will be applied to each learner at the stage level. This means that if a learner is currently in Foundations, they will retain the previous version of Transition to Discipline and Foundations but will be given the new version of Core and Transition to Practice. Please ensure that all of your learners are in the correct stage (on their CBME Dashboards) prior to publishing the new curriculum version. Remember: a green checkmark on the stage indicates that the learner has completed that stage.

Which CSV files do I need to have ready?

1. EPA Template

2. Milestone Template

3. If applicable, any new program-specific Key and Enabling Competencies

How will my assessors know which EPA version to pick for each learner?

Great news: When triggering an assessment, only the EPAs and associated forms relevant to that learner will be displayed. You will not have to choose between multiple versions for a given learner.

Do I need to build new assessment tools?

You only need to build new assessment tools for new EPAs or EPAs that have changed. Assessment tools will be carried over for all EPAs that were marked as "not changing."

Please note, as soon as you have published your new version, you will need to build tools for all of the new EPAs before residents can be assessed on them.

Which learners will get the new version of the curriculum?

The new curriculum version will be applied to each learner at the stage level. This means that if a learner is currently in Foundations, they will retain the previous version of Transition to Discipline and Foundations but will be given the new version of Core and Transition to Practice. Please ensure that all of your learners are in the correct stage (on their CBME Dashboards) prior to publishing the new curriculum version. Remember: a green checkmark on the stage indicates that the learner has completed that stage. A learner is currently in the stage that comes after the last green stage checkmark.

Building Form Templates for New EPA Versions

Please note that versioning support for Procedure Form Templates is awaiting review and is not currently included in Elentra.

1. Navigate to Admin > Assessment and Evaluation.

2. Click ‘Form Templates’ on the tab menu.

3. Click the green ‘Add Form Template’ button in the top right and a pop-up window will appear.

4. Type in a form name and select the form type from the dropdown menu. Select ‘Supervisor Form’. If you have permission to access multiple programs, use the dropdown menu to select the appropriate program for this form. This option will only show up if you have access to multiple programs.

Now that you have two (or more) curriculum versions in the system, the Form Template Builder will default to loading the most recent version. In the "EPA Version" tab, simply select the appropriate version. Click Save.

If you want to build new forms for learners using Version 1, change the EPA Version to Version 1, click Save, and it will load the appropriate EPAs.

EPA Selection

Select which EPAs can be assessed using forms generated from this template.

To remove EPAs, click on the small 'x' to the left of the EPA code. You can add back any required EPAs by clicking on the dropdown menu and checking off the tick box for a desired EPA.

Click the grey badge beside an EPA to select or remove specific milestones for forms built from this template.

Click Save.

Contextual Variables

If you want all EPAs to have the contextual variables, leave all EPAs checked off. If you’d rather specify which contextual variables apply to which EPAs simply uncheck an EPA and it will appear below with its own customizable list of contextual variables.

You can remove specific contextual variable responses by clicking on the grey button beside a contextual variable.

You may only select between 1 and 6 contextual variables per EPA per supervisor form.

Click Save.

Enabling Competency/Milestone Scale Selector

Use the first dropdown menu to select the scale you want to use to assess enabling competencies or milestones.

Indicate whether comments are disabled, optional, mandatory, or prompted. By selecting ‘Prompted’, you can set the system to prompt and require a comment when any flagged response is selected by an assessor.

The default response feature allows you to pre-populate a form with the selected response.

Disabled - Comments are disabled at the milestone level.

Optional - An optional comment box will appear for each milestone. Comments are not required for a form to be submitted.

Mandatory - A mandatory comment box will appear for each milestone. Comments are required for all milestones before a form can be submitted.

Prompted - A mandatory comment box will appear only for the prompted responses indicated. This is a popular option for responses at the lower end of a scale.

Global Rating Scale Selector

From the first dropdown menu, select a Global Rating Scale. This will populate the Item Text and the Responses sections.

From the second dropdown menu, indicate whether comments are disabled, optional, mandatory, or prompted. By selecting ‘Prompted’ from the Comments dropdown menu, you can set the system to prompt and require a comment when any flagged response is selected by an assessor.

Publish Form Template

Click 'Publish' to make your template available for use. The forms will be available within the hour.

Once a form template has been published, you can rearrange the template components for each form; however, you cannot makes changes to the scales or contextual variables. To make these changes, copy the form template and create a new version.

Default Content

On each form template you create you’ll notice a greyed out area at the bottom including Next Steps, Concerns, and a place for feedback. This is default form content that cannot be changed.

Rubrics and PPAs

Within a given form, you can only tag curricular objectives (e.g., EPAs or milestones) from the same curriculum version. To ensure that you do not accidentally add an EPA from a different version, you must create the form first and then "Create & Attach" new items to the form.

Build a PPA or Rubric/Flex Form

1. Click Admin > Assessment & Evaluation.

2. Click on Forms from the subtab menu

3. Click Add Form.

4. Provide a form name and select PPA Form or Rubric/Flex Form from the Form Type dropdown menu; then click Add Form.

Now that you have two (or more) curriculum versions in the system, the Form Editor will default to loading the most recent version. Under "EPA Version", simply select the appropriate version. Click Save.

If you want to build new forms for learners using Version 1, simply change the EPA Version to Version 1 and it will load the appropriate EPAs.

Note: In order to use the "Programs" filter in the form bank, you need to add Program-level permissions to each form.

1. Click "Individual", change to "Program"

2. Begin typing in your program name in the "Permissions" box.

3. Click on your program to add it.

If applicable, select the relevant contextual variables. Click the grey badge beside a variable to select or remove specific responses.

Indicate whether comments are disabled, optional, mandatory, or prompted. By selecting ‘Prompted’, you can set the system to prompt and require a comment when any flagged response is selected by an assessor.

Feedback and Concerns items will be added when the form is published.

Add Items

Click "Add Item(s)" to add an item, or click the down arrow for more options. "Add Free Text" will add an instruction box. Do not use the "Add Curriculum Tag Set" at this time.

Create & Attach a New Item

Click "Create & Attach a New Item" to add an item.

Create New Item

Click "Create & Attach a New Item" to add an item. Select the Item Type and add any item responses, if applicable.

Tag Curriculum Objectives

Because you are using a form that is mapped to "Version 2", the curriculum tag sets will be locked to "Version 2". This will ensure that you do not accidentally tag an EPA from a different version.

Add Additional Items & Publish

Continue adding items as desired. When you have finished creating and attaching new items, click "Publish" to publish your new form.

Although Elentra allows programs to upload specific key and enabling competencies, organizations are still required to upload a standard list of key and enabling competencies at the organization level. For Canadian schools using CBME we recommend making the CanMEDS Key and Enabling Competencies your standard list. These are available online or by contacting Elentra's Training and Development Coordinator.

To download the required templates, complete them, and upload standard key and enabling competencies information

• Navigate to Admin > Manage Curriculum.

• Click 'Curriculum Tags' from the Manage Curriculum card on the left sidebar.

• Click on a curriculum tag set name (e.g. CanMEDS Key Competencies) and then click 'Import from CSV' in the top right to access and download the template to complete.

After you complete each required template, save it in a .csv format and upload it from the same screen you were on.

Accessing CanMEDS Key and Enabling Competencies

You can access the CanMEDS Key and Enabling Competencies at http://canmeds.royalcollege.ca/guide. The Key Competencies are preceded by one number like 1 or 2, and are stored under CanMEDS roles. The Enabling Competencies are preceded by a number like 1.1 or 2.2 and are also stored under CanMEDS roles.

Both of these are from the Medical Expert role so would be coded with the prefix ME. Alphanumeric codes are provided in the templates, you will need to input the key and enabling competency text.

You can export the text from the CanMEDS guide.

CanMEDS Key Competencies

Download the template through Manage Curriculum > Curriculum Tags > CanMEDS Key Competencies.

Note that the first column, objective_code, is completed for you with alphanumeric codes based on the CanMEDS roles.

Complete the second column, objective_name, by providing the key competency text. You can access the key competencies at http://canmeds.royalcollege.ca/guide or request them from another institution.

Save your file as a CSV.

CanMEDS Enabling Competencies

Download the template through Manage Curriculum > Curriculum Tags > CanMEDS Enabling Competencies.

Note that the first column, objective_code, is completed for you with alphanumeric codes based on the CanMEDS roles.

Complete the second column, objective_name, by providing the enabling competency text. You can access the enabling competencies at http://canmeds.royalcollege.ca/guide or request them from another institution.

Save your file as a CSV.

Uploading Templates and Setting as Standardized Curriculum Tag Set

To upload completed templates:

• Navigate to Admin > Manage Curriculum.

• Click 'Curriculum Tags' from the Manage Curriculum card on the left sidebar.

• Click on a curriculum tag set name (e.g. CanMEDS Key Competencies) and then click 'Import from CSV' in the top right.

• Drag and drop or select the appropriate file from your computer and click 'Upload'.

• You will get a success message and the curriculum tags you've added will appear on the screen.

To use the CBME Module, nine required curriculum tag sets must be in place. An auto-setup tool will help users create these. Only certain users have permission to complete this step.

Note: If you are a UGME organization using CBME some of these may be changed; please speak to a developer or the Elentra Core Team as needed.

Navigate to Admin > Manage Course > select a program, and click the CBME tab.

If the required curriculum tag sets aren't yet configured the program will prompt users to auto-setup the required curriculum tag sets and will populate the roles, stages, and contextual variables tags.

Users without permission to use the auto-setup feature will be directed to contact a system administrator with the message below.

After an administrative user has run the auto-setup tool, several tag sets will be created and three of them populated (users can view these via Admin > Manage Curriculum). Unless you are working with a developer, you cannot change the tags in these tags sets or after you do, you will be prompted to run the auto-setup feature again.

A program needs to complete this template OR the Milestones Template, not both. Most programs will map milestones and there is no need to complete this template. Users define which option they will use in in Step 2 of the import CBME process.

Navigate to Admin>Manage course and select a course. Click 'CBME' and 'Import CBME Data' to access and download the template. (See here for more information on downloading templates if needed.)

Columns to Complete

• EPA code: Codes should be recorded in uppercase letters. Codes should refer to the stage of training and the EPA number. e.g. D1, D2, D3, F1, C2. Using consistent codes across your program and school is important.The system is setup to accept the following codes for learner stage: D: Transition to Discipline F: Foundations of Discipline C: Core Discipline P: Transition to Practice

• Enabling Competency Code: Note that the template already includes a list of Enabling Competency Codes using the CanMEDS role prefixes. In the first column, simply add the EPAs codes you wish to map to each enabling competency.

  If there are unmapped Enabling Competency Codes left over you must delete those rows before you upload your .csv file.

Mapping multiple EPAs to one enabling competency

The system is designed to allow multiple EPAs to map to one milestone. To do so, simply insert a comma between the EPA code entries in the first column.

Save as a .csv and upload

After saving your file as a .csv, scroll to step 3 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

Use the Key Competencies Template if a program wants to import program specific key competencies. This template organizes the information about Key Competencies decided by each national specialty committee and allows the information to be uploaded to the system.

Navigate to Admin>Manage Course and select a course. Click 'CBME' and 'Import CBME Data' to access and download the required template. (See here for more information on downloading templates if needed.)

Template Columns to Complete

Note that the first column, Key Competency Code, is completed for you with alphanumeric codes based on the CanMEDS roles. Adjust this list as needed to match the information from the national specialty committee.

Complete the second column, Title, by providing the key competency text.

The third column, Description, is optional. This will store information but will not be visible to users.

Save your file as a CSV.

Uploading Your File

After saving your file as a .csv, scroll to step 3 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

Use the Enabling Competencies Template if a program wants to import program specific enabling competencies. This template organizes the information about Enabling Competencies decided by each national specialty committee and allows the information to be uploaded to the system.

Navigate to Admin>Manage Course and select a course. Click 'CBME' and 'Import CBME Data' to access and download the required template. (See here for more information on downloading templates if needed.)

Template Columns to Complete

Note that the first column, Enabling Competency Code, is completed for you with alphanumeric codes based on the CanMEDS roles. Adjust this list as needed to match the information from the national specialty committee.

Complete the second column, Title, by providing the enabling competency text.

The third column, Description, is optional. This will store information but will not be visible to users.

Save your file as a CSV.

Uploading Your File

After saving your file as a .csv, scroll to step 4 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

Reordering Contextual Variable Responses

• Navigate to Admin > Manage Courses/Programs.

• Search for the appropriate course as needed.

• From the cog menu on the right, select 'CBME'.

• Click on 'CV Responses' from the tab menu below the Competency-Based Medical Education heading.

• Click on a contextual variable to show its existing responses.

• Click on the double arrow in the Order column to drag and drop a response to a new location.

• You'll get a green success message that your change has been saved.

Reordering Contextual Variables List

If users want to reorder the overall list of contextual variables they can do so from Admin > Manage Curriculum. To change a tag's display order you need to edit the tag by clicking on the pencil icon beside a tag name. Then you need to adjust the display order for each individual tag.

When importing data to create EPA maps, programs are required to upload a contextual variable template to provide contextual variable response options (e.g. case complexity: low, medium, high). In addition to this information, programs will need to add specific details about any procedures included in their contextual variable response options. This additional information, called procedure attributes, plays an important part in building procedure assessment forms. Think of the procedure attributes as assessment criteria (e.g. obtained informed consent, appropriately directed assistants, etc.) for each procedure .

Required Information

For each procedure in your program, you must define headings, rubrics and list items that may be used to assess a learner completing the procedure. Headings appear at the top of a form, rubric becomes the subtitle of a rubric, and list items are the things that will actually be assessed on the rubric.

All this information must be stored and uploaded in a CSV spreadsheet that uses two columns: Type and Title.

Within ‘Type’ there are three things you can include: H to represent heading (e.g. Participate in Surgical Procedures, NG Insertion) R to represent rubric (e.g. Procedure, Pre-Procedure) L to represent list items (e.g. Apply knowledge of anatomy)

The three procedure criteria form a hierarchy. A heading can have multiple rubrics, and a rubric can have multiple list items. Arrange the criteria in your spreadsheet to reflect their nesting hierarchies.

In the 'Title' column enter the information to be uploaded.

When complete, save the file in a CSV format.

Sample Procedure Form On this sample form, the following information was uploaded as procedure attributes: H: Participate in Surgical Procedures R: Procedure L: (ME1.3.2) Apply knowledge... (ME3.4.2) Perform procedural tasks... Use common surgical instruments...

Tip: You can use the same file for multiple procedures if they share the same heading, rubric, and list information.

Uploading Procedure Attributes

• Ensure you are in a Program Coordinator or Program Director role.

• Navigate to Admin > Manage Courses/Programs.

• Scroll down or type in text and click on the relevant program name to select it.

• Click on ‘CBME’ on the tab menu.

• Click on ‘Contextual Variable Responses’ on the second tab menu.

• From the list of contextual variables, click on ‘Procedure’.

• Beside each procedure you will see a small black up arrow beside a green disk icon.

• Click on the up arrow to upload a .csv file of information for each procedure.

• Either choose your file name from a list or drag and drop a file into place.

• Use the dropdown menu to select the EPA to apply your procedure attributes to.

• Click ‘Save and upload criteria’.

Repeat this process for each procedure and each EPA. Remember, a program can use the same procedure attributes for multiple procedures and EPAs if appropriate.

Managing Existing Procedure Attributes

You can view what attributes were already uploaded to a procedure by clicking on the green check mark in the criteria column. You can review the criteria posted for a specific EPA and opt to replace the criteria for that specific EPA.

After a program has uploaded an initial set of contextual variable responses, they can be managed through the CBME tab.

Navigate to Admin>Manage course and select a course. Click CBME and then click the Manage Contextual Variable Responses tab below.

If contextual variable response information has been uploaded for the program, you will see a list of contextual response variables.

To modify the contextual variables within a category, click the title of the contextual variable or click 'Show' to the right of the category you want to edit.

To edit an existing response, make the required change and click the green disk icon.

To delete an existing response, click the red minus button beside the response.

To add a new contextual variable response, click the black plus button at the bottom of the category window. This will open a blank response space at the end of the list. Fill in the required content and save.

Troubleshooting

If you get a yellow bar across the screen when you try to modify contextual variables it means none have been uploaded for the program you are working in. Return to the Import CBME tab and complete Step 4.

If you need to add new information to your EPAs, enabling competencies, or milestones templates, simply add new information to the existing template and upload it again in the appropriate spot. The system will identify any NEWLY ADDED EPAs/milestones/enabling competencies and append them to the existing EPA information.

Because of this feature, it can be valuable to keep a copy of the most recent file you’ve uploaded at all times.

If you need to correct a typo or minor error in an EPA, milestone, etc. you can now do so in through the user interface. Navigate to the Import CBME Data screen and follow the instructions on the screen to edit existing EPAs, milestones, etc.

Contextual Variable Responses are used to describe the context in which a learner completes something. Examples of contextual variables include diagnosis, clinical presentation, clinical setting, case complexity, patient demographics, etc. For each variable, programs must define a list of response options. For example, under clinical presentation, a program might include cough, dyspnea, hemoptysis, etc.

Elentra will autosetup a specific list of contextual variables in the Manage Curriculum interface. Institutions can also add contextual variables (e.g., assessor's role) to this list before uploading contextual variable responses (e.g., nurse, physician, senior resident) via each program's CBME tab.

Do not delete any contextual variables from the existing list within Manage Curriculum or the CBME auto-setup feature will prompt you to re-add them the next time you use CBME. If you wish to add to the contextual variables list you may do so through Manage Curriculum and then import your responses through a program's CBME tab. By default Elentra is set up to provide access to the same Contextual Variables for all programs. You don't necessarily have to provide responses for all CVs in all programs but by default each program will see the same list of CVs.

Adding Additional Contextual Variables

Institutions can add additional contextual variables to make available to their programs through Admin > Manage Curriculum. You'll need to be a medtech:admin or staff:admin user to do this.

• Navigate to Admin > Manage Curriculum.

• Click on 'Curriculum Tags' from the Manage Curriculum Card on the left sidebar.

• Click on the Contextual Variables tag set.

• Any new tags you add to the Contextual Variable tag set (e.g. observer role, consult type, domain) will be visible and useable to all programs within the organization.

• When you add new tags to the tag set, you'll be required to provide a code and title. It is recommended that you make the code and title the same, but separate the words in the code with underscores. For example: Title: Observer Role Code: observer_role

• Please note that there is by default a 24 character limit for curriculum tag codes and a 240 character limit for curriculum tag titles. If you enter more characters than the limit, the system will automatically cut off your entry at the maximum characters.

• After you have added Contextual Variables to the existing tag set, programs will be able to add the new response codes (and responses) to the contextual variable response template and successfully import them.

Importing Contextual Variable Responses

Once you have built out the list of all contextual variables you want included in the system you can import specific contextual variable responses for each program.

Template Columns to Complete

• Response Code: This column is pre-populated by fields including assessor’s role, basis of assessment, case complexity, case type, etc. You can add to this list if you have built additional contextual variables through Manage Curriculum (see above). Make sure to use the exact response code from the newly created contextual variable tag in the response_code column.

• Response: In this column, you can add the response variables required (e.g. Case complexity response variables: low, medium, high). To add more than one response variable per response code, simply insert another line and fill in both the response code and response columns.

• If there are items in the response code column that you don’t use, you must delete them before you upload the file.

• Description (optional): This field is not seen by users at this point and simply serves as metadata.

Save as a .csv and upload

After saving your file as a .csv, scroll to step 4 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

If you want to manually map a connection between an EPA and a milestone, you can do so through the Map Curriculum Tags tab. This allows you to map additional milestones to EPAs after you have uploaded your templates.

• Navigate to admin, manage course and select a course.

• Click CBME and then map curriculum tags.

• Scroll down to the Map Curriculum Tags area.

• Either type in and select the tags required to sort the displayed results OR click through the displayed options to create the mapping structure you require.

  • You can display the results for more than one role at a time by clicking on multiple roles. Note that any already mapped milestones will still show up in the milestone list.

• Click to highlight the appropriate tags and then click ‘Save Curriculum Map’.

To modify existing EPAs and milestones navigate to the CBME tab of a program page. In the appropriate step (e.g. Import EPAs or Import Milestones), look for "Click here if you wish to make changes to existing EPAs" and click on the blue text.

This will open a window where you can edit the text, description, and entrustment of EPAs. Note that this should only be used to make small modifications to this information and should not change the meaning of the text.

If you edit an EPA, it will be updated automatically across the system including on all completed forms. If you edit any milestones, you will have to republish forms on which they are used. Edits made to milestones will not be made on already published forms.

Uploading Templates

All templates you upload to the system must be in .csv format.

If you are working in Excel or another spreadsheet manager use the “Save as” function to select and save your file as a .csv

To upload your files, either search for or drag and drop the file into place. You'll often see a green checkmark when your file is successfully uploaded.

If you receive an error message make sure that you have:

• Deleted any unused lines. This is especially relevant in templates with pre-populated columns including the contextual variables template, and enabling competencies mapping template.

• Completed all required columns. If a column is empty for a specific line the file may fail to upload.

Viewing EPA Maps

If you've imported all CBME data for a program but you are not able to see EPA maps in the EPA Encyclopedia or in the Map Curriculum Tags tab of CBME, double check that you've uploaded your files without spaces between the EPA Code letter and number, nor between the CanMEDS Role and Key Competency number. Having spaces in the incorrect places will prevent the system from retrieving the required information to produce maps.

Mistakes When Mapping Relationships

While you can correct minor typos and add additional mapping info. through the user interface you can not delete, or undo mapping between, EPAs and milestones or enabling competencies.

The option to reset all CBME data for a program has been removed since you can now create a new version of your CBME curriculum if significant mistakes were made.

A program needs to complete this template OR the Enabling Competencies Template, not both. Most programs will complete the Milestones Template. Users define which option they will use in Step 2 of the import CBME process.

Navigate to Admin>Manage course and select a course. Click 'CBME' and 'Import CBME Data' to access and download the template. (See here for more information on downloading templates if needed.)

Three Columns to Complete

2) Milestone code: Codes should be recorded in uppercase letters. The format for the milestone code is: Leaner Stage letter, followed by a space, CanMEDS Role letters, Key Competency number, followed by a period, Enabling Competency number, followed by a period, Milestone number.

Note that there should be no space between the CanMEDS Role letter and the Key Competency number.

Creating and confirming your Milestones codes takes patience. You'll likely notice that in some Royal College (RC) documents there is only a two digit milestone code. For the purposes of mapping your curriculum in Entrada, you must have three digit milestone codes. We recommend you add the third digit in the order the milestones appear in the RC documents you're using. Make sure you check for duplication as you go so that unique milestones have their own codes but different codes aren't applied to repeating milestones (within one stage). Using the data organization tools to reorder the milestone code columns and title columns can help you identify unneeded duplication.

You may also notice that the RC allows programs to use milestones coded from one stage in another stage (so you may see an F milestone in a C stage EPA). How programs and organizations handle this is ultimately up to them but we recommend that you align the milestone code with the stage that it is actually being assessed in/mapped to. For example, if it’s a “D” milestone being mapped to an F EPA, rename it as an “F” milestone. This is because you likely have a different expectation of performance on the milestone in a different stage, even if it’s the same task.

3) Title: The text of the milestone should be copied verbatim from the national specialty committee EPAs.

Mapping multiple EPAs to one milestone

The system is designed to allow multiple EPAs to map to one milestone. To do so, simply insert a comma between the EPA code entries in column one. e.g. D1,D2,F1

Save as a .csv and upload

After saving your file as a .csv, scroll to step 3 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

After sufficient information has been added to the system, a user can generate a map showing the connections between EPAs, roles, key competencies, enabling competencies, and milestones. This information can be access through the EPA Encyclopedia or a program's CBME page.

EPA Encyclopedia

From the dashboard, access the EPA Encyclopedia from the left sidebar on the Helpful Links card.

There are several filter options when viewing the EPA Encyclopedia.

1. Search for a specific learner. After you select a learner the page will refresh to show that learner's program(s).

2. Switch between multiple programs as needed.

3. Search for a specific EP by typing in an EPA code or key word. The tool will search the EPA title, detailed description and entrustment.

4. Switch between CBME Curriculum versions as needed.

After clicking on an EPA, click on Detailed Description, Entrustment or Program Map to view details about the EPA.

Viewing Maps from the CBME page

This option is only available to users with Admin > Manage Courses access (e.g. program direcotrs, program coordinators).

• Navigate to Admin > Manage course/program and select a course.

• Click CBME and the Map Curriculum tab.

• Scroll down to click on an EPA or type the beginning of the EPA code and title into the dialogue box and click on the relevant EPA when it appears below.

• Hover the mouse over a point on the map to view the complete text for the mapped item.

Elentra allows you to map EPAs to specific rotations and indicate for each rotation whether an EPA is a priority and how likely a learner is to encounter the EPA. For this information to be useful for learners, you must be using the Elentra Rotation Scheduler.

From the CBME dashboard, EPAs can display their priority and the likelihood of being encountered within a learner's rotation. Additionally, EPAs relevant to the learner's current rotation are outlined.

When learners and faculty trigger assessments they will also have the option to apply pre-set filters (e.g. priority EPA, current stage EPAs). All of these tools can help learners quickly identify EPAs to focus on in their clinical work.

Adding Priority and Likelihood Levels

• Navigate to Admin > Clinical Experience.

• Click on the Rotation Schedule tab.

• Create a draft schedule or open a published schedule. Note that if you are creating a new rotation schedule, you will also need to add rotations to the schedule.

• Once rotations exist within a schedule a small Objectives badge will appear beside the rotation name.

• Click on the CBME Objectives badge to open a list of EPAs tagged to the rotation.

• Click the plus sign beside any EPA you'd like to label with priority or likelihood.

• Indicate the likelihood by clicking on the appropriate button in the Likelihood.

• Set an EPA as a priority by clicking the exclamation mark in the priority column.

• Repeat as necessary.

• Return to the rotations list using the Back to Rotations button in the top right.

After you upload an EPA Template, you will be prompted to select how you'd like to continue mapping your EPAs. Each program has two decisions to make: 1. Competency Options: Decide whether or not to upload course/program specific key and enabling competencies. If you select to use the standard key and enabling competencies (e.g., CanMEDS key and enabling competencies) the system will copy the items previously uploaded via Manage Curriculum and make them the program's default key and enabling competencies. 2. Curriculum Tag Options: Decide whether to map EPAs to Milestones or Enabling Competencies. Most programs will map their EPAs to Milestones.

The EPA Template needs to be completed by all programs. This template organizes the information about EPAs decided by each national specialty committee and allows the information to be uploaded to Elentra.

Navigate to Admin > Manage Course and select a course. Click CBME and Import CBME Data to access and download the required template. (See here for more information on downloading templates if needed.)

Template Columns to Complete

Each program should complete an EPA template with the following information:

• Code: This is the EPA code. It should be recorded in uppercase letters and include the learner stage of training and the EPA number. Use the learner stage letters outlined below to ensure that the EPAs correctly map to other curriculum tags. D: Transition to Discipline F: Foundations of Discipline C: Core Discipline P: Transition to Practice Sample EPA Codes include D1, D2, D3, F1, F2, F3, C1, C2, etc. Note that there is no space between the letter and number in the EPA Code.

• Title: This should be the EPA text. This can be copied from the EPAs as decided by each national specialty committee.

• Detailed Description: In some cases, these may be the same as the EPA title; in other cases, an EPA has an additional description. Not every program will use the ‘Detailed Description’ column. Note that this information is displayed to users in the EPA Encyclopedia.

• Entrustment required (optional): This field is optional but will populate the EPA Encyclopedia which is visible to all users.

Save as a .csv and upload

• After saving your file as a .csv, scroll to Step 1 on the Import CBME screen and either search for or drag and drop the file into place. You'll see a green checkmark when your file is successfully uploaded.

The assessment plan builder is undergoing significant rebuilding during the winter of 2019. We do not recommend using the assessment plan builder at this time and if your institution is operating Elentra ME 1.14 the assessment plan builder will appear broken. You will likely see something like below:

Planned enhancements for a future release include:

1) Adding version scoping to the AP builder

2) Adding ability to group contextual variable responses for use in the AP builder (will not be reflected in forms as they will still honour the CV Response ordering)

3) Adding ability to indicate if the AP should look at specific CV responses (same as original version of AP builder whereby you indicate how many of each response is required) , a group of CV responses (i.e. 3 different responses from a given CV response group), or a general number of CV responses (i.e. 3 different responses).

_______________________________________________________________________________________________________

Introduced in Entrada ME 1.12, the assessment plan builder allows you to create assessment plans for a program. Currently these simply house information about the requirements for each program, but eventually reporting tools will be built to reflect the assessment plan information.

The assessment plan allows you to set minimum completion requirements for the different assessment tools built for a program. These minimums can apply to the number of assessments, the number of assessors, and specific contextual variable responses.

Please note that currently the assessment plan does not apply to procedure forms.

You need to have EPAs mapped and forms built before you can create an assessment plan.

Navigate to the CBME Sub-Module

1. Log in to Elentra.

2. Click "Admin" at top right

3. Click "Manage Programs"

4. Select your program.

5. Click "CBME"

6. Click "Import CBME Data".

Add New Version

1. Click "Actions"

2. Click "Add New Version"

Versioning Wizard Step 1

In this step, simply indicate which EPAs will be:

1. Replaced

2. Changing

3. Not Changing

4. Retired

Note: Each program can only have one version "in progress" at a time. You can continue editing that new version, or reset the unsaved progress.

Replaced

Select this option if you wish to replace the EPA entirely, including all milestones. This is the easiest option to work through the wizard. This option will require you to have a complete set of CSV files (all EPAs, all Milestones), rather than just the changes. Use this option if you have many changes to implement across EPAs and milestones.

Retired

Select this option to retire an EPA (remember, think in terms of EPA Code, e.g. "C11") from the next version. This means that there will be no new version of that EPA Code (C11) in the next curriculum version. If you include an EPA with the retired code in the spreadsheet, it will not get uploaded. You are unlikely to use this status for EPAs unless there is an EPA at the end of your numbering that is being removed. For example, you have D1, D2, D3, D4 currently. You are retiring D4 from the next version and no new D4 will be uploaded.

For milestones, a “retired” status will remove that milestone from that EPA in the new version. This means that there will be no new version of that milestone for that EPA. It will not retire it from other EPAs unless you explicitly tell it to.

Changing

The objective will be changing. This will allow you to upload new EPA and Milestone text. For a milestone (or competency), if you select “changing” for that milestone under one EPA, the system will update all other instances of that milestone to “changing”. This will ensure that you only have one version of that milestone.

Not changing

The objectives will remain unchanged. This exact version of the objective will be carried-forward into the new curriculum version. Note that selecting this option will stop any objectives with the same code from being uploaded.

Versioning Wizard Step 2

In this step, indicate which KCs, ECs, and Milestones will be: Changing, Not changing, or Retired.

If you marked all of your EPAs as "Replaced," you can click next without needing to go through each EPA.

PLEASE NOTE: If there are any milestones in a Changing EPA that you want removed, you MUST mark them as RETIRED. This is with respect to the Milestone code - if "C PR4.3.2" will no longer be in this EPA, mark it as retired.

If you would like to add a new Milestone to an EPA only, then just set the EPA to changing and nothing else.

Note that "retiring" a milestone will only retire it from that EPA and no others unless otherwise indicated.

Import CBME Data

Upload CSV file with new and/or changing EPAs

You will notice that the yellow box at the top will list all objectives that have been set to changing. You can still upload new EPAs and Milestones that were not already in the system.

Choose the file and click "Save and upload EPAs"

IF APPLICABLE: Upload Key and Enabling Competencies

Two programs (Otolaryngology and Medical Oncology) have added new Key or Enabling competencies. If you are one of these programs, upload the new KC and/or EC files here.

If your program has mapped your EPAs to the EC level, and is not using milestones, upload the standard (or program specific) ECs here, then go to Uploader Step 6. All other programs can skip the EC and KC steps.

Upload CSV file with new and/or changing Milestones

Choose the file and click "Save and upload Milestones".

Some programs at Queen's only mapped to the Enabling competency level. If you will continue mapping to enabling competencies, and are not converting to milestones, skip the milestone upload and use Step 6 to upload your EC map. If you are converting to milestones, use the milestone upload.

When the upload is complete, click "Next".

Versioning Wizard Step 3

Confirm the Version

This overview lists all of the EPAs and their version status. When you are ready to publish the new version of the curriculum, click the blue "Publish" button.

The new curriculum version will be available the next day. New versions are published nightly.

Optional: Map Existing Milestones to EPAs

Step through the Map Curriculum Tags wizard to map milestones to EPAs.

1. Select stage

2. Select EPA

3. Select CanMEDs Role

4. Select Milestone(s)

5. Click "Save Curriculum Map"

Repeat as needed for any additional milestones.

Build and Publish Assessment Tools for the New Version

Now that you have finished creating your new curriculum version, please ensure that you create new assessment tools for any changed/replaced/new EPAs. After the published version is live, learners will not be able to be assessed on any new/changed EPAs until the new tools have been built. Remember that in most cases this will have minimal impact, since the learners only get the new version for the stages beyond the one they are currently in.

Please do not delete forms or form templates from previous versions unless you are certain that there are no longer any learners using that version.