The SP Admin is used by the Center for Experiential Learning and Assessments [CELA] for delivering the Post-Encounter Notes portion of the CCA (Comprehensive Clinical Assessment) as well as for remediation for CCA. It is also used by the Doctoring program to deliver Post-Encounter Notes from SP activities as part of the Doctoring class. 

Overview of Components

SP Encounters Admin

For administrators Part of the Admin portal in ARB. SP Encounters Admin has 4 components. (Access to all 4 components is handled in App Access/Pen Admin role, additional access is required for SP Outcomes).

• Exam Management - Used to set up sessions and manage the delivery of the exams
• SP Instrument - Used to create the forms
• Dashboard  - Customized views of active testing blocks and active students across sessions 
• SP Outcomes - Manage and publish SP Outcomes from SIM IQ to Learning Outcomes (access to this tab is controlled in App Access "Admin - Standardized Patient Outcome")
• CCA Seasons - Aggregate and publish CCA Exams to Learning Outcomes from SIM IQ, Exam Management, and external exam software
  

SP Encounters in ARB

For graders and remediating students

• Grading page  - This is where Graders grade PENs and Remediation (access is handled in App Access/PEN Grader role).
• Remediation/ non CCA delivery - This is where students complete Remediation for all of CCA as well as Doctoring Write ups(all students have access automatically, non students can be set up in app access as a test student in order to access this and do the work here)

Post Encounter Notes - Learning Outcomes - A page in Learning Outcomes where students (and others) can view published PEN's

Exam Delivery
Application outside of ARB for delivering the exam portion of CCA exams - users cannot switch user to view as another users - access is given by being a test taker in a session. Logging in to this component requires Level 1 but not Duo two factor authentication.

SP Encounter Note "Quick" Instructions

Step 1 - Create a New Instrument (If Needed)

(Either start from scratch by clicking the "Create Instrument" button and selecting the correct type - Doctoring type for Doctoring, PEN for CCA Post Encounter Notes, non-pen for remediation of other CCA stations - OR copy an existing Doctoring type and make edits to it)

1. Drag from the left hand pallet Items into the center canvas, then configure the item in the right hand panel.
   
   
   Drag Items from ITEMS Pallet into center canvas and configure in the Item Configuration Panel.  In this example instrument the should always show, so All Steps was selected in the Delivery Plan Step.
   
   
   1. Title/Heading and other organizations structures (In Layout or Text) can have a Delivery Plan Step of either "All Steps" (if it should always show) or the phase it should be first seen.
      (For Example: a subheading called "Remediation" makes more sense to have a Delivery Plan Step of "Remediation" instead of All Steps, so that it only appears once a student is Remediating)
   2. Select the first Item type that the Student should complete, this should have a Delivery Plan Step of "Exam"
   3. Select and place the Item type that should be used by the grader to grade either the student's individual item above or the whole exam. This should have a Delivery Plan Step of "Grading"
      
      
      In this example instrument, the first item a student will complete is a write up, which uses a Comment Item type. The first item a grader will complete is a Rating (which is a scorable multiple choice options), this is set to a Delivery Plan Step of Grading. The Rubric used for is called 0-5 and is displayed in the center panel how it will look to the grader.
      
      
      • If there should be different text for the prompt when the student first completes it, vs when the grader grades it, you can uncheck "match prompt" for the Outcome Label and enter a different label that will be used whenever the response to that item is displayed.
      • (For example - If there is an Item for faculty to provide comments back to the student, the Prompt might be "Enter Comments on the student's write up" which provides the grader specific instructions. That prompt makes less sense when viewing the completed assessment, so the Outcome Label could say "Faculty Comments")
      • There can be more than one item type used in grading, such as a Rating to score and a Comment type for providing feedback.
      • Rating, Multiple Choice, Matrix, Dropdown, and Checkbox all required a Rubric (which determines what the selection choices are). Currently Dropdown and Checkbox items do not have any Rubrics configured for them. Software Delivery can create Rubrics as needed.
   4. Select and place the Item type that a student should use in Remediation. This should have a Delivery Plan Step of "Remediation"
   5. Finally Select and place items types necessary for Remediation Grading and give them the Delivery Plan Step of "Remediation Grading"
      • Don't forget to mark "Overall Outcome" on the item that is the overall outcome of remediation (aka what the student should see in Learning Outcomes as the final Outcome).
        
        
        
        The Outcome label has been unchecked from Match Prompt and an alternative label for use in the Outcome has been entered. "Faculty Comments" is displayed instead of "Enter Comments on the student's write up" in the Outcome view.
        
        
2. Save as needed (before previewing or after configuring each item)
3. Use Preview to review and confirm what the Exam will look like at each Phase.
4. Once you are satisfied with the Exam change the Status from Draft to Active (Exam Management will not show Exams unless they are in the Active Status.)
   
   
   
   
   Additional information about Instruments.

Step 2 - Create a New Exam Session

PEN Admin - Exam Management is used to create a session and to manage all the elements of a session

(Start by clicking the "Create Exam Session" button and selecting the correct type (Doctoring, PEN, non-pen remediation)

1. Set the Open and Close dates for Grading, Remediation and Remediation Grading (these can be changed at any time and can be blank until a date is determined). If you want to test these parts of the process you must include the dates you will be testing in the range
   • Name and Instrument are the only required fields.
   • If you wish to test the timeline limit and alert at, those fields will be required.
   • If no cut point is entered, there will be no cut point considered for the station
   • Everything for a station can be edited at any time (except instrument after a test has been taken)
2. Add a Station. The Instrument you wish to use must be ACTIVE in Instrument Admin for you to see it here.
   1. Set the Group Type to "Coaching Groups"
   2. Set the Test Taker Role to "Students"
   3. Set the Grader Role to "Doctoring Assessor"
   4. The name of the station is what the student and grader will see when they are completing their phases.
   5. Select Group for grading type to use current doctoring group assignments (for both Grading and Remediation Grading)
   6. Cut Point… Marginal Point… Still to be decided.
3. Add one or more testing blocks - this is when students will be able to complete the Exam phase
   1. Name it something meaningful for you
   2. Set the open and close date/time
4. Add Students (Manage Test Takers)
   • A sample file is available on the upload test takers page.
   • Either add students one by one by selecting "Add New Test Taker" entering the student's name/uniqname and selecting them from the drop down and then selecting a Testing Block. (make sure it has saved before closing the box)
   • Or add them in mass, by creating a CSV and uploading that 
   • For testing PEN types, a user who will test as a test taker must be added as a test taker (switch user is not possible in Exam Delivery).
5. The Exam Session has now been created and is available to students as soon as the first Testing block open time passes.

Step 3 - Students Complete Exam Phase

For PEN this will happen in Exam Delivery

• Exam Delivery URL (requires a Level 1 log in, but does not require Duo Login)
• While the Testing Block is open students can see all of the stations that are part of that session
• As soon as the student clicks Start any timer will begin counting down. If the student does not save and submit the exam before the timer ends the system will save the exam, close the exam window and submit it
• If the window is accidentally closed before a student has clicked save and submit, they will be able to click "Resume" and reopen the exam and use any remaining time
• If an issue occurs, such as a student starting an incorrect station, administrative users can reset the station back to not started (any entered text will be deleted) on the active testing block dashboard
• If an exam does not have a time limit and a student submits it, they can reopen the exam and make changes as long as the testing block is open 

For NON-PEN Remediation and Doctoring, this will happen in the SP Encounters Hub in ARB

• SP Encounters - Assignments URL
  
• In this list they will see any Sessions in which the Testing Block or Remediation Phase are active (aka while it is open). This may include CCA related notes
• The due date (when the Testing Block or Remediation Phase close) displays both on the table and at the top of the Note while they are completing it. 
• They can Save and Continue or Save and Submit. (If the student does not submit it before the Testing Block ends, they will NOT be able to submit it and a grader will not see it in grading - the testing block would need to be extended past the current time in order for them to submit)

Step 4 - Graders Grade Each Student's Note

This happens in the SP Encounters Hub in ARB

• Graders will receive and email notification with a link to SP Encounters when a student submits their Note. (Make sure the Grading Phase is open before students begin to submit or graders will not be able to see it when they receive the email)
• In this page they will see a list of all of the students who they are a Doctoring Assessor for that have been added as test takers to the session AND have been Submitted
• By default this table sorts by status - Not Started, Started, and Submitted to make sure what is needed to be completed is at the top.
• They can Save and Continue or Save and Submit. (If the grader does not submit it before the Grading Phase ends, they will NOT be able to submit it and when the Session is Published the student will not see their outcome - the Grading Phase would need to be extended past the current time in order for them to submit)
• All Graders except Doctoring Assessors must be given access in order to see the SP Encounters Hub
  • They must have "PEN - Grader" access for the ARB Suite in App Access
    
  • Any person who is in a Coaching Group, in the role of Doctoring Assessor will also see the SP Encounters Hub

Step 5 - Monitor Timelines and Grading

• On the Statistics Page you can see all of the test takers, their outcome (score and satisfactory/unsatisfactory) and the status of it.
• Clicking on View Item Analysis allows you to see
  • responses to individual questions the grader completed as part of the grading phase (except comment type)
  • the date/time the student submitted the note/exam 
  • the date/time the grader submitted the grade 
• Clicking Scores by Student downloads a csv that has all of the scores for each student for all stations and includes the individual grader responses shown in item analysis
• During the Exam Phase (while a testing block is open) the Current Testing Block Dashboard shows all Sessions with active Testing Blocks [This is mostly used during CCA week]
  • Information is grouped by testing block
    • It shows a row for each test taker for each station and the current status
    • This page updates every 15 seconds
  • Clicking on a test taker/station opens up a modal for more detailed information and administrative capabilities
    • These include resetting the station back to not started (all entered text is deleted), adding additional time, and viewing the actual exam the student is completing (the modal shows exactly how much has been entered up until the modal is opened, but it does not refresh as the student enters text)
• Active Student Dashboard excludes Doctoring type Sessions but provides an overview of every student who is in an open session (exam testing block, grading, remediation, remediation grading) and displays the status of each station that student is assigned to.

Step 6 - Publish Exam Session to the SP Encounters Tab in Learning Outcomes

• When you are ready for Students to see the results, on the Manage Session page, toggle "Publish Results" from unpublished to published, this session will now appear on a students "SP Encounters" Tab in Learning Outcomes
• From here the student can see their score (if applicable) and the final outcome
• Clicking on the Outcome opens a window where the student (and others) can see the full details of the note/exam including what they wrote initially and at remediation and what the grader has entered. (if any item should be hidden from students to see here or when they remediate, the Instrument used for that station should have Visible On unchecked for "Test Taker Edit View" and Test Taker Outcome View" for those items the student should not see)

Step 7 - Assign Students to Remediation If They Need It

• On the Remediation page you can see a list of all Students, their Exam Testing block and the Outcome/Score of the Grading
• Checking the box next to the score to assign that student to remediate
• As soon as the Remediation Phase is open and the box has been checked a student will be able to see the station in their SP Encounters Hub in ARB
• For Non-PEN Remediation types, no station is assigned to a student unless it has been checked on the remediation page

Testing the Set Up Before Go Live

Access Information

• In order to create an instrument or manage exams a user must have "Admin - PEN Admin" access for the ARB Suite in App Access
• In order to test grading a user must have "PEN - Grader" access for the ARB Suite in App Access
• In order to test remediation or non pen exams a user must have "PEN Test Student" access in the ARB Suite in App Access. (actual students do not need this and will have access to this page)
• Exam Delivery is open to anyone with a umich uniqname, but nothing will display on the page for them unless they are in an active session with an active testing block.

Testing Exam Delivery (AKA Day of CCA Exam)

Ensure that the person testing is set up as a Test Taker for a currently active Testing Block, and is in a group of the PEN Pilot Test type in Group Admin.

Go to Exam Delivery and log in.
Select an exam from the list and click Start Exam - the exam will now open in the same window and you can complete the exam. 

If the exam has a timer, that will show in the upper right corner, counting down. 

You can Save and Submit at any point before the timer ends, (or once the timer ends it will save and submit and log you out), the page autosaves whenever there is a curser action on the page.

In Exam Management on the Statistics page you should be able to see the status of the station for the Test Taker you are testing as.

• Exam - Not Started
• Exam - Started
• Exam - Submitted
• Grading - Started
• Grading - Submitted
• Remediation - Not Started (only if that test taker has had remediation for that station checked on the remediation page)
• Remediation - Started
• Remediation - Submitted
• Remediation Grading - Started
• Remediation Grading - Submitted

Testing Grading (All Types)

To test this, a user either needs to be a grader for the required station, or able to switch user to that grader.

As a grader, as soon as students have submitted an Exam, and while the grading step is open, submitted exams will appear in the PEN - Grading page in ARB.

• Ensure that the Open and Close dates include todays date for the Grading step
• Ensure that the person testing is set up as a grader for the station
• Go to PEN - Grading page in ARB
• Click on the exam (the status will now change to Grading Started)
• If you click "continue later" the exam will save and the status will stay Grading Started
• If you click Save and Submit" the exam will be submitted and the status will update to Grading Submitted. If the Session is published at this point the graded exam will be published in the test takers Learning Outcome.

Non-PEN Remediation Notes

• Make sure to select Non PEN when creating the Instrument and Exam type. This will give you instruments and exams configured to only have 2 steps, Remediation and Remediation Grading.
• There also will not be testing blocks, all students are just added to the session as a whole, the Remediation open/close dates on the Manage Session page is what will control when students can see/start their remediation. 
• You must check the check box for each station that Test Taker should remediate on the Remediation page in order for that station to appear for that Test Taker in the Remediation page on the SP Encounters hub in ARB.

Doctoring Notes

Make sure to select Doctoring type when create both the Instrument and the Exam type. This will give you instruments and exams configured to have 4 Steps (Write up, Grading, Remediation, Remediation Grading). The write up step is configured to show up in the SP Encounters hub for students.