assignment open response 3 (50 points)

• Suggest edit

Create an Open Response Assessment Assignment

10.27.2. create an open response assessment assignment #.

Creating an open response assessment (ORA) assignment is a multi-step process. This section covers each step in detail.

In addition, see these other topics about different aspects of open response assessments.

Components of an open response assessment: Open Response Assessments Two

Viewing metrics and learner responses for released open response assessments: Accessing Metrics for ORA Assignments

Actions you can take for an active ORA assignment: Managing Open Response Assessment Assignments

10.27.2.1. Team Open Response Assessments #

Open response assessments can also be configured to work for teams (Team ORAs), allowing students to submit and be graded on a submission as a group. The process of creating and configuring Team ORAs is similar to that for individual ORAs, but deviations and notes for Teams are included in the instructions below.

Before setting up a Team Open Response Assessment, make sure that you have set up your course’s Team Configuration to define the desired team-sets in the Settings > Advanced Configuration menu. See Enable and Configure Teams for help setting up team-sets.

10.27.2.2. Step 1. Add the Component #

To add the open response assessment component to your course, complete these steps.

Do not add more than one ORA component in a course unit. Multiple ORA assignments in a unit cause errors when learners submit their assessments.

In Studio, open the unit where you want to create the open response assessment.

Under Add New Component , select Open Response .

Select one of the the Open Response Assessment templates listed.

In the problem component that appears, select Edit .

You use this component editor to add prompts and the rubric, and to specify other settings for the open response assessment component.

Select Save each time you complete an editing session. You can continue to edit the problem until you publish the unit.

After you publish an ORA assignment, you can no longer change the structure of the rubric or the point values associated with each criterion in the rubric. If you correct typographical errors in the text of the rubric, only learners who have not yet started the assignment will see the corrections. However, you can modify due dates and the weight of the ORA assignment after you publish an ORA assignment.

10.27.2.3. Step 2. Add Prompts #

You can format text and add images inside an open response assessment prompt the same way you would for a Text component. For more information, see The Visual Editor .

To add prompts , or questions, to your ORA assignment, complete these steps.

In the open response assessment component editor, select Prompt .

Replace the example prompt with your prompt.

To add another prompt in the assignment, select Add a Prompt , and then repeat step 2.

10.27.2.4. Step 3. Add the Rubric #

In this step, you add your rubric to provide guidance for assessing responses within the assignment. You add one rubric for each problem, regardless of the number of prompts in the problem.

The most effective rubrics for peer grading are written in clear, simple language, have concrete details, and are as specific as possible. Many novice learners will find it difficult to make the types of value judgments required by more holistic rubrics.

For each step below, replace any default text with your own text.

All open response assessments include a feedback field below the rubric so that learners can provide written feedback on a peer’s overall response. You can also allow or require learners to provide feedback for individual criteria. See step 4 in the following procedure for instructions. For more information, see Feedback Options .

To add the rubric, follow these steps.

In the ORA component editor, select the Rubric tab.

In the first Criterion section, enter the name and prompt text of your first criterion (100 character limit).

In the Option sections for this criterion, for each option that you provide for the criterion enter a name, explanation, and point value.

To remove options, select Remove at the top right of the option section.

To add more options, select Add Option .

Next to Feedback for This Criterion , select a value in the dropdown list.

If you do not want to allow feedback for this individual criterion, select None .

To require feedback for this criterion, select Required .

To allow feedback, but not require it, select Optional .

Repeat steps 2-4 to create additional criteria. To add more criteria than provided for in the template, select Add Criterion at the end of the list of criteria.

Under Feedback for This Response , add instructions for learners to provide overall written feedback on responses that they assess. You can leave the default text in the Feedback Instructions and Default Feedback Text fields, or replace it with your own text.

If you wish to allow learners to view the rubric as they work on their response, see Show Rubric During Response

10.27.2.4.1. Provide Only Comment Fields for Individual Criteria #

For an individual criterion, you can omit options, but if you do not include options, you must include the ability to add feedback comments.

To provide a comment field without options, complete these steps.

In the Criterion section for the criterion that you want to only provide a comment field for, select Remove to remove each option.

Next to Feedback for This Criterion , select Required from the list.

10.27.2.4.2. Reuse Rubrics #

If your course uses multiple Open Response Assessments with similar assessment criteria, reusing rubrics may save time and prevent having to re-author the same rubric across many ORAs.

To reuse a rubric, complete the following steps:

In Studio, navigate to an ORA with a completed rubric you’d like to copy.

Expand the Clone Rubric section (if it is not already expanded) and copy or note the value following “Block ID for this ORA”. This will allow you to uniquely identify this ORA later.

Navigate to (or create) an unpublished ORA where you’d like to reuse the rubric.

In the ORA component editor, select the Rubric tab and expand the Clone Rubric section.

In the “Block ID” dropdown, paste or select the ID of the ORA to copy rubric data from.

Click “Clone”. A banner should appear saying “Rubric Successfully Cloned from Block ID: <block ID>”.

Cloning rubric data clears Learner Training examples. If applicable, follow the instructions at learner training step to add updated Learner Training examples.

When a rubric is cloned, the version of the rubric that is cloned is the currently saved version, not the currently published version. This is done to support the ability to clone from ORAs that have not yet been published.

10.27.2.5. Step 4. Specify Additional Settings #

After you have added a prompt and rubric, you must specify additional settings for the assignment. These settings include the type of response that learners must submit, assignment dates, and whether learners will see a list of top scoring responses.

10.27.2.5.1. Team vs. Individual ORA #

To make a Team ORA, set the option Teams Enabled to True . This reveals a drop down for Selected Team-set that defines which group of teams will be able to submit a response for this assignment.

Students can be in one team per team-set, so configuring multiple team-sets allows you to create groups for different assignments. You could, for example, create separate team-sets for Homework, Midterms, and the Final Project, each selected for the relevant Team ORAs to allow students to collaborate with different classmates for each assignment type. For more on how to configure team-sets for your course, see Enable and Configure Teams .

For Team ORAs, File Upload is the main intended form of response. Learners on a team can collaboratively upload and review files as part of a team response. A text response is still acceptable but can only be drafted by the one submitting member of the team.

Team Open Response Assessments are designed to only be assessed by staff.

10.27.2.5.2. Specify the Response Type #

Learners can submit written responses, files, or both in their responses to the assigment. If you want learners to upload files, make sure the text of your prompt includes adequate instructions for learners to upload the required files, including the file types that learners can upload.

Before you ask learners to submit files for your open response assessment, be sure to read about limitations and best practices. For more information, see Asking Learners to Upload Files in Responses .

If you allow or require learners to upload image files, learners must also provide a brief written description of each image for accessibility.

To specify the response type that learners must submit, follow these steps.

In the ORA component editor, select Settings .

For Text Response , select one of the following options.

The Response Editor field allows you to select an editor that the students will use to format their responses. Select one of the following options:

Simple text editor : a simple text field without formatting options.

WYSIWYG Editor : a visual text editor that allows text formatting.

For File Uploads Response , select one of the following options.

If you select Required or Optional , Allow Multiple Files and File Upload Types will appear.

For Allow Multiple Files , select either True or False . If Allow Multiple Files is True , learners will be able to upload multiple files in their response. If you would like to restrict learner responses to a single file, set Allow Multiple Files to False .

For File Upload Types , select one of the following options.

PDF or Image Files

Image Files

Custom File Types

If you select Custom File Types , the File Types field appears. Enter the file name extensions, separated by commas, of the types of files that you want learners to submit.

To reduce the potential for problems from files with malicious content, learners cannot upload certain file types. For more information, see Prohibited File Extensions .

For Allow LaTeX Responses , select True or False .

10.27.2.5.3. Allow Learners to View Rubric While Responding #

By default, learners cannot see the rubric while they are working on their response. However, you may decide that it would be helpful for learners to be able to view the rubric while they work on their response so they know how they will be evaluated. To enable this functionality:

Set Show Rubric During Response to True .

When this setting is enabled, a collapsable section will appear in the Response step, above the first prompt, that shows learners a detailed breakdown of how their response will be graded.

This is the rubric you set up in Add Rubric . For each Criterion, learners will see all Option names, descriptions, and point values.

10.27.2.5.4. Include Top Responses #

You can specify whether learners see a section that shows the highest scoring responses that were submitted for each question in the assignment. If offered, this section displays only after each learner has completed all steps in the assignment. You specify the number of highest scoring responses to show.

Because each response can be up to 300 pixels in height, we recommend that you set the number of top responses lower than 20, to prevent the page from becoming too long.

In the Top Responses field, specify the number of responses that you want to appear in the Top Responses section below the learner’s final score.

If you do not want this section to appear, set the number to 0. The maximum number is 100.

10.27.2.6. Step 5. Select Assignment Steps #

Open response assessment assignments can include learner training, peer assessment, self assessment, and staff assessment steps.

When adding an ORA problem, the component editor provides some predefined ORA problem templates with different steps set up in a sequence that works well for most courses. While you can change the order of the peer, self, and staff assessment steps, edX recommends that you include them in this order.

If you include a learner training step, you must also include a peer assessment step. The learner training step must come before peer or self assessment steps.

If you include both peer and self assessment steps, edX recommends that you place the peer assessment before the self assessment.

If you include a staff assessment step, it should be the final step in the assignment.

For Team ORAs, Staff assessments are the only assessment step allowed, as these are not intended for peer or self assessment.

To add steps to the open response assignment, complete these actions.

In the ORA component editor, select the Assessment Steps tab.

Locate the following headings.

Step: Learner Training

Step: Peer Assessment

Step: Self Assessment

Step: Staff Assessment

Select the check boxes for the steps that you want the assignment to include.

(optional) To change the order of the steps, drag the steps into the order that you want using the bar at the left side of the steps.

10.27.2.7. Step 6. Assignment Schedule #

Under the Schedule tab of the ORA comonent editor, you can choose between three different modes for configuring the schedule and due dates for an ORA assignment.

The three different date configuration modes are: * Configure deadlines manually * Match deadlines to the subsection due date * Match deadlines to the course end date

The behavior of each option is as follows:

10.27.2.7.1. Configure deadlines manually #

With this option, you have the ability to set individual deadlines for each step of the Open Response Assessment individually. The options that you can set include:

Response Start Date and Response Start Time : These settings define when learners can begin working on their response. Before this date and time, learners cannot type a response or upload files. After, they can begin to work on their response and upload files.

Response Due Date and Response Due Time : These settings define the date / time by which learners must complete and submit their response. After this date / time passes, learners can no longer submit responses to the problem.

Peer Assessment Start Date and Peer Assessment Start Time : These settings define when learners can begin to assess peer responses. Before this time, learners will receive a message that peer assessment has not yet begun.

Peer Assessment Due Date and Peer Assessment Due Time : These settings define the date / time by which learners must complete assessing peer responses. After this date / time has passed learners will no longer be able to assess peers. If learners have not assessed the required number of peers before this deadline, they will not be able to recieve a grade.

Self Assessment Start Date and Self Assessment Start TIme : These settings define when learners can begin to self-assess their responses. Before this time, learners will receive a message that self assessment has not yet begun.

Self Assessment Due Date and Self Assessment Due Time : These settings define the date / time by which learners must complete their self-assessment. After this date / time has passed learners will no longer be able to complete their self-assessment.

The times that you set are in Coordinated Universal Time (UTC). To verify that you have specified the times that you intend, use a time zone converter such as Time and Date Time Zone Converter .

If you choose to specify these dates manually, the course grace period setting and individual learner extensions will not apply to open response assessments. For more information about the grace period setting, see Set the Grace Period .

You should allow sufficient time for peer assessments to be performed after learners have submitted their own responses. EdX recommends that you allow at least one week between the due date for responses and the due date for peer assessments. If the response due time and peer assessment due time are close together, and a learner submits a response just before responses are due, other learners may not have time to perform peer assessments before peer assessments are due.

10.27.2.7.2. Match deadlines to the subsection due date #

When this setting is selected, all ORA due dates will be set to the due date of the subsection that they are contained within. Rather than specifying individual dates and times for the submission, peer, and self due dates, they are all set to the due date of the subsection they are contained within. This has multiple benefits:

Alignment with other assignment dates : Rather than having their own separate due dates, ORAs can use the same due date as all other problems within a subsection, reducing complexity and simplifying the course timeline for students.

Ability to use grace period and individual extensions : Setting the date configuration to this setting allows ORA problems to use the grace period and learner extension features.

Because the submission and assessment deadlines are all set to the same date under this option, there will be no “buffer” time between the response due date and the peer assessment due date. If you are using this setting for a peer assessment ORA, you must make it clear to learners that they must submit early enough to give their peers time to review their response.

10.27.2.7.3. Match deadlines to the course end date #

When this setting is selected, all ORA due dates will be set to the end date of the course. Rather than specifying individual dates and times for the submission, peer, and self due dates, they are all set to the end date of the course. This setting is useful for self-paced courses.

10.27.2.8. Step 7. Specify Step Settings #

After you select the steps that you want, you can specify settings for those steps.

If you make changes to a step, and then clear the check box for that step, the step will no longer be part of the assignment and your changes will not be saved.

For Team ORAs, Staff assessments are the only assessment step allowed, therefore no changes can be made in the step settings.

10.27.2.8.1. Learner Training #

For the learner training step , you enter one or more example responses that you have created, then specify the expected option for each criterion in your rubric.

You must enter your complete rubric on the Rubric tab before you can select options for the learner training responses. If you later change one of your criteria or any of its options, you must also update the learner training step.

To add and score learner training responses, follow these steps.

Under Step: Learner Training , select View / Add Sample Responses . The section will expand and display the sample responses already set up.

Select Add sample response .

In the Response field, enter the text of your example response.

Under Response Score , for each criterion, select the option that you want.

10.27.2.8.2. Peer Assessment #

For the peer assessment step , you specify the number of responses that each learner must grade, the number of learners who must grade each response, and start and due dates. All fields are required.

To specify peer assessment settings, follow these steps.

Locate the Step: Peer Assessment heading.

Select View Options & configuration to display the step settings.

Next to Must Grade , enter the number of responses that each learner must grade.

Next to Graded By , enter the number of learners that must grade each response.

Next to Enable Flexible Peer Grade Averaging , select True if you want to enable Flexible Peer Grade Averaging .

For more information about peer assessment steps, see Peer Assessment Step .

10.27.2.8.3. Self Assessment #

For the self assessment step , you specify when the step starts and ends.

Locate the Step: Self Assessment heading and enable it.

Switch to the Schedule tab.

Locate the Self Assessment Deadlines heading.

Next to Start Date and Start Time , enter the date and time when learners can begin assessing their peers’ responses.

Next to Due Date and Due Time , enter the date and time by which all peer assessments must be complete.

Note The times that you set are in Coordinated Universal Time (UTC). To verify that you have specified the times that you intend, use a time zone converter such as Time and Date Time Zone Converter . Additionally, the course grace period setting does not apply to open response assessments. For more information about the grace period setting, see Set the Grace Period .

10.27.2.8.4. Staff Assessment #

For the staff assessment step , there are no additional settings to specify after you have selected the step for inclusion in the assignment.

10.27.2.9. Step 8. Test the Assignment #

To test your ORA assignment, you can set up the assignment in your course, set the section or subsection date in the future, publish the unit, and ask one or more beta testers to submit responses and grade each other. The beta testers can then let you know if they found the question and the rubric easy to understand or if they had any problems with the assignment.

For more information about beta testing, see Beta Testing a Course .

Scoring and Leaving Feedback on Open Response Questions

There are two ways to score and provide feedback on student open responses:

If you want to grade the entire class on the same open response question, we recommend doing so from within the assignment report. 

• Open the assignment report that includes the open response question you wish to grade.
• Click ‘assess’ in any student row to access, score and comment on students' open responses.

• Input the score for each response on a 4-point scale and write feedback comments as needed.

•  The feedback will now appear for you in the Assignment Report and for students in the Student Report.  (You may need to refresh the Assignment Report to see the updated scores).

If you want to give one student feedback on multiple open response questions, we recommend doing so from within the Student Details Report 

• Once you’re in the Assignment Report, click any student’s name to access the “Details Report.”

• Input the score for each open response question on a 4-point scale and write feedback comments as needed.

• The feedback will now appear for you in the Assignment Report and for students in the Student Report.

Didn't Find What You Were Looking For?

ASSISTments is a free public service operated jointly by Worcester Polytechnic Institute and the ASSISTments Foundation. ‍ The ASSISTments Foundation is a 501(c)(3) non-profit organization sponsored by Worcester Polytechnic Institute created in 2019. ‍ Worcester Polytechnic Institute is a 501(c)(3) non profit created in 1865. ‍ The ASSISTments Project work at WPI is led by Professor Neil Heffernan. ‍ The ASSISTments Foundation is led by Cristina Heffernan.

• Suggest edit

Managing Open Response Assessment Assignments

10.27.3. managing open response assessment assignments #.

After you publish an open response assessment (ORA) assignment and learners start to submit responses and perform assessments, members of the course team can take the following actions.

The following topics provide additional information about open response assessments.

Best practices for ORA assignments

Components of an ORA assignment

Instructions for creating an ORA assignment

Viewing metrics and learner responses for released ORA assignments

10.27.3.1. View a Specific Learner’s Response and Assessments #

You can view the following information about an individual learner’s open response assessment assignment.

The text of the learner’s response, including any files that the learner uploaded.

The peer assessments that other learners performed on the learner’s response, including feedback on individual criteria and on the overall response.

The peer assessments that the learner performed on other learners’ responses, including feedback on individual criteria and on the overall responses.

The learner’s self assessment.

The learner’s grade for the assignment.

An override grade provided by course staff.

For more details about accessing information for a specific learner, and for an example that shows a learner’s response with peer assessments, see Access a Specific Learner’s Information .

To determine whether a learner has received the required number of assessments from other learners and has completed the required number of assessments for other learners, refer to the Graded By and Must Grade values that were set for the open response assessment assignment in Studio. For more information about these settings, see Specify Step Settings .

10.27.3.1.1. Access a Specific Learner’s Information #

In order to access information about a specific learner’s assignment, you need that learner’s username or email address. For more information, see Download or View Learner Data .

To access information about a specific learner, follow these steps.

View the live version of your course in the LMS, and then go to the ORA assignment.

Scroll to the end of the problem, and then select Manage Individual Learners .

Enter the learner’s username or email address, and then select Submit .

The Manage Individual Learners dialog box updates with expandable sections for each of the assessment steps in the assignment and other actions you can take on the learner’s response. Only the types of assessment steps (self, peer, or staff) that are included in the assignment are shown.

If the learner uploaded files along with her response, select View the files associated with this submission to review or download the files.

Select any of the section headings to expand that section.

10.27.3.2. Perform Staff Assessments in an ORA Assignment #

The staff grading experience has gotten a makeover! Learn about the new ORA staff grading experience .

When a staff assessment is included in an open response assessment assignment, course team members see a Grade Available Responses option at the end of the assignment in the course, and learners see a Staff Assessment step in their assignment. For information about the possible assessment steps in an ORA assignment, see Assessment Steps .

Submitting a staff assessment has the following results.

The score that you give a learner in a staff assessment overrides scores from any other assessment type in the assignment.

Peer assessments that are completed before or after your staff assessment have no effect on the learner’s final assignment grade.

To perform a staff assessment in an assignment, follow these steps.

Scroll to the end of the problem, and then select Grade Available Responses .

In the dialog that opens, the number of available and checked out responses is shown. Checked out responses are responses that are currently being graded by you or another course team member.

Select the Staff Assessment heading to open a response that is available for grading.

Perform an evaluation of the response using the problem’s rubric.

Select Submit assessment to submit the assessment and close the grading dialog. Alternatively, select Submit assessment and continue grading to submit the assessment and immediately grade another submission.

10.27.3.3. Performing Staff Assessments in a Team ORA Assignment #

Team-based open response assessments can only be assessed by staff. Learners will see this as a Staff Assessment step in their assignment. Course team members will see a Grade Available Responses option at the end of the assignment.

Staff assessments for Team ORAs follow the same process as for individual ORAs except that the same score will be applied to all members of the submitting team.

To perform a staff assessment in a team assignment, follow the steps outlined in Perform a Staff Assessment . Before submitting, you will receive an alert to notify you that the grade will apply to all members of the team. Click Ok to apply the grade and close the grading dialog.

10.27.3.4. Override a Learner’s Assessment Grade #

For any open response assessment, whether or not a staff assessment is already included, you can override a learner’s final grade for the assignment. The ability to override the final grade can be useful if, for example, a learner’s submission was inappropriately or inadequately graded by peers, or if there are not enough peer reviewers to complete the required number of peer assessments.

Submitting an override assessment has the following results.

The score that you give a learner in an override assessment overrides scores from any other assessment type in the assignment.

Any steps that the learner did not complete for the assignment are marked as complete.

You can perform override assessments more than once on the same response, regardless of the due date of the assignment. The learner’s final grade on the assessment is updated to reflect the most recent staff override assessment grade.

Once grades are frozen 30 days after the course end date, you will no longer be able to perform override assessments for learners.

Learners who receive override grades for their submissions see a Staff Assessment step in their assignments, where they can view the rubric and any comments provided in the staff assessments.

10.27.3.4.1. Perform an Override Assessment #

In order to perform an override assessment for a learner, you need that learner’s username or email address. For more information, see Download or View Learner Data .

To perform an override assessment, follow these steps.

Scroll to the end of the problem, then select Manage Individual Learners .

Enter the learner’s username or email, then select Submit .

The Manage Individual Learners dialog updates with expandable sections for each of the assessment steps in the assignment and other actions you can take on the learner’s response.

Select Submit Assessment Grade Override .

Perform an assessment of the learner’s response using the problem’s rubric.

When you have finished the assessment, select Submit assessment .

The grade that you have given this learner’s response becomes the learner’s final grade on the assignment. Peer assessments are not taken into account in calculating the learner’s final assignment grade when a staff override grade exists.

Learners who have an override grade for their submission see a Staff Assessment step in their assignment, where they can view the rubric and any comments provided in the staff assessment.

Override assessments can be performed more than once on the same response, regardless of the due date of the assignment. The learner’s final grade on the assessment is updated to reflect the most recent staff override assessment grade.

10.27.3.5. Overriding a Team’s Assessment Grade #

As with individual ORAs, you can override the final grade for a Team ORA if it has previously been graded. Although only staff may grade team assignments, this ability to override a grade can be useful for fixing improperly submitted grades or amending grades after-the-fact. For individual ORAs, see Override a Learner’s Assessment Grade .

Like individual ORAs, overrides can be performed more than once on the same Team Open Response Assessment.

In order to perform an override assessment for a team, you need the edX username or edX email address of a learner on the team. For more information, see Download or View Learner Data .

To perform an override assessment, follow these steps:

View the live version of your course in the LMS and navigate to the Team ORA.

Scroll to the end of the problem where staff have a Manage Team Responses option. Select this to open the Manage Teams panel.

Enter the edX username or edX email of any member of the team and select Submit .

The Manage Teams dialog updates with the team’s response, grade, and existing assessments, along with options to submit overrides and remove the submission from grading.

Select Submit Team Grade Override .

Perform an assessment of the team’s response using the problem’s rubric.

The grade given for the team’s response overrides the grade for each member of the team for this assignment.

Like individual ORAs, overrides can be performed more than once on the same Team Open Response Assessment up until grades freeze for the course (30 days after the course end date).

10.27.3.6. Remove a Learner’s Response #

In a course that contains assignments with peer assessment steps, learners might alert you to inappropriate responses that they have seen while performing peer assessments. In such a situation you can locate and remove the response. Doing so removes the response so that it is no longer shown to other learners for peer assessment.

Removing a learner’s response is an irreversible action.

When you remove a response, the response is immediately taken out of the pool of submissions available for peer assessment. If the inappropriate response has already been sent to other learners for peer assessment, it is also removed from their queues. However, if any learner has already graded the inappropriate response, it is counted as one of the submissions they have graded.

After you remove an inappropriate response, you can decide whether the learner who submitted that response is allowed to submit a replacement response.

If you do not want to allow the learner to submit a replacement response, you do not need to take any additional action. The learner receives a grade of zero for the entire submission.

To allow the learner to resubmit a response for a cancelled submission, you must delete the learner’s state for the problem.

To remove a submitted response, follow these steps.

Identify the learner who submitted the inappropriate response by following the steps in the Locate a Specific Submission in an ORA Assignment topic.

View the live version of your course in the LMS, and then go to the ORA assignment that contains the submission you want to remove.

Enter the learner’s username or email, and then select Submit .

Select Remove Submission from Peer Grading .

Enter a comment to explain the removal. The learner sees this comment when she views her response in the open response assessment problem.

Select Remove submission .

The inappropriate submission is permanently removed from peer assessment. Removed submissions are also removed from the list of Top Responses if they were previously listed.

Optionally, delete the learner’s state for the problem. This step allows the learner to submit another response. For more information, see Delete a Learner’s State for a Problem .

When you access this learner’s information again by selecting Manage Individual Learners , instead of the response, you see a note showing the date and time that the submission was removed, and the comments that you entered.

When the learner views the assignment in the course, she sees that all steps in the assignment have a status of “Cancelled”. Under Your Response , instead of the text of their response, she sees the date and time that their response was cancelled, and the comments relating to the removal of their submission.

10.27.3.7. Remove a Team’s Response #

Removing a team’s response is very similar to removing a single learner’s response. To remove a submitted Team ORA response, follow these steps.

Identify a learner on the team that needs their submission removed from the staff grading workflow. If you only know the team name, use the Teams browser to find learners on a specific team. You can pick the username of any learner on the team by hovering over the learner avatar icons in the Team Details panel.

Scroll to the end of the problem, and then select Manage Team Responses .

The Manage Teams dialog updates with expandable sections for each of the assessment steps in the assignment and other actions you can take on the team’s response.

Select Remove Team Submission from Grading .

Enter a comment to explain the removal. The learners see this comment when they view their response in the open response assessment problem.

Select Remove submission . The selected submission is permanently removed from staff assessment.

Optionally, delete the team’s state for the problem. This allows the team to submit another response. For more information, see Delete a Learner’s State for a Problem . If you follow the instructions on that page and enter the username of a student on a team, the team state will be cleared.

When you access this learner’s information again by selecting Manage Team Responses , instead of the response, you see a note showing the date and time that the submission was removed, and the comments that you entered.

When a learner on the team views the assignment in the course, they will see that all steps in the assignment have a status of “Cancelled”. Under Your Response , instead of the text of their response, they will see the date and time that their response was cancelled, and the comments relating to the removal of their submission.

10.27.3.8. Locate a Specific Submission in an ORA Assignment #

If you are alerted to an inappropriate ORA submission that you want to cancel and remove from peer assessment , locate the specific submission by following these steps.

Ask the person who reported the incident to send you a sample of text from the inappropriate response.

Generate an ORA data report .

Search the report for text that matches the sample text from the inappropriate response.

From any matching entries in the spreadsheet, locate the username of the learner who posted the submission.

Make a note of the username, and follow the steps to remove a learner response from peer grading .