Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add endpoint to create responses for current user in bulk #4380

Merged
merged 8 commits into from
Dec 14, 2023

Conversation

jfcalvo
Copy link
Member

@jfcalvo jfcalvo commented Dec 5, 2023

Description

This PR adds a new endpoint to create responses in bulk to support the new bulk annotation feature on Argilla.

The new endpoint is POST /api/v1/me/responses/bulk with the following implementations details:

  • It can accept a maximum of 50 responses to be created or updated.
  • Responses are created or updated for the current user doing the request.
  • The endpoint is not transactional.
    • One response failing to be created or updated is not avoiding to process the rest of the responses in the request.
  • If the request is successful the number and order of the items in the request body and response will match.

Example of a body request with a response to update and a second response to update (but using an invalid question):

{
  "items": [
    {
      "values": {"prompt-quality": {"value": 10}},
      "status": "submitted",
      "record_id": "8df12fc4-aaa4-4c5c-a883-1b50ab877c97"
    },
    {
      "values": {"non-existent-question": {"value": 1}},
      "status": "draft",
      "record_id": "f62786d4-eba7-4ecb-8225-730509150668"
    }
  ]
}

Note

For updating responses we don't need the id of the response, from the context of the request we have the current user_id and the record_id is mandatory using this endpoint. These two attributes are the ones used to unambiguously know when to create or update a response.

A response for the previous request with the first response successfully updated and an error trying to create the second response:

{
  "items": [
    {
      "item": {
        "id": "e27cdcde-1393-4bfb-a42a-d557a1975b35",
        "values": {"prompt-quality": {"value": 10}},
        "status": "submitted",
        "user_id": "3063e7c6-5b2a-4952-83b1-8e05277e8b51",
        "record_id": "8df12fc4-aaa4-4c5c-a883-1b50ab877c97",
        "inserted_at": "2023-12-05T12:39:19.165603",
        "updated_at": "2023-12-05T12:50:26.593117",
      },
      "error": null
    },
    {
      "item": null,
      "error": {
        "detail": "found responses for non configured questions: ['non-existent-question']"
      }
    }
  ]
}

/cc @frascuchon @gabrielmbmb there are some TODO comments on the code to discuss possible improvements.
/cc @damianpumar

Closes #4367

Type of change

(Please delete options that are not relevant. Remember to title the PR according to the type of change)

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested

  • Running tests locally using SQLite as database.
  • Running tests locally using PostgreSQL as database.

Checklist

  • I added relevant documentation
  • follows the style guidelines of this project
  • I did a self-review of my code
  • I made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I filled out the contributor form (see text above)
  • I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. area: api Indicates that an issue or pull request is related to the Fast API server or REST endpoints area: server Indicates that an issue or pull request is related to the server area: tests Indicates that an issue or pull request is related to the tests language: python Pull requests or issues that update Python code status: help wanted Indicates that a maintainer wants help on an issue or pull request team: backend Indicates that the issue or pull request is owned by the backend team type: enhancement Indicates new feature requests labels Dec 5, 2023
Copy link

github-actions bot commented Dec 5, 2023

The URL of the deployed environment for this PR is https://argilla-quickstart-pr-4380-ki24f765kq-no.a.run.app

@jfcalvo
Copy link
Member Author

jfcalvo commented Dec 11, 2023

@damianpumar notice that we made a change to this PR so now responses that need to be updated doesn't need to provide the response id in the body.

Copy link

codecov bot commented Dec 11, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6630d7b) 90.13% compared to head (870b328) 65.96%.
Report is 532 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff              @@
##           develop    #4380       +/-   ##
============================================
- Coverage    90.13%   65.96%   -24.18%     
============================================
  Files          233      330       +97     
  Lines        12493    19115     +6622     
============================================
+ Hits         11261    12609     +1348     
- Misses        1232     6506     +5274     
Flag Coverage Δ
pytest ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Dec 13, 2023
@jfcalvo jfcalvo merged commit 4ceb23d into develop Dec 14, 2023
@jfcalvo jfcalvo deleted the feat/add-current-user-responses-bulk-endpoint branch December 14, 2023 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: api Indicates that an issue or pull request is related to the Fast API server or REST endpoints area: server Indicates that an issue or pull request is related to the server area: tests Indicates that an issue or pull request is related to the tests language: python Pull requests or issues that update Python code lgtm This PR has been approved by a maintainer size:XL This PR changes 500-999 lines, ignoring generated files. status: help wanted Indicates that a maintainer wants help on an issue or pull request team: backend Indicates that the issue or pull request is owned by the backend team type: enhancement Indicates new feature requests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Create a new endpoint supporting bulk annotation
2 participants