[PERF][IMPROVEMENT] argilla server: improve computation for dataset progress and metrics

[frascuchon]

Description

Computing dataset progress and metrics using the search engine drastically reduces the time required to run the Argilla server in HF spaces when persistent storage is enabled.

TODO

• Add search engine tests
• Adapt existing test to mock engine results

Type of change

• Refactor (change restructuring the codebase without changing functionality)
• Improvement (change adding some improvement to an existing functionality)

How Has This Been Tested

Checklist

• I added relevant documentation
• I followed the style guidelines of this project
• I did a self-review of my code
• I made corresponding changes to the documentation
• I confirm My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

[codecov]

Codecov Report

Attention: Patch coverage is 95.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 91.21%. Comparing base (eb6741e) to head (79a578c).
Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
...la-server/src/argilla_server/search_engine/base.py	77.77%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #5618      +/-   ##
===========================================
- Coverage    91.22%   91.21%   -0.02%     
===========================================
  Files          145      145              
  Lines         6051     6065      +14     
===========================================
+ Hits          5520     5532      +12     
- Misses         531      533       +2     

Flag	Coverage Δ
argilla-server	91.21% <95.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.