Feature/6076 add proximity search

[tmpayton]

Summary (required)

• Resolves Implement Proximity Search #6076

This PR adds a proximity search to our legal search endpoint for AO, ADR, MUR, and AFs. The four new filters are q_proximity, max_gaps, proximity_filter, and proximity_filter_term.

Note: I removed highlighting for proximity search because it does not work with intervals queries.
Please see the previous research ticket for an in depth breakdown.

Required reviewers 2 - 3 developers

Impacted areas of the application

General components of the application that this PR will affect:

• legal search

How to test

• checkout this branch
• start elasticsearch ./elasticsearch
• pytest
• create case index: python cli.py create_index case_index
• create ao index: python cli.py create_index ao_index
• load sample data for ao, mur, af, and adrs(at least three of each) : python cli.py load_current_murs python cli.py load_admin_fines python cli.py load_advisory_opinions python cli.py load_adrs
• flask run
• see test URLs below

Test URLs reference the following documents: MUR 8285, ADR 1175, AF 4774, AO 2024-15

MUR:

One phrase:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=Dowell%20daunting&max_gaps=2

Multiple phrases:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=daunting%20task&q_proximity=actively%20campaigning&q_proximity=represent%20Illinois&max_gaps=12

With filter:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=Dowell%20daunting&max_gaps=2&proximity_filter=after&proximity_filter_term=remediation

http://127.0.0.1:5000/v1/legal/search/?q_proximity=Dowell%20daunting&max_gaps=2&proximity_filter=before&proximity_filter_term=remediation

ADR:

One phrase:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=RAD%20Wisconsin&max_gaps=5

Multiple phrases:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=Polls%20Wisconsin&q_proximity=twenty-two&max_gaps=13

With filter:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=Polls%20Wisconsin&q_proximity=twenty-two&max_gaps=13&proximity_filter=after&proximity_filter_term=RAD

http://127.0.0.1:5000/v1/legal/search/?q_proximity=Polls%20Wisconsin&q_proximity=twenty-two&max_gaps=13&proximity_filter=before&proximity_filter_term=RAD

AF:

One phrase:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=Datwyler%20McGuire&max_gaps=4

Multiple phrases:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=%20Thomas%20Datwyler&q_proximity=reason%20to%20believe&max_gaps=5

With filter:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=Datwyler%20McGuire&max_gaps=4&proximity_filter=after&proximity_filter_term=%20Pre-Primary%20Report

http://127.0.0.1:5000/v1/legal/search/?q_proximity=Datwyler%20McGuire&max_gaps=4&proximity_filter=before&proximity_filter_term=%20Pre-Primary%20Report

AO:

One phrase:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=LAMA%20illegitimate%20&max_gaps=6

Multiple phrases:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=misappropriation%20&q_proximity=membership%20dues&max_gaps=2

With filter:

http://127.0.0.1:5000/v1/legal/search/?q_proximity=LAMA%20illegitimate%20&max_gaps=6&proximity_filter=before&proximity_filter_term=misappropriation

http://127.0.0.1:5000/v1/legal/search/?q_proximity=LAMA%20illegitimate%20&max_gaps=6&proximity_filter=after&proximity_filter_term=misappropriation

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.47%. Comparing base (02b2fbd) to head (288b9d2).
Report is 5 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #6087      +/-   ##
===========================================
+ Coverage    88.46%   88.47%   +0.01%     
===========================================
  Files           82       82              
  Lines         9091     9122      +31     
===========================================
+ Hits          8042     8071      +29     
- Misses        1049     1051       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cnlucas]

Fabulous job @tmpayton! Tested adding q search to find the highlights, everything works well.

[fec-jli]

great, it works well. Thanks