Skip to content

Commit

Permalink
Correct the fixtures for scoring of signal bug probe#2627 (#842)
Browse files Browse the repository at this point in the history
* Add correct fixture for bug 2627

* Correct the fixtures for scoring of bug 2627

* The integration test was using as a fixture measurement from
  before the incident
(https://explorer.ooni.org/m/20210427000432.751743_AU_signal_490428d8ebc76e6b),
but was expecting it to be marked as failed
* The unit test was using a measurement that was from a newer version of
  probe
(https://explorer.ooni.org/measurement/20221118T104419Z_signal_IT_30722_n1_Q02UUAiiHlVU0VE6),
which should not be marked as failed either

As part of #831 we improved the
logic for scoring to be more precise and so these checks were now
failing.

We now use in both cases a correct "bad case" and in the unit test we
also include the previous case as a "good case".

* Apply path filters to fastpath and legacy API tests

* Add clickhouse-driver to requirement.txt
  • Loading branch information
hellais authored Apr 24, 2024
1 parent 9223f48 commit 548bb4e
Show file tree
Hide file tree
Showing 6 changed files with 21 additions and 7 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/test_fastpath.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
name: test fastpath
on:
push:
branches:
- master
paths:
- 'fastpath/**'

jobs:
test:
Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/test_legacy_ooniapi.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
name: test legacy/ooniapi
on:
push:
branches:
- master
paths:
- "api/**"
- "analysis/**"

workflow_dispatch:
inputs:
debug_enabled:
Expand Down
1 change: 1 addition & 0 deletions fastpath/fastpath/tests/data/signal_2627.json

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions fastpath/fastpath/tests/test_functional.py
Original file line number Diff line number Diff line change
Expand Up @@ -1148,12 +1148,12 @@ def test_score_http_invalid_request_line():


def test_score_signal():
for can_fn, msm in minicans("signal", date(2021, 4, 27), date(2021, 4, 27), 1):
for can_fn, msm in minicans("signal", date(2022, 10, 27), date(2022, 10, 27), 1):
assert msm["test_name"] == "signal"
scores = fp.score_measurement(msm)
assert scores
rid = msm["report_id"]
if rid == "20210427T000430Z_signal_AU_45671_n1_Zq1z77FuiG2IkqqC":
if rid == "20221027T000314Z_signal_AM_44395_n1_2Tjx8RckKKKGB5Fv":
# Failed due to https://github.com/ooni/probe/issues/2627
assert scores == {
"accuracy": 0.0,
Expand Down
12 changes: 11 additions & 1 deletion fastpath/fastpath/tests/test_unit.py
Original file line number Diff line number Diff line change
Expand Up @@ -574,7 +574,7 @@ def test_score_http_requests():
def test_score_signal_2627():
# https://explorer.ooni.org/measurement/20221118T104419Z_signal_IT_30722_n1_Q02UUAiiHlVU0VE6
# Failed due to https://github.com/ooni/probe/issues/2627
msm = loadj("signal_022")
msm = loadj("signal_2627")
scores = fp.score_measurement(msm)
assert scores == {
"accuracy": 0.0,
Expand All @@ -585,6 +585,16 @@ def test_score_signal_2627():
"blocking_local": 0.0,
}

msm = loadj("signal_022")
scores = fp.score_measurement(msm)
assert scores == {
"blocking_country": 0.0,
"blocking_general": 0.0,
"blocking_global": 0.0,
"blocking_isp": 0.0,
"blocking_local": 0.0,
}


def test_score_signal_679():
# https://explorer.ooni.org/m/20230530180835.892478_US_signal_b091ac0f3794bbcd
Expand Down
1 change: 1 addition & 0 deletions fastpath/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ boto3
gunicorn
psycopg2-binary
# systemd <- This is an optional requirement on linux
clickhouse-driver

0 comments on commit 548bb4e

Please sign in to comment.