Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIH TS Model Initial Commit Round 5 2024-08-11 #96

Merged
merged 6 commits into from
Aug 30, 2024

Conversation

aperofsky
Copy link
Contributor

@aperofsky aperofsky commented Aug 16, 2024

NIH TS Model Round 5

  • Locations: US + 38 states
  • All ages combined.
  • Submission does not include projections for incident deaths.
  • 'sample' target includes 300 representative simulations per run grouping.
  • 'quantile' and 'cdf' targets are estimated from 1000 simulations per run grouping.

Targets included:

  • 'sample': weekly incident hospitalizations
  • 'quantile': weekly cumulative hospitalizations, weekly incident hospitalizations, peak size hospitalizations
  • 'cdf': weekly peak timing hospitalizations

Simulated trajectories of weekly incident hospitalizations:

  • Run groupings: N = 78 (39 locations x 2 IAV subtype scenarios)
  • Stochastic runs: Each run grouping set has 300 stochastic simulations (N = 23400, 78 run groupings x 300 simulations/grouping).

Copy link

Run validation on files: 2024-08-11-NIH-Flu_TS.gz.parquet

Required values:

❌ Error 006: The submission is missing some required values, please check:
origin_date: 2024-08-11
scenario_id: c("A-2024-08-01", "C-2024-08-01", "E-2024-08-01", "B-2024-08-01", "D-2024-08-01", "F-2024-08-01")
target: inc death
horizon: c("1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "40", "41", "42", "43")
age_group: 0-130
location: US

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

❌ Error 5041: All values should be greater or equal to 0 the data frame contains some error, please verify.
🟡 Warning 5043: All values associated with output type 'sample' should have a maximum of 1 decimal place
❌ Error 511: The cumulative values are decreasing, 1787 unique groups have been identified with this issue. For example:
group: 2024-08-11, E-2024-08-01, 36, cum hosp, 2, 0-130, quantile, 0.01;
group: 2024-08-11, A-2024-08-01, US, cum hosp, 3, 0-130, quantile, 0.01;
group: 2024-08-11, B-2024-08-01, US, cum hosp, 3, 0-130, quantile, 0.01;
etc.
❌ Error 5041: All values should be less or equal to 1 the data frame contains some error, please verify.

Target Columns:

❌ Error 602: The data frame does not contain projections for 'inc death' target(s).
🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.

Locations:

No errors or warnings found on Location

Sample:

❌ Error 902: The minimal accepted grouping includes the column(s): horizon, age_group, scenario_id, the column(s): scenario_id seem to be missing (or is missing a sub-group), please verify.

Column Pairing information:

Run grouping pairing:
horizon
Stochastic run pairing:
horizon
Number of Samples: 300

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

If the submission contains projection file(s) with quantile projection, a pdf containing visualization plots of the submission is available and downloadable in the GitHub actions. Please click on 'details' on the right of the 'Validate submission' checks. The pdf is available in a ZIP file as an artifact of the GH Actions. For more information, please see here

@aperofsky
Copy link
Contributor Author

aperofsky commented Aug 16, 2024

@LucieContamin I will look into the errors associated with the value columns and round values to 1 decimal place. For the run groupings, they are based on scenario ID x location. Each location has its own model, and I separately produce 300 simulations for each scenario id x location pair. I don't have age specific projections. Perhaps you could provide some insight into what I'm potentially doing wrong? Happy to chat on slack to diagnose if that's easier.

Copy link

Run validation on files: 2024-08-11-NIH-Flu_TS.gz.parquet

Required values:

❌ Error 006: The submission is missing some required values, please check:
origin_date: 2024-08-11
scenario_id: c("A-2024-08-01", "C-2024-08-01", "E-2024-08-01", "B-2024-08-01", "D-2024-08-01", "F-2024-08-01")
target: inc death
horizon: c("1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "40", "41", "42", "43")
age_group: 0-130
location: US

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

❌ Error 5041: All values should be less or equal to 1 the data frame contains some error, please verify.

Target Columns:

❌ Error 602: The data frame does not contain projections for 'inc death' target(s).
🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.

Locations:

No errors or warnings found on Location

Sample:

❌ Error 902: The minimal accepted grouping includes the column(s): horizon, age_group, scenario_id, the column(s): scenario_id seem to be missing (or is missing a sub-group), please verify.

Column Pairing information:

Run grouping pairing:
horizon
Stochastic run pairing:
horizon
Number of Samples: 300

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

If the submission contains projection file(s) with quantile projection, a pdf containing visualization plots of the submission is available and downloadable in the GitHub actions. Please click on 'details' on the right of the 'Validate submission' checks. The pdf is available in a ZIP file as an artifact of the GH Actions. For more information, please see here

Copy link

Run validation on files: 2024-08-11-NIH-Flu_TS.gz.parquet

Required values:

❌ Error 006: The submission is missing some required values, please check:
origin_date: 2024-08-11
scenario_id: c("A-2024-08-01", "C-2024-08-01", "E-2024-08-01", "B-2024-08-01", "D-2024-08-01", "F-2024-08-01")
target: inc death
horizon: c("1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "40", "41", "42", "43")
age_group: 0-130
location: US

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

No errors or warnings found on Value and Type columns

Target Columns:

❌ Error 602: The data frame does not contain projections for 'inc death' target(s).
🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.

Locations:

No errors or warnings found on Location

Sample:

❌ Error 902: The minimal accepted grouping includes the column(s): horizon, age_group, scenario_id, the column(s): scenario_id seem to be missing (or is missing a sub-group), please verify.

Column Pairing information:

Run grouping pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon")
Stochastic run pairing:
horizon
Number of Samples: 300

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
horizon
Number of Samples: 300

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

If the submission contains projection file(s) with quantile projection, a pdf containing visualization plots of the submission is available and downloadable in the GitHub actions. Please click on 'details' on the right of the 'Validate submission' checks. The pdf is available in a ZIP file as an artifact of the GH Actions. For more information, please see here

Copy link

Run validation on files: 2024-08-11-NIH-Flu_TS.gz.parquet

Required values:

❌ Error 006: The submission is missing some required values, please check:
origin_date: 2024-08-11
scenario_id: c("A-2024-08-01", "C-2024-08-01", "E-2024-08-01", "B-2024-08-01", "D-2024-08-01", "F-2024-08-01")
target: inc death
horizon: c("1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "40", "41", "42", "43")
age_group: 0-130
location: US

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

No errors or warnings found on Value and Type columns

Target Columns:

❌ Error 602: The data frame does not contain projections for 'inc death' target(s).
🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.

Locations:

No errors or warnings found on Location

Sample:

Column Pairing information:

Run grouping pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon")
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon")
Number of Samples: 300

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon")
Number of Samples: 300

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

If the submission contains projection file(s) with quantile projection, a pdf containing visualization plots of the submission is available and downloadable in the GitHub actions. Please click on 'details' on the right of the 'Validate submission' checks. The pdf is available in a ZIP file as an artifact of the GH Actions. For more information, please see here

Copy link

Run validation on files: 2024-08-11-NIH-Flu_TS.gz.parquet

Required values:

❌ Error 006: The submission is missing some required values, please check:
origin_date: 2024-08-11
scenario_id: c("A-2024-08-01", "C-2024-08-01", "E-2024-08-01", "B-2024-08-01", "D-2024-08-01", "F-2024-08-01")
target: inc death
horizon: c("1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "40", "41", "42", "43")
age_group: 0-130
location: US

Columns:

❌ Error 102: The data frame should contains 9 columns, not 10. Please verify if one or multiple columns have been added.

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

No errors or warnings found on Value and Type columns

Target Columns:

❌ Error 602: The data frame does not contain projections for 'inc death' target(s).
🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.

Locations:

No errors or warnings found on Location

Sample:

Column Pairing information:

Run grouping pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "iav_type")
Stochastic run pairing:
c("scenario_id (B-2024-08-01, D-2024-08-01, F-2024-08-01)", "horizon"),
c("scenario_id (A-2024-08-01, C-2024-08-01, E-2024-08-01)", "horizon")
Number of Samples: 300

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (B-2024-08-01, D-2024-08-01, F-2024-08-01)", "horizon"),
c("scenario_id (A-2024-08-01, C-2024-08-01, E-2024-08-01)", "horizon")
Number of Samples: 300

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

If the submission contains projection file(s) with quantile projection, a pdf containing visualization plots of the submission is available and downloadable in the GitHub actions. Please click on 'details' on the right of the 'Validate submission' checks. The pdf is available in a ZIP file as an artifact of the GH Actions. For more information, please see here

Copy link

Run validation on files: 2024-08-11-NIH-Flu_TS.gz.parquet

Required values:

❌ Error 006: The submission is missing some required values, please check:
origin_date: 2024-08-11
scenario_id: c("A-2024-08-01", "C-2024-08-01", "E-2024-08-01", "B-2024-08-01", "D-2024-08-01", "F-2024-08-01")
target: inc death
horizon: c("1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "40", "41", "42", "43")
age_group: 0-130
location: US

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

No errors or warnings found on Value and Type columns

Target Columns:

❌ Error 602: The data frame does not contain projections for 'inc death' target(s).
🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.

Locations:

No errors or warnings found on Location

Sample:

Column Pairing information:

Run grouping pairing:
c("scenario_id (B-2024-08-01, D-2024-08-01, F-2024-08-01)", "horizon"),
c("scenario_id (A-2024-08-01, C-2024-08-01, E-2024-08-01)", "horizon")
Stochastic run pairing:
c("scenario_id (B-2024-08-01, D-2024-08-01, F-2024-08-01)", "horizon"),
c("scenario_id (A-2024-08-01, C-2024-08-01, E-2024-08-01)", "horizon")
Number of Samples: 300

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

If the submission contains projection file(s) with quantile projection, a pdf containing visualization plots of the submission is available and downloadable in the GitHub actions. Please click on 'details' on the right of the 'Validate submission' checks. The pdf is available in a ZIP file as an artifact of the GH Actions. For more information, please see here

@LucieContamin LucieContamin merged commit f1e50fa into midas-network:main Aug 30, 2024
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants