From 71581f7937a1b6ba5db56e527bece4de9fdefac9 Mon Sep 17 00:00:00 2001 From: Aliaksei Katyshou Date: Wed, 28 Feb 2024 15:12:34 +0300 Subject: [PATCH 1/2] Add value_source_concept_id field in measurement and observation tables --- inst/csv/OMOP_CDMv5.4_Field_Level.csv | 2 ++ 1 file changed, 2 insertions(+) diff --git a/inst/csv/OMOP_CDMv5.4_Field_Level.csv b/inst/csv/OMOP_CDMv5.4_Field_Level.csv index 86ea6a1c..2f7a2b88 100644 --- a/inst/csv/OMOP_CDMv5.4_Field_Level.csv +++ b/inst/csv/OMOP_CDMv5.4_Field_Level.csv @@ -167,6 +167,7 @@ measurement,measurement_source_concept_id,No,integer,"This is the concept repres measurement,unit_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the unit of the Measurement that occurred.,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,NA,NA,NA,NA,NA measurement,unit_source_concept_id,No,integer,"""This is the concept representing the UNIT_SOURCE_VALUE and may not necessarily be standard. This field is discouraged from use in analysis because it is not required to contain Standard Concepts that are used across the OHDSI community, and should only be used when Standard Concepts do not adequately represent the source detail for the Measurement necessary for a given analytic use case. Consider using UNIT_CONCEPT_ID instead to enable standardized analytics that can be consistent across the network.""",If the UNIT_SOURCE_VALUE is coded in the source data using an OMOP supported vocabulary put the concept id representing the source value here.,No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA measurement,value_source_value,No,varchar(50),This field houses the verbatim result value of the Measurement from the source data .,"If both a continuous and categorical result are given in the source data such that both VALUE_AS_NUMBER and VALUE_AS_CONCEPT_ID are both included, store the verbatim value that was mapped to VALUE_AS_CONCEPT_ID here.",No,No,NA,NA,NA,NA,NA +measurement,value_source_concept_id,No,integer,"Observations differ from Measurements in that they do not require a standardized test or some other activity to generate clinical fact. Typical observations are medical history, family history, the stated need for certain treatment, social circumstances, lifestyle choices, healthcare utilization patterns, etc. If the generation clinical facts requires a standardized testing such as lab testing or imaging and leads to a standardized result, the data item is recorded in the MEASUREMENT table. If the clinical fact observed determines a sign, symptom, diagnosis of a disease or other medical condition, it is recorded in the CONDITION_OCCURRENCE table. Valid Observation Concepts are not enforced to be from any domain though they still should be Standard Concepts.","Records whose Source Values map to any domain besides Condition, Procedure, Drug, Measurement or Device should be stored in the Observation table. Observations can be stored as attribute value pairs, with the attribute as the Observation Concept and the value representing the clinical fact. This fact can be a Concept (stored in VALUE_AS_CONCEPT), a numerical value (VALUE_AS_NUMBER), a verbatim string (VALUE_AS_STRING), or a datetime (VALUE_AS_DATETIME). Even though Observations do not have an explicit result, the clinical fact can be stated separately from the type of Observation in the VALUE_AS_ fields. In cases where the measurement recorded is a part of the standardized survey with values represented by concepts, such concepts should be placed into the value_source_concept_id field, and the value_as_concept_id should be filled with a corresponding Standard concept. It is recommended for Observations that are suggestive statements of positive assertion should have a value of ‘Yes’ (concept_id=4188539), recorded, even though the null value is the equivalent.",No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA measurement,measurement_event_id,No,integer,"If the Measurement record is related to another record in the database, this field is the primary key of the linked record.","Put the primary key of the linked record, if applicable, here.",No,No,NA,NA,NA,NA,NA measurement,meas_event_field_concept_id,No,integer,"If the Measurement record is related to another record in the database, this field is the CONCEPT_ID that identifies which table the primary key of the linked record came from.",Put the CONCEPT_ID that identifies which table and field the MEASUREMENT_EVENT_ID came from.,No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA observation,observation_id,Yes,integer,The unique key given to an Observation record for a Person. Refer to the ETL for how duplicate Observations during the same Visit were handled.,Each instance of an observation present in the source data should be assigned this unique key.,Yes,No,NA,NA,NA,NA,NA @@ -188,6 +189,7 @@ observation,observation_source_concept_id,No,integer,"This is the concept repres observation,unit_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the unit of the Observation that occurred.,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,NA,NA,NA,NA,NA observation,qualifier_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the qualifier of the Observation that occurred.,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,NA,NA,NA,NA,NA observation,value_source_value,No,varchar(50),This field houses the verbatim result value of the Observation from the source data. Do not get confused with the Observation_source_value which captures source value of the observation mapped to observation_concept_id. This field is the observation result value from the source.,"If the observation_source_value was a question, for example, or an observation that requires a result then this field is the answer/ result from the source data. Store the verbatim value that represents the result of the observation_source_value.",No,No,NA,NA,NA,NA,NA +observation,value_source_concept_id,No,integer,"Observations differ from Measurements in that they do not require a standardized test or some other activity to generate clinical fact. Typical observations are medical history, family history, the stated need for certain treatment, social circumstances, lifestyle choices, healthcare utilization patterns, etc. One of the common sources of observations are surveys. If the generation clinical facts requires a standardized testing such as lab testing or imaging and leads to a standardized result, the data item is recorded in the MEASUREMENT table. If the clinical fact observed determines a sign, symptom, diagnosis of a disease or other medical condition, it is recorded in the CONDITION_OCCURRENCE table. Valid Observation Concepts are not enforced to be from any domain though they still should be Standard Concepts.","Records whose Source Values map to any domain besides Condition, Procedure, Drug, Measurement or Device should be stored in the Observation table. Observations can be stored as attribute value pairs, with the attribute as the Observation Concept and the value representing the clinical fact. This fact can be a Concept (stored in VALUE_AS_CONCEPT), a numerical value (VALUE_AS_NUMBER), a verbatim string (VALUE_AS_STRING), or a datetime (VALUE_AS_DATETIME). Even though Observations do not have an explicit result, the clinical fact can be stated separately from the type of Observation in the VALUE_AS_ fields. In cases where the observation recorded is a part of the standardized survey with values represented by concepts, such concepts should be placed into the value_source_concept_id field, and the value_as_concept_id should be filled with a corresponding Standard concept. It is recommended for Observations that are suggestive statements of positive assertion should have a value of ‘Yes’ (concept_id=4188539), recorded, even though the null value is the equivalent.",No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA observation,observation_event_id,No,integer,"If the Observation record is related to another record in the database, this field is the primary key of the linked record.","Put the primary key of the linked record, if applicable, here. See the [ETL Conventions for the OBSERVATION](https://ohdsi.github.io/CommonDataModel/cdm60.html#observation) table for more details.",No,No,NA,NA,NA,NA,NA observation,obs_event_field_concept_id,No,integer,"If the Observation record is related to another record in the database, this field is the CONCEPT_ID that identifies which table the primary key of the linked record came from.",Put the CONCEPT_ID that identifies which table and field the OBSERVATION_EVENT_ID came from.,No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA death,person_id,Yes,integer,NA,NA,No,Yes,PERSON,PERSON_ID,NA,NA,NA From 509c2e36ca649885b4e9c0b9573c21870268c579 Mon Sep 17 00:00:00 2001 From: AlaikseiKatyshou Date: Wed, 24 Apr 2024 15:53:26 +0300 Subject: [PATCH 2/2] Fix descriptions for userGuidance and etlConventions in measurement and observation tables --- inst/csv/OMOP_CDMv5.4_Field_Level.csv | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/inst/csv/OMOP_CDMv5.4_Field_Level.csv b/inst/csv/OMOP_CDMv5.4_Field_Level.csv index 2f7a2b88..19f06597 100644 --- a/inst/csv/OMOP_CDMv5.4_Field_Level.csv +++ b/inst/csv/OMOP_CDMv5.4_Field_Level.csv @@ -167,7 +167,7 @@ measurement,measurement_source_concept_id,No,integer,"This is the concept repres measurement,unit_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the unit of the Measurement that occurred.,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,NA,NA,NA,NA,NA measurement,unit_source_concept_id,No,integer,"""This is the concept representing the UNIT_SOURCE_VALUE and may not necessarily be standard. This field is discouraged from use in analysis because it is not required to contain Standard Concepts that are used across the OHDSI community, and should only be used when Standard Concepts do not adequately represent the source detail for the Measurement necessary for a given analytic use case. Consider using UNIT_CONCEPT_ID instead to enable standardized analytics that can be consistent across the network.""",If the UNIT_SOURCE_VALUE is coded in the source data using an OMOP supported vocabulary put the concept id representing the source value here.,No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA measurement,value_source_value,No,varchar(50),This field houses the verbatim result value of the Measurement from the source data .,"If both a continuous and categorical result are given in the source data such that both VALUE_AS_NUMBER and VALUE_AS_CONCEPT_ID are both included, store the verbatim value that was mapped to VALUE_AS_CONCEPT_ID here.",No,No,NA,NA,NA,NA,NA -measurement,value_source_concept_id,No,integer,"Observations differ from Measurements in that they do not require a standardized test or some other activity to generate clinical fact. Typical observations are medical history, family history, the stated need for certain treatment, social circumstances, lifestyle choices, healthcare utilization patterns, etc. If the generation clinical facts requires a standardized testing such as lab testing or imaging and leads to a standardized result, the data item is recorded in the MEASUREMENT table. If the clinical fact observed determines a sign, symptom, diagnosis of a disease or other medical condition, it is recorded in the CONDITION_OCCURRENCE table. Valid Observation Concepts are not enforced to be from any domain though they still should be Standard Concepts.","Records whose Source Values map to any domain besides Condition, Procedure, Drug, Measurement or Device should be stored in the Observation table. Observations can be stored as attribute value pairs, with the attribute as the Observation Concept and the value representing the clinical fact. This fact can be a Concept (stored in VALUE_AS_CONCEPT), a numerical value (VALUE_AS_NUMBER), a verbatim string (VALUE_AS_STRING), or a datetime (VALUE_AS_DATETIME). Even though Observations do not have an explicit result, the clinical fact can be stated separately from the type of Observation in the VALUE_AS_ fields. In cases where the measurement recorded is a part of the standardized survey with values represented by concepts, such concepts should be placed into the value_source_concept_id field, and the value_as_concept_id should be filled with a corresponding Standard concept. It is recommended for Observations that are suggestive statements of positive assertion should have a value of ‘Yes’ (concept_id=4188539), recorded, even though the null value is the equivalent.",No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA +measurement,value_source_concept_id,No,integer,"This is the Concept representing the VALUE_SOURCE_VALUE and may not necessarily be standard. This field is discouraged from use in analysis because it is not required to contain Standard Concepts that are used across the OHDSI community, and should only be used when Standard Concepts do not adequately represent the source detail for the Measurement necessary for a given analytic use case. Consider using VALUE_AS_CONCEPT_ID instead to enable standardized analytics that can be consistent across the network.","If the VALUE_SOURCE_VALUE is coded in the source data using an OMOP-supported vocabulary put the Concept ID representing the source value here.",No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA measurement,measurement_event_id,No,integer,"If the Measurement record is related to another record in the database, this field is the primary key of the linked record.","Put the primary key of the linked record, if applicable, here.",No,No,NA,NA,NA,NA,NA measurement,meas_event_field_concept_id,No,integer,"If the Measurement record is related to another record in the database, this field is the CONCEPT_ID that identifies which table the primary key of the linked record came from.",Put the CONCEPT_ID that identifies which table and field the MEASUREMENT_EVENT_ID came from.,No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA observation,observation_id,Yes,integer,The unique key given to an Observation record for a Person. Refer to the ETL for how duplicate Observations during the same Visit were handled.,Each instance of an observation present in the source data should be assigned this unique key.,Yes,No,NA,NA,NA,NA,NA @@ -189,7 +189,7 @@ observation,observation_source_concept_id,No,integer,"This is the concept repres observation,unit_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the unit of the Observation that occurred.,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,NA,NA,NA,NA,NA observation,qualifier_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the qualifier of the Observation that occurred.,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,NA,NA,NA,NA,NA observation,value_source_value,No,varchar(50),This field houses the verbatim result value of the Observation from the source data. Do not get confused with the Observation_source_value which captures source value of the observation mapped to observation_concept_id. This field is the observation result value from the source.,"If the observation_source_value was a question, for example, or an observation that requires a result then this field is the answer/ result from the source data. Store the verbatim value that represents the result of the observation_source_value.",No,No,NA,NA,NA,NA,NA -observation,value_source_concept_id,No,integer,"Observations differ from Measurements in that they do not require a standardized test or some other activity to generate clinical fact. Typical observations are medical history, family history, the stated need for certain treatment, social circumstances, lifestyle choices, healthcare utilization patterns, etc. One of the common sources of observations are surveys. If the generation clinical facts requires a standardized testing such as lab testing or imaging and leads to a standardized result, the data item is recorded in the MEASUREMENT table. If the clinical fact observed determines a sign, symptom, diagnosis of a disease or other medical condition, it is recorded in the CONDITION_OCCURRENCE table. Valid Observation Concepts are not enforced to be from any domain though they still should be Standard Concepts.","Records whose Source Values map to any domain besides Condition, Procedure, Drug, Measurement or Device should be stored in the Observation table. Observations can be stored as attribute value pairs, with the attribute as the Observation Concept and the value representing the clinical fact. This fact can be a Concept (stored in VALUE_AS_CONCEPT), a numerical value (VALUE_AS_NUMBER), a verbatim string (VALUE_AS_STRING), or a datetime (VALUE_AS_DATETIME). Even though Observations do not have an explicit result, the clinical fact can be stated separately from the type of Observation in the VALUE_AS_ fields. In cases where the observation recorded is a part of the standardized survey with values represented by concepts, such concepts should be placed into the value_source_concept_id field, and the value_as_concept_id should be filled with a corresponding Standard concept. It is recommended for Observations that are suggestive statements of positive assertion should have a value of ‘Yes’ (concept_id=4188539), recorded, even though the null value is the equivalent.",No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA +observation,value_source_concept_id,No,integer,"This is the Concept representing the VALUE_SOURCE_VALUE and may not necessarily be standard. This field is discouraged from use in analysis because it is not required to contain Standard Concepts that are used across the OHDSI community, and should only be used when Standard Concepts do not adequately represent the source detail for the Observation necessary for a given analytic use case. Consider using VALUE_AS_CONCEPT_ID instead to enable standardized analytics that can be consistent across the network.","If the VALUE_SOURCE_VALUE is coded in the source data using an OMOP-supported vocabulary put the Concept ID representing the source value here.",No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA observation,observation_event_id,No,integer,"If the Observation record is related to another record in the database, this field is the primary key of the linked record.","Put the primary key of the linked record, if applicable, here. See the [ETL Conventions for the OBSERVATION](https://ohdsi.github.io/CommonDataModel/cdm60.html#observation) table for more details.",No,No,NA,NA,NA,NA,NA observation,obs_event_field_concept_id,No,integer,"If the Observation record is related to another record in the database, this field is the CONCEPT_ID that identifies which table the primary key of the linked record came from.",Put the CONCEPT_ID that identifies which table and field the OBSERVATION_EVENT_ID came from.,No,Yes,CONCEPT,CONCEPT_ID,NA,NA,NA death,person_id,Yes,integer,NA,NA,No,Yes,PERSON,PERSON_ID,NA,NA,NA