3_Use Embedding for Prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c0049a4d",
   "metadata": {},
   "source": [
    "# Code to test HAIM framework on a specific application using HAIM-MIMIC-MM dataset\n",
    "\n",
    "This notebook is dedicated to demonstrate how an user can use the embedding csv files to make predictions of their task of interest. We here choose to predict a patient's mortality status in the next 48 hours as a demonstration. A patient's death status is defined by their hospital discharge location, if it is morgue, hospice or expired, we consider the patient has died. \n",
    "\n",
    "### Project Info\n",
    " ->Copyright 2020 (Last Update: June 07, 2022)\n",
    " \n",
    " -> Authors: \n",
    "        Luis R Soenksen (<soenksen@mit.edu>),\n",
    "        Yu Ma (<midsumer@mit.edu>),\n",
    "        Cynthia Zeng (<czeng12@mit.edu>),\n",
    "        Ignacio Fuentes (<ifuentes@mit.edu>),\n",
    "        Leonard David Jean Boussioux (<leobix@mit.edu>),\n",
    "        Agni Orfanoudaki (<agniorf@mit.edu>),\n",
    "        Holly Mika Wiberg (<hwiberg@mit.edu>),\n",
    "        Michael Lingzhi Li (<mlli@mit.edu>),\n",
    "        Kimberly M Villalobos Carballo (<kimvc@mit.edu>),\n",
    "        Liangyuan Na (<lyna@mit.edu>),\n",
    "        Dimitris J Bertsimas (<dbertsim@mit.edu>),\n",
    "\n",
    "```\n",
    "**Licensed under the Apache License, Version 2.0**\n",
    "You may not use this file except in compliance with the License. You may obtain a copy of the License at\n",
    "https://www.apache.org/licenses/LICENSE-2.0\n",
    "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cf11828",
   "metadata": {},
   "source": [
    "### Requires \n",
    "```\n",
    " -> Previously generated pickle files from HAIM-MIMIC-MM Dataset\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e297a4f",
   "metadata": {},
   "source": [
    "## I. Import Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "504b3619",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn import metrics\n",
    "import xgboost as xgb\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64d8aaa3",
   "metadata": {},
   "source": [
    "## II. Reading Data and Constructing Prediction Target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53e8d4d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(fname)\n",
    "df_death_small48 = df[((df['img_length_of_stay'] < 48) & (df['death_status'] == 1))]\n",
    "df_alive_big48 = df[((df['img_length_of_stay'] >= 48) & (df['death_status'] == 0))]\n",
    "df_death_big48 = df[((df['img_length_of_stay'] >= 48) & (df['death_status'] == 1))]\n",
    "\n",
    "df_death_small48['y'] = 1\n",
    "df_alive_big48['y'] = 0\n",
    "df_death_big48['y'] = 0\n",
    "df = pd.concat([df_death_small48, df_alive_big48, df_death_big48], axis = 0)\n",
    "df = df.drop(['img_id', 'img_charttime', 'img_deltacharttime', 'discharge_location', 'img_length_of_stay', \n",
    "        'death_status'], axis = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5be2eeac",
   "metadata": {},
   "source": [
    "## III. Training/Testing Set Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32c366c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "pkl_list = df['haim_id'].unique().tolist()\n",
    "\n",
    "train_id, test_id = train_test_split(pkl_list, test_size=0.3, random_state=seed)\n",
    "#get the index for training and testing set\n",
    "train_idx = df[df['haim_id'].isin(train_id)]['haim_id'].tolist()\n",
    "test_idx = df[df['haim_id'].isin(test_id)]['haim_id'].tolist()\n",
    "\n",
    "x_y = x_y[~x_y.isna().any(axis=1)]\n",
    "#split train and test according to pkl list\n",
    "y_train = x_y[x_y['haim_id'].isin(train_idx)]['y']\n",
    "y_test = x_y[x_y['haim_id'].isin(test_idx)]['y']\n",
    "\n",
    "x_train = x_y[x_y['haim_id'].isin(train_idx)].drop(['y','haim_id'],axis=1)\n",
    "x_test = x_y[x_y['haim_id'].isin(test_idx)].drop(['y','haim_id'],axis=1)\n",
    "\n",
    "print('train, test shapes', x_train.shape, x_test.shape, y_train.shape, y_test.shape)\n",
    "print('train set, death outcome case = %s, percentage = %s' %(y_train.sum(),  y_train.sum()/len(y_train)))\n",
    "print('test set, death outcome case = %s, percentage = %s' %(y_test.sum(),  y_test.sum()/len(y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52c3504a",
   "metadata": {},
   "source": [
    "## IV: Model Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f84dd50b",
   "metadata": {},
   "outputs": [],
   "source": [
    "cv_folds = 5\n",
    "gs_metric = 'roc_auc'\n",
    "param_grid = {'max_depth': [5, 6, 7, 8],\n",
    "             'n_estimators': [200, 300],\n",
    "             'learning_rate': [0.3, 0.1, 0.05],\n",
    "             }\n",
    "\n",
    "est = xgb.XGBClassifier(verbosity=0, scale_pos_weight = (len(y_train) - sum(y_train))/sum(y_train), seed = 42,\n",
    "                        eval_metric='logloss')\n",
    "\n",
    "gs = GridSearchCV(estimator = est, param_grid=param_grid, scoring=gs_metric, cv= cv_folds)\n",
    "gs.fit(x_train, y_train)\n",
    "\n",
    "y_pred_prob_train = gs.predict_proba(x_train)\n",
    "y_pred_train = gs.predict(x_train)\n",
    "\n",
    "y_pred_prob_test = gs.predict_proba(x_test)\n",
    "y_pred_test = gs.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d930832",
   "metadata": {},
   "source": [
    "## V:Reporting Performance Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c59da1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "f1_train = metrics.f1_score(y_train, y_pred_train, average='macro')\n",
    "accu_train = metrics.accuracy_score(y_train, y_pred_train)\n",
    "accu_bl_train = metrics.balanced_accuracy_score(y_train, y_pred_train)\n",
    "auc_train =  metrics.roc_auc_score(y_train, y_pred_prob_train)\n",
    "conf_matrix_train = metrics.confusion_matrix(y_train, y_pred_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81a1a2f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'F1 Score for Training Set is: {f1_train}')\n",
    "print(f'Accuracy for Training Set is: {accu_train}')\n",
    "print(f'Balanced Accuracy for Training Set is: {accu_bl_train}')\n",
    "print(f'AUC for Training Set is: {auc_train}')\n",
    "print(f'Confusion Matrix for Training Set is: {conf_matrix_train}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57bfc194",
   "metadata": {},
   "outputs": [],
   "source": [
    "f1_test = metrics.f1_score(y_test, y_pred_test, average='macro')\n",
    "accu_test = metrics.accuracy_score(y_test, y_pred_test)\n",
    "accu_bl_test = metrics.balanced_accuracy_score(y_test, y_pred_test)\n",
    "auc_test =  metrics.roc_auc_score(y_test, y_pred_prob_test)\n",
    "conf_matrix_test = metrics.confusion_matrix(y_test, y_pred_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c0670df",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'F1 Score for Testing Set is: {f1_test}')\n",
    "print(f'Accuracy for Testing Set is: {accu_test}')\n",
    "print(f'Balanced Accuracy for Testing Set is: {accu_bl_test}')\n",
    "print(f'AUC for Testing Set is: {auc_test}')\n",
    "print(f'Confusion Matrix for Testing Set is: {conf_matrix_test}')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}