_Draft.Rmd

---
title: "Pmetrics User Manual"
output: 
  html_document:
    theme: spacelab
    toc: true
    toc_float: true
    toc_depth: 3
    highlight: tango
---


```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
options(knitr.kable.NA = ".")
```

```{css echo=F}
h1 {
  color:#446e9b
}

.legacy {
  background-color: DimGrey;
  color: white;
  text-shadow: 1px 1px black;
  padding: 3px;
}

.r6 {
  background-color: #446e9b;
  color: white;
  text-shadow: 1px 1px black;
  padding: 3px;
}

.update {
  background-color: red;
  color: white;
  text-shadow: 1px 1px black;
  padding: 3px;
}

```

# Introduction 

Thank you for your interest in Pmetrics! This guide provides
instructions and examples to assist users of the Pmetrics R package, by
the Laboratory of Applied Pharmacokinetics and Bioinformatics at the
Saban Research Institute, Children's Hospital Los Angeles, and the
Department of Pediatrics, Keck School of Medicine, University of
Southern California. Please see our website at
[[http://www.lapk.org]{.underline}](http://www.lapk.org) for more
information.

## R6 architecture

**As of v. 2.0**, Pmetrics is shifting to an architecture less dependent on
reading and writing files. Data files are unchanged and described later in
this manual.

Model files can now be defined as an R object, instead of in a text file. 
Pmetrics can suppoort legacy runs with the old-style text file models, but
users are encouraged to change to the new methods. Throughout this manual we
will indicate the new style with the <span class="r6">R6</span> designation to reflect the
object-oriented R6 style of programming available in R that makes it more
consistent with object-oriented languages such as Python. We will indicate
old-style approaches with <span class="legacy">Legacy</span>.


**Here are some tips for using this guide.**

* The table of contents to the left is expandable and navigable.

* Items that are hyperlinked can be selected to cross reference
within this manual or link to external sites.

* `Items` correspond to inline examples of R code, which are not evaluted
in this document, but serve as templates for what may be typed into your 
R console or script. They may not necessarily be executable if typed verbatim.

## Citing Pmetrics

Please help us maintain our funding to provide Pmetrics as a free
research tool to the pharmacometric community. If you use Pmetrics in a
publication, you can cite it as below.

```{r} 
citation("Pmetrics")
```

## Disclaimer

You, the user, assume all responsibility for acting on the results
obtained from Pmetrics. The Laboratory of Applied Pharmacokinetics and
Bioinformatics (LAPKB), members and consultants to LAPKB, and Children's
Hospital Los Angeles and the University of Southern California and their
employees assume no liability whatsoever. Your use of the package
constitutes your agreement to this provision.

## System Requirements and Installation


Pmetrics and all required components will run under Mac (Unix), Windows,
and Linux. There are three *required* software components
which must be installed on your system **in this order**:

1.  The statistical programming language and environment "**R**"
    + After installing R, we highly recommended that you also install
**Rstudio**, a user-friendly environment for R.

2.  The **Pmetrics** package for R

3.  The **gfortran** Fortran compiler.

 
All components have versions for Mac, Windows, and Linux environments,
and 64- bit processors. Systems with 32-bit processors are no longer
supported. All are free of charge.

#### R

R is a free software environment for statistical computing and graphics,
which can be obtained from
[[http://www.R-project.org]{.underline}](http://www.R-project.org)/.
Pmetrics is a library for R.

#### Rstudio

We strongly recommend using [Rstudio](http://rstudio.org/) rather than any
other R interface.

#### Pmetrics

If you are reading this manual, then you have likely visited our website
at [[http://www.lapk.org]{.underline}](http://www.lapk.org), where you
can select the software tab to access instructions. 
As of version 1.9, Pmetrics is distributed on github and is a self-contained
package that will install gfortran with your permission, if it is not
already installed on your computer. Installing from github will also install
all packages upon which Pmetrics depends.


#### Gfortran

In order to run Pmetrics, a Fortran compiler compiler is required.
Pmetrics is designed to work with gfortran, a free compiler. After you
have installed Pmetrics, it will check your system for an active
gfortran installation. If it doesn't find one, it will offer to download
and install it. From there, installation should proceed automatically.
This is by far the easiest and most reliable way to complete
installation. Rest assured that no files are installed without your
permission.

If you do not wish to do this, you will have to get components manually,
and the first command to run is `PMbuild()`. You can get detailed
instructions on how to obtain and install gfortran appropriate for your
system on our [LAPKB website](http://www.lapk.org/Pmetrics_install_manual.php).

## What This Manual Is Not

We assume that the user has familiarity with population modeling and R,
and thus this manual is not a tutorial for basic concepts and techniques
in either domain. We have tried to make the R code simple, regular and
well documented. A very good free online resource for learning the
basics of R can be found at [Stat Methods](http://www.statmethods.net/index.html).

We recognize that initial use of a new software package can be complex,
so please feel free to contact us at any time, preferably through the
[Pmetrics forum](http:/www.lapk.org) or directly by [email](mailto:contact@lapk.org).

This manual is also not intended to be a theoretical treatise on the
algorithms used in IT2B or NPAG. For that, the user is directed to our
[website](http://www.lapk.org).

## Getting Help and Updates

<!--There is an active [LAPK forum](http://www.lapk.org) with all kinds
of useful tips and help with Pmetrics. Register and feel free to post! 
-->
Within R, you can  use
`help("command")` or `?command` in the R console to see detailed help files
for any Pmetrics command. Many commands have examples included in this
documentation and you can execute the examples with `example(command)`.

<!--You can also type `PMmanual()` to launch this manual from within Pmetrics 
as well as a catalogue of all Pmetrics functions. Finally, 
`PMnews()` will display the Pmetrics change log.
-->

Pmetrics will check for updates automatically every time you load it
with `library(Pmetrics)` and you are connected to the internet. 
If an update is available, it will provide a
brief message to inform you. You can then reinstall the package from github.

<!--When bugs arise in Pmetrics, you may see a start up message to inform you of 
the bug and a patch can be installed by the command `PMpatch()` if available. 
Note that patches must be reinstalled with this command every time you launch
Pmetrics, until the bug is corrected in the next version.
-->

<!--As of version 1.0.0 Pmetrics has graphical user interface (GUI)
capability for many functions. Using `PMcode("function")` will launch the
GUI in your default browser. While you are interacting with the GUI, R
is "listening" and no other activity is possible. The GUI is designed to
generate Pmetrics R code in response to your input in a friendly,
intuitive environment. That code can be copied and pasted into your
Pmetrics R script. You can also see live plot previews with the GUI. All
this is made possible with the 'shiny' package for R.


Currently, the following GUIs are available: `PMcode("run")` and
`PMcode("plot")`. 
--> 

## Customizing Pmetrics Options

You can change global options in Pmetrics with 
`setPMoptions(sep, dec,server\_address)`.

Currently you can change three options: `sep` and `dec` will allow Pmetrics
to read data files whose field separators are semicolons and decimal
separators are commas, e.g. `setPMoptions(sep=";", dec=",")`. These
options will persist from session to session until changed. The third
option, `server\_address`, allows you to specify the address of a remote
server with Pmetrics installed, to allow remote runs.

`getPMoptions()` will return the current options.

# Pmetrics Components

## Software engines

There are three main software engines that Pmetrics controls.

* **IT2B** is the ITerative 2-stage Bayesian parametric population PK modeling
program. It is generally used to estimate parameter ranges to pass to
NPAG. It will estimate values for population model parameters under the
assumption that the underlying distributions of those values are normal
or transformed to normal.

* **NPAG** is the Non-parametric Adaptive Grid software. It will create a
non-parametric population model consisting of discrete support points,
each with a set of estimates for all parameters in the model plus an
associated probability (weight) of that set of estimates. There can be
at most one point for each subject in the study population. There is no
need for any assumption about the underlying distribution of model
parameter values.

* The **Simulator** is a semi-parametric Monte Carlo simulation software
program that can use the output of IT2B or NPAG to build randomly
generated response profiles (e.g. time-concentration curves) for a given
population model, parameter estimates, and data input. Simulation from a
non-parametric joint density model, i.e. NPAG output, is possible, with
each point serving as the mean of a multivariate normal distribution,
weighted according to the weight of the point. The covariance matrix of
the entire set of support points is divided equally among the points for
the purposes of simulation.

## Pmetrics control functions

<span class="r6">R6</span>

Pmetrics uses `PM_model` to create model objects and `PM_fit` to create
objects that combine the model with the data, ready to be run (fitted),
generating probability distributions for primary model parameters.

These functions replace the following Legacy functions: `ITrun`, `ERRrun`, 
`NPrun`.

`PMload` is aliased as `PM_load` for consistency with the above functions.
Similar to the Legacy version, it loads the results of a run into R to be 
accessible to the user for analysis, plotting, etc. Different from the Legacy
version, it loads the results into another R6 object, `PM_result` instead of
the current environment.  

Invoking the simulator in R6 is unchanged, i.e. use `SIMrun`.


<span class="legacy">Legacy</span>

Pmetrics has groups of R functions named logically to run each of these
programs and to extract the output. Again, these are extensively
documented within R by using the `help(command)` or `?command syntax`.

* `ITrun`, `ITparse`, `ERRrun`

* `NPrun`, `NPparse`

* `PMload`, `PMsave`, `PMreport`

* `SIMrun`, `SIMparse`

### Run functions

<span class="r6">R6</span>

Once a `PM_fit` object is created, which combines a model with a data file,
it can be run by using the syntax `$run()` to access the appropriate function
defined for the `PM_fit` object. 

```{r echo=T, eval=FALSE}
run1 <- PM_fit(model,data)
run1$run(options)
```


<span class="r6">R6</span> <span class="legacy">Legacy</span>

For IT2B and NPAG, the "run" functions generate batch files, which when
executed, launch the software programs to do the analysis. `$run(engine="err")`
or `ERRrun()` is a special implementation of IT2B designed to estimate 
the assay error polynomial coefficients from the data, when they cannot be calculated from assay validation data (using `makeErrorPoly()`) supplied by the
analytical laboratory. The batch files contain all the information
necessary to complete a run, tidy the output into a date/time stamped
directory with meaningful subdirectories, extract the information,
generate a report, and a saved Rdata file of parsed output which can be
quickly and easily loaded into R. On Mac (Unix) and Linux systems, 
the batch file automatically launches in a Terminal window. 
Prior to v1.9, on Windows systems, the batch file was launched manually, 
but as of v1.9, this manual step is no longer necessary. The execution of
the program to do the actual model parameter estimation is independent
of R, so that the user is free to use R for other purposes.

For the Simulator, the `SIMrun` function will execute the program
directly within R.

### Parse functions

<span class="r6">R6</span> <span class="legacy">Legacy</span>

For all programs, the "parse" functions will extract the primary output
from the program into meaningful R data objects. For IT2B and NPAG, this
is done automatically at the end of a successful run, and the objects
are saved in the output subdirectory as IT2Bout.Rdata or NPAGout.Rdata,
respectively. These functions generally run automatically and are not
necessary for the user to access.

### Saving and loading functions

<span class="r6">R6</span> 

After a successful IT2B or NPAG run, `PMload` or `PM_load` creates a 
`PM_result` object rather than loading run results into the current 
environment and suffixed with the run number.

```{r echo=T, eval=FALSE}
fit1 <- PM_load(1)
plot(fit1$op)
```

<span class="legacy">Legacy</span> 

For IT2B and NPAG, the `PM_load` function can be used to load either
of the above .Rdata files after a successful run. Objects will be loaded
into the current environment in R and suffixed with ".run", where "run" is
the run number.

```{r echo=T, eval=F}
PM_load(1)
plot(op.1)
```

<span class="update">Update</span>
`PMsave` is the
companion to `PMload` and can re-save modified objects to the .Rdata file.


### Report generation

<span class="r6">R6</span> <span class="legacy">Legacy</span>

The `PMreport` function is automatically run at the end of a successful
NPAG and IT2B run, and it will generate an HTML page with summaries of
the run, as well as the .Rdata files and other objects. The default
browser will be automatically launched for viewing of the HTML report
page. <!--It will also generate a .tex file suitable for processing by a
LATEX engine to generate a pdf report.--> See the [Pmetrics
Outputs](#outputs) section.

## Data manipulation functions

<span class="r6">R6</span> <span class="legacy">Legacy</span>

Within Pmetrics there are also functions to manipulate data .csv files
and process and plot extracted data.

* Manipulate data .csv files: `PMreadMatrix`, `PMcheck`, `PMwriteMatrix`,
`PMmatrixRelTime`, `PMwrk2csv`, `NM2PM`, `PMmb2csv`

* Process data: `makeAUC`, `makeCov`, `makeCycle`, `makeFinal`, `makeOP`, `makePTA`,
`makeErrorPoly`

* Plot data: `plot.PMcov`, `plot.PMcycle`, `plot.PMfinal`, `plot.PMmatrix`,
`plot.PMop`,`plot.PMsim`, `plot.PMvalid`, `plot.PMpta`

* Model selection and diagnostics: `PMcompare`, `plot.PMop` (with residual
option), `makeValid`, `plot.PMvalid`, `PMstep`

* Pmetrics function defaults: `setPMoptions`, `getPMoptions`

Again, all functions have extensive help files and examples which can be
examined in R by using the `help(command)` or `?command` syntax.

# General Workflow

<span class="r6">R6</span>

The general Pmetrics workflow in R6 for IT2B and NPAG is shown in the
following diagram.

The user supplies the items in **yellow**. R is used to specify the working directory containing the data .csv file. The model file is created in R using
the `PM_model` function. When combined using `PM_fit` and the `$run()` function
on the resulting object, a batch file is generated by R, causing the preparation
program to be compiled and executed. An instruction file is generated
automatically by the contents of the data and model, and by
arguments to the `$run()` function. The batch file
will then compile and execute the engine file according to the
instructions, which will generate several output files upon completion.
Finally, the batch file will call the R script to generate the summary
report and several data objects, including the IT2Bout.Rdata or
NPAGout.Rdata files which can be loaded into R subsequently using
`PM_load`. Objects that are modified can be saved back to the .Rdata
files with `PMsave`.


![](Images/Slide1.png)

<span class="legacy">Legacy</span>

The general Pmetrics workflow in Legacy for IT2B and NPAG is shown in the
following diagram. The major differences compared to R6 are that the model is a text file, and the commands to start the run are different.


![](Images/Slide2.png)

The user supplies the items in **yellow**. R is used to specify the working directory containing the data.csv and
model.txt files. Through the batch file generated by R, the preparation
program is compiled and executed. The instruction file is generated
automatically by the contents of the data and model files, and by
arguments to the `NPrun`, `ITrun` or `ERRrun` commands. The batch file
will then compile and execute the engine file according to the
instructions, which will generate several output files upon completion.
Finally, the batch file will call the R script to generate the summary
report and several data objects, including the IT2Bout.Rdata or
NPAGout.Rdata files which can be loaded into R subsequently using
`PMload`. Objects that are modified can be saved back to the .Rdata
files with `PMsave`.

Both input files (data, model) are text files which can be edited
directly.

# Pmetrics Input Files

## Data.csv Files

Pmetrics accepts input as a spreadsheet "matrix" format. It is designed
for input of multiple records in a concise way. **Please keep the number
of characters in the file name ≤ 8.**

### Data file format

Files are in comma-separated-values (.csv) format. Examples of programs
that can save .csv files are any text editor (e.g. TextEdit on Mac,
Notepad on Windows) or spreadsheet program (e.g. Excel). Click on
hyperlinked items to see an explanation.

**IMPORTANT**: The order, capitalization and names of the header and the
first 12 columns are fixed. All entries must be numeric, with the
exception of ID and "." for non-required placeholder entries.

***POPDATA DEC\_11***
```{r echo=F, results='asis'}
library(knitr)
tab <- read.csv("Data/mdata.csv",na.strings=".")
names(tab)[1] <- "#ID"
tab$OUT <- as.character(tab$OUT)
kable(tab)
```

* ***POPDATA DEC\_11*** This is the fixed header for the file and must be
in the first line. It identifies the version. It is not the date of your
data file.

* ***\#ID*** This field must be preceded by the "\#" symbol to confirm
that this is the header row. It can be numeric or character and
identifies each individual. All rows must contain an ID, and all
records from one individual must be contiguous. Any subsequent row
that begins with "\#" will be ignored, which is helpful if you want to
exclude data from the analysis, but preserve the integrity of the
original dataset, or to add comment lines. IDs should be 11 characters
or less but may be any alphanumeric combination. **There can be at
most 800 subjects per run.**

* ***EVID*** This is the event ID field. It can be 0, 1, or 4. Every row
must have an entry.

    + 0 = observation

    + 1 = input (e.g. dose)

    + 2, 3 are currently unused

    + 4 = reset, where all compartment values are set to 0 and the time
counter is reset to 0. This is useful when an individual has multiple
sampling episodes that are widely spaced in time with no new
information gathered. This is a dose event, so dose information needs
to be complete.

* ***TIME*** This is the elapsed time in decimal hours since the first
event. It is not clock time (e.g. 09:30), although the `PMmatrixRelTime`
function can convert dates and clock times to decimal hours. 
Every row must have an entry, and within a given ID, rows
must be sorted chronologically, earliest to latest.

* ***DUR*** This is the duration of an infusion in hours. If EVID=1,
there must be an entry, otherwise it is ignored. For a bolus (i.e. an
oral dose), set the value equal to 0.

* ***DOSE*** This is the dose amount. If EVID=1, there must be an entry,
otherwise it is ignored.

* ***ADDL*** This specifies the number of additional doses to give at
interval II. It may be missing for dose events (EVID=1 or 4), in which
case it is assumed to be 0. It is ignored for observation (EVID=0)
events. Be sure to adjust the time entry for the subsequent row, if
necessary, to account for the extra doses. If set to -1, the dose is
assumed to be given under steady-state conditions. ADDL=-1 can only be
used for the first dose event for a given subject, or an EVID=4 event,
as you cannot suddenly be at steady state in the middle of dosing
record, unless all compartments/times are reset to 0 (as for an EVID=4
event). To clarify further, when ADDL=-1, all compartments in the
model will contain the predicted amounts of drug at the end of the
100th II interval.

* ***II*** This is the interdose interval and is only relevant if ADDL
is not equal to 0, in which case it cannot be missing. If ADDL=0 or is
missing, II is ignored.

* ***INPUT*** This defines which input (i.e. drug) the DOSE corresponds
to. Inputs are defined in the model file.

* ***OUT*** This is the observation, or output value. If EVID=0, there
must be an entry; if missing, this must be coded as -99. It will be
ignored for any other EVID and therefore can be ".". There can be at
most 150 observations for a given subject.

* ***OUTEQ*** This is the output equation number that corresponds to the
OUT value. Output equations are defined in the model file.

* ***C0, C1, C2, C3*** These are the coefficients for the assay error
polynomial for that observation. Each subject may have up to one set
of coefficients per output equation. If more than one set is detected
for a given subject and output equation, the last set will be used. If
there are no available coefficients, these cells may be left blank or
filled with "." as a placeholder.

* ***COV***\... Any column after the assay error coefficients is assumed
to be a covariate, one column per covariate. The first row for any subject
must have a value for all covariates, since the first row is always a dose.
Covariate values are applied at the time of doses.