L. panamensis 202011: TMRC3 Sample Estimation

1 Introduction

This document is intended to provide an overview of TMRC3 samples which have been sequenced. It includes some plots and analyses showing the relationships among the samples as well as some differential analyses when possible.

2 Annotation

We take the annotation data from ensembl’s biomart instance. The genome which was used to map the data was hg38 revision 91. My default when using biomart is to load the data from 1 year before the current date, which provides annotations which match hg38 91 almost perfectly.

hs_annot <- load_biomart_annotations()

## The biomart annotations file already exists, loading from it.

hs_annot <- hs_annot[["annotation"]]
hs_annot[["transcript"]] <- paste0(rownames(hs_annot), ".", hs_annot[["version"]])
rownames(hs_annot) <- make.names(hs_annot[["ensembl_gene_id"]], unique=TRUE)
tx_gene_map <- hs_annot[, c("transcript", "ensembl_gene_id")]

3 Sample Estimation

I used two mapping methods for this data, hisat2 and salmon. Most analyses use hisat2, which is a more traditional map-and-count method. In contrast, salmon uses what may be thought of as a indexed voting method (so that multi-matches are discounted and the votes split among all matches). Salmon also required a pre-existing database of known transcripts (though later versions may actually use mapping from things like hisat), while hisat uses the genome and a database of known transcripts (and optionally can search for splicing junctions to find new transcripts).

3.1 Generate expressionsets

samplesheet <- "sample_sheets/tmrc3_samples_20201105.xlsx"

Caveat: This initial section is using salmon quantifications. A majority of analyses used hisat2.

3.1.1 Salmon expressionsets

Currently, I have these disabled.

hs_expt <- sm(create_expt(samplesheet,
                          file_column="hg3891salmonfile",
                          gene_info=hs_annot, tx_gene_map=tx_gene_map))

libsizes <- plot_libsize(hs_expt)
libsizes$plot
nonzero <- plot_nonzero(hs_expt)
box <- plot_boxplot(hs_expt)

hs_write <- write_expt(hs_expt, excel=glue("excel/hs_written_salmon-v{ver}.xlsx"))

hs_valid <- subset_expt(hs_expt, coverage=100000)
valid_write <- write_expt(hs_valid, excel=glue("excel/hs_valid_salmon-v{ver}.xlsx"))

3.1.2 Hisat2 expressionsets

The first thing to note is the large range in coverage. There are multiple samples with coverage which is too low to use. These will be removed shortly.

hs_expt <- create_expt(samplesheet,
                       file_column="hg3891hisatfile", savefile="hs_expt_all.rda",
                       gene_info=hs_annot)

## Reading the sample metadata.

## Dropped 164 rows from the sample metadata because they were blank.

## The sample definitions comprises: 70 rows(samples) and 79 columns(metadata fields).

## Reading count tables.

## Reading count files with read.table().

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30001/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30002/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30003/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30004/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30005/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30006/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30007/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30009/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30010/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30015/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30011/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30012/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30013/outputs/hisat2_hg38_91/forward.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30016/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30017/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30050/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30052/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30071/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30056/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30058/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30018/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30019/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30014/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30021/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30029/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30020/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30038/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30039/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30023/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30025/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30022/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30046/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30047/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30048/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30026/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30030/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30031/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30032/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30024/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30040/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30033/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30049/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30053/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30054/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30037/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30027/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30028/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30034/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30035/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30036/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30044/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30055/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30068/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30070/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30041/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30042/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30043/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30045/outputs/hisat2_hg38_91/concatenated.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30059/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30060/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30061/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30062/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30063/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30051/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30064/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30065/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30066/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30067/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30057/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## /mnt/sshfs/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/rnaseq/lpanamensis_tmrc_2019/preprocessing/tmrc30069/outputs/hisat2_hg38_91/r1_trimmed.count_hg38_91_sno_gene_gene_id.count.xz contains 58307 rows and merges to 58307 rows.

## Finished reading count data.

## Matched 57768 annotations and counts.

## Bringing together the count matrix and gene information.

## Some annotations were lost in merging, setting them to 'undefined'.

## The final expressionset has 58302 rows and 70 columns.

hs_expt <- exclude_genes_expt(hs_expt, column="gene_biotype",
                              method="keep", pattern="protein_coding")

## Before removal, there were 58302 entries.

## Now there are 19754 entries.

## Percent kept: 95.178, 94.234, 94.315, 94.748, 94.024, 93.570, 96.754, 97.306, 96.776, 84.755, 97.443, 97.423, 97.265, 93.491, 88.446, 97.881, 97.278, 97.417, 97.693, 97.133, 96.366, 90.828, 97.594, 97.264, 96.787, 90.052, 97.111, 96.690, 96.935, 93.115, 91.153, 98.074, 97.532, 97.672, 94.754, 97.628, 97.255, 96.931, 97.374, 96.524, 96.219, 98.214, 97.534, 97.489, 97.492, 96.859, 96.661, 97.571, 97.178, 96.855, 86.749, 98.027, 97.371, 97.531, 97.507, 96.959, 97.297, 80.597, 97.823, 97.741, 97.930, 98.008, 97.976, 98.054, 97.572, 97.353, 97.547, 97.568, 97.741, 98.048

## Percent removed: 4.822, 5.766, 5.685, 5.252, 5.976, 6.430, 3.246, 2.694, 3.224, 15.245, 2.557, 2.577, 2.735, 6.509, 11.554, 2.119, 2.722, 2.583, 2.307, 2.867, 3.634, 9.172, 2.406, 2.736, 3.213, 9.948, 2.889, 3.310, 3.065, 6.885, 8.847, 1.926, 2.468, 2.328, 5.246, 2.372, 2.745, 3.069, 2.626, 3.476, 3.781, 1.786, 2.466, 2.511, 2.508, 3.141, 3.339, 2.429, 2.822, 3.145, 13.251, 1.973, 2.629, 2.469, 2.493, 3.041, 2.703, 19.403, 2.177, 2.259, 2.070, 1.992, 2.024, 1.946, 2.428, 2.647, 2.453, 2.432, 2.259, 1.952

libsizes <- plot_libsize(hs_expt)

## The scale difference between the smallest and largest
## libraries is > 10. Assuming a log10 scale is better, set scale=FALSE if not.

libsizes$plot

nonzero <- plot_nonzero(hs_expt)
nonzero$plot

## Warning: ggrepel: 44 unlabeled data points (too many overlaps). Consider
## increasing max.overlaps

box <- plot_boxplot(hs_expt)

## This data will benefit from being displayed on the log scale.

## If this is not desired, set scale='raw'

## Some entries are 0.  We are on log scale, adding 1 to the data.

## Changed 431739 zero count features.

box

3.2 Minimum coverage sample filtering

I arbitrarily chose 3,000,000 counts as a minimal level of coverage. We may want this to be higher.

hs_valid <- subset_expt(hs_expt, coverage=3000000)

## Subsetting given a minimal number of counts/sample.

## There were 70, now there are 64 samples.

plot_libsize(hs_valid)$plot

## The scale difference between the smallest and largest
## libraries is > 10. Assuming a log10 scale is better, set scale=FALSE if not.

valid_write <- sm(write_expt(hs_valid, excel=glue("excel/hs_valid-v{ver}.xlsx")))

## Error in checkModelStatus(fit, showWarnings = showWarnings, colinearityCutoff = colinearityCutoff) : 
##   The variables specified in this model are redundant,
## so the design matrix is not full rank

## Error in checkModelStatus(fit, showWarnings = showWarnings, colinearityCutoff = colinearityCutoff) : 
##   The variables specified in this model are redundant,
## so the design matrix is not full rank

4 Questions from Maria Adelaida

The following comes from an email 20190830 from Maria Adelaida.

1. Samples WT1010 and WT1011 PBMCs from two healthy donors processed 2h, 7h and 12h after sample procurement. This is an analysis to explore the time-effect on gene expression and define steps for data analysis for patient samples considering time-dependent effects.

   1. An initial PCA on the raw data would be very useful to see if there is clustering based on time or (as usual), mostly a donor-specific effect. Then I think a hierarchical clustering of genes based on time-dependent modifications to see what is mostly affected (if any) - like what you guys did for T.cruzi.

2. Samples from SU1017, SU1034 Samples from TMRC CL patients. m= monocyte, n= neutrophil. Samples labeled “1” are taken before treatment and those “2” mid way through treatment. This is exiting, because these will be our first neutrophil transcriptomes.

In an attempt to poke at these questions, I mapped the reads to hg38_91 using salmon and hisat2. It is very noteworthy that the salmon mappings are exhibiting some serious problems and should be looked into further. The hisat2 mappings are significantly more ‘normal’. Having said that, two samples remain basically unusable: tmrc30009 (1034n1) and (to a smaller degree) tmrc30007 (1017n1) have too few reads as shown above.

4.1 Preparation

To address these, I added to the end of the sample sheet columns named ‘condition’, ‘batch’, ‘donor’, and ‘time’. These are filled in with shorthand values according to the above.

4.2 Global view

Before addressing the questions explicitly by subsetting the data, I want to get a look at the samples as they are.

hs_valid <- set_expt_batches(hs_valid, fact="donor")
hs_valid <- set_expt_samplenames(hs_valid, newnames=pData(hs_valid)[["samplename"]])
all_norm <- normalize_expt(hs_valid, convert="cpm", batch="svaseq",
                           filter=TRUE)

## This function will replace the expt$expressionset slot with:

## svaseq(cpm(cbcb(data)))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Leaving the data in its current base format, keep in mind that
##  some metrics are easier to see when the data is log2 transformed, but
##  EdgeR/DESeq do not accept transformed data.

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Step 1: performing count filter with option: cbcb

## Removing 5330 low-count genes (14424 remaining).

## Step 2: not normalizing the data.

## Step 3: converting the data with cpm.

## The method is: svaseq.

## Step 4: doing batch correction with svaseq.

## Using the current state of normalization.

## Passing the data to all_adjusters using the svaseq estimate type.

## batch_counts: Before batch/surrogate estimation, 673268 entries are x>1: 73%.

## batch_counts: Before batch/surrogate estimation, 110099 entries are x==0: 12%.

## batch_counts: Before batch/surrogate estimation, 139769 entries are 0<x<1: 15%.

## The be method chose 10 surrogate variables.

## Attempting svaseq estimation with 10 surrogates.

## There are 25617 (3%) elements which are < 0 after batch correction.

## Setting low elements to zero.

## Step 4: not transforming the data.

all_norm <- normalize_expt(all_norm, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

## transform_counts: Found 25617 values equal to 0, adding 1 to the matrix.

plt <- plot_pca(all_norm, plot_labels=FALSE)$plot

## Not putting labels on the PC plot.

pp(file=glue("images/tmrc3_pca_nolabels-v{ver}.pdf"), image=plt)

## Writing the image to: images/tmrc3_pca_nolabels-v202011.pdf and calling dev.off().

plt

all_ts <- plot_tsne(all_norm)
all_ts$plot

## Warning: ggrepel: 3 unlabeled data points (too many overlaps). Consider
## increasing max.overlaps

knitr::kable(all_pca$table)

	sampleid	condition	batch	batch_int	colors	labels	PC1	PC2	pc_1	pc_2	pc_3	pc_4	pc_5	pc_6	pc_7	pc_8	pc_9	pc_10	pc_11	pc_12	pc_13	pc_14	pc_15	pc_16	pc_17	pc_18	pc_19	pc_20	pc_21	pc_22	pc_23	pc_24	pc_25	pc_26	pc_27	pc_28	pc_29	pc_30	pc_31	pc_32	pc_33	pc_34	pc_35	pc_36	pc_37	pc_38	pc_39	pc_40	pc_41	pc_42	pc_43	pc_44	pc_45	pc_46	pc_47	pc_48	pc_49	pc_50	pc_51	pc_52	pc_53	pc_54	pc_55	pc_56	pc_57	pc_58	pc_59	pc_60	pc_61	pc_62	pc_63
1010-2	1010-2	PBMC	d1010	1	#1B9E77	1010-2	-0.1625	-0.1077	-0.1625	-0.1077	-0.0168	-0.0473	0.2822	-0.0556	0.1021	-0.1039	0.0532	-0.0064	0.0322	-0.0849	0.1536	0.0818	-0.1769	0.2848	-0.1958	0.0130	-0.2244	0.0820	-0.1367	0.0351	-0.0468	0.0084	-0.0805	0.0383	0.1238	-0.0075	-0.1143	0.0149	0.0473	0.0303	0.0483	-0.1890	-0.0009	-0.0242	-0.0531	-0.2497	0.0327	0.0020	0.1702	0.3754	-0.0029	-0.0593	-0.1377	-0.0657	-0.0245	-0.0112	-0.0272	0.0133	0.2162	0.0211	0.0720	0.0214	0.0081	0.0452	-0.2030	-0.2586	-0.2092	-0.0013	-0.0094	-0.1357	-0.0012
1010-7	1010-7	PBMC	d1010	1	#1B9E77	1010-7	-0.1070	-0.0109	-0.1070	-0.0109	-0.1094	-0.0352	0.2833	-0.0207	-0.0372	0.0430	0.0109	-0.0320	-0.0410	0.0155	-0.0297	0.0479	-0.0545	-0.2129	-0.1861	0.1607	-0.0549	0.0663	-0.0176	0.0298	-0.0289	-0.1390	-0.0738	0.0195	-0.1038	0.0437	-0.1952	-0.1579	-0.0641	0.0328	0.0018	0.2088	0.0891	-0.4388	-0.0568	0.3631	0.1370	-0.0790	-0.2365	-0.0691	0.0313	0.0922	0.2117	0.0422	-0.0361	0.0518	-0.1078	-0.0169	0.1293	-0.0339	-0.0157	-0.0484	-0.0477	0.0133	-0.2251	0.0635	-0.0953	0.0589	0.0960	-0.0808	0.0494
1010-12	1010-12	PBMC	d1010	1	#1B9E77	1010-12	-0.1560	-0.0437	-0.1560	-0.0437	-0.0912	-0.0652	0.3206	0.1441	-0.0429	0.0647	0.0304	-0.0297	-0.0654	0.1591	-0.0464	0.0085	0.0204	-0.3028	-0.0209	0.0833	0.2264	0.0401	0.0622	0.1488	-0.0668	-0.1022	0.1293	0.0191	0.0091	0.0424	0.0381	0.0781	0.0743	-0.0633	-0.0469	-0.0883	-0.1190	0.4483	0.0047	-0.1903	-0.2363	0.0106	0.0391	-0.2801	-0.0848	0.0418	0.0778	-0.0412	0.0048	-0.0733	0.0357	0.0097	0.1113	-0.0063	0.0270	-0.0442	-0.0019	-0.0146	-0.3000	-0.0069	-0.0417	0.0359	0.1189	-0.0984	0.0126
1011-2	1011-2	PBMC	d1011	2	#1B9E77	1011-2	-0.0276	-0.0308	-0.0276	-0.0308	-0.0941	-0.1307	0.3178	-0.0606	0.0986	0.0192	0.0278	-0.0272	0.0140	-0.1551	0.1067	-0.0149	0.0661	0.3169	-0.0041	-0.0942	-0.2626	-0.1314	-0.0207	-0.1867	0.0386	0.1262	-0.1555	-0.0556	0.0106	-0.0253	-0.0184	-0.0262	-0.0705	0.0036	0.0173	0.0295	-0.0314	0.1633	-0.1907	-0.0428	-0.0164	-0.2735	-0.1549	-0.5195	0.0573	0.0842	0.0394	-0.0713	0.0298	-0.0025	-0.0606	-0.0010	-0.0473	-0.0103	-0.0218	0.0151	0.0220	-0.0424	0.1795	0.0168	0.0955	-0.0405	-0.0947	0.0479	-0.0231
1011-7	1011-7	PBMC	d1011	2	#1B9E77	1011-7	0.0020	-0.0523	0.0020	-0.0523	-0.0896	-0.1424	0.3563	0.0968	0.0429	0.0738	0.0485	-0.0247	0.0101	-0.0309	-0.0199	-0.0322	0.0953	0.0375	0.0619	-0.0788	-0.0130	-0.0491	0.0601	-0.0426	0.0389	0.0694	0.0122	-0.1090	-0.0606	-0.0946	0.1307	-0.0138	-0.0108	-0.0783	-0.0632	0.1566	-0.0432	-0.0470	0.2170	0.3079	-0.2544	0.3976	0.2431	0.2217	-0.0987	0.0931	0.0620	-0.2084	0.0407	-0.1733	-0.1628	-0.0192	0.0374	-0.0126	0.0059	0.1125	0.0395	-0.0246	0.2259	-0.0014	0.0839	-0.0091	-0.1073	0.0724	-0.0130
1011-12	1011-12	PBMC	d1011	2	#1B9E77	1011-12	0.0613	-0.0626	0.0613	-0.0626	-0.1137	-0.1145	0.3041	0.1334	-0.0363	0.1128	-0.0322	-0.0188	-0.0151	0.0759	-0.0873	-0.0886	0.1097	-0.1464	0.2004	-0.0452	0.2531	-0.0107	0.0633	0.0183	0.0415	0.0054	0.1471	0.1278	-0.0371	0.0917	0.1471	0.0448	-0.0188	0.0508	0.0179	-0.1213	0.1102	-0.1762	-0.0482	-0.2206	0.3708	-0.2458	-0.0728	0.1578	0.0322	-0.0434	-0.2356	-0.0745	-0.0224	0.0970	-0.0839	0.0002	0.1136	-0.0213	0.0316	0.0194	0.0119	0.0325	0.3685	-0.0154	0.0582	0.0272	0.0216	-0.0878	-0.0261
1034n1	1034n1	Neutrophil	d1034	3	#D95F02	1034n1	0.1631	0.0949	0.1631	0.0949	0.0675	0.0300	0.0255	-0.2904	0.0653	0.1071	-0.0215	0.0613	-0.2872	0.1535	0.0963	0.1069	0.2141	-0.1358	0.0554	-0.0382	-0.1260	0.2002	-0.2121	-0.1026	-0.1820	-0.0195	0.1185	0.2069	0.0575	-0.1542	0.0717	0.1191	0.1232	0.2559	0.0842	0.0838	-0.3890	-0.1319	0.0342	-0.0683	0.0289	0.1345	-0.0033	-0.1213	0.0898	-0.0209	0.0309	-0.0683	0.0467	-0.0059	-0.0011	-0.0119	-0.0384	-0.0502	0.0202	-0.0733	-0.0186	0.0314	0.0701	-0.1422	-0.1269	0.1092	-0.0887	0.0223	-0.0496
1034bp1	1034bp1	Biopsy	d1034	3	#E7298A	1034bp1	-0.0445	-0.0424	-0.0445	-0.0424	0.0565	-0.0498	0.0275	0.0148	-0.1410	0.0235	0.1075	-0.0365	-0.0161	-0.0809	0.0746	0.0304	-0.0230	-0.0061	0.0387	0.0771	0.1601	0.0223	-0.0797	-0.0627	0.1336	-0.0713	0.1777	-0.0147	-0.1323	-0.2931	-0.3400	0.0414	-0.0610	-0.0425	0.0560	0.0011	-0.0169	0.0638	0.0263	-0.0224	0.0282	-0.0512	0.0436	0.0387	0.0238	-0.0187	-0.0414	-0.0011	0.0277	0.0187	-0.0011	-0.0044	-0.0642	-0.0521	0.0079	-0.1203	-0.0280	0.0401	0.0616	0.2481	-0.3672	-0.4518	-0.1059	0.2612	-0.2743
1034n2	1034n2	Neutrophil	d1034	3	#D95F02	1034n2	0.1418	0.0973	0.1418	0.0973	0.0888	0.0358	0.0355	-0.1996	0.1498	0.1158	0.0432	0.0018	-0.4285	0.1062	-0.1113	0.0818	0.0596	-0.0872	0.0623	0.0495	0.0138	0.0330	0.0691	-0.0998	0.0535	0.2075	-0.1184	-0.2503	0.0585	0.1201	-0.1117	0.0659	0.0708	-0.3925	-0.1372	0.0221	0.3523	0.1103	-0.0488	0.0454	-0.0303	-0.0959	0.0172	0.1093	-0.0100	-0.0196	0.0162	0.0340	0.0059	0.0679	0.0139	0.0479	-0.0786	-0.0483	0.0195	-0.1554	-0.0541	0.0513	0.0078	-0.1957	-0.1318	0.0704	-0.0978	-0.0107	-0.0505
1034m2	1034m2	Monocyte	d1034	3	#7570B3	1034m2	-0.1228	0.0277	-0.1228	0.0277	-0.1672	0.0345	-0.0622	-0.4329	0.0357	0.0027	-0.0934	0.0066	0.1319	0.1028	0.1366	-0.0300	-0.0205	0.1194	0.0743	-0.0048	0.1976	-0.2394	0.0629	0.1747	0.0365	-0.0899	-0.0328	-0.1281	0.0170	0.0938	-0.0052	0.1037	-0.0391	0.0540	-0.1018	-0.0981	-0.1821	0.2514	-0.1131	0.2795	0.3968	0.1277	0.0359	0.0632	-0.0316	0.1949	-0.0110	-0.0646	-0.0047	0.0273	-0.0017	-0.0103	0.0409	-0.0503	0.0280	-0.0662	-0.0493	0.0547	-0.0303	0.0858	-0.0640	0.1259	-0.1193	-0.0175	-0.0155
1034m2-	1034m2-	Monocyte	d1034	3	#7570B3	1034m2-	-0.0754	0.0261	-0.0754	0.0261	-0.1667	0.0345	-0.0501	-0.4259	0.0443	-0.0058	-0.1900	-0.0405	0.0950	0.0958	0.0832	-0.0570	-0.0605	0.0825	-0.0021	0.0457	0.1835	-0.0835	0.0698	0.2055	0.0938	-0.0678	-0.0054	0.0451	-0.0790	-0.0633	0.1531	-0.2136	0.0073	0.0005	0.0557	0.0471	0.1862	-0.2523	0.0694	-0.2728	-0.4590	-0.0883	0.0058	-0.0730	-0.0171	-0.1736	0.0333	0.0174	-0.0309	-0.0070	-0.0872	-0.0191	0.0392	-0.0292	0.0282	-0.0723	-0.0211	0.0146	0.0732	0.0848	-0.0605	0.0750	-0.1485	0.0203	-0.0319
2050bp1	2050bp1	Biopsy	d2050	4	#E7298A	2050bp1	-0.1037	0.1760	-0.1037	0.1760	0.1821	-0.1971	-0.1246	0.2244	0.3601	-0.0178	-0.1628	0.0809	0.0857	0.4623	-0.1755	0.0527	-0.0095	0.0259	-0.1011	-0.0324	-0.1444	-0.0941	0.0629	-0.0307	0.0338	-0.0040	0.0356	-0.0652	-0.1126	-0.2927	-0.0187	0.0630	-0.2016	0.0185	-0.0141	0.0695	-0.0062	-0.0054	0.0083	-0.0739	0.0917	-0.0477	0.0102	0.0123	-0.0398	-0.0160	0.0386	0.0176	0.0347	-0.0088	0.0244	0.0138	0.1154	-0.0407	0.0428	-0.0054	0.0058	0.0043	-0.0494	0.1165	0.0430	-0.0397	-0.2712	-0.2286	0.0272
2052bp1	2052bp1	Biopsy	d2052	5	#E7298A	2052bp1	0.0418	0.0159	0.0418	0.0159	-0.0758	0.0653	-0.0078	0.0996	0.0713	-0.0239	0.0392	-0.0183	-0.0614	-0.0483	-0.1087	-0.0033	0.0025	-0.1035	0.0035	-0.0699	-0.1719	0.0659	0.0618	0.2364	-0.2149	0.0159	-0.2548	-0.1178	0.2493	0.2430	0.4360	0.0446	0.0288	0.0878	-0.0452	-0.0097	-0.0126	-0.0905	-0.0257	-0.0126	0.0424	-0.0239	-0.0170	0.0082	0.0068	-0.0140	-0.0335	-0.0345	0.0074	-0.0086	-0.0096	-0.0168	-0.0689	0.0019	0.0103	0.0235	0.0054	-0.0272	-0.1310	0.3163	-0.2062	-0.3672	-0.2025	0.0608	-0.1348
2052e1	2052e1	Eosinophil	d2052	5	#66A61E	2052e1	0.1261	-0.1792	0.1261	-0.1792	0.0452	-0.0823	-0.0318	-0.0254	-0.0618	0.1660	0.3870	-0.0287	-0.0909	0.2974	-0.0197	-0.1116	-0.1264	0.2162	0.5422	-0.0542	-0.1396	-0.0357	-0.1504	0.1298	0.0296	-0.0775	-0.1005	0.1346	0.0080	-0.0425	-0.0482	-0.1368	-0.0118	-0.0649	0.0382	0.0310	0.0291	-0.0216	-0.0586	-0.0388	-0.0115	0.0350	-0.0182	0.0489	-0.0271	0.0354	0.0911	0.0910	-0.0377	-0.0343	0.1123	0.0013	0.0853	0.0871	-0.0441	0.0903	0.0148	-0.0291	-0.1489	0.1910	0.0430	0.0789	0.1043	0.0102	0.0450
2052n2	2052n2	Neutrophil	d2052	5	#D95F02	2052n2	0.2495	0.1901	0.2495	0.1901	-0.1010	-0.1146	0.0273	-0.0176	-0.0749	0.1279	-0.0116	-0.5635	0.0368	-0.0193	-0.2543	0.0594	0.1225	0.3463	-0.2811	-0.1212	0.1670	0.0924	-0.0748	0.0125	0.0115	0.0702	0.1876	0.0200	0.1282	-0.0006	-0.0261	0.1066	0.0790	-0.0149	-0.0182	-0.0473	-0.0169	-0.0357	0.0362	0.0378	0.0028	0.0431	-0.0514	0.0234	0.0311	-0.0463	0.0313	0.1374	-0.0204	0.0213	0.1340	-0.0166	0.0053	0.0457	-0.0227	0.0125	-0.0123	-0.0123	-0.1230	0.1548	0.0729	0.0449	0.0144	-0.0383	0.0204
2065bp1	2065bp1	Biopsy	d2065	6	#E7298A	2065bp1	0.0498	-0.1187	0.0498	-0.1187	-0.1240	0.0629	0.0698	0.0486	0.0631	0.0143	-0.0372	0.0348	0.0904	0.2520	0.0137	0.0740	-0.0057	0.0372	-0.0960	0.1873	0.0189	0.0284	-0.1045	0.0742	0.0987	-0.0329	0.0042	0.0160	-0.0358	-0.0323	-0.0341	-0.0447	0.0350	-0.0572	-0.0287	0.0079	-0.0030	-0.0008	0.0073	0.0285	0.0428	-0.0336	-0.0189	0.0198	0.0966	-0.0643	-0.2421	-0.1300	0.0776	0.0320	-0.1252	-0.0475	-0.7065	0.0683	-0.0514	0.2937	0.1138	-0.1178	-0.1570	-0.0483	0.0053	0.0754	0.0246	0.0140	-0.0146
2066bp1	2066bp1	Biopsy	d2066	7	#E7298A	2066bp1	-0.0886	0.0405	-0.0886	0.0405	0.0794	-0.0060	-0.0324	-0.1214	-0.0127	-0.0179	-0.0936	0.0134	0.0031	-0.1509	-0.0581	-0.0824	0.1028	0.0117	0.2190	-0.2577	0.0335	-0.1472	0.2350	-0.4091	0.0851	0.0857	0.1446	-0.1092	-0.0018	-0.0919	0.1804	0.0734	-0.1215	0.0725	0.0312	0.0177	0.0202	-0.1138	0.1006	-0.0744	0.0229	-0.0534	-0.0411	0.0310	-0.0378	0.0203	-0.0089	-0.1128	-0.0106	0.0146	-0.1055	-0.0043	-0.0553	0.0384	-0.0081	0.1386	0.0219	-0.0437	-0.4353	0.0111	-0.2523	0.0359	0.2724	0.0073	0.0928
2068m1	2068m1	Monocyte	d2068	8	#7570B3	2068m1	-0.1505	-0.0567	-0.1505	-0.0567	-0.1275	-0.0176	-0.1925	0.3168	-0.1501	-0.0091	0.0673	-0.1267	-0.2051	0.1392	0.1703	-0.0695	0.0785	0.0059	-0.0841	-0.3221	-0.0631	-0.3084	-0.0529	0.0547	0.0593	0.0128	0.1135	-0.2116	-0.0850	0.3166	-0.1680	-0.0994	0.1302	0.2217	0.0382	0.1035	-0.0953	-0.0689	-0.0101	-0.0577	-0.0665	-0.0753	0.0068	0.0559	-0.0281	-0.0547	0.0429	0.0375	0.0360	-0.1131	-0.0336	0.0477	-0.1124	-0.1436	0.0012	-0.0996	-0.0145	0.0579	0.0350	-0.1166	-0.1124	0.1415	-0.0802	0.0092	-0.0562
2068n1	2068n1	Neutrophil	d2068	8	#D95F02	2068n1	0.1164	0.1292	0.1164	0.1292	0.0341	0.0169	0.0392	0.1734	-0.1570	-0.0546	-0.0463	0.2250	0.0983	0.0368	0.1432	-0.0835	0.1231	0.1561	-0.0202	-0.0490	0.1205	-0.0240	-0.1966	-0.1025	-0.3460	-0.1298	-0.1401	0.0154	0.0242	-0.1298	0.0287	0.0631	0.0078	0.0289	-0.2023	0.0219	0.2625	-0.0068	0.0922	0.0518	-0.0692	0.0063	-0.0442	-0.0680	-0.1879	0.1684	-0.3337	0.1980	-0.0749	-0.0596	-0.0297	-0.2450	0.0052	-0.1083	0.0173	-0.1435	-0.0856	0.0532	-0.0698	0.0343	-0.0639	0.1941	0.0060	0.0548	-0.0717
2068e1	2068e1	Eosinophil	d2068	8	#66A61E	2068e1	0.0406	-0.2274	0.0406	-0.2274	0.1538	0.0895	-0.0434	0.1191	-0.2620	-0.0105	-0.1073	-0.0305	0.0109	0.0734	0.0316	0.1818	0.1261	0.0893	-0.0563	0.0703	0.0629	-0.0407	0.3142	-0.0085	-0.2125	0.1125	-0.0797	0.0081	0.1978	-0.1549	0.0116	-0.2737	-0.0176	-0.2245	0.3987	-0.0890	-0.0824	0.0141	-0.1037	-0.0220	0.0769	0.1602	-0.2078	0.0163	-0.1139	-0.0037	0.0038	-0.0611	0.1670	0.0075	-0.1132	0.1130	0.0255	0.0267	0.0265	-0.0993	-0.0214	0.0200	0.0100	0.0201	-0.0182	0.1358	-0.0957	0.0283	-0.0251
2068bp1	2068bp1	Biopsy	d2068	8	#E7298A	2068bp1	-0.1184	0.1442	-0.1184	0.1442	0.2589	-0.2740	-0.0968	-0.0173	-0.1239	-0.0343	0.0549	0.0178	-0.0018	-0.1074	-0.0164	-0.0110	-0.0702	-0.0285	0.0248	0.0384	-0.0245	0.1910	-0.0883	-0.1208	0.2571	-0.4892	0.1255	-0.1883	0.3569	0.0539	0.1188	-0.2102	0.1942	-0.0810	0.0467	-0.0409	-0.0067	0.0178	0.0070	0.0771	0.0057	-0.0897	0.0720	-0.0623	0.0217	0.0503	-0.0778	-0.0869	0.0020	0.0068	-0.0990	-0.0215	-0.0249	-0.0280	-0.0478	0.0035	-0.0001	-0.0105	0.0805	0.0271	0.0315	0.0918	-0.1319	-0.1031	0.0812
2072n1	2072n1	Neutrophil	d2072	10	#D95F02	2072n1	0.1316	0.1433	0.1316	0.1433	-0.0533	0.0781	0.0420	0.0653	-0.1600	-0.0653	0.0307	0.2492	0.0647	0.1171	0.2641	0.0015	0.0869	0.1477	-0.0485	-0.2136	-0.0191	0.1411	0.1754	0.0628	0.0194	-0.1604	0.0814	0.0673	-0.0241	-0.0336	0.0477	0.0764	0.0364	-0.0761	-0.1688	-0.1595	0.1485	-0.0668	-0.0037	0.0760	0.0356	-0.0998	0.0506	0.0349	0.1595	-0.1682	0.3534	-0.2610	0.2793	0.0572	0.1609	0.0504	-0.0219	-0.0230	0.0656	-0.0921	-0.0223	0.0882	-0.1074	-0.0539	0.2134	-0.1190	0.0701	0.0542	-0.0517
2072e1	2072e1	Eosinophil	d2072	10	#66A61E	2072e1	-0.0093	-0.1685	-0.0093	-0.1685	0.0620	0.0722	0.0283	0.0529	-0.2525	-0.1121	-0.6493	-0.1039	-0.3048	-0.0252	-0.1668	-0.0598	-0.1469	0.1308	0.2231	0.1064	-0.1437	0.0173	-0.1539	0.0526	0.0553	-0.0678	-0.0401	-0.0238	-0.0556	0.0254	-0.0557	0.0831	-0.0561	0.0211	-0.1465	-0.0140	-0.0879	0.0101	0.0571	0.0154	-0.0087	0.0024	0.0298	0.0024	0.0114	-0.0308	-0.0162	-0.0275	0.0221	0.0215	-0.0914	-0.0205	0.0358	-0.0327	0.0341	-0.0797	-0.0341	0.0254	-0.0763	-0.0095	0.2326	-0.1246	0.0363	0.0301	-0.0062
2072bp1	2072bp1	Biopsy	d2072	10	#E7298A	2072bp1	-0.2574	0.0973	-0.2574	0.0973	0.2706	-0.3588	-0.0528	-0.0251	-0.1273	0.0214	0.1428	-0.0442	-0.0038	0.0420	0.1950	0.0384	0.1532	0.0752	-0.0346	0.3530	0.1716	-0.1018	-0.0755	0.0168	0.0565	0.2813	-0.2103	0.0092	-0.0651	0.0768	0.2405	0.0720	-0.0080	0.0596	-0.0313	0.0945	-0.1009	-0.0954	-0.0051	-0.0272	-0.0060	0.0223	0.0015	0.0643	0.0652	-0.0468	-0.0153	0.0468	0.0051	0.0598	0.0465	0.0311	-0.0885	-0.0555	0.0255	-0.1574	-0.0639	0.0625	-0.0711	-0.0742	0.2064	-0.1741	0.1962	0.0342	0.0341
2071bp1	2071bp1	Biopsy	d2071	9	#E7298A	2071bp1	-0.0192	0.1473	-0.0192	0.1473	0.0668	-0.0974	-0.0878	-0.0813	-0.1831	-0.0082	0.1753	-0.0281	-0.1553	-0.3655	0.0485	0.0103	-0.0748	-0.1249	0.0758	-0.1180	0.0481	0.0890	0.0301	0.3526	-0.2512	0.1013	0.0166	-0.1498	-0.1822	-0.2175	-0.1651	0.1244	-0.1881	0.0402	0.0056	-0.0141	0.0295	-0.0257	-0.0398	-0.0661	-0.0028	-0.0752	-0.0202	0.0087	0.0036	-0.0034	-0.0764	-0.1207	0.0487	-0.0736	-0.1082	-0.0120	-0.1079	0.0461	0.0250	0.1371	0.0405	-0.0430	-0.0648	0.0186	0.1846	0.0886	-0.2337	-0.2144	0.1963
2073m1	2073m1	Monocyte	d2073	11	#7570B3	2073m1	-0.0467	-0.0478	-0.0467	-0.0478	-0.2485	-0.1266	-0.1244	0.0138	-0.1442	0.0087	-0.0104	0.2424	-0.1000	-0.0674	-0.1482	0.2898	0.1318	-0.0068	-0.0661	-0.0709	-0.0344	-0.1242	-0.0082	0.0215	0.0037	0.0245	0.0163	0.0111	-0.0264	-0.0178	-0.0143	-0.2017	0.0088	0.0135	-0.0535	0.1604	0.0208	0.0443	0.0806	-0.0097	-0.0393	0.0501	0.1983	-0.0557	0.2434	0.1405	-0.1917	0.0076	-0.0131	0.4454	0.2083	0.0259	0.1595	0.3248	0.1193	0.0668	-0.0263	-0.0745	-0.0175	0.0038	0.0077	0.0029	-0.0131	0.0118	-0.0111
2073n1	2073n1	Neutrophil	d2073	11	#D95F02	2073n1	0.2337	0.0775	0.2337	0.0775	-0.0594	-0.1337	0.0110	0.0061	-0.0929	-0.0478	-0.1375	-0.1249	0.1810	0.0381	0.2127	0.0238	0.0952	-0.1827	0.0829	0.0151	-0.0368	-0.0018	-0.0047	-0.1238	-0.1165	-0.1256	-0.1924	-0.0136	0.0147	0.0021	-0.0617	0.0044	-0.1081	0.1058	0.0348	-0.0328	0.1147	0.1313	0.0213	-0.0187	0.0256	-0.0064	0.2039	0.0564	0.1308	-0.0616	0.1588	0.2416	-0.2915	-0.0148	-0.1750	0.5293	-0.0487	0.0867	-0.0688	0.0477	0.0620	-0.1109	-0.0043	-0.0524	-0.0040	-0.0177	-0.0369	-0.0039	-0.0101
2073e1	2073e1	Eosinophil	d2073	11	#66A61E	2073e1	0.1099	-0.2401	0.1099	-0.2401	0.0664	-0.1073	-0.0391	-0.0025	0.0385	-0.0103	-0.0916	0.1264	-0.1767	-0.0293	0.0662	-0.2369	-0.0331	-0.0090	-0.2190	0.0070	0.1334	-0.0212	0.1490	-0.0746	-0.1204	0.0122	-0.0119	-0.0367	0.0360	-0.0354	-0.0255	-0.0696	-0.0343	-0.0905	0.0772	-0.0186	-0.0679	-0.0439	-0.1278	-0.0443	0.0369	0.0061	0.1782	0.0193	0.2871	0.0091	0.1366	-0.0711	-0.4791	-0.1281	0.2727	-0.2907	-0.0432	-0.1592	-0.0957	0.1376	0.0519	0.0253	0.0111	0.1013	0.0572	0.0300	0.0163	-0.0110	-0.0254
2073bp1	2073bp1	Biopsy	d2073	11	#E7298A	2073bp1	-0.0291	0.1201	-0.0291	0.1201	0.0049	-0.0122	-0.0751	0.0273	0.0493	-0.0044	-0.0160	0.0469	-0.0864	-0.1280	0.0376	-0.0268	-0.0388	-0.0174	0.0498	0.0634	0.0798	-0.1692	0.0404	-0.2905	0.0097	0.0575	-0.0324	0.5710	-0.0421	0.3008	-0.1826	-0.0246	0.1678	0.0071	-0.0028	-0.1467	0.1110	0.0101	-0.0074	0.0479	-0.0687	0.1762	-0.0312	-0.0106	-0.0324	0.0216	-0.0111	-0.1074	0.0350	-0.0546	-0.0455	-0.0066	-0.0597	0.0424	0.0326	0.0346	0.0044	0.0016	-0.0933	0.1694	0.0601	-0.1140	-0.2892	-0.2818	0.0050
2068m2	2068m2	Monocyte	d2068	8	#7570B3	2068m2	-0.1711	-0.0310	-0.1711	-0.0310	-0.1232	-0.0872	-0.1519	0.0823	0.1750	-0.0523	-0.0096	-0.1763	0.0416	-0.1287	0.0177	-0.1607	0.0686	0.0853	0.0792	0.0960	-0.1945	0.2168	0.0862	0.0019	-0.1518	-0.2269	0.0046	0.0691	-0.3901	0.1342	0.1171	0.2433	0.0352	-0.2761	0.3269	0.0639	0.0588	0.0550	0.1530	-0.0035	0.0410	0.0410	-0.0019	-0.0063	0.1611	0.0851	-0.0405	-0.0140	0.0533	0.0844	0.0709	0.0135	-0.0515	-0.0192	0.0473	-0.1310	-0.0402	0.0260	0.0282	0.0098	-0.0764	0.1902	0.0369	0.0631	-0.0745
2068n2	2068n2	Neutrophil	d2068	8	#D95F02	2068n2	0.1037	0.1086	0.1037	0.1086	0.0990	-0.0600	0.0610	0.0786	0.1447	-0.0570	0.0022	0.0841	0.0252	-0.2056	-0.1629	-0.1143	-0.0851	-0.0296	0.0511	0.2249	-0.1406	-0.3235	-0.1479	0.1988	-0.1239	0.1428	0.3065	0.1140	0.1714	-0.0387	0.0696	-0.1586	-0.1682	0.1311	0.0727	-0.1990	0.0739	0.1018	0.1327	0.1989	-0.0514	-0.1177	0.0408	0.0452	0.0759	-0.0988	0.1479	-0.0458	0.0338	0.0747	0.0696	0.0659	-0.1202	-0.0511	0.0252	-0.1586	-0.0395	0.0504	0.0287	-0.0025	-0.1110	0.2258	0.0918	0.0786	-0.0971
2068e2	2068e2	Eosinophil	d2068	8	#66A61E	2068e2	-0.0065	-0.2581	-0.0065	-0.2581	0.1976	0.0429	-0.0770	0.1183	0.2558	-0.0281	0.0188	-0.1467	0.1195	-0.2524	-0.0096	0.0701	0.2727	-0.0100	0.0833	-0.0528	0.0158	0.1654	0.1532	0.1474	0.1672	-0.0494	-0.1131	0.1297	0.0025	-0.0626	-0.0908	-0.1633	-0.0206	0.1136	-0.5019	0.0923	-0.0230	0.0558	-0.0797	-0.1181	0.0226	0.1463	-0.0773	0.0021	0.1010	-0.0745	0.0319	-0.0269	-0.0307	0.0635	-0.0673	-0.0033	-0.0303	-0.0726	-0.0135	-0.1246	-0.0527	0.0633	-0.0020	0.0286	-0.0793	0.1925	0.0114	-0.0201	-0.0628
2072m2	2072m2	Monocyte	d2072	10	#7570B3	2072m2	-0.1266	-0.0184	-0.1266	-0.0184	-0.2279	-0.0265	-0.1329	-0.0172	0.1543	-0.0273	0.0501	-0.0847	0.0813	0.0031	-0.0188	-0.1143	-0.0474	-0.0204	0.1119	0.1418	-0.0267	0.0823	0.0972	-0.0681	-0.2381	0.0935	0.0925	-0.1184	0.2564	-0.0491	-0.2532	-0.0647	0.1605	0.0561	-0.0398	0.1105	0.1118	-0.1509	0.1353	-0.1300	0.2434	0.1105	0.2579	-0.2460	-0.3454	-0.1374	0.0281	-0.1402	-0.0498	0.0615	0.0245	-0.0029	-0.0540	0.0192	0.0135	-0.0784	-0.0152	0.0366	-0.0006	-0.0609	0.2083	-0.1052	0.1032	0.0634	-0.0274
2072n2	2072n2	Neutrophil	d2072	10	#D95F02	2072n2	0.1243	0.1618	0.1243	0.1618	-0.0204	0.0475	0.0506	-0.0353	0.0375	-0.0429	0.0909	0.1345	-0.0294	-0.0403	0.0093	0.0156	-0.0799	-0.0188	-0.0526	-0.0799	-0.2391	-0.0634	0.2875	0.1715	0.1853	0.0257	0.2696	0.1172	-0.0598	-0.0101	0.0427	-0.0449	0.0681	-0.2021	-0.0106	0.0056	-0.1591	-0.0963	-0.1329	0.0146	0.0526	0.1022	0.0677	-0.0847	0.0225	0.0251	-0.3211	0.3177	-0.2015	-0.1858	-0.2066	0.0778	-0.0178	-0.0375	0.0646	-0.1736	-0.0742	0.0403	-0.1231	-0.0370	0.2005	-0.1083	0.1021	0.0913	-0.0578
2072e2	2072e2	Eosinophil	d2072	10	#66A61E	2072e2	0.0134	-0.2113	0.0134	-0.2113	0.0986	0.0927	-0.0075	-0.0829	0.0927	-0.0182	-0.0395	-0.0415	0.1276	-0.0486	-0.0505	0.0962	0.0567	-0.1614	-0.0328	-0.1618	-0.0184	-0.0569	-0.2429	0.0248	0.0592	-0.0019	0.1297	-0.0028	-0.0266	-0.1098	0.1170	0.0959	0.2153	0.2106	0.2977	0.2114	0.4089	0.1532	-0.3749	0.0511	-0.0256	0.0727	-0.0103	0.1050	-0.0221	0.1021	0.0101	-0.0909	-0.0261	0.0043	0.0048	-0.0271	-0.0130	-0.0516	0.0217	-0.0915	-0.0366	0.0368	-0.1011	-0.0116	0.2463	-0.1142	0.0269	0.0247	-0.0208
2073m2	2073m2	Monocyte	d2073	11	#7570B3	2073m2	-0.0634	-0.1002	-0.0634	-0.1002	-0.2147	-0.1560	-0.1068	0.0116	0.0105	0.0124	-0.0066	0.2392	0.0395	-0.1634	-0.2276	0.2086	0.0840	0.0944	0.0888	-0.0012	0.0633	0.0483	-0.0251	-0.0689	-0.0896	-0.0979	-0.0050	0.0545	-0.1269	-0.0321	0.0821	0.0837	0.0829	-0.0858	-0.0263	-0.0423	-0.2209	0.0079	-0.2793	0.1946	-0.1086	-0.1820	-0.0170	0.1296	-0.3763	-0.3494	0.1301	0.1805	-0.0944	0.0363	0.0459	0.0143	-0.0549	-0.1184	-0.1144	0.0567	0.0574	-0.0021	0.0324	-0.0970	-0.0345	-0.0123	-0.0151	-0.0466	-0.0137
2073n2	2073n2	Neutrophil	d2073	11	#D95F02	2073n2	0.2121	-0.0307	0.2121	-0.0307	0.0315	-0.0992	-0.0364	0.0318	-0.0323	-0.0799	-0.0096	-0.2375	0.1411	-0.0287	-0.0127	0.0582	-0.2265	-0.1096	0.1305	0.0095	0.0672	-0.1558	0.0962	-0.0055	-0.0177	-0.0609	-0.0896	0.0319	-0.1222	-0.0729	0.1314	-0.1244	0.0774	0.0173	-0.0247	0.0479	0.0452	0.0072	0.0282	0.1222	0.1212	0.0518	0.0179	-0.1955	0.1237	-0.1356	-0.0479	0.1131	0.1935	-0.1716	0.1990	-0.2605	0.0741	0.0297	0.0359	0.0797	0.0261	0.0297	-0.0245	-0.5038	-0.1741	-0.1655	-0.1288	-0.0148	0.0434
2073e2	2073e2	Eosinophil	d2073	11	#66A61E	2073e2	0.0798	-0.2785	0.0798	-0.2785	0.1088	-0.1340	-0.0294	-0.0965	0.1182	-0.0222	-0.0588	0.0789	-0.1333	-0.0880	0.0504	-0.2845	-0.0999	-0.0319	-0.2521	-0.0572	0.1019	0.0564	-0.0219	-0.0018	0.0097	-0.0086	0.0611	0.0585	-0.0642	-0.0241	0.1022	-0.0486	0.0158	-0.0004	-0.1341	0.0051	-0.0281	-0.0959	0.0151	-0.0354	0.0470	-0.0896	0.0157	0.0116	-0.2869	0.3123	0.0093	0.1840	0.3219	-0.1154	0.2322	0.3418	-0.0685	0.2210	-0.0331	0.0239	0.0349	-0.0929	0.0211	-0.0121	0.0095	-0.0042	-0.0076	0.0353	-0.0418
2068m3	2068m3	Monocyte	d2068	8	#7570B3	2068m3	-0.1008	-0.0035	-0.1008	-0.0035	-0.1771	-0.0032	-0.1313	-0.0008	-0.2551	-0.0019	0.0894	-0.0701	0.0513	0.1146	-0.0294	-0.1020	-0.2452	-0.1450	-0.1220	-0.0509	-0.1165	0.3127	-0.1692	-0.2387	0.1067	0.2606	0.0694	0.0445	-0.1079	0.0161	0.2282	-0.1334	-0.3510	-0.0070	-0.1428	-0.1510	0.0809	0.0697	-0.2375	0.0027	0.0446	0.2096	0.0516	-0.0776	-0.0484	-0.0825	-0.0042	-0.0604	-0.0297	0.0300	-0.0288	-0.0254	-0.0109	-0.0202	0.0380	-0.0907	-0.0391	0.0529	0.0468	0.0189	-0.0650	0.1480	-0.0707	0.0536	-0.0334
2068n3	2068n3	Neutrophil	d2068	8	#D95F02	2068n3	0.1720	0.1523	0.1720	0.1523	0.0434	0.0349	0.0639	0.0842	-0.1154	-0.0326	0.0442	0.1805	0.2174	-0.0900	-0.2837	-0.1127	-0.3661	-0.0158	-0.0490	-0.0242	0.1277	-0.1449	-0.0606	-0.0399	0.0668	0.0289	-0.2378	-0.0450	-0.0027	-0.0069	-0.0608	0.2288	0.2219	-0.0937	-0.0080	0.2665	-0.1948	-0.0883	-0.1018	-0.2717	0.0293	0.1171	-0.0086	-0.0035	0.0944	0.0131	0.1366	-0.1387	0.0446	0.0966	-0.0597	0.1342	-0.0400	-0.0104	-0.0059	-0.1017	-0.0277	0.0311	0.0300	0.0258	-0.0649	0.1998	-0.0006	0.0675	-0.0488
2068e3	2068e3	Eosinophil	d2068	8	#66A61E	2068e3	0.0662	-0.2231	0.0662	-0.2231	0.1278	0.1375	-0.0131	-0.0142	-0.1315	0.0091	0.0579	-0.0203	0.1973	0.0644	-0.0289	0.1752	-0.0706	-0.0488	0.0248	-0.0237	-0.0300	0.1817	0.1595	0.0069	0.2064	0.2399	-0.1211	0.0920	0.0250	0.0560	-0.0471	0.2017	0.0423	0.1425	0.1219	-0.0182	-0.0634	-0.0524	0.2586	0.2182	-0.0814	-0.3960	0.3394	-0.0983	-0.0832	0.1103	-0.0750	-0.0084	-0.0587	-0.0333	0.0772	-0.0290	0.0064	-0.0936	0.0859	-0.0941	-0.0287	0.0483	-0.0117	0.0805	-0.0039	0.1571	-0.0845	-0.1055	-0.0151
2072m3	2072m3	Monocyte	d2072	10	#7570B3	2072m3	-0.1004	-0.0089	-0.1004	-0.0089	-0.2396	0.0320	-0.1714	-0.0081	0.1000	0.0194	0.0505	-0.0314	0.1193	0.0695	-0.0987	-0.1409	-0.1087	-0.0390	0.1026	0.0168	0.0918	0.1505	0.0623	-0.1816	-0.2193	0.1106	0.0245	-0.0015	0.1886	-0.0997	-0.0854	-0.1140	0.0562	0.0403	-0.1045	0.0419	-0.1712	0.0960	-0.0216	0.0960	-0.2312	-0.2597	-0.2069	0.2704	0.3526	0.2356	-0.0725	0.0279	0.1179	-0.1057	-0.1271	0.0282	0.0608	-0.0714	0.0126	-0.0841	-0.0089	0.0093	-0.0051	-0.1172	0.1679	-0.1231	0.0128	0.0224	-0.0217
2072n3	2072n3	Neutrophil	d2072	10	#D95F02	2072n3	0.1494	0.1314	0.1494	0.1314	0.0215	0.0166	0.0285	-0.0575	0.0826	0.1143	0.1236	0.0134	-0.1199	-0.0036	-0.1902	0.0006	-0.0911	0.0374	-0.1206	-0.0754	0.0560	0.0616	0.2002	-0.0318	0.0792	-0.2775	-0.2653	0.0219	-0.0957	0.1165	-0.0866	-0.1457	-0.3207	0.3277	0.2010	-0.0821	-0.0404	0.1417	0.1270	-0.0772	-0.0669	0.0586	-0.0820	0.1042	-0.2171	0.0977	-0.0358	-0.0734	-0.0551	0.1694	0.0191	-0.0996	-0.1179	-0.0828	-0.0137	-0.1070	-0.0226	0.0163	0.0116	-0.1867	0.1712	-0.1156	0.0971	0.1182	-0.0646
2072e3	2072e3	Eosinophil	d2072	10	#66A61E	2072e3	0.0412	-0.2091	0.0412	-0.2091	0.0825	0.1376	-0.0721	-0.0900	0.1482	0.0427	0.1183	-0.0312	0.1584	-0.0525	-0.0480	0.1219	0.0948	-0.1813	-0.0864	-0.2037	0.0754	-0.2031	-0.4191	-0.0974	-0.0801	-0.0872	-0.0574	-0.1002	-0.0724	0.1053	-0.0014	-0.0636	-0.1542	-0.3637	-0.0179	-0.2510	-0.1672	-0.1395	0.2558	-0.0609	0.0410	-0.0244	0.0056	-0.0848	0.0399	-0.0644	0.0001	-0.0536	-0.0024	-0.0091	-0.1088	-0.0038	0.0762	-0.0492	0.0429	-0.0697	-0.0152	0.0307	-0.0757	-0.0021	0.2177	-0.1010	0.0060	-0.0013	-0.0078
2159bp1	2159bp1	Biopsy	d2159	12	#E7298A	2159bp1	-0.0546	0.1199	-0.0546	0.1199	0.2846	-0.2221	-0.1533	-0.1104	-0.0266	0.0207	-0.1452	0.0280	0.1449	0.1513	-0.1313	0.0547	0.0107	-0.0035	-0.0712	-0.1819	-0.0533	0.1055	-0.0017	0.2088	-0.1780	0.1243	0.0111	0.1438	-0.0935	0.2419	-0.1810	-0.0367	0.0989	-0.0562	-0.0540	-0.0828	0.0654	0.0270	-0.0761	0.0639	-0.0401	-0.0546	-0.0054	-0.0745	-0.0507	0.0924	0.0042	-0.1998	-0.0519	-0.0813	-0.1954	-0.0507	0.1369	0.0634	-0.0205	0.2437	0.1129	-0.0904	0.1419	-0.0704	-0.1483	-0.0423	0.2507	0.2798	-0.0793
2073m3	2073m3	Monocyte	d2073	11	#7570B3	2073m3	-0.0505	-0.0790	-0.0505	-0.0790	-0.2310	-0.1772	-0.0878	-0.0489	-0.0489	-0.0542	-0.0289	0.2652	0.0107	-0.0665	-0.1373	0.2687	0.0565	-0.0039	0.0211	0.0508	-0.0449	0.0714	-0.0360	0.0576	0.2082	0.0669	-0.0320	-0.0590	0.1507	0.0377	-0.0766	0.1636	-0.0663	0.0914	0.0793	-0.1279	0.1969	-0.0121	0.2460	-0.1838	0.1055	0.2039	-0.2266	-0.0360	0.1253	0.0981	0.1431	0.1641	0.0842	-0.4314	0.0552	-0.0425	-0.0215	-0.0833	-0.1191	0.0271	-0.0069	0.0088	0.0763	0.0159	-0.0040	0.0332	0.0822	-0.0099	-0.0197
2073n3	2073n3	Neutrophil	d2073	11	#D95F02	2073n3	0.2452	0.1116	0.2452	0.1116	-0.0719	-0.1284	0.0216	-0.0061	-0.0040	-0.1352	-0.1494	-0.0896	0.2223	-0.0291	0.3032	0.0187	0.0687	-0.2959	0.1158	0.0795	-0.2414	-0.0197	0.0010	0.0037	0.1342	0.0954	0.0711	-0.0914	0.0866	0.1156	-0.1382	0.0478	-0.0642	-0.1449	-0.0012	0.0210	-0.1651	-0.0361	-0.0685	-0.1226	-0.1657	-0.0071	-0.2130	0.1979	-0.2317	0.1389	-0.0193	-0.0338	0.0241	0.1838	0.2679	-0.2400	0.0253	0.0212	-0.0815	0.0656	-0.0224	-0.0146	-0.0024	0.0642	0.0519	0.0215	-0.0149	0.0152	-0.0219
2073e3	2073e3	Eosinophil	d2073	11	#66A61E	2073e3	0.1166	-0.2414	0.1166	-0.2414	0.0761	-0.1165	-0.0248	-0.0684	0.0607	-0.0023	0.0826	0.1279	-0.0402	0.0509	0.1267	-0.2266	-0.1204	-0.0409	-0.1480	-0.0016	-0.0118	0.0311	0.0894	-0.0456	0.0310	0.0535	0.0358	-0.1323	0.0400	0.0601	-0.0670	0.2183	0.0969	0.1561	0.0376	-0.0157	0.0294	0.1490	0.1727	0.1584	-0.0504	0.1113	-0.2354	-0.0383	0.0774	-0.4517	-0.1251	0.1725	0.1095	0.2483	-0.2233	-0.0778	0.2102	0.0879	0.0052	0.1115	-0.0039	-0.0524	0.0006	0.1192	0.0730	0.0425	-0.0118	0.0364	-0.0068
2162m1	2162m1	Monocyte	d2162	13	#7570B3	2162m1	-0.0187	0.0568	-0.0187	0.0568	-0.2048	0.0914	-0.2833	0.1429	0.0104	0.0193	0.0710	-0.1572	-0.2306	-0.0333	0.0974	-0.0975	0.0598	-0.1100	-0.1636	0.0808	-0.0403	-0.1994	-0.0381	0.0979	0.2566	-0.0839	-0.2376	0.2397	0.2029	-0.3555	0.0921	0.2322	0.0028	-0.1019	-0.0074	-0.0731	0.0331	-0.0112	-0.0113	0.0788	-0.0023	-0.1078	-0.0265	-0.0221	-0.1108	0.0520	0.0053	-0.1703	-0.0773	-0.0078	-0.1392	-0.0076	0.1802	0.0350	0.0128	0.1412	0.0570	-0.0452	0.0505	-0.1059	-0.0204	0.0494	0.1261	0.0958	-0.0411
2162n1	2162n1	Neutrophil	d2162	13	#D95F02	2162n1	0.2259	0.1946	0.2259	0.1946	-0.0766	0.1583	-0.0758	0.2015	0.2194	-0.1219	-0.0605	0.1629	-0.0530	-0.0999	0.1044	-0.0885	0.1321	0.1582	0.0984	0.1460	0.2810	0.2662	-0.2144	0.0698	0.2144	0.2069	-0.0150	-0.1659	-0.1511	0.1025	0.0190	-0.2504	0.0218	0.0451	0.2166	0.0510	-0.0858	0.0478	-0.0264	0.0424	0.0847	-0.0371	0.0469	-0.0748	-0.0306	0.0011	0.0121	-0.0644	0.0056	-0.0943	-0.0951	0.0237	0.2442	0.0446	0.0162	0.0956	0.0461	-0.0354	-0.1611	-0.0140	-0.0256	0.0017	-0.0264	0.0086	-0.0148
2162e1	2162e1	Eosinophil	d2162	13	#66A61E	2162e1	0.2072	-0.1195	0.2072	-0.1195	0.0608	0.1901	-0.1400	0.0073	-0.0576	0.0631	0.1861	-0.0271	-0.0039	0.0116	0.0116	0.2388	-0.0384	0.1506	-0.0803	0.4565	-0.0732	-0.1294	0.1136	-0.1528	-0.1312	-0.1712	0.3020	-0.1386	-0.1352	0.1829	0.1061	0.1486	-0.0996	0.1165	-0.2014	0.0716	0.0458	-0.0326	-0.0902	-0.1455	-0.0519	-0.0581	0.0129	0.0024	-0.0282	0.1332	0.0573	-0.1294	-0.0130	-0.0925	-0.1246	0.0155	0.1774	0.0142	-0.0029	0.1158	0.0405	-0.0394	0.0508	0.0062	0.0178	0.0091	-0.0510	0.0496	0.0312
2162bp1	2162bp1	Biopsy	d2162	13	#E7298A	2162bp1	0.1274	0.0023	0.1274	0.0023	-0.0061	0.0920	-0.0965	-0.0449	-0.0477	-0.0097	-0.0040	0.0011	0.0139	0.0112	0.0626	-0.0243	-0.0620	0.0672	-0.0416	-0.0314	-0.0556	0.0053	-0.0290	0.0463	-0.0411	0.0091	0.0050	-0.0964	-0.0448	-0.0468	0.0393	-0.0595	0.0110	-0.0391	0.0018	0.0775	-0.0191	0.0750	0.0284	0.0546	-0.0658	0.0832	-0.0413	-0.0538	-0.0273	0.0311	-0.0121	-0.0478	-0.0056	0.0345	-0.0366	0.0348	-0.0675	-0.0230	0.0451	-0.0361	-0.0069	0.0410	0.2792	0.0535	-0.1727	-0.2044	0.4532	-0.6905	-0.1331
macrofagos	Macrofagos	Macrophage	unknown	14	#E6AB02	Macrofagos	-0.1400	0.0708	-0.1400	0.0708	0.0645	0.0952	0.0275	0.0994	0.1802	0.0372	-0.0828	-0.0354	-0.0957	-0.0679	0.2488	0.2738	-0.2768	0.0268	0.0494	-0.1501	0.2114	0.0469	-0.0293	-0.0252	-0.0590	-0.0491	0.0693	0.1442	0.2543	0.1337	0.0270	0.0876	-0.2990	-0.1043	0.0561	0.3201	-0.0162	0.0458	-0.0401	-0.0126	0.0281	-0.0321	0.0874	0.0324	0.0048	-0.0545	0.1396	0.2458	0.0899	-0.0060	-0.0552	-0.1426	-0.0763	0.1083	0.2299	0.1005	0.0768	0.0285	0.1241	0.0739	0.0430	0.0709	0.1158	0.0796	-0.0240
macrofagos+sbv	Macrofagos+SbV	Macrophage	unknown	14	#E6AB02	Mcrfgs+SbV	-0.1148	0.0839	-0.1148	0.0839	0.0507	0.1470	0.0639	0.0443	0.1032	-0.2500	0.0582	-0.0920	-0.0915	0.0480	0.0054	0.1215	-0.0621	-0.0372	0.0612	-0.0185	0.0738	-0.0336	0.0524	-0.0638	-0.0340	0.0043	-0.0633	-0.0877	-0.0758	-0.0602	-0.0180	0.0328	0.0880	0.0868	0.0119	-0.2830	-0.0160	-0.1311	-0.1555	0.0215	-0.0416	0.0560	0.1803	-0.0011	0.0877	0.0201	0.0329	0.1133	0.1371	0.0468	-0.0053	-0.1647	0.0072	0.0939	-0.3139	-0.1577	-0.1367	-0.5760	0.1264	0.0955	0.0416	0.0729	0.1255	0.0011	-0.0364
macrofagos+10772	Macrofagos+10772	Macrophage	unknown	14	#E6AB02	Mcrf+10772	-0.1301	0.0875	-0.1301	0.0875	0.0540	0.1574	0.0185	0.0592	0.0524	0.2442	-0.0823	-0.0042	-0.0083	-0.0490	0.1529	0.1638	-0.2313	0.0224	0.0231	-0.0622	0.0232	0.0158	-0.0083	0.0140	-0.0237	0.0269	0.0376	0.0189	0.0034	-0.0068	0.0437	-0.0281	-0.0013	-0.0227	-0.0381	0.1024	0.0405	0.0114	0.1245	0.0917	-0.0063	0.0607	-0.1820	-0.0501	0.0238	-0.1323	-0.2954	-0.1826	-0.0954	-0.0098	0.3556	0.3544	0.1233	-0.2864	-0.3765	0.0883	-0.0103	-0.1075	0.0050	0.0097	0.0281	0.0144	0.0158	0.0613	0.0139
macrofagos+10772+sbv	Macrofagos+10772+SbV	Macrophage	unknown	14	#E6AB02	M+10772+SV	-0.1299	0.0675	-0.1299	0.0675	0.0744	0.1465	0.0706	0.0311	0.0764	-0.2177	0.0439	-0.0529	-0.0303	0.0670	-0.0627	0.0365	0.0132	-0.0281	0.0570	0.0066	0.0599	-0.0009	0.0216	-0.0572	-0.0193	-0.0613	-0.0548	-0.1217	-0.0218	-0.0095	-0.0475	0.0836	0.0013	0.0881	0.0203	-0.1606	-0.0361	-0.0951	-0.0970	0.1004	-0.1400	0.0204	-0.0817	-0.0507	0.0347	-0.0586	-0.0999	-0.0280	-0.1166	-0.1213	0.1445	0.1048	-0.0341	0.4812	-0.1269	0.1398	-0.1807	0.5702	0.1129	0.0301	0.0319	0.0468	0.0410	0.0472	-0.0499
macrofagos+2169	Macrofagos+2169	Macrophage	unknown	14	#E6AB02	Mcrfg+2169	-0.1445	0.0433	-0.1445	0.0433	0.0818	0.1579	0.0062	0.0974	-0.0019	0.4194	-0.0869	0.0645	0.1244	-0.0621	0.0394	-0.0679	-0.0125	0.0149	0.0595	0.0421	-0.0411	0.0338	-0.0010	0.0017	0.0604	0.0027	0.0035	-0.1675	-0.0248	-0.0411	-0.0154	0.0539	0.1345	0.0702	-0.0234	-0.2134	-0.0106	-0.0934	-0.0106	0.0445	-0.0408	0.0811	-0.2113	0.0059	-0.0102	-0.0034	0.0763	0.0378	-0.2562	-0.0644	0.1709	-0.0286	-0.0690	0.2915	0.4243	-0.1576	0.2192	-0.2478	0.0589	-0.0988	-0.0154	-0.0081	-0.0620	0.0061	0.0028
macrofagos+2169+sbv	Macrofagos+2169+SbV	Macrophage	unknown	14	#E6AB02	Mc+2169+SV	-0.1605	-0.0566	-0.1605	-0.0566	0.1122	0.2087	0.0406	0.0179	-0.1095	-0.2317	0.1156	0.0589	0.0797	0.0756	-0.0823	-0.1932	0.1097	0.1415	0.0442	-0.0492	0.0800	0.0623	-0.0363	0.1405	0.0444	0.0585	0.2008	0.1055	0.1548	0.0760	0.0212	0.1547	-0.1482	-0.0916	0.0203	0.1028	0.0232	-0.0498	0.1067	0.0104	-0.0287	0.0844	-0.1409	-0.0856	-0.0208	0.1626	0.1735	0.1242	-0.2311	0.2656	-0.0319	0.0198	-0.0401	-0.0061	-0.1901	0.0991	-0.0581	-0.1130	0.0366	-0.3585	-0.0487	-0.1658	-0.2455	-0.0860	0.0036
macrofagos+12309	Macrofagos+12309	Macrophage	unknown	14	#E6AB02	Mcrf+12309	0.0599	-0.0375	0.0599	-0.0375	-0.0357	0.0336	-0.0203	0.0645	0.0772	0.0157	-0.0336	0.0371	0.0185	0.0190	-0.0079	-0.0472	0.0299	0.0094	-0.0171	0.0146	0.0120	0.0187	-0.0176	0.0195	0.0033	-0.0236	-0.0210	0.0030	0.0336	0.0131	-0.0749	0.0268	0.0060	0.0015	0.0258	-0.1042	-0.0043	-0.0236	-0.0875	0.0070	-0.0769	0.0601	0.0381	-0.0386	0.0053	0.0245	-0.0186	0.0292	0.0358	0.0406	0.0533	0.1014	-0.0683	-0.2058	0.2858	0.1339	-0.4911	-0.0329	0.1458	0.0488	-0.1611	-0.1273	0.0796	0.1195	0.6511
macrofagos+12309+sbv	Macrofagos+12309+SbV	Macrophage	unknown	14	#E6AB02	M+12309+SV	-0.0927	0.0846	-0.0927	0.0846	0.0441	0.1406	0.0742	-0.0320	-0.0021	-0.2662	0.0810	-0.0707	-0.0682	0.0145	-0.0493	0.0407	-0.0134	-0.0449	0.0269	0.0281	-0.0182	-0.0098	0.0237	-0.0476	-0.0089	0.0300	-0.0330	-0.0655	-0.0869	-0.0883	0.0120	-0.0720	0.1823	0.0513	-0.0349	-0.1898	0.0282	-0.0342	0.0166	-0.0295	-0.0105	0.1029	-0.0123	-0.0320	0.0118	0.1364	0.0342	0.2296	0.0688	0.2242	0.0830	0.0861	0.0121	-0.3868	0.2067	0.2441	0.5188	0.1833	0.0723	0.1060	0.0540	0.0327	0.0458	0.0742	0.0450
macrofagos+12367+sbv	Macrofagos+12367+SbV	Macrophage	unknown	14	#E6AB02	M+12367+SV	-0.0953	0.1030	-0.0953	0.1030	0.0411	0.1661	0.0893	-0.1336	-0.1309	-0.2716	0.0888	-0.0426	-0.0398	0.0230	-0.0536	0.0156	0.0302	-0.0479	-0.1105	-0.0023	-0.1221	-0.0104	-0.0187	-0.0770	-0.0245	0.0246	-0.0269	0.0864	-0.1471	-0.0688	0.0980	-0.2204	0.1859	-0.0283	-0.0826	0.2227	-0.0273	0.2900	0.2853	-0.0798	0.2196	-0.1820	-0.1711	0.2091	-0.1151	-0.1592	0.1156	-0.0147	-0.1669	-0.1888	0.0820	-0.0589	-0.0153	0.1013	0.1376	0.0796	-0.1793	-0.1613	0.0919	0.1198	0.0716	0.0318	-0.0105	0.0614	0.0411
macrofagos+1126	Macrofagos+1126	Macrophage	unknown	14	#E6AB02	Mcrfg+1126	0.0363	-0.0131	0.0363	-0.0131	-0.0362	0.0654	-0.0018	0.0107	0.0067	-0.0117	-0.0192	0.0400	0.0032	0.0293	0.0241	-0.0165	0.0249	0.0093	-0.0726	-0.0240	-0.0387	-0.0076	-0.0162	0.0288	0.0036	0.0193	0.0094	0.0781	-0.0085	0.0156	0.0058	-0.0081	-0.0062	-0.0298	-0.0052	0.0835	-0.0019	0.0941	0.0657	-0.0034	0.0355	-0.0362	0.0320	0.0513	-0.0005	-0.0272	-0.0168	-0.0289	-0.0089	-0.0087	-0.0532	-0.1064	-0.0175	0.2053	-0.3404	-0.4018	0.4541	0.1342	0.0809	0.0740	-0.1218	-0.1033	0.0347	0.0578	0.5724
macrofagos+12251	Macrofagos+12251	Macrophage	unknown	14	#E6AB02	Mcrf+12251	-0.1193	0.1087	-0.1193	0.1087	0.0478	0.1937	-0.0031	0.0308	-0.0629	0.4981	-0.1015	0.0359	0.0849	-0.1058	0.0396	-0.0880	0.0296	-0.0481	-0.0605	0.0510	-0.1475	-0.0099	0.0123	0.0129	0.0023	0.0677	-0.0251	-0.0154	-0.0930	-0.0695	0.0476	-0.1143	0.1532	0.0040	0.0132	-0.0759	0.0052	0.0912	0.0562	-0.1232	0.0994	-0.0378	0.1813	0.0617	0.0092	0.0880	0.1945	0.2845	0.1577	0.1170	-0.1387	-0.1830	-0.0026	0.0087	-0.3135	0.2365	-0.1835	0.2172	0.0101	0.0002	0.0362	-0.0172	-0.0246	-0.0184	-0.0233
macrofagos+12251+sbv	Macrofagos+12251+SbV	Macrophage	unknown	14	#E6AB02	M+12251+SV	-0.1099	0.0930	-0.1099	0.0930	0.0636	0.2177	0.0503	-0.1269	-0.1408	0.0439	0.0272	0.0378	0.1080	-0.0111	-0.1997	-0.2820	0.3484	-0.0647	-0.0457	0.1065	-0.0978	-0.0508	-0.0028	-0.0212	0.0311	-0.1066	-0.0870	0.0204	0.1326	0.1221	-0.1779	0.0409	-0.2584	-0.0218	0.0644	0.1864	-0.0085	0.0040	-0.1866	0.0070	-0.0380	0.0129	0.1875	0.0173	0.0746	-0.2364	-0.1944	0.0649	0.2689	-0.2147	0.2019	0.0191	0.1036	-0.0409	0.0504	0.0779	0.1359	-0.1150	0.0283	0.0103	0.0151	0.0146	0.0444	-0.0455	-0.0391

write.csv(all_pca$table, file="coords/hs_donor_pca_coords.csv")
plot_corheat(all_norm)$plot

plot_topn(hs_valid)$plot

## `geom_smooth()` using formula 'y ~ x'

4.3 All celltypes clinical outcome

hs_clinical <- subset_expt(hs_valid, subset="condition!='PBMC'&condition!='Macrophage'")

## Using a subset expression.

## There were 64, now there are 46 samples.

hs_clinical <- set_expt_conditions(hs_clinical, fact="clinicaloutcome")
hs_clinical <- set_expt_batches(hs_clinical, fact="typeofcells")
chosen_colors <- c("#D95F02", "#7570B3", "#1B9E77")
names(chosen_colors) <- c("cure", "failure", "lost")
hs_clinical <- set_expt_colors(expt=hs_clinical, colors=chosen_colors)

## The new colors are a character, changing according to condition.

hs_clinical_norm <- normalize_expt(hs_clinical, filter=TRUE, convert="cpm", norm="quant")

## This function will replace the expt$expressionset slot with:

## cpm(quant(cbcb(data)))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Leaving the data in its current base format, keep in mind that
##  some metrics are easier to see when the data is log2 transformed, but
##  EdgeR/DESeq do not accept transformed data.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: performing count filter with option: cbcb

## Removing 5632 low-count genes (14122 remaining).

## Step 2: normalizing the data with quant.

## Step 3: converting the data with cpm.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: not transforming the data.

hs_clinical_norm <- normalize_expt(hs_clinical, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

## transform_counts: Found 277629 values equal to 0, adding 1 to the matrix.

plt <- plot_pca(hs_clinical_norm, plot_labels=FALSE)$plot

## Not putting labels on the PC plot.

pp(file=glue("images/all_clinical_nobatch_pca-v{ver}.pdf"), image=plt)

## Writing the image to: images/all_clinical_nobatch_pca-v202011.pdf and calling dev.off().

plt

5 Cell types

The following blocks split the samples into a few groups by sample type and look at the distributions between them.

5.1 Implementation details

Get top/bottom n genes for each cell type, using clinical outcome as the factor of interest. For the moment, use sva for the DE analysis. Provide cpms for the top/bottom n genes.

Start with top/bottom 200. Perform default logFC and p-value as well.

5.1.1 Shared contrasts

Here is the contrast we will use throughput, I am leaving open the option to add more.

keepers <- list(
  "fail_vs_cure" = c("failure", "cure"))

5.2 Monocytes

mono <- subset_expt(hs_valid, subset="typeofcells=='Monocytes'")

## Using a subset expression.

## There were 64, now there are 11 samples.

mono <- set_expt_conditions(mono, fact="clinicaloutcome")
mono <- set_expt_batches(mono, fact="donor")
mono <- set_expt_colors(expt=mono, colors=chosen_colors)

## The new colors are a character, changing according to condition.

save_result <- save(mono, file="rda/monocyte_expt.rda")
mono_norm <- normalize_expt(mono, convert="cpm", batch="svaseq", filter=TRUE)

## This function will replace the expt$expressionset slot with:

## svaseq(cpm(cbcb(data)))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Leaving the data in its current base format, keep in mind that
##  some metrics are easier to see when the data is log2 transformed, but
##  EdgeR/DESeq do not accept transformed data.

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Step 1: performing count filter with option: cbcb

## Removing 9100 low-count genes (10654 remaining).

## Step 2: not normalizing the data.

## Step 3: converting the data with cpm.

## The method is: svaseq.

## Step 4: doing batch correction with svaseq.

## Using the current state of normalization.

## Passing the data to all_adjusters using the svaseq estimate type.

## batch_counts: Before batch/surrogate estimation, 109909 entries are x>1: 94%.

## batch_counts: Before batch/surrogate estimation, 2700 entries are x==0: 2%.

## batch_counts: Before batch/surrogate estimation, 4585 entries are 0<x<1: 4%.

## The be method chose 3 surrogate variables.

## Attempting svaseq estimation with 3 surrogates.

## There are 358 (0%) elements which are < 0 after batch correction.

## Setting low elements to zero.

## Step 4: not transforming the data.

mono_norm <- normalize_expt(mono_norm, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

## transform_counts: Found 358 values equal to 0, adding 1 to the matrix.

plt <- plot_pca(mono_norm, plot_labels=FALSE)$plot

## Not putting labels on the PC plot.

pp(file=glue("images/mono_pca_normalized-v{ver}.pdf"), image=plt)

## Writing the image to: images/mono_pca_normalized-v202011.pdf and calling dev.off().

plt

mono_de <- sm(all_pairwise(mono, model_batch="svaseq", filter=TRUE))

mono_tables <- combine_de_tables(mono_de, keepers=keepers,
                                 excel=glue::glue("excel/monocyte_clinical_all_tables-v{ver}.xlsx"))

## Deleting the file excel/monocyte_clinical_all_tables-v202011.xlsx before writing the tables.

## Writing a legend of columns.

## Printing a pca plot before/after surrogates/batch estimation.

## Working on 1/1: fail_vs_cure which is: failure/cure.

## Found table with failure_vs_cure

## Adding venn plots for fail_vs_cure.

## Limma expression coefficients for fail_vs_cure; R^2: 0.908; equation: y = 1.08x - 0.714

## Deseq expression coefficients for fail_vs_cure; R^2: 0.932; equation: y = 1.01x - 0.122

## Edger expression coefficients for fail_vs_cure; R^2: 0.931; equation: y = 1.01x - 0.12

## Writing summary information, compare_plot is: TRUE.

## Performing save of excel/monocyte_clinical_all_tables-v202011.xlsx.

written <- write_xlsx(data=mono_tables[["data"]][[1]],
                      excel=glue::glue("excel/monocyte_clinical_table-v{ver}.xlsx"))

## Saving to: excel/monocyte_clinical_table-v202011.xlsx

mono_sig <- extract_significant_genes(mono_tables, according_to="deseq")
written <- write_xlsx(data=mono_sig[["deseq"]][["ups"]][[1]],
                      excel=glue::glue("excel/monocyte_clinical_sigup-v{ver}.xlsx"))

## Saving to: excel/monocyte_clinical_sigup-v202011.xlsx

written <- write_xlsx(data=mono_sig[["deseq"]][["downs"]][[1]],
                      excel=glue::glue("excel/monocyte_clinical_sigdown-v{ver}.xlsx"))

## Saving to: excel/monocyte_clinical_sigdown-v202011.xlsx

mono_pct_sig <- extract_significant_genes(mono_tables, n=200, lfc=NULL, p=NULL, according_to="deseq")

## Getting the top and bottom 200 genes.

written <- write_xlsx(data=mono_pct_sig[["deseq"]][["ups"]][[1]],
                      excel=glue::glue("excel/monocyte_clinical_sigup_pct-v{ver}.xlsx"))

## Saving to: excel/monocyte_clinical_sigup_pct-v202011.xlsx

written <- write_xlsx(data=mono_pct_sig[["deseq"]][["downs"]][[1]],
                      excel=glue::glue("excel/monocyte_clinical_sigdown_pct-v{ver}.xlsx"))

## Saving to: excel/monocyte_clinical_sigdown_pct-v202011.xlsx

mono_sig$summary_df

## data frame with 0 columns and 1 row

mono_cpm <- sm(normalize_expt(mono, convert="cpm"))
written <- write_xlsx(data=exprs(mono_cpm),
                      excel=glue::glue("excel/monocyte_cpm_before_batch-v{ver}.xlsx"))

## Saving to: excel/monocyte_cpm_before_batch-v202011.xlsx

mono_bcpm <- sm(normalize_expt(mono, filter=TRUE, convert="cpm", batch="svaseq"))
written <- write_xlsx(data=exprs(mono_bcpm),
                      excel=glue::glue("excel/monocyte_cpm_after_batch-v{ver}.xlsx"))

## Saving to: excel/monocyte_cpm_after_batch-v202011.xlsx

5.3 Neutrophils

neut <- subset_expt(hs_valid, subset="typeofcells=='Neutrophils'")

## Using a subset expression.

## There were 64, now there are 13 samples.

neut <- set_expt_conditions(neut, fact="clinicaloutcome")
neut <- set_expt_batches(neut, fact="donor")
neut <- set_expt_colors(expt=neut, colors=chosen_colors)

## The new colors are a character, changing according to condition.

save_result <- save(neut, file="rda/neutrophil_expt.rda")
neut_norm <- sm(normalize_expt(neut, convert="cpm", batch="svaseq", filter=TRUE))
neut_norm <- normalize_expt(neut_norm, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

## transform_counts: Found 1824 values equal to 0, adding 1 to the matrix.

plt <- plot_pca(neut_norm, plot_labels=FALSE)$plot

## Not putting labels on the PC plot.

pp(file=glue("images/neut_pca_normalized-v{ver}.pdf"), image=plt)

## Writing the image to: images/neut_pca_normalized-v202011.pdf and calling dev.off().

plt

neut_de <- sm(all_pairwise(neut, model_batch="svaseq", filter=TRUE))

neut_tables <- combine_de_tables(neut_de, keepers=keepers,
                                 excel=glue::glue("excel/neutrophil_clinical_all_tables-v{ver}.xlsx"))

## Deleting the file excel/neutrophil_clinical_all_tables-v202011.xlsx before writing the tables.

## Writing a legend of columns.

## Printing a pca plot before/after surrogates/batch estimation.

## Working on 1/1: fail_vs_cure which is: failure/cure.

## Found table with failure_vs_cure

## Adding venn plots for fail_vs_cure.

## Limma expression coefficients for fail_vs_cure; R^2: 0.863; equation: y = 0.998x - 0.256

## Deseq expression coefficients for fail_vs_cure; R^2: 0.888; equation: y = 0.917x + 0.744

## Edger expression coefficients for fail_vs_cure; R^2: 0.887; equation: y = 0.918x + 0.891

## Writing summary information, compare_plot is: TRUE.

## Performing save of excel/neutrophil_clinical_all_tables-v202011.xlsx.

written <- write_xlsx(data=neut_tables[["data"]][[1]],
                      excel=glue::glue("excel/neutrophil_clinical_table-v{ver}.xlsx"))

## Saving to: excel/neutrophil_clinical_table-v202011.xlsx

neut_sig <- extract_significant_genes(neut_tables, according_to="deseq")
written <- write_xlsx(data=neut_sig[["deseq"]][["ups"]][[1]],
                      excel=glue::glue("excel/neutrophil_clinical_sigup-v{ver}.xlsx"))

## Saving to: excel/neutrophil_clinical_sigup-v202011.xlsx

written <- write_xlsx(data=neut_sig[["deseq"]][["downs"]][[1]],
                      excel=glue::glue("excel/neutrophil_clinical_sigdown-v{ver}.xlsx"))

## Saving to: excel/neutrophil_clinical_sigdown-v202011.xlsx

neut_pct_sig <- extract_significant_genes(neut_tables, n=200, lfc=NULL, p=NULL, according_to="deseq")

## Getting the top and bottom 200 genes.

written <- write_xlsx(data=neut_sig[["deseq"]][["ups"]][[1]],
                      excel=glue::glue("excel/neutrophil_clinical_sigup_pct-v{ver}.xlsx"))

## Saving to: excel/neutrophil_clinical_sigup_pct-v202011.xlsx

written <- write_xlsx(data=neut_sig[["deseq"]][["downs"]][[1]],
                      excel=glue::glue("excel/neutrophil_clinical_sigdown_pct-v{ver}.xlsx"))

## Saving to: excel/neutrophil_clinical_sigdown_pct-v202011.xlsx

neut_cpm <- sm(normalize_expt(neut, convert="cpm"))
written <- write_xlsx(data=exprs(neut_cpm),
                      excel=glue::glue("excel/neutrophil_cpm_before_batch-v{ver}.xlsx"))

## Saving to: excel/neutrophil_cpm_before_batch-v202011.xlsx

neut_bcpm <- sm(normalize_expt(neut, filter=TRUE, convert="cpm", batch="svaseq"))
written <- write_xlsx(data=exprs(neut_bcpm),
                      excel=glue::glue("excel/neutrophil_cpm_after_batch-v{ver}.xlsx"))

## Saving to: excel/neutrophil_cpm_after_batch-v202011.xlsx

5.4 Eosinophils

eo <- subset_expt(hs_valid, subset="typeofcells=='Eosinophils'")

## Using a subset expression.

## There were 64, now there are 11 samples.

eo <- set_expt_conditions(eo, fact="clinicaloutcome")
eo <- set_expt_batches(eo, fact="donor")
eo <- set_expt_colors(expt=eo, colors=chosen_colors)

## The new colors are a character, changing according to condition.

save_result <- save(eo, file="rda/eosinophil_expt.rda")
eo_norm <- sm(normalize_expt(eo, convert="cpm", batch="svaseq", filter=TRUE))
eo_norm <- normalize_expt(eo_norm, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

## transform_counts: Found 1554 values equal to 0, adding 1 to the matrix.

plt <- plot_pca(eo_norm, plot_labels=FALSE)$plot

## Not putting labels on the PC plot.

pp(file=glue("images/eo_pca_normalized-v{ver}.pdf"), image=plt)

## Writing the image to: images/eo_pca_normalized-v202011.pdf and calling dev.off().

plt

eo_de <- sm(all_pairwise(eo, model_batch="svaseq", filter=TRUE))

eo_tables <- combine_de_tables(eo_de, keepers=keepers,
                               excel=glue::glue("excel/eosinophil_clinical_all_tables-v{ver}.xlsx"))

## Deleting the file excel/eosinophil_clinical_all_tables-v202011.xlsx before writing the tables.

## Writing a legend of columns.

## Printing a pca plot before/after surrogates/batch estimation.

## Working on 1/1: fail_vs_cure which is: failure/cure.

## Found table with failure_vs_cure

## Adding venn plots for fail_vs_cure.

## Limma expression coefficients for fail_vs_cure; R^2: 0.914; equation: y = 1.08x - 0.747

## Deseq expression coefficients for fail_vs_cure; R^2: 0.931; equation: y = 1x - 0.0128

## Edger expression coefficients for fail_vs_cure; R^2: 0.931; equation: y = 1x - 0.0567

## Writing summary information, compare_plot is: TRUE.

## Performing save of excel/eosinophil_clinical_all_tables-v202011.xlsx.

written <- write_xlsx(data=eo_tables[["data"]][[1]],
                      excel=glue::glue("excel/eosinophil_clinical_table-v{ver}.xlsx"))

## Saving to: excel/eosinophil_clinical_table-v202011.xlsx

eo_sig <- extract_significant_genes(eo_tables, according_to="deseq")
written <- write_xlsx(data=eo_sig[["deseq"]][["ups"]][[1]],
                      excel=glue::glue("excel/eosinophil_clinical_sigup-v{ver}.xlsx"))

## Saving to: excel/eosinophil_clinical_sigup-v202011.xlsx

written <- write_xlsx(data=eo_sig[["deseq"]][["downs"]][[1]],
                      excel=glue::glue("excel/eosinophil_clinical_sigdown-v{ver}.xlsx"))

## Saving to: excel/eosinophil_clinical_sigdown-v202011.xlsx

eo_pct_sig <- extract_significant_genes(eo_tables, n=200, lfc=NULL, p=NULL, according_to="deseq")

## Getting the top and bottom 200 genes.

written <- write_xlsx(data=eo_pct_sig[["deseq"]][["ups"]][[1]],
                      excel=glue::glue("excel/eosinophil_clinical_sigup_pct-v{ver}.xlsx"))

## Saving to: excel/eosinophil_clinical_sigup_pct-v202011.xlsx

written <- write_xlsx(data=eo_pct_sig[["deseq"]][["downs"]][[1]],
                      excel=glue::glue("excel/eosinophil_clinical_sigdown_pct-v{ver}.xlsx"))

## Saving to: excel/eosinophil_clinical_sigdown_pct-v202011.xlsx

eo_cpm <- sm(normalize_expt(eo, convert="cpm"))
written <- write_xlsx(data=exprs(eo_cpm),
                      excel=glue::glue("excel/eosinophil_cpm_before_batch-v{ver}.xlsx"))

## Saving to: excel/eosinophil_cpm_before_batch-v202011.xlsx

eo_bcpm <- sm(normalize_expt(eo, filter=TRUE, convert="cpm", batch="svaseq"))
written <- write_xlsx(data=exprs(eo_bcpm),
                      excel=glue::glue("excel/eosinophil_cpm_after_batch-v{ver}.xlsx"))

## Saving to: excel/eosinophil_cpm_after_batch-v202011.xlsx

5.5 Biopsies

biop <- subset_expt(hs_valid, subset="typeofcells=='Biopsy'")

## Using a subset expression.

## There were 64, now there are 11 samples.

biop <- set_expt_conditions(biop, fact="clinicaloutcome")
biop <- set_expt_batches(biop, fact="donor")
biop <- set_expt_colors(expt=biop, colors=chosen_colors)

## The new colors are a character, changing according to condition.

save_result <- save(biop, file="rda/biopsy_expt.rda")
biop_norm <- normalize_expt(biop, batch="svaseq", filter=TRUE, convert="cpm")

## This function will replace the expt$expressionset slot with:

## svaseq(cpm(cbcb(data)))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Leaving the data in its current base format, keep in mind that
##  some metrics are easier to see when the data is log2 transformed, but
##  EdgeR/DESeq do not accept transformed data.

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Step 1: performing count filter with option: cbcb

## Removing 6342 low-count genes (13412 remaining).

## Step 2: not normalizing the data.

## Step 3: converting the data with cpm.

## The method is: svaseq.

## Step 4: doing batch correction with svaseq.

## Using the current state of normalization.

## Passing the data to all_adjusters using the svaseq estimate type.

## batch_counts: Before batch/surrogate estimation, 143597 entries are x>1: 97%.

## batch_counts: Before batch/surrogate estimation, 535 entries are x==0: 0%.

## batch_counts: Before batch/surrogate estimation, 3400 entries are 0<x<1: 2%.

## The be method chose 3 surrogate variables.

## Attempting svaseq estimation with 3 surrogates.

## There are 90 (0%) elements which are < 0 after batch correction.

## Setting low elements to zero.

## Step 4: not transforming the data.

biop_norm <- normalize_expt(biop_norm, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

## transform_counts: Found 90 values equal to 0, adding 1 to the matrix.

plt <- plot_pca(biop_norm)$plot
pp(file=glue("images/biop_pca_normalized-v{ver}.pdf"), image=plt)

## Writing the image to: images/biop_pca_normalized-v202011.pdf and calling dev.off().

plt

biop_de <- sm(all_pairwise(biop, model_batch="svaseq", filter=TRUE))

biop_tables <- combine_de_tables(biop_de, keepers=keepers,
                                 excel=glue::glue("excel/biopsy_clinical_all_tables-v{ver}.xlsx"))

## Deleting the file excel/biopsy_clinical_all_tables-v202011.xlsx before writing the tables.

## Writing a legend of columns.

## Printing a pca plot before/after surrogates/batch estimation.

## Working on 1/1: fail_vs_cure which is: failure/cure.

## Found table with failure_vs_cure

## Adding venn plots for fail_vs_cure.

## Limma expression coefficients for fail_vs_cure; R^2: 0.961; equation: y = 0.954x + 0.241

## Deseq expression coefficients for fail_vs_cure; R^2: 0.962; equation: y = 0.959x + 0.359

## Edger expression coefficients for fail_vs_cure; R^2: 0.962; equation: y = 0.959x + 0.513

## Writing summary information, compare_plot is: TRUE.

## Performing save of excel/biopsy_clinical_all_tables-v202011.xlsx.

written <- write_xlsx(data=biop_tables[["data"]][[1]],
                      excel=glue::glue("excel/biopsy_clinical_table-v{ver}.xlsx"))

## Saving to: excel/biopsy_clinical_table-v202011.xlsx

biop_sig <- extract_significant_genes(biop_tables, according_to="deseq")
##written <- write_xlsx(data=biop_sig[["deseq"]][["ups"]][[1]],
##                      excel=glue::glue("excel/biopsy_clinical_sigup-v{ver}.xlsx"))
written <- write_xlsx(data=biop_sig[["deseq"]][["downs"]][[1]],
                      excel=glue::glue("excel/biopsy_clinical_sigdown-v{ver}.xlsx"))

## Saving to: excel/biopsy_clinical_sigdown-v202011.xlsx

biop_pct_sig <- extract_significant_genes(biop_tables, n=200, lfc=NULL, p=NULL, according_to="deseq")

## Getting the top and bottom 200 genes.

written <- write_xlsx(data=biop_pct_sig[["deseq"]][["ups"]][[1]],
                      excel=glue::glue("excel/biopsy_clinical_sigup_pct-v{ver}.xlsx"))

## Saving to: excel/biopsy_clinical_sigup_pct-v202011.xlsx

written <- write_xlsx(data=biop_pct_sig[["deseq"]][["downs"]][[1]],
                      excel=glue::glue("excel/biopsy_clinical_sigdown_pct-v{ver}.xlsx"))

## Saving to: excel/biopsy_clinical_sigdown_pct-v202011.xlsx

biop_cpm <- sm(normalize_expt(biop, convert="cpm"))
written <- write_xlsx(data=exprs(biop_cpm),
                      excel=glue::glue("excel/biopsy_cpm_before_batch-v{ver}.xlsx"))

## Saving to: excel/biopsy_cpm_before_batch-v202011.xlsx

biop_bcpm <- sm(normalize_expt(biop, filter=TRUE, convert="cpm", batch="svaseq"))
written <- write_xlsx(data=exprs(biop_bcpm),
                      excel=glue::glue("excel/biopsy_cpm_after_batch-v{ver}.xlsx"))

## Saving to: excel/biopsy_cpm_after_batch-v202011.xlsx

5.6 Macrophages

These samples are rather different from all of the others. The following section is therefore written primarily in response to a separate set of emails from Olga and Maria Adelaida; here is a snippet:

Dear all, about the samples corresponding to infected macrophages with three sensitive (2.2) and three resistant (2.3) clinical strains of L. (V.) panamensis, I send you the results of parasite burden by detection of 7SLRNA. I think these results are interesting, but the sample size is very small. Doctor Najib or Trey could you please send me the quality data and PCA analysis of these samples?

and

Hi Doctor, thank you. These samples corresponding to primary macrophages infected with clinical strains 2.2 (n=3) and 2.3 (n = 3). These information is in the file: TMRC project 3: excel Host TMRC3 v1.1, rows 137 to 150.

Thus I added 3 columns to the end of the sample sheet which seek to include this information. The first is ‘drug’ and encodes both the infection state (no for the two controls and yes for everything else), the second is zymodeme which I took from the tmrc2 sample sheet, the last is drug, which is either no or sb.

macr <- subset_expt(hs_valid, subset="typeofcells=='Macrophages'")

## Using a subset expression.

## There were 64, now there are 12 samples.

macr <- set_expt_conditions(macr, fact="zymodeme")
macr <- set_expt_batches(macr, fact="macrdrug")

macr_norm <- normalize_expt(macr, norm="quant", convert="cpm", filter=TRUE)

## This function will replace the expt$expressionset slot with:

## cpm(quant(cbcb(data)))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Leaving the data in its current base format, keep in mind that
##  some metrics are easier to see when the data is log2 transformed, but
##  EdgeR/DESeq do not accept transformed data.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: performing count filter with option: cbcb

## Removing 9365 low-count genes (10389 remaining).

## Step 2: normalizing the data with quant.

## Step 3: converting the data with cpm.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: not transforming the data.

macr_norm <- normalize_expt(macr_norm, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

plt <- plot_pca(macr_norm, plot_labels=FALSE)$plot

## Not putting labels on the PC plot.

pp(file=glue("images/macrophage_side_experiment_norm_pca-v{ver}.pdf"), image=plt)

## Writing the image to: images/macrophage_side_experiment_norm_pca-v202011.pdf and calling dev.off().

plt

macr_nb <- normalize_expt(macr, batch="svaseq", filter=TRUE)

## This function will replace the expt$expressionset slot with:

## svaseq(cbcb(data))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Leaving the data in its current base format, keep in mind that
##  some metrics are easier to see when the data is log2 transformed, but
##  EdgeR/DESeq do not accept transformed data.

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Step 1: performing count filter with option: cbcb

## Removing 9365 low-count genes (10389 remaining).

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: svaseq.

## Step 4: doing batch correction with svaseq.

## Using the current state of normalization.

## Passing the data to all_adjusters using the svaseq estimate type.

## batch_counts: Before batch/surrogate estimation, 124526 entries are x>1: 100%.

## batch_counts: Before batch/surrogate estimation, 66 entries are x==0: 0%.

## The be method chose 2 surrogate variables.

## Attempting svaseq estimation with 2 surrogates.

## There are 20 (0%) elements which are < 0 after batch correction.

## Setting low elements to zero.

## Step 4: not transforming the data.

macr_nb <- normalize_expt(macr_nb, norm="quant", convert="cpm", transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(cpm(quant(data)))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: normalizing the data with quant.

## Step 3: converting the data with cpm.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

plt <- plot_pca(macr_nb)$plot
pp(file=glue("images/macrophage_side_experiment-v{ver}/normbatch_pca.pdf"), image=plt)

## Warning in cairo_pdf(filename = file, ...): cairo error 'error while writing to
## output stream'

## Error in cairo_pdf(filename = file, ...): unable to start device 'cairo_pdf'

plt

macr_written <- write_expt(macr, excel="excel/macrophage_side_experiment/macrophage_expt.xlsx")

## Writing the first sheet, containing a legend and some summary data.

## Writing the raw reads.

## Graphing the raw reads.

## `geom_smooth()` using formula 'y ~ x'
## `geom_smooth()` using formula 'y ~ x'

## Warning in doTryCatch(return(expr), name, parentenv, handler): display list
## redraw incomplete

## Warning in doTryCatch(return(expr), name, parentenv, handler): display list
## redraw incomplete

## Warning: ggrepel: 7 unlabeled data points (too many overlaps). Consider
## increasing max.overlaps

## Warning: ggrepel: 7 unlabeled data points (too many overlaps). Consider
## increasing max.overlaps

## Attempting mixed linear model with: ~  condition + batch

## Fitting the expressionset to the model, this is slow.

## 
## Total:105 s

## Placing factor: condition at the beginning of the model.

## Writing the normalized reads.

## Graphing the normalized reads.

## `geom_smooth()` using formula 'y ~ x'

## `geom_smooth()` using formula 'y ~ x'

## Warning in doTryCatch(return(expr), name, parentenv, handler): display list
## redraw incomplete

## Warning in doTryCatch(return(expr), name, parentenv, handler): display list
## redraw incomplete

## Warning: ggrepel: 8 unlabeled data points (too many overlaps). Consider
## increasing max.overlaps

## Warning: ggrepel: 8 unlabeled data points (too many overlaps). Consider
## increasing max.overlaps

## Attempting mixed linear model with: ~  condition + batch

## Fitting the expressionset to the model, this is slow.

## 
## Total:62 s

## Placing factor: condition at the beginning of the model.

## Writing the median reads by factor.

zymo_de <- all_pairwise(macr, model_batch="svaseq", filter=TRUE)

## batch_counts: Before batch/surrogate estimation, 124526 entries are x>1: 100%.

## batch_counts: Before batch/surrogate estimation, 66 entries are x==0: 0%.

## The be method chose 2 surrogate variables.

## Attempting svaseq estimation with 2 surrogates.

## Plotting a PCA before surrogate/batch inclusion.

## Not putting labels on the PC plot.

## Using svaseq to visualize before/after batch inclusion.

## Performing a test normalization with: raw

## This function will replace the expt$expressionset slot with:

## log2(svaseq(cpm(cbcb(data))))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Step 1: performing count filter with option: cbcb

## Removing 0 low-count genes (10389 remaining).

## Step 2: not normalizing the data.

## Step 3: converting the data with cpm.

## The method is: svaseq.

## Step 4: doing batch correction with svaseq.

## Using the current state of normalization.

## Passing the data to all_adjusters using the svaseq estimate type.

## batch_counts: Before batch/surrogate estimation, 121056 entries are x>1: 97%.

## batch_counts: Before batch/surrogate estimation, 66 entries are x==0: 0%.

## batch_counts: Before batch/surrogate estimation, 3546 entries are 0<x<1: 3%.

## The be method chose 2 surrogate variables.

## Attempting svaseq estimation with 2 surrogates.

## There are 175 (0%) elements which are < 0 after batch correction.

## Setting low elements to zero.

## Step 4: transforming the data with log2.

## transform_counts: Found 175 values equal to 0, adding 1 to the matrix.

## Not putting labels on the PC plot.

## Finished running DE analyses, collecting outputs.

## Comparing analyses.

zymo_table <- combine_de_tables(zymo_de, excel="images/macrophage_side_experiment/macrophage_de.xlsx")

## Writing a legend of columns.

## Printing a pca plot before/after surrogates/batch estimation.

## Working on table 1/3: z22_vs_undef

## Working on table 2/3: z23_vs_undef

## Working on table 3/3: z23_vs_z22

## Adding venn plots for z22_vs_undef.

## Limma expression coefficients for z22_vs_undef; R^2: 0.96; equation: y = 0.971x + 0.159

## Deseq expression coefficients for z22_vs_undef; R^2: 0.959; equation: y = 1.02x - 0.176

## Edger expression coefficients for z22_vs_undef; R^2: 0.959; equation: y = 1.02x - 0.153

## Adding venn plots for z23_vs_undef.

## Limma expression coefficients for z23_vs_undef; R^2: 0.955; equation: y = 0.976x + 0.157

## Deseq expression coefficients for z23_vs_undef; R^2: 0.952; equation: y = 0.971x + 0.32

## Edger expression coefficients for z23_vs_undef; R^2: 0.952; equation: y = 0.971x + 0.296

## Adding venn plots for z23_vs_z22.

## Limma expression coefficients for z23_vs_z22; R^2: 0.968; equation: y = 0.988x + 0.0835

## Deseq expression coefficients for z23_vs_z22; R^2: 0.965; equation: y = 0.943x + 0.632

## Edger expression coefficients for z23_vs_z22; R^2: 0.965; equation: y = 0.943x + 0.432

## Writing summary information, compare_plot is: TRUE.

## Performing save of images/macrophage_side_experiment/macrophage_de.xlsx.

5.6.1 Followup email from Olga

• Macrophages infected with 2.3 vs macrophagues infected with 2.3 + SbV
• Macrophages uninfected (M0) vs macrophages uninfected + SbV
• Macrophages uninfected vs macrophages infected with 2.3
• Macrophages uninfected + SbV vs Macrophagues infected with 2.3 + SbV

With this comparisons we can define the effect of infection with 2.3 and the drug effect with and without infection. In this moment we can’t evaluate the conditions with 2.2 because the samples are incomplete. Mariana will complete the samples in the next shipment.

new_factor <- paste0(pData(macr)[["macrdrug"]], "_", pData(macr)[["zymodeme"]])
new_factor <- gsub(x=new_factor, pattern="undef", replacement="uninf")
macr_v2 <- set_expt_conditions(macr, fact=new_factor)
macr_v2$conditions

##  [1] "no_uninf" "sb_uninf" "no_z2.3"  "sb_z2.3"  "no_z2.3"  "sb_z2.3" 
##  [7] "no_z2.2"  "sb_z2.2"  "sb_z2.2"  "no_z2.2"  "no_z2.3"  "sb_z2.3"

macr_v2_norm <- normalize_expt(macr_v2, onvert="cpm", filter=TRUE, norm="quant")

## This function will replace the expt$expressionset slot with:

## quant(cbcb(data))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Leaving the data in its current base format, keep in mind that
##  some metrics are easier to see when the data is log2 transformed, but
##  EdgeR/DESeq do not accept transformed data.

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: performing count filter with option: cbcb

## Removing 9365 low-count genes (10389 remaining).

## Step 2: normalizing the data with quant.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: not transforming the data.

macr_v2_norm <- normalize_expt(macr_v2_norm, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

plot_pca(macr_v2_norm)$plot

keepers <- list(
  "zymo_sbv" = c("sb_z22", "sb_z23"),
  "uninf_drug" = c("no_uninf", "sb_uninf"),
  "uninfnodrug_z23" = c("no_uninf", "no_z23"),
  "uninfdrug_z23" = c("sb_uninf", "sb_z23"),
  "z23_drug_nodrug" = c("sb_z23", "no_z23"),
  "z22_drug_nodrug" = c("sb_z22", "no_z22")
)
olga_pairwise <- all_pairwise(macr_v2, do_deseq=FALSE, do_edger=FALSE, do_ebseq=FALSE)

## Plotting a PCA before surrogate/batch inclusion.

## Not putting labels on the PC plot.

## Using limma's removeBatchEffect to visualize with(out) batch inclusion.

## Not putting labels on the PC plot.

## Finished running DE analyses, collecting outputs.

## Comparing analyses.

olga_zymo_sb <- olga_pairwise[["limma"]][["all_tables"]][["sb_z23_vs_sb_z22"]]
olga_zymo_sb <- merge(olga_zymo_sb, fData(macr_v2), by="row.names")
rownames(olga_zymo_sb) <- olga_zymo_sb[["Row.names"]]
olga_zymo_sb[["Row.names"]] <- NULL
olga_zymo_sb[["transcript"]] <- NULL
written <- write_xlsx(data=olga_zymo_sb,
                      excel=glue::glue("excel/sb_23_vs22-v{ver}.xlsx"))

## Saving to: excel/sb_23_vs22-v202011.xlsx

olga_uninf_drug <- olga_pairwise[["limma"]][["all_tables"]][["sb_uninf_vs_no_uninf"]]
olga_uninf_drug <- merge(olga_uninf_drug, fData(macr_v2), by="row.names")
rownames(olga_uninf_drug) <- olga_uninf_drug[["Row.names"]]
olga_uninf_drug[["Row.names"]] <- NULL
written <- write_xlsx(data=olga_uninf_drug,
                      excel=glue::glue("excel/uninf_sb_vs_nosb-v{ver}.xlsx"))

## Saving to: excel/uninf_sb_vs_nosb-v202011.xlsx

olga_uninfnodrug_z23 <- olga_pairwise[["limma"]][["all_tables"]][["no_z23_vs_no_uninf"]]
olga_uninfnodrug_z23 <- merge(olga_uninfnodrug_z23, fData(macr_v2), by="row.names")
rownames(olga_uninfnodrug_z23) <- olga_uninfnodrug_z23[["Row.names"]]
olga_uninfnodrug_z23[["Row.names"]] <- NULL
written <- write_xlsx(data=olga_uninfnodrug_z23,
                      excel=glue::glue("excel/no_z23_vs_nosb_uninf-v{ver}.xlsx"))

## Saving to: excel/no_z23_vs_nosb_uninf-v202011.xlsx

olga_uninfdrug_z23 <- olga_pairwise[["limma"]][["all_tables"]][["sb_z23_vs_sb_uninf"]]
olga_uninfdrug_z23 <- merge(olga_uninfdrug_z23, fData(macr_v2), by="row.names")
rownames(olga_uninfdrug_z23) <- olga_uninfdrug_z23[["Row.names"]]
olga_uninfdrug_z23[["Row.names"]] <- NULL
written <- write_xlsx(data=olga_uninfdrug_z23,
                      excel=glue::glue("excel/sb_z23_vs_sb_uninf-v{ver}.xlsx"))

## Saving to: excel/sb_z23_vs_sb_uninf-v202011.xlsx

olga_z23_drugnodrug <- olga_pairwise[["limma"]][["all_tables"]][["sb_z23_vs_no_z23"]]
olga_z23_drugnodrug <- merge(olga_z23_drugnodrug, fData(macr_v2), by="row.names")
rownames(olga_z23_drugnodrug) <- olga_z23_drugnodrug[["Row.names"]]
olga_z23_drugnodrug[["Row.names"]] <- NULL
olga_z23_drugnodrug[["transcript"]] <- NULL
written <- write_xlsx(data=olga_z23_drugnodrug,
                      excel=glue::glue("excel/z23_drug_vs_nodrug-v{ver}.xlsx"))

## Saving to: excel/z23_drug_vs_nodrug-v202011.xlsx

olga_z22_drugnodrug <- olga_pairwise[["limma"]][["all_tables"]][["sb_z22_vs_no_z22"]]
olga_z22_drugnodrug <- merge(olga_z22_drugnodrug, fData(macr_v2), by="row.names")
rownames(olga_z22_drugnodrug) <- olga_z22_drugnodrug[["Row.names"]]
olga_z22_drugnodrug[["Row.names"]] <- NULL
olga_z22_drugnodrug[["transcript"]] <- NULL
written <- write_xlsx(data=olga_z22_drugnodrug,
                      excel=glue::glue("excel/z22_drug_vs_nodrug-v{ver}.xlsx"))

## Saving to: excel/z22_drug_vs_nodrug-v202011.xlsx

5.7 Donor effect

donor <- set_expt_conditions(hs_expt, fact="donor")
donor <- set_expt_batches(donor, fact="time")
save_result <- save(donor, file="rda/donor_expt.rda")
test <- normalize_expt(donor, filter=TRUE, transform="log2", convert="cpm", norm="quant")

## This function will replace the expt$expressionset slot with:

## log2(cpm(quant(cbcb(data))))

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: performing count filter with option: cbcb

## Removing 4756 low-count genes (14998 remaining).

## Step 2: normalizing the data with quant.

## Step 3: converting the data with cpm.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

## transform_counts: Found 4080 values equal to 0, adding 1 to the matrix.

plot_pca(test)$plot

## Warning in MASS::cov.trob(data[, vars]): Probable convergence failure

## Warning in MASS::cov.trob(data[, vars]): Probable convergence failure

## Warning: ggrepel: 70 unlabeled data points (too many overlaps). Consider
## increasing max.overlaps

5.8 Question 1

I interpreted question 1 as: pull out tmrc3000[1-6] and look at them.

I am not entirely certain what is meant by the heirarchical clustering request. I can see a couple of possibilities for what this means. The most similar thing I recall in the cruzi context was a post-DE visualization of some fairly specific genes.

hs_q1 <- subset_expt(hs_valid, subset="donor=='d1010'|donor=='d1011'")

## Using a subset expression.

## There were 64, now there are 6 samples.

q1_norm <- sm(normalize_expt(hs_q1, norm="quant", transform="log2", convert="cpm", batch=FALSE,
                           filter=TRUE))
q1_pca <- plot_pca(q1_norm)
q1_pca$plot

knitr::kable(q1_pca$table)

	sampleid	condition	batch	batch_int	colors	labels	PC1	PC2	pc_1	pc_2	pc_3	pc_4	pc_5
1010-2	1010-2	PBMC	d1010	1	#1B9E77	1010-2	-0.5597	0.2860	-0.5597	0.2860	0.4198	-0.1032	0.5014
1010-7	1010-7	PBMC	d1010	1	#1B9E77	1010-7	0.0137	0.4819	0.0137	0.4819	-0.6300	0.4508	0.0283
1010-12	1010-12	PBMC	d1010	1	#1B9E77	1010-12	0.4174	0.4406	0.4174	0.4406	0.3520	-0.3314	-0.4809
1011-2	1011-2	PBMC	d1011	2	#1B9E77	1011-2	-0.4953	-0.4061	-0.4953	-0.4061	-0.3185	-0.3369	-0.4563
1011-7	1011-7	PBMC	d1011	2	#1B9E77	1011-7	0.1215	-0.4505	0.1215	-0.4505	0.3933	0.6660	-0.1319
1011-12	1011-12	PBMC	d1011	2	#1B9E77	1011-12	0.5023	-0.3520	0.5023	-0.3520	-0.2166	-0.3452	0.5394

write.csv(q1_pca$table, file="coords/q1_pca_coords.csv")

## Looks like PC1 == Time and PC2 is donor, and they have pretty much the same amount of variance

hs_time <- set_expt_conditions(hs_q1, fact="time")
time_norm <- sm(normalize_expt(hs_time, batch="svaseq", filter=TRUE))
time_norm <- normalize_expt(time_norm, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

## transform_counts: Found 4 values equal to 0, adding 1 to the matrix.

## Get a set of genes with high PC loads for PC1(time), and PC2(donor):
high_scores <- pca_highscores(time_norm, n=40)

time_stuff <- high_scores$highest[, c(1, 2)]
time_stuff

##       Comp.1                  Comp.2                 
##  [1,] "12.67:ENSG00000073756" "10.66:ENSG00000129824"
##  [2,] "10.5:ENSG00000169245"  "9.97:ENSG00000243716" 
##  [3,] "10.33:ENSG00000125538" "9.643:ENSG00000012817"
##  [4,] "9.445:ENSG00000093134" "8.492:ENSG00000114374"
##  [5,] "8.952:ENSG00000112299" "8.434:ENSG00000206503"
##  [6,] "8.307:ENSG00000140092" "8.396:ENSG00000067048"
##  [7,] "8.252:ENSG00000174946" "7.703:ENSG00000167874"
##  [8,] "7.877:ENSG00000168329" "7.393:ENSG00000067646"
##  [9,] "7.569:ENSG00000126243" "7.182:ENSG00000125538"
## [10,] "7.383:ENSG00000171049" "6.958:ENSG00000196735"
## [11,] "7.347:ENSG00000170915" "6.819:ENSG00000169245"
## [12,] "7.216:ENSG00000187474" "6.808:ENSG00000114315"
## [13,] "7.16:ENSG00000173114"  "6.751:ENSG00000196436"
## [14,] "6.877:ENSG00000086570" "6.741:ENSG00000080007"
## [15,] "6.839:ENSG00000143365" "6.587:ENSG00000100298"
## [16,] "6.722:ENSG00000125703" "6.584:ENSG00000011426"
## [17,] "6.662:ENSG00000123700" "6.578:ENSG00000204642"
## [18,] "6.6:ENSG00000263002"   "6.523:ENSG00000104951"
## [19,] "6.571:ENSG00000162739" "6.451:ENSG00000241945"
## [20,] "6.246:ENSG00000120526" "6.426:ENSG00000128641"
## [21,] "6.226:ENSG00000168310" "6.353:ENSG00000169071"
## [22,] "6.149:ENSG00000164484" "6.351:ENSG00000179344"
## [23,] "6.146:ENSG00000091106" "6.339:ENSG00000183878"
## [24,] "5.928:ENSG00000171115" "6.29:ENSG00000185499" 
## [25,] "5.864:ENSG00000120262" "6.172:ENSG00000197272"
## [26,] "5.86:ENSG00000133574"  "6.101:ENSG00000164877"
## [27,] "5.831:ENSG00000081041" "6.071:ENSG00000234745"
## [28,] "5.819:ENSG00000056050" "6.013:ENSG00000154736"
## [29,] "5.783:ENSG00000213203" "5.754:ENSG00000243708"
## [30,] "5.712:ENSG00000250510" "5.637:ENSG00000112139"
## [31,] "5.679:ENSG00000143167" "5.48:ENSG00000051180" 
## [32,] "5.665:ENSG00000050405" "5.44:ENSG00000111671" 
## [33,] "5.629:ENSG00000119471" "5.423:ENSG00000198692"
## [34,] "5.62:ENSG00000165118"  "5.404:ENSG00000166793"
## [35,] "5.618:ENSG00000164011" "5.39:ENSG00000108556" 
## [36,] "5.604:ENSG00000163563" "5.333:ENSG00000223609"
## [37,] "5.582:ENSG00000197044" "5.321:ENSG00000121931"
## [38,] "5.582:ENSG00000131355" "5.277:ENSG00000206077"
## [39,] "5.576:ENSG00000180626" "5.246:ENSG00000106341"
## [40,] "5.568:ENSG00000173409" "5.235:ENSG00000143333"

## Column 1 should be winners vs. time and column 2 by donor.
time_genes <- gsub(x=time_stuff[, "Comp.1"], pattern="^.*:(.*)$", replacement="\\1")
head(time_genes)

## [1] "ENSG00000073756" "ENSG00000169245" "ENSG00000125538" "ENSG00000093134"
## [5] "ENSG00000112299" "ENSG00000140092"

time_subset <- exprs(time_norm)[time_genes, ]
plot_sample_heatmap(time_norm, row_label=rownames(time_norm))

hs_donor <- set_expt_conditions(hs_q1, fact="donor")
donor_norm <- sm(normalize_expt(hs_donor, convert="cpm", batch="svaseq", filter=TRUE))
donor_norm <- normalize_expt(donor_norm, transform="log2")

## This function will replace the expt$expressionset slot with:

## log2(data)

## It will save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep libsizes in mind
##  when invoking limma.  The appropriate libsize is non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize

## Filter is false, this should likely be set to something, good
##  choices include cbcb, kofa, pofa (anything but FALSE).  If you want this to
##  stay FALSE, keep in mind that if other normalizations are performed, then the
##  resulting libsizes are likely to be strange (potentially negative!)

## Leaving the data unconverted.  It is often advisable to cpm/rpkm
##  the data to normalize for sampling differences, keep in mind though that rpkm
##  has some annoying biases, and voom() by default does a cpm (though hpgl_voom()
##  will try to detect this).

## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.

## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.

## Step 1: not doing count filtering.

## Step 2: not normalizing the data.

## Step 3: not converting the data.

## The method is: raw.

## Step 4: not doing batch correction.

## Step 4: transforming the data with log2.

## transform_counts: Found 28 values equal to 0, adding 1 to the matrix.

## Get a set of genes with high PC loads for PC1(donor), and PC2(donor):
high_scores <- pca_highscores(donor_norm, n=40)

donor_stuff <- high_scores$highest[, c(1, 2)]
donor_stuff

##       Comp.1                  Comp.2                 
##  [1,] "40.57:ENSG00000129824" "4.176:ENSG00000159842"
##  [2,] "38.82:ENSG00000067048" "4.164:ENSG00000206077"
##  [3,] "37.62:ENSG00000012817" "4.016:ENSG00000108556"
##  [4,] "33.33:ENSG00000179344" "3.888:ENSG00000037280"
##  [5,] "32.49:ENSG00000114374" "3.829:ENSG00000144152"
##  [6,] "32.14:ENSG00000206503" "3.773:ENSG00000112212"
##  [7,] "30.9:ENSG00000196735"  "3.696:ENSG00000105856"
##  [8,] "28.32:ENSG00000183878" "3.599:ENSG00000145850"
##  [9,] "28.16:ENSG00000234745" "3.579:ENSG00000143786"
## [10,] "27.56:ENSG00000099725" "3.422:ENSG00000113088"
## [11,] "25.15:ENSG00000067646" "3.349:ENSG00000213638"
## [12,] "21.36:ENSG00000198692" "3.342:ENSG00000243716"
## [13,] "19.44:ENSG00000204642" "3.305:ENSG00000214756"
## [14,] "16.14:ENSG00000112139" "3.207:ENSG00000268041"
## [15,] "13.69:ENSG00000049247" "3.179:ENSG00000168000"
## [16,] "13.12:ENSG00000115602" "3.174:ENSG00000258315"
## [17,] "12.75:ENSG00000131042" "3.138:ENSG00000167874"
## [18,] "12.45:ENSG00000129422" "3.104:ENSG00000099814"
## [19,] "12.35:ENSG00000127022" "3.103:ENSG00000172602"
## [20,] "11.3:ENSG00000144115"  "3.082:ENSG00000162545"
## [21,] "11.3:ENSG00000196436"  "3.079:ENSG00000139517"
## [22,] "10.98:ENSG00000154620" "3.026:ENSG00000095397"
## [23,] "10.85:ENSG00000196126" "3.022:ENSG00000062524"
## [24,] "10.47:ENSG00000241945" "3.015:ENSG00000149257"
## [25,] "9.861:ENSG00000109321" "2.999:ENSG00000154620"
## [26,] "9.415:ENSG00000182534" "2.993:ENSG00000254413"
## [27,] "9.188:ENSG00000204001" "2.992:ENSG00000196169"
## [28,] "9.093:ENSG00000160201" "2.991:ENSG00000159640"
## [29,] "9.031:ENSG00000248905" "2.983:ENSG00000099725"
## [30,] "8.879:ENSG00000144290" "2.971:ENSG00000186827"
## [31,] "8.832:ENSG00000137959" "2.967:ENSG00000171219"
## [32,] "8.644:ENSG00000169507" "2.954:ENSG00000167191"
## [33,] "8.589:ENSG00000243716" "2.948:ENSG00000198832"
## [34,] "8.418:ENSG00000168765" "2.946:ENSG00000187862"
## [35,] "8.267:ENSG00000080007" "2.944:ENSG00000188001"
## [36,] "7.976:ENSG00000160307" "2.92:ENSG00000232434" 
## [37,] "7.945:ENSG00000189283" "2.911:ENSG00000176845"
## [38,] "7.91:ENSG00000142156"  "2.908:ENSG00000106333"
## [39,] "7.886:ENSG00000088827" "2.897:ENSG00000153563"
## [40,] "7.695:ENSG00000186193" "2.891:ENSG00000061656"

## Column 1 should be winners vs. donor and column 2 by donor.
donor_genes <- gsub(x=donor_stuff[, "Comp.1"], pattern="^.*:(.*)$", replacement="\\1")
head(donor_genes)

## [1] "ENSG00000129824" "ENSG00000067048" "ENSG00000012817" "ENSG00000179344"
## [5] "ENSG00000114374" "ENSG00000206503"

donor_subset <- exprs(donor_norm)[donor_genes, ]
plot_sample_heatmap(donor_norm, row_label=rownames(donor_norm))

time_keepers <- list(
  "2hr_vs_7hr" = c("t2hr", "t7hr"),
  "2hr_vs_12hr" = c("t2hr", "t12hr"),
  "7hr_vs_12hr" = c("t7hr", "t12hr"))

q1_time <- set_expt_conditions(hs_q1, fact="time")
time_de <- sm(all_pairwise(q1_time, model_batch=FALSE, parallel=FALSE))

time_all_tables <- sm(combine_de_tables(time_de,
                                        excel=glue::glue("excel/time_de_tables-v{ver}.xlsx")))

save_result <- save(time_all_tables, file="rda/time_all_tables.rda")

time_all_tables_all <- sm(combine_de_tables(
  time_de,
  keepers="all",
  excel=glue::glue("excel/time_de_all_tables-v{ver}.xlsx")))

time_all_tables <- sm(combine_de_tables(
  time_de,
  keepers=time_keepers,
  excel=glue::glue("excel/{rundate}-time_de_tables-v{ver}.xlsx")))

q1_donor <- set_expt_conditions(hs_q1, fact="donor")
donor_de <- sm(all_pairwise(q1_donor, model_batch=FALSE))
donor_tables <- sm(combine_de_tables(
                     donor_de, excel=glue::glue("excel/donor_de_tables-v{ver}.xlsx")))

save_result <- save(donor_tables, file="rda/donor_tables.rda")

hs_q2 <- subset_expt(hs_valid, subset="donor!='d1010'&donor!='d1011'")

## Using a subset expression.

## There were 64, now there are 58 samples.

q2_norm <- sm(normalize_expt(hs_q2, transform="log2", convert="cpm", norm="quant", filter=TRUE))

q2_pca <- plot_pca(q2_norm)
knitr::kable(q2_pca$table)

	sampleid	condition	batch	batch_int	colors	labels	PC1	PC2	pc_1	pc_2	pc_3	pc_4	pc_5	pc_6	pc_7	pc_8	pc_9	pc_10	pc_11	pc_12	pc_13	pc_14	pc_15	pc_16	pc_17	pc_18	pc_19	pc_20	pc_21	pc_22	pc_23	pc_24	pc_25	pc_26	pc_27	pc_28	pc_29	pc_30	pc_31	pc_32	pc_33	pc_34	pc_35	pc_36	pc_37	pc_38	pc_39	pc_40	pc_41	pc_42	pc_43	pc_44	pc_45	pc_46	pc_47	pc_48	pc_49	pc_50	pc_51	pc_52	pc_53	pc_54	pc_55	pc_56	pc_57
1034n1	1034n1	Neutrophil	d1034	1	#D95F02	1034n1	-0.1732	-0.0652	-0.1732	-0.0652	-0.1739	-0.0278	-0.1152	-0.0301	0.3016	-0.1039	0.0725	-0.0426	0.0010	-0.0902	0.0592	-0.2204	0.2706	-0.0473	0.0435	-0.1543	0.0304	-0.0518	0.1216	-0.0918	0.1039	0.1429	-0.3789	0.1433	0.1095	0.4072	-0.3134	-0.1268	-0.0691	-0.2514	0.0893	0.0546	0.0820	-0.0505	-0.1490	-0.0167	-0.0018	0.0013	0.0360	0.0609	0.0135	0.0380	-0.0341	-0.0107	-0.0032	-0.0009	-0.0023	-0.0005	-0.0061	-0.0015	-0.0019	-0.0129	-0.0038	-0.0013	-0.0035
1034bp1	1034bp1	Biopsy	d1034	1	#E7298A	1034bp1	0.1762	-0.1888	0.1762	-0.1888	-0.0237	0.0923	0.0544	0.2045	0.1656	0.0443	-0.0950	0.1514	0.0101	-0.0174	0.0537	0.0705	0.0156	0.0877	-0.1220	-0.0184	-0.0904	0.1402	0.0894	-0.0356	-0.0363	0.0116	0.1017	0.0136	0.2713	-0.3070	-0.2344	-0.1409	0.5649	-0.2757	0.0181	0.1077	0.1672	0.0282	0.1369	-0.1128	0.0022	0.0826	0.0070	0.0102	0.0115	-0.0100	0.0244	-0.0216	0.0336	-0.0192	0.0100	-0.0148	0.0079	-0.0049	-0.0103	0.0036	-0.0125	0.0040	0.0078
1034n2	1034n2	Neutrophil	d1034	1	#D95F02	1034n2	-0.1732	-0.0685	-0.1732	-0.0685	-0.2007	-0.0245	-0.1192	0.0139	0.3133	-0.1842	0.0757	-0.0046	0.0074	-0.0383	0.0289	-0.3516	0.2551	-0.2494	0.0248	-0.1618	-0.0104	0.0983	-0.1980	-0.0745	0.0042	-0.0901	0.2311	0.0598	0.0529	-0.2845	0.4046	-0.1543	0.0201	0.2216	-0.1433	0.0372	-0.0302	0.0686	0.0313	0.0255	0.0019	0.0445	0.0334	0.0202	-0.0252	-0.0065	0.0239	0.0184	0.0085	-0.0096	0.0073	-0.0034	-0.0009	-0.0041	-0.0003	0.0036	-0.0038	0.0007	-0.0008
1034m2	1034m2	Monocyte	d1034	1	#7570B3	1034m2	-0.0026	-0.0366	-0.0026	-0.0366	0.0441	-0.2625	0.0345	-0.1574	0.3522	0.0227	0.1218	0.0299	-0.0334	0.0069	0.0977	0.2175	0.1543	0.1613	-0.1777	0.1187	0.1068	-0.0182	0.0026	0.0956	0.0062	-0.0235	0.0678	-0.1457	-0.0447	-0.0719	-0.0973	0.1860	-0.0625	0.0892	-0.0301	0.0015	0.0090	-0.0322	-0.0595	0.0061	-0.0307	-0.0548	-0.0146	0.0376	0.0280	-0.0501	0.0526	0.3455	0.4693	0.2443	-0.1891	0.1586	-0.0377	0.0226	0.0252	0.0139	0.0118	-0.0047	0.0054
1034m2-	1034m2-	Monocyte	d1034	1	#7570B3	1034m2-	-0.0087	-0.0382	-0.0087	-0.0382	0.0512	-0.2531	0.0278	-0.1560	0.3592	0.0446	0.1269	0.0225	-0.1167	0.0546	0.0606	0.1766	0.1334	0.1533	-0.1852	0.1693	0.1120	-0.0542	0.0080	0.0241	0.0278	-0.0524	0.0950	-0.1207	-0.0364	-0.0170	-0.0096	0.1541	-0.0555	0.0287	0.0524	-0.0063	-0.0108	-0.0109	0.0685	-0.0418	0.0310	-0.0374	-0.0131	-0.0432	-0.0642	-0.0354	-0.0716	-0.3326	-0.5125	-0.2223	0.1560	-0.2012	0.0279	-0.0204	-0.0023	-0.0130	0.0017	-0.0138	-0.0073
2050bp1	2050bp1	Biopsy	d2050	2	#E7298A	2050bp1	0.1642	-0.1684	0.1642	-0.1684	-0.0071	0.0628	0.0352	0.0689	-0.1245	-0.3370	0.2811	-0.3254	-0.2397	0.1066	-0.1360	0.3616	0.1130	-0.2199	0.0587	-0.1000	0.0364	0.0699	-0.0581	-0.0689	-0.0748	-0.1800	-0.1509	-0.1414	-0.0269	-0.0859	-0.0986	-0.1156	-0.0400	-0.0942	0.0173	0.2229	-0.3200	0.0218	0.0601	0.0503	0.0117	-0.0404	-0.0516	0.0441	-0.0107	0.0704	0.0228	-0.0147	0.0195	-0.0023	-0.0008	-0.0019	0.0134	-0.0195	0.0232	-0.0363	-0.0050	-0.0031	0.0067
2052bp1	2052bp1	Biopsy	d2052	3	#E7298A	2052bp1	0.1733	-0.1944	0.1733	-0.1944	-0.0204	0.1107	0.0626	0.2373	0.1161	0.0895	-0.1535	0.2038	0.0128	0.0056	0.0178	0.1007	-0.0271	-0.0242	-0.0335	-0.0968	-0.1946	0.1301	0.0140	-0.0178	0.2467	-0.3689	-0.1855	0.0261	-0.2049	-0.1566	0.0063	-0.0670	-0.5152	-0.0793	-0.0546	-0.1367	0.2697	-0.0480	0.0254	-0.0429	0.0219	-0.0220	0.0114	0.0141	0.0027	-0.0347	0.0155	-0.0097	-0.0059	-0.0085	-0.0111	-0.0099	-0.0158	-0.0034	-0.0039	-0.0078	0.0074	0.0093	-0.0001
2052e1	2052e1	Eosinophil	d2052	3	#66A61E	2052e1	-0.0956	0.1103	-0.0956	0.1103	0.1745	0.2235	0.1420	-0.0572	-0.0363	0.0127	0.0271	0.0100	0.3013	-0.1252	0.1920	0.1848	0.1446	-0.2729	0.0388	0.1760	0.0726	0.2438	-0.1947	-0.2125	0.2942	0.2643	-0.2621	0.0166	-0.0244	-0.1357	0.0671	0.3190	0.0911	-0.0165	-0.0525	0.0049	-0.0106	0.0980	-0.0218	-0.0836	0.0751	-0.0236	-0.0245	-0.0162	-0.0808	0.0044	-0.0301	-0.0325	-0.0416	0.0180	-0.0081	0.0575	0.0227	0.0068	0.0111	0.0198	-0.0170	-0.0245	-0.0061
2052n2	2052n2	Neutrophil	d2052	3	#D95F02	2052n2	-0.1354	0.0861	-0.1354	0.0861	-0.2232	0.1496	0.1831	-0.1019	-0.0400	0.1529	-0.0883	-0.0630	0.0647	0.6311	0.3107	0.0527	-0.0694	-0.3422	-0.3212	-0.0395	-0.0090	-0.1798	0.1552	0.0247	-0.0411	-0.0114	0.0220	-0.0141	0.0366	0.0228	-0.0196	-0.0873	-0.0006	0.0266	-0.0083	0.0046	0.0174	-0.0028	-0.0451	0.0400	-0.0057	-0.0020	0.0085	-0.0024	0.0244	-0.0183	-0.0160	0.0065	0.0104	-0.0010	-0.0037	-0.0044	-0.0057	-0.0003	0.0044	0.0035	0.0046	0.0002	0.0029
2065bp1	2065bp1	Biopsy	d2065	4	#E7298A	2065bp1	0.1781	-0.1922	0.1781	-0.1922	-0.0426	0.0959	0.0790	0.3802	0.1598	0.1509	-0.1483	0.0717	-0.0232	0.0271	-0.0006	0.0476	-0.0438	-0.1456	0.3489	0.0064	0.5252	-0.3238	-0.0814	0.1286	0.1151	-0.0074	0.0714	0.0396	-0.0119	0.1699	0.0174	0.0991	0.1068	0.1991	0.0460	0.0565	-0.0652	-0.0248	-0.0469	-0.0041	0.0039	-0.0101	0.0104	-0.0202	-0.0269	0.0075	0.0053	0.0083	0.0021	0.0163	0.0108	0.0027	0.0004	0.0031	0.0072	-0.0032	-0.0081	-0.0011	-0.0022
2066bp1	2066bp1	Biopsy	d2066	5	#E7298A	2066bp1	0.1731	-0.1954	0.1731	-0.1954	-0.0385	0.1033	0.0456	0.0986	0.0949	0.0993	-0.1617	0.1681	0.0758	-0.0617	0.0649	-0.0497	0.0004	0.0730	-0.1384	0.0370	-0.1938	0.0607	-0.1091	-0.2162	-0.4334	0.3161	-0.1170	-0.2396	0.0042	0.1334	0.1984	-0.0075	-0.1706	0.1325	0.3312	0.2016	-0.1816	-0.0801	0.0577	0.1159	0.0033	-0.0219	-0.0321	-0.0077	0.0285	0.0492	-0.0020	0.0030	0.0040	0.0084	-0.0018	0.0291	-0.0115	-0.0097	-0.0016	0.0027	0.0081	-0.0106	-0.0016
2068m1	2068m1	Monocyte	d2068	6	#7570B3	2068m1	0.0018	-0.0496	0.0018	-0.0496	0.0854	-0.2698	0.1071	0.1350	-0.2366	-0.0503	-0.1160	-0.0342	0.0627	0.1020	0.0342	-0.2312	0.2202	-0.0199	0.1336	0.0369	0.0894	0.1987	0.2635	-0.0212	-0.0605	-0.0644	-0.1086	-0.2195	0.1468	-0.1908	-0.0460	0.1622	0.0294	0.0653	-0.0105	-0.1889	-0.0560	-0.3804	-0.2167	0.1319	-0.2559	-0.0405	0.1106	0.0391	0.0602	-0.0308	0.0936	-0.0619	-0.0371	-0.0511	0.0781	0.0433	0.0373	0.1367	-0.0221	-0.0664	-0.0233	0.0026	0.0117
2068n1	2068n1	Neutrophil	d2068	6	#D95F02	2068n1	-0.1653	-0.0725	-0.1653	-0.0725	-0.1361	-0.0365	-0.0734	0.1103	-0.2210	0.0024	-0.0494	-0.0091	0.0160	-0.1549	-0.0471	0.0903	0.1200	0.1056	-0.1939	0.0812	0.1610	-0.0418	0.2283	-0.1532	-0.0802	0.0066	-0.0946	0.3095	0.0890	-0.0825	-0.1090	-0.0199	-0.1200	0.3104	-0.1355	0.1308	0.0143	0.1364	0.2113	-0.0536	-0.0917	-0.1048	-0.1039	-0.4658	0.1729	-0.0661	0.0148	-0.1128	0.0681	0.0070	-0.0470	-0.0119	0.0000	0.0364	0.0066	-0.0101	0.0013	-0.0076	-0.0065
2068e1	2068e1	Eosinophil	d2068	6	#66A61E	2068e1	-0.1251	-0.0549	-0.1251	-0.0549	0.2402	0.0559	-0.1248	0.0511	-0.1172	0.0485	-0.0999	-0.0498	-0.0627	0.0772	-0.0572	-0.1008	0.1229	0.1167	-0.1589	-0.1393	0.2578	0.0939	0.1064	-0.0965	-0.0210	0.0141	0.1325	0.0215	-0.1875	-0.1197	-0.0241	-0.0813	-0.0566	-0.0565	0.2807	-0.1394	-0.0942	0.0997	-0.2613	-0.1587	0.2853	0.2331	-0.2966	-0.0053	-0.1607	-0.0300	-0.1660	0.0079	0.1553	-0.0273	0.1881	0.0170	0.1508	-0.0940	-0.0172	0.0336	0.0031	0.0043	0.0069
2068bp1	2068bp1	Biopsy	d2068	6	#E7298A	2068bp1	0.1695	-0.1829	0.1695	-0.1829	-0.0222	0.0745	-0.0026	-0.2783	-0.1798	-0.2938	0.0341	-0.0065	-0.0384	0.0316	-0.0356	-0.0173	-0.0044	-0.0174	-0.0370	0.0194	-0.0053	-0.0119	-0.1250	-0.1874	0.0764	-0.1439	0.0078	-0.1895	0.0352	0.2836	-0.0040	-0.0009	0.2023	0.3741	0.1755	-0.1713	0.4681	-0.0495	0.0101	-0.1988	0.0218	0.0811	-0.0070	-0.0432	0.0174	-0.0247	0.0371	-0.0069	0.0201	0.0199	0.0006	0.0019	-0.0195	-0.0076	-0.0260	0.0093	-0.0170	0.0008	-0.0166
2072n1	2072n1	Neutrophil	d2072	8	#D95F02	2072n1	-0.1727	-0.0670	-0.1727	-0.0670	-0.1501	-0.0338	-0.1051	-0.0662	-0.1421	0.1806	-0.0384	0.0312	-0.0411	-0.2156	-0.0213	0.2129	0.0458	-0.0083	0.0928	0.0129	-0.0253	-0.1262	0.3460	-0.1152	0.0575	0.0717	-0.0100	-0.1460	-0.1327	-0.0975	0.0434	-0.1545	0.0576	0.1771	0.0274	-0.0509	-0.0080	0.1686	0.1012	0.0041	0.0632	-0.0006	0.3879	0.4853	-0.0539	0.0397	-0.1682	-0.1066	0.0337	0.0601	-0.0016	0.0287	-0.0046	-0.0144	0.0117	0.0041	0.0047	-0.0061	0.0078
2072e1	2072e1	Eosinophil	d2072	8	#66A61E	2072e1	-0.1071	-0.0611	-0.1071	-0.0611	0.2041	0.0342	-0.1324	-0.0882	-0.0707	0.2913	-0.1837	0.0596	-0.6336	0.1340	-0.2685	-0.1781	0.0791	-0.1591	-0.0757	0.1535	-0.0936	0.0053	-0.2120	-0.0253	0.1297	0.1138	-0.1209	0.0301	0.0157	-0.0174	-0.0220	0.1133	0.0638	-0.0170	-0.0908	0.0849	0.0573	-0.0204	0.0817	0.0605	-0.0566	-0.1082	-0.0110	0.0703	0.1030	0.0062	0.0189	0.0447	0.0559	-0.0097	-0.0071	-0.0664	-0.0013	0.0006	0.0013	0.0037	-0.0023	0.0000	-0.0110
2072bp1	2072bp1	Biopsy	d2072	8	#E7298A	2072bp1	0.1683	-0.1774	0.1683	-0.1774	-0.0366	0.0650	-0.0178	-0.3460	-0.1186	-0.1229	0.0744	-0.0689	0.0035	0.0033	0.0168	-0.0590	-0.0415	-0.0122	0.1358	0.0203	0.1688	-0.1203	0.1390	0.2208	-0.0205	0.2783	0.0650	0.2784	-0.0039	-0.3052	0.1707	0.0945	-0.2099	-0.2417	0.2449	0.1897	0.2728	-0.1774	0.1116	0.0515	0.0040	-0.0300	0.0608	0.0280	-0.0016	-0.0654	0.0219	0.0154	0.0163	-0.0215	0.0172	0.0033	-0.0095	0.0062	0.0073	0.0113	0.0148	-0.0077	0.0126
2071bp1	2071bp1	Biopsy	d2071	7	#E7298A	2071bp1	0.1588	-0.1814	0.1588	-0.1814	-0.0312	0.0722	-0.0236	-0.4326	-0.0847	-0.0890	-0.2399	0.2592	0.1192	-0.0666	0.0751	-0.1642	-0.0792	0.0914	-0.0361	-0.0274	-0.0399	0.0214	0.0313	0.0626	0.3171	-0.2412	0.0552	0.0628	0.0465	0.0342	-0.1553	0.0304	0.0477	0.0627	-0.0357	0.1870	-0.4883	0.0799	-0.0789	0.1494	-0.0004	-0.0204	-0.0263	0.0178	0.0171	-0.0141	-0.0452	0.0133	-0.0311	-0.0154	-0.0042	-0.0027	0.0019	0.0027	-0.0193	0.0175	0.0115	-0.0050	0.0021
2073m1	2073m1	Monocyte	d2073	9	#7570B3	2073m1	0.0162	0.1191	0.0162	0.1191	0.0391	-0.0986	0.3085	-0.0591	-0.0423	0.0549	-0.0411	0.0040	-0.0384	-0.2096	-0.1181	-0.1005	-0.0560	-0.1105	-0.1295	-0.2965	0.1135	0.0049	0.0903	0.0568	-0.0284	0.0122	-0.0372	-0.0726	0.0150	-0.0402	0.0091	0.0244	0.0154	0.0005	0.0096	-0.0470	-0.0052	-0.0397	0.0164	-0.0914	-0.0919	-0.0592	-0.0672	-0.0106	-0.0858	0.1716	0.0867	-0.0193	-0.0889	-0.0035	-0.3395	-0.0830	-0.1737	-0.5905	0.0795	0.1764	0.0850	-0.0175	0.0052
2073n1	2073n1	Neutrophil	d2073	9	#D95F02	2073n1	-0.1310	0.0947	-0.1310	0.0947	-0.1839	0.1349	0.2066	-0.0449	-0.0406	0.0410	-0.0423	-0.0200	-0.0858	0.0682	-0.0399	0.0672	0.1146	0.3159	0.1446	-0.0676	0.0120	0.0439	-0.0167	-0.0362	-0.0078	0.0073	-0.0557	0.0618	-0.0014	-0.0373	-0.0197	0.0150	-0.0117	0.0731	0.0182	-0.0414	-0.0220	0.0681	0.1209	-0.0220	-0.0231	0.1195	0.0425	0.0161	-0.0357	0.1287	0.2401	0.6201	-0.3304	-0.1206	0.1772	0.0527	0.1450	-0.0230	-0.0156	-0.0169	-0.0107	0.0233	-0.0040
2073e1	2073e1	Eosinophil	d2073	9	#66A61E	2073e1	-0.0894	0.1058	-0.0894	0.1058	0.1825	0.2105	0.1554	0.0209	0.0195	-0.0892	0.0064	0.0552	-0.0720	-0.1109	-0.0543	-0.1074	0.0889	-0.0819	0.0040	0.2536	-0.0291	-0.0688	0.1528	0.0493	-0.1384	-0.1084	0.1241	0.0529	-0.0431	-0.0101	-0.0720	-0.1194	-0.0400	-0.0004	0.1106	-0.0184	-0.0373	0.0181	-0.0716	-0.0968	0.1029	0.0357	-0.0789	0.0383	-0.1346	0.0459	0.0499	0.0683	-0.2046	0.0334	-0.3791	0.0002	-0.4138	0.4364	0.0397	-0.0911	-0.0333	0.0038	0.0221
2073bp1	2073bp1	Biopsy	d2073	9	#E7298A	2073bp1	0.1592	-0.1766	0.1592	-0.1766	-0.0372	0.0531	0.0030	-0.1695	0.0037	-0.1734	-0.1636	0.0709	-0.0403	-0.0144	0.0087	0.0260	-0.0367	0.0204	0.0106	0.0528	0.1571	-0.1311	-0.1038	-0.0906	-0.3568	0.1975	-0.0519	-0.0191	-0.0696	-0.0261	-0.0170	-0.0572	-0.0500	-0.2211	-0.5403	-0.4268	-0.0274	0.1840	-0.1293	0.0020	-0.0300	0.0055	0.0188	-0.0085	-0.0031	0.0136	-0.0574	-0.0063	-0.0386	-0.0036	-0.0182	-0.0084	0.0232	0.0227	-0.0013	0.0139	0.0036	0.0072	0.0012
2068m2	2068m2	Monocyte	d2068	6	#7570B3	2068m2	-0.0045	-0.0436	-0.0045	-0.0436	0.0766	-0.2755	0.1129	0.1425	-0.0380	-0.1599	0.0631	0.0203	0.0474	0.1611	0.0356	-0.1273	-0.1191	0.1090	0.0158	0.1692	-0.1148	-0.1219	0.0073	-0.1796	0.1015	0.0986	-0.0708	0.1693	-0.3036	0.0327	-0.0045	-0.0312	0.1003	0.0480	-0.2489	0.1438	-0.0393	-0.3180	0.1861	0.0838	0.3214	0.3549	0.0209	0.0363	0.0061	0.0985	-0.0461	-0.0048	0.0334	-0.0960	-0.1275	0.0649	-0.0196	-0.1085	0.0130	0.0058	0.0164	0.0080	0.0032
2068n2	2068n2	Neutrophil	d2068	6	#D95F02	2068n2	-0.1757	-0.0735	-0.1757	-0.0735	-0.1694	-0.0309	-0.0802	0.1808	-0.0712	-0.1863	0.1034	0.0500	0.0289	-0.0436	-0.0382	-0.1362	-0.2962	0.0266	-0.2992	0.2505	0.1713	0.1421	-0.1763	0.4944	0.0645	0.0471	-0.3208	-0.0980	0.0145	-0.0581	0.0171	-0.0985	0.0410	0.0273	0.1577	-0.1887	-0.0793	0.1012	0.0903	0.0244	0.0264	0.0471	0.0764	0.0692	0.0129	-0.0052	0.0264	0.0199	-0.0211	0.0026	0.0106	-0.0016	0.0096	0.0027	0.0005	-0.0039	-0.0014	-0.0009	-0.0028
2068e2	2068e2	Eosinophil	d2068	6	#66A61E	2068e2	-0.1229	-0.0559	-0.1229	-0.0559	0.2477	0.0561	-0.1005	0.2041	0.0062	-0.2655	0.0603	-0.0144	0.0938	0.1751	0.0456	-0.1582	-0.1851	0.2332	-0.0966	-0.1226	-0.0079	-0.1800	0.1075	-0.1922	0.1022	0.0000	0.0052	-0.0495	-0.1620	0.0404	0.1536	0.1553	0.0819	-0.1099	-0.0201	0.1077	0.0232	0.1644	0.0911	-0.0936	-0.3325	-0.4138	-0.0066	0.1246	-0.1313	0.0004	-0.0718	0.1208	0.0113	-0.0581	0.0053	-0.0815	-0.0520	0.0362	-0.0157	0.0001	-0.0202	-0.0136	-0.0002
2072m2	2072m2	Monocyte	d2072	8	#7570B3	2072m2	-0.0136	-0.0439	-0.0136	-0.0439	0.0737	-0.2714	0.0784	-0.0387	0.0301	0.0375	0.0426	0.0231	-0.0098	0.0595	0.0135	0.0928	-0.1778	-0.0257	0.1406	0.0858	-0.1401	-0.0104	-0.0830	-0.0499	0.0293	0.0377	-0.0636	0.2230	-0.0711	-0.1128	0.0444	-0.2163	0.0577	0.0526	0.1441	0.0258	0.0241	0.1130	-0.3002	-0.0364	-0.3115	0.0912	-0.0565	-0.1121	0.0753	0.3230	-0.1253	-0.0084	-0.1469	0.4661	0.1860	-0.0452	-0.0764	0.0162	-0.0099	0.0690	0.0008	-0.0115	-0.0059
2072n2	2072n2	Neutrophil	d2072	8	#D95F02	2072n2	-0.1823	-0.0676	-0.1823	-0.0676	-0.1833	-0.0267	-0.1201	-0.0585	-0.0099	0.0805	0.0563	0.0625	-0.0157	-0.1357	-0.0097	0.0557	-0.2060	-0.0970	0.1597	0.0281	-0.0807	-0.0022	0.1319	0.0209	0.0950	0.0912	0.0571	-0.3594	-0.2307	-0.0353	0.0839	-0.1490	0.1225	-0.1334	-0.0880	0.1095	0.0003	-0.1998	-0.2507	-0.1852	0.2174	-0.2822	-0.1305	-0.3252	0.1794	-0.0654	0.1083	0.1362	-0.0610	-0.0985	0.0212	-0.0100	0.0174	0.0263	0.0005	0.0115	0.0155	0.0044	0.0045
2072e2	2072e2	Eosinophil	d2072	8	#66A61E	2072e2	-0.1365	-0.0551	-0.1365	-0.0551	0.2266	0.0690	-0.1506	-0.0747	0.1045	0.0635	0.0244	-0.0345	-0.0638	0.0486	-0.0273	0.0756	-0.2212	0.0308	0.1077	-0.1378	-0.1028	-0.0510	0.0626	0.0381	-0.0499	-0.0511	-0.1725	-0.0851	0.2546	-0.0216	0.0805	0.0324	-0.0238	0.0453	-0.0540	0.0380	-0.0363	-0.0288	-0.0603	0.0173	-0.0104	0.2575	0.3705	-0.3994	-0.4685	-0.1582	-0.0383	0.0102	0.0259	-0.0094	-0.0584	0.0732	-0.0117	-0.0177	0.0126	-0.0077	-0.0053	-0.0245	0.0030
2073m2	2073m2	Monocyte	d2073	9	#7570B3	2073m2	0.0163	0.1175	0.0163	0.1175	0.0288	-0.0949	0.3189	-0.0041	0.0368	-0.0361	-0.0120	-0.0300	-0.0152	-0.1782	-0.1489	-0.0470	-0.1909	-0.0704	-0.1799	-0.2607	-0.0175	-0.0717	-0.0529	-0.0667	-0.0446	0.0647	-0.0110	0.1097	-0.0588	0.0097	-0.0805	0.0603	0.0128	0.0554	-0.0567	0.0476	-0.0440	-0.0362	-0.0295	-0.0411	0.1792	-0.0545	0.1093	0.1136	-0.0394	-0.3636	0.3121	-0.0705	0.0627	0.2392	0.3786	-0.1816	-0.0454	0.1902	0.0449	-0.1224	-0.0854	0.0121	-0.0009
2073n2	2073n2	Neutrophil	d2073	9	#D95F02	2073n2	-0.1356	0.0939	-0.1356	0.0939	-0.2185	0.1421	0.1960	-0.0260	0.0407	-0.0253	-0.0259	-0.0553	-0.0608	0.2016	-0.0695	-0.0777	-0.0534	0.2857	0.2205	-0.0763	-0.0066	0.2303	-0.2206	0.0085	-0.1119	-0.0365	0.0963	0.0672	-0.0756	-0.0499	-0.1437	0.1891	-0.0338	-0.0065	0.0853	-0.0110	-0.0827	-0.0332	-0.0416	-0.2673	0.1174	-0.1541	0.3302	-0.0788	0.1621	0.1114	-0.2448	-0.2523	0.1734	0.0055	-0.1136	-0.0499	-0.0574	0.0253	0.0014	0.0027	0.0048	-0.0068	-0.0017
2073e2	2073e2	Eosinophil	d2073	9	#66A61E	2073e2	-0.0919	0.1074	-0.0919	0.1074	0.1771	0.2161	0.1510	0.0068	0.1245	-0.1371	0.0415	0.0529	-0.0518	-0.0736	-0.0369	-0.1103	-0.0126	-0.0593	0.0572	0.2871	-0.1149	-0.1201	0.1381	0.0733	-0.1334	-0.1616	0.0824	0.0334	0.0359	0.0633	-0.0234	-0.0337	-0.0395	-0.0176	-0.0152	0.0159	-0.0102	-0.0199	0.0429	-0.0671	0.0006	-0.1404	0.0141	0.0300	0.0586	-0.0232	0.0969	-0.1804	-0.0246	0.1519	0.1650	0.5049	0.3777	-0.2483	-0.0780	0.0705	0.0450	0.0440	-0.0123
2068m3	2068m3	Monocyte	d2068	6	#7570B3	2068m3	-0.0174	-0.0463	-0.0174	-0.0463	0.0674	-0.2701	0.0766	-0.0464	-0.0984	0.1065	-0.0583	-0.0047	0.0788	0.0542	0.0293	0.0381	0.0167	-0.0576	0.1081	0.1048	0.0842	0.1897	-0.0504	-0.0819	0.0004	-0.1173	0.0848	0.1189	0.2612	0.2399	0.1663	-0.2412	-0.0176	-0.1843	0.0507	-0.2051	-0.0763	-0.1040	0.3600	0.0006	0.2369	-0.3604	-0.0202	-0.0413	-0.1995	-0.0316	-0.1721	0.1462	0.0247	0.1390	0.0600	-0.0431	-0.0196	0.0081	0.0389	0.0336	0.0004	-0.0094	0.0032
2068n3	2068n3	Neutrophil	d2068	6	#D95F02	2068n3	-0.1804	-0.0687	-0.1804	-0.0687	-0.1715	-0.0214	-0.0929	0.0887	-0.1441	0.0129	-0.0048	0.0216	0.0495	-0.1027	-0.0738	0.1115	-0.2101	-0.0463	-0.1408	0.1445	0.1351	0.2426	-0.1176	-0.0974	-0.1199	-0.0992	0.2995	0.0602	0.1766	0.1691	-0.0809	0.1471	-0.1668	-0.0736	-0.2035	0.3430	0.1986	-0.1116	-0.2729	0.0439	-0.0721	0.0667	-0.0190	0.2446	-0.2075	0.0948	-0.0206	0.0458	-0.0614	-0.0399	0.0227	0.0205	0.0111	-0.0447	-0.0050	0.0189	-0.0042	0.0106	0.0029
2068e3	2068e3	Eosinophil	d2068	6	#66A61E	2068e3	-0.1365	-0.0487	-0.1365	-0.0487	0.2448	0.0714	-0.1386	-0.0020	-0.0123	0.0459	-0.0009	-0.0500	0.0815	0.0499	0.0128	0.1004	-0.0349	0.1017	-0.0791	-0.1443	0.2108	0.1002	-0.0348	-0.1238	0.1143	-0.0282	0.1918	-0.0655	0.0340	0.1679	0.1644	-0.0604	-0.0394	-0.2358	0.0476	-0.0630	-0.0181	0.0523	0.0712	0.1910	0.0158	0.1885	0.2323	-0.0469	0.4785	0.0554	0.3445	-0.1122	-0.1040	0.1497	-0.1264	0.0799	-0.0845	0.0148	0.0279	-0.0266	0.0253	0.0204	0.0040
2072m3	2072m3	Monocyte	d2072	8	#7570B3	2072m3	-0.0179	-0.0438	-0.0179	-0.0438	0.0579	-0.2706	0.0761	-0.0426	0.0445	0.0710	-0.0192	-0.0514	0.0459	0.0445	-0.0058	0.1115	-0.1558	-0.0632	0.1593	0.0737	-0.1111	0.0497	-0.1050	-0.0881	-0.0583	-0.0012	-0.0117	0.2714	0.0020	0.0153	0.0041	-0.1375	0.0290	0.0443	0.2047	-0.0833	-0.0302	0.2549	-0.1882	-0.0164	-0.1057	-0.0625	-0.0222	0.1145	0.0338	-0.2239	0.2745	-0.0543	0.1382	-0.5181	-0.2360	0.0749	0.1229	-0.0109	-0.0501	-0.0436	-0.0096	0.0260	-0.0079
2072n3	2072n3	Neutrophil	d2072	8	#D95F02	2072n3	-0.1843	-0.0697	-0.1843	-0.0697	-0.2022	-0.0213	-0.1140	-0.0240	0.0881	0.0064	-0.0087	-0.0319	0.0111	-0.0549	-0.0340	-0.0396	-0.1934	-0.2320	0.2028	-0.0295	-0.1265	0.0634	0.0747	-0.0928	-0.0860	-0.0382	0.1777	-0.0530	-0.2122	0.0159	-0.2517	0.3489	0.0283	-0.1084	0.1680	-0.2538	-0.0483	0.0447	0.3577	0.2006	-0.1753	0.1858	-0.2518	-0.0790	-0.0749	-0.0552	0.0109	-0.0263	0.0188	0.0978	0.0118	-0.0091	-0.0185	0.0097	0.0046	-0.0183	-0.0117	-0.0065	-0.0012
2072e3	2072e3	Eosinophil	d2072	8	#66A61E	2072e3	-0.1316	-0.0537	-0.1316	-0.0537	0.2331	0.0608	-0.1458	-0.0535	0.1310	0.0082	0.0147	-0.0636	0.0915	0.0336	0.0487	0.0956	-0.2160	0.0275	0.1681	-0.2112	-0.1076	-0.0121	0.0982	0.1621	-0.1563	-0.0251	-0.1672	0.0507	0.3516	-0.0674	0.0020	0.1368	-0.0447	0.2521	-0.1420	-0.0456	0.0175	-0.0793	0.0459	-0.0869	0.2235	-0.0144	-0.3399	0.2738	0.2863	0.1398	-0.0741	0.0101	-0.0303	-0.0354	0.0213	-0.0999	-0.0274	0.0410	0.0080	-0.0105	0.0028	0.0217	0.0014
2159bp1	2159bp1	Biopsy	d2159	10	#E7298A	2159bp1	0.1862	-0.1877	0.1862	-0.1877	-0.0181	0.0991	0.0496	0.1313	-0.0033	0.0233	0.1579	-0.0954	-0.1084	0.0208	-0.0261	0.0746	0.1125	0.0001	-0.0918	-0.0654	-0.2721	0.3162	0.3075	0.3120	0.1163	0.2564	0.2050	0.2221	-0.0900	0.3334	0.2009	0.1438	0.0861	0.1061	-0.1408	-0.1474	-0.0877	0.0496	-0.1076	-0.0220	-0.0305	0.0007	0.0207	-0.0439	-0.0373	0.0025	-0.0134	-0.0048	-0.0099	0.0078	0.0023	-0.0022	0.0078	0.0020	0.0089	-0.0026	-0.0067	0.0050	-0.0108
2073m3	2073m3	Monocyte	d2073	9	#7570B3	2073m3	0.0117	0.1232	0.0117	0.1232	0.0448	-0.0983	0.3170	-0.0340	-0.0016	0.0241	0.0530	0.0386	-0.0597	-0.2101	-0.1116	0.0060	-0.1047	-0.0903	-0.1432	-0.2746	0.0593	-0.0598	-0.0290	0.0398	0.0961	0.0249	0.0425	-0.1161	0.0467	0.0595	0.0857	-0.0300	-0.0061	-0.0738	-0.0068	0.0314	0.0992	0.0328	0.0802	0.1453	-0.1010	0.0940	-0.0383	-0.1000	0.1297	0.1724	-0.4211	0.0326	0.0581	-0.2426	-0.0123	0.2307	0.2243	0.3750	-0.1231	-0.0570	-0.0013	-0.0056	-0.0029
2073n3	2073n3	Neutrophil	d2073	9	#D95F02	2073n3	-0.1324	0.0991	-0.1324	0.0991	-0.1863	0.1367	0.1856	-0.0240	-0.0605	0.0229	0.0404	0.0442	-0.1028	0.0765	0.0029	0.0870	0.0691	0.3923	0.1850	-0.0151	0.0042	-0.0049	-0.0587	0.0324	0.1548	0.0618	-0.0852	-0.1294	0.0884	0.0245	0.2025	-0.1663	0.0440	-0.0348	-0.0958	0.0292	0.0806	-0.0136	-0.0328	0.2834	-0.0950	0.0456	-0.4131	0.0795	-0.1727	-0.2137	0.0641	-0.3176	0.1071	0.1202	-0.0737	0.0268	-0.0798	0.0124	0.0027	0.0092	0.0046	-0.0116	0.0059
2073e3	2073e3	Eosinophil	d2073	9	#66A61E	2073e3	-0.0984	0.1119	-0.0984	0.1119	0.1787	0.2235	0.1452	-0.0068	0.0523	-0.0774	0.0728	0.0524	0.0571	-0.1334	0.0334	0.0384	0.0628	-0.0421	0.0851	0.2554	-0.0301	-0.0209	0.0585	0.0535	-0.0519	-0.0819	0.0895	-0.0361	-0.0096	0.0822	-0.0010	-0.1596	-0.0316	-0.0131	-0.0052	-0.0097	0.0216	-0.0633	-0.0129	0.1847	-0.1595	0.1286	0.0976	-0.0629	0.1673	-0.0347	-0.1335	0.1220	0.2904	-0.1955	0.2095	-0.5639	-0.0066	-0.1914	0.0319	0.0041	-0.0003	-0.0269	-0.0048
2162m1	2162m1	Monocyte	d2162	11	#7570B3	2162m1	0.0053	-0.0464	0.0053	-0.0464	0.0792	-0.2706	0.0935	0.0582	-0.2036	0.0197	-0.0828	-0.0183	0.1394	0.0882	0.0552	-0.1699	0.2197	-0.0421	0.1962	0.0123	-0.1410	-0.0389	-0.0329	0.2799	-0.0738	-0.0068	0.0336	-0.2845	0.0205	0.0984	-0.0505	0.0926	-0.1199	-0.1123	-0.1011	0.2973	0.1552	0.5044	0.0860	-0.1346	0.1176	0.1413	-0.0094	-0.0478	0.0425	-0.0633	0.0220	-0.0087	0.0005	0.0491	0.0578	-0.0376	-0.0457	-0.0276	0.0125	-0.0036	0.0036	0.0036	0.0003
2162n1	2162n1	Neutrophil	d2162	11	#D95F02	2162n1	-0.1551	-0.0752	-0.1551	-0.0752	-0.1245	-0.0431	-0.0529	0.1173	-0.2299	-0.0306	-0.0392	0.0536	0.0571	-0.1486	0.0019	0.1905	0.2561	0.0285	-0.1347	-0.0415	-0.2982	-0.4785	-0.2401	0.0882	-0.0052	-0.1571	0.0345	0.1310	0.1266	0.0033	0.1853	0.2534	0.0899	-0.2160	0.1268	-0.1655	-0.1166	-0.1651	-0.1233	-0.0951	0.0340	0.0404	0.0063	-0.0130	0.0222	0.0456	0.0040	-0.0135	0.0353	-0.0149	-0.0078	-0.0234	-0.0106	-0.0252	-0.0129	0.0023	-0.0012	0.0049	-0.0025
2162e1	2162e1	Eosinophil	d2162	11	#66A61E	2162e1	-0.1244	-0.0542	-0.1244	-0.0542	0.2357	0.0626	-0.1266	-0.0346	-0.0916	0.0801	-0.0563	-0.0513	0.2054	-0.0275	0.0971	0.1002	0.1876	0.0163	-0.0112	-0.2331	-0.0561	0.0133	-0.3255	0.2950	-0.1026	0.1131	0.1218	0.0353	-0.2954	-0.0029	-0.3365	-0.2688	0.0544	0.1733	-0.0671	0.0306	0.0858	-0.2260	0.0980	0.1220	-0.1360	-0.1671	0.0335	0.0018	-0.1277	-0.0202	0.0030	-0.0439	-0.0976	-0.0157	-0.0202	0.0755	0.0420	0.0208	-0.0218	0.0068	0.0009	-0.0063	-0.0036
2162bp1	2162bp1	Biopsy	d2162	11	#E7298A	2162bp1	0.1779	-0.1948	0.1779	-0.1948	-0.0342	0.1390	0.0428	-0.0297	-0.0533	0.4917	0.4540	-0.3747	0.2069	-0.0755	-0.0612	-0.3288	-0.0459	0.1192	-0.0690	0.1245	-0.0348	-0.1435	-0.0967	-0.1260	0.0318	-0.1115	-0.0064	-0.0508	0.0510	-0.0786	-0.0933	-0.0141	-0.0205	-0.0025	-0.0813	-0.0852	-0.0321	0.0343	-0.0397	0.0306	-0.0031	-0.0200	-0.0174	0.0142	0.0081	0.0090	-0.0176	0.0192	-0.0108	0.0068	-0.0060	0.0056	0.0006	0.0040	0.0091	-0.0008	0.0034	-0.0012	0.0006
macrofagos	Macrofagos	Macrophage	unknown	12	#E6AB02	Macrofagos	0.1323	0.1849	0.1323	0.1849	-0.0069	-0.0252	-0.0942	0.1296	-0.0780	-0.1075	0.0693	0.0271	-0.1846	-0.0698	0.3001	-0.0687	-0.0627	0.0336	0.0209	-0.0274	-0.0871	-0.0769	-0.0178	-0.0546	0.0965	0.1017	0.1333	-0.0354	0.1214	-0.0835	-0.1647	0.0429	-0.0801	0.0793	0.0695	-0.0910	0.0938	0.1388	-0.0914	0.3029	0.2538	-0.1861	0.0561	-0.0951	-0.1506	0.3338	0.1921	-0.0663	0.0505	0.0309	-0.0843	-0.1764	0.2671	0.0793	0.1693	0.0137	0.1427	0.0898	0.1352
macrofagos+sbv	Macrofagos+SbV	Macrophage	unknown	12	#E6AB02	Mcrfgs+SbV	0.1246	0.1910	0.1246	0.1910	-0.0122	-0.0174	-0.1228	0.0891	-0.0912	-0.0250	0.2206	0.2401	-0.0039	0.0857	-0.0362	-0.0019	0.0047	-0.0361	0.0411	-0.0452	-0.0057	-0.0319	-0.0182	-0.0600	0.0555	0.1499	0.1010	-0.0333	0.1208	-0.0458	-0.1093	-0.0372	-0.1265	0.0202	0.0125	-0.0386	-0.0806	-0.0312	0.0065	-0.0824	-0.0101	0.0412	0.0162	0.0399	0.0373	-0.2007	-0.0538	0.0656	-0.1032	0.1419	-0.1982	-0.1485	0.1351	-0.0282	-0.4241	0.0461	-0.0173	0.1099	-0.5621
macrofagos+10772	Macrofagos+10772	Macrophage	unknown	12	#E6AB02	Mcrf+10772	0.1287	0.1845	0.1287	0.1845	-0.0286	-0.0179	-0.1087	0.0431	0.0117	-0.0368	-0.1053	-0.1609	-0.0967	-0.0763	0.1624	0.0059	-0.0550	0.0301	-0.0086	0.0428	-0.0389	-0.0347	-0.0344	-0.0076	0.1243	0.1055	0.1070	-0.0424	0.0964	-0.0394	-0.0686	-0.0477	-0.0942	0.0212	-0.0071	-0.0734	-0.0165	0.0227	0.0159	0.0232	0.0242	-0.0509	-0.0043	-0.0493	-0.0122	0.1567	0.0217	-0.0195	0.1000	-0.1692	0.2783	0.2537	-0.4810	-0.2060	-0.3112	-0.3133	-0.2286	-0.1297	-0.0627
macrofagos+10772+sbv	Macrofagos+10772+SbV	Macrophage	unknown	12	#E6AB02	M+10772+SV	0.1220	0.1913	0.1220	0.1913	-0.0205	-0.0132	-0.1234	0.0941	-0.0359	-0.0412	0.1644	0.1765	0.0288	0.1001	-0.1429	0.0266	0.0241	-0.0289	0.0231	-0.0349	-0.0326	-0.0219	-0.0240	-0.0488	0.0095	0.0694	0.0469	0.0156	0.0732	-0.0196	-0.1021	0.0113	-0.0481	0.0424	0.0204	-0.0511	-0.0156	0.0261	-0.0414	0.0784	0.0367	-0.0046	0.0395	0.0190	0.1140	-0.2256	-0.0839	0.0750	-0.0804	-0.0568	0.0777	0.1080	-0.1674	0.0225	-0.0711	0.5431	0.0012	-0.3445	0.4602
macrofagos+2169	Macrofagos+2169	Macrophage	unknown	12	#E6AB02	Mcrfg+2169	0.1233	0.1765	0.1233	0.1765	-0.0475	-0.0091	-0.0924	0.0704	0.0809	-0.0969	-0.2807	-0.3484	0.0366	-0.0406	-0.0688	0.0309	-0.0364	0.0308	-0.0323	0.0854	-0.0842	-0.0397	-0.0412	-0.0216	0.1531	0.1071	0.0762	-0.0585	0.0825	-0.0289	-0.0374	-0.0545	-0.0621	0.0234	0.0203	-0.0128	-0.0293	-0.0421	-0.0081	-0.0426	-0.0890	0.0508	-0.0023	0.0105	0.0800	-0.2097	-0.1723	0.0631	-0.0637	-0.0437	0.0580	0.1036	-0.1276	0.0214	0.3396	0.2200	0.2643	0.4774	-0.1253
macrofagos+2169+sbv	Macrofagos+2169+SbV	Macrophage	unknown	12	#E6AB02	Mc+2169+SV	0.1162	0.1862	0.1162	0.1862	-0.0425	0.0003	-0.1314	-0.0119	0.0579	-0.0077	-0.0351	0.0510	0.1831	0.1080	-0.3328	0.0292	0.0631	0.0021	-0.0179	0.0056	-0.0136	-0.0024	0.0588	0.0280	-0.0920	-0.1071	-0.0847	0.0479	-0.0774	-0.0214	0.0396	0.0682	0.1341	0.0005	0.0386	0.0159	0.1315	0.0889	-0.1204	0.3108	0.1494	-0.1225	0.0153	-0.0493	0.0219	0.0600	-0.0703	0.0213	-0.0422	-0.0219	-0.0044	0.0789	0.0128	-0.0587	0.4101	-0.0523	-0.3353	-0.2184	-0.3897
macrofagos+12309	Macrofagos+12309	Macrophage	unknown	12	#E6AB02	Mcrf+12309	0.1277	0.1869	0.1277	0.1869	-0.0192	-0.0094	-0.1151	0.0260	-0.0647	0.0614	0.0051	-0.0684	-0.1767	-0.1354	0.3551	-0.0416	-0.0326	0.0242	0.0115	-0.0435	0.0540	0.0919	-0.0248	0.0084	-0.1812	-0.2174	-0.1283	0.1142	-0.1436	0.0496	0.1023	0.0697	0.1548	-0.0324	0.0034	0.0671	0.0471	0.0064	-0.0472	0.0269	-0.0057	0.0247	0.0038	0.0408	0.0960	-0.2421	-0.1466	0.0811	-0.1045	0.0586	-0.0253	0.0425	-0.0148	-0.0315	0.0372	-0.2037	0.4921	-0.3411	-0.1053
macrofagos+12309+sbv	Macrofagos+12309+SbV	Macrophage	unknown	12	#E6AB02	M+12309+SV	0.1243	0.1901	0.1243	0.1901	-0.0190	-0.0094	-0.1385	-0.0020	-0.0689	0.0384	0.1741	0.2570	0.0295	0.0745	-0.0551	-0.0011	0.0086	-0.0285	0.0268	-0.0294	0.0172	-0.0045	-0.0402	-0.0190	0.0212	0.0747	0.0382	-0.0053	0.0449	0.0109	-0.0134	-0.0546	-0.0562	-0.0016	-0.0068	-0.0095	-0.0644	-0.0506	0.0322	-0.1943	-0.1452	0.0700	-0.0388	0.0307	0.0582	-0.1341	-0.1302	0.0328	-0.0662	0.0638	-0.0949	0.0258	0.1109	-0.1498	0.2946	-0.5455	-0.1641	0.1922	0.4052
macrofagos+12367+sbv	Macrofagos+12367+SbV	Macrophage	unknown	12	#E6AB02	M+12367+SV	0.1206	0.1911	0.1206	0.1911	-0.0289	-0.0048	-0.1552	-0.0971	-0.0276	0.1224	0.1479	0.2442	0.0721	0.0703	-0.0704	0.0025	0.0349	-0.0318	0.0155	-0.0208	0.0737	0.0177	0.0247	0.0336	-0.0315	-0.0196	-0.0463	-0.0115	-0.0402	0.0703	0.0537	-0.0504	-0.0240	-0.0443	-0.0727	0.0057	-0.1036	-0.0785	0.1647	-0.3127	-0.1683	0.1006	-0.0421	0.0300	-0.1400	0.3138	0.2890	-0.1562	0.2392	-0.1718	0.2286	0.0441	-0.0804	0.2039	0.1918	0.1189	0.2684	-0.0428	-0.1686
macrofagos+1126	Macrofagos+1126	Macrophage	unknown	12	#E6AB02	Mcrfg+1126	0.1254	0.1878	0.1254	0.1878	-0.0220	-0.0086	-0.1267	-0.0386	-0.0490	0.1129	0.0222	-0.0389	-0.1465	-0.1287	0.3366	-0.0386	-0.0101	0.0236	0.0065	-0.0416	0.0882	0.0883	0.0150	0.0444	-0.1819	-0.2289	-0.1593	0.0720	-0.1733	0.0713	0.1397	0.0558	0.1250	-0.0584	-0.0208	0.0852	0.0111	-0.0216	0.0375	-0.0261	-0.0301	0.0441	0.0058	0.0076	0.0109	-0.0075	0.0152	-0.0250	0.0214	-0.0046	-0.0135	-0.0236	0.0087	0.0584	-0.0908	0.3100	-0.5101	0.3781	0.0643
macrofagos+12251	Macrofagos+12251	Macrophage	unknown	12	#E6AB02	Mcrf+12251	0.1242	0.1759	0.1242	0.1759	-0.0555	-0.0039	-0.1083	-0.0094	0.0911	-0.0215	-0.3345	-0.3700	0.0667	-0.0548	-0.0485	-0.0029	-0.0378	0.0351	-0.0365	0.1067	-0.0446	-0.0271	0.0162	0.0347	0.1174	0.0462	0.0391	-0.0723	0.0209	0.0254	0.0593	-0.0867	-0.0712	-0.0282	-0.0339	0.0373	-0.0639	-0.1052	0.0857	-0.2488	-0.1160	0.1093	-0.0469	0.0449	-0.0378	0.0522	0.0906	-0.0388	0.0083	0.1164	-0.2020	-0.1902	0.3499	0.1025	-0.0658	0.0164	-0.1975	-0.4336	0.0908
macrofagos+12251+sbv	Macrofagos+12251+SbV	Macrophage	unknown	12	#E6AB02	M+12251+SV	0.1153	0.1846	0.1153	0.1846	-0.0549	0.0054	-0.1355	-0.0568	0.1012	0.0337	-0.1230	-0.0661	0.2360	0.0939	-0.3743	0.0483	0.0774	-0.0134	-0.0374	0.0423	0.0201	0.0263	0.0735	0.0429	-0.0830	-0.1056	-0.1358	0.0369	-0.1151	0.0165	0.1009	0.0648	0.1471	-0.0118	-0.0011	0.0469	0.0842	0.0390	-0.0371	0.1688	0.1021	-0.0772	0.0107	-0.0407	-0.0778	0.1195	0.0439	-0.0139	0.0255	0.0540	-0.0220	-0.1121	-0.0023	-0.0051	-0.4753	-0.1500	0.2873	0.2732	0.2536

write.csv(q2_pca$table, file="coords/q2_pca_coords.csv")

6 Request on 202101

Maria Adelaida sent an email which said: “I have been working on the gene lists and the easiest and cleanest way I found was to get the GSEA hallmark gene sets that may be of interest to us. The idea is that we use some of these gene lists (to star the xenobiotic list) to map the changes in gene expression over the course of treatment and in the different cell populations.”

Upon speaking with her, I think the goal is to have a plot with one line per MSigDB Hallmark category gene. The x-axis will therefore be the three time points for those patients for whom we have 3 visits; the y-axis will be normalized cpm. Each dot will be a single Hallmark gene.

With that in mind, the most difficult challenge is picking out the samples for which we have the three time points.

meta <- hpgltools::extract_metadata(samplesheet)

## Dropped 164 rows from the sample metadata because they were blank.

na_idx <- is.na(meta)
meta[na_idx] <- ""

visit_3_samples <- meta[["visitnumber"]] == 3
threes <- meta[visit_3_samples, ]
visit_2_samples <- meta[["visitnumber"]] == 2
twos <- meta[visit_2_samples, ]
visit_1_samples <- meta[["visitnumber"]] == 1
ones <- meta[visit_1_samples, ]

ones[["tubelabelorigin"]] %in% twos[["tubelabelorigin"]]

##  [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE FALSE  TRUE
## [13]  TRUE  TRUE  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
## [25]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE

ones_twos <- ones[["tubelabelorigin"]] %in% twos[["tubelabelorigin"]]
one_two <- meta[ones_twos, ]
all_three_idx <- threes[["tubelabelorigin"]] %in% one_two[["tubelabelorigin"]]
all_three <- threes[all_three_idx, ]
wanted_patients <- levels(as.factor(all_three[["tubelabelorigin"]]))

## So, it appears we have 3 visits for patients: su2068, su2072, and su2073.
plot_expt <- subset_expt(expt=all_norm, subset="tubelabelorigin=='su2068'|tubelabelorigin=='su2072'|tubelabelorigin=='su2073'")

## Using a subset expression.

## There were 64, now there are 29 samples.

plot_pca(plot_expt)$plot

plot_monocyte <- subset_expt(plot_expt, subset="typeofcells=='Monocytes'&tubelabelorigin!='su2072'")

## Using a subset expression.

## There were 29, now there are 6 samples.

up_monocyte_genes <- rownames(mono_sig[["deseq"]][["ups"]][["fail_vs_cure"]])
down_monocyte_genes <- rownames(mono_sig[["deseq"]][["downs"]][["fail_vs_cure"]])
plot_monocyte <- exclude_genes_expt(plot_monocyte, ids=down_monocyte_genes, method="keep")

## Before removal, there were 14424 entries.

## Now there are 1516 entries.

## Percent kept: 9.822, 9.065, 10.364, 9.124, 9.889, 9.335

## Percent removed: 90.178, 90.935, 89.636, 90.876, 90.111, 90.665

hallmark_table <- openxlsx::read.xlsx(xlsxFile="reference/geneset_selection_GSEA_Hallmark_modified.xlsx")
first_category <- hallmark_table[[1]]

hallmark_genes_idx <- fData(plot_monocyte)[["hgnc_symbol"]] %in% first_category
hallmark_genes <- fData(plot_monocyte)[hallmark_genes_idx, ]
wanted <- rownames(hallmark_genes)

plotted <- exprs(plot_monocyte)[wanted, ]
plot_long <- reshape2::melt(plotted)
colnames(plot_long) <- c("gene", "samplename", "exprs")
dim(plot_long)

## [1] 84  3

plot_long <- merge(plot_long, fData(plot_monocyte), by.x="gene", by.y="row.names")
dim(plot_long)

## [1] 84 16

plot_long <- merge(plot_long, pData(plot_monocyte), by="samplename")
dim(plot_long)

## [1] 84 95

library(ggplot2)

## RStudio Community is a great place to get help:
## https://community.rstudio.com/c/tidyverse

plot_2068_idx <- plot_long[["tubelabelorigin"]] == "su2068"
plot_2068 <- plot_long[plot_2068_idx, ]
plt <- ggplot(data=plot_2068, mapping=aes_string(x="visitnumber", y="exprs",
                                                 colour="gene")) +
  geom_line() +
  theme(legend.position="none")
plt

probably_not <- ggplotly_url(plt, "probably_terrible_su2068.html")

## Warning in ggplotly_url(plt, "probably_terrible_su2068.html"): No url df was
## provided, nor is there a column: url, not much to do.

plot_2073_idx <- plot_long[["tubelabelorigin"]] == "su2073"
plot_2073 <- plot_long[plot_2073_idx, ]
plt <- ggplot(data=plot_2073, mapping=aes_string(x="visitnumber", y="exprs",
                                                 colour="gene")) +
  geom_line() +
  theme(legend.position="none")
plt

probably_not <- ggplotly_url(plt, "images/probably_terrible_su2073.html")

## Warning in ggplotly_url(plt, "images/probably_terrible_su2073.html"): No url df
## was provided, nor is there a column: url, not much to do.

if (!isTRUE(get0("skip_load"))) {
  pander::pander(sessionInfo())
  message(paste0("This is hpgltools commit: ", get_git_commit()))
  message(paste0("Saving to ", savefile))
  tmp <- sm(saveme(filename=savefile))
}

## If you wish to reproduce this exact build of hpgltools, invoke the following:

## > git clone http://github.com/abelew/hpgltools.git

## > git reset 052640a0b091e9a740e487957e087f265d0c74b5

## This is hpgltools commit: Thu Dec 3 13:42:02 2020 -0500: 052640a0b091e9a740e487957e087f265d0c74b5

## Saving to 01_annotation_v202011.rda.xz

tmp <- loadme(filename=savefile)