Annotation
We take the annotation data from ensembl’s biomart instance. The genome which was used to map the data was hg38 revision 91. My default when using biomart is to load the data from 1 year before the current date, which provides annotations which match hg38 91 almost perfectly.
hs_annot <- load_biomart_annotations()
## The biomart annotations file already exists, loading from it.
hs_annot <- hs_annot[["annotation"]]
hs_annot[["transcript"]] <- paste0(rownames(hs_annot), ".", hs_annot[["version"]])
rownames(hs_annot) <- make.names(hs_annot[["ensembl_gene_id"]], unique=TRUE)
tx_gene_map <- hs_annot[, c("transcript", "ensembl_gene_id")]
Sample Estimation
I used two mapping methods for this data, hisat2 and salmon. Most analyses use hisat2, which is a more traditional map-and-count method. In contrast, salmon uses what may be thought of as a indexed voting method (so that multi-matches are discounted and the votes split among all matches). Salmon also required a pre-existing database of known transcripts (though later versions may actually use mapping from things like hisat), while hisat uses the genome and a database of known transcripts (and optionally can search for splicing junctions to find new transcripts).
Generate expressionsets
The first thing to note is the large range in coverage. There are multiple samples with coverage which is too low to use. These will be removed shortly.
hs_expt <- sm(create_expt("sample_sheets/tmrc3_samples_20201105.xlsx",
file_column="hg3891hisatfile", savefile="hs_expt_all.rda",
gene_info=hs_annot))
Minimum coverage sample filtering
I arbitrarily chose 3,000,000 counts as a minimal level of coverage. We may want this to be higher.
hs_valid <- subset_expt(hs_expt, coverage=3000000)
## Subsetting given a minimal number of counts/sample.
## There were 70, now there are 64 samples.
Macrophages
These samples are rather different from all of the others. The following section is therefore written primarily in response to a separate set of emails from Olga and Maria Adelaida; here is a snippet:
Dear all, about the samples corresponding to infected macrophages with three sensitive (2.2) and three resistant (2.3) clinical strains of L. (V.) panamensis, I send you the results of parasite burden by detection of 7SLRNA. I think these results are interesting, but the sample size is very small. Doctor Najib or Trey could you please send me the quality data and PCA analysis of these samples?
and
Hi Doctor, thank you. These samples corresponding to primary macrophages infected with clinical strains 2.2 (n=3) and 2.3 (n = 3). These information is in the file: TMRC project 3: excel Host TMRC3 v1.1, rows 137 to 150.
Thus I added 3 columns to the end of the sample sheet which seek to include this information. The first is ‘drug’ and encodes both the infection state (no for the two controls and yes for everything else), the second is zymodeme which I took from the tmrc2 sample sheet, the last is drug, which is either no or sb.
macr <- subset_expt(hs_expt, subset="typeofcells=='Macrophages'")
## Using a subset expression.
## There were 70, now there are 12 samples.
macr <- set_expt_conditions(macr, fact="zymodeme")
macr <- set_expt_batches(macr, fact="macrdrug")
macr_norm <- sm(normalize_expt(macr, transform="log2", norm="quant", convert="cpm", filter=TRUE))
norm_pca <- plot_pca(macr_norm, plot_labels=FALSE)
## Not putting labels on the PC plot.
pp(file="macrophage_side_experiment/norm_pca.png", image=norm_pca$plot)
## Writing the image to: macrophage_side_experiment/norm_pca.png and calling dev.off().

plot_3d_pca(norm_pca)
## Warning: `arrange_()` is deprecated as of dplyr 0.7.0.
## Please use `arrange()` instead.
## See vignette('programming') for more help
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_warnings()` to see where this warning was generated.
## $plot
##
## $file
## [1] "3dpca.html"
macr_nb <- sm(normalize_expt(macr, batch="svaseq", filter=TRUE))
macr_nb <- sm(normalize_expt(macr_nb, norm="quant", convert="cpm", transform="log2"))
nb_pca <- plot_pca(macr_nb, plot_labels=FALSE)
## Not putting labels on the PC plot.
pp(file="macrophage_side_experiment/normbatch_pca.png", image=nb_pca$plot)
## Writing the image to: macrophage_side_experiment/normbatch_pca.png and calling dev.off().

Write the data
macr_written <- sm(write_expt(macr, excel="macrophage_side_experiment/macrophage_expt.xlsx"))
LS0tCnRpdGxlOiAiTC4gcGFuYW1lbnNpcyAyMDIwMTE6IEEgTWFjcm9waGFnZSBTaWRlIEV4cGVyaW1lbnQiCmF1dGhvcjogImF0YiBhYmVsZXdAZ21haWwuY29tIgpkYXRlOiAiYHIgU3lzLkRhdGUoKWAiCm91dHB1dDoKIGh0bWxfZG9jdW1lbnQ6CiAgY29kZV9kb3dubG9hZDogdHJ1ZQogIGNvZGVfZm9sZGluZzogc2hvdwogIGZpZ19jYXB0aW9uOiB0cnVlCiAgZmlnX2hlaWdodDogNwogIGZpZ193aWR0aDogNwogIGhpZ2hsaWdodDogZGVmYXVsdAogIGtlZXBfbWQ6IGZhbHNlCiAgbW9kZTogc2VsZmNvbnRhaW5lZAogIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogIHNlbGZfY29udGFpbmVkOiB0cnVlCiAgdGhlbWU6IHJlYWRhYmxlCiAgdG9jOiB0cnVlCiAgdG9jX2Zsb2F0OgogICBjb2xsYXBzZWQ6IGZhbHNlCiAgIHNtb290aF9zY3JvbGw6IGZhbHNlCi0tLQoKPHN0eWxlPgogIGJvZHkgLm1haW4tY29udGFpbmVyIHsKICAgIG1heC13aWR0aDogMTYwMHB4OwogIH0KPC9zdHlsZT4KCmBgYHtyIG9wdGlvbnMsIGluY2x1ZGU9RkFMU0V9CmlmICghaXNUUlVFKGdldDAoInNraXBfbG9hZCIpKSkgewogIGxpYnJhcnkoaHBnbHRvb2xzKQogIHR0IDwtIHNtKGRldnRvb2xzOjpsb2FkX2FsbCgiL2RhdGEvaHBnbHRvb2xzIikpCiAga25pdHI6Om9wdHNfa25pdCRzZXQocHJvZ3Jlc3M9VFJVRSwKICAgICAgICAgICAgICAgICAgICAgICB2ZXJib3NlPVRSVUUsCiAgICAgICAgICAgICAgICAgICAgICAgd2lkdGg9OTAsCiAgICAgICAgICAgICAgICAgICAgICAgZWNobz1UUlVFKQogIGtuaXRyOjpvcHRzX2NodW5rJHNldChlcnJvcj1UUlVFLAogICAgICAgICAgICAgICAgICAgICAgICBmaWcud2lkdGg9OCwKICAgICAgICAgICAgICAgICAgICAgICAgZmlnLmhlaWdodD04LAogICAgICAgICAgICAgICAgICAgICAgICBkcGk9OTYpCiAgb2xkX29wdGlvbnMgPC0gb3B0aW9ucyhkaWdpdHM9NCwKICAgICAgICAgICAgICAgICAgICAgICAgIHN0cmluZ3NBc0ZhY3RvcnM9RkFMU0UsCiAgICAgICAgICAgICAgICAgICAgICAgICBrbml0ci5kdXBsaWNhdGUubGFiZWw9ImFsbG93IikKICBnZ3Bsb3QyOjp0aGVtZV9zZXQoZ2dwbG90Mjo6dGhlbWVfYncoYmFzZV9zaXplPTEyKSkKICB2ZXIgPC0gIjIwMjAxMSIKICBwcmV2aW91c19maWxlIDwtIHBhc3RlMCgibm9uZSIsIHZlciwgIi5SbWQiKQogIHJ1bmRhdGUgPC0gZm9ybWF0KFN5cy5EYXRlKCksIGZvcm1hdD0iJVklbSVkIikKICB0bXAgPC0gdHJ5KHNtKGxvYWRtZShmaWxlbmFtZT1nc3ViKHBhdHRlcm49IlxcLlJtZCIsIHJlcGxhY2U9IlxcLnJkYVxcLnh6IiwgeD1wcmV2aW91c19maWxlKSkpKQogIHJtZF9maWxlIDwtICJtYWNyb3BoYWdlX3NpZGVfZXhwZXJpbWVudC5SbWQiCiAgc2F2ZWZpbGUgPC0gZ3N1YihwYXR0ZXJuPSJcXC5SbWQiLCByZXBsYWNlPSJcXC5yZGFcXC54eiIsIHg9cm1kX2ZpbGUpCn0KYGBgCgojIEludHJvZHVjdGlvbgoKVGhpcyBkb2N1bWVudCBpcyBpbnRlbmRlZCB0byBwcm92aWRlIGFuIG92ZXJ2aWV3IG9mIFRNUkMzIHNhbXBsZXMgd2hpY2ggaGF2ZQpiZWVuIHNlcXVlbmNlZC4gIEl0IGluY2x1ZGVzIHNvbWUgcGxvdHMgYW5kIGFuYWx5c2VzIHNob3dpbmcgdGhlIHJlbGF0aW9uc2hpcHMKYW1vbmcgdGhlIHNhbXBsZXMgYXMgd2VsbCBhcyBzb21lIGRpZmZlcmVudGlhbCBhbmFseXNlcyB3aGVuIHBvc3NpYmxlLgoKIyBBbm5vdGF0aW9uCgpXZSB0YWtlIHRoZSBhbm5vdGF0aW9uIGRhdGEgZnJvbSBlbnNlbWJsJ3MgYmlvbWFydCBpbnN0YW5jZS4gIFRoZSBnZW5vbWUgd2hpY2gKd2FzIHVzZWQgdG8gbWFwIHRoZSBkYXRhIHdhcyBoZzM4IHJldmlzaW9uIDkxLiAgTXkgZGVmYXVsdCB3aGVuIHVzaW5nIGJpb21hcnQgaXMKdG8gbG9hZCB0aGUgZGF0YSBmcm9tIDEgeWVhciBiZWZvcmUgdGhlIGN1cnJlbnQgZGF0ZSwgd2hpY2ggcHJvdmlkZXMgYW5ub3RhdGlvbnMKd2hpY2ggbWF0Y2ggaGczOCA5MSBhbG1vc3QgcGVyZmVjdGx5LgoKYGBge3IgaHNfYW5ub3R9CmhzX2Fubm90IDwtIGxvYWRfYmlvbWFydF9hbm5vdGF0aW9ucygpCmhzX2Fubm90IDwtIGhzX2Fubm90W1siYW5ub3RhdGlvbiJdXQpoc19hbm5vdFtbInRyYW5zY3JpcHQiXV0gPC0gcGFzdGUwKHJvd25hbWVzKGhzX2Fubm90KSwgIi4iLCBoc19hbm5vdFtbInZlcnNpb24iXV0pCnJvd25hbWVzKGhzX2Fubm90KSA8LSBtYWtlLm5hbWVzKGhzX2Fubm90W1siZW5zZW1ibF9nZW5lX2lkIl1dLCB1bmlxdWU9VFJVRSkKdHhfZ2VuZV9tYXAgPC0gaHNfYW5ub3RbLCBjKCJ0cmFuc2NyaXB0IiwgImVuc2VtYmxfZ2VuZV9pZCIpXQpgYGAKCiMgU2FtcGxlIEVzdGltYXRpb24KCkkgdXNlZCB0d28gbWFwcGluZyBtZXRob2RzIGZvciB0aGlzIGRhdGEsIGhpc2F0MiBhbmQgc2FsbW9uLiAgTW9zdCBhbmFseXNlcyB1c2UKaGlzYXQyLCB3aGljaCBpcyBhIG1vcmUgdHJhZGl0aW9uYWwgbWFwLWFuZC1jb3VudCBtZXRob2QuICBJbiBjb250cmFzdCwgc2FsbW9uCnVzZXMgd2hhdCBtYXkgYmUgdGhvdWdodCBvZiBhcyBhIGluZGV4ZWQgdm90aW5nIG1ldGhvZCAoc28gdGhhdCBtdWx0aS1tYXRjaGVzIGFyZQpkaXNjb3VudGVkIGFuZCB0aGUgdm90ZXMgc3BsaXQgYW1vbmcgYWxsIG1hdGNoZXMpLiAgU2FsbW9uIGFsc28gcmVxdWlyZWQgYQpwcmUtZXhpc3RpbmcgZGF0YWJhc2Ugb2Yga25vd24gdHJhbnNjcmlwdHMgKHRob3VnaCBsYXRlciB2ZXJzaW9ucyBtYXkgYWN0dWFsbHkKdXNlIG1hcHBpbmcgZnJvbSB0aGluZ3MgbGlrZSBoaXNhdCksIHdoaWxlIGhpc2F0IHVzZXMgdGhlIGdlbm9tZSBhbmQgYSBkYXRhYmFzZQpvZiBrbm93biB0cmFuc2NyaXB0cyAoYW5kIG9wdGlvbmFsbHkgY2FuIHNlYXJjaCBmb3Igc3BsaWNpbmcganVuY3Rpb25zIHRvIGZpbmQKbmV3IHRyYW5zY3JpcHRzKS4KCiMjIEdlbmVyYXRlIGV4cHJlc3Npb25zZXRzCgpUaGUgZmlyc3QgdGhpbmcgdG8gbm90ZSBpcyB0aGUgbGFyZ2UgcmFuZ2UgaW4gY292ZXJhZ2UuICBUaGVyZSBhcmUgbXVsdGlwbGUKc2FtcGxlcyB3aXRoIGNvdmVyYWdlIHdoaWNoIGlzIHRvbyBsb3cgdG8gdXNlLiAgVGhlc2Ugd2lsbCBiZSByZW1vdmVkIHNob3J0bHkuCgpgYGB7ciBhbGxfbmV3X2hpc2F0Mn0KaHNfZXhwdCA8LSBzbShjcmVhdGVfZXhwdCgic2FtcGxlX3NoZWV0cy90bXJjM19zYW1wbGVzXzIwMjAxMTA1Lnhsc3giLAogICAgICAgICAgICAgICAgICAgICAgICAgIGZpbGVfY29sdW1uPSJoZzM4OTFoaXNhdGZpbGUiLCBzYXZlZmlsZT0iaHNfZXhwdF9hbGwucmRhIiwKICAgICAgICAgICAgICAgICAgICAgICAgICBnZW5lX2luZm89aHNfYW5ub3QpKQpgYGAKCiMjIE1pbmltdW0gY292ZXJhZ2Ugc2FtcGxlIGZpbHRlcmluZwoKSSBhcmJpdHJhcmlseSBjaG9zZSAzLDAwMCwwMDAgY291bnRzIGFzIGEgbWluaW1hbCBsZXZlbCBvZiBjb3ZlcmFnZS4gIFdlIG1heQp3YW50IHRoaXMgdG8gYmUgaGlnaGVyLgoKYGBge3IgaGlzYXQyX3dyaXRlLCBmaWcuc2hvdz0iaGlkZSJ9CmhzX3ZhbGlkIDwtIHN1YnNldF9leHB0KGhzX2V4cHQsIGNvdmVyYWdlPTMwMDAwMDApCmBgYAoKIyBNYWNyb3BoYWdlcwoKVGhlc2Ugc2FtcGxlcyBhcmUgcmF0aGVyIGRpZmZlcmVudCBmcm9tIGFsbCBvZiB0aGUgb3RoZXJzLiAgVGhlIGZvbGxvd2luZwpzZWN0aW9uIGlzIHRoZXJlZm9yZSB3cml0dGVuIHByaW1hcmlseSBpbiByZXNwb25zZSB0byBhIHNlcGFyYXRlIHNldCBvZiBlbWFpbHMKZnJvbSBPbGdhIGFuZCBNYXJpYSBBZGVsYWlkYTsgaGVyZSBpcyBhIHNuaXBwZXQ6CgogRGVhciBhbGwsIGFib3V0IHRoZSBzYW1wbGVzIGNvcnJlc3BvbmRpbmcgdG8gaW5mZWN0ZWQgbWFjcm9waGFnZXMgd2l0aCB0aHJlZQogc2Vuc2l0aXZlICgyLjIpIGFuZCB0aHJlZSByZXNpc3RhbnQgKDIuMykgY2xpbmljYWwgc3RyYWlucyBvZiBMLiAoVi4pCiBwYW5hbWVuc2lzLCBJIHNlbmQgeW91IHRoZSByZXN1bHRzIG9mIHBhcmFzaXRlIGJ1cmRlbiBieSBkZXRlY3Rpb24gb2YgN1NMUk5BLiBJCiB0aGluayB0aGVzZSByZXN1bHRzIGFyZSBpbnRlcmVzdGluZywgYnV0IHRoZSBzYW1wbGUgc2l6ZSBpcyB2ZXJ5IHNtYWxsLgpEb2N0b3IgTmFqaWIgb3IgVHJleSBjb3VsZCB5b3UgcGxlYXNlIHNlbmQgbWUgdGhlIHF1YWxpdHkgZGF0YSBhbmQgUENBIGFuYWx5c2lzCiBvZiB0aGVzZSBzYW1wbGVzPwoKYW5kCgogSGkgRG9jdG9yLCB0aGFuayB5b3UuICBUaGVzZSBzYW1wbGVzIGNvcnJlc3BvbmRpbmcgdG8gcHJpbWFyeSBtYWNyb3BoYWdlcwogaW5mZWN0ZWQgd2l0aCBjbGluaWNhbCBzdHJhaW5zIDIuMiAobj0zKSBhbmQgMi4zIChuID0gMykuIFRoZXNlIGluZm9ybWF0aW9uIGlzCiBpbiB0aGUgZmlsZTogVE1SQyBwcm9qZWN0IDM6IGV4Y2VsIEhvc3QgVE1SQzMgdjEuMSwgcm93cyAxMzcgdG8gMTUwLgoKVGh1cyBJIGFkZGVkIDMgY29sdW1ucyB0byB0aGUgZW5kIG9mIHRoZSBzYW1wbGUgc2hlZXQgd2hpY2ggc2VlayB0bwppbmNsdWRlIHRoaXMgaW5mb3JtYXRpb24uICBUaGUgZmlyc3QgaXMgJ2RydWcnIGFuZCBlbmNvZGVzIGJvdGggdGhlIGluZmVjdGlvbgpzdGF0ZSAobm8gZm9yIHRoZSB0d28gY29udHJvbHMgYW5kIHllcyBmb3IgZXZlcnl0aGluZyBlbHNlKSwgdGhlIHNlY29uZCBpcwp6eW1vZGVtZSB3aGljaCBJIHRvb2sgZnJvbSB0aGUgdG1yYzIgc2FtcGxlIHNoZWV0LCB0aGUgbGFzdCBpcyBkcnVnLCB3aGljaCBpcwplaXRoZXIgbm8gb3Igc2IuCgpgYGB7ciBtYWNyb3BoYWdlX3p5bW9kZW1lX2V4cGVyaW1lbnR9Cm1hY3IgPC0gc3Vic2V0X2V4cHQoaHNfZXhwdCwgc3Vic2V0PSJ0eXBlb2ZjZWxscz09J01hY3JvcGhhZ2VzJyIpCm1hY3IgPC0gc2V0X2V4cHRfY29uZGl0aW9ucyhtYWNyLCBmYWN0PSJ6eW1vZGVtZSIpCm1hY3IgPC0gc2V0X2V4cHRfYmF0Y2hlcyhtYWNyLCBmYWN0PSJtYWNyZHJ1ZyIpCgptYWNyX25vcm0gPC0gc20obm9ybWFsaXplX2V4cHQobWFjciwgdHJhbnNmb3JtPSJsb2cyIiwgbm9ybT0icXVhbnQiLCBjb252ZXJ0PSJjcG0iLCBmaWx0ZXI9VFJVRSkpCm5vcm1fcGNhIDwtIHBsb3RfcGNhKG1hY3Jfbm9ybSwgcGxvdF9sYWJlbHM9RkFMU0UpCnBwKGZpbGU9Im1hY3JvcGhhZ2Vfc2lkZV9leHBlcmltZW50L25vcm1fcGNhLnBuZyIsIGltYWdlPW5vcm1fcGNhJHBsb3QpCnBsb3RfM2RfcGNhKG5vcm1fcGNhKQoKbWFjcl9uYiA8LSBzbShub3JtYWxpemVfZXhwdChtYWNyLCBiYXRjaD0ic3Zhc2VxIiwgZmlsdGVyPVRSVUUpKQptYWNyX25iIDwtIHNtKG5vcm1hbGl6ZV9leHB0KG1hY3JfbmIsIG5vcm09InF1YW50IiwgY29udmVydD0iY3BtIiwgdHJhbnNmb3JtPSJsb2cyIikpCm5iX3BjYSA8LSBwbG90X3BjYShtYWNyX25iLCBwbG90X2xhYmVscz1GQUxTRSkKcHAoZmlsZT0ibWFjcm9waGFnZV9zaWRlX2V4cGVyaW1lbnQvbm9ybWJhdGNoX3BjYS5wbmciLCBpbWFnZT1uYl9wY2EkcGxvdCkKYGBgCgojIyBXcml0ZSB0aGUgZGF0YQoKYGBge3Igd3JpdGUsIGZpZy5zaG93PSJoaWRlIn0KbWFjcl93cml0dGVuIDwtIHNtKHdyaXRlX2V4cHQobWFjciwgZXhjZWw9Im1hY3JvcGhhZ2Vfc2lkZV9leHBlcmltZW50L21hY3JvcGhhZ2VfZXhwdC54bHN4IikpCmBgYAoKIyMgUGVyZm9ybSBERQoKYGBge3IgZGUsIGZpZy5zaG93PSJoaWRlIn0KIyN0bXAgPC0gbm9ybWFsaXplX2V4cHQobWFjciwgZmlsdGVyPVRSVUUpCiMjenlfZGUgPC0gZGVzZXFfcGFpcndpc2UodG1wLCBtb2RlbF9iYXRjaD0ic3Zhc2VxIikKIyN6eV9lZCA8LSBlZGdlcl9wYWlyd2lzZSh0bXAsIG1vZGVsX2JhdGNoPSJzdmFzZXEiKQp6eW1vX2RlIDwtIHNtKGFsbF9wYWlyd2lzZShtYWNyLCBtb2RlbF9iYXRjaD0ic3Zhc2VxIiwgZmlsdGVyPVRSVUUsIHBhcmFsbGVsPUZBTFNFKSkKenltb190YWJsZSA8LSBzbShjb21iaW5lX2RlX3RhYmxlcyh6eW1vX2RlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGV4Y2VsPSJtYWNyb3BoYWdlX3NpZGVfZXhwZXJpbWVudC9tYWNyb3BoYWdlX2RlLnhsc3giLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhcmFsbGVsPUZBTFNFKSkKYGBgCgpgYGB7ciBzYXZlbWV9CmlmICghaXNUUlVFKGdldDAoInNraXBfbG9hZCIpKSkgewogIHBhbmRlcjo6cGFuZGVyKHNlc3Npb25JbmZvKCkpCiAgbWVzc2FnZShwYXN0ZTAoIlRoaXMgaXMgaHBnbHRvb2xzIGNvbW1pdDogIiwgZ2V0X2dpdF9jb21taXQoKSkpCiAgbWVzc2FnZShwYXN0ZTAoIlNhdmluZyB0byAiLCBzYXZlZmlsZSkpCiAgdG1wIDwtIHNtKHNhdmVtZShmaWxlbmFtZT1zYXZlZmlsZSkpCn0KYGBgCgpgYGB7ciBsb2FkbWVfYWZ0ZXIsIGV2YWw9RkFMU0V9CnRtcCA8LSBsb2FkbWUoZmlsZW5hbWU9c2F2ZWZpbGUpCmBgYAo=