1 A fresh running of all proteomics tasks

I think I finally worked out all(most?) of the kinks in the processing of DIA data. Thus I want to have a fresh run of all the tasks required to interpret the results.

2 Annotation version: 20180817

2.1 Genome annotation input

2.1.1 Read a gff file

In contrast, it is possible to load most annotations of interest directly from the gff files used in the alignments. More in-depth information for the human transcriptome may be extracted from biomart.

## The old way of getting genome/annotation data
mtb_gff <- "reference/mycobacterium_tuberculosis_h37rv_2.gff.gz"

mtb_genome <- "reference/mtuberculosis_h37rv_genbank.fasta"
mtb_cds <- "reference/mtb_cds.fasta"

mtb_annotations <- sm(load_gff_annotations(mtb_gff, type="gene"))
colnames(mtb_annotations) <- gsub(pattern="\\.", replacement="", x=colnames(mtb_annotations))
mtb_annotations[["description"]] <- gsub(pattern="\\+", replacement=" ",
                                         x=mtb_annotations[["description"]])
mtb_annotations[["function"]] <- gsub(pattern="\\+", replacement=" ",
                                         x=mtb_annotations[["function"]])
rownames(mtb_annotations) <- mtb_annotations[["ID"]]

2.1.2 Download from microbesonline

Apparently I queried the microbesonline too often and now I get an error whenever I try to use them, this disappoints me.

## First figure out the ID for the Mtb genome:
ids <- get_microbesonline_ids("37")
head(ids)
## Mycobacterium tuberculosis H37Rv is the first entry and has id: 83332
## I made a nifty function to do this stuff: load_uniprotws_annotations().
## It is slow, though.
mtb_microbes <- load_microbesonline_annotations(id=83332)
## The species being downloaded is: Mycobacterium tuberculosis H37Rv
## mtb_uniprot_annot <- load_uniprotws_annotations()
mtb_uniprot_annot <- load_uniprot_annotations(file="reference/uniprot_3AUP000001584.txt.gz")
## 
  |                                                                       
  |                                                                 |   0%
  |                                                                       
  |                                                                 |   1%
  |                                                                       
  |=                                                                |   1%
  |                                                                       
  |=                                                                |   2%
  |                                                                       
  |==                                                               |   2%
  |                                                                       
  |==                                                               |   3%
  |                                                                       
  |==                                                               |   4%
  |                                                                       
  |===                                                              |   4%
  |                                                                       
  |===                                                              |   5%
  |                                                                       
  |====                                                             |   5%
  |                                                                       
  |====                                                             |   6%
  |                                                                       
  |====                                                             |   7%
  |                                                                       
  |=====                                                            |   7%
  |                                                                       
  |=====                                                            |   8%
  |                                                                       
  |======                                                           |   8%
  |                                                                       
  |======                                                           |   9%
  |                                                                       
  |======                                                           |  10%
  |                                                                       
  |=======                                                          |  10%
  |                                                                       
  |=======                                                          |  11%
  |                                                                       
  |=======                                                          |  12%
  |                                                                       
  |========                                                         |  12%
  |                                                                       
  |========                                                         |  13%
  |                                                                       
  |=========                                                        |  13%
  |                                                                       
  |=========                                                        |  14%
  |                                                                       
  |=========                                                        |  15%
  |                                                                       
  |==========                                                       |  15%
  |                                                                       
  |==========                                                       |  16%
  |                                                                       
  |===========                                                      |  16%
  |                                                                       
  |===========                                                      |  17%
  |                                                                       
  |===========                                                      |  18%
  |                                                                       
  |============                                                     |  18%
  |                                                                       
  |============                                                     |  19%
  |                                                                       
  |=============                                                    |  19%
  |                                                                       
  |=============                                                    |  20%
  |                                                                       
  |=============                                                    |  21%
  |                                                                       
  |==============                                                   |  21%
  |                                                                       
  |==============                                                   |  22%
  |                                                                       
  |===============                                                  |  22%
  |                                                                       
  |===============                                                  |  23%
  |                                                                       
  |===============                                                  |  24%
  |                                                                       
  |================                                                 |  24%
  |                                                                       
  |================                                                 |  25%
  |                                                                       
  |=================                                                |  25%
  |                                                                       
  |=================                                                |  26%
  |                                                                       
  |=================                                                |  27%
  |                                                                       
  |==================                                               |  27%
  |                                                                       
  |==================                                               |  28%
  |                                                                       
  |===================                                              |  28%
  |                                                                       
  |===================                                              |  29%
  |                                                                       
  |===================                                              |  30%
  |                                                                       
  |====================                                             |  30%
  |                                                                       
  |====================                                             |  31%
  |                                                                       
  |====================                                             |  32%
  |                                                                       
  |=====================                                            |  32%
  |                                                                       
  |=====================                                            |  33%
  |                                                                       
  |======================                                           |  33%
  |                                                                       
  |======================                                           |  34%
  |                                                                       
  |======================                                           |  35%
  |                                                                       
  |=======================                                          |  35%
  |                                                                       
  |=======================                                          |  36%
  |                                                                       
  |========================                                         |  36%
  |                                                                       
  |========================                                         |  37%
  |                                                                       
  |========================                                         |  38%
  |                                                                       
  |=========================                                        |  38%
  |                                                                       
  |=========================                                        |  39%
  |                                                                       
  |==========================                                       |  39%
  |                                                                       
  |==========================                                       |  40%
  |                                                                       
  |==========================                                       |  41%
  |                                                                       
  |===========================                                      |  41%
  |                                                                       
  |===========================                                      |  42%
  |                                                                       
  |============================                                     |  42%
  |                                                                       
  |============================                                     |  43%
  |                                                                       
  |============================                                     |  44%
  |                                                                       
  |=============================                                    |  44%
  |                                                                       
  |=============================                                    |  45%
  |                                                                       
  |==============================                                   |  45%
  |                                                                       
  |==============================                                   |  46%
  |                                                                       
  |==============================                                   |  47%
  |                                                                       
  |===============================                                  |  47%
  |                                                                       
  |===============================                                  |  48%
  |                                                                       
  |================================                                 |  48%
  |                                                                       
  |================================                                 |  49%
  |                                                                       
  |================================                                 |  50%
  |                                                                       
  |=================================                                |  50%
  |                                                                       
  |=================================                                |  51%
  |                                                                       
  |=================================                                |  52%
  |                                                                       
  |==================================                               |  52%
  |                                                                       
  |==================================                               |  53%
  |                                                                       
  |===================================                              |  53%
  |                                                                       
  |===================================                              |  54%
  |                                                                       
  |===================================                              |  55%
  |                                                                       
  |====================================                             |  55%
  |                                                                       
  |====================================                             |  56%
  |                                                                       
  |=====================================                            |  56%
  |                                                                       
  |=====================================                            |  57%
  |                                                                       
  |=====================================                            |  58%
  |                                                                       
  |======================================                           |  58%
  |                                                                       
  |======================================                           |  59%
  |                                                                       
  |=======================================                          |  59%
  |                                                                       
  |=======================================                          |  60%
  |                                                                       
  |=======================================                          |  61%
  |                                                                       
  |========================================                         |  61%
  |                                                                       
  |========================================                         |  62%
  |                                                                       
  |=========================================                        |  62%
  |                                                                       
  |=========================================                        |  63%
  |                                                                       
  |=========================================                        |  64%
  |                                                                       
  |==========================================                       |  64%
  |                                                                       
  |==========================================                       |  65%
  |                                                                       
  |===========================================                      |  65%
  |                                                                       
  |===========================================                      |  66%
  |                                                                       
  |===========================================                      |  67%
  |                                                                       
  |============================================                     |  67%
  |                                                                       
  |============================================                     |  68%
  |                                                                       
  |=============================================                    |  68%
  |                                                                       
  |=============================================                    |  69%
  |                                                                       
  |=============================================                    |  70%
  |                                                                       
  |==============================================                   |  70%
  |                                                                       
  |==============================================                   |  71%
  |                                                                       
  |==============================================                   |  72%
  |                                                                       
  |===============================================                  |  72%
  |                                                                       
  |===============================================                  |  73%
  |                                                                       
  |================================================                 |  73%
  |                                                                       
  |================================================                 |  74%
  |                                                                       
  |================================================                 |  75%
  |                                                                       
  |=================================================                |  75%
  |                                                                       
  |=================================================                |  76%
  |                                                                       
  |==================================================               |  76%
  |                                                                       
  |==================================================               |  77%
  |                                                                       
  |==================================================               |  78%
  |                                                                       
  |===================================================              |  78%
  |                                                                       
  |===================================================              |  79%
  |                                                                       
  |====================================================             |  79%
  |                                                                       
  |====================================================             |  80%
  |                                                                       
  |====================================================             |  81%
  |                                                                       
  |=====================================================            |  81%
  |                                                                       
  |=====================================================            |  82%
  |                                                                       
  |======================================================           |  82%
  |                                                                       
  |======================================================           |  83%
  |                                                                       
  |======================================================           |  84%
  |                                                                       
  |=======================================================          |  84%
  |                                                                       
  |=======================================================          |  85%
  |                                                                       
  |========================================================         |  85%
  |                                                                       
  |========================================================         |  86%
  |                                                                       
  |========================================================         |  87%
  |                                                                       
  |=========================================================        |  87%
  |                                                                       
  |=========================================================        |  88%
  |                                                                       
  |==========================================================       |  88%
  |                                                                       
  |==========================================================       |  89%
  |                                                                       
  |==========================================================       |  90%
  |                                                                       
  |===========================================================      |  90%
  |                                                                       
  |===========================================================      |  91%
  |                                                                       
  |===========================================================      |  92%
  |                                                                       
  |============================================================     |  92%
  |                                                                       
  |============================================================     |  93%
  |                                                                       
  |=============================================================    |  93%
  |                                                                       
  |=============================================================    |  94%
  |                                                                       
  |=============================================================    |  95%
  |                                                                       
  |==============================================================   |  95%
  |                                                                       
  |==============================================================   |  96%
  |                                                                       
  |===============================================================  |  96%
  |                                                                       
  |===============================================================  |  97%
  |                                                                       
  |===============================================================  |  98%
  |                                                                       
  |================================================================ |  98%
  |                                                                       
  |================================================================ |  99%
  |                                                                       
  |=================================================================|  99%
  |                                                                       
  |=================================================================| 100%
## Finished parsing, creating data frame.

2.2 Getting ontology data

mtb_go <- load_microbesonline_go(id=83332)
if (!isTRUE(get0("skip_load"))) {
  pander::pander(sessionInfo())
  message(paste0("This is hpgltools commit: ", get_git_commit()))
  this_save <- paste0(gsub(pattern="\\.Rmd", replace="", x=rmd_file), "-v", ver, ".rda.xz")
  message(paste0("Saving to ", this_save))
  tmp <- sm(saveme(filename=this_save))
}
## If you wish to reproduce this exact build of hpgltools, invoke the following:
## > git clone http://github.com/abelew/hpgltools.git
## > git reset 0b63ce6abcbb822832fe4631b4916f94931d8648
## R> packrat::restore()
## This is hpgltools commit: Wed Sep 5 12:04:45 2018 -0400: 0b63ce6abcbb822832fe4631b4916f94931d8648
## Saving to 01_annotation_20180817-v20180817.rda.xz
LS0tCnRpdGxlOiAiTS50dWJlcmN1bG9zaXMgMjAxODA4MTcgcHJvdGVvbWljczogQ29sbGVjdGluZyBhbm5vdGF0aW9uIGRhdGEuIgphdXRob3I6ICJhdGIgYWJlbGV3QGdtYWlsLmNvbSIKZGF0ZTogImByIFN5cy5EYXRlKClgIgpvdXRwdXQ6CiBodG1sX2RvY3VtZW50OgogIGNvZGVfZG93bmxvYWQ6IHRydWUKICBjb2RlX2ZvbGRpbmc6IHNob3cKICBmaWdfY2FwdGlvbjogdHJ1ZQogIGZpZ19oZWlnaHQ6IDcKICBmaWdfd2lkdGg6IDcKICBoaWdobGlnaHQ6IGRlZmF1bHQKICBrZWVwX21kOiBmYWxzZQogIG1vZGU6IHNlbGZjb250YWluZWQKICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICBzZWxmX2NvbnRhaW5lZDogdHJ1ZQogIHRoZW1lOiByZWFkYWJsZQogIHRvYzogdHJ1ZQogIHRvY19mbG9hdDoKICAgIGNvbGxhcHNlZDogZmFsc2UKICAgIHNtb290aF9zY3JvbGw6IGZhbHNlCi0tLQoKPHN0eWxlPgogIGJvZHkgLm1haW4tY29udGFpbmVyIHsKICAgIG1heC13aWR0aDogMTYwMHB4OwogIH0KPC9zdHlsZT4KCmBgYHtyIG9wdGlvbnMsIGluY2x1ZGU9RkFMU0V9CmlmICghaXNUUlVFKGdldDAoInNraXBfbG9hZCIpKSkgewogIGxpYnJhcnkoaHBnbHRvb2xzKQogIHR0IDwtIGRldnRvb2xzOjpsb2FkX2FsbCgifi9ocGdsdG9vbHMiKQogIGtuaXRyOjpvcHRzX2tuaXQkc2V0KHByb2dyZXNzPVRSVUUsCiAgICAgICAgICAgICAgICAgICAgICAgdmVyYm9zZT1UUlVFLAogICAgICAgICAgICAgICAgICAgICAgIHdpZHRoPTkwLAogICAgICAgICAgICAgICAgICAgICAgIGVjaG89VFJVRSkKICBrbml0cjo6b3B0c19jaHVuayRzZXQoZXJyb3I9VFJVRSwKICAgICAgICAgICAgICAgICAgICAgICAgZmlnLndpZHRoPTgsCiAgICAgICAgICAgICAgICAgICAgICAgIGZpZy5oZWlnaHQ9OCwKICAgICAgICAgICAgICAgICAgICAgICAgZHBpPTk2KQogIG9sZF9vcHRpb25zIDwtIG9wdGlvbnMoZGlnaXRzPTQsCiAgICAgICAgICAgICAgICAgICAgICAgICBzdHJpbmdzQXNGYWN0b3JzPUZBTFNFLAogICAgICAgICAgICAgICAgICAgICAgICAga25pdHIuZHVwbGljYXRlLmxhYmVsPSJhbGxvdyIpCiAgZ2dwbG90Mjo6dGhlbWVfc2V0KGdncGxvdDI6OnRoZW1lX2J3KGJhc2Vfc2l6ZT0xMCkpCiAgdmVyIDwtICIyMDE4MDgxNyIKICBwcmV2aW91c19maWxlIDwtICJpbmRleC5SbWQiCgogIHRtcCA8LSB0cnkoc20obG9hZG1lKGZpbGVuYW1lPXBhc3RlMChnc3ViKHBhdHRlcm49IlxcLlJtZCIsIHJlcGxhY2U9IiIsIHg9cHJldmlvdXNfZmlsZSksICItdiIsIHZlciwgIi5yZGEueHoiKSkpKQogIHJtZF9maWxlIDwtIHBhc3RlMCgiMDFfYW5ub3RhdGlvbl8iLCB2ZXIsICIuUm1kIikKfQpgYGAKCkEgZnJlc2ggcnVubmluZyBvZiBhbGwgcHJvdGVvbWljcyB0YXNrcwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KCkkgdGhpbmsgSSBmaW5hbGx5IHdvcmtlZCBvdXQgYWxsKG1vc3Q/KSBvZiB0aGUga2lua3MgaW4gdGhlIHByb2Nlc3Npbmcgb2YgRElBIGRhdGEuClRodXMgSSB3YW50IHRvIGhhdmUgYSBmcmVzaCBydW4gb2YgYWxsIHRoZSB0YXNrcyByZXF1aXJlZCB0byBpbnRlcnByZXQgdGhlIHJlc3VsdHMuCgojIEFubm90YXRpb24gdmVyc2lvbjogYHIgdmVyYAoKIyMgR2Vub21lIGFubm90YXRpb24gaW5wdXQKCiMjIyBSZWFkIGEgZ2ZmIGZpbGUKCkluIGNvbnRyYXN0LCBpdCBpcyBwb3NzaWJsZSB0byBsb2FkIG1vc3QgYW5ub3RhdGlvbnMgb2YgaW50ZXJlc3QgZGlyZWN0bHkgZnJvbSB0aGUgZ2ZmIGZpbGVzIHVzZWQgaW4KdGhlIGFsaWdubWVudHMuICBNb3JlIGluLWRlcHRoIGluZm9ybWF0aW9uIGZvciB0aGUgaHVtYW4gdHJhbnNjcmlwdG9tZSBtYXkgYmUgZXh0cmFjdGVkIGZyb20gYmlvbWFydC4KCmBgYHtyIGdlbm9tZV9pbnB1dH0KIyMgVGhlIG9sZCB3YXkgb2YgZ2V0dGluZyBnZW5vbWUvYW5ub3RhdGlvbiBkYXRhCm10Yl9nZmYgPC0gInJlZmVyZW5jZS9teWNvYmFjdGVyaXVtX3R1YmVyY3Vsb3Npc19oMzdydl8yLmdmZi5neiIKCm10Yl9nZW5vbWUgPC0gInJlZmVyZW5jZS9tdHViZXJjdWxvc2lzX2gzN3J2X2dlbmJhbmsuZmFzdGEiCm10Yl9jZHMgPC0gInJlZmVyZW5jZS9tdGJfY2RzLmZhc3RhIgoKbXRiX2Fubm90YXRpb25zIDwtIHNtKGxvYWRfZ2ZmX2Fubm90YXRpb25zKG10Yl9nZmYsIHR5cGU9ImdlbmUiKSkKY29sbmFtZXMobXRiX2Fubm90YXRpb25zKSA8LSBnc3ViKHBhdHRlcm49IlxcLiIsIHJlcGxhY2VtZW50PSIiLCB4PWNvbG5hbWVzKG10Yl9hbm5vdGF0aW9ucykpCm10Yl9hbm5vdGF0aW9uc1tbImRlc2NyaXB0aW9uIl1dIDwtIGdzdWIocGF0dGVybj0iXFwrIiwgcmVwbGFjZW1lbnQ9IiAiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHg9bXRiX2Fubm90YXRpb25zW1siZGVzY3JpcHRpb24iXV0pCm10Yl9hbm5vdGF0aW9uc1tbImZ1bmN0aW9uIl1dIDwtIGdzdWIocGF0dGVybj0iXFwrIiwgcmVwbGFjZW1lbnQ9IiAiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHg9bXRiX2Fubm90YXRpb25zW1siZnVuY3Rpb24iXV0pCnJvd25hbWVzKG10Yl9hbm5vdGF0aW9ucykgPC0gbXRiX2Fubm90YXRpb25zW1siSUQiXV0KYGBgCgojIyMgRG93bmxvYWQgZnJvbSBtaWNyb2Jlc29ubGluZQoKQXBwYXJlbnRseSBJIHF1ZXJpZWQgdGhlIG1pY3JvYmVzb25saW5lIHRvbyBvZnRlbiBhbmQgbm93IEkgZ2V0IGFuIGVycm9yCndoZW5ldmVyIEkgdHJ5IHRvIHVzZSB0aGVtLCB0aGlzIGRpc2FwcG9pbnRzIG1lLgoKYGBge3IgbWljcm9iZXNvbmxpbmUsIGV2YWw9RkFMU0V9CiMjIEZpcnN0IGZpZ3VyZSBvdXQgdGhlIElEIGZvciB0aGUgTXRiIGdlbm9tZToKaWRzIDwtIGdldF9taWNyb2Jlc29ubGluZV9pZHMoIjM3IikKaGVhZChpZHMpCiMjIE15Y29iYWN0ZXJpdW0gdHViZXJjdWxvc2lzIEgzN1J2IGlzIHRoZSBmaXJzdCBlbnRyeSBhbmQgaGFzIGlkOiA4MzMzMgpgYGAKCmBgYHtyIGdlbmJhbmt9CiMjIEkgbWFkZSBhIG5pZnR5IGZ1bmN0aW9uIHRvIGRvIHRoaXMgc3R1ZmY6IGxvYWRfdW5pcHJvdHdzX2Fubm90YXRpb25zKCkuCiMjIEl0IGlzIHNsb3csIHRob3VnaC4KbXRiX21pY3JvYmVzIDwtIGxvYWRfbWljcm9iZXNvbmxpbmVfYW5ub3RhdGlvbnMoaWQ9ODMzMzIpCiMjIG10Yl91bmlwcm90X2Fubm90IDwtIGxvYWRfdW5pcHJvdHdzX2Fubm90YXRpb25zKCkKbXRiX3VuaXByb3RfYW5ub3QgPC0gbG9hZF91bmlwcm90X2Fubm90YXRpb25zKGZpbGU9InJlZmVyZW5jZS91bmlwcm90XzNBVVAwMDAwMDE1ODQudHh0Lmd6IikKYGBgCgojIyBHZXR0aW5nIG9udG9sb2d5IGRhdGEKCmBgYHtyIG9udG9sb2d5LCBldmFsPUZBTFNFfQptdGJfZ28gPC0gbG9hZF9taWNyb2Jlc29ubGluZV9nbyhpZD04MzMzMikKYGBgCgpgYGB7ciBzYXZlbWV9CmlmICghaXNUUlVFKGdldDAoInNraXBfbG9hZCIpKSkgewogIHBhbmRlcjo6cGFuZGVyKHNlc3Npb25JbmZvKCkpCiAgbWVzc2FnZShwYXN0ZTAoIlRoaXMgaXMgaHBnbHRvb2xzIGNvbW1pdDogIiwgZ2V0X2dpdF9jb21taXQoKSkpCiAgdGhpc19zYXZlIDwtIHBhc3RlMChnc3ViKHBhdHRlcm49IlxcLlJtZCIsIHJlcGxhY2U9IiIsIHg9cm1kX2ZpbGUpLCAiLXYiLCB2ZXIsICIucmRhLnh6IikKICBtZXNzYWdlKHBhc3RlMCgiU2F2aW5nIHRvICIsIHRoaXNfc2F2ZSkpCiAgdG1wIDwtIHNtKHNhdmVtZShmaWxlbmFtZT10aGlzX3NhdmUpKQp9CmBgYAo=