1 TODO

Remove MSstats logging. – done
Make explicit spearman correlations between methods. – done
- Do both for all data and for the top-50/bottom-50 – done, but weird.
Make 100% certain that the samples are annotated correctly. – done.
Limma/MSstats/EdgeR venn disagram for all up in CF
- Repeat in all down CF
- Repeat with a cutoff, top-50
Individual plots for P/PE proteins.

2 Analyzing data from openMS and friends.

In preprocessing_comet_highres.Rmd, I used the openMS tutorials and supplemental materials from a couple papers to hopefully correctly perform the various preprocessing tasks required to extract intensity data from DIA/SWATH transitions.

The final steps of that process combined the transition intensities from every sample into a metadata frile (results/tric/HCD_meta.tsv), an intensity matrix (results/tric/HCD_outmatrix.tsv), and a feature aligned output matrix (results/tric/aligned_comet_HCD.tsv).

My reading of the SWATH2stats and MSstats source code suggests to me that the log2(intensities) of the feature aligned data are our final proxy for protein abundance. At first glance, this suggests to me that these data might follow a distribution similar to RNASeq data (negative binomial, but perhaps with a bigger tail?). In addition, by the time we use tric on the data, we have a count matrix and sample annotation data frames which look remarkably similar to those used in a RNASeq expressionset. Indeed, by the end of the MSstats processing, it creates a MSnSet class of its own which uses fData/exprs/pData.

For the curious, my reasoning for saying that the log intensities are our proxy for abundance comes from MSstats/R/DataProcess.R in a clause which looks like:

if (logTrans == 2) {
  work[["ABUNDANCE"]] <- log2(work[["ABUNDANCE"]])
} else if (logTrans == 10) {
  work[["ABUNDANCE"]] <- log10(work[["ABUNDANCE"]])
} else {
  ## Above there was a check for only log 2 and 10, but we can do e if we want.
  ## I might go back up there and remove that check. Long live e! 2.718282 rules!
  work[["ABUNDANCE"]] <- log(work[["ABUNDANCE"]]) / log(logTrans)
}

(Note: I added the natural log to the set of conditions, but otherwise the logic is unchanged.)

With that in mind, I want to use some tools with which I am familiar in order to try to understand these data. Therefore I will first attempt to coerce my tric aligned data and annotations into a ‘normal’ expressionset. Then I want to do some diagnostic plots which, if I am wrong and these distributions are not as expected, will be conceptually incorrect (I don’t yet think I am wrong).

2.1 Sample annotation via SWATH2stats

I am using the SWATH2stats vignette as my primary source of information. Thus I see that it uses the OpenSWATH_SM3_GoldStandardAutomatedResults_human_peakgroups.txt which has a format nearly identical to my tric output matrix. Thus for the moment I will assume that the proper input for SWATH2stats is ‘results/tric/comet_HCD.tsv’ and not the metadata nor output matrix.

I keep a sample sheet of all the DIA samples used in this analysis in ‘sample_sheets/dia_samples.xlsx’ It should contain all the other required data with one important caveat, I removed 1 sample by ‘commenting’ it (e.g. prefixing it with ‘##’ – which is an admittedly dumb thing to do in an excel file.

One last caveat: I hacked the SWATH2stats sample_annotation() function to add a couple columns in an attempt to make it a little more robust when faced with sample sheets with differently named columns.

In addition, SWATH2stats provides some nice filtering and combination functions which should be considered when generating various expressionset data structures later.

tric_data <- read.csv("results/tric_201805/comet_HCD.tsv", sep="\t")

sample_annot <- openxlsx::read.xlsx("sample_sheets/Mtb_dia_samples.xlsx")
rownames(sample_annot) <- make.names(sample_annot[["sampleid"]], unique=TRUE)
## Drop samples starting with comments
keep_idx <- ! grepl(pattern="##", x=sample_annot[["sampleid"]])
sample_annot <- sample_annot[keep_idx, ]
expt_idx <- sample_annot[["expt_id"]] == "may2018" | sample_annot[["expt_id"]] == "mar2018"
expt_idx[is.na(sample_annot[["expt_id"]])] <- FALSE
sample_annot <- sample_annot[expt_idx, ]
mz_idx <- sample_annot[["windowsize"]] == "8"
sample_annot <- sample_annot[mz_idx, ]
## Set the mzXML column to match the filename column in the data.
devtools::load_all("~/scratch/git/SWATH2stats")

## Loading SWATH2stats

## s2s, my witty way of shortening SWATH2stats...
s2s_exp <- sample_annotation(data=tric_data,
                             sample_annotation=sample_annot,
                             fullpeptidename_column="fullunimodpeptidename")

## Found the same mzXML files in the annotations and data.

Now I have a couple data structures which should prove useful for the metrics provided by SWATH2stats, MSstats, and my own hpgltools.

3 SWATH2stats continued

Lets return to some of the metrics provided by swath2stats.

## Get correlations on a sample by sample basis
pp(file="images/20180523_swath2stats_sample_cor.png")

## Going to write the image to: images/20180523_swath2stats_sample_cor.png when dev.off() is called.

sample_cor <- plot_correlation_between_samples(s2s_exp,
                                               fun.aggregate=sum,
                                               column.values="intensity")
dev.off()

## png 
##   2

sample_cond_rep_cor <- plot_correlation_between_samples(s2s_exp,
                                                        comparison=transition_group_id ~
                                                          condition + bioreplicate + run,
                                                        fun.aggregate=sum,
                                                        column.values="intensity")

## I am a little concerned that these values do not seem to change when I took
## filtered/normalized data.  So I am rerunning them manually for a moment --
## perhaps I messed something up when I rewrote portions of the
## sample_annotation() function in SWATH2stats.

## ahh I think I see the problem.  The default value for fun.aggregate is NULL,
## which causes dcast to default to length.  I think this is not likely to be
## valid for this data.  I am not certain, however, what is the appropriate
## function.  If I had to guess, I would go with sum()?

assess_decoy_rate(s2s_exp)

## Number of non-decoy peptides: 18096
## Number of decoy peptides: 1436
## Decoy rate: 0.0794

## This seems a bit high to me, yesno?
fdr_overall <- assess_fdr_overall(s2s_exp, output="Rconsole", plot=TRUE)

byrun_fdr <- assess_fdr_byrun(s2s_exp, FFT=0.7, plot=TRUE, output="Rconsole")

## The average FDR by run on assay level is 0.011

## The average FDR by run on peptide level is 0.012

## The average FDR by run on protein level is 0.049

chosen_mscore <- mscore4assayfdr(s2s_exp, FFT=0.7, fdr_target=0.02)

## Target assay FDR: 0.02

## Required overall m-score cutoff:0.0039811
## achieving assay FDR =0.0186

prot_score <- mscore4protfdr(s2s_exp, FFT=0.7, fdr_target=0.02)

## Target protein FDR:0.02

## Required overall m-score cutoff:0.00089125
## achieving protein FDR =0.0194

mscore_filtered <- filter_mscore(s2s_exp, chosen_mscore)

## Dimension difference: 15921, 0

data_filtered_mscore <- filter_mscore_freqobs(s2s_exp, 0.01, 0.8, rm.decoy=FALSE)

## Treshold, peptides need to have been quantified in more conditions than: 18.4

## Fraction of peptides selected: 1

## Dimension difference: 0, 0

data_filtered_fdr <- filter_mscore_fdr(mscore_filtered, FFT=0.7,
                                       overall_protein_fdr_target=prot_score,
                                       upper_overall_peptide_fdr_limit=0.05)

## Target protein FDR:0.000891250938133746

## Required overall m-score cutoff:0.01
## achieving protein FDR =0

## filter_mscore_fdr is filtering the data...

## finding m-score cutoff to achieve desired protein FDR in protein master list..

## finding m-score cutoff to achieve desired global peptide FDR..

## Target peptide FDR: 0.05

## Required overall m-score cutoff: 0.01

## Achieving peptide FDR: 0

## Proteins selected: 
## Total proteins selected: 2583
## Thereof target proteins: 2583
## Thereof decoy proteins: 0

## Peptides mapping to these protein entries selected:
## Total mapping peptides: 16956
## Thereof target peptides: 16956
## Thereof decoy peptides: 0

## Total peptides selected from:
## Total peptides: 16956
## Thereof target peptides: 16956
## Thereof decoy peptides: 0

## Individual run FDR quality of the peptides was not calculated
## as not every run contains a decoy.

## The decoys have been removed from the returned data.

only_proteotypic <- filter_proteotypic_peptides(data_filtered_fdr)

## Number of proteins detected: 2544
## Protein identifiers: Rv2224c, Rv2220, Rv3267, Rv2241, Rv1613, Rv2427c

## Number of proteins detected that are supported by a proteotypic peptide: 2505
## Number of proteotypic peptides detected: 16822

all_filtered <- filter_all_peptides(only_proteotypic)

## Number of proteins detected: 2505

## First 6 protein identifiers: Rv2224c, Rv2220, Rv3267, Rv2241, Rv1613, Rv2427c

only_strong <- filter_on_max_peptides(data=all_filtered, n_peptides=10)

## Before filtering: 
##   Number of proteins: 2505
##   Number of peptides: 16822
## 
## Percentage of peptides removed: 21.43%
## 
## After filtering: 
##   Number of proteins: 2491
##   Number of peptides: 13217

only_minimum <- filter_on_min_peptides(data=only_strong, n_peptides=3)

## Before filtering: 
##   Number of proteins: 2491
##   Number of peptides: 13217
## 
## Percentage of peptides removed: 0.03%
## 
## After filtering: 
##   Number of proteins: 2329
##   Number of peptides: 13213

## I think these matrixes are probably smarter to use than the raw outmatrix from tric.
## But I am not a fan of rerwriting the sample column names.
protein_matrix_all <- write_matrix_proteins(
  s2s_exp, write.csv=TRUE,
  filename=paste0("results/swath2stats_", ver, "/protein_all.csv"))

## Protein overview matrix results/swath2stats_20180528/protein_all.csv written to working folder.

dim(protein_matrix_all)

## [1] 3614   24

protein_matrix_mscore <- write_matrix_proteins(
  mscore_filtered, write.csv=TRUE,
  filename=paste0("results/swath2stats_", ver, "/protein_matrix_mscore.csv"))

## Protein overview matrix results/swath2stats_20180528/protein_matrix_mscore.csv written to working folder.

dim(protein_matrix_mscore)

## [1] 2583   24

peptide_matrix_mscore <- write_matrix_peptides(
  mscore_filtered, write.csv=TRUE,
  filename=paste0("results/swath2stats_", ver, "/peptide_matrix_mscore.csv"))

## Peptide overview matrix results/swath2stats_20180528/peptide_matrix_mscore.csv written to working folder.

dim(peptide_matrix_mscore)

## [1] 16956    24

protein_matrix_minimum <- write_matrix_proteins(
  only_minimum, write.csv=TRUE,
  filename=paste0("results/swath2stats_", ver, "/protein_matrix_minimum.csv"))

## Protein overview matrix results/swath2stats_20180528/protein_matrix_minimum.csv written to working folder.

dim(protein_matrix_minimum)

## [1] 2329   24

peptide_matrix_minimum <- write_matrix_peptides(
  only_minimum, write.csv=TRUE,
  filename=paste0("results/swath2stats_", ver, "/peptide_matrix_minimum.csv"))

## Peptide overview matrix results/swath2stats_20180528/peptide_matrix_minimum.csv written to working folder.

dim(peptide_matrix_minimum)

## [1] 115412     24

rt_cor <- plot_correlation_between_samples(only_minimum,
                                           column.values="intensity",
                                           fun.aggregate=sum)

## I have no effing clue what this plot means.
variation <- plot_variation(only_minimum, fun.aggregate=sum,
                            comparison=transition_group_id ~ condition)

## Something in SWATH2stats::disaggregate was written poorly and is looking for
## a variable named 'cols'
cols <- colnames(only_minimum)
disaggregated <- disaggregate(only_minimum, all.columns=TRUE)

## The library contains 6 transitions per precursor.
##                   
## The data table was transformed into a table containing one row per transition.

msstats_input <- convert4MSstats(disaggregated)

## One or several columns required by MSstats were not in the data. The columns were created and filled with NAs.
## Missing columns: fragmention, productcharge, isotopelabeltype

## isotopelabeltype was filled with light.

##alfq_input <- sm(convert4aLFQ(disaggregated))
##mapdia_input <- sm(convert4mapDIA(disaggregated, RT=TRUE))

3.1 MSstats

msstats.org seems to provide a complete solution for performing reasonable metrics of this data.

I am currently reading: http://msstats.org/wp-content/uploads/2017/01/MSstats_v3.7.3_manual.pdf

I made some moderately intrusive changes to MSstats to make it clearer, as well.

devtools::load_all("~/scratch/git/MSstats")

## Loading MSstats

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

## Warning: character(0)

msstats_quant <- sm(dataProcess(msstats_input))
msstats_plots <- sm(dataProcessPlots(msstats_quant, type="QCPLOT"))

my_levels <- levels(as.factor(msstats_input$condition))
my_levels

## [1] "comp_cf"     "comp_whole"  "delta_cf"    "delta_whole" "wt_cf"      
## [6] "wt_whole"

comparisons <- ghetto_contrast_matrix(
  numerators=c("wt_cf", "delta_cf", "comp_cf",
               "delta_cf", "comp_cf", "delta_whole",
               "comp_whole"),
  denominators=c("wt_whole", "delta_whole", "comp_whole",
                 "wt_cf", "wt_cf", "wt_whole",
                 "wt_whole"))
results <- list()
for (c in 1:length(comparisons)) {
  name <- rownames(comparisons)[c]
  message("Starting ", name)
  comp <- comparisons[c, ]
  comparison <- t(as.matrix(comp))
  rownames(comparison) <- name
  results[name] <- sm(MSstats::groupComparison(contrast.matrix=comparison,
                                               data=msstats_quant))
}

## Starting wt_cf-wt_whole

## Warning in results[name] <- sm(MSstats::groupComparison(contrast.matrix =
## comparison, : number of items to replace is not a multiple of replacement
## length

## Starting delta_cf-delta_whole

## Warning in results[name] <- sm(MSstats::groupComparison(contrast.matrix =
## comparison, : number of items to replace is not a multiple of replacement
## length

## Starting comp_cf-comp_whole

## Warning in results[name] <- sm(MSstats::groupComparison(contrast.matrix =
## comparison, : number of items to replace is not a multiple of replacement
## length

## Starting delta_cf-wt_cf

## Warning in results[name] <- sm(MSstats::groupComparison(contrast.matrix =
## comparison, : number of items to replace is not a multiple of replacement
## length

## Starting comp_cf-wt_cf

## Warning in results[name] <- sm(MSstats::groupComparison(contrast.matrix =
## comparison, : number of items to replace is not a multiple of replacement
## length

## Starting delta_whole-wt_whole

## Warning in results[name] <- sm(MSstats::groupComparison(contrast.matrix =
## comparison, : number of items to replace is not a multiple of replacement
## length

## Starting comp_whole-wt_whole

## Warning in results[name] <- sm(MSstats::groupComparison(contrast.matrix =
## comparison, : number of items to replace is not a multiple of replacement
## length

## Starting NA

## Error in comparisons[c, ]: subscript out of bounds

3.1.1 P/PE protein QC plots for Yan

Yan asked for the p/pe protein qc plots. ok. I changed the dataProcessPlots to return something useful, so that should be possible now.

pe_genes <- read.table("reference/annotated_pe_genes.txt")[[1]]

## Unfortunately, the names did not get set in my changed version of dataProcessPlots...
plotlst <- msstats_plots$QCPLOT
available_plots <- gsub(pattern="^1/", replacement="", x=levels(msstats_quant$ProcessedData$PROTEIN))
names(plotlst) <- available_plots

pe_in_avail_idx <- pe_genes %in% available_plots
pe_in_avail <- pe_genes[pe_in_avail_idx]
pe_plots <- plotlst[pe_in_avail]
pdf(file="pe_qc_plots.pdf")
for (p in 1:length(pe_plots)) {
  plot(pe_plots[[p]])
}
dev.off()

4 Create hpgltools expressionset

Since I am not certain I understand these data, I will take the intensities from SWATH2stats, metadata, and annotation data; attempt to create a ‘normal’ expressionset; poke at it to see what I can learn.

4.1 Massaging the metadata

I want to use the same metadata as were used for MSstats. It has a few important differences from the requirements of hpgltools: pretty much only that I do not allow rownames/sampleIDs to start with a number.

metadata <- sample_annot
metadata[["sampleid"]] <- paste0("s", metadata[["sampleid"]])
rownames(metadata) <- metadata[["sampleid"]]

4.2 Massaging the gene annotation data and adding the msstats data.

I have my own annotation data from the gff file/microbesonline/whatever, I can add the MSstats result to it so that later I can print them all together.

## Here is a neat little thing I can do:  Add the MSstats results to my annotation data.
## Then when I print out the tables of the limma/etc results, they MSstats
## results will come along for free.
big_table <- results[[1]]
for (r in 2:length(results)) {
  big_table <- merge(big_table, results[[r]], by="Protein")
}

## Warning in merge.data.frame(big_table, results[[r]], by = "Protein"):
## column names 'Label.x', 'log2FC.x', 'SE.x', 'Tvalue.x', 'DF.x', 'pvalue.x',
## 'adj.pvalue.x', 'issue.x', 'MissingPercentage.x', 'ImputationPercentage.x',
## 'Label.y', 'log2FC.y', 'SE.y', 'Tvalue.y', 'DF.y', 'pvalue.y',
## 'adj.pvalue.y', 'issue.y', 'MissingPercentage.y', 'ImputationPercentage.y'
## are duplicated in the result

## Warning in merge.data.frame(big_table, results[[r]], by = "Protein"):
## column names 'Label.x', 'log2FC.x', 'SE.x', 'Tvalue.x', 'DF.x', 'pvalue.x',
## 'adj.pvalue.x', 'issue.x', 'MissingPercentage.x', 'ImputationPercentage.x',
## 'Label.y', 'log2FC.y', 'SE.y', 'Tvalue.y', 'DF.y', 'pvalue.y',
## 'adj.pvalue.y', 'issue.y', 'MissingPercentage.y', 'ImputationPercentage.y'
## are duplicated in the result

## Warning in merge.data.frame(big_table, results[[r]], by = "Protein"):
## column names 'Label.x', 'log2FC.x', 'SE.x', 'Tvalue.x', 'DF.x', 'pvalue.x',
## 'adj.pvalue.x', 'issue.x', 'MissingPercentage.x', 'ImputationPercentage.x',
## 'Label.y', 'log2FC.y', 'SE.y', 'Tvalue.y', 'DF.y', 'pvalue.y',
## 'adj.pvalue.y', 'issue.y', 'MissingPercentage.y', 'ImputationPercentage.y',
## 'Label.x', 'log2FC.x', 'SE.x', 'Tvalue.x', 'DF.x', 'pvalue.x',
## 'adj.pvalue.x', 'issue.x', 'MissingPercentage.x', 'ImputationPercentage.x',
## 'Label.y', 'log2FC.y', 'SE.y', 'Tvalue.y', 'DF.y', 'pvalue.y',
## 'adj.pvalue.y', 'issue.y', 'MissingPercentage.y', 'ImputationPercentage.y'
## are duplicated in the result

## Warning in merge.data.frame(big_table, results[[r]], by = "Protein"):
## column names 'Label.x', 'log2FC.x', 'SE.x', 'Tvalue.x', 'DF.x', 'pvalue.x',
## 'adj.pvalue.x', 'issue.x', 'MissingPercentage.x', 'ImputationPercentage.x',
## 'Label.y', 'log2FC.y', 'SE.y', 'Tvalue.y', 'DF.y', 'pvalue.y',
## 'adj.pvalue.y', 'issue.y', 'MissingPercentage.y', 'ImputationPercentage.y',
## 'Label.x', 'log2FC.x', 'SE.x', 'Tvalue.x', 'DF.x', 'pvalue.x',
## 'adj.pvalue.x', 'issue.x', 'MissingPercentage.x', 'ImputationPercentage.x',
## 'Label.y', 'log2FC.y', 'SE.y', 'Tvalue.y', 'DF.y', 'pvalue.y',
## 'adj.pvalue.y', 'issue.y', 'MissingPercentage.y', 'ImputationPercentage.y'
## are duplicated in the result

rownames(big_table) <- big_table[["Protein"]]
big_table <- big_table[, -1]

mtb_annotations_with_msstats <- merge(mtb_annotations, big_table,
                                      by="row.names", all.x=TRUE)

## Error in merge(mtb_annotations, big_table, by = "row.names", all.x = TRUE): object 'mtb_annotations' not found

rownames(mtb_annotations_with_msstats) <- mtb_annotations_with_msstats[["Row.names"]]

## Error in eval(expr, envir, enclos): object 'mtb_annotations_with_msstats' not found

mtb_annotations_with_msstats <- mtb_annotations_with_msstats[, -1]

## Error in eval(expr, envir, enclos): object 'mtb_annotations_with_msstats' not found

4.3 Massaging the intensity matrix

I do not want the \1 before the protein names, I already merged them into one entry per gene vis SWATH2stats.

prot_mtrx <- read.csv(paste0("results/swath2stats_", ver, "/protein_matrix_minimum.csv"))
rownames(prot_mtrx) <- gsub(pattern="^1\\/", replacement="", x=prot_mtrx[["proteinname"]])
prot_mtrx <- prot_mtrx[, -1]
## Important question: Did SWATH2stats reorder my data?
fun <- gsub(pattern="^.*_(2018.*$)", replacement="\\1", x=colnames(prot_mtrx))
colnames(prot_mtrx) <- paste0("s", fun)

4.4 Merge the pieces

Now we should have sufficient pieces to make an expressionset.

## Drop the metadata not in the protein matrix:
## And ensure that they are the same order.
reordered <- colnames(prot_mtrx)
metadata <- metadata[reordered, ]

protein_expt <- create_expt(metadata,
                            count_dataframe=prot_mtrx,
                            gene_info=mtb_annotations_with_msstats)

## Reading the sample metadata.

## The sample definitions comprises: 23, 21 rows, columns.

## Error in create_expt(metadata, count_dataframe = prot_mtrx, gene_info = mtb_annotations_with_msstats): object 'mtb_annotations_with_msstats' not found

whole_expt <- subset_expt(protein_expt, subset="collectiontype=='whole'")

## Error in sampleNames(expt): object 'protein_expt' not found

cf_expt <- subset_expt(protein_expt, subset="collectiontype=='cf'")

## Error in sampleNames(expt): object 'protein_expt' not found

protein_metrics <- sm(graph_metrics(protein_expt))

## Error in corheat[["plot"]]: subscript out of bounds

protein_norm <- sm(normalize_expt(protein_expt, transform="log2", convert="cpm",
                                  norm="quant", filter=TRUE))

## Error in normalize_expt(protein_expt, transform = "log2", convert = "cpm", : object 'protein_expt' not found

protein_norm_metrics <- sm(graph_metrics(protein_norm))

## Error in corheat[["plot"]]: subscript out of bounds

protein_fsva <- sm(normalize_expt(protein_expt, transform="log2", convert="cpm",
                                  batch="fsva", filter=TRUE))

## Error in normalize_expt(protein_expt, transform = "log2", convert = "cpm", : object 'protein_expt' not found

protein_fsva_metrics <- sm(graph_metrics(protein_fsva))

## Error in corheat[["plot"]]: subscript out of bounds

whole_metrics <- sm(graph_metrics(whole_expt))

## Error in corheat[["plot"]]: subscript out of bounds

whole_norm <- sm(normalize_expt(whole_expt, transform="log2", convert="cpm",
                                  norm="quant", filter=TRUE))

## Error in normalize_expt(whole_expt, transform = "log2", convert = "cpm", : object 'whole_expt' not found

whole_norm_metrics <- sm(graph_metrics(whole_norm))

## Error in corheat[["plot"]]: subscript out of bounds

whole_fsva <- sm(normalize_expt(whole_expt, transform="log2", convert="cpm",
                                  batch="fsva", filter=TRUE))

## Error in normalize_expt(whole_expt, transform = "log2", convert = "cpm", : object 'whole_expt' not found

whole_fsva_metrics <- sm(graph_metrics(whole_fsva))

## Error in corheat[["plot"]]: subscript out of bounds

cf_metrics <- sm(graph_metrics(cf_expt))

## Error in corheat[["plot"]]: subscript out of bounds

cf_norm <- sm(normalize_expt(cf_expt, transform="log2", convert="cpm",
                                  norm="quant", filter=TRUE))

## Error in normalize_expt(cf_expt, transform = "log2", convert = "cpm", : object 'cf_expt' not found

cf_norm_metrics <- sm(graph_metrics(cf_norm))

## Error in corheat[["plot"]]: subscript out of bounds

cf_fsva <- sm(normalize_expt(cf_expt, transform="log2", convert="cpm",
                                  batch="fsva", filter=TRUE))

## Error in normalize_expt(cf_expt, transform = "log2", convert = "cpm", : object 'cf_expt' not found

cf_fsva_metrics <- sm(graph_metrics(cf_fsva))

## Error in corheat[["plot"]]: subscript out of bounds

pp(image=protein_metrics$libsize, file="images/20180523_libsize.png")

## Error in pp(image = protein_metrics$libsize, file = "images/20180523_libsize.png"): object 'protein_metrics' not found

pp(image=protein_norm_metrics$pcaplot, file="images/20180523_norm_pca.png")

## Error in pp(image = protein_norm_metrics$pcaplot, file = "images/20180523_norm_pca.png"): object 'protein_norm_metrics' not found

pp(image=protein_fsva_metrics$pcaplot, file="images/20180523_fsva_pca.png")

## Error in pp(image = protein_fsva_metrics$pcaplot, file = "images/20180523_fsva_pca.png"): object 'protein_fsva_metrics' not found

pp(image=protein_norm_metrics$corheat, file="images/20180523_norm_corheat.png")

## Error in pp(image = protein_norm_metrics$corheat, file = "images/20180523_norm_corheat.png"): object 'protein_norm_metrics' not found

pp(image=protein_metrics$density, file="images/20180523_raw_density.png")

## Error in pp(image = protein_metrics$density, file = "images/20180523_raw_density.png"): object 'protein_metrics' not found

pp(image=protein_metrics$boxplot, file="images/20180523_boxplot.png")

## Error in pp(image = protein_metrics$boxplot, file = "images/20180523_boxplot.png"): object 'protein_metrics' not found

pp(image=whole_metrics$libsize, file="images/20180523_whole_libsize.png")

## Error in pp(image = whole_metrics$libsize, file = "images/20180523_whole_libsize.png"): object 'whole_metrics' not found

pp(image=whole_norm_metrics$pcaplot, file="images/20180523_whole_norm_pca.png")

## Error in pp(image = whole_norm_metrics$pcaplot, file = "images/20180523_whole_norm_pca.png"): object 'whole_norm_metrics' not found

pp(image=whole_fsva_metrics$pcaplot, file="images/20180523_whole_fsva_pca.png")

## Error in pp(image = whole_fsva_metrics$pcaplot, file = "images/20180523_whole_fsva_pca.png"): object 'whole_fsva_metrics' not found

pp(image=whole_norm_metrics$corheat, file="images/20180523_whole_norm_corheat.png")

## Error in pp(image = whole_norm_metrics$corheat, file = "images/20180523_whole_norm_corheat.png"): object 'whole_norm_metrics' not found

pp(image=whole_metrics$density, file="images/20180523_whole_raw_density.png")

## Error in pp(image = whole_metrics$density, file = "images/20180523_whole_raw_density.png"): object 'whole_metrics' not found

pp(image=whole_metrics$boxplot, file="images/20180523_whole_boxplot.png")

## Error in pp(image = whole_metrics$boxplot, file = "images/20180523_whole_boxplot.png"): object 'whole_metrics' not found

pp(image=cf_metrics$libsize, file="images/20180523_libsize.png")

## Error in pp(image = cf_metrics$libsize, file = "images/20180523_libsize.png"): object 'cf_metrics' not found

pp(image=cf_norm_metrics$pcaplot, file="images/20180523_norm_pca.png")

## Error in pp(image = cf_norm_metrics$pcaplot, file = "images/20180523_norm_pca.png"): object 'cf_norm_metrics' not found

pp(image=cf_fsva_metrics$pcaplot, file="images/20180523_fsva_pca.png")

## Error in pp(image = cf_fsva_metrics$pcaplot, file = "images/20180523_fsva_pca.png"): object 'cf_fsva_metrics' not found

pp(image=cf_norm_metrics$corheat, file="images/20180523_norm_corheat.png")

## Error in pp(image = cf_norm_metrics$corheat, file = "images/20180523_norm_corheat.png"): object 'cf_norm_metrics' not found

pp(image=cf_metrics$density, file="images/20180523_raw_density.png")

## Error in pp(image = cf_metrics$density, file = "images/20180523_raw_density.png"): object 'cf_metrics' not found

pp(image=cf_metrics$boxplot, file="images/20180523_boxplot.png")

## Error in pp(image = cf_metrics$boxplot, file = "images/20180523_boxplot.png"): object 'cf_metrics' not found

5 Attempt some quantification comparisons?

pairwise_filt <- sm(normalize_expt(protein_expt, filter=TRUE))

## Error in normalize_expt(protein_expt, filter = TRUE): object 'protein_expt' not found

pairwise_comp <- sm(all_pairwise(pairwise_filt, model_batch="fsva", force=TRUE))

## Error in get_model_adjust(input, estimate_type = model_batch, surrogates = surrogates): object 'pairwise_filt' not found

6 For each msstats run, do a DE table

keepers <- list(
  "wt_cf_minus_wt_whole" = c("wt_cf", "wt_whole"),
  "delta_cf_minus_delta_whole" = c("delta_cf", "delta_whole"),
  "comp_cf_minus_comp_whole" = c("comp_cf", "comp_whole"),
  "delta_cf_minus_wt_cf" = c("delta_cf", "wt_cf"),
  "comp_cf_minus_wt_cf" = c("comp_cf", "wt_cf"),
  "delta_whole_minus_wt_whole" = c("delta_whole", "wt_whole"),
  "comp_whole_minus_wt_whole" = c("comp_whole", "wt_whole"))
pairwise_tables <- sm(combine_de_tables(
  pairwise_comp,
  excel=paste0("excel/pairwise_de_with_msstats-v", ver, ".xlsx"),
  keepers=keepers))

## Error in combine_de_tables(pairwise_comp, excel = paste0("excel/pairwise_de_with_msstats-v", : object 'pairwise_comp' not found

pairwise_sig <- sm(extract_significant_genes(
  pairwise_tables,
  excel=paste0("excel/test_pairwise_sig_with_msstats-v", ver, ".xlsx")))

## Error in extract_significant_genes(pairwise_tables, excel = paste0("excel/test_pairwise_sig_with_msstats-v", : object 'pairwise_tables' not found

droppers <- c("undefined")
names(droppers) <- "log2fc"
pairwise_onlyall <- sm(combine_de_tables(
  pairwise_comp,
  excel=paste0("excel/test_pairwise_de_only_msstats-v", ver, ".xlsx"),
  keepers=keepers,
  excludes=droppers))

solo_proteins <- features_in_single_condition(protein_expt)

## Error in pData(expt): object 'protein_expt' not found

proteins_only_cf <- solo_proteins[["solo_this"]][["wt_cf"]]

## Error in eval(expr, envir, enclos): object 'solo_proteins' not found

proteins_only_cf

## Error in eval(expr, envir, enclos): object 'proteins_only_cf' not found

proteins_only_whole <- solo_proteins[["solo_this"]][["wt_whole"]]

## Error in eval(expr, envir, enclos): object 'solo_proteins' not found

length(proteins_only_whole)

## Error in eval(expr, envir, enclos): object 'proteins_only_whole' not found

6.1 Compare hpgltools/MSstats

Can we compare limma and riends to MSstats?

6.1.1 wt cf / wt whole

hpgl_table <- pairwise_tables[["data"]][["wt_cf_vs_wt_whole"]]
msstats_table <- results[["


msstats
msstats_table <- msstats_comparison$ComparisonResult
rownames(msstats_table) <- gsub(pattern="^1/", replacement="", x=msstats_table$Protein)

merged_table <- merge(hpgl_table, msstats_table, by="row.names")
write.csv(file="images/merged_table.csv", merged_table)
cor.test(merged_table$limma_logfc, merged_table$log2FC)
cor.test(merged_table$limma_logfc, merged_table$log2FC, method="spearman")
cor.test(merged_table$deseq_logfc, merged_table$log2FC)
cor.test(merged_table$deseq_logfc, merged_table$log2FC, method="spearman")
cor.test(merged_table$edger_logfc, merged_table$log2FC)
cor.test(merged_table$edger_logfc, merged_table$log2FC, method="spearman")

combined_table <- pairwise_tables$data[[1]]
undefined_idx <- combined_table[["log2fc"]] == "undefined"
combined_table[undefined_idx, "log2fc"] <- NA
combined_table[["log2fc"]] <- as.numeric(combined_table[["log2fc"]])
cor.test(combined_table[["limma_logfc"]], combined_table[["log2fc"]])
cor.test(combined_table[["limma_logfc"]], combined_table[["log2fc"]], method="spearman")

high_to_low_idx <- order(combined_table[["log2fc"]], na.last=TRUE, decreasing=TRUE)
low_to_high_idx <- order(combined_table[["log2fc"]], na.last=TRUE, decreasing=FALSE)
top_50 <- head(combined_table[high_to_low_idx, ], n=100)
bottom_50 <- head(combined_table[low_to_high_idx, ], n=100)

cor.test(top_50$limma_logfc, top_50$log2fc)
cor.test(top_50$limma_logfc, top_50$log2fc, method="spearman")
cor.test(top_50$deseq_logfc, top_50$log2fc)
cor.test(top_50$deseq_logfc, top_50$log2fc, method="spearman")
cor.test(top_50$edger_logfc, top_50$log2fc)
cor.test(top_50$edger_logfc, top_50$log2fc, method="spearman")

cor.test(bottom_50$limma_logfc, bottom_50$log2fc)
cor.test(bottom_50$limma_logfc, bottom_50$log2fc, method="spearman")
cor.test(bottom_50$deseq_logfc, bottom_50$log2fc)
cor.test(bottom_50$deseq_logfc, bottom_50$log2fc, method="spearman")
cor.test(bottom_50$edger_logfc, bottom_50$log2fc)
cor.test(bottom_50$edger_logfc, bottom_50$log2fc, method="spearman")

up_in_msstats <- rownames(combined_table)[combined_table[["log2fc"]] > 0]
up_in_msstats <- up_in_msstats[!is.na(up_in_msstats)]
up_in_limma <- rownames(combined_table)[combined_table[["limma_logfc"]] > 0]
up_in_limma <- up_in_limma[!is.na(up_in_limma)]
up_in_edger <- rownames(combined_table)[combined_table[["edger_logfc"]] > 0]
up_in_edger <- up_in_edger[!is.na(up_in_edger)]
up_venn_sets <- list(
  "msstats" = up_in_msstats,
  "limma" = up_in_limma,
  "edger" = up_in_edger)
testing <- Vennerable::Venn(Sets=up_venn_sets, )
pp(file="/tmp/up_venn.png")
Vennerable::plot(testing, doWeights=FALSE)
dev.off()
Vennerable::plot(testing, doWeights=FALSE)

down_in_msstats <- rownames(combined_table)[combined_table[["log2fc"]] < 0]
down_in_msstats <- down_in_msstats[!is.na(down_in_msstats)]
down_in_limma <- rownames(combined_table)[combined_table[["limma_logfc"]] < 0]
down_in_limma <- down_in_limma[!is.na(down_in_limma)]
down_in_edger <- rownames(combined_table)[combined_table[["edger_logfc"]] < 0]
down_in_edger <- down_in_edger[!is.na(down_in_edger)]
down_venn_sets <- list(
  "msstats" = down_in_msstats,
  "limma" = down_in_limma,
  "edger" = down_in_edger)
testing <- Vennerable::Venn(Sets=down_venn_sets, )
pp(file="/tmp/down_venn.png")
Vennerable::plot(testing, doWeights=FALSE)
dev.off()

## Error: <text>:7:61: unexpected string constant
## 8: 
## 9: merged_table <- merge(hpgl_table, msstats_table, by="
##                                                                ^

7 Repeat with the modified data

hpgl_table <- pairwise_modified_tables$data[[1]]

## Error in eval(expr, envir, enclos): object 'pairwise_modified_tables' not found

msstats_table <- msstats_modified_comp$ComparisonResult

## Error in eval(expr, envir, enclos): object 'msstats_modified_comp' not found

rownames(msstats_table) <- gsub(pattern="^1/", replacement="", x=msstats_table_modified$Protein)

## Error in gsub(pattern = "^1/", replacement = "", x = msstats_table_modified$Protein): object 'msstats_table_modified' not found

merged_table <- merge(hpgl_table, msstats_table, by="row.names")

## Error in merge(hpgl_table, msstats_table, by = "row.names"): object 'hpgl_table' not found

write.csv(file="images/merged_table_modified.csv", merged_table)

## Error in is.data.frame(x): object 'merged_table' not found

cor.test(merged_table$limma_logfc, merged_table$log2FC)

## Error in cor.test(merged_table$limma_logfc, merged_table$log2FC): object 'merged_table' not found

cor.test(merged_table$limma_logfc, merged_table$log2FC, method="spearman")

## Error in cor.test(merged_table$limma_logfc, merged_table$log2FC, method = "spearman"): object 'merged_table' not found

cor.test(merged_table$deseq_logfc, merged_table$log2FC)

## Error in cor.test(merged_table$deseq_logfc, merged_table$log2FC): object 'merged_table' not found

cor.test(merged_table$deseq_logfc, merged_table$log2FC, method="spearman")

## Error in cor.test(merged_table$deseq_logfc, merged_table$log2FC, method = "spearman"): object 'merged_table' not found

cor.test(merged_table$edger_logfc, merged_table$log2FC)

## Error in cor.test(merged_table$edger_logfc, merged_table$log2FC): object 'merged_table' not found

cor.test(merged_table$edger_logfc, merged_table$log2FC, method="spearman")

## Error in cor.test(merged_table$edger_logfc, merged_table$log2FC, method = "spearman"): object 'merged_table' not found

combined_table <- pairwise_tables$data[[1]]

## Error in eval(expr, envir, enclos): object 'pairwise_tables' not found

undefined_idx <- combined_table[["log2fc"]] == "undefined"

## Error in eval(expr, envir, enclos): object 'combined_table' not found

combined_table[undefined_idx, "log2fc"] <- NA

## Error in combined_table[undefined_idx, "log2fc"] <- NA: object 'combined_table' not found

combined_table[["log2fc"]] <- as.numeric(combined_table[["log2fc"]])

## Error in eval(expr, envir, enclos): object 'combined_table' not found

cor.test(combined_table[["limma_logfc"]], combined_table[["log2fc"]])

## Error in cor.test(combined_table[["limma_logfc"]], combined_table[["log2fc"]]): object 'combined_table' not found

cor.test(combined_table[["limma_logfc"]], combined_table[["log2fc"]], method="spearman")

## Error in cor.test(combined_table[["limma_logfc"]], combined_table[["log2fc"]], : object 'combined_table' not found

high_to_low_idx <- order(combined_table[["log2fc"]], na.last=TRUE, decreasing=TRUE)

## Error in order(combined_table[["log2fc"]], na.last = TRUE, decreasing = TRUE): object 'combined_table' not found

low_to_high_idx <- order(combined_table[["log2fc"]], na.last=TRUE, decreasing=FALSE)

## Error in order(combined_table[["log2fc"]], na.last = TRUE, decreasing = FALSE): object 'combined_table' not found

top_50 <- head(combined_table[high_to_low_idx, ], n=100)

## Error in head(combined_table[high_to_low_idx, ], n = 100): object 'combined_table' not found

bottom_50 <- head(combined_table[low_to_high_idx, ], n=100)

## Error in head(combined_table[low_to_high_idx, ], n = 100): object 'combined_table' not found

cor.test(top_50$limma_logfc, top_50$log2fc)

## Error in cor.test(top_50$limma_logfc, top_50$log2fc): object 'top_50' not found

cor.test(top_50$limma_logfc, top_50$log2fc, method="spearman")

## Error in cor.test(top_50$limma_logfc, top_50$log2fc, method = "spearman"): object 'top_50' not found

cor.test(top_50$deseq_logfc, top_50$log2fc)

## Error in cor.test(top_50$deseq_logfc, top_50$log2fc): object 'top_50' not found

cor.test(top_50$deseq_logfc, top_50$log2fc, method="spearman")

## Error in cor.test(top_50$deseq_logfc, top_50$log2fc, method = "spearman"): object 'top_50' not found

cor.test(top_50$edger_logfc, top_50$log2fc)

## Error in cor.test(top_50$edger_logfc, top_50$log2fc): object 'top_50' not found

cor.test(top_50$edger_logfc, top_50$log2fc, method="spearman")

## Error in cor.test(top_50$edger_logfc, top_50$log2fc, method = "spearman"): object 'top_50' not found

cor.test(bottom_50$limma_logfc, bottom_50$log2fc)

## Error in cor.test(bottom_50$limma_logfc, bottom_50$log2fc): object 'bottom_50' not found

cor.test(bottom_50$limma_logfc, bottom_50$log2fc, method="spearman")

## Error in cor.test(bottom_50$limma_logfc, bottom_50$log2fc, method = "spearman"): object 'bottom_50' not found

cor.test(bottom_50$deseq_logfc, bottom_50$log2fc)

## Error in cor.test(bottom_50$deseq_logfc, bottom_50$log2fc): object 'bottom_50' not found

cor.test(bottom_50$deseq_logfc, bottom_50$log2fc, method="spearman")

## Error in cor.test(bottom_50$deseq_logfc, bottom_50$log2fc, method = "spearman"): object 'bottom_50' not found

cor.test(bottom_50$edger_logfc, bottom_50$log2fc)

## Error in cor.test(bottom_50$edger_logfc, bottom_50$log2fc): object 'bottom_50' not found

cor.test(bottom_50$edger_logfc, bottom_50$log2fc, method="spearman")

## Error in cor.test(bottom_50$edger_logfc, bottom_50$log2fc, method = "spearman"): object 'bottom_50' not found

up_in_msstats <- rownames(combined_table)[combined_table[["log2fc"]] > 0]

## Error in rownames(combined_table): object 'combined_table' not found

up_in_msstats <- up_in_msstats[!is.na(up_in_msstats)]

## Error in eval(expr, envir, enclos): object 'up_in_msstats' not found

up_in_limma <- rownames(combined_table)[combined_table[["limma_logfc"]] > 0]

## Error in rownames(combined_table): object 'combined_table' not found

up_in_limma <- up_in_limma[!is.na(up_in_limma)]

## Error in eval(expr, envir, enclos): object 'up_in_limma' not found

up_in_edger <- rownames(combined_table)[combined_table[["edger_logfc"]] > 0]

## Error in rownames(combined_table): object 'combined_table' not found

up_in_edger <- up_in_edger[!is.na(up_in_edger)]

## Error in eval(expr, envir, enclos): object 'up_in_edger' not found

up_venn_sets <- list(
  "msstats" = up_in_msstats,
  "limma" = up_in_limma,
  "edger" = up_in_edger)

## Error in eval(expr, envir, enclos): object 'up_in_msstats' not found

testing <- Vennerable::Venn(Sets=up_venn_sets, )

## Error in Vennerable::Venn(Sets = up_venn_sets, ): object 'up_venn_sets' not found

pp(file="/tmp/up_venn.png")

## Going to write the image to: /tmp/up_venn.png when dev.off() is called.

Vennerable::plot(testing, doWeights=FALSE)

## Error in Vennerable::plot(testing, doWeights = FALSE): object 'testing' not found

dev.off()

## png 
##   2

Vennerable::plot(testing, doWeights=FALSE)

## Error in Vennerable::plot(testing, doWeights = FALSE): object 'testing' not found

down_in_msstats <- rownames(combined_table)[combined_table[["log2fc"]] < 0]

## Error in rownames(combined_table): object 'combined_table' not found

down_in_msstats <- down_in_msstats[!is.na(down_in_msstats)]

## Error in eval(expr, envir, enclos): object 'down_in_msstats' not found

down_in_limma <- rownames(combined_table)[combined_table[["limma_logfc"]] < 0]

## Error in rownames(combined_table): object 'combined_table' not found

down_in_limma <- down_in_limma[!is.na(down_in_limma)]

## Error in eval(expr, envir, enclos): object 'down_in_limma' not found

down_in_edger <- rownames(combined_table)[combined_table[["edger_logfc"]] < 0]

## Error in rownames(combined_table): object 'combined_table' not found

down_in_edger <- down_in_edger[!is.na(down_in_edger)]

## Error in eval(expr, envir, enclos): object 'down_in_edger' not found

down_venn_sets <- list(
  "msstats" = down_in_msstats,
  "limma" = down_in_limma,
  "edger" = down_in_edger)

## Error in eval(expr, envir, enclos): object 'down_in_msstats' not found

testing <- Vennerable::Venn(Sets=down_venn_sets, )

## Error in Vennerable::Venn(Sets = down_venn_sets, ): object 'down_venn_sets' not found

pp(file="/tmp/down_venn.png")

## Going to write the image to: /tmp/down_venn.png when dev.off() is called.

Vennerable::plot(testing, doWeights=FALSE)

## Error in Vennerable::plot(testing, doWeights = FALSE): object 'testing' not found

dev.off()

## png 
##   2

8 Everything below here might get dropped?

I think I wedged all of the following work into that short block above. So, stop evaluating the following blocks and see if that is true.

8.0.1 Expressionset from the TRIC intensity matrix

intensity_mtrx <- read.csv(file="results/tric/HCD_outmatrix.tsv", sep="\t")
## The HCD_outmatrix has columns including: Peptide, Protein,
## A series of Intensity_sample_id columns
## A series of RT_sample_id columns
## A series of score_sample_id columns
## And at the end: RT_mean, RT_std, and pg_pvalue.
## Since SWATH2stats/MSstats uses the intensities, lets get those columns out,
## standardize the column names to match the annotation data, and use them for
## creating an expressionset.
intensity_mtrx[["rownames"]] <- intensity_mtrx[["Protein"]]
intensity_mtrx[["rownames"]] <- gsub(pattern="^[[:digit:]]+\\/",
                                     replacement="", x=intensity_mtrx[["rownames"]])
## Standardize the rownames, this might be a bad idea, as this will keep
## separate every peptide from each protein and not do anything to
## sum/median/whatever them.  But for the purposes of testing out the data I
## think it is ok.
rownames(intensity_mtrx) <- make.names(intensity_mtrx[["rownames"]], unique=TRUE)

## Now lets get rid of the extraneous text in the column names and simplify them
## to the sample names as referenced in the sample sheet.
all_columns <- colnames(intensity_mtrx)
intense_columns <- grepl(pattern="Intensity", x=all_columns)
intensity_mtrx <- intensity_mtrx[, intense_columns]
all_columns <- colnames(intensity_mtrx)
new_columns <- gsub(pattern="Intensity_", replacement="", x=all_columns)
new_columns <- gsub(pattern="_dia_.*$", replacement="", x=new_columns)
colnames(intensity_mtrx) <- paste0("s", new_columns)
intensity_mtrx[is.na(intensity_mtrx)] <- 0

## No columns in an expression set are allowed to start with a number.  I have
## unsed prefixing samples with 's' as a standard to handle this problem...
metadata <- sample_annot
metadata$sampleid <- paste0("s", metadata$sampleid)
colnames(intensity_mtrx) <- gsub(pattern="_vs_HCD", replacement="", x=colnames(intensity_mtrx))
rownames(metadata) <- metadata$sampleid
## Theoretically this is not needed anymore...
intensity_mtrx <- intensity_mtrx[, rownames(metadata)]
## Ok, at this point, we should have all the pieces for a more or less normal expressionset.
dim(intensity_mtrx)
test_expt <- create_expt(metadata=metadata,
                         count_dataframe=intensity_mtrx,
                         gene_info=mtb_annotations)

8.0.2 Play with the hpgltools derived expressionset of peptide intensities

Lets see if anything makes sense in this intensity expressionset.

## First, log2 and normalize the data.
test_norm <- normalize_expt(test_expt, transform="log2", convert="cpm", norm="quantile", filter=TRUE)
test_normbatch <- sm(normalize_expt(test_expt, transform="log2", convert="cpm",
                                    norm="quantile", filter=TRUE, batch="limma"))

test_metrics <- sm(graph_metrics(test_expt))
test_metrics_norm <- sm(graph_metrics(test_norm))
test_metrics_normbatch <- sm(graph_metrics(test_normbatch))

Now lets see what the data looks like, assuming I did not do anything horrible to it.

test_metrics$legend
test_metrics$libsize  ## Wow the intensities get ridiculous, what?
test_metrics$nonzero
## Interesting, but nothing which super-jumps out at me
test_metrics$density
## hmm very interesting, the good news is that they all have basically the same
## distribution.  I did not expect the CF samples to have such a stronger
## distribution though.

test_metrics_norm$corheat
## This suggests to me a likely batch effect
test_metrics_norm$disheat
test_metrics$smc
test_metrics$tsneplot
## Yeah there is some wonkyness between the old/new samples.

test_metrics_normbatch$pcaplot
## Wow, this is after invoking 'removeBatchEffect' from limma.  That suggests
## pretty strongly to me that we should probalby not examine the new and old
## data together.  In addition I am thinking that one whole-cell lysate sample
## might not be.

8.1 Remove old batch/weirdo sample

kept_testing <- subset_expt(test_expt,
                            subset="bioreplicate == 'mar'")
## The new batch IDs: x,y,z are associated with the 3 runs of these samples, one
## which is 8 m/z, one which is 20 m/z, and one which is a reordered 20 m/z.

test_norm <- normalize_expt(kept_testing, transform="log2", convert="cpm", norm="quantile", filter=TRUE)
test_normbatch <- sm(normalize_expt(kept_testing, transform="log2", convert="cpm",
                                    filter=TRUE, batch="limma"))
new_metrics <- sm(graph_metrics(kept_testing))
new_metrics_norm <- sm(graph_metrics(test_norm))
new_metrics_normbatch <- sm(graph_metrics(test_normbatch))

8.1.1 Plot only new data

Removing the January data seems to have made this a lot easier to look at. There might be some batchyness associated with reordering the samples, but I am thinking it is not huge.

tt = plot_topn(kept_testing, direct=TRUE)
new_metrics$legend
new_metrics$libsize
pp("images/kept_norm_topn.png", image=new_metrics$topnplot)
pp("images/kept_norm_hcd_pca.png", image=new_metrics_norm$pcaplot)
new_metrics_norm$tsneplot
pp("images/kept_norm_hcd_corheat.png", image=new_metrics_norm$corheat)
new_metrics_norm$disheat
new_metrics_normbatch$pcaplot
pp("images/normbatch_tsne.png", image=new_metrics_normbatch$tsneplot)

8.1.2 How is the variance?

Lets see what variancePartition has to say about this data.

new_varpart <- varpart(kept_testing, factors=c("condition", "batch"))
pp("images/varpart_partition.png", image=new_varpart$partition_plot)
## That is not terrible.

## the modified_expt slow from varpart() adds some metadata to the expressionset
## including the calculated % variance for condition/batch/residual for each peptide.
kept_testing <- new_varpart$modified_expt

pe_genes <- read.table("reference/annotated_pe_genes.txt")[[1]]
sum(pe_genes %in% rownames(exprs(protein_norm)))

## Error in exprs(protein_norm): object 'protein_norm' not found

found_pe_idx <- rownames(exprs(protein_expt)) %in% pe_genes

## Error in exprs(protein_expt): object 'protein_expt' not found

found_pe <- rownames(exprs(protein_expt))[found_pe_idx]

## Error in exprs(protein_expt): object 'protein_expt' not found

pe_expt <- exclude_genes_expt(protein_expt, method="keep", ids=found_pe)

## Error in exclude_genes_expt(protein_expt, method = "keep", ids = found_pe): object 'protein_expt' not found

plot_boxplot(pe_expt)

## Error in plot_boxplot(pe_expt): object 'pe_expt' not found

exprs(pe_expt)

## Error in exprs(pe_expt): object 'pe_expt' not found

8.2 Back to hpgltools

If you will recall, in the original block where I read the data into SWATH2stats, I invoked write_matrix_proteins() and write_matrix_peptides() and wrote out 2 csv files ‘swath2stats_protein_matrix.csv’ and ’_peptide_matrix.csv’. If my earlier work is correct, I should be able to use the protein matrix to get a truer sense of the relative abundances between my experimental conditions.

library(aLFQ)
alfq_process <- ProteinInference(alfq_input,
                                 peptide_method="top",
                                 peptide_topx=3,
                                 peptide_strictness="loose",
                                 peptide_summary="mean",
                                 transition_topx=3,
                                 transition_strictness="loose",
                                 transition_summary="sum",
                                 fasta=NA,
                                 model=NA,
                                 combine_precursors=FALSE)

9 Index version: 20180528

10 TODO

2018-04-10: Make sure my invocations of SWATH2stats/MSstats are correct.

if (!isTRUE(get0("skip_load"))) {
  message(paste0("This is hpgltools commit: ", get_git_commit()))
  this_save <- paste0(gsub(pattern="\\.Rmd", replace="", x=rmd_file), "-v", ver, ".rda.xz")
  message(paste0("Saving to ", this_save))
  tmp <- sm(saveme(filename=this_save))
  pander::pander(sessionInfo())
}

LS0tCnRpdGxlOiAiTS50dWJlcmN1bG9zaXMgMjAxODA1OiBBbmFseXppbmcgZGF0YSBmcm9tIE9wZW5Td2F0aFdvcmtGbG93L1RSSUMuIgphdXRob3I6ICJhdGIgYWJlbGV3QGdtYWlsLmNvbSIKZGF0ZTogImByIFN5cy5EYXRlKClgIgpvdXRwdXQ6CiBodG1sX2RvY3VtZW50OgogIGNvZGVfZG93bmxvYWQ6IHRydWUKICBjb2RlX2ZvbGRpbmc6IHNob3cKICBmaWdfY2FwdGlvbjogdHJ1ZQogIGZpZ19oZWlnaHQ6IDcKICBmaWdfd2lkdGg6IDcKICBoaWdobGlnaHQ6IGRlZmF1bHQKICBrZWVwX21kOiBmYWxzZQogIG1vZGU6IHNlbGZjb250YWluZWQKICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICBzZWxmX2NvbnRhaW5lZDogdHJ1ZQogIHRoZW1lOiByZWFkYWJsZQogIHRvYzogdHJ1ZQogIHRvY19mbG9hdDoKICAgIGNvbGxhcHNlZDogZmFsc2UKICAgIHNtb290aF9zY3JvbGw6IGZhbHNlCi0tLQoKPHN0eWxlPgogIGJvZHkgLm1haW4tY29udGFpbmVyIHsKICAgIG1heC13aWR0aDogMTYwMHB4OwogIH0KPC9zdHlsZT4KCmBgYHtyIG9wdGlvbnMsIGluY2x1ZGU9RkFMU0V9CmlmICghaXNUUlVFKGdldDAoInNraXBfbG9hZCIpKSkgewogIGxpYnJhcnkoaHBnbHRvb2xzKQogIHR0IDwtIGRldnRvb2xzOjpsb2FkX2FsbCgifi9ocGdsdG9vbHMiKQogIGtuaXRyOjpvcHRzX2tuaXQkc2V0KHByb2dyZXNzPVRSVUUsCiAgICAgICAgICAgICAgICAgICAgICAgdmVyYm9zZT1UUlVFLAogICAgICAgICAgICAgICAgICAgICAgIHdpZHRoPTkwLAogICAgICAgICAgICAgICAgICAgICAgIGVjaG89VFJVRSkKICBrbml0cjo6b3B0c19jaHVuayRzZXQoZXJyb3I9VFJVRSwKICAgICAgICAgICAgICAgICAgICAgICAgZmlnLndpZHRoPTgsCiAgICAgICAgICAgICAgICAgICAgICAgIGZpZy5oZWlnaHQ9OCwKICAgICAgICAgICAgICAgICAgICAgICAgZHBpPTk2KQogIG9sZF9vcHRpb25zIDwtIG9wdGlvbnMoZGlnaXRzPTQsCiAgICAgICAgICAgICAgICAgICAgICAgICBzdHJpbmdzQXNGYWN0b3JzPUZBTFNFLAogICAgICAgICAgICAgICAgICAgICAgICAga25pdHIuZHVwbGljYXRlLmxhYmVsPSJhbGxvdyIpCiAgZ2dwbG90Mjo6dGhlbWVfc2V0KGdncGxvdDI6OnRoZW1lX2J3KGJhc2Vfc2l6ZT0xMCkpCiAgdmVyIDwtICIyMDE4MDUyOCIKICBwcmV2aW91c19maWxlIDwtICIwMV9wcmVwcm9jZXNzaW5nX2NvbWV0X2hpZ2hyZXMuUm1kIgoKICB0bXAgPC0gdHJ5KHNtKGxvYWRtZShmaWxlbmFtZT1wYXN0ZTAoZ3N1YihwYXR0ZXJuPSJcXC5SbWQiLCByZXBsYWNlPSIiLCB4PXByZXZpb3VzX2ZpbGUpLCAiLXYiLCB2ZXIsICIucmRhLnh6IikpKSkKICBwcmV2aW91c19maWxlIDwtICIwMV9hbm5vdGF0aW9uLlJtZCIKICB0bXAgPC0gdHJ5KHNtKGxvYWRtZShmaWxlbmFtZT1wYXN0ZTAoZ3N1YihwYXR0ZXJuPSJcXC5SbWQiLCByZXBsYWNlPSIiLCB4PXByZXZpb3VzX2ZpbGUpLCAiLXYiLCB2ZXIsICIucmRhLnh6IikpKSkKCiAgcm1kX2ZpbGUgPC0gIjAyX3N3YXRoMnN0YXRzXzIwMTgwNTIwLlJtZCIKfQpgYGAKCiMgVE9ETwoKKiBSZW1vdmUgTVNzdGF0cyBsb2dnaW5nLiAtLSBkb25lCiogTWFrZSBleHBsaWNpdCBzcGVhcm1hbiBjb3JyZWxhdGlvbnMgYmV0d2VlbiBtZXRob2RzLiAtLSBkb25lCiAgICAqIERvIGJvdGggZm9yIGFsbCBkYXRhIGFuZCBmb3IgdGhlIHRvcC01MC9ib3R0b20tNTAgLS0gZG9uZSwgYnV0IHdlaXJkLgoqIE1ha2UgMTAwJSBjZXJ0YWluIHRoYXQgdGhlIHNhbXBsZXMgYXJlIGFubm90YXRlZCBjb3JyZWN0bHkuIC0tIGRvbmUuCiogTGltbWEvTVNzdGF0cy9FZGdlUiB2ZW5uIGRpc2FncmFtIGZvciBhbGwgdXAgaW4gQ0YKICAgKiBSZXBlYXQgaW4gYWxsIGRvd24gQ0YKICAgKiBSZXBlYXQgd2l0aCBhIGN1dG9mZiwgdG9wLTUwCiogSW5kaXZpZHVhbCBwbG90cyBmb3IgUC9QRSBwcm90ZWlucy4KCkFuYWx5emluZyBkYXRhIGZyb20gb3Blbk1TIGFuZCBmcmllbmRzLgo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KCkluIHByZXByb2Nlc3NpbmdfY29tZXRfaGlnaHJlcy5SbWQsIEkgdXNlZCB0aGUgb3Blbk1TIHR1dG9yaWFscyBhbmQgc3VwcGxlbWVudGFsCm1hdGVyaWFscyBmcm9tIGEgY291cGxlIHBhcGVycyB0byBob3BlZnVsbHkgY29ycmVjdGx5IHBlcmZvcm0gdGhlIHZhcmlvdXMKcHJlcHJvY2Vzc2luZyB0YXNrcyByZXF1aXJlZCB0byBleHRyYWN0IGludGVuc2l0eSBkYXRhIGZyb20gRElBL1NXQVRICnRyYW5zaXRpb25zLgoKVGhlIGZpbmFsIHN0ZXBzIG9mIHRoYXQgcHJvY2VzcyBjb21iaW5lZCB0aGUgdHJhbnNpdGlvbiBpbnRlbnNpdGllcyBmcm9tIGV2ZXJ5CnNhbXBsZSBpbnRvIGEgbWV0YWRhdGEgZnJpbGUgKHJlc3VsdHMvdHJpYy9IQ0RfbWV0YS50c3YpLCBhbiBpbnRlbnNpdHkgbWF0cml4CihyZXN1bHRzL3RyaWMvSENEX291dG1hdHJpeC50c3YpLCBhbmQgYSBmZWF0dXJlIGFsaWduZWQgb3V0cHV0IG1hdHJpeAoocmVzdWx0cy90cmljL2FsaWduZWRfY29tZXRfSENELnRzdikuCgpNeSByZWFkaW5nIG9mIHRoZSBTV0FUSDJzdGF0cyBhbmQgTVNzdGF0cyBzb3VyY2UgY29kZSBzdWdnZXN0cyB0byBtZSB0aGF0IHRoZQpsb2cyKGludGVuc2l0aWVzKSBvZiB0aGUgZmVhdHVyZSBhbGlnbmVkIGRhdGEgYXJlIG91ciBmaW5hbCBwcm94eSBmb3IgcHJvdGVpbgphYnVuZGFuY2UuICBBdCBmaXJzdCBnbGFuY2UsIHRoaXMgc3VnZ2VzdHMgdG8gbWUgdGhhdCB0aGVzZSBkYXRhIG1pZ2h0IGZvbGxvdyBhCmRpc3RyaWJ1dGlvbiBzaW1pbGFyIHRvIFJOQVNlcSBkYXRhIChuZWdhdGl2ZSBiaW5vbWlhbCwgYnV0IHBlcmhhcHMgd2l0aCBhCmJpZ2dlciB0YWlsPykuICBJbiBhZGRpdGlvbiwgYnkgdGhlIHRpbWUgd2UgdXNlIHRyaWMgb24gdGhlIGRhdGEsIHdlIGhhdmUgYQpjb3VudCBtYXRyaXggYW5kIHNhbXBsZSBhbm5vdGF0aW9uIGRhdGEgZnJhbWVzIHdoaWNoIGxvb2sgcmVtYXJrYWJseSBzaW1pbGFyIHRvCnRob3NlIHVzZWQgaW4gYSBSTkFTZXEgZXhwcmVzc2lvbnNldC4gIEluZGVlZCwgYnkgdGhlIGVuZCBvZiB0aGUgTVNzdGF0cwpwcm9jZXNzaW5nLCBpdCBjcmVhdGVzIGEgTVNuU2V0IGNsYXNzIG9mIGl0cyBvd24gd2hpY2ggdXNlcyBmRGF0YS9leHBycy9wRGF0YS4KCkZvciB0aGUgY3VyaW91cywgbXkgcmVhc29uaW5nIGZvciBzYXlpbmcgdGhhdCB0aGUgbG9nIGludGVuc2l0aWVzIGFyZSBvdXIgcHJveHkKZm9yIGFidW5kYW5jZSBjb21lcyBmcm9tIE1Tc3RhdHMvUi9EYXRhUHJvY2Vzcy5SIGluIGEgY2xhdXNlIHdoaWNoIGxvb2tzIGxpa2U6CgpgYGB7ciBzbmlwcGV0LCBldmFsPUZBTFNFfQppZiAobG9nVHJhbnMgPT0gMikgewogIHdvcmtbWyJBQlVOREFOQ0UiXV0gPC0gbG9nMih3b3JrW1siQUJVTkRBTkNFIl1dKQp9IGVsc2UgaWYgKGxvZ1RyYW5zID09IDEwKSB7CiAgd29ya1tbIkFCVU5EQU5DRSJdXSA8LSBsb2cxMCh3b3JrW1siQUJVTkRBTkNFIl1dKQp9IGVsc2UgewogICMjIEFib3ZlIHRoZXJlIHdhcyBhIGNoZWNrIGZvciBvbmx5IGxvZyAyIGFuZCAxMCwgYnV0IHdlIGNhbiBkbyBlIGlmIHdlIHdhbnQuCiAgIyMgSSBtaWdodCBnbyBiYWNrIHVwIHRoZXJlIGFuZCByZW1vdmUgdGhhdCBjaGVjay4gTG9uZyBsaXZlIGUhIDIuNzE4MjgyIHJ1bGVzIQogIHdvcmtbWyJBQlVOREFOQ0UiXV0gPC0gbG9nKHdvcmtbWyJBQlVOREFOQ0UiXV0pIC8gbG9nKGxvZ1RyYW5zKQp9CmBgYAoKKE5vdGU6IEkgYWRkZWQgdGhlIG5hdHVyYWwgbG9nIHRvIHRoZSBzZXQgb2YgY29uZGl0aW9ucywgYnV0IG90aGVyd2lzZSB0aGUgbG9naWMKaXMgdW5jaGFuZ2VkLikKCldpdGggdGhhdCBpbiBtaW5kLCBJIHdhbnQgdG8gdXNlIHNvbWUgdG9vbHMgd2l0aCB3aGljaCBJIGFtIGZhbWlsaWFyIGluIG9yZGVyIHRvCnRyeSB0byB1bmRlcnN0YW5kIHRoZXNlIGRhdGEuICBUaGVyZWZvcmUgSSB3aWxsIGZpcnN0IGF0dGVtcHQgdG8gY29lcmNlIG15IHRyaWMKYWxpZ25lZCBkYXRhIGFuZCBhbm5vdGF0aW9ucyBpbnRvIGEgJ25vcm1hbCcgZXhwcmVzc2lvbnNldC4gIFRoZW4gSSB3YW50IHRvIGRvCnNvbWUgZGlhZ25vc3RpYyBwbG90cyB3aGljaCwgaWYgSSBhbSB3cm9uZyBhbmQgdGhlc2UgZGlzdHJpYnV0aW9ucyBhcmUgbm90IGFzCmV4cGVjdGVkLCB3aWxsIGJlIGNvbmNlcHR1YWxseSBpbmNvcnJlY3QgKEkgZG9uJ3QgeWV0IHRoaW5rIEkgYW0gd3JvbmcpLgoKIyMgU2FtcGxlIGFubm90YXRpb24gdmlhIFNXQVRIMnN0YXRzCgpJIGFtIHVzaW5nIHRoZSBTV0FUSDJzdGF0cyB2aWduZXR0ZSBhcyBteSBwcmltYXJ5IHNvdXJjZSBvZiBpbmZvcm1hdGlvbi4gIFRodXMgSQpzZWUgdGhhdCBpdCB1c2VzIHRoZQpPcGVuU1dBVEhfU00zX0dvbGRTdGFuZGFyZEF1dG9tYXRlZFJlc3VsdHNfaHVtYW5fcGVha2dyb3Vwcy50eHQgd2hpY2ggaGFzIGEKZm9ybWF0IG5lYXJseSBpZGVudGljYWwgdG8gbXkgdHJpYyBvdXRwdXQgbWF0cml4LiAgVGh1cyBmb3IgdGhlIG1vbWVudCBJIHdpbGwKYXNzdW1lIHRoYXQgdGhlIHByb3BlciBpbnB1dCBmb3IgU1dBVEgyc3RhdHMgaXMgJ3Jlc3VsdHMvdHJpYy9jb21ldF9IQ0QudHN2JyBhbmQKbm90IHRoZSBtZXRhZGF0YSBub3Igb3V0cHV0IG1hdHJpeC4KCkkga2VlcCBhIHNhbXBsZSBzaGVldCBvZiBhbGwgdGhlIERJQSBzYW1wbGVzIHVzZWQgaW4gdGhpcyBhbmFseXNpcyBpbgonc2FtcGxlX3NoZWV0cy9kaWFfc2FtcGxlcy54bHN4JyAgSXQgc2hvdWxkIGNvbnRhaW4gYWxsIHRoZSBvdGhlciByZXF1aXJlZCBkYXRhCndpdGggb25lIGltcG9ydGFudCBjYXZlYXQsIEkgcmVtb3ZlZCAxIHNhbXBsZSBieSAnY29tbWVudGluZycgaXQgKGUuZy4gcHJlZml4aW5nCml0IHdpdGggJyMjJyAtLSB3aGljaCBpcyBhbiBhZG1pdHRlZGx5IGR1bWIgdGhpbmcgdG8gZG8gaW4gYW4gZXhjZWwgZmlsZS4KCk9uZSBsYXN0IGNhdmVhdDogSSBoYWNrZWQgdGhlIFNXQVRIMnN0YXRzIHNhbXBsZV9hbm5vdGF0aW9uKCkgZnVuY3Rpb24gdG8gYWRkIGEKY291cGxlIGNvbHVtbnMgaW4gYW4gYXR0ZW1wdCB0byBtYWtlIGl0IGEgbGl0dGxlIG1vcmUgcm9idXN0IHdoZW4gZmFjZWQgd2l0aApzYW1wbGUgc2hlZXRzIHdpdGggZGlmZmVyZW50bHkgbmFtZWQgY29sdW1ucy4KCkluIGFkZGl0aW9uLCBTV0FUSDJzdGF0cyBwcm92aWRlcyBzb21lIG5pY2UgZmlsdGVyaW5nIGFuZCBjb21iaW5hdGlvbgpmdW5jdGlvbnMgd2hpY2ggc2hvdWxkIGJlIGNvbnNpZGVyZWQgd2hlbiBnZW5lcmF0aW5nIHZhcmlvdXMgZXhwcmVzc2lvbnNldCBkYXRhCnN0cnVjdHVyZXMgbGF0ZXIuCgpgYGB7ciBzd2F0aDJzdGF0c19zYW1wbGVfYW5ub3RhdGlvbnN9CnRyaWNfZGF0YSA8LSByZWFkLmNzdigicmVzdWx0cy90cmljXzIwMTgwNS9jb21ldF9IQ0QudHN2Iiwgc2VwPSJcdCIpCgpzYW1wbGVfYW5ub3QgPC0gb3Blbnhsc3g6OnJlYWQueGxzeCgic2FtcGxlX3NoZWV0cy9NdGJfZGlhX3NhbXBsZXMueGxzeCIpCnJvd25hbWVzKHNhbXBsZV9hbm5vdCkgPC0gbWFrZS5uYW1lcyhzYW1wbGVfYW5ub3RbWyJzYW1wbGVpZCJdXSwgdW5pcXVlPVRSVUUpCiMjIERyb3Agc2FtcGxlcyBzdGFydGluZyB3aXRoIGNvbW1lbnRzCmtlZXBfaWR4IDwtICEgZ3JlcGwocGF0dGVybj0iIyMiLCB4PXNhbXBsZV9hbm5vdFtbInNhbXBsZWlkIl1dKQpzYW1wbGVfYW5ub3QgPC0gc2FtcGxlX2Fubm90W2tlZXBfaWR4LCBdCmV4cHRfaWR4IDwtIHNhbXBsZV9hbm5vdFtbImV4cHRfaWQiXV0gPT0gIm1heTIwMTgiIHwgc2FtcGxlX2Fubm90W1siZXhwdF9pZCJdXSA9PSAibWFyMjAxOCIKZXhwdF9pZHhbaXMubmEoc2FtcGxlX2Fubm90W1siZXhwdF9pZCJdXSldIDwtIEZBTFNFCnNhbXBsZV9hbm5vdCA8LSBzYW1wbGVfYW5ub3RbZXhwdF9pZHgsIF0KbXpfaWR4IDwtIHNhbXBsZV9hbm5vdFtbIndpbmRvd3NpemUiXV0gPT0gIjgiCnNhbXBsZV9hbm5vdCA8LSBzYW1wbGVfYW5ub3RbbXpfaWR4LCBdCiMjIFNldCB0aGUgbXpYTUwgY29sdW1uIHRvIG1hdGNoIHRoZSBmaWxlbmFtZSBjb2x1bW4gaW4gdGhlIGRhdGEuCmRldnRvb2xzOjpsb2FkX2FsbCgifi9zY3JhdGNoL2dpdC9TV0FUSDJzdGF0cyIpCiMjIHMycywgbXkgd2l0dHkgd2F5IG9mIHNob3J0ZW5pbmcgU1dBVEgyc3RhdHMuLi4KczJzX2V4cCA8LSBzYW1wbGVfYW5ub3RhdGlvbihkYXRhPXRyaWNfZGF0YSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzYW1wbGVfYW5ub3RhdGlvbj1zYW1wbGVfYW5ub3QsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZnVsbHBlcHRpZGVuYW1lX2NvbHVtbj0iZnVsbHVuaW1vZHBlcHRpZGVuYW1lIikKYGBgCgpOb3cgSSBoYXZlIGEgY291cGxlIGRhdGEgc3RydWN0dXJlcyB3aGljaCBzaG91bGQgcHJvdmUgdXNlZnVsIGZvciB0aGUgbWV0cmljcwpwcm92aWRlZCBieSBTV0FUSDJzdGF0cywgTVNzdGF0cywgYW5kIG15IG93biBocGdsdG9vbHMuCgojIFNXQVRIMnN0YXRzIGNvbnRpbnVlZAoKTGV0cyByZXR1cm4gdG8gc29tZSBvZiB0aGUgbWV0cmljcyBwcm92aWRlZCBieSBzd2F0aDJzdGF0cy4KCmBgYHtyIHN3YXRoMnN0YXRzX3Byb2Nlc3Npbmd9CiMjIEdldCBjb3JyZWxhdGlvbnMgb24gYSBzYW1wbGUgYnkgc2FtcGxlIGJhc2lzCnBwKGZpbGU9ImltYWdlcy8yMDE4MDUyM19zd2F0aDJzdGF0c19zYW1wbGVfY29yLnBuZyIpCnNhbXBsZV9jb3IgPC0gcGxvdF9jb3JyZWxhdGlvbl9iZXR3ZWVuX3NhbXBsZXMoczJzX2V4cCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmdW4uYWdncmVnYXRlPXN1bSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb2x1bW4udmFsdWVzPSJpbnRlbnNpdHkiKQpkZXYub2ZmKCkKc2FtcGxlX2NvbmRfcmVwX2NvciA8LSBwbG90X2NvcnJlbGF0aW9uX2JldHdlZW5fc2FtcGxlcyhzMnNfZXhwLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbXBhcmlzb249dHJhbnNpdGlvbl9ncm91cF9pZCB+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25kaXRpb24gKyBiaW9yZXBsaWNhdGUgKyBydW4sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZnVuLmFnZ3JlZ2F0ZT1zdW0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29sdW1uLnZhbHVlcz0iaW50ZW5zaXR5IikKIyMgSSBhbSBhIGxpdHRsZSBjb25jZXJuZWQgdGhhdCB0aGVzZSB2YWx1ZXMgZG8gbm90IHNlZW0gdG8gY2hhbmdlIHdoZW4gSSB0b29rCiMjIGZpbHRlcmVkL25vcm1hbGl6ZWQgZGF0YS4gIFNvIEkgYW0gcmVydW5uaW5nIHRoZW0gbWFudWFsbHkgZm9yIGEgbW9tZW50IC0tCiMjIHBlcmhhcHMgSSBtZXNzZWQgc29tZXRoaW5nIHVwIHdoZW4gSSByZXdyb3RlIHBvcnRpb25zIG9mIHRoZQojIyBzYW1wbGVfYW5ub3RhdGlvbigpIGZ1bmN0aW9uIGluIFNXQVRIMnN0YXRzLgoKIyMgYWhoIEkgdGhpbmsgSSBzZWUgdGhlIHByb2JsZW0uICBUaGUgZGVmYXVsdCB2YWx1ZSBmb3IgZnVuLmFnZ3JlZ2F0ZSBpcyBOVUxMLAojIyB3aGljaCBjYXVzZXMgZGNhc3QgdG8gZGVmYXVsdCB0byBsZW5ndGguICBJIHRoaW5rIHRoaXMgaXMgbm90IGxpa2VseSB0byBiZQojIyB2YWxpZCBmb3IgdGhpcyBkYXRhLiAgSSBhbSBub3QgY2VydGFpbiwgaG93ZXZlciwgd2hhdCBpcyB0aGUgYXBwcm9wcmlhdGUKIyMgZnVuY3Rpb24uICBJZiBJIGhhZCB0byBndWVzcywgSSB3b3VsZCBnbyB3aXRoIHN1bSgpPwoKYXNzZXNzX2RlY295X3JhdGUoczJzX2V4cCkKIyMgVGhpcyBzZWVtcyBhIGJpdCBoaWdoIHRvIG1lLCB5ZXNubz8KZmRyX292ZXJhbGwgPC0gYXNzZXNzX2Zkcl9vdmVyYWxsKHMyc19leHAsIG91dHB1dD0iUmNvbnNvbGUiLCBwbG90PVRSVUUpCgpieXJ1bl9mZHIgPC0gYXNzZXNzX2Zkcl9ieXJ1bihzMnNfZXhwLCBGRlQ9MC43LCBwbG90PVRSVUUsIG91dHB1dD0iUmNvbnNvbGUiKQpjaG9zZW5fbXNjb3JlIDwtIG1zY29yZTRhc3NheWZkcihzMnNfZXhwLCBGRlQ9MC43LCBmZHJfdGFyZ2V0PTAuMDIpCnByb3Rfc2NvcmUgPC0gbXNjb3JlNHByb3RmZHIoczJzX2V4cCwgRkZUPTAuNywgZmRyX3RhcmdldD0wLjAyKQoKbXNjb3JlX2ZpbHRlcmVkIDwtIGZpbHRlcl9tc2NvcmUoczJzX2V4cCwgY2hvc2VuX21zY29yZSkKZGF0YV9maWx0ZXJlZF9tc2NvcmUgPC0gZmlsdGVyX21zY29yZV9mcmVxb2JzKHMyc19leHAsIDAuMDEsIDAuOCwgcm0uZGVjb3k9RkFMU0UpCmRhdGFfZmlsdGVyZWRfZmRyIDwtIGZpbHRlcl9tc2NvcmVfZmRyKG1zY29yZV9maWx0ZXJlZCwgRkZUPTAuNywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgb3ZlcmFsbF9wcm90ZWluX2Zkcl90YXJnZXQ9cHJvdF9zY29yZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdXBwZXJfb3ZlcmFsbF9wZXB0aWRlX2Zkcl9saW1pdD0wLjA1KQpvbmx5X3Byb3Rlb3R5cGljIDwtIGZpbHRlcl9wcm90ZW90eXBpY19wZXB0aWRlcyhkYXRhX2ZpbHRlcmVkX2ZkcikKYWxsX2ZpbHRlcmVkIDwtIGZpbHRlcl9hbGxfcGVwdGlkZXMob25seV9wcm90ZW90eXBpYykKb25seV9zdHJvbmcgPC0gZmlsdGVyX29uX21heF9wZXB0aWRlcyhkYXRhPWFsbF9maWx0ZXJlZCwgbl9wZXB0aWRlcz0xMCkKb25seV9taW5pbXVtIDwtIGZpbHRlcl9vbl9taW5fcGVwdGlkZXMoZGF0YT1vbmx5X3N0cm9uZywgbl9wZXB0aWRlcz0zKQoKIyMgSSB0aGluayB0aGVzZSBtYXRyaXhlcyBhcmUgcHJvYmFibHkgc21hcnRlciB0byB1c2UgdGhhbiB0aGUgcmF3IG91dG1hdHJpeCBmcm9tIHRyaWMuCiMjIEJ1dCBJIGFtIG5vdCBhIGZhbiBvZiByZXJ3cml0aW5nIHRoZSBzYW1wbGUgY29sdW1uIG5hbWVzLgpwcm90ZWluX21hdHJpeF9hbGwgPC0gd3JpdGVfbWF0cml4X3Byb3RlaW5zKAogIHMyc19leHAsIHdyaXRlLmNzdj1UUlVFLAogIGZpbGVuYW1lPXBhc3RlMCgicmVzdWx0cy9zd2F0aDJzdGF0c18iLCB2ZXIsICIvcHJvdGVpbl9hbGwuY3N2IikpCmRpbShwcm90ZWluX21hdHJpeF9hbGwpCnByb3RlaW5fbWF0cml4X21zY29yZSA8LSB3cml0ZV9tYXRyaXhfcHJvdGVpbnMoCiAgbXNjb3JlX2ZpbHRlcmVkLCB3cml0ZS5jc3Y9VFJVRSwKICBmaWxlbmFtZT1wYXN0ZTAoInJlc3VsdHMvc3dhdGgyc3RhdHNfIiwgdmVyLCAiL3Byb3RlaW5fbWF0cml4X21zY29yZS5jc3YiKSkKZGltKHByb3RlaW5fbWF0cml4X21zY29yZSkKcGVwdGlkZV9tYXRyaXhfbXNjb3JlIDwtIHdyaXRlX21hdHJpeF9wZXB0aWRlcygKICBtc2NvcmVfZmlsdGVyZWQsIHdyaXRlLmNzdj1UUlVFLAogIGZpbGVuYW1lPXBhc3RlMCgicmVzdWx0cy9zd2F0aDJzdGF0c18iLCB2ZXIsICIvcGVwdGlkZV9tYXRyaXhfbXNjb3JlLmNzdiIpKQpkaW0ocGVwdGlkZV9tYXRyaXhfbXNjb3JlKQpwcm90ZWluX21hdHJpeF9taW5pbXVtIDwtIHdyaXRlX21hdHJpeF9wcm90ZWlucygKICBvbmx5X21pbmltdW0sIHdyaXRlLmNzdj1UUlVFLAogIGZpbGVuYW1lPXBhc3RlMCgicmVzdWx0cy9zd2F0aDJzdGF0c18iLCB2ZXIsICIvcHJvdGVpbl9tYXRyaXhfbWluaW11bS5jc3YiKSkKZGltKHByb3RlaW5fbWF0cml4X21pbmltdW0pCnBlcHRpZGVfbWF0cml4X21pbmltdW0gPC0gd3JpdGVfbWF0cml4X3BlcHRpZGVzKAogIG9ubHlfbWluaW11bSwgd3JpdGUuY3N2PVRSVUUsCiAgZmlsZW5hbWU9cGFzdGUwKCJyZXN1bHRzL3N3YXRoMnN0YXRzXyIsIHZlciwgIi9wZXB0aWRlX21hdHJpeF9taW5pbXVtLmNzdiIpKQpkaW0ocGVwdGlkZV9tYXRyaXhfbWluaW11bSkKCnJ0X2NvciA8LSBwbG90X2NvcnJlbGF0aW9uX2JldHdlZW5fc2FtcGxlcyhvbmx5X21pbmltdW0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb2x1bW4udmFsdWVzPSJpbnRlbnNpdHkiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZnVuLmFnZ3JlZ2F0ZT1zdW0pCiMjIEkgaGF2ZSBubyBlZmZpbmcgY2x1ZSB3aGF0IHRoaXMgcGxvdCBtZWFucy4KdmFyaWF0aW9uIDwtIHBsb3RfdmFyaWF0aW9uKG9ubHlfbWluaW11bSwgZnVuLmFnZ3JlZ2F0ZT1zdW0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb21wYXJpc29uPXRyYW5zaXRpb25fZ3JvdXBfaWQgfiBjb25kaXRpb24pCgojIyBTb21ldGhpbmcgaW4gU1dBVEgyc3RhdHM6OmRpc2FnZ3JlZ2F0ZSB3YXMgd3JpdHRlbiBwb29ybHkgYW5kIGlzIGxvb2tpbmcgZm9yCiMjIGEgdmFyaWFibGUgbmFtZWQgJ2NvbHMnCmNvbHMgPC0gY29sbmFtZXMob25seV9taW5pbXVtKQpkaXNhZ2dyZWdhdGVkIDwtIGRpc2FnZ3JlZ2F0ZShvbmx5X21pbmltdW0sIGFsbC5jb2x1bW5zPVRSVUUpCm1zc3RhdHNfaW5wdXQgPC0gY29udmVydDRNU3N0YXRzKGRpc2FnZ3JlZ2F0ZWQpCgojI2FsZnFfaW5wdXQgPC0gc20oY29udmVydDRhTEZRKGRpc2FnZ3JlZ2F0ZWQpKQojI21hcGRpYV9pbnB1dCA8LSBzbShjb252ZXJ0NG1hcERJQShkaXNhZ2dyZWdhdGVkLCBSVD1UUlVFKSkKYGBgCgojIyBNU3N0YXRzCgptc3N0YXRzLm9yZyBzZWVtcyB0byBwcm92aWRlIGEgY29tcGxldGUgc29sdXRpb24gZm9yIHBlcmZvcm1pbmcgcmVhc29uYWJsZSBtZXRyaWNzIG9mIHRoaXMgZGF0YS4KCkkgYW0gY3VycmVudGx5IHJlYWRpbmc6IGh0dHA6Ly9tc3N0YXRzLm9yZy93cC1jb250ZW50L3VwbG9hZHMvMjAxNy8wMS9NU3N0YXRzX3YzLjcuM19tYW51YWwucGRmCgpJIG1hZGUgc29tZSBtb2RlcmF0ZWx5IGludHJ1c2l2ZSBjaGFuZ2VzIHRvIE1Tc3RhdHMgdG8gbWFrZSBpdCBjbGVhcmVyLCBhcyB3ZWxsLgoKYGBge3IgbXNzdGF0c19xdWFudH0KZGV2dG9vbHM6OmxvYWRfYWxsKCJ+L3NjcmF0Y2gvZ2l0L01Tc3RhdHMiKQptc3N0YXRzX3F1YW50IDwtIHNtKGRhdGFQcm9jZXNzKG1zc3RhdHNfaW5wdXQpKQptc3N0YXRzX3Bsb3RzIDwtIHNtKGRhdGFQcm9jZXNzUGxvdHMobXNzdGF0c19xdWFudCwgdHlwZT0iUUNQTE9UIikpCgpteV9sZXZlbHMgPC0gbGV2ZWxzKGFzLmZhY3Rvcihtc3N0YXRzX2lucHV0JGNvbmRpdGlvbikpCm15X2xldmVscwpjb21wYXJpc29ucyA8LSBnaGV0dG9fY29udHJhc3RfbWF0cml4KAogIG51bWVyYXRvcnM9Yygid3RfY2YiLCAiZGVsdGFfY2YiLCAiY29tcF9jZiIsCiAgICAgICAgICAgICAgICJkZWx0YV9jZiIsICJjb21wX2NmIiwgImRlbHRhX3dob2xlIiwKICAgICAgICAgICAgICAgImNvbXBfd2hvbGUiKSwKICBkZW5vbWluYXRvcnM9Yygid3Rfd2hvbGUiLCAiZGVsdGFfd2hvbGUiLCAiY29tcF93aG9sZSIsCiAgICAgICAgICAgICAgICAgInd0X2NmIiwgInd0X2NmIiwgInd0X3dob2xlIiwKICAgICAgICAgICAgICAgICAid3Rfd2hvbGUiKSkKcmVzdWx0cyA8LSBsaXN0KCkKZm9yIChjIGluIDE6bGVuZ3RoKGNvbXBhcmlzb25zKSkgewogIG5hbWUgPC0gcm93bmFtZXMoY29tcGFyaXNvbnMpW2NdCiAgbWVzc2FnZSgiU3RhcnRpbmcgIiwgbmFtZSkKICBjb21wIDwtIGNvbXBhcmlzb25zW2MsIF0KICBjb21wYXJpc29uIDwtIHQoYXMubWF0cml4KGNvbXApKQogIHJvd25hbWVzKGNvbXBhcmlzb24pIDwtIG5hbWUKICByZXN1bHRzW25hbWVdIDwtIHNtKE1Tc3RhdHM6Omdyb3VwQ29tcGFyaXNvbihjb250cmFzdC5tYXRyaXg9Y29tcGFyaXNvbiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkYXRhPW1zc3RhdHNfcXVhbnQpKQp9CmBgYAoKIyMjIFAvUEUgcHJvdGVpbiBRQyBwbG90cyBmb3IgWWFuCgpZYW4gYXNrZWQgZm9yIHRoZSBwL3BlIHByb3RlaW4gcWMgcGxvdHMuIG9rLiAgSSBjaGFuZ2VkIHRoZSBkYXRhUHJvY2Vzc1Bsb3RzIHRvCnJldHVybiBzb21ldGhpbmcgdXNlZnVsLCBzbyB0aGF0IHNob3VsZCBiZSBwb3NzaWJsZSBub3cuCgpgYGB7ciBwZSwgZXZhbD1GQUxTRX0KcGVfZ2VuZXMgPC0gcmVhZC50YWJsZSgicmVmZXJlbmNlL2Fubm90YXRlZF9wZV9nZW5lcy50eHQiKVtbMV1dCgojIyBVbmZvcnR1bmF0ZWx5LCB0aGUgbmFtZXMgZGlkIG5vdCBnZXQgc2V0IGluIG15IGNoYW5nZWQgdmVyc2lvbiBvZiBkYXRhUHJvY2Vzc1Bsb3RzLi4uCnBsb3Rsc3QgPC0gbXNzdGF0c19wbG90cyRRQ1BMT1QKYXZhaWxhYmxlX3Bsb3RzIDwtIGdzdWIocGF0dGVybj0iXjEvIiwgcmVwbGFjZW1lbnQ9IiIsIHg9bGV2ZWxzKG1zc3RhdHNfcXVhbnQkUHJvY2Vzc2VkRGF0YSRQUk9URUlOKSkKbmFtZXMocGxvdGxzdCkgPC0gYXZhaWxhYmxlX3Bsb3RzCgpwZV9pbl9hdmFpbF9pZHggPC0gcGVfZ2VuZXMgJWluJSBhdmFpbGFibGVfcGxvdHMKcGVfaW5fYXZhaWwgPC0gcGVfZ2VuZXNbcGVfaW5fYXZhaWxfaWR4XQpwZV9wbG90cyA8LSBwbG90bHN0W3BlX2luX2F2YWlsXQpwZGYoZmlsZT0icGVfcWNfcGxvdHMucGRmIikKZm9yIChwIGluIDE6bGVuZ3RoKHBlX3Bsb3RzKSkgewogIHBsb3QocGVfcGxvdHNbW3BdXSkKfQpkZXYub2ZmKCkKYGBgCgojIENyZWF0ZSBocGdsdG9vbHMgZXhwcmVzc2lvbnNldAoKU2luY2UgSSBhbSBub3QgY2VydGFpbiBJIHVuZGVyc3RhbmQgdGhlc2UgZGF0YSwgSSB3aWxsIHRha2UgdGhlIGludGVuc2l0aWVzIGZyb20KU1dBVEgyc3RhdHMsIG1ldGFkYXRhLCBhbmQgYW5ub3RhdGlvbiBkYXRhOyAgYXR0ZW1wdCB0byBjcmVhdGUgYSAnbm9ybWFsJwpleHByZXNzaW9uc2V0OyBwb2tlIGF0IGl0IHRvIHNlZSB3aGF0IEkgY2FuIGxlYXJuLgoKIyMgTWFzc2FnaW5nIHRoZSBtZXRhZGF0YQoKSSB3YW50IHRvIHVzZSB0aGUgc2FtZSBtZXRhZGF0YSBhcyB3ZXJlIHVzZWQgZm9yIE1Tc3RhdHMuICBJdCBoYXMgYSBmZXcKaW1wb3J0YW50IGRpZmZlcmVuY2VzIGZyb20gdGhlIHJlcXVpcmVtZW50cyBvZiBocGdsdG9vbHM6IHByZXR0eSBtdWNoIG9ubHkgdGhhdApJIGRvIG5vdCBhbGxvdyByb3duYW1lcy9zYW1wbGVJRHMgdG8gc3RhcnQgd2l0aCBhIG51bWJlci4KCmBgYHtyIHByb3RlaW5fcGVwdGlkZV9tYXRyaWNlc30KbWV0YWRhdGEgPC0gc2FtcGxlX2Fubm90Cm1ldGFkYXRhW1sic2FtcGxlaWQiXV0gPC0gcGFzdGUwKCJzIiwgbWV0YWRhdGFbWyJzYW1wbGVpZCJdXSkKcm93bmFtZXMobWV0YWRhdGEpIDwtIG1ldGFkYXRhW1sic2FtcGxlaWQiXV0KYGBgCgojIyBNYXNzYWdpbmcgdGhlIGdlbmUgYW5ub3RhdGlvbiBkYXRhIGFuZCBhZGRpbmcgdGhlIG1zc3RhdHMgZGF0YS4KCkkgaGF2ZSBteSBvd24gYW5ub3RhdGlvbiBkYXRhIGZyb20gdGhlIGdmZiBmaWxlL21pY3JvYmVzb25saW5lL3doYXRldmVyLCBJIGNhbgphZGQgdGhlIE1Tc3RhdHMgcmVzdWx0IHRvIGl0IHNvIHRoYXQgbGF0ZXIgSSBjYW4gcHJpbnQgdGhlbSBhbGwgdG9nZXRoZXIuCgpgYGB7ciBtc3N0YXRzX2Fubm90YXRpb25zfQojIyBIZXJlIGlzIGEgbmVhdCBsaXR0bGUgdGhpbmcgSSBjYW4gZG86ICBBZGQgdGhlIE1Tc3RhdHMgcmVzdWx0cyB0byBteSBhbm5vdGF0aW9uIGRhdGEuCiMjIFRoZW4gd2hlbiBJIHByaW50IG91dCB0aGUgdGFibGVzIG9mIHRoZSBsaW1tYS9ldGMgcmVzdWx0cywgdGhleSBNU3N0YXRzCiMjIHJlc3VsdHMgd2lsbCBjb21lIGFsb25nIGZvciBmcmVlLgpiaWdfdGFibGUgPC0gcmVzdWx0c1tbMV1dCmZvciAociBpbiAyOmxlbmd0aChyZXN1bHRzKSkgewogIGJpZ190YWJsZSA8LSBtZXJnZShiaWdfdGFibGUsIHJlc3VsdHNbW3JdXSwgYnk9IlByb3RlaW4iKQp9CnJvd25hbWVzKGJpZ190YWJsZSkgPC0gYmlnX3RhYmxlW1siUHJvdGVpbiJdXQpiaWdfdGFibGUgPC0gYmlnX3RhYmxlWywgLTFdCgptdGJfYW5ub3RhdGlvbnNfd2l0aF9tc3N0YXRzIDwtIG1lcmdlKG10Yl9hbm5vdGF0aW9ucywgYmlnX3RhYmxlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJ5PSJyb3cubmFtZXMiLCBhbGwueD1UUlVFKQpyb3duYW1lcyhtdGJfYW5ub3RhdGlvbnNfd2l0aF9tc3N0YXRzKSA8LSBtdGJfYW5ub3RhdGlvbnNfd2l0aF9tc3N0YXRzW1siUm93Lm5hbWVzIl1dCm10Yl9hbm5vdGF0aW9uc193aXRoX21zc3RhdHMgPC0gbXRiX2Fubm90YXRpb25zX3dpdGhfbXNzdGF0c1ssIC0xXQpgYGAKCiMjIE1hc3NhZ2luZyB0aGUgaW50ZW5zaXR5IG1hdHJpeAoKSSBkbyBub3Qgd2FudCB0aGUgXDEgYmVmb3JlIHRoZSBwcm90ZWluIG5hbWVzLCBJIGFscmVhZHkgbWVyZ2VkIHRoZW0gaW50byBvbmUKZW50cnkgcGVyIGdlbmUgdmlzIFNXQVRIMnN0YXRzLgoKYGBge3IgcHJvdGVpbl9tYXRyaXh9CnByb3RfbXRyeCA8LSByZWFkLmNzdihwYXN0ZTAoInJlc3VsdHMvc3dhdGgyc3RhdHNfIiwgdmVyLCAiL3Byb3RlaW5fbWF0cml4X21pbmltdW0uY3N2IikpCnJvd25hbWVzKHByb3RfbXRyeCkgPC0gZ3N1YihwYXR0ZXJuPSJeMVxcLyIsIHJlcGxhY2VtZW50PSIiLCB4PXByb3RfbXRyeFtbInByb3RlaW5uYW1lIl1dKQpwcm90X210cnggPC0gcHJvdF9tdHJ4WywgLTFdCiMjIEltcG9ydGFudCBxdWVzdGlvbjogRGlkIFNXQVRIMnN0YXRzIHJlb3JkZXIgbXkgZGF0YT8KZnVuIDwtIGdzdWIocGF0dGVybj0iXi4qXygyMDE4LiokKSIsIHJlcGxhY2VtZW50PSJcXDEiLCB4PWNvbG5hbWVzKHByb3RfbXRyeCkpCmNvbG5hbWVzKHByb3RfbXRyeCkgPC0gcGFzdGUwKCJzIiwgZnVuKQpgYGAKCiMjIE1lcmdlIHRoZSBwaWVjZXMKCk5vdyB3ZSBzaG91bGQgaGF2ZSBzdWZmaWNpZW50IHBpZWNlcyB0byBtYWtlIGFuIGV4cHJlc3Npb25zZXQuCgpgYGB7ciBleHB0fQojIyBEcm9wIHRoZSBtZXRhZGF0YSBub3QgaW4gdGhlIHByb3RlaW4gbWF0cml4OgojIyBBbmQgZW5zdXJlIHRoYXQgdGhleSBhcmUgdGhlIHNhbWUgb3JkZXIuCnJlb3JkZXJlZCA8LSBjb2xuYW1lcyhwcm90X210cngpCm1ldGFkYXRhIDwtIG1ldGFkYXRhW3Jlb3JkZXJlZCwgXQoKcHJvdGVpbl9leHB0IDwtIGNyZWF0ZV9leHB0KG1ldGFkYXRhLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgY291bnRfZGF0YWZyYW1lPXByb3RfbXRyeCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIGdlbmVfaW5mbz1tdGJfYW5ub3RhdGlvbnNfd2l0aF9tc3N0YXRzKQp3aG9sZV9leHB0IDwtIHN1YnNldF9leHB0KHByb3RlaW5fZXhwdCwgc3Vic2V0PSJjb2xsZWN0aW9udHlwZT09J3dob2xlJyIpCmNmX2V4cHQgPC0gc3Vic2V0X2V4cHQocHJvdGVpbl9leHB0LCBzdWJzZXQ9ImNvbGxlY3Rpb250eXBlPT0nY2YnIikKYGBgCgpgYGB7ciBwcm90ZWluX21ldHJpY3MsIGZpZy5zaG93PSdoaWRlJ30KcHJvdGVpbl9tZXRyaWNzIDwtIHNtKGdyYXBoX21ldHJpY3MocHJvdGVpbl9leHB0KSkKcHJvdGVpbl9ub3JtIDwtIHNtKG5vcm1hbGl6ZV9leHB0KHByb3RlaW5fZXhwdCwgdHJhbnNmb3JtPSJsb2cyIiwgY29udmVydD0iY3BtIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5vcm09InF1YW50IiwgZmlsdGVyPVRSVUUpKQpwcm90ZWluX25vcm1fbWV0cmljcyA8LSBzbShncmFwaF9tZXRyaWNzKHByb3RlaW5fbm9ybSkpCnByb3RlaW5fZnN2YSA8LSBzbShub3JtYWxpemVfZXhwdChwcm90ZWluX2V4cHQsIHRyYW5zZm9ybT0ibG9nMiIsIGNvbnZlcnQ9ImNwbSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiYXRjaD0iZnN2YSIsIGZpbHRlcj1UUlVFKSkKcHJvdGVpbl9mc3ZhX21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyhwcm90ZWluX2ZzdmEpKQpgYGAKCmBgYHtyIHdob2xlX21ldHJpY3MsIGZpZy5zaG93PSdoaWRlJ30Kd2hvbGVfbWV0cmljcyA8LSBzbShncmFwaF9tZXRyaWNzKHdob2xlX2V4cHQpKQp3aG9sZV9ub3JtIDwtIHNtKG5vcm1hbGl6ZV9leHB0KHdob2xlX2V4cHQsIHRyYW5zZm9ybT0ibG9nMiIsIGNvbnZlcnQ9ImNwbSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBub3JtPSJxdWFudCIsIGZpbHRlcj1UUlVFKSkKd2hvbGVfbm9ybV9tZXRyaWNzIDwtIHNtKGdyYXBoX21ldHJpY3Mod2hvbGVfbm9ybSkpCndob2xlX2ZzdmEgPC0gc20obm9ybWFsaXplX2V4cHQod2hvbGVfZXhwdCwgdHJhbnNmb3JtPSJsb2cyIiwgY29udmVydD0iY3BtIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJhdGNoPSJmc3ZhIiwgZmlsdGVyPVRSVUUpKQp3aG9sZV9mc3ZhX21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyh3aG9sZV9mc3ZhKSkKYGBgCgpgYGB7ciBjZl9tZXRyaWNzLCBmaWcuc2hvdz0naGlkZSd9CmNmX21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyhjZl9leHB0KSkKY2Zfbm9ybSA8LSBzbShub3JtYWxpemVfZXhwdChjZl9leHB0LCB0cmFuc2Zvcm09ImxvZzIiLCBjb252ZXJ0PSJjcG0iLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbm9ybT0icXVhbnQiLCBmaWx0ZXI9VFJVRSkpCmNmX25vcm1fbWV0cmljcyA8LSBzbShncmFwaF9tZXRyaWNzKGNmX25vcm0pKQpjZl9mc3ZhIDwtIHNtKG5vcm1hbGl6ZV9leHB0KGNmX2V4cHQsIHRyYW5zZm9ybT0ibG9nMiIsIGNvbnZlcnQ9ImNwbSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiYXRjaD0iZnN2YSIsIGZpbHRlcj1UUlVFKSkKY2ZfZnN2YV9tZXRyaWNzIDwtIHNtKGdyYXBoX21ldHJpY3MoY2ZfZnN2YSkpCmBgYAoKYGBge3IgcHJpbnRfbWV0cmljc30KcHAoaW1hZ2U9cHJvdGVpbl9tZXRyaWNzJGxpYnNpemUsIGZpbGU9ImltYWdlcy8yMDE4MDUyM19saWJzaXplLnBuZyIpCnBwKGltYWdlPXByb3RlaW5fbm9ybV9tZXRyaWNzJHBjYXBsb3QsIGZpbGU9ImltYWdlcy8yMDE4MDUyM19ub3JtX3BjYS5wbmciKQpwcChpbWFnZT1wcm90ZWluX2ZzdmFfbWV0cmljcyRwY2FwbG90LCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfZnN2YV9wY2EucG5nIikKcHAoaW1hZ2U9cHJvdGVpbl9ub3JtX21ldHJpY3MkY29yaGVhdCwgZmlsZT0iaW1hZ2VzLzIwMTgwNTIzX25vcm1fY29yaGVhdC5wbmciKQpwcChpbWFnZT1wcm90ZWluX21ldHJpY3MkZGVuc2l0eSwgZmlsZT0iaW1hZ2VzLzIwMTgwNTIzX3Jhd19kZW5zaXR5LnBuZyIpCnBwKGltYWdlPXByb3RlaW5fbWV0cmljcyRib3hwbG90LCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfYm94cGxvdC5wbmciKQoKcHAoaW1hZ2U9d2hvbGVfbWV0cmljcyRsaWJzaXplLCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfd2hvbGVfbGlic2l6ZS5wbmciKQpwcChpbWFnZT13aG9sZV9ub3JtX21ldHJpY3MkcGNhcGxvdCwgZmlsZT0iaW1hZ2VzLzIwMTgwNTIzX3dob2xlX25vcm1fcGNhLnBuZyIpCnBwKGltYWdlPXdob2xlX2ZzdmFfbWV0cmljcyRwY2FwbG90LCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfd2hvbGVfZnN2YV9wY2EucG5nIikKcHAoaW1hZ2U9d2hvbGVfbm9ybV9tZXRyaWNzJGNvcmhlYXQsIGZpbGU9ImltYWdlcy8yMDE4MDUyM193aG9sZV9ub3JtX2NvcmhlYXQucG5nIikKcHAoaW1hZ2U9d2hvbGVfbWV0cmljcyRkZW5zaXR5LCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfd2hvbGVfcmF3X2RlbnNpdHkucG5nIikKcHAoaW1hZ2U9d2hvbGVfbWV0cmljcyRib3hwbG90LCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfd2hvbGVfYm94cGxvdC5wbmciKQoKcHAoaW1hZ2U9Y2ZfbWV0cmljcyRsaWJzaXplLCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfbGlic2l6ZS5wbmciKQpwcChpbWFnZT1jZl9ub3JtX21ldHJpY3MkcGNhcGxvdCwgZmlsZT0iaW1hZ2VzLzIwMTgwNTIzX25vcm1fcGNhLnBuZyIpCnBwKGltYWdlPWNmX2ZzdmFfbWV0cmljcyRwY2FwbG90LCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfZnN2YV9wY2EucG5nIikKcHAoaW1hZ2U9Y2Zfbm9ybV9tZXRyaWNzJGNvcmhlYXQsIGZpbGU9ImltYWdlcy8yMDE4MDUyM19ub3JtX2NvcmhlYXQucG5nIikKcHAoaW1hZ2U9Y2ZfbWV0cmljcyRkZW5zaXR5LCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfcmF3X2RlbnNpdHkucG5nIikKcHAoaW1hZ2U9Y2ZfbWV0cmljcyRib3hwbG90LCBmaWxlPSJpbWFnZXMvMjAxODA1MjNfYm94cGxvdC5wbmciKQpgYGAKCiMgQXR0ZW1wdCBzb21lIHF1YW50aWZpY2F0aW9uIGNvbXBhcmlzb25zPwoKYGBge3IgZGVfdGVzdGluZ30KcGFpcndpc2VfZmlsdCA8LSBzbShub3JtYWxpemVfZXhwdChwcm90ZWluX2V4cHQsIGZpbHRlcj1UUlVFKSkKcGFpcndpc2VfY29tcCA8LSBzbShhbGxfcGFpcndpc2UocGFpcndpc2VfZmlsdCwgbW9kZWxfYmF0Y2g9ImZzdmEiLCBmb3JjZT1UUlVFKSkKYGBgCgojIEZvciBlYWNoIG1zc3RhdHMgcnVuLCBkbyBhIERFIHRhYmxlCgpgYGB7ciBtdWx0aV90YWxlc30Ka2VlcGVycyA8LSBsaXN0KAogICJ3dF9jZl9taW51c193dF93aG9sZSIgPSBjKCJ3dF9jZiIsICJ3dF93aG9sZSIpLAogICJkZWx0YV9jZl9taW51c19kZWx0YV93aG9sZSIgPSBjKCJkZWx0YV9jZiIsICJkZWx0YV93aG9sZSIpLAogICJjb21wX2NmX21pbnVzX2NvbXBfd2hvbGUiID0gYygiY29tcF9jZiIsICJjb21wX3dob2xlIiksCiAgImRlbHRhX2NmX21pbnVzX3d0X2NmIiA9IGMoImRlbHRhX2NmIiwgInd0X2NmIiksCiAgImNvbXBfY2ZfbWludXNfd3RfY2YiID0gYygiY29tcF9jZiIsICJ3dF9jZiIpLAogICJkZWx0YV93aG9sZV9taW51c193dF93aG9sZSIgPSBjKCJkZWx0YV93aG9sZSIsICJ3dF93aG9sZSIpLAogICJjb21wX3dob2xlX21pbnVzX3d0X3dob2xlIiA9IGMoImNvbXBfd2hvbGUiLCAid3Rfd2hvbGUiKSkKcGFpcndpc2VfdGFibGVzIDwtIHNtKGNvbWJpbmVfZGVfdGFibGVzKAogIHBhaXJ3aXNlX2NvbXAsCiAgZXhjZWw9cGFzdGUwKCJleGNlbC9wYWlyd2lzZV9kZV93aXRoX21zc3RhdHMtdiIsIHZlciwgIi54bHN4IiksCiAga2VlcGVycz1rZWVwZXJzKSkKcGFpcndpc2Vfc2lnIDwtIHNtKGV4dHJhY3Rfc2lnbmlmaWNhbnRfZ2VuZXMoCiAgcGFpcndpc2VfdGFibGVzLAogIGV4Y2VsPXBhc3RlMCgiZXhjZWwvdGVzdF9wYWlyd2lzZV9zaWdfd2l0aF9tc3N0YXRzLXYiLCB2ZXIsICIueGxzeCIpKSkKYGBgCgpgYGB7ciBtc3N0YXRzX29ubHksIGV2YWw9RkFMU0V9CmRyb3BwZXJzIDwtIGMoInVuZGVmaW5lZCIpCm5hbWVzKGRyb3BwZXJzKSA8LSAibG9nMmZjIgpwYWlyd2lzZV9vbmx5YWxsIDwtIHNtKGNvbWJpbmVfZGVfdGFibGVzKAogIHBhaXJ3aXNlX2NvbXAsCiAgZXhjZWw9cGFzdGUwKCJleGNlbC90ZXN0X3BhaXJ3aXNlX2RlX29ubHlfbXNzdGF0cy12IiwgdmVyLCAiLnhsc3giKSwKICBrZWVwZXJzPWtlZXBlcnMsCiAgZXhjbHVkZXM9ZHJvcHBlcnMpKQpgYGAKCmBgYHtyIHNvbG9fZmVhdHVyZXN9CnNvbG9fcHJvdGVpbnMgPC0gZmVhdHVyZXNfaW5fc2luZ2xlX2NvbmRpdGlvbihwcm90ZWluX2V4cHQpCnByb3RlaW5zX29ubHlfY2YgPC0gc29sb19wcm90ZWluc1tbInNvbG9fdGhpcyJdXVtbInd0X2NmIl1dCnByb3RlaW5zX29ubHlfY2YKcHJvdGVpbnNfb25seV93aG9sZSA8LSBzb2xvX3Byb3RlaW5zW1sic29sb190aGlzIl1dW1sid3Rfd2hvbGUiXV0KbGVuZ3RoKHByb3RlaW5zX29ubHlfd2hvbGUpCmBgYAoKIyMgQ29tcGFyZSBocGdsdG9vbHMvTVNzdGF0cwoKQ2FuIHdlIGNvbXBhcmUgbGltbWEgYW5kIHJpZW5kcyB0byBNU3N0YXRzPwoKIyMjIHd0IGNmIC8gd3Qgd2hvbGUKCmBgYHtyIGNvbXBhcmVfY29tcGFyaXNvbnN9CmhwZ2xfdGFibGUgPC0gcGFpcndpc2VfdGFibGVzW1siZGF0YSJdXVtbInd0X2NmX3ZzX3d0X3dob2xlIl1dCm1zc3RhdHNfdGFibGUgPC0gcmVzdWx0c1tbIgoKCm1zc3RhdHMKbXNzdGF0c190YWJsZSA8LSBtc3N0YXRzX2NvbXBhcmlzb24kQ29tcGFyaXNvblJlc3VsdApyb3duYW1lcyhtc3N0YXRzX3RhYmxlKSA8LSBnc3ViKHBhdHRlcm49Il4xLyIsIHJlcGxhY2VtZW50PSIiLCB4PW1zc3RhdHNfdGFibGUkUHJvdGVpbikKCm1lcmdlZF90YWJsZSA8LSBtZXJnZShocGdsX3RhYmxlLCBtc3N0YXRzX3RhYmxlLCBieT0icm93Lm5hbWVzIikKd3JpdGUuY3N2KGZpbGU9ImltYWdlcy9tZXJnZWRfdGFibGUuY3N2IiwgbWVyZ2VkX3RhYmxlKQpjb3IudGVzdChtZXJnZWRfdGFibGUkbGltbWFfbG9nZmMsIG1lcmdlZF90YWJsZSRsb2cyRkMpCmNvci50ZXN0KG1lcmdlZF90YWJsZSRsaW1tYV9sb2dmYywgbWVyZ2VkX3RhYmxlJGxvZzJGQywgbWV0aG9kPSJzcGVhcm1hbiIpCmNvci50ZXN0KG1lcmdlZF90YWJsZSRkZXNlcV9sb2dmYywgbWVyZ2VkX3RhYmxlJGxvZzJGQykKY29yLnRlc3QobWVyZ2VkX3RhYmxlJGRlc2VxX2xvZ2ZjLCBtZXJnZWRfdGFibGUkbG9nMkZDLCBtZXRob2Q9InNwZWFybWFuIikKY29yLnRlc3QobWVyZ2VkX3RhYmxlJGVkZ2VyX2xvZ2ZjLCBtZXJnZWRfdGFibGUkbG9nMkZDKQpjb3IudGVzdChtZXJnZWRfdGFibGUkZWRnZXJfbG9nZmMsIG1lcmdlZF90YWJsZSRsb2cyRkMsIG1ldGhvZD0ic3BlYXJtYW4iKQoKY29tYmluZWRfdGFibGUgPC0gcGFpcndpc2VfdGFibGVzJGRhdGFbWzFdXQp1bmRlZmluZWRfaWR4IDwtIGNvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dID09ICJ1bmRlZmluZWQiCmNvbWJpbmVkX3RhYmxlW3VuZGVmaW5lZF9pZHgsICJsb2cyZmMiXSA8LSBOQQpjb21iaW5lZF90YWJsZVtbImxvZzJmYyJdXSA8LSBhcy5udW1lcmljKGNvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dKQpjb3IudGVzdChjb21iaW5lZF90YWJsZVtbImxpbW1hX2xvZ2ZjIl1dLCBjb21iaW5lZF90YWJsZVtbImxvZzJmYyJdXSkKY29yLnRlc3QoY29tYmluZWRfdGFibGVbWyJsaW1tYV9sb2dmYyJdXSwgY29tYmluZWRfdGFibGVbWyJsb2cyZmMiXV0sIG1ldGhvZD0ic3BlYXJtYW4iKQoKaGlnaF90b19sb3dfaWR4IDwtIG9yZGVyKGNvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dLCBuYS5sYXN0PVRSVUUsIGRlY3JlYXNpbmc9VFJVRSkKbG93X3RvX2hpZ2hfaWR4IDwtIG9yZGVyKGNvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dLCBuYS5sYXN0PVRSVUUsIGRlY3JlYXNpbmc9RkFMU0UpCnRvcF81MCA8LSBoZWFkKGNvbWJpbmVkX3RhYmxlW2hpZ2hfdG9fbG93X2lkeCwgXSwgbj0xMDApCmJvdHRvbV81MCA8LSBoZWFkKGNvbWJpbmVkX3RhYmxlW2xvd190b19oaWdoX2lkeCwgXSwgbj0xMDApCgpjb3IudGVzdCh0b3BfNTAkbGltbWFfbG9nZmMsIHRvcF81MCRsb2cyZmMpCmNvci50ZXN0KHRvcF81MCRsaW1tYV9sb2dmYywgdG9wXzUwJGxvZzJmYywgbWV0aG9kPSJzcGVhcm1hbiIpCmNvci50ZXN0KHRvcF81MCRkZXNlcV9sb2dmYywgdG9wXzUwJGxvZzJmYykKY29yLnRlc3QodG9wXzUwJGRlc2VxX2xvZ2ZjLCB0b3BfNTAkbG9nMmZjLCBtZXRob2Q9InNwZWFybWFuIikKY29yLnRlc3QodG9wXzUwJGVkZ2VyX2xvZ2ZjLCB0b3BfNTAkbG9nMmZjKQpjb3IudGVzdCh0b3BfNTAkZWRnZXJfbG9nZmMsIHRvcF81MCRsb2cyZmMsIG1ldGhvZD0ic3BlYXJtYW4iKQoKY29yLnRlc3QoYm90dG9tXzUwJGxpbW1hX2xvZ2ZjLCBib3R0b21fNTAkbG9nMmZjKQpjb3IudGVzdChib3R0b21fNTAkbGltbWFfbG9nZmMsIGJvdHRvbV81MCRsb2cyZmMsIG1ldGhvZD0ic3BlYXJtYW4iKQpjb3IudGVzdChib3R0b21fNTAkZGVzZXFfbG9nZmMsIGJvdHRvbV81MCRsb2cyZmMpCmNvci50ZXN0KGJvdHRvbV81MCRkZXNlcV9sb2dmYywgYm90dG9tXzUwJGxvZzJmYywgbWV0aG9kPSJzcGVhcm1hbiIpCmNvci50ZXN0KGJvdHRvbV81MCRlZGdlcl9sb2dmYywgYm90dG9tXzUwJGxvZzJmYykKY29yLnRlc3QoYm90dG9tXzUwJGVkZ2VyX2xvZ2ZjLCBib3R0b21fNTAkbG9nMmZjLCBtZXRob2Q9InNwZWFybWFuIikKCnVwX2luX21zc3RhdHMgPC0gcm93bmFtZXMoY29tYmluZWRfdGFibGUpW2NvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dID4gMF0KdXBfaW5fbXNzdGF0cyA8LSB1cF9pbl9tc3N0YXRzWyFpcy5uYSh1cF9pbl9tc3N0YXRzKV0KdXBfaW5fbGltbWEgPC0gcm93bmFtZXMoY29tYmluZWRfdGFibGUpW2NvbWJpbmVkX3RhYmxlW1sibGltbWFfbG9nZmMiXV0gPiAwXQp1cF9pbl9saW1tYSA8LSB1cF9pbl9saW1tYVshaXMubmEodXBfaW5fbGltbWEpXQp1cF9pbl9lZGdlciA8LSByb3duYW1lcyhjb21iaW5lZF90YWJsZSlbY29tYmluZWRfdGFibGVbWyJlZGdlcl9sb2dmYyJdXSA+IDBdCnVwX2luX2VkZ2VyIDwtIHVwX2luX2VkZ2VyWyFpcy5uYSh1cF9pbl9lZGdlcildCnVwX3Zlbm5fc2V0cyA8LSBsaXN0KAogICJtc3N0YXRzIiA9IHVwX2luX21zc3RhdHMsCiAgImxpbW1hIiA9IHVwX2luX2xpbW1hLAogICJlZGdlciIgPSB1cF9pbl9lZGdlcikKdGVzdGluZyA8LSBWZW5uZXJhYmxlOjpWZW5uKFNldHM9dXBfdmVubl9zZXRzLCApCnBwKGZpbGU9Ii90bXAvdXBfdmVubi5wbmciKQpWZW5uZXJhYmxlOjpwbG90KHRlc3RpbmcsIGRvV2VpZ2h0cz1GQUxTRSkKZGV2Lm9mZigpClZlbm5lcmFibGU6OnBsb3QodGVzdGluZywgZG9XZWlnaHRzPUZBTFNFKQoKZG93bl9pbl9tc3N0YXRzIDwtIHJvd25hbWVzKGNvbWJpbmVkX3RhYmxlKVtjb21iaW5lZF90YWJsZVtbImxvZzJmYyJdXSA8IDBdCmRvd25faW5fbXNzdGF0cyA8LSBkb3duX2luX21zc3RhdHNbIWlzLm5hKGRvd25faW5fbXNzdGF0cyldCmRvd25faW5fbGltbWEgPC0gcm93bmFtZXMoY29tYmluZWRfdGFibGUpW2NvbWJpbmVkX3RhYmxlW1sibGltbWFfbG9nZmMiXV0gPCAwXQpkb3duX2luX2xpbW1hIDwtIGRvd25faW5fbGltbWFbIWlzLm5hKGRvd25faW5fbGltbWEpXQpkb3duX2luX2VkZ2VyIDwtIHJvd25hbWVzKGNvbWJpbmVkX3RhYmxlKVtjb21iaW5lZF90YWJsZVtbImVkZ2VyX2xvZ2ZjIl1dIDwgMF0KZG93bl9pbl9lZGdlciA8LSBkb3duX2luX2VkZ2VyWyFpcy5uYShkb3duX2luX2VkZ2VyKV0KZG93bl92ZW5uX3NldHMgPC0gbGlzdCgKICAibXNzdGF0cyIgPSBkb3duX2luX21zc3RhdHMsCiAgImxpbW1hIiA9IGRvd25faW5fbGltbWEsCiAgImVkZ2VyIiA9IGRvd25faW5fZWRnZXIpCnRlc3RpbmcgPC0gVmVubmVyYWJsZTo6VmVubihTZXRzPWRvd25fdmVubl9zZXRzLCApCnBwKGZpbGU9Ii90bXAvZG93bl92ZW5uLnBuZyIpClZlbm5lcmFibGU6OnBsb3QodGVzdGluZywgZG9XZWlnaHRzPUZBTFNFKQpkZXYub2ZmKCkKYGBgCgojIFJlcGVhdCB3aXRoIHRoZSBtb2RpZmllZCBkYXRhCgpgYGB7ciBjb21wYXJlX2NvbXBhcmlzb25zX21vZGlmaWVkfQpocGdsX3RhYmxlIDwtIHBhaXJ3aXNlX21vZGlmaWVkX3RhYmxlcyRkYXRhW1sxXV0KbXNzdGF0c190YWJsZSA8LSBtc3N0YXRzX21vZGlmaWVkX2NvbXAkQ29tcGFyaXNvblJlc3VsdApyb3duYW1lcyhtc3N0YXRzX3RhYmxlKSA8LSBnc3ViKHBhdHRlcm49Il4xLyIsIHJlcGxhY2VtZW50PSIiLCB4PW1zc3RhdHNfdGFibGVfbW9kaWZpZWQkUHJvdGVpbikKCm1lcmdlZF90YWJsZSA8LSBtZXJnZShocGdsX3RhYmxlLCBtc3N0YXRzX3RhYmxlLCBieT0icm93Lm5hbWVzIikKd3JpdGUuY3N2KGZpbGU9ImltYWdlcy9tZXJnZWRfdGFibGVfbW9kaWZpZWQuY3N2IiwgbWVyZ2VkX3RhYmxlKQpjb3IudGVzdChtZXJnZWRfdGFibGUkbGltbWFfbG9nZmMsIG1lcmdlZF90YWJsZSRsb2cyRkMpCmNvci50ZXN0KG1lcmdlZF90YWJsZSRsaW1tYV9sb2dmYywgbWVyZ2VkX3RhYmxlJGxvZzJGQywgbWV0aG9kPSJzcGVhcm1hbiIpCmNvci50ZXN0KG1lcmdlZF90YWJsZSRkZXNlcV9sb2dmYywgbWVyZ2VkX3RhYmxlJGxvZzJGQykKY29yLnRlc3QobWVyZ2VkX3RhYmxlJGRlc2VxX2xvZ2ZjLCBtZXJnZWRfdGFibGUkbG9nMkZDLCBtZXRob2Q9InNwZWFybWFuIikKY29yLnRlc3QobWVyZ2VkX3RhYmxlJGVkZ2VyX2xvZ2ZjLCBtZXJnZWRfdGFibGUkbG9nMkZDKQpjb3IudGVzdChtZXJnZWRfdGFibGUkZWRnZXJfbG9nZmMsIG1lcmdlZF90YWJsZSRsb2cyRkMsIG1ldGhvZD0ic3BlYXJtYW4iKQoKY29tYmluZWRfdGFibGUgPC0gcGFpcndpc2VfdGFibGVzJGRhdGFbWzFdXQp1bmRlZmluZWRfaWR4IDwtIGNvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dID09ICJ1bmRlZmluZWQiCmNvbWJpbmVkX3RhYmxlW3VuZGVmaW5lZF9pZHgsICJsb2cyZmMiXSA8LSBOQQpjb21iaW5lZF90YWJsZVtbImxvZzJmYyJdXSA8LSBhcy5udW1lcmljKGNvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dKQpjb3IudGVzdChjb21iaW5lZF90YWJsZVtbImxpbW1hX2xvZ2ZjIl1dLCBjb21iaW5lZF90YWJsZVtbImxvZzJmYyJdXSkKY29yLnRlc3QoY29tYmluZWRfdGFibGVbWyJsaW1tYV9sb2dmYyJdXSwgY29tYmluZWRfdGFibGVbWyJsb2cyZmMiXV0sIG1ldGhvZD0ic3BlYXJtYW4iKQoKaGlnaF90b19sb3dfaWR4IDwtIG9yZGVyKGNvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dLCBuYS5sYXN0PVRSVUUsIGRlY3JlYXNpbmc9VFJVRSkKbG93X3RvX2hpZ2hfaWR4IDwtIG9yZGVyKGNvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dLCBuYS5sYXN0PVRSVUUsIGRlY3JlYXNpbmc9RkFMU0UpCnRvcF81MCA8LSBoZWFkKGNvbWJpbmVkX3RhYmxlW2hpZ2hfdG9fbG93X2lkeCwgXSwgbj0xMDApCmJvdHRvbV81MCA8LSBoZWFkKGNvbWJpbmVkX3RhYmxlW2xvd190b19oaWdoX2lkeCwgXSwgbj0xMDApCgpjb3IudGVzdCh0b3BfNTAkbGltbWFfbG9nZmMsIHRvcF81MCRsb2cyZmMpCmNvci50ZXN0KHRvcF81MCRsaW1tYV9sb2dmYywgdG9wXzUwJGxvZzJmYywgbWV0aG9kPSJzcGVhcm1hbiIpCmNvci50ZXN0KHRvcF81MCRkZXNlcV9sb2dmYywgdG9wXzUwJGxvZzJmYykKY29yLnRlc3QodG9wXzUwJGRlc2VxX2xvZ2ZjLCB0b3BfNTAkbG9nMmZjLCBtZXRob2Q9InNwZWFybWFuIikKY29yLnRlc3QodG9wXzUwJGVkZ2VyX2xvZ2ZjLCB0b3BfNTAkbG9nMmZjKQpjb3IudGVzdCh0b3BfNTAkZWRnZXJfbG9nZmMsIHRvcF81MCRsb2cyZmMsIG1ldGhvZD0ic3BlYXJtYW4iKQoKY29yLnRlc3QoYm90dG9tXzUwJGxpbW1hX2xvZ2ZjLCBib3R0b21fNTAkbG9nMmZjKQpjb3IudGVzdChib3R0b21fNTAkbGltbWFfbG9nZmMsIGJvdHRvbV81MCRsb2cyZmMsIG1ldGhvZD0ic3BlYXJtYW4iKQpjb3IudGVzdChib3R0b21fNTAkZGVzZXFfbG9nZmMsIGJvdHRvbV81MCRsb2cyZmMpCmNvci50ZXN0KGJvdHRvbV81MCRkZXNlcV9sb2dmYywgYm90dG9tXzUwJGxvZzJmYywgbWV0aG9kPSJzcGVhcm1hbiIpCmNvci50ZXN0KGJvdHRvbV81MCRlZGdlcl9sb2dmYywgYm90dG9tXzUwJGxvZzJmYykKY29yLnRlc3QoYm90dG9tXzUwJGVkZ2VyX2xvZ2ZjLCBib3R0b21fNTAkbG9nMmZjLCBtZXRob2Q9InNwZWFybWFuIikKCnVwX2luX21zc3RhdHMgPC0gcm93bmFtZXMoY29tYmluZWRfdGFibGUpW2NvbWJpbmVkX3RhYmxlW1sibG9nMmZjIl1dID4gMF0KdXBfaW5fbXNzdGF0cyA8LSB1cF9pbl9tc3N0YXRzWyFpcy5uYSh1cF9pbl9tc3N0YXRzKV0KdXBfaW5fbGltbWEgPC0gcm93bmFtZXMoY29tYmluZWRfdGFibGUpW2NvbWJpbmVkX3RhYmxlW1sibGltbWFfbG9nZmMiXV0gPiAwXQp1cF9pbl9saW1tYSA8LSB1cF9pbl9saW1tYVshaXMubmEodXBfaW5fbGltbWEpXQp1cF9pbl9lZGdlciA8LSByb3duYW1lcyhjb21iaW5lZF90YWJsZSlbY29tYmluZWRfdGFibGVbWyJlZGdlcl9sb2dmYyJdXSA+IDBdCnVwX2luX2VkZ2VyIDwtIHVwX2luX2VkZ2VyWyFpcy5uYSh1cF9pbl9lZGdlcildCnVwX3Zlbm5fc2V0cyA8LSBsaXN0KAogICJtc3N0YXRzIiA9IHVwX2luX21zc3RhdHMsCiAgImxpbW1hIiA9IHVwX2luX2xpbW1hLAogICJlZGdlciIgPSB1cF9pbl9lZGdlcikKdGVzdGluZyA8LSBWZW5uZXJhYmxlOjpWZW5uKFNldHM9dXBfdmVubl9zZXRzLCApCnBwKGZpbGU9Ii90bXAvdXBfdmVubi5wbmciKQpWZW5uZXJhYmxlOjpwbG90KHRlc3RpbmcsIGRvV2VpZ2h0cz1GQUxTRSkKZGV2Lm9mZigpClZlbm5lcmFibGU6OnBsb3QodGVzdGluZywgZG9XZWlnaHRzPUZBTFNFKQoKZG93bl9pbl9tc3N0YXRzIDwtIHJvd25hbWVzKGNvbWJpbmVkX3RhYmxlKVtjb21iaW5lZF90YWJsZVtbImxvZzJmYyJdXSA8IDBdCmRvd25faW5fbXNzdGF0cyA8LSBkb3duX2luX21zc3RhdHNbIWlzLm5hKGRvd25faW5fbXNzdGF0cyldCmRvd25faW5fbGltbWEgPC0gcm93bmFtZXMoY29tYmluZWRfdGFibGUpW2NvbWJpbmVkX3RhYmxlW1sibGltbWFfbG9nZmMiXV0gPCAwXQpkb3duX2luX2xpbW1hIDwtIGRvd25faW5fbGltbWFbIWlzLm5hKGRvd25faW5fbGltbWEpXQpkb3duX2luX2VkZ2VyIDwtIHJvd25hbWVzKGNvbWJpbmVkX3RhYmxlKVtjb21iaW5lZF90YWJsZVtbImVkZ2VyX2xvZ2ZjIl1dIDwgMF0KZG93bl9pbl9lZGdlciA8LSBkb3duX2luX2VkZ2VyWyFpcy5uYShkb3duX2luX2VkZ2VyKV0KZG93bl92ZW5uX3NldHMgPC0gbGlzdCgKICAibXNzdGF0cyIgPSBkb3duX2luX21zc3RhdHMsCiAgImxpbW1hIiA9IGRvd25faW5fbGltbWEsCiAgImVkZ2VyIiA9IGRvd25faW5fZWRnZXIpCnRlc3RpbmcgPC0gVmVubmVyYWJsZTo6VmVubihTZXRzPWRvd25fdmVubl9zZXRzLCApCnBwKGZpbGU9Ii90bXAvZG93bl92ZW5uLnBuZyIpClZlbm5lcmFibGU6OnBsb3QodGVzdGluZywgZG9XZWlnaHRzPUZBTFNFKQpkZXYub2ZmKCkKYGBgCgojIEV2ZXJ5dGhpbmcgYmVsb3cgaGVyZSBtaWdodCBnZXQgZHJvcHBlZD8KCkkgdGhpbmsgSSB3ZWRnZWQgYWxsIG9mIHRoZSBmb2xsb3dpbmcgd29yayBpbnRvIHRoYXQgc2hvcnQgYmxvY2sgYWJvdmUuClNvLCBzdG9wIGV2YWx1YXRpbmcgdGhlIGZvbGxvd2luZyBibG9ja3MgYW5kIHNlZSBpZiB0aGF0IGlzIHRydWUuCgojIyMgRXhwcmVzc2lvbnNldCBmcm9tIHRoZSBUUklDIGludGVuc2l0eSBtYXRyaXgKCmBgYHtyIGhwZ2x0b29sc19leHB0LCBldmFsPUZBTFNFfQppbnRlbnNpdHlfbXRyeCA8LSByZWFkLmNzdihmaWxlPSJyZXN1bHRzL3RyaWMvSENEX291dG1hdHJpeC50c3YiLCBzZXA9Ilx0IikKIyMgVGhlIEhDRF9vdXRtYXRyaXggaGFzIGNvbHVtbnMgaW5jbHVkaW5nOiBQZXB0aWRlLCBQcm90ZWluLAojIyBBIHNlcmllcyBvZiBJbnRlbnNpdHlfc2FtcGxlX2lkIGNvbHVtbnMKIyMgQSBzZXJpZXMgb2YgUlRfc2FtcGxlX2lkIGNvbHVtbnMKIyMgQSBzZXJpZXMgb2Ygc2NvcmVfc2FtcGxlX2lkIGNvbHVtbnMKIyMgQW5kIGF0IHRoZSBlbmQ6IFJUX21lYW4sIFJUX3N0ZCwgYW5kIHBnX3B2YWx1ZS4KIyMgU2luY2UgU1dBVEgyc3RhdHMvTVNzdGF0cyB1c2VzIHRoZSBpbnRlbnNpdGllcywgbGV0cyBnZXQgdGhvc2UgY29sdW1ucyBvdXQsCiMjIHN0YW5kYXJkaXplIHRoZSBjb2x1bW4gbmFtZXMgdG8gbWF0Y2ggdGhlIGFubm90YXRpb24gZGF0YSwgYW5kIHVzZSB0aGVtIGZvcgojIyBjcmVhdGluZyBhbiBleHByZXNzaW9uc2V0LgppbnRlbnNpdHlfbXRyeFtbInJvd25hbWVzIl1dIDwtIGludGVuc2l0eV9tdHJ4W1siUHJvdGVpbiJdXQppbnRlbnNpdHlfbXRyeFtbInJvd25hbWVzIl1dIDwtIGdzdWIocGF0dGVybj0iXltbOmRpZ2l0Ol1dK1xcLyIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICByZXBsYWNlbWVudD0iIiwgeD1pbnRlbnNpdHlfbXRyeFtbInJvd25hbWVzIl1dKQojIyBTdGFuZGFyZGl6ZSB0aGUgcm93bmFtZXMsIHRoaXMgbWlnaHQgYmUgYSBiYWQgaWRlYSwgYXMgdGhpcyB3aWxsIGtlZXAKIyMgc2VwYXJhdGUgZXZlcnkgcGVwdGlkZSBmcm9tIGVhY2ggcHJvdGVpbiBhbmQgbm90IGRvIGFueXRoaW5nIHRvCiMjIHN1bS9tZWRpYW4vd2hhdGV2ZXIgdGhlbS4gIEJ1dCBmb3IgdGhlIHB1cnBvc2VzIG9mIHRlc3Rpbmcgb3V0IHRoZSBkYXRhIEkKIyMgdGhpbmsgaXQgaXMgb2suCnJvd25hbWVzKGludGVuc2l0eV9tdHJ4KSA8LSBtYWtlLm5hbWVzKGludGVuc2l0eV9tdHJ4W1sicm93bmFtZXMiXV0sIHVuaXF1ZT1UUlVFKQoKIyMgTm93IGxldHMgZ2V0IHJpZCBvZiB0aGUgZXh0cmFuZW91cyB0ZXh0IGluIHRoZSBjb2x1bW4gbmFtZXMgYW5kIHNpbXBsaWZ5IHRoZW0KIyMgdG8gdGhlIHNhbXBsZSBuYW1lcyBhcyByZWZlcmVuY2VkIGluIHRoZSBzYW1wbGUgc2hlZXQuCmFsbF9jb2x1bW5zIDwtIGNvbG5hbWVzKGludGVuc2l0eV9tdHJ4KQppbnRlbnNlX2NvbHVtbnMgPC0gZ3JlcGwocGF0dGVybj0iSW50ZW5zaXR5IiwgeD1hbGxfY29sdW1ucykKaW50ZW5zaXR5X210cnggPC0gaW50ZW5zaXR5X210cnhbLCBpbnRlbnNlX2NvbHVtbnNdCmFsbF9jb2x1bW5zIDwtIGNvbG5hbWVzKGludGVuc2l0eV9tdHJ4KQpuZXdfY29sdW1ucyA8LSBnc3ViKHBhdHRlcm49IkludGVuc2l0eV8iLCByZXBsYWNlbWVudD0iIiwgeD1hbGxfY29sdW1ucykKbmV3X2NvbHVtbnMgPC0gZ3N1YihwYXR0ZXJuPSJfZGlhXy4qJCIsIHJlcGxhY2VtZW50PSIiLCB4PW5ld19jb2x1bW5zKQpjb2xuYW1lcyhpbnRlbnNpdHlfbXRyeCkgPC0gcGFzdGUwKCJzIiwgbmV3X2NvbHVtbnMpCmludGVuc2l0eV9tdHJ4W2lzLm5hKGludGVuc2l0eV9tdHJ4KV0gPC0gMAoKIyMgTm8gY29sdW1ucyBpbiBhbiBleHByZXNzaW9uIHNldCBhcmUgYWxsb3dlZCB0byBzdGFydCB3aXRoIGEgbnVtYmVyLiAgSSBoYXZlCiMjIHVuc2VkIHByZWZpeGluZyBzYW1wbGVzIHdpdGggJ3MnIGFzIGEgc3RhbmRhcmQgdG8gaGFuZGxlIHRoaXMgcHJvYmxlbS4uLgptZXRhZGF0YSA8LSBzYW1wbGVfYW5ub3QKbWV0YWRhdGEkc2FtcGxlaWQgPC0gcGFzdGUwKCJzIiwgbWV0YWRhdGEkc2FtcGxlaWQpCmNvbG5hbWVzKGludGVuc2l0eV9tdHJ4KSA8LSBnc3ViKHBhdHRlcm49Il92c19IQ0QiLCByZXBsYWNlbWVudD0iIiwgeD1jb2xuYW1lcyhpbnRlbnNpdHlfbXRyeCkpCnJvd25hbWVzKG1ldGFkYXRhKSA8LSBtZXRhZGF0YSRzYW1wbGVpZAojIyBUaGVvcmV0aWNhbGx5IHRoaXMgaXMgbm90IG5lZWRlZCBhbnltb3JlLi4uCmludGVuc2l0eV9tdHJ4IDwtIGludGVuc2l0eV9tdHJ4Wywgcm93bmFtZXMobWV0YWRhdGEpXQojIyBPaywgYXQgdGhpcyBwb2ludCwgd2Ugc2hvdWxkIGhhdmUgYWxsIHRoZSBwaWVjZXMgZm9yIGEgbW9yZSBvciBsZXNzIG5vcm1hbCBleHByZXNzaW9uc2V0LgpkaW0oaW50ZW5zaXR5X210cngpCnRlc3RfZXhwdCA8LSBjcmVhdGVfZXhwdChtZXRhZGF0YT1tZXRhZGF0YSwKICAgICAgICAgICAgICAgICAgICAgICAgIGNvdW50X2RhdGFmcmFtZT1pbnRlbnNpdHlfbXRyeCwKICAgICAgICAgICAgICAgICAgICAgICAgIGdlbmVfaW5mbz1tdGJfYW5ub3RhdGlvbnMpCmBgYAoKIyMjIFBsYXkgd2l0aCB0aGUgaHBnbHRvb2xzIGRlcml2ZWQgZXhwcmVzc2lvbnNldCBvZiBwZXB0aWRlIGludGVuc2l0aWVzCgpMZXRzIHNlZSBpZiBhbnl0aGluZyBtYWtlcyBzZW5zZSBpbiB0aGlzIGludGVuc2l0eSBleHByZXNzaW9uc2V0LgoKYGBge3IgaHBnbHRvb2xzX2V4cHRfZXhhbWluZSwgZmlnLnNob3c9ImhpZGUiLCBldmFsPUZBTFNFfQojIyBGaXJzdCwgbG9nMiBhbmQgbm9ybWFsaXplIHRoZSBkYXRhLgp0ZXN0X25vcm0gPC0gbm9ybWFsaXplX2V4cHQodGVzdF9leHB0LCB0cmFuc2Zvcm09ImxvZzIiLCBjb252ZXJ0PSJjcG0iLCBub3JtPSJxdWFudGlsZSIsIGZpbHRlcj1UUlVFKQp0ZXN0X25vcm1iYXRjaCA8LSBzbShub3JtYWxpemVfZXhwdCh0ZXN0X2V4cHQsIHRyYW5zZm9ybT0ibG9nMiIsIGNvbnZlcnQ9ImNwbSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5vcm09InF1YW50aWxlIiwgZmlsdGVyPVRSVUUsIGJhdGNoPSJsaW1tYSIpKQoKdGVzdF9tZXRyaWNzIDwtIHNtKGdyYXBoX21ldHJpY3ModGVzdF9leHB0KSkKdGVzdF9tZXRyaWNzX25vcm0gPC0gc20oZ3JhcGhfbWV0cmljcyh0ZXN0X25vcm0pKQp0ZXN0X21ldHJpY3Nfbm9ybWJhdGNoIDwtIHNtKGdyYXBoX21ldHJpY3ModGVzdF9ub3JtYmF0Y2gpKQpgYGAKCk5vdyBsZXRzIHNlZSB3aGF0IHRoZSBkYXRhIGxvb2tzIGxpa2UsIGFzc3VtaW5nIEkgZGlkIG5vdCBkbyBhbnl0aGluZyBob3JyaWJsZQp0byBpdC4KCmBgYHtyIGhwZ2x0b29sc19wbG90cywgZXZhbD1GQUxTRX0KdGVzdF9tZXRyaWNzJGxlZ2VuZAp0ZXN0X21ldHJpY3MkbGlic2l6ZSAgIyMgV293IHRoZSBpbnRlbnNpdGllcyBnZXQgcmlkaWN1bG91cywgd2hhdD8KdGVzdF9tZXRyaWNzJG5vbnplcm8KIyMgSW50ZXJlc3RpbmcsIGJ1dCBub3RoaW5nIHdoaWNoIHN1cGVyLWp1bXBzIG91dCBhdCBtZQp0ZXN0X21ldHJpY3MkZGVuc2l0eQojIyBobW0gdmVyeSBpbnRlcmVzdGluZywgdGhlIGdvb2QgbmV3cyBpcyB0aGF0IHRoZXkgYWxsIGhhdmUgYmFzaWNhbGx5IHRoZSBzYW1lCiMjIGRpc3RyaWJ1dGlvbi4gIEkgZGlkIG5vdCBleHBlY3QgdGhlIENGIHNhbXBsZXMgdG8gaGF2ZSBzdWNoIGEgc3Ryb25nZXIKIyMgZGlzdHJpYnV0aW9uIHRob3VnaC4KCnRlc3RfbWV0cmljc19ub3JtJGNvcmhlYXQKIyMgVGhpcyBzdWdnZXN0cyB0byBtZSBhIGxpa2VseSBiYXRjaCBlZmZlY3QKdGVzdF9tZXRyaWNzX25vcm0kZGlzaGVhdAp0ZXN0X21ldHJpY3Mkc21jCnRlc3RfbWV0cmljcyR0c25lcGxvdAojIyBZZWFoIHRoZXJlIGlzIHNvbWUgd29ua3luZXNzIGJldHdlZW4gdGhlIG9sZC9uZXcgc2FtcGxlcy4KCnRlc3RfbWV0cmljc19ub3JtYmF0Y2gkcGNhcGxvdAojIyBXb3csIHRoaXMgaXMgYWZ0ZXIgaW52b2tpbmcgJ3JlbW92ZUJhdGNoRWZmZWN0JyBmcm9tIGxpbW1hLiAgVGhhdCBzdWdnZXN0cwojIyBwcmV0dHkgc3Ryb25nbHkgdG8gbWUgdGhhdCB3ZSBzaG91bGQgcHJvYmFsYnkgbm90IGV4YW1pbmUgdGhlIG5ldyBhbmQgb2xkCiMjIGRhdGEgdG9nZXRoZXIuICBJbiBhZGRpdGlvbiBJIGFtIHRoaW5raW5nIHRoYXQgb25lIHdob2xlLWNlbGwgbHlzYXRlIHNhbXBsZQojIyBtaWdodCBub3QgYmUuCmBgYAoKIyMgUmVtb3ZlIG9sZCBiYXRjaC93ZWlyZG8gc2FtcGxlCgpgYGB7ciByZW1vdmFscywgZmlnLnNob3c9ImhpZGUiLCBldmFsPUZBTFNFfQprZXB0X3Rlc3RpbmcgPC0gc3Vic2V0X2V4cHQodGVzdF9leHB0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgc3Vic2V0PSJiaW9yZXBsaWNhdGUgPT0gJ21hciciKQojIyBUaGUgbmV3IGJhdGNoIElEczogeCx5LHogYXJlIGFzc29jaWF0ZWQgd2l0aCB0aGUgMyBydW5zIG9mIHRoZXNlIHNhbXBsZXMsIG9uZQojIyB3aGljaCBpcyA4IG0veiwgb25lIHdoaWNoIGlzIDIwIG0veiwgYW5kIG9uZSB3aGljaCBpcyBhIHJlb3JkZXJlZCAyMCBtL3ouCgp0ZXN0X25vcm0gPC0gbm9ybWFsaXplX2V4cHQoa2VwdF90ZXN0aW5nLCB0cmFuc2Zvcm09ImxvZzIiLCBjb252ZXJ0PSJjcG0iLCBub3JtPSJxdWFudGlsZSIsIGZpbHRlcj1UUlVFKQp0ZXN0X25vcm1iYXRjaCA8LSBzbShub3JtYWxpemVfZXhwdChrZXB0X3Rlc3RpbmcsIHRyYW5zZm9ybT0ibG9nMiIsIGNvbnZlcnQ9ImNwbSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZpbHRlcj1UUlVFLCBiYXRjaD0ibGltbWEiKSkKbmV3X21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyhrZXB0X3Rlc3RpbmcpKQpuZXdfbWV0cmljc19ub3JtIDwtIHNtKGdyYXBoX21ldHJpY3ModGVzdF9ub3JtKSkKbmV3X21ldHJpY3Nfbm9ybWJhdGNoIDwtIHNtKGdyYXBoX21ldHJpY3ModGVzdF9ub3JtYmF0Y2gpKQpgYGAKCiMjIyBQbG90IG9ubHkgbmV3IGRhdGEKClJlbW92aW5nIHRoZSBKYW51YXJ5IGRhdGEgc2VlbXMgdG8gaGF2ZSBtYWRlIHRoaXMgYSBsb3QgZWFzaWVyIHRvIGxvb2sgYXQuClRoZXJlIG1pZ2h0IGJlIHNvbWUgYmF0Y2h5bmVzcyBhc3NvY2lhdGVkIHdpdGggcmVvcmRlcmluZyB0aGUgc2FtcGxlcywgYnV0IEkgYW0KdGhpbmtpbmcgaXQgaXMgbm90IGh1Z2UuCgoKYGBge3IgbmV3X2RhdGFfcGxvdHMsIGV2YWw9RkFMU0V9CnR0ID0gcGxvdF90b3BuKGtlcHRfdGVzdGluZywgZGlyZWN0PVRSVUUpCm5ld19tZXRyaWNzJGxlZ2VuZApuZXdfbWV0cmljcyRsaWJzaXplCnBwKCJpbWFnZXMva2VwdF9ub3JtX3RvcG4ucG5nIiwgaW1hZ2U9bmV3X21ldHJpY3MkdG9wbnBsb3QpCnBwKCJpbWFnZXMva2VwdF9ub3JtX2hjZF9wY2EucG5nIiwgaW1hZ2U9bmV3X21ldHJpY3Nfbm9ybSRwY2FwbG90KQpuZXdfbWV0cmljc19ub3JtJHRzbmVwbG90CnBwKCJpbWFnZXMva2VwdF9ub3JtX2hjZF9jb3JoZWF0LnBuZyIsIGltYWdlPW5ld19tZXRyaWNzX25vcm0kY29yaGVhdCkKbmV3X21ldHJpY3Nfbm9ybSRkaXNoZWF0Cm5ld19tZXRyaWNzX25vcm1iYXRjaCRwY2FwbG90CnBwKCJpbWFnZXMvbm9ybWJhdGNoX3RzbmUucG5nIiwgaW1hZ2U9bmV3X21ldHJpY3Nfbm9ybWJhdGNoJHRzbmVwbG90KQpgYGAKCiMjIyBIb3cgaXMgdGhlIHZhcmlhbmNlPwoKTGV0cyBzZWUgd2hhdCB2YXJpYW5jZVBhcnRpdGlvbiBoYXMgdG8gc2F5IGFib3V0IHRoaXMgZGF0YS4KCmBgYHtyIHZhcnBhcnQsIGV2YWw9RkFMU0V9Cm5ld192YXJwYXJ0IDwtIHZhcnBhcnQoa2VwdF90ZXN0aW5nLCBmYWN0b3JzPWMoImNvbmRpdGlvbiIsICJiYXRjaCIpKQpwcCgiaW1hZ2VzL3ZhcnBhcnRfcGFydGl0aW9uLnBuZyIsIGltYWdlPW5ld192YXJwYXJ0JHBhcnRpdGlvbl9wbG90KQojIyBUaGF0IGlzIG5vdCB0ZXJyaWJsZS4KCiMjIHRoZSBtb2RpZmllZF9leHB0IHNsb3cgZnJvbSB2YXJwYXJ0KCkgYWRkcyBzb21lIG1ldGFkYXRhIHRvIHRoZSBleHByZXNzaW9uc2V0CiMjIGluY2x1ZGluZyB0aGUgY2FsY3VsYXRlZCAlIHZhcmlhbmNlIGZvciBjb25kaXRpb24vYmF0Y2gvcmVzaWR1YWwgZm9yIGVhY2ggcGVwdGlkZS4Ka2VwdF90ZXN0aW5nIDwtIG5ld192YXJwYXJ0JG1vZGlmaWVkX2V4cHQKYGBgCgpgYGB7ciBwZV9wcm90ZWluc30KcGVfZ2VuZXMgPC0gcmVhZC50YWJsZSgicmVmZXJlbmNlL2Fubm90YXRlZF9wZV9nZW5lcy50eHQiKVtbMV1dCnN1bShwZV9nZW5lcyAlaW4lIHJvd25hbWVzKGV4cHJzKHByb3RlaW5fbm9ybSkpKQpmb3VuZF9wZV9pZHggPC0gcm93bmFtZXMoZXhwcnMocHJvdGVpbl9leHB0KSkgJWluJSBwZV9nZW5lcwpmb3VuZF9wZSA8LSByb3duYW1lcyhleHBycyhwcm90ZWluX2V4cHQpKVtmb3VuZF9wZV9pZHhdCnBlX2V4cHQgPC0gZXhjbHVkZV9nZW5lc19leHB0KHByb3RlaW5fZXhwdCwgbWV0aG9kPSJrZWVwIiwgaWRzPWZvdW5kX3BlKQpwbG90X2JveHBsb3QocGVfZXhwdCkKZXhwcnMocGVfZXhwdCkKYGBgCgoKIyMgQmFjayB0byBocGdsdG9vbHMKCklmIHlvdSB3aWxsIHJlY2FsbCwgaW4gdGhlIG9yaWdpbmFsIGJsb2NrIHdoZXJlIEkgcmVhZCB0aGUgZGF0YSBpbnRvClNXQVRIMnN0YXRzLCBJIGludm9rZWQgd3JpdGVfbWF0cml4X3Byb3RlaW5zKCkgYW5kIHdyaXRlX21hdHJpeF9wZXB0aWRlcygpIGFuZAp3cm90ZSBvdXQgMiBjc3YgZmlsZXMgJ3N3YXRoMnN0YXRzX3Byb3RlaW5fbWF0cml4LmNzdicgYW5kCidfcGVwdGlkZV9tYXRyaXguY3N2Jy4gIElmIG15IGVhcmxpZXIgd29yayBpcyBjb3JyZWN0LCBJIHNob3VsZCBiZSBhYmxlIHRvIHVzZQp0aGUgcHJvdGVpbiBtYXRyaXggdG8gZ2V0IGEgdHJ1ZXIgc2Vuc2Ugb2YgdGhlIHJlbGF0aXZlIGFidW5kYW5jZXMgYmV0d2VlbiBteQpleHBlcmltZW50YWwgY29uZGl0aW9ucy4KCmBgYHtyIGFsZnFfcXVhbnQsIGV2YWw9RkFMU0V9CmxpYnJhcnkoYUxGUSkKYWxmcV9wcm9jZXNzIDwtIFByb3RlaW5JbmZlcmVuY2UoYWxmcV9pbnB1dCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcGVwdGlkZV9tZXRob2Q9InRvcCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBlcHRpZGVfdG9weD0zLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwZXB0aWRlX3N0cmljdG5lc3M9Imxvb3NlIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcGVwdGlkZV9zdW1tYXJ5PSJtZWFuIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhbnNpdGlvbl90b3B4PTMsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYW5zaXRpb25fc3RyaWN0bmVzcz0ibG9vc2UiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFuc2l0aW9uX3N1bW1hcnk9InN1bSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZhc3RhPU5BLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtb2RlbD1OQSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29tYmluZV9wcmVjdXJzb3JzPUZBTFNFKQpgYGAKCiMgSW5kZXggdmVyc2lvbjogYHIgdmVyYAoKIyBUT0RPCgoqIDIwMTgtMDQtMTA6ICBNYWtlIHN1cmUgbXkgaW52b2NhdGlvbnMgb2YgU1dBVEgyc3RhdHMvTVNzdGF0cyBhcmUgY29ycmVjdC4KCmBgYHtyIHNhdmVtZSwgZXZhbD1GQUxTRX0KaWYgKCFpc1RSVUUoZ2V0MCgic2tpcF9sb2FkIikpKSB7CiAgbWVzc2FnZShwYXN0ZTAoIlRoaXMgaXMgaHBnbHRvb2xzIGNvbW1pdDogIiwgZ2V0X2dpdF9jb21taXQoKSkpCiAgdGhpc19zYXZlIDwtIHBhc3RlMChnc3ViKHBhdHRlcm49IlxcLlJtZCIsIHJlcGxhY2U9IiIsIHg9cm1kX2ZpbGUpLCAiLXYiLCB2ZXIsICIucmRhLnh6IikKICBtZXNzYWdlKHBhc3RlMCgiU2F2aW5nIHRvICIsIHRoaXNfc2F2ZSkpCiAgdG1wIDwtIHNtKHNhdmVtZShmaWxlbmFtZT10aGlzX3NhdmUpKQogIHBhbmRlcjo6cGFuZGVyKHNlc3Npb25JbmZvKCkpCn0KYGBgCg==

M.tuberculosis 201805: Analyzing data from OpenSwathWorkFlow/TRIC.

atb abelew@gmail.com

2018-05-30