1 S. cerevisiae sample estimation of all samples (old and new)

This document should make clear the suitability of our yeast data for differential expression analyses. It should also give some ideas about the depth and distribution of the data.

2 Gathering samples

Later, we will likely choose to exclude one of the three experimental batches in the data. Therefore, I am making three expressionsets, one with all data, and one the various subsets.

2.1 Only look at wt/cbf5, old and new

In response to some ideas around 20180606, we will extract all the samples which are wt/cbf5 and not excessively batch-ridden (eg. some of our wt samples)

  • all_expt: All samples bundled into one experiment.
  • E1_expt: Only the samples from the first experiment.
  • E2_expt: Only the smaples from the second experiment.
  • E1E2B1_expt: The first experiment and the first batch of the second experiment.
  • E1E2B2_expt: The first experiment and the second batch of the second experiment.
  • E1E2B2_cbf5wt: Ibid, but only the samples which compare the D95A and wt samples for the CBF5 gene.
all_expt <- sm(create_expt(metadata="sample_sheets/all_samples.xlsx",
                              gene_info=sc_all_annotations,
                              file_column="allfile"))
## Don't forget, I need to change the condition names.
E1_expt <- subset_expt(all_expt, subset="batch=='E1'")
## There were 24, now there are 8 samples.
E2_expt <- subset_expt(all_expt, subset="batch!='E1'")
## There were 24, now there are 16 samples.
E1E2B2_expt <- subset_expt(all_expt, subset="batch!='E2B1'")
## There were 24, now there are 16 samples.
E1E2B1_expt <- subset_expt(all_expt, subset="batch!='E2B2'")
## There were 24, now there are 16 samples.
E1E2B2_cbf5wt <- subset_expt(E1E2B2_expt, subset="condition=='WT'|condition=='cbf5_D95A'")
## There were 16, now there are 12 samples.

3 Visualizing raw data

There are lots of methods we have to examine raw data and explore stuff like batch effects or non-canonical distributions or skewed counts. hpgltools provides some functionality to make this process easier. The graphs shown below and many more are generated with the wrapper ‘graph_metrics()’, when invoked it provides a list of plots including: library sizes, the number of non-zero genes, density/box plots by sample, distance/correlation heatmaps, standard median correlation/distance, PCA/TSNE clustering, top-n genes, a legend describing the colors/symbols used, and some tables describing the data. Optionally, it can also perform quantile-quantile plots showing the distribution of each sample vs. the median of all samples and MA plots of the same.

Caveat: some plots do not work well with gene IDs that are all-0, thus I first filter the data to remove them.

I also added a neat function ‘plot_libsize_prepost()’ which shows how many genes are poorly represented before/after filtering the data.

The plots printed here are all metrics which are useful when considering raw data.

all_filt <- sm(normalize_expt(all_expt, filter=TRUE))
all_metrics <- sm(graph_metrics(all_expt))

Now print some plots of interest of the raw data.

pp(file="illustrator_input/01_legend.pdf", image=all_metrics$legend)
## Writing the image to: illustrator_input/01_legend.pdf and calling dev.off().

pp(file="illustrator_input/02_raw_libsize.pdf", image=all_metrics$libsize)
## Writing the image to: illustrator_input/02_raw_libsize.pdf and calling dev.off().

prepost <- plot_libsize_prepost(all_expt)
pp(file="illustrator_input/03_libsize_changed_lowgenes.pdf", image=prepost$lowgene_plot)
## Writing the image to: illustrator_input/03_libsize_changed_lowgenes.pdf and calling dev.off().

pp(file="illustrator_input/04_nonzero_genes.pdf", image=all_metrics$nonzero)
## Writing the image to: illustrator_input/04_nonzero_genes.pdf and calling dev.off().

pp(file="illustrator_input/05_raw_boxplot.pdf", image=all_metrics$boxplot)
## Writing the image to: illustrator_input/05_raw_boxplot.pdf and calling dev.off().

pp(file="illustrator_input/06_raw_density.pdf", image=all_metrics$density)
## Writing the image to: illustrator_input/06_raw_density.pdf and calling dev.off().

pp(file="illustrator_input/07_raw_boxplot.pdf", image=all_metrics$boxplot)
## Writing the image to: illustrator_input/07_raw_boxplot.pdf and calling dev.off().

pp(file="illustrator_input/08_topn.pdf", image=all_metrics$topnplot)
## Writing the image to: illustrator_input/08_topn.pdf and calling dev.off().

4 Normalize and visualize

Other metrics are more useful when used with data on the log scale and normalized by number of reads/library and/or by quantile.

all_exptm <- normalize_expt(all_expt, transform="log2", convert="cpm", filter=TRUE)
## This function will replace the expt$expressionset slot with:
## log2(cpm(hpgl(data)))
## It backs up the current data into a slot named:
##  expt$backup_expressionset. It will also save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep the libsizes in mind
##  when invoking limma.  The appropriate libsize is the non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize
## Leaving the data unnormalized.  This is necessary for DESeq, but
##  EdgeR/limma might benefit from normalization.  Good choices include quantile,
##  size-factor, tmm, etc.
## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.
## Step 1: performing count filter with option: hpgl
## Removing 1030 low-count genes (6095 remaining).
## Step 2: not normalizing the data.
## Step 3: converting the data with cpm.
## Step 4: transforming the data with log2.
## transform_counts: Found 1152 values equal to 0, adding 1 to the matrix.
## Step 5: not doing batch correction.
all_metrics <- graph_metrics(all_exptm)
## Graphing number of non-zero genes with respect to CPM by library.
## Graphing library sizes.
## Graphing a boxplot.
## Graphing a correlation heatmap.
## Graphing a standard median correlation.
## Performing correlation.
## Graphing a distance heatmap.
## Graphing a standard median distance.
## Performing distance.
## Graphing a PCA plot.
## Graphing a T-SNE plot.
## Plotting a density plot.
## Plotting the representation of the top-n genes.
## Printing a color to condition legend.

The data should now be normalized, lets view some metrics post-facto.

pp(file="illustrator_input/09_norm_corheat.pdf", image=all_metrics$corheat)
## Writing the image to: illustrator_input/09_norm_corheat.pdf and calling dev.off().

pp(file="illustrator_input/10_norm_disheat.pdf", image=all_metrics$disheat)
## Writing the image to: illustrator_input/10_norm_disheat.pdf and calling dev.off().

## It appears that just the normalization is sufficient to split the samples completely by type and deeply separate them from the heterologous samples

pp(file="illustrator_input/11_norm_smc.pdf", image=all_metrics$smc)
## Writing the image to: illustrator_input/11_norm_smc.pdf and calling dev.off().

pp(file="illustrator_input/12_norm_smd.pdf", image=all_metrics$smd)
## Writing the image to: illustrator_input/12_norm_smd.pdf and calling dev.off().

## The samples are very well behaved, none fall below the red line.

pp(file="illustrator_input/13_norm_pca.pdf", image=all_metrics$pcaplot)
## Writing the image to: illustrator_input/13_norm_pca.pdf and calling dev.off().

pp(file="illustrator_input/14_norm_tsne.pdf", image=all_metrics$tsneplot)
## Writing the image to: illustrator_input/14_norm_tsne.pdf and calling dev.off().

## The homogeneous wt/mutant are nicely separated, and what is more, the exogeneous samples also split wt/mutant, that might prove to be quite useful.

4.1 wt/cbf5 normalized metrics

Above, we split out the wt/cbf5 samples, now lets normalize and make a few plots of potential interest. Let us use the same batch method (fsva).

We will do this for both only the old samples and also for the old+new samples.

E1_norm <- normalize_expt(E1_expt, filter=TRUE, norm="quant", convert="cpm", transform="log")
## This function will replace the expt$expressionset slot with:
## log(cpm(quant(hpgl(data))))
## It backs up the current data into a slot named:
##  expt$backup_expressionset. It will also save copies of each step along the way
##  in expt$normalized with the corresponding libsizes. Keep the libsizes in mind
##  when invoking limma.  The appropriate libsize is the non-log(cpm(normalized)).
##  This is most likely kept at:
##  'new_expt$normalized$intermediate_counts$normalization$libsizes'
##  A copy of this may also be found at:
##  new_expt$best_libsize
## Not correcting the count-data for batch effects.  If batch is
##  included in EdgerR/limma's model, then this is probably wise; but in extreme
##  batch effects this is a good parameter to play with.
## Step 1: performing count filter with option: hpgl
## Removing 1280 low-count genes (5845 remaining).
## Step 2: normalizing the data with quant.
## Step 3: converting the data with cpm.
## Step 4: transforming the data with log.
## Step 5: not doing batch correction.
cbf5_norm <- sm(normalize_expt(E1E2B2_cbf5wt, filter=TRUE, norm="quant", convert="cpm",
                               transform="log"))
cbf5_batch <- sm(normalize_expt(E1E2B2_cbf5wt, filter=TRUE, norm="quant", convert="cpm",
                                transform="log", batch="fsva"))

E1_raw_metrics <- sm(graph_metrics(E1_expt))
E1_norm_metrics <- sm(graph_metrics(E1_norm))
cbf5_raw_metrics <- sm(graph_metrics(E1E2B2_cbf5wt))
cbf5_norm_metrics <- sm(graph_metrics(cbf5_norm))
cbf5_batch_metrics <- sm(graph_metrics(cbf5_batch))

Now print out some of the interesting plots.

pp(file="illustrator_input/32_E1_norm_corheat.pdf", image=E1_norm_metrics$corheat)
## Writing the image to: illustrator_input/32_E1_norm_corheat.pdf and calling dev.off().

pp(file="illustrator_input/33_E1_norm_pca.pdf", image=E1_norm_metrics$pcaplot)
## Writing the image to: illustrator_input/33_E1_norm_pca.pdf and calling dev.off().

pp(file="illustrator_input/34_E1_norm_disheat.pdf", image=E1_norm_metrics$disheat)
## Writing the image to: illustrator_input/34_E1_norm_disheat.pdf and calling dev.off().

5 Now look at the data without batch ‘E2B1’

E1E2B2_norm <- sm(normalize_expt(E1E2B2_expt, transform="log2", convert="cpm", norm="quant", filter=TRUE))
E1E2B2_plots <- sm(graph_metrics(E1E2B2_norm))
E1E2B2_batchnorm <- sm(normalize_expt(E1E2B2_expt, transform="log2", convert="cpm", norm="quant", filter=TRUE,
                                      batch="fsva"))
E1E2B2_batchplots <- sm(graph_metrics(E1E2B2_batchnorm))

6 Write the expt

## hmm I was going to silence this line, but looking at the report it seems to be missing some pictures.
all_written <- write_expt(all_expt, excel=paste0("excel/samples_written_all_expt-v", ver, ".xlsx"),
                          filter=TRUE, norm="raw", convert="cpm", batch="fsva", transform="log2")
## Writing the legend.
## Writing the raw reads.
## Graphing the raw reads.
## Writing the normalized reads.
## Graphing the normalized reads.
## Writing the median reads by factor.
## The factor WT has 8 rows.
## The factor cbf5_D95A has 12 rows.
## The factor upf1d has 8 rows.
## The factor cbf5_D95A upf1d has 4 rows.
E1_written <- write_expt(all_expt, excel=paste0("excel/samples_written_E1_expt-v", ver, ".xlsx"),
                         filter=TRUE, norm="raw", convert="cpm", transform="log2")
## Writing the legend.
## Writing the raw reads.
## Graphing the raw reads.
## Writing the normalized reads.
## Graphing the normalized reads.
## Writing the median reads by factor.
## The factor WT has 8 rows.
## The factor cbf5_D95A has 12 rows.
## The factor upf1d has 8 rows.
## The factor cbf5_D95A upf1d has 4 rows.
E1E2B2_written <- write_expt(E1E2B2_expt, excel=paste0("excel/samples_written_E1E2B2_expt-v", ver, ".xlsx"),
                             filter=TRUE, norm="raw", convert="cpm", batch="fsva", transform="log2")
## Writing the legend.
## Writing the raw reads.
## Graphing the raw reads.
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Writing the normalized reads.
## Graphing the normalized reads.
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Too few points to calculate an ellipse
## Writing the median reads by factor.
## The factor WT has 6 rows.
## The factor cbf5_D95A has 8 rows.
## The factor upf1d has 4 rows.
## The factor cbf5_D95A upf1d has 2 rows.
LS0tCnRpdGxlOiAiUy4gY2VyZXZpc2lhZSBzYW1wbGUgZXN0aW1hdGlvbiwgbWVyZ2VkIHNhbXBsZSBlZGl0aW9uICgyMDE4MDYwNikuIgphdXRob3I6ICJhdGIgYWJlbGV3QGdtYWlsLmNvbSIKZGF0ZTogImByIFN5cy5EYXRlKClgIgpvdXRwdXQ6CiBodG1sX2RvY3VtZW50OgogIGNvZGVfZG93bmxvYWQ6IHRydWUKICBjb2RlX2ZvbGRpbmc6IHNob3cKICBmaWdfY2FwdGlvbjogdHJ1ZQogIGZpZ19oZWlnaHQ6IDcKICBmaWdfd2lkdGg6IDcKICBoaWdobGlnaHQ6IGRlZmF1bHQKICBrZWVwX21kOiBmYWxzZQogIG1vZGU6IHNlbGZjb250YWluZWQKICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICBzZWxmX2NvbnRhaW5lZDogdHJ1ZQogIHRoZW1lOiByZWFkYWJsZQogIHRvYzogdHJ1ZQogIHRvY19mbG9hdDoKICAgIGNvbGxhcHNlZDogZmFsc2UKICAgIHNtb290aF9zY3JvbGw6IGZhbHNlCi0tLQoKPHN0eWxlPgogIGJvZHkgLm1haW4tY29udGFpbmVyIHsKICAgIG1heC13aWR0aDogMTYwMHB4OwogIH0KPC9zdHlsZT4KCmBgYHtyIG9wdGlvbnMsIGluY2x1ZGU9RkFMU0V9CmlmICghaXNUUlVFKGdldDAoInNraXBfbG9hZCIpKSkgewogIGxpYnJhcnkoImhwZ2x0b29scyIpCiAgdHQgPC0gZGV2dG9vbHM6OmxvYWRfYWxsKCJ+L2hwZ2x0b29scyIpCiAga25pdHI6Om9wdHNfa25pdCRzZXQocHJvZ3Jlc3M9VFJVRSwKICAgICAgICAgICAgICAgICAgICAgICB2ZXJib3NlPVRSVUUsCiAgICAgICAgICAgICAgICAgICAgICAgd2lkdGg9OTAsCiAgICAgICAgICAgICAgICAgICAgICAgZWNobz1UUlVFKQogIGtuaXRyOjpvcHRzX2NodW5rJHNldChlcnJvcj1UUlVFLAogICAgICAgICAgICAgICAgICAgICAgICBmaWcud2lkdGg9OCwKICAgICAgICAgICAgICAgICAgICAgICAgZmlnLmhlaWdodD04LAogICAgICAgICAgICAgICAgICAgICAgICBkcGk9OTYpCiAgb2xkX29wdGlvbnMgPC0gb3B0aW9ucyhkaWdpdHM9NCwKICAgICAgICAgICAgICAgICAgICAgICAgIHN0cmluZ3NBc0ZhY3RvcnM9RkFMU0UsCiAgICAgICAgICAgICAgICAgICAgICAgICBrbml0ci5kdXBsaWNhdGUubGFiZWw9ImFsbG93IikKICB2ZXIgPC0gIjIwMTgwNjA2IgogIHByZXZpb3VzX2ZpbGUgPC0gcGFzdGUwKCIwMV9hbm5vdGF0aW9uLXYiLCB2ZXIsICIuUm1kIikKCiAgdG1wIDwtIHRyeShzbShsb2FkbWUoZmlsZW5hbWU9cGFzdGUwKGdzdWIocGF0dGVybj0iXFwuUm1kIiwgcmVwbGFjZT0iIiwgeD1wcmV2aW91c19maWxlKSwgIi12IiwgdmVyLCAiLnJkYS54eiIpKSkpCiAgcm1kX2ZpbGUgPC0gcGFzdGUwKCIwMl9zYW1wbGVfZXN0aW1hdGlvbl9tZXJnZWQtdiIsIHZlciwgIi5SbWQiKQp9CmBgYAoKUy4gY2VyZXZpc2lhZSBzYW1wbGUgZXN0aW1hdGlvbiBvZiBhbGwgc2FtcGxlcyAob2xkIGFuZCBuZXcpCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQoKVGhpcyBkb2N1bWVudCBzaG91bGQgbWFrZSBjbGVhciB0aGUgc3VpdGFiaWxpdHkgb2Ygb3VyIHllYXN0IGRhdGEgZm9yIGRpZmZlcmVudGlhbCBleHByZXNzaW9uCmFuYWx5c2VzLiAgSXQgc2hvdWxkIGFsc28gZ2l2ZSBzb21lIGlkZWFzIGFib3V0IHRoZSBkZXB0aCBhbmQgZGlzdHJpYnV0aW9uIG9mCnRoZSBkYXRhLgoKIyBHYXRoZXJpbmcgc2FtcGxlcwoKTGF0ZXIsIHdlIHdpbGwgbGlrZWx5IGNob29zZSB0byBleGNsdWRlIG9uZSBvZiB0aGUgdGhyZWUgZXhwZXJpbWVudGFsIGJhdGNoZXMgaW4KdGhlIGRhdGEuICBUaGVyZWZvcmUsIEkgYW0gbWFraW5nIHRocmVlIGV4cHJlc3Npb25zZXRzLCBvbmUgd2l0aCBhbGwgZGF0YSwgYW5kCm9uZSB0aGUgdmFyaW91cyBzdWJzZXRzLgoKIyMgT25seSBsb29rIGF0IHd0L2NiZjUsIG9sZCBhbmQgbmV3CgpJbiByZXNwb25zZSB0byBzb21lIGlkZWFzIGFyb3VuZCAyMDE4MDYwNiwgd2Ugd2lsbCBleHRyYWN0IGFsbCB0aGUgc2FtcGxlcyB3aGljaAphcmUgd3QvY2JmNSBhbmQgbm90IGV4Y2Vzc2l2ZWx5IGJhdGNoLXJpZGRlbiAoZWcuIHNvbWUgb2Ygb3VyIHd0IHNhbXBsZXMpCgoqIGFsbF9leHB0OiBBbGwgc2FtcGxlcyBidW5kbGVkIGludG8gb25lIGV4cGVyaW1lbnQuCiogRTFfZXhwdDogT25seSB0aGUgc2FtcGxlcyBmcm9tIHRoZSBmaXJzdCBleHBlcmltZW50LgoqIEUyX2V4cHQ6IE9ubHkgdGhlIHNtYXBsZXMgZnJvbSB0aGUgc2Vjb25kIGV4cGVyaW1lbnQuCiogRTFFMkIxX2V4cHQ6IFRoZSBmaXJzdCBleHBlcmltZW50IGFuZCB0aGUgZmlyc3QgYmF0Y2ggb2YgdGhlIHNlY29uZCBleHBlcmltZW50LgoqIEUxRTJCMl9leHB0OiBUaGUgZmlyc3QgZXhwZXJpbWVudCBhbmQgdGhlIHNlY29uZCBiYXRjaCBvZiB0aGUgc2Vjb25kIGV4cGVyaW1lbnQuCiogRTFFMkIyX2NiZjV3dDogSWJpZCwgYnV0IG9ubHkgdGhlIHNhbXBsZXMgd2hpY2ggY29tcGFyZSB0aGUgRDk1QSBhbmQgd3Qgc2FtcGxlcyBmb3IgdGhlIENCRjUgZ2VuZS4KCmBgYHtyIGdhdGhlcl9hbGx9CmFsbF9leHB0IDwtIHNtKGNyZWF0ZV9leHB0KG1ldGFkYXRhPSJzYW1wbGVfc2hlZXRzL2FsbF9zYW1wbGVzLnhsc3giLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBnZW5lX2luZm89c2NfYWxsX2Fubm90YXRpb25zLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmaWxlX2NvbHVtbj0iYWxsZmlsZSIpKQojIyBEb24ndCBmb3JnZXQsIEkgbmVlZCB0byBjaGFuZ2UgdGhlIGNvbmRpdGlvbiBuYW1lcy4KRTFfZXhwdCA8LSBzdWJzZXRfZXhwdChhbGxfZXhwdCwgc3Vic2V0PSJiYXRjaD09J0UxJyIpCkUyX2V4cHQgPC0gc3Vic2V0X2V4cHQoYWxsX2V4cHQsIHN1YnNldD0iYmF0Y2ghPSdFMSciKQpFMUUyQjJfZXhwdCA8LSBzdWJzZXRfZXhwdChhbGxfZXhwdCwgc3Vic2V0PSJiYXRjaCE9J0UyQjEnIikKRTFFMkIxX2V4cHQgPC0gc3Vic2V0X2V4cHQoYWxsX2V4cHQsIHN1YnNldD0iYmF0Y2ghPSdFMkIyJyIpCgpFMUUyQjJfY2JmNXd0IDwtIHN1YnNldF9leHB0KEUxRTJCMl9leHB0LCBzdWJzZXQ9ImNvbmRpdGlvbj09J1dUJ3xjb25kaXRpb249PSdjYmY1X0Q5NUEnIikKYGBgCgojIFZpc3VhbGl6aW5nIHJhdyBkYXRhCgpUaGVyZSBhcmUgbG90cyBvZiBtZXRob2RzIHdlIGhhdmUgdG8gZXhhbWluZSByYXcgZGF0YSBhbmQgZXhwbG9yZSBzdHVmZiBsaWtlCmJhdGNoIGVmZmVjdHMgb3Igbm9uLWNhbm9uaWNhbCBkaXN0cmlidXRpb25zIG9yIHNrZXdlZCBjb3VudHMuICBocGdsdG9vbHMKcHJvdmlkZXMgc29tZSBmdW5jdGlvbmFsaXR5IHRvIG1ha2UgdGhpcyBwcm9jZXNzIGVhc2llci4gIFRoZSBncmFwaHMgc2hvd24gYmVsb3cKYW5kIG1hbnkgbW9yZSBhcmUgZ2VuZXJhdGVkIHdpdGggdGhlIHdyYXBwZXIgJ2dyYXBoX21ldHJpY3MoKScsIHdoZW4gaW52b2tlZCBpdApwcm92aWRlcyBhIGxpc3Qgb2YgcGxvdHMgaW5jbHVkaW5nOiBsaWJyYXJ5IHNpemVzLCB0aGUgbnVtYmVyIG9mIG5vbi16ZXJvIGdlbmVzLApkZW5zaXR5L2JveCBwbG90cyBieSBzYW1wbGUsIGRpc3RhbmNlL2NvcnJlbGF0aW9uIGhlYXRtYXBzLCBzdGFuZGFyZCBtZWRpYW4KY29ycmVsYXRpb24vZGlzdGFuY2UsIFBDQS9UU05FIGNsdXN0ZXJpbmcsIHRvcC1uIGdlbmVzLCBhIGxlZ2VuZCBkZXNjcmliaW5nIHRoZQpjb2xvcnMvc3ltYm9scyB1c2VkLCBhbmQgc29tZSB0YWJsZXMgZGVzY3JpYmluZyB0aGUgZGF0YS4gIE9wdGlvbmFsbHksIGl0IGNhbgphbHNvIHBlcmZvcm0gcXVhbnRpbGUtcXVhbnRpbGUgcGxvdHMgc2hvd2luZyB0aGUgZGlzdHJpYnV0aW9uIG9mIGVhY2ggc2FtcGxlCnZzLiB0aGUgbWVkaWFuIG9mIGFsbCBzYW1wbGVzIGFuZCBNQSBwbG90cyBvZiB0aGUgc2FtZS4KCkNhdmVhdDogc29tZSBwbG90cyBkbyBub3Qgd29yayB3ZWxsIHdpdGggZ2VuZSBJRHMgdGhhdCBhcmUgYWxsLTAsIHRodXMgSSBmaXJzdApmaWx0ZXIgdGhlIGRhdGEgdG8gcmVtb3ZlIHRoZW0uCgpJIGFsc28gYWRkZWQgYSBuZWF0IGZ1bmN0aW9uICdwbG90X2xpYnNpemVfcHJlcG9zdCgpJyB3aGljaCBzaG93cyBob3cgbWFueSBnZW5lcwphcmUgcG9vcmx5IHJlcHJlc2VudGVkIGJlZm9yZS9hZnRlciBmaWx0ZXJpbmcgdGhlIGRhdGEuCgpUaGUgcGxvdHMgcHJpbnRlZCBoZXJlIGFyZSBhbGwgbWV0cmljcyB3aGljaCBhcmUgdXNlZnVsIHdoZW4gY29uc2lkZXJpbmcgcmF3CmRhdGEuCgpgYGB7ciByYXdfZXhwbG9yZSwgZmlnLnNob3c9ImhpZGUifQphbGxfZmlsdCA8LSBzbShub3JtYWxpemVfZXhwdChhbGxfZXhwdCwgZmlsdGVyPVRSVUUpKQphbGxfbWV0cmljcyA8LSBzbShncmFwaF9tZXRyaWNzKGFsbF9leHB0KSkKYGBgCgpOb3cgcHJpbnQgc29tZSBwbG90cyBvZiBpbnRlcmVzdCBvZiB0aGUgcmF3IGRhdGEuCgpgYGB7ciByYXdfc2hvd30KcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMDFfbGVnZW5kLnBkZiIsIGltYWdlPWFsbF9tZXRyaWNzJGxlZ2VuZCkKcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMDJfcmF3X2xpYnNpemUucGRmIiwgaW1hZ2U9YWxsX21ldHJpY3MkbGlic2l6ZSkKCnByZXBvc3QgPC0gcGxvdF9saWJzaXplX3ByZXBvc3QoYWxsX2V4cHQpCnBwKGZpbGU9ImlsbHVzdHJhdG9yX2lucHV0LzAzX2xpYnNpemVfY2hhbmdlZF9sb3dnZW5lcy5wZGYiLCBpbWFnZT1wcmVwb3N0JGxvd2dlbmVfcGxvdCkKCnBwKGZpbGU9ImlsbHVzdHJhdG9yX2lucHV0LzA0X25vbnplcm9fZ2VuZXMucGRmIiwgaW1hZ2U9YWxsX21ldHJpY3Mkbm9uemVybykKcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMDVfcmF3X2JveHBsb3QucGRmIiwgaW1hZ2U9YWxsX21ldHJpY3MkYm94cGxvdCkKcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMDZfcmF3X2RlbnNpdHkucGRmIiwgaW1hZ2U9YWxsX21ldHJpY3MkZGVuc2l0eSkKcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMDdfcmF3X2JveHBsb3QucGRmIiwgaW1hZ2U9YWxsX21ldHJpY3MkYm94cGxvdCkKcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMDhfdG9wbi5wZGYiLCBpbWFnZT1hbGxfbWV0cmljcyR0b3BucGxvdCkKYGBgCgojIE5vcm1hbGl6ZSBhbmQgdmlzdWFsaXplCgpPdGhlciBtZXRyaWNzIGFyZSBtb3JlIHVzZWZ1bCB3aGVuIHVzZWQgd2l0aCBkYXRhIG9uIHRoZSBsb2cgc2NhbGUgYW5kCm5vcm1hbGl6ZWQgYnkgbnVtYmVyIG9mIHJlYWRzL2xpYnJhcnkgYW5kL29yIGJ5IHF1YW50aWxlLgoKYGBge3Igbm9ybWFsaXplLCBmaWcuc2hvdz0iaGlkZSJ9CmFsbF9leHB0bSA8LSBub3JtYWxpemVfZXhwdChhbGxfZXhwdCwgdHJhbnNmb3JtPSJsb2cyIiwgY29udmVydD0iY3BtIiwgZmlsdGVyPVRSVUUpCmFsbF9tZXRyaWNzIDwtIGdyYXBoX21ldHJpY3MoYWxsX2V4cHRtKQpgYGAKClRoZSBkYXRhIHNob3VsZCBub3cgYmUgbm9ybWFsaXplZCwgbGV0cyB2aWV3IHNvbWUgbWV0cmljcyBwb3N0LWZhY3RvLgoKYGBge3Igbm9ybXZpen0KcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMDlfbm9ybV9jb3JoZWF0LnBkZiIsIGltYWdlPWFsbF9tZXRyaWNzJGNvcmhlYXQpCnBwKGZpbGU9ImlsbHVzdHJhdG9yX2lucHV0LzEwX25vcm1fZGlzaGVhdC5wZGYiLCBpbWFnZT1hbGxfbWV0cmljcyRkaXNoZWF0KQojIyBJdCBhcHBlYXJzIHRoYXQganVzdCB0aGUgbm9ybWFsaXphdGlvbiBpcyBzdWZmaWNpZW50IHRvIHNwbGl0IHRoZSBzYW1wbGVzIGNvbXBsZXRlbHkgYnkgdHlwZSBhbmQgZGVlcGx5IHNlcGFyYXRlIHRoZW0gZnJvbSB0aGUgaGV0ZXJvbG9nb3VzIHNhbXBsZXMKCnBwKGZpbGU9ImlsbHVzdHJhdG9yX2lucHV0LzExX25vcm1fc21jLnBkZiIsIGltYWdlPWFsbF9tZXRyaWNzJHNtYykKcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMTJfbm9ybV9zbWQucGRmIiwgaW1hZ2U9YWxsX21ldHJpY3Mkc21kKQojIyBUaGUgc2FtcGxlcyBhcmUgdmVyeSB3ZWxsIGJlaGF2ZWQsIG5vbmUgZmFsbCBiZWxvdyB0aGUgcmVkIGxpbmUuCgpwcChmaWxlPSJpbGx1c3RyYXRvcl9pbnB1dC8xM19ub3JtX3BjYS5wZGYiLCBpbWFnZT1hbGxfbWV0cmljcyRwY2FwbG90KQpwcChmaWxlPSJpbGx1c3RyYXRvcl9pbnB1dC8xNF9ub3JtX3RzbmUucGRmIiwgaW1hZ2U9YWxsX21ldHJpY3MkdHNuZXBsb3QpCiMjIFRoZSBob21vZ2VuZW91cyB3dC9tdXRhbnQgYXJlIG5pY2VseSBzZXBhcmF0ZWQsIGFuZCB3aGF0IGlzIG1vcmUsIHRoZSBleG9nZW5lb3VzIHNhbXBsZXMgYWxzbyBzcGxpdCB3dC9tdXRhbnQsIHRoYXQgbWlnaHQgcHJvdmUgdG8gYmUgcXVpdGUgdXNlZnVsLgpgYGAKCiMjIHd0L2NiZjUgbm9ybWFsaXplZCBtZXRyaWNzCgpBYm92ZSwgd2Ugc3BsaXQgb3V0IHRoZSB3dC9jYmY1IHNhbXBsZXMsIG5vdyBsZXRzIG5vcm1hbGl6ZSBhbmQgbWFrZSBhIGZldyBwbG90cwpvZiBwb3RlbnRpYWwgaW50ZXJlc3QuICBMZXQgdXMgdXNlIHRoZSBzYW1lIGJhdGNoIG1ldGhvZCAoZnN2YSkuCgpXZSB3aWxsIGRvIHRoaXMgZm9yIGJvdGggX29ubHlfIHRoZSBvbGQgc2FtcGxlcyBhbmQgYWxzbyBmb3IgdGhlIG9sZCtuZXcgc2FtcGxlcy4KCmBgYHtyIGNiZjVfcGxvdHMsIGZpZy5zaG93PSJoaWRlIn0KRTFfbm9ybSA8LSBub3JtYWxpemVfZXhwdChFMV9leHB0LCBmaWx0ZXI9VFJVRSwgbm9ybT0icXVhbnQiLCBjb252ZXJ0PSJjcG0iLCB0cmFuc2Zvcm09ImxvZyIpCgpjYmY1X25vcm0gPC0gc20obm9ybWFsaXplX2V4cHQoRTFFMkIyX2NiZjV3dCwgZmlsdGVyPVRSVUUsIG5vcm09InF1YW50IiwgY29udmVydD0iY3BtIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYW5zZm9ybT0ibG9nIikpCmNiZjVfYmF0Y2ggPC0gc20obm9ybWFsaXplX2V4cHQoRTFFMkIyX2NiZjV3dCwgZmlsdGVyPVRSVUUsIG5vcm09InF1YW50IiwgY29udmVydD0iY3BtIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFuc2Zvcm09ImxvZyIsIGJhdGNoPSJmc3ZhIikpCgpFMV9yYXdfbWV0cmljcyA8LSBzbShncmFwaF9tZXRyaWNzKEUxX2V4cHQpKQpFMV9ub3JtX21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyhFMV9ub3JtKSkKCmNiZjVfcmF3X21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyhFMUUyQjJfY2JmNXd0KSkKY2JmNV9ub3JtX21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyhjYmY1X25vcm0pKQpjYmY1X2JhdGNoX21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyhjYmY1X2JhdGNoKSkKYGBgCgpOb3cgcHJpbnQgb3V0IHNvbWUgb2YgdGhlIGludGVyZXN0aW5nIHBsb3RzLgoKYGBge3IgY2JmNV9zaG93X3Bsb3RzfQpwcChmaWxlPSJpbGx1c3RyYXRvcl9pbnB1dC8zMl9FMV9ub3JtX2NvcmhlYXQucGRmIiwgaW1hZ2U9RTFfbm9ybV9tZXRyaWNzJGNvcmhlYXQpCnBwKGZpbGU9ImlsbHVzdHJhdG9yX2lucHV0LzMzX0UxX25vcm1fcGNhLnBkZiIsIGltYWdlPUUxX25vcm1fbWV0cmljcyRwY2FwbG90KQpwcChmaWxlPSJpbGx1c3RyYXRvcl9pbnB1dC8zNF9FMV9ub3JtX2Rpc2hlYXQucGRmIiwgaW1hZ2U9RTFfbm9ybV9tZXRyaWNzJGRpc2hlYXQpCmBgYAoKIyMgUHJpbnQgc29tZSB2YXJpYW5jZSBwYXJ0aXRpb24gaW5mb3JtYXRpb24KClRoaXMgaXMgYW4gaW50ZXJlc3RpbmcgYXNpZGUgd2hpY2ggY2FtZSB1cCBsYXN0IHdlZWsgZm9yIHNvbWUgb3RoZXIgZGF0YS4gIEl0Cm1pZ2h0IGJlIGdvb2QgdG8gaW5jbHVkZSB0aGUgJSB2YXJpYW5jZSBjb3JyZWxhdGVkIHdpdGggJ2NvbmRpdGlvbicgYXMgYSBjb2x1bW4KaW4gdGhlIGFubm90YXRpb24gZGF0YSBmb3IgdGhlIGV4cHJlc3Npb25zZXQuICBUaHVzLCB3aGVuIHdlIGRvIHRoZSBkaWZmZXJlbnRpYWwKZXhwcmVzc2lvbiBhbmFseXNpcyBsYXRlciwgd2UgY2FuIGxvb2sgYW5kIHNlZSBpZiBhICdzaWduaWZpY2FudCcgZ2VuZSBoYXMKdmFyaWFuY2Ugd2hpY2ggaXMgYWN0dWFsbHkgY29ycmVsYXRlZCB3aXRoIGNvbmRpdGlvbi4KClNpbmNlIHdyaXRpbmcgdGhlc2UgYW5hbHlzZXMsIEkgaW1wbGVtZW50ZWQgdGhpcyBpZGVhIGFuZCBzbyBhbSBpbmNsdWRpbmcgaXQgaGVyZS4KCmBgYHtyIHZhcmlhbmNlX3BhcnRpdGlvbnN9CnZwIDwtIHZhcnBhcnQoYWxsX2V4cHQsIHByZWRpY3Rvcj1OVUxMLCBmYWN0b3JzPWMoImNvbmRpdGlvbiIsICJiYXRjaCIpKQpwcChmaWxlPSJpbGx1c3RyYXRvcl9pbnB1dC8xNV9tZXJnZWRfdmFycGFydF9wYXJ0aXRpb24ucGRmIiwgaW1hZ2U9dnAkcGFydGl0aW9uX3Bsb3QpCiMjIENoZWNrIG91dCB0aGUgbGFzdCB0d28gY29sdW1ucyEKaGVhZChmRGF0YSh2cCRtb2RpZmllZF9leHB0KSkKCm1lcmdlZF9zdmEgPC0gc20obm9ybWFsaXplX2V4cHQoYWxsX2V4cHQsIHRyYW5zZm9ybT0ibG9nMiIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmlsdGVyPVRSVUUsIGJhdGNoPSJzdmEiLCBsb3dfdG9femVybz1UUlVFKSkKdnBzdmEgPC0gdmFycGFydChtZXJnZWRfc3ZhLCBwcmVkaWN0b3I9TlVMTCwgZmFjdG9ycz1jKCJjb25kaXRpb24iLCAiYmF0Y2giKSkKcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMTZfbWVyZ2VkX3ZhcnBhcnRfc3ZhX3BhcnRpdGlvbi5wZGYiLCBpbWFnZT12cHN2YSRwYXJ0aXRpb25fcGxvdCkKaGVhZChmRGF0YSh2cHN2YSRtb2RpZmllZF9leHB0KSkKYGBgCgojIE5vdyBsb29rIGF0IHRoZSBkYXRhIHdpdGhvdXQgYmF0Y2ggJ0UyQjEnCgpgYGB7ciBub3IsIGZpZy5zaG93PSJoaWRlIn0KRTFFMkIyX25vcm0gPC0gc20obm9ybWFsaXplX2V4cHQoRTFFMkIyX2V4cHQsIHRyYW5zZm9ybT0ibG9nMiIsIGNvbnZlcnQ9ImNwbSIsIG5vcm09InF1YW50IiwgZmlsdGVyPVRSVUUpKQpFMUUyQjJfcGxvdHMgPC0gc20oZ3JhcGhfbWV0cmljcyhFMUUyQjJfbm9ybSkpCkUxRTJCMl9iYXRjaG5vcm0gPC0gc20obm9ybWFsaXplX2V4cHQoRTFFMkIyX2V4cHQsIHRyYW5zZm9ybT0ibG9nMiIsIGNvbnZlcnQ9ImNwbSIsIG5vcm09InF1YW50IiwgZmlsdGVyPVRSVUUsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmF0Y2g9ImZzdmEiKSkKRTFFMkIyX2JhdGNocGxvdHMgPC0gc20oZ3JhcGhfbWV0cmljcyhFMUUyQjJfYmF0Y2hub3JtKSkKYGBgCgojIyBQcmludCB0aGUgc21hbGxlciBzZXQgb2YgZGF0YQoKYGBge3IgRTFFMkIyX3Nob3d9CiMjIEJlZm9yZSBmc3ZhLCBjb3JyZWxhdGlvbiBoZWF0bWFwIHdpdGhvdXQgJ3InCnBwKGZpbGU9ImlsbHVzdHJhdG9yX2lucHV0LzE3X0UxRTJCMl9jb3JoZWF0LnBkZiIsIGltYWdlPUUxRTJCMl9wbG90cyRjb3JoZWF0KQojIyBCZWZvcmUgZnN2YSwgcGNhIHdpdGhvdXQgJ3InCnBwKGZpbGU9ImlsbHVzdHJhdG9yX2lucHV0LzE4X0UxRTJCMl9wY2EucGRmIiwgaW1hZ2U9RTFFMkIyX3Bsb3RzJHBjYXBsb3QpCiMjIEFmdGVyIGZzdmEsIGNvcnJlbGF0aW9uIGhlYXRtYXAgd2l0aG91dCAncicKcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMTlfRTFFMkIyX2ZzdmFfY29yaGVhdC5wZGYiLCBpbWFnZT1FMUUyQjJfYmF0Y2hwbG90cyRjb3JoZWF0KQojIyBBZnRlciBmc3ZhLCBwY2Egd2l0aG91dCAncicKcHAoZmlsZT0iaWxsdXN0cmF0b3JfaW5wdXQvMjBfRTFFMkIyX2ZzdmFfcGNhLnBkZiIsIGltYWdlPUUxRTJCMl9iYXRjaHBsb3RzJHBjYXBsb3QpCmBgYAoKIyBXcml0ZSB0aGUgZXhwdAoKYGBge3IgZnN2YV93cml0dGVuLCBmaWcuc2hvdz0naGlkZSd9CiMjIGhtbSBJIHdhcyBnb2luZyB0byBzaWxlbmNlIHRoaXMgbGluZSwgYnV0IGxvb2tpbmcgYXQgdGhlIHJlcG9ydCBpdCBzZWVtcyB0byBiZSBtaXNzaW5nIHNvbWUgcGljdHVyZXMuCmFsbF93cml0dGVuIDwtIHdyaXRlX2V4cHQoYWxsX2V4cHQsIGV4Y2VsPXBhc3RlMCgiZXhjZWwvc2FtcGxlc193cml0dGVuX2FsbF9leHB0LXYiLCB2ZXIsICIueGxzeCIpLAogICAgICAgICAgICAgICAgICAgICAgICAgIGZpbHRlcj1UUlVFLCBub3JtPSJyYXciLCBjb252ZXJ0PSJjcG0iLCBiYXRjaD0iZnN2YSIsIHRyYW5zZm9ybT0ibG9nMiIpCkUxX3dyaXR0ZW4gPC0gd3JpdGVfZXhwdChhbGxfZXhwdCwgZXhjZWw9cGFzdGUwKCJleGNlbC9zYW1wbGVzX3dyaXR0ZW5fRTFfZXhwdC12IiwgdmVyLCAiLnhsc3giKSwKICAgICAgICAgICAgICAgICAgICAgICAgIGZpbHRlcj1UUlVFLCBub3JtPSJyYXciLCBjb252ZXJ0PSJjcG0iLCB0cmFuc2Zvcm09ImxvZzIiKQpFMUUyQjJfd3JpdHRlbiA8LSB3cml0ZV9leHB0KEUxRTJCMl9leHB0LCBleGNlbD1wYXN0ZTAoImV4Y2VsL3NhbXBsZXNfd3JpdHRlbl9FMUUyQjJfZXhwdC12IiwgdmVyLCAiLnhsc3giKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmaWx0ZXI9VFJVRSwgbm9ybT0icmF3IiwgY29udmVydD0iY3BtIiwgYmF0Y2g9ImZzdmEiLCB0cmFuc2Zvcm09ImxvZzIiKQoKYGBgCgpgYGB7ciBzYXZlbWUsIGluY2x1ZGU9RkFMU0V9CmlmICghaXNUUlVFKGdldDAoInNraXBfbG9hZCIpKSkgewogIHBhbmRlcjo6cGFuZGVyKHNlc3Npb25JbmZvKCkpCiAgbWVzc2FnZShwYXN0ZTAoIlRoaXMgaXMgaHBnbHRvb2xzIGNvbW1pdDogIiwgZ2V0X2dpdF9jb21taXQoKSkpCiAgdGhpc19zYXZlIDwtIHBhc3RlMChnc3ViKHBhdHRlcm49IlxcLlJtZCIsIHJlcGxhY2U9IiIsIHg9cm1kX2ZpbGUpLCAiLXYiLCB2ZXIsICIucmRhLnh6IikKICB0bXAgPC0gc20oc2F2ZW1lKGZpbGVuYW1lPXRoaXNfc2F2ZSkpCn0KYGBgCg==