1 Calculating error rates.

I wrote the function ‘create_matrices()’ to collect mutation counts. At least in theory the results from it should be able to address most/any question regarding the counts of mutations observed in the data.

1.1 Categorize the data with at least 3 indexes per mutant

devtools::load_all("Rerrrt")

## Loading Rerrrt

## Loading required package: dplyr

## 
## Attaching package: 'dplyr'

## The following object is masked from 'package:hpgltools':
## 
##     combine

## The following object is masked from 'package:Biobase':
## 
##     combine

## The following objects are masked from 'package:BiocGenerics':
## 
##     combine, intersect, setdiff, union

## The following objects are masked from 'package:stats':
## 
##     filter, lag

## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

## Loading required package: tidyr

sample_sheet <- "sample_sheets/all_samples.xlsx"
ident_column <- "identtable"
mut_column <- "mutationtable"
min_reads <- 3
min_indexes <- 3
min_sequencer <- 10
min_position <- 24
max_position <- 176
max_mutations_per_read <- NULL
prune_n <- TRUE
verbose <- TRUE
excel <- glue::glue("excel/{rundate}_triples-v{ver}.xlsx")
triples <- create_matrices(sample_sheet=sample_sheet,
                           ident_column=ident_column, mut_column=mut_column,
                           min_reads=min_reads, min_indexes=min_indexes,
                           min_sequencer=min_sequencer,
                           min_position=min_position, max_position=max_position,
                           prune_n=prune_n, verbose=verbose, excel=excel)

## Dropped 3 rows from the sample metadata because they were blank.

## Starting sample: s4.

##   Reading the file containing mutations: preprocessing/s4/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s4/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s4/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2239016 reads.

##     Mutation data: after min-position pruning, there are: 2183747 reads: 55269 lost or 2.47%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 2183747 reads.

##     Mutation data: after max-position pruning, there are: 837996 reads: 1345751 lost or 61.63%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 832549 reads: 5447 lost or 0.65%.

##   Mutation data: all filters removed 1406467 reads, or 62.82%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 973940 indexes in all the data.

##     After reads/index pruning, there are: 354890 indexes: 619050 lost or 63.56%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 832549 changed reads.

##     All data: before reads/index pruning, there are: 1975296 identical reads.

##     All data: after index pruning, there are: 454061 changed reads: 54.54%.

##     All data: after index pruning, there are: 1351672 identical reads: 68.43%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 1351672 identical reads.

##   Before classification, there are 454061 reads with mutations.

##   After classification, there are 779988 reads/indexes which are only identical.

##   After classification, there are 3025 reads/indexes which are strictly sequencer.

##   After classification, there are 38821 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 1634797 forward reads and 2107382 reverse_reads.

## Subsetting based on mutations with at least 3 indexes.

## Classified mutation strings according to various criteria.

## Starting sample: s5.

##   Reading the file containing mutations: preprocessing/s5/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s5/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s5/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2259996 reads.

##     Mutation data: after min-position pruning, there are: 2204009 reads: 55987 lost or 2.48%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 2204009 reads.

##     Mutation data: after max-position pruning, there are: 825004 reads: 1379005 lost or 62.57%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 817693 reads: 7311 lost or 0.89%.

##   Mutation data: all filters removed 1442303 reads, or 63.82%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 965529 indexes in all the data.

##     After reads/index pruning, there are: 316034 indexes: 649495 lost or 67.27%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 817693 changed reads.

##     All data: before reads/index pruning, there are: 1764804 identical reads.

##     All data: after index pruning, there are: 420259 changed reads: 51.40%.

##     All data: after index pruning, there are: 1113889 identical reads: 63.12%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 1113889 identical reads.

##   Before classification, there are 420259 reads with mutations.

##   After classification, there are 650056 reads/indexes which are only identical.

##   After classification, there are 1554 reads/indexes which are strictly sequencer.

##   After classification, there are 63275 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 1429423 forward reads and 1637306 reverse_reads.

## Subsetting based on mutations with at least 3 indexes.

## Classified mutation strings according to various criteria.

## Starting sample: s6.

##   Reading the file containing mutations: preprocessing/s6/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s6/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s6/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2012128 reads.

##     Mutation data: after min-position pruning, there are: 1966909 reads: 45219 lost or 2.25%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 1966909 reads.

##     Mutation data: after max-position pruning, there are: 747617 reads: 1219292 lost or 61.99%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 734242 reads: 13375 lost or 1.79%.

##   Mutation data: all filters removed 1277886 reads, or 63.51%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 844764 indexes in all the data.

##     After reads/index pruning, there are: 206713 indexes: 638051 lost or 75.53%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 734242 changed reads.

##     All data: before reads/index pruning, there are: 1254041 identical reads.

##     All data: after index pruning, there are: 316103 changed reads: 43.05%.

##     All data: after index pruning, there are: 627064 identical reads: 50.00%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 627064 identical reads.

##   Before classification, there are 316103 reads with mutations.

##   After classification, there are 375136 reads/indexes which are only identical.

##   After classification, there are 270 reads/indexes which are strictly sequencer.

##   After classification, there are 71568 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 727996 forward reads and 950480 reverse_reads.

## Subsetting based on mutations with at least 3 indexes.

## Classified mutation strings according to various criteria.

## Plotting index densities.

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in dir.create(excel_dir, recursive = TRUE): cannot create dir 'excel',
## reason 'Transport endpoint is not connected'

##   Writing a legend.

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/pre_chng_density.pdf'

## Warning: Removed 12000 rows containing non-finite values (stat_density).

## Warning: Removed 12000 rows containing non-finite values (stat_density).

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/pre_ident_density.pdf'

## Warning: Removed 11672 rows containing non-finite values (stat_density).

## Warning: Removed 11672 rows containing non-finite values (stat_density).

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/pre_all_index_density.pdf'

## Warning: Removed 30533 rows containing non-finite values (stat_density).

## Warning: Removed 30533 rows containing non-finite values (stat_density).

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/post_chng_density.pdf'

## Warning: Removed 4037 rows containing non-finite values (stat_density).

## Warning: Removed 4037 rows containing non-finite values (stat_density).

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/post_ident_density.pdf'

## Warning: Removed 1431 rows containing non-finite values (stat_density).

## Warning: Removed 1431 rows containing non-finite values (stat_density).

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/post_all_index_density.pdf'

## Warning: Removed 3916 rows containing non-finite values (stat_density).

## Warning: Removed 3916 rows containing non-finite values (stat_density).

##   Writing raw data.

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_indexes_by_refnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_sequencer_by_refnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_indexes_by_hitnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_sequencer_by_hitnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_indexes_by_type.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_sequencer_by_type.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_indexes_by_trans.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_sequencer_by_trans.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_indexes_by_strength.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_sequencer_by_strength.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_sequencer_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_insert_indexes_by_nt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_sequencer_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_insert_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_indexes_by_string.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_miss_sequencer_by_string.pdf'

##   Writing cpm data.

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_indexes_by_refnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_sequencer_by_refnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_indexes_by_hitnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_sequencer_by_hitnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_indexes_by_type.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_sequencer_by_type.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_indexes_by_trans.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_sequencer_by_trans.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_indexes_by_strength.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_sequencer_by_strength.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_sequencer_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_insert_indexes_by_nt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_sequencer_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_insert_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_indexes_by_string.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_cpm_miss_sequencer_by_string.pdf'

##   Writing data normalized by reads/indexes.

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_indexes_by_refnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_sequencer_by_refnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_indexes_by_hitnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_sequencer_by_hitnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_indexes_by_type.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_sequencer_by_type.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_indexes_by_trans.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_sequencer_by_trans.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_indexes_by_strength.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_sequencer_by_strength.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_sequencer_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_insert_indexes_by_nt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_sequencer_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_insert_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_indexes_by_string.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_counts_miss_sequencer_by_string.pdf'

##   Writing data normalized by reads/indexes and length.

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_indexes_by_refnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_sequencer_by_refnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_indexes_by_hitnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_sequencer_by_hitnt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_indexes_by_type.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_sequencer_by_type.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_indexes_by_trans.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_sequencer_by_trans.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_indexes_by_strength.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_sequencer_by_strength.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_sequencer_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_insert_indexes_by_nt.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_sequencer_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_insert_indexes_by_position.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_indexes_by_string.pdf'

## Warning in dir.create(savedir, recursive = TRUE): cannot create dir
## 'saved_plots', reason 'Transport endpoint is not connected'

## Error in pdf(file = high_quality) : 
##   cannot open file 'saved_plots/matrices_countslength_miss_sequencer_by_string.pdf'

## Error in setwd(wd): character argument expected

max_mutations_per_read <- 10
excel <- glue::glue("excel/{rundate}_triples_tenmpr-v{ver}.xlsx")
triples_tenmpr <- create_matrices(sample_sheet=sample_sheet,
                                  ident_column=ident_column, mut_column=mut_column,
                                  min_reads=min_reads, min_indexes=min_indexes,
                                  min_sequencer=min_sequencer,
                                  min_position=min_position, max_position=max_position,
                                  prune_n=prune_n, verbose=verbose, excel=excel)

## Error in read.xlsx.default(xlsxFile = file, sheet = 1) : 
##   File does not exist.

## Error in read_metadata(meta_file, ...): Unable to read the metadata file: sample_sheets/all_samples.xlsx

max_mutations_per_read <- 5
excel <- glue::glue("excel/{rundate}_triples_fivempr-v{ver}.xlsx")
triples_fivempr <- create_matrices(sample_sheet=sample_sheet,
                                   ident_column=ident_column, mut_column=mut_column,
                                   min_reads=min_reads, min_indexes=min_indexes,
                                   min_sequencer=min_sequencer,
                                   min_position=min_position, max_position=max_position,
                                   prune_n=prune_n, verbose=verbose, excel=excel)

## Error in read.xlsx.default(xlsxFile = file, sheet = 1) : 
##   File does not exist.

## Error in read_metadata(meta_file, ...): Unable to read the metadata file: sample_sheets/all_samples.xlsx

1.2 Categorize the data with at least 5 indexes per mutant

min_indexes <- 5
max_mutations_per_read <- NULL
excel <- glue::glue("excel/{rundate}_quints-v{ver}.xlsx")
quints <- create_matrices(sample_sheet=sample_sheet,
                          ident_column=ident_column, mut_column=mut_column,
                          min_reads=min_reads, min_indexes=min_indexes,
                          min_sequencer=min_sequencer,
                          min_position=min_position, max_position=max_position,
                          prune_n=prune_n, verbose=verbose, excel=excel)

## Dropped 3 rows from the sample metadata because they were blank.

## Starting sample: s4.

##   Reading the file containing mutations: preprocessing/s4/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s4/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s4/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2239016 reads.

##     Mutation data: after min-position pruning, there are: 2183747 reads: 55269 lost or 2.47%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 2183747 reads.

##     Mutation data: after max-position pruning, there are: 837996 reads: 1345751 lost or 61.63%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 832549 reads: 5447 lost or 0.65%.

##   Mutation data: all filters removed 1406467 reads, or 62.82%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 973940 indexes in all the data.

##     After reads/index pruning, there are: 354890 indexes: 619050 lost or 63.56%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 832549 changed reads.

##     All data: before reads/index pruning, there are: 1975296 identical reads.

##     All data: after index pruning, there are: 454061 changed reads: 54.54%.

##     All data: after index pruning, there are: 1351672 identical reads: 68.43%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 1351672 identical reads.

##   Before classification, there are 454061 reads with mutations.

##   After classification, there are 779988 reads/indexes which are only identical.

##   After classification, there are 3025 reads/indexes which are strictly sequencer.

##   After classification, there are 38821 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 1634797 forward reads and 2107382 reverse_reads.

## Subsetting based on mutations with at least 5 indexes.

## Classified mutation strings according to various criteria.

## Starting sample: s5.

##   Reading the file containing mutations: preprocessing/s5/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s5/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s5/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2259996 reads.

##     Mutation data: after min-position pruning, there are: 2204009 reads: 55987 lost or 2.48%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 2204009 reads.

##     Mutation data: after max-position pruning, there are: 825004 reads: 1379005 lost or 62.57%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 817693 reads: 7311 lost or 0.89%.

##   Mutation data: all filters removed 1442303 reads, or 63.82%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 965529 indexes in all the data.

##     After reads/index pruning, there are: 316034 indexes: 649495 lost or 67.27%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 817693 changed reads.

##     All data: before reads/index pruning, there are: 1764804 identical reads.

##     All data: after index pruning, there are: 420259 changed reads: 51.40%.

##     All data: after index pruning, there are: 1113889 identical reads: 63.12%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 1113889 identical reads.

##   Before classification, there are 420259 reads with mutations.

##   After classification, there are 650056 reads/indexes which are only identical.

##   After classification, there are 1554 reads/indexes which are strictly sequencer.

##   After classification, there are 63275 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 1429423 forward reads and 1637306 reverse_reads.

## Subsetting based on mutations with at least 5 indexes.

## Classified mutation strings according to various criteria.

## Starting sample: s6.

##   Reading the file containing mutations: preprocessing/s6/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s6/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s6/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2012128 reads.

##     Mutation data: after min-position pruning, there are: 1966909 reads: 45219 lost or 2.25%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 1966909 reads.

##     Mutation data: after max-position pruning, there are: 747617 reads: 1219292 lost or 61.99%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 734242 reads: 13375 lost or 1.79%.

##   Mutation data: all filters removed 1277886 reads, or 63.51%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 844764 indexes in all the data.

##     After reads/index pruning, there are: 206713 indexes: 638051 lost or 75.53%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 734242 changed reads.

##     All data: before reads/index pruning, there are: 1254041 identical reads.

##     All data: after index pruning, there are: 316103 changed reads: 43.05%.

##     All data: after index pruning, there are: 627064 identical reads: 50.00%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 627064 identical reads.

##   Before classification, there are 316103 reads with mutations.

##   After classification, there are 375136 reads/indexes which are only identical.

##   After classification, there are 270 reads/indexes which are strictly sequencer.

##   After classification, there are 71568 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 727996 forward reads and 950480 reverse_reads.

## Subsetting based on mutations with at least 5 indexes.

## Classified mutation strings according to various criteria.

## Plotting index densities.

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

##   Writing a legend.

## Warning: Removed 12000 rows containing non-finite values (stat_density).

## Warning: Removed 12000 rows containing non-finite values (stat_density).

## Warning: Removed 11672 rows containing non-finite values (stat_density).

## Warning: Removed 11672 rows containing non-finite values (stat_density).

## Warning: Removed 30533 rows containing non-finite values (stat_density).

## Warning: Removed 30533 rows containing non-finite values (stat_density).

## Warning: Removed 4037 rows containing non-finite values (stat_density).

## Warning: Removed 4037 rows containing non-finite values (stat_density).

## Warning: Removed 1431 rows containing non-finite values (stat_density).

## Warning: Removed 1431 rows containing non-finite values (stat_density).

## Warning: Removed 3916 rows containing non-finite values (stat_density).

## Warning: Removed 3916 rows containing non-finite values (stat_density).

##   Writing raw data.

##   Writing cpm data.

##   Writing data normalized by reads/indexes.

##   Writing data normalized by reads/indexes and length.

max_mutations_per_read <- 10
excel <- glue::glue("excel/{rundate}_quints_tenmpr-v{ver}.xlsx")
quints_tenmpr <- create_matrices(sample_sheet=sample_sheet,
                                 ident_column=ident_column, mut_column=mut_column,
                                 min_reads=min_reads, min_indexes=min_indexes,
                                 min_sequencer=min_sequencer,
                                 min_position=min_position, max_position=max_position,
                                 prune_n=prune_n, verbose=verbose, excel=excel)

## Dropped 3 rows from the sample metadata because they were blank.

## Starting sample: s4.

##   Reading the file containing mutations: preprocessing/s4/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s4/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s4/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2239016 reads.

##     Mutation data: after min-position pruning, there are: 2183747 reads: 55269 lost or 2.47%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 2183747 reads.

##     Mutation data: after max-position pruning, there are: 837996 reads: 1345751 lost or 61.63%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 832549 reads: 5447 lost or 0.65%.

##   Mutation data: all filters removed 1406467 reads, or 62.82%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 973940 indexes in all the data.

##     After reads/index pruning, there are: 354890 indexes: 619050 lost or 63.56%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 832549 changed reads.

##     All data: before reads/index pruning, there are: 1975296 identical reads.

##     All data: after index pruning, there are: 454061 changed reads: 54.54%.

##     All data: after index pruning, there are: 1351672 identical reads: 68.43%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 1351672 identical reads.

##   Before classification, there are 454061 reads with mutations.

##   After classification, there are 779988 reads/indexes which are only identical.

##   After classification, there are 3025 reads/indexes which are strictly sequencer.

##   After classification, there are 38821 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 1634797 forward reads and 2107382 reverse_reads.

## Subsetting based on mutations with at least 5 indexes.

## Classified mutation strings according to various criteria.

## Starting sample: s5.

##   Reading the file containing mutations: preprocessing/s5/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s5/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s5/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2259996 reads.

##     Mutation data: after min-position pruning, there are: 2204009 reads: 55987 lost or 2.48%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 2204009 reads.

##     Mutation data: after max-position pruning, there are: 825004 reads: 1379005 lost or 62.57%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 817693 reads: 7311 lost or 0.89%.

##   Mutation data: all filters removed 1442303 reads, or 63.82%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 965529 indexes in all the data.

##     After reads/index pruning, there are: 316034 indexes: 649495 lost or 67.27%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 817693 changed reads.

##     All data: before reads/index pruning, there are: 1764804 identical reads.

##     All data: after index pruning, there are: 420259 changed reads: 51.40%.

##     All data: after index pruning, there are: 1113889 identical reads: 63.12%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 1113889 identical reads.

##   Before classification, there are 420259 reads with mutations.

##   After classification, there are 650056 reads/indexes which are only identical.

##   After classification, there are 1554 reads/indexes which are strictly sequencer.

##   After classification, there are 63275 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 1429423 forward reads and 1637306 reverse_reads.

## Subsetting based on mutations with at least 5 indexes.

## Classified mutation strings according to various criteria.

## Starting sample: s6.

##   Reading the file containing mutations: preprocessing/s6/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s6/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s6/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2012128 reads.

##     Mutation data: after min-position pruning, there are: 1966909 reads: 45219 lost or 2.25%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 1966909 reads.

##     Mutation data: after max-position pruning, there are: 747617 reads: 1219292 lost or 61.99%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 734242 reads: 13375 lost or 1.79%.

##   Mutation data: all filters removed 1277886 reads, or 63.51%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 844764 indexes in all the data.

##     After reads/index pruning, there are: 206713 indexes: 638051 lost or 75.53%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 734242 changed reads.

##     All data: before reads/index pruning, there are: 1254041 identical reads.

##     All data: after index pruning, there are: 316103 changed reads: 43.05%.

##     All data: after index pruning, there are: 627064 identical reads: 50.00%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 627064 identical reads.

##   Before classification, there are 316103 reads with mutations.

##   After classification, there are 375136 reads/indexes which are only identical.

##   After classification, there are 270 reads/indexes which are strictly sequencer.

##   After classification, there are 71568 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 727996 forward reads and 950480 reverse_reads.

## Subsetting based on mutations with at least 5 indexes.

## Classified mutation strings according to various criteria.

## Plotting index densities.

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

##   Writing a legend.

## Warning: Removed 12000 rows containing non-finite values (stat_density).

## Warning: Removed 12000 rows containing non-finite values (stat_density).

## Warning: Removed 11672 rows containing non-finite values (stat_density).

## Warning: Removed 11672 rows containing non-finite values (stat_density).

## Warning: Removed 30533 rows containing non-finite values (stat_density).

## Warning: Removed 30533 rows containing non-finite values (stat_density).

## Warning: Removed 4037 rows containing non-finite values (stat_density).

## Warning: Removed 4037 rows containing non-finite values (stat_density).

## Warning: Removed 1431 rows containing non-finite values (stat_density).

## Warning: Removed 1431 rows containing non-finite values (stat_density).

## Warning: Removed 3916 rows containing non-finite values (stat_density).

## Warning: Removed 3916 rows containing non-finite values (stat_density).

##   Writing raw data.

##   Writing cpm data.

##   Writing data normalized by reads/indexes.

##   Writing data normalized by reads/indexes and length.

max_mutations_per_read <- 5
excel <- glue::glue("excel/{rundate}_quints_fivempr-v{ver}.xlsx")
quints_fivempr <- create_matrices(sample_sheet=sample_sheet,
                                  ident_column=ident_column, mut_column=mut_column,
                                  min_reads=min_reads, min_indexes=min_indexes,
                                  min_sequencer=min_sequencer,
                                  min_position=min_position, max_position=max_position,
                                  prune_n=prune_n, verbose=verbose, excel=excel)

## Dropped 3 rows from the sample metadata because they were blank.

## Starting sample: s4.

##   Reading the file containing mutations: preprocessing/s4/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s4/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s4/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2239016 reads.

##     Mutation data: after min-position pruning, there are: 2183747 reads: 55269 lost or 2.47%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 2183747 reads.

##     Mutation data: after max-position pruning, there are: 837996 reads: 1345751 lost or 61.63%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 832549 reads: 5447 lost or 0.65%.

##   Mutation data: all filters removed 1406467 reads, or 62.82%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 973940 indexes in all the data.

##     After reads/index pruning, there are: 354890 indexes: 619050 lost or 63.56%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 832549 changed reads.

##     All data: before reads/index pruning, there are: 1975296 identical reads.

##     All data: after index pruning, there are: 454061 changed reads: 54.54%.

##     All data: after index pruning, there are: 1351672 identical reads: 68.43%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 1351672 identical reads.

##   Before classification, there are 454061 reads with mutations.

##   After classification, there are 779988 reads/indexes which are only identical.

##   After classification, there are 3025 reads/indexes which are strictly sequencer.

##   After classification, there are 38821 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 1634797 forward reads and 2107382 reverse_reads.

## Subsetting based on mutations with at least 5 indexes.

## Classified mutation strings according to various criteria.

## Starting sample: s5.

##   Reading the file containing mutations: preprocessing/s5/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s5/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s5/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2259996 reads.

##     Mutation data: after min-position pruning, there are: 2204009 reads: 55987 lost or 2.48%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 2204009 reads.

##     Mutation data: after max-position pruning, there are: 825004 reads: 1379005 lost or 62.57%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 817693 reads: 7311 lost or 0.89%.

##   Mutation data: all filters removed 1442303 reads, or 63.82%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 965529 indexes in all the data.

##     After reads/index pruning, there are: 316034 indexes: 649495 lost or 67.27%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 817693 changed reads.

##     All data: before reads/index pruning, there are: 1764804 identical reads.

##     All data: after index pruning, there are: 420259 changed reads: 51.40%.

##     All data: after index pruning, there are: 1113889 identical reads: 63.12%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 1113889 identical reads.

##   Before classification, there are 420259 reads with mutations.

##   After classification, there are 650056 reads/indexes which are only identical.

##   After classification, there are 1554 reads/indexes which are strictly sequencer.

##   After classification, there are 63275 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 1429423 forward reads and 1637306 reverse_reads.

## Subsetting based on mutations with at least 5 indexes.

## Classified mutation strings according to various criteria.

## Starting sample: s6.

##   Reading the file containing mutations: preprocessing/s6/step4.txt.xz

##   Reading the file containing the identical reads: preprocessing/s6/step2_identical_reads.txt.xz

## Warning: 1 parsing failure.
## row    col expected   actual                                            file
##   1 readid a number read_num 'preprocessing/s6/step2_identical_reads.txt.xz'

##   Counting indexes before filtering.

##     Mutation data: removing any differences before position: 24.

##     Mutation data: before pruning, there are: 2012128 reads.

##     Mutation data: after min-position pruning, there are: 1966909 reads: 45219 lost or 2.25%.

##     Mutation data: removing any differences after position: 176.

##     Mutation data: before pruning, there are: 1966909 reads.

##     Mutation data: after max-position pruning, there are: 747617 reads: 1219292 lost or 61.99%.

##     Mutation data: removing any reads with 'N' as the hit.

##     Mutation data: after N pruning, there are: 734242 reads: 13375 lost or 1.79%.

##   Mutation data: all filters removed 1277886 reads, or 63.51%.

##     Gathering information about the number of reads per index.

##     Before reads/index pruning, there are: 844764 indexes in all the data.

##     After reads/index pruning, there are: 206713 indexes: 638051 lost or 75.53%.

##     All data: removing indexes with fewer than 3 reads/index.

##     All data: before reads/index pruning, there are: 734242 changed reads.

##     All data: before reads/index pruning, there are: 1254041 identical reads.

##     All data: after index pruning, there are: 316103 changed reads: 43.05%.

##     All data: after index pruning, there are: 627064 identical reads: 50.00%.

##   Gathering identical, mutant, and sequencer reads/indexes.

##   Before classification, there are 627064 identical reads.

##   Before classification, there are 316103 reads with mutations.

##   After classification, there are 375136 reads/indexes which are only identical.

##   After classification, there are 270 reads/indexes which are strictly sequencer.

##   After classification, there are 71568 reads/indexes which are deemed from reverse transcriptase.

##   Counted by direction: 727996 forward reads and 950480 reverse_reads.

## Subsetting based on mutations with at least 5 indexes.

## Classified mutation strings according to various criteria.

## Plotting index densities.

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(matrix_melted[["category"]]))): no non-missing
## arguments to max; returning -Inf

##   Writing a legend.

## Warning: Removed 12000 rows containing non-finite values (stat_density).

## Warning: Removed 12000 rows containing non-finite values (stat_density).

## Warning: Removed 11672 rows containing non-finite values (stat_density).

## Warning: Removed 11672 rows containing non-finite values (stat_density).

## Warning: Removed 30533 rows containing non-finite values (stat_density).

## Warning: Removed 30533 rows containing non-finite values (stat_density).

## Warning: Removed 4037 rows containing non-finite values (stat_density).

## Warning: Removed 4037 rows containing non-finite values (stat_density).

## Warning: Removed 1431 rows containing non-finite values (stat_density).

## Warning: Removed 1431 rows containing non-finite values (stat_density).

## Warning: Removed 3916 rows containing non-finite values (stat_density).

## Warning: Removed 3916 rows containing non-finite values (stat_density).

##   Writing raw data.

##   Writing cpm data.

##   Writing data normalized by reads/indexes.

##   Writing data normalized by reads/indexes and length.

2 Questions from Dr. DeStefano

I think what is best is to get the number of recovered mutations of each type from each data set. That would be A to T, A to G, A to C; T to A, T to G, T to C; G to A, G to C, G to T; and C to A, C to G, C to T; as well as deletions and insertions. I would then need the sum number of the reads that met all our criteria (i.e. at least 3 good recovered reads for that 14 nt index). Each set of 3 or more would ct as “1” read of that particular index so I would need the total with this in mind. I also need to know the total number of nucleotides that were in the region we decided to consider in the analysis. We may want to try this for 3 or more and 5 or more recovered indexes if it is not hard. This information does not include specific positions on the template where errors occurred but we can look at that latter. Right now I just want to get a general error rate and type of error. It would basically be calculated by dividing the number of recovered mutations of a particular type by sum number of the reads times the number of nucleotides screened in the template. As it ends up, this number does not really have a lot of meaning but it can be used to calculate the overall mutation rate as well as the rate for transversions, transitions, and deletions and insertions.

3 Answers

In order to address those queries, I invoked create_matrices() with a minimum index count of 3 and 5. It should be noted that this is not the same as requiring 3 or 5 reads per index. In both cases I require 3 reads per index.

3.1 Recovered mutations of each type

I am interpreting this question as the number of indexes recovered for each mutation type. I collect this information in 2 ways of interest: the indexes by type which are deemed to be from the RT and from the sequencer. In addition, I calculate a normalized (cpm) version of this information which may be used to look for changes across samples.

3.1.1 Mutations by RT index

This following block should print out tables of the numbers of mutant indexes observed for each type for the RT and the sequencer. One would hope that the sequencer will be consistent for all samples, but I think the results will instead suggest that my metric is not yet stringent enough.

knitr::kable(triples[["matrices"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples[["matrices"]][["miss_indexes_by_type"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["matrices"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples_tenmpr[["matrices"]][["miss_indexes_by_type"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["matrices"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples_fivempr[["matrices"]][["miss_indexes_by_type"]]): object 'triples_fivempr' not found

knitr::kable(quints[["matrices"]][["miss_indexes_by_type"]])

	s4	s5	s6
A_C	1148	1390	1930
A_G	689	3474	8702
A_T	1597	2141	2774
C_A	21676	21121	18344
C_G	1545	8123	8239
C_T	1565	4569	10301
G_A	1440	11668	9513
G_C	635	584	497
G_T	5632	5387	4898
T_A	930	1790	2275
T_C	382	852	1305
T_G	1381	1952	2574

knitr::kable(quints_tenmpr[["matrices"]][["miss_indexes_by_type"]])

	s4	s5	s6
A_C	1148	1390	1930
A_G	689	3474	8702
A_T	1597	2141	2774
C_A	21676	21121	18344
C_G	1545	8123	8239
C_T	1565	4569	10301
G_A	1440	11668	9513
G_C	635	584	497
G_T	5632	5387	4898
T_A	930	1790	2275
T_C	382	852	1305
T_G	1381	1952	2574

knitr::kable(quints_fivempr[["matrices"]][["miss_indexes_by_type"]])

	s4	s5	s6
A_C	1148	1390	1930
A_G	689	3474	8702
A_T	1597	2141	2774
C_A	21676	21121	18344
C_G	1545	8123	8239
C_T	1565	4569	10301
G_A	1440	11668	9513
G_C	635	584	497
G_T	5632	5387	4898
T_A	930	1790	2275
T_C	382	852	1305
T_G	1381	1952	2574

knitr::kable(triples[["matrices"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples[["matrices"]][["miss_sequencer_by_type"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["matrices"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples_tenmpr[["matrices"]][["miss_sequencer_by_type"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["matrices"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples_fivempr[["matrices"]][["miss_sequencer_by_type"]]): object 'triples_fivempr' not found

knitr::kable(quints[["matrices"]][["miss_sequencer_by_type"]])

	s4	s5	s6
A_C	177	120	0
A_T	233	128	15
C_A	1247	635	90
C_G	121	36	0
C_T	10	5	0
G_A	13	0	0
G_C	7	0	0
G_T	219	79	0
T_A	243	108	5
T_C	16	5	0
T_G	372	190	12

knitr::kable(quints_tenmpr[["matrices"]][["miss_sequencer_by_type"]])

	s4	s5	s6
A_C	177	120	0
A_T	233	128	15
C_A	1247	635	90
C_G	121	36	0
C_T	10	5	0
G_A	13	0	0
G_C	7	0	0
G_T	219	79	0
T_A	243	108	5
T_C	16	5	0
T_G	372	190	12

knitr::kable(quints_fivempr[["matrices"]][["miss_sequencer_by_type"]])

	s4	s5	s6
A_C	177	120	0
A_T	233	128	15
C_A	1247	635	90
C_G	121	36	0
C_T	10	5	0
G_A	13	0	0
G_C	7	0	0
G_T	219	79	0
T_A	243	108	5
T_C	16	5	0
T_G	372	190	12

Plots of this information

triples[["plots"]][["counts"]][["miss_indexes_by_type"]]

## Error in eval(expr, envir, enclos): object 'triples' not found

triples_tenmpr[["plots"]][["counts"]][["miss_indexes_by_type"]]

## Error in eval(expr, envir, enclos): object 'triples_tenmpr' not found

triples_fivempr[["plots"]][["counts"]][["miss_indexes_by_type"]]

## Error in eval(expr, envir, enclos): object 'triples_fivempr' not found

quints[["plots"]][["counts"]][["miss_indexes_by_type"]]

quints_tenmpr[["plots"]][["counts"]][["miss_indexes_by_type"]]

quints_fivempr[["plots"]][["counts"]][["miss_indexes_by_type"]]

This suggests to me that this information needs to be normalized in some more sensible fashion. Thus the following:

3.1.2 Mutations by RT index, post normalization

The same numbers may be expressed in the context of the number of indexes observed / sample and/or as a cpm across samples. Thus in the first instance one can look at the apparent error rate for each sample, and in the second instance one may look for relative changes in apparent error rate across samples.

3.1.2.1 Rewriting the matrices as cpm to account for library sizes.

knitr::kable(triples[["normalized"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples[["normalized"]][["miss_indexes_by_type"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["normalized"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples_tenmpr[["normalized"]][["miss_indexes_by_type"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["normalized"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples_fivempr[["normalized"]][["miss_indexes_by_type"]]): object 'triples_fivempr' not found

knitr::kable(quints[["normalized"]][["miss_indexes_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_tenmpr[["normalized"]][["miss_indexes_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_fivempr[["normalized"]][["miss_indexes_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(triples[["normalized"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples[["normalized"]][["miss_sequencer_by_type"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["normalized"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples_tenmpr[["normalized"]][["miss_sequencer_by_type"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["normalized"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples_fivempr[["normalized"]][["miss_sequencer_by_type"]]): object 'triples_fivempr' not found

knitr::kable(quints[["normalized"]][["miss_sequencer_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_tenmpr[["normalized"]][["miss_sequencer_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_fivempr[["normalized"]][["miss_sequencer_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

3.1.2.2 Rewriting the matrices by dividing by all indexes

This I think starts to address the later text in your query.

knitr::kable(triples[["matrices_by_counts"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples[["matrices_by_counts"]][["miss_indexes_by_type"]]): object 'triples' not found

knitr::kable(quints[["matrices_by_counts"]][["miss_indexes_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(triples[["matrices_by_counts"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples[["matrices_by_counts"]][["miss_sequencer_by_type"]]): object 'triples' not found

knitr::kable(quints[["matrices_by_counts"]][["miss_sequencer_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

3.1.2.3 Rewriting the matrices by dividing by all indexes and cpm

I think this might prove to be where we get the most meaningful results.

The nicest thing in it is that after accounting for library sizes and total indexes observed, we finally see that the sequencer error is mostly consistent across all samples and mutation types – with a couple of notable exceptions.

By the same token, for the mutations which are identical for the sequencer, we have some which are decidedly different for the non-sequencer data. The most notable examples I think are A to G but _not G to A; and C to T.

knitr::kable(triples[["normalized_by_counts"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples[["normalized_by_counts"]][["miss_indexes_by_type"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["normalized_by_counts"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples_tenmpr[["normalized_by_counts"]][["miss_indexes_by_type"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["normalized_by_counts"]][["miss_indexes_by_type"]])

## Error in knitr::kable(triples_fivempr[["normalized_by_counts"]][["miss_indexes_by_type"]]): object 'triples_fivempr' not found

knitr::kable(quints[["normalized_by_counts"]][["miss_indexes_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_tenmpr[["normalized_by_counts"]][["miss_indexes_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_fivempr[["normalized_by_counts"]][["miss_indexes_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(triples[["normalized_by_counts"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples[["normalized_by_counts"]][["miss_sequencer_by_type"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["normalized_by_counts"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples_tenmpr[["normalized_by_counts"]][["miss_sequencer_by_type"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["normalized_by_counts"]][["miss_sequencer_by_type"]])

## Error in knitr::kable(triples_fivempr[["normalized_by_counts"]][["miss_sequencer_by_type"]]): object 'triples_fivempr' not found

knitr::kable(quints[["normalized_by_counts"]][["miss_sequencer_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_tenmpr[["normalized_by_counts"]][["miss_sequencer_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_fivempr[["normalized_by_counts"]][["miss_sequencer_by_type"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

3.1.3 Indels by RT index

The following blocks will repeat the above, but looking for insertions. This data does not observe sufficient deletions to make a proper count for them.

knitr::kable(triples[["matrices"]][["insert_indexes_by_nt"]])

## Error in knitr::kable(triples[["matrices"]][["insert_indexes_by_nt"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["matrices"]][["insert_indexes_by_nt"]])

## Error in knitr::kable(triples_tenmpr[["matrices"]][["insert_indexes_by_nt"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["matrices"]][["insert_indexes_by_nt"]])

## Error in knitr::kable(triples_fivempr[["matrices"]][["insert_indexes_by_nt"]]): object 'triples_fivempr' not found

knitr::kable(quints[["matrices"]][["insert_indexes_by_nt"]])

	s5	s6
A	34	82
C	12	12
G	0	8
T	81	35

knitr::kable(quints_tenmpr[["matrices"]][["insert_indexes_by_nt"]])

	s5	s6
A	34	82
C	12	12
G	0	8
T	81	35

knitr::kable(quints_fivempr[["matrices"]][["insert_indexes_by_nt"]])

	s5	s6
A	34	82
C	12	12
G	0	8
T	81	35

knitr::kable(triples[["matrices"]][["insert_sequencer_by_nt"]])

## Error in knitr::kable(triples[["matrices"]][["insert_sequencer_by_nt"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["matrices"]][["insert_sequencer_by_nt"]])

## Error in knitr::kable(triples_tenmpr[["matrices"]][["insert_sequencer_by_nt"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["matrices"]][["insert_sequencer_by_nt"]])

## Error in knitr::kable(triples_fivempr[["matrices"]][["insert_sequencer_by_nt"]]): object 'triples_fivempr' not found

knitr::kable(quints[["matrices"]][["insert_sequencer_by_nt"]])

s.x	s.y	s

knitr::kable(quints_tenmpr[["matrices"]][["insert_sequencer_by_nt"]])

s.x	s.y	s

knitr::kable(quints_fivempr[["matrices"]][["insert_sequencer_by_nt"]])

s.x	s.y	s

Plots of this information

triples[["plots"]][["counts"]][["insert_indexes_by_nt"]]

## Error in eval(expr, envir, enclos): object 'triples' not found

triples_tenmpr[["plots"]][["counts"]][["insert_indexes_by_nt"]]

## Error in eval(expr, envir, enclos): object 'triples_tenmpr' not found

triples_fivempr[["plots"]][["counts"]][["insert_indexes_by_nt"]]

## Error in eval(expr, envir, enclos): object 'triples_fivempr' not found

quints[["plots"]][["counts"]][["insert_indexes_by_nt"]]

quints_tenmpr[["plots"]][["counts"]][["insert_indexes_by_nt"]]

quints_fivempr[["plots"]][["counts"]][["insert_indexes_by_nt"]]

3.1.4 Insertions by RT index, post normalization

3.1.4.1 Rewriting the matrices as cpm to account for library sizes.

knitr::kable(triples[["normalized"]][["insert_indexes_by_nt"]])

## Error in knitr::kable(triples[["normalized"]][["insert_indexes_by_nt"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["normalized"]][["insert_indexes_by_nt"]])

## Error in knitr::kable(triples_tenmpr[["normalized"]][["insert_indexes_by_nt"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["normalized"]][["insert_indexes_by_nt"]])

## Error in knitr::kable(triples_fivempr[["normalized"]][["insert_indexes_by_nt"]]): object 'triples_fivempr' not found

knitr::kable(quints[["normalized"]][["insert_indexes_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_tenmpr[["normalized"]][["insert_indexes_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_fivempr[["normalized"]][["insert_indexes_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(triples[["normalized"]][["insert_sequencer_by_nt"]])

## Error in knitr::kable(triples[["normalized"]][["insert_sequencer_by_nt"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["normalized"]][["insert_sequencer_by_nt"]])

## Error in knitr::kable(triples_tenmpr[["normalized"]][["insert_sequencer_by_nt"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["normalized"]][["insert_sequencer_by_nt"]])

## Error in knitr::kable(triples_fivempr[["normalized"]][["insert_sequencer_by_nt"]]): object 'triples_fivempr' not found

knitr::kable(quints[["normalized"]][["insert_sequencer_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_tenmpr[["normalized"]][["insert_sequencer_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_fivempr[["normalized"]][["insert_sequencer_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

3.1.4.2 Rewriting the matrices by dividing by all indexes

I think that there are few enough insertion events that this gets a bit messed up. I will double check the logic of this, but that is my initial guess given how few insertions I was seeing when reading the outputs manually. Unfortunately, this means that for these I also cannot provide a cpm measurement.

knitr::kable(triples[["matrices_by_counts"]][["insert_indexes_by_nt"]])

## Error in knitr::kable(triples[["matrices_by_counts"]][["insert_indexes_by_nt"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["matrices_by_counts"]][["insert_indexes_by_nt"]])

## Error in knitr::kable(triples_tenmpr[["matrices_by_counts"]][["insert_indexes_by_nt"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["matrices_by_counts"]][["insert_indexes_by_nt"]])

## Error in knitr::kable(triples_fivempr[["matrices_by_counts"]][["insert_indexes_by_nt"]]): object 'triples_fivempr' not found

knitr::kable(quints[["matrices_by_counts"]][["insert_indexes_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_tenmpr[["matrices_by_counts"]][["insert_indexes_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_fivempr[["matrices_by_counts"]][["insert_indexes_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(triples[["matrices_by_counts"]][["insert_sequencer_by_nt"]])

## Error in knitr::kable(triples[["matrices_by_counts"]][["insert_sequencer_by_nt"]]): object 'triples' not found

knitr::kable(triples_tenmpr[["matrices_by_counts"]][["insert_sequencer_by_nt"]])

## Error in knitr::kable(triples_tenmpr[["matrices_by_counts"]][["insert_sequencer_by_nt"]]): object 'triples_tenmpr' not found

knitr::kable(triples_fivempr[["matrices_by_counts"]][["insert_sequencer_by_nt"]])

## Error in knitr::kable(triples_fivempr[["matrices_by_counts"]][["insert_sequencer_by_nt"]]): object 'triples_fivempr' not found

knitr::kable(quints[["matrices_by_counts"]][["insert_sequencer_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_tenmpr[["matrices_by_counts"]][["insert_sequencer_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

knitr::kable(quints_fivempr[["matrices_by_counts"]][["insert_sequencer_by_nt"]])

## Warning in kable_markdown(x, padding = padding, ...): The table should have a
## header (column names)

|| || || ||

The following is my previous writing of this worksheet which just dumped the various tables.

4 Print raw tables

for (t in 1:length(triples[["matrices"]])) {
  table_name <- names(triples[["matrices"]])[t]
  message("Raw table: ", table_name, ".")
  print(knitr::kable(triples[["matrices"]][t]))
}

## Error in eval(expr, envir, enclos): object 'triples' not found

5 Print raw plots

for (t in 1:length(triples[["plots"]][["matrices"]])) {
  message("Raw table: ", table_name, ".")
  print(triples[["plots"]][["matrices"]][t])
}

## Error in eval(expr, envir, enclos): object 'triples' not found

6 Print normalized tables

for (t in 1:length(triples[["matrices_counts"]])) {
  table_name <- names(triples[["matrices_counts"]])[t]
  message("Normalized table: ", table_name, ".")
  print(knitr::kable(triples[["matrices_counts"]][t]))
}

## Error in eval(expr, envir, enclos): object 'triples' not found

7 Print normalized plots

for (t in 1:length(triples[["plots"]][["counts"]])) {
  message("Normalized table: ", table_name, ".")
  print(triples[["plots"]][["counts"]][t])
}

## Error in eval(expr, envir, enclos): object 'triples' not found

pander::pander(sessionInfo())

R version 3.6.1 (2019-07-05)

Platform: x86_64-pc-linux-gnu (64-bit)

locale: LC_CTYPE=en_US.UTF-8, LC_NUMERIC=C, LC_TIME=en_US.UTF-8, LC_COLLATE=en_US.UTF-8, LC_MONETARY=en_US.UTF-8, LC_MESSAGES=en_US.UTF-8, LC_PAPER=en_US.UTF-8, LC_NAME=C, LC_ADDRESS=C, LC_TELEPHONE=C, LC_MEASUREMENT=en_US.UTF-8 and LC_IDENTIFICATION=C

attached base packages: parallel, stats, graphics, grDevices, utils, datasets, methods and base

other attached packages: Rerrrt(v.1.0), tidyr(v.1.0.2), dplyr(v.0.8.4), hpgltools(v.1.0), Biobase(v.2.46.0) and BiocGenerics(v.0.32.0)

loaded via a namespace (and not attached): tidyselect(v.1.0.0), lme4(v.1.1-21), RSQLite(v.2.2.0), AnnotationDbi(v.1.48.0), grid(v.3.6.1), BiocParallel(v.1.20.1), devtools(v.2.2.2), munsell(v.0.5.0), codetools(v.0.2-16), withr(v.2.1.2), colorspace(v.1.4-1), GOSemSim(v.2.12.0), highr(v.0.8), knitr(v.1.28), rstudioapi(v.0.11), stats4(v.3.6.1), DOSE(v.3.12.0), labeling(v.0.3), urltools(v.1.7.3), GenomeInfoDbData(v.1.2.2), polyclip(v.1.10-0), bit64(v.0.9-7), farver(v.2.0.3), rprojroot(v.1.3-2), vctrs(v.0.2.4), xfun(v.0.12), BiocFileCache(v.1.10.2), R6(v.2.4.1), doParallel(v.1.0.15), GenomeInfoDb(v.1.22.0), graphlayouts(v.0.5.0), locfit(v.1.5-9.1), bitops(v.1.0-6), fgsea(v.1.12.0), gridGraphics(v.0.4-1), DelayedArray(v.0.12.2), assertthat(v.0.2.1), scales(v.1.1.0), ggraph(v.2.0.1), enrichplot(v.1.6.1), gtable(v.0.3.0), sva(v.3.34.0), processx(v.3.4.2), tidygraph(v.1.1.2), rlang(v.0.4.5), genefilter(v.1.68.0), splines(v.3.6.1), rtracklayer(v.1.46.0), lazyeval(v.0.2.2), europepmc(v.0.3), BiocManager(v.1.30.10), yaml(v.2.2.1), reshape2(v.1.4.3), GenomicFeatures(v.1.38.2), backports(v.1.1.5), qvalue(v.2.18.0), clusterProfiler(v.3.14.3), tools(v.3.6.1), usethis(v.1.5.1), ggplotify(v.0.0.4), ggplot2(v.3.2.1), ellipsis(v.0.3.0), gplots(v.3.0.1.2), RColorBrewer(v.1.1-2), sessioninfo(v.1.1.1), ggridges(v.0.5.2), Rcpp(v.1.0.3), plyr(v.1.8.5), base64enc(v.0.1-3), progress(v.1.2.2), zlibbioc(v.1.32.0), purrr(v.0.3.3), RCurl(v.1.98-1.1), ps(v.1.3.2), prettyunits(v.1.1.1), openssl(v.1.4.1), viridis(v.0.5.1), cowplot(v.1.0.0), S4Vectors(v.0.24.3), SummarizedExperiment(v.1.16.1), ggrepel(v.0.8.1), colorRamps(v.2.3), fs(v.1.3.1), variancePartition(v.1.16.1), magrittr(v.1.5), data.table(v.1.12.8), DO.db(v.2.9), openxlsx(v.4.1.4), triebeard(v.0.3.0), matrixStats(v.0.55.0), pkgload(v.1.0.2), hms(v.0.5.3), evaluate(v.0.14), xtable(v.1.8-4), pbkrtest(v.0.4-7), XML(v.3.99-0.3), IRanges(v.2.20.2), gridExtra(v.2.3), testthat(v.2.3.1), compiler(v.3.6.1), biomaRt(v.2.42.0), tibble(v.2.1.3), KernSmooth(v.2.23-16), crayon(v.1.3.4), minqa(v.1.2.4), htmltools(v.0.4.0), mgcv(v.1.8-31), DBI(v.1.1.0), tweenr(v.1.0.1), dbplyr(v.1.4.2), MASS(v.7.3-51.5), rappdirs(v.0.3.1), boot(v.1.3-24), Matrix(v.1.2-18), readr(v.1.3.1), cli(v.2.0.1), gdata(v.2.18.0), igraph(v.1.2.4.2), GenomicRanges(v.1.38.0), pkgconfig(v.2.0.3), rvcheck(v.0.1.7), GenomicAlignments(v.1.22.1), xml2(v.1.2.2), foreach(v.1.4.8), annotate(v.1.64.0), XVector(v.0.26.0), stringr(v.1.4.0), callr(v.3.4.2), digest(v.0.6.25), Biostrings(v.2.54.0), rmarkdown(v.2.1), fastmatch(v.1.1-0), edgeR(v.3.28.0), curl(v.4.3), Rsamtools(v.2.2.2), gtools(v.3.8.1), nloptr(v.1.2.1), lifecycle(v.0.1.0), nlme(v.3.1-144), jsonlite(v.1.6.1), desc(v.1.2.0), viridisLite(v.0.3.0), askpass(v.1.1), limma(v.3.42.2), fansi(v.0.4.1), pillar(v.1.4.3), lattice(v.0.20-40), httr(v.1.4.1), pkgbuild(v.1.0.6), survival(v.3.1-8), GO.db(v.3.10.0), glue(v.1.3.1), remotes(v.2.1.1), zip(v.2.0.4), iterators(v.1.0.12), pander(v.0.6.3), bit(v.1.1-15.2), ggforce(v.0.3.1), stringi(v.1.4.6), blob(v.1.2.1), caTools(v.1.18.0) and memoise(v.1.1.0)

message(paste0("This is hpgltools commit: ", get_git_commit()))

## If you wish to reproduce this exact build of hpgltools, invoke the following:

## > git clone http://github.com/abelew/hpgltools.git

## > git reset 84603e9a66d4b380ca5e981f7dd93f6f0ad33bb9

## This is hpgltools commit: Thu Mar 12 12:16:38 2020 -0400: 84603e9a66d4b380ca5e981f7dd93f6f0ad33bb9

this_save <- paste0(gsub(pattern="\\.Rmd", replace="", x=rmd_file), "-v", ver, ".rda.xz")
message(paste0("Saving to ", this_save))

## Saving to error_quant-v20200314.rda.xz

tmp <- sm(saveme(filename=this_save))

loadme(filename=this_save)

LS0tCnRpdGxlOiAiQ291bnRpbmcgUlQgbXV0YXRpb25zIGZyb20gaWxsdW1pbmEgc2VxdWVuY2luZyBkYXRhLiIKYXV0aG9yOiAiYXRiIGFiZWxld0BnbWFpbC5jb20iCmRhdGU6ICJgciBTeXMuRGF0ZSgpYCIKb3V0cHV0OgogIGh0bWxfZG9jdW1lbnQ6CiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICBjb2RlX2ZvbGRpbmc6IHNob3cKICAgIGZpZ19jYXB0aW9uOiB0cnVlCiAgICBmaWdfaGVpZ2h0OiA3CiAgICBmaWdfd2lkdGg6IDcKICAgIGhpZ2hsaWdodDogdGFuZ28KICAgIGtlZXBfbWQ6IGZhbHNlCiAgICBtb2RlOiBzZWxmY29udGFpbmVkCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICAgIHNlbGZfY29udGFpbmVkOiB0cnVlCiAgICB0aGVtZTogcmVhZGFibGUKICAgIHRvYzogdHJ1ZQogICAgdG9jX2Zsb2F0OgogICAgICBjb2xsYXBzZWQ6IGZhbHNlCiAgICAgIHNtb290aF9zY3JvbGw6IGZhbHNlCiAgcm1kZm9ybWF0czo6cmVhZHRoZWRvd246CiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICBjb2RlX2ZvbGRpbmc6IHNob3cKICAgIGRmX3ByaW50OiBwYWdlZAogICAgZmlnX2NhcHRpb246IHRydWUKICAgIGZpZ19oZWlnaHQ6IDcKICAgIGZpZ193aWR0aDogNwogICAgaGlnaGxpZ2h0OiB0YW5nbwogICAgd2lkdGg6IDMwMAogICAga2VlcF9tZDogZmFsc2UKICAgIG1vZGU6IHNlbGZjb250YWluZWQKICAgIHRvY19mbG9hdDogdHJ1ZQogIEJpb2NTdHlsZTo6aHRtbF9kb2N1bWVudDoKICAgIGNvZGVfZG93bmxvYWQ6IHRydWUKICAgIGNvZGVfZm9sZGluZzogc2hvdwogICAgZmlnX2NhcHRpb246IHRydWUKICAgIGZpZ19oZWlnaHQ6IDcKICAgIGZpZ193aWR0aDogNwogICAgaGlnaGxpZ2h0OiB0YW5nbwogICAga2VlcF9tZDogZmFsc2UKICAgIG1vZGU6IHNlbGZjb250YWluZWQKICAgIHRvY19mbG9hdDogdHJ1ZQotLS0KCjxzdHlsZSB0eXBlPSJ0ZXh0L2NzcyI+CmJvZHksIHRkIHsKICBmb250LXNpemU6IDE2cHg7Cn0KY29kZS5yewogIGZvbnQtc2l6ZTogMTZweDsKfQpwcmUgewogZm9udC1zaXplOiAxNnB4Cn0KPC9zdHlsZT4KCmBgYHtyIG9wdGlvbnMsIGluY2x1ZGU9RkFMU0V9CmxpYnJhcnkoImhwZ2x0b29scyIpCnR0IDwtIGRldnRvb2xzOjpsb2FkX2FsbCgiL2RhdGEvaHBnbHRvb2xzIikKa25pdHI6Om9wdHNfa25pdCRzZXQod2lkdGg9MTIwLAogICAgICAgICAgICAgICAgICAgICBwcm9ncmVzcz1UUlVFLAogICAgICAgICAgICAgICAgICAgICB2ZXJib3NlPVRSVUUsCiAgICAgICAgICAgICAgICAgICAgIGVjaG89VFJVRSkKa25pdHI6Om9wdHNfY2h1bmskc2V0KGVycm9yPVRSVUUsCiAgICAgICAgICAgICAgICAgICAgICBkcGk9OTYpCm9sZF9vcHRpb25zIDwtIG9wdGlvbnMoZGlnaXRzPTQsCiAgICAgICAgICAgICAgICAgICAgICAgc3RyaW5nc0FzRmFjdG9ycz1GQUxTRSwKICAgICAgICAgICAgICAgICAgICAgICBrbml0ci5kdXBsaWNhdGUubGFiZWw9ImFsbG93IikKZ2dwbG90Mjo6dGhlbWVfc2V0KGdncGxvdDI6OnRoZW1lX2J3KGJhc2Vfc2l6ZT0xMCkpCnJ1bmRhdGUgPC0gZm9ybWF0KFN5cy5EYXRlKCksIGZvcm1hdD0iJVklbSVkIikKcHJldmlvdXNfZmlsZSA8LSAiaW5kZXguUm1kIgp2ZXIgPC0gIjIwMjAwMzE0IgoKIyN0bXAgPC0gc20obG9hZG1lKGZpbGVuYW1lPXBhc3RlMChnc3ViKHBhdHRlcm49IlxcLlJtZCIsIHJlcGxhY2U9IiIsIHg9cHJldmlvdXNfZmlsZSksICItdiIsIHZlciwgIi5yZGEueHoiKSkpCnJtZF9maWxlIDwtICJlcnJvcl9xdWFudC5SbWQiCmBgYAoKIyBDYWxjdWxhdGluZyBlcnJvciByYXRlcy4KCkkgd3JvdGUgdGhlIGZ1bmN0aW9uICdjcmVhdGVfbWF0cmljZXMoKScgdG8gY29sbGVjdCBtdXRhdGlvbiBjb3VudHMuICBBdCBsZWFzdAppbiB0aGVvcnkgdGhlIHJlc3VsdHMgZnJvbSBpdCBzaG91bGQgYmUgYWJsZSB0byBhZGRyZXNzIG1vc3QvYW55IHF1ZXN0aW9uCnJlZ2FyZGluZyB0aGUgY291bnRzIG9mIG11dGF0aW9ucyBvYnNlcnZlZCBpbiB0aGUgZGF0YS4KCiMjIENhdGVnb3JpemUgdGhlIGRhdGEgd2l0aCBhdCBsZWFzdCAzIGluZGV4ZXMgcGVyIG11dGFudAoKYGBge3IgdHJpcGxlc30KZGV2dG9vbHM6OmxvYWRfYWxsKCJSZXJycnQiKQpzYW1wbGVfc2hlZXQgPC0gInNhbXBsZV9zaGVldHMvYWxsX3NhbXBsZXMueGxzeCIKaWRlbnRfY29sdW1uIDwtICJpZGVudHRhYmxlIgptdXRfY29sdW1uIDwtICJtdXRhdGlvbnRhYmxlIgptaW5fcmVhZHMgPC0gMwptaW5faW5kZXhlcyA8LSAzCm1pbl9zZXF1ZW5jZXIgPC0gMTAKbWluX3Bvc2l0aW9uIDwtIDI0Cm1heF9wb3NpdGlvbiA8LSAxNzYKbWF4X211dGF0aW9uc19wZXJfcmVhZCA8LSBOVUxMCnBydW5lX24gPC0gVFJVRQp2ZXJib3NlIDwtIFRSVUUKZXhjZWwgPC0gZ2x1ZTo6Z2x1ZSgiZXhjZWwve3J1bmRhdGV9X3RyaXBsZXMtdnt2ZXJ9Lnhsc3giKQp0cmlwbGVzIDwtIGNyZWF0ZV9tYXRyaWNlcyhzYW1wbGVfc2hlZXQ9c2FtcGxlX3NoZWV0LAogICAgICAgICAgICAgICAgICAgICAgICAgICBpZGVudF9jb2x1bW49aWRlbnRfY29sdW1uLCBtdXRfY29sdW1uPW11dF9jb2x1bW4sCiAgICAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9yZWFkcz1taW5fcmVhZHMsIG1pbl9pbmRleGVzPW1pbl9pbmRleGVzLAogICAgICAgICAgICAgICAgICAgICAgICAgICBtaW5fc2VxdWVuY2VyPW1pbl9zZXF1ZW5jZXIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9wb3NpdGlvbj1taW5fcG9zaXRpb24sIG1heF9wb3NpdGlvbj1tYXhfcG9zaXRpb24sCiAgICAgICAgICAgICAgICAgICAgICAgICAgIHBydW5lX249cHJ1bmVfbiwgdmVyYm9zZT12ZXJib3NlLCBleGNlbD1leGNlbCkKCm1heF9tdXRhdGlvbnNfcGVyX3JlYWQgPC0gMTAKZXhjZWwgPC0gZ2x1ZTo6Z2x1ZSgiZXhjZWwve3J1bmRhdGV9X3RyaXBsZXNfdGVubXByLXZ7dmVyfS54bHN4IikKdHJpcGxlc190ZW5tcHIgPC0gY3JlYXRlX21hdHJpY2VzKHNhbXBsZV9zaGVldD1zYW1wbGVfc2hlZXQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpZGVudF9jb2x1bW49aWRlbnRfY29sdW1uLCBtdXRfY29sdW1uPW11dF9jb2x1bW4sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtaW5fcmVhZHM9bWluX3JlYWRzLCBtaW5faW5kZXhlcz1taW5faW5kZXhlcywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9zZXF1ZW5jZXI9bWluX3NlcXVlbmNlciwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9wb3NpdGlvbj1taW5fcG9zaXRpb24sIG1heF9wb3NpdGlvbj1tYXhfcG9zaXRpb24sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcnVuZV9uPXBydW5lX24sIHZlcmJvc2U9dmVyYm9zZSwgZXhjZWw9ZXhjZWwpCm1heF9tdXRhdGlvbnNfcGVyX3JlYWQgPC0gNQpleGNlbCA8LSBnbHVlOjpnbHVlKCJleGNlbC97cnVuZGF0ZX1fdHJpcGxlc19maXZlbXByLXZ7dmVyfS54bHN4IikKdHJpcGxlc19maXZlbXByIDwtIGNyZWF0ZV9tYXRyaWNlcyhzYW1wbGVfc2hlZXQ9c2FtcGxlX3NoZWV0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGlkZW50X2NvbHVtbj1pZGVudF9jb2x1bW4sIG11dF9jb2x1bW49bXV0X2NvbHVtbiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtaW5fcmVhZHM9bWluX3JlYWRzLCBtaW5faW5kZXhlcz1taW5faW5kZXhlcywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtaW5fc2VxdWVuY2VyPW1pbl9zZXF1ZW5jZXIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbWluX3Bvc2l0aW9uPW1pbl9wb3NpdGlvbiwgbWF4X3Bvc2l0aW9uPW1heF9wb3NpdGlvbiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcnVuZV9uPXBydW5lX24sIHZlcmJvc2U9dmVyYm9zZSwgZXhjZWw9ZXhjZWwpCmBgYAoKIyMgQ2F0ZWdvcml6ZSB0aGUgZGF0YSB3aXRoIGF0IGxlYXN0IDUgaW5kZXhlcyBwZXIgbXV0YW50CgpgYGB7ciBxdWludHN9Cm1pbl9pbmRleGVzIDwtIDUKbWF4X211dGF0aW9uc19wZXJfcmVhZCA8LSBOVUxMCmV4Y2VsIDwtIGdsdWU6OmdsdWUoImV4Y2VsL3tydW5kYXRlfV9xdWludHMtdnt2ZXJ9Lnhsc3giKQpxdWludHMgPC0gY3JlYXRlX21hdHJpY2VzKHNhbXBsZV9zaGVldD1zYW1wbGVfc2hlZXQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgaWRlbnRfY29sdW1uPWlkZW50X2NvbHVtbiwgbXV0X2NvbHVtbj1tdXRfY29sdW1uLAogICAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9yZWFkcz1taW5fcmVhZHMsIG1pbl9pbmRleGVzPW1pbl9pbmRleGVzLAogICAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9zZXF1ZW5jZXI9bWluX3NlcXVlbmNlciwKICAgICAgICAgICAgICAgICAgICAgICAgICBtaW5fcG9zaXRpb249bWluX3Bvc2l0aW9uLCBtYXhfcG9zaXRpb249bWF4X3Bvc2l0aW9uLAogICAgICAgICAgICAgICAgICAgICAgICAgIHBydW5lX249cHJ1bmVfbiwgdmVyYm9zZT12ZXJib3NlLCBleGNlbD1leGNlbCkKbWF4X211dGF0aW9uc19wZXJfcmVhZCA8LSAxMApleGNlbCA8LSBnbHVlOjpnbHVlKCJleGNlbC97cnVuZGF0ZX1fcXVpbnRzX3Rlbm1wci12e3Zlcn0ueGxzeCIpCnF1aW50c190ZW5tcHIgPC0gY3JlYXRlX21hdHJpY2VzKHNhbXBsZV9zaGVldD1zYW1wbGVfc2hlZXQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGlkZW50X2NvbHVtbj1pZGVudF9jb2x1bW4sIG11dF9jb2x1bW49bXV0X2NvbHVtbiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbWluX3JlYWRzPW1pbl9yZWFkcywgbWluX2luZGV4ZXM9bWluX2luZGV4ZXMsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9zZXF1ZW5jZXI9bWluX3NlcXVlbmNlciwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbWluX3Bvc2l0aW9uPW1pbl9wb3NpdGlvbiwgbWF4X3Bvc2l0aW9uPW1heF9wb3NpdGlvbiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcHJ1bmVfbj1wcnVuZV9uLCB2ZXJib3NlPXZlcmJvc2UsIGV4Y2VsPWV4Y2VsKQptYXhfbXV0YXRpb25zX3Blcl9yZWFkIDwtIDUKZXhjZWwgPC0gZ2x1ZTo6Z2x1ZSgiZXhjZWwve3J1bmRhdGV9X3F1aW50c19maXZlbXByLXZ7dmVyfS54bHN4IikKcXVpbnRzX2ZpdmVtcHIgPC0gY3JlYXRlX21hdHJpY2VzKHNhbXBsZV9zaGVldD1zYW1wbGVfc2hlZXQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpZGVudF9jb2x1bW49aWRlbnRfY29sdW1uLCBtdXRfY29sdW1uPW11dF9jb2x1bW4sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtaW5fcmVhZHM9bWluX3JlYWRzLCBtaW5faW5kZXhlcz1taW5faW5kZXhlcywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9zZXF1ZW5jZXI9bWluX3NlcXVlbmNlciwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9wb3NpdGlvbj1taW5fcG9zaXRpb24sIG1heF9wb3NpdGlvbj1tYXhfcG9zaXRpb24sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcnVuZV9uPXBydW5lX24sIHZlcmJvc2U9dmVyYm9zZSwgZXhjZWw9ZXhjZWwpCmBgYAoKIyBRdWVzdGlvbnMgZnJvbSBEci4gRGVTdGVmYW5vCgpJIHRoaW5rIHdoYXQgaXMgYmVzdCBpcyB0byBnZXQgdGhlIG51bWJlciBvZiByZWNvdmVyZWQgbXV0YXRpb25zIG9mIGVhY2ggdHlwZQpmcm9tIGVhY2ggZGF0YSBzZXQuICBUaGF0IHdvdWxkIGJlIEEgdG8gVCwgQSB0byBHLCBBIHRvIEM7IFQgdG8gQSwgVCB0byBHLCBUIHRvCkM7IEcgdG8gQSwgRyB0byBDLCBHIHRvIFQ7IGFuZCBDIHRvIEEsIEMgdG8gRywgQyB0byBUOyBhcyB3ZWxsIGFzIGRlbGV0aW9ucyBhbmQKaW5zZXJ0aW9ucy4gIEkgd291bGQgdGhlbiBuZWVkIHRoZSBzdW0gbnVtYmVyIG9mIHRoZSByZWFkcyB0aGF0IG1ldCBhbGwgb3VyCmNyaXRlcmlhIChpLmUuIGF0IGxlYXN0IDMgZ29vZCByZWNvdmVyZWQgcmVhZHMgZm9yIHRoYXQgMTQgbnQgaW5kZXgpLiAgRWFjaCBzZXQKb2YgMyBvciBtb3JlIHdvdWxkIGN0IGFzICIxIiByZWFkIG9mIHRoYXQgcGFydGljdWxhciBpbmRleCBzbyBJIHdvdWxkIG5lZWQgdGhlCnRvdGFsIHdpdGggdGhpcyBpbiBtaW5kLiAgSSBhbHNvIG5lZWQgdG8ga25vdyB0aGUgdG90YWwgbnVtYmVyIG9mIG51Y2xlb3RpZGVzCnRoYXQgd2VyZSBpbiB0aGUgcmVnaW9uIHdlIGRlY2lkZWQgdG8gY29uc2lkZXIgaW4gdGhlIGFuYWx5c2lzLiAgV2UgbWF5IHdhbnQgdG8KdHJ5IHRoaXMgZm9yIDMgb3IgbW9yZSBhbmQgNSBvciBtb3JlIHJlY292ZXJlZCBpbmRleGVzIGlmIGl0IGlzIG5vdCBoYXJkLiAgVGhpcwppbmZvcm1hdGlvbiBkb2VzIG5vdCBpbmNsdWRlIHNwZWNpZmljIHBvc2l0aW9ucyBvbiB0aGUgdGVtcGxhdGUgd2hlcmUgZXJyb3JzCm9jY3VycmVkIGJ1dCB3ZSBjYW4gbG9vayBhdCB0aGF0IGxhdHRlci4gIFJpZ2h0IG5vdyBJIGp1c3Qgd2FudCB0byBnZXQgYSBnZW5lcmFsCmVycm9yIHJhdGUgYW5kIHR5cGUgb2YgZXJyb3IuICBJdCB3b3VsZCBiYXNpY2FsbHkgYmUgY2FsY3VsYXRlZCBieSBkaXZpZGluZyB0aGUKbnVtYmVyIG9mIHJlY292ZXJlZCBtdXRhdGlvbnMgb2YgYSBwYXJ0aWN1bGFyIHR5cGUgYnkgc3VtIG51bWJlciBvZiB0aGUgcmVhZHMKdGltZXMgdGhlIG51bWJlciBvZiBudWNsZW90aWRlcyBzY3JlZW5lZCBpbiB0aGUgdGVtcGxhdGUuICBBcyBpdCBlbmRzIHVwLCB0aGlzCm51bWJlciBkb2VzIG5vdCByZWFsbHkgaGF2ZSBhIGxvdCBvZiBtZWFuaW5nIGJ1dCBpdCBjYW4gYmUgdXNlZCB0byBjYWxjdWxhdGUgdGhlCm92ZXJhbGwgbXV0YXRpb24gcmF0ZSBhcyB3ZWxsIGFzIHRoZSByYXRlIGZvciB0cmFuc3ZlcnNpb25zLCB0cmFuc2l0aW9ucywgYW5kCmRlbGV0aW9ucyBhbmQgaW5zZXJ0aW9ucy4KCiMgQW5zd2VycwoKSW4gb3JkZXIgdG8gYWRkcmVzcyB0aG9zZSBxdWVyaWVzLCBJIGludm9rZWQgY3JlYXRlX21hdHJpY2VzKCkgd2l0aCBhIG1pbmltdW0KaW5kZXggY291bnQgb2YgMyBhbmQgNS4gIEl0IHNob3VsZCBiZSBub3RlZCB0aGF0IHRoaXMgaXMgbm90IHRoZSBzYW1lIGFzCnJlcXVpcmluZyAzIG9yIDUgcmVhZHMgcGVyIGluZGV4LiAgSW4gYm90aCBjYXNlcyBJIHJlcXVpcmUgMyByZWFkcyBwZXIgaW5kZXguCgojIyBSZWNvdmVyZWQgbXV0YXRpb25zIG9mIGVhY2ggdHlwZQoKSSBhbSBpbnRlcnByZXRpbmcgdGhpcyBxdWVzdGlvbiBhcyB0aGUgbnVtYmVyIG9mIGluZGV4ZXMgcmVjb3ZlcmVkIGZvciBlYWNoCm11dGF0aW9uIHR5cGUuICBJIGNvbGxlY3QgdGhpcyBpbmZvcm1hdGlvbiBpbiAyIHdheXMgb2YgaW50ZXJlc3Q6IHRoZSBpbmRleGVzIGJ5CnR5cGUgd2hpY2ggYXJlIGRlZW1lZCB0byBiZSBmcm9tIHRoZSBSVCBhbmQgZnJvbSB0aGUgc2VxdWVuY2VyLiAgSW4gYWRkaXRpb24sIEkKY2FsY3VsYXRlIGEgbm9ybWFsaXplZCAoY3BtKSB2ZXJzaW9uIG9mIHRoaXMgaW5mb3JtYXRpb24gd2hpY2ggbWF5IGJlIHVzZWQgdG8gbG9vayBmb3IKY2hhbmdlcyBhY3Jvc3Mgc2FtcGxlcy4KCiMjIyBNdXRhdGlvbnMgYnkgUlQgaW5kZXgKClRoaXMgZm9sbG93aW5nIGJsb2NrIHNob3VsZCBwcmludCBvdXQgdGFibGVzIG9mIHRoZSBudW1iZXJzIG9mIG11dGFudCBpbmRleGVzCm9ic2VydmVkIGZvciBlYWNoIHR5cGUgZm9yIHRoZSBSVCBhbmQgdGhlIHNlcXVlbmNlci4gIE9uZSB3b3VsZCBob3BlIHRoYXQgdGhlCnNlcXVlbmNlciB3aWxsIGJlIGNvbnNpc3RlbnQgZm9yIGFsbCBzYW1wbGVzLCBidXQgSSB0aGluayB0aGUgcmVzdWx0cyB3aWxsCmluc3RlYWQgc3VnZ2VzdCB0aGF0IG15IG1ldHJpYyBpcyBub3QgeWV0IHN0cmluZ2VudCBlbm91Z2guCgpgYGB7ciBtdXRhdGlvbl9pbmRleF9jb3VudCwgcmVzdWx0cz0nYXNpcyd9CmtuaXRyOjprYWJsZSh0cmlwbGVzW1sibWF0cmljZXMiXV1bWyJtaXNzX2luZGV4ZXNfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHRyaXBsZXNfdGVubXByW1sibWF0cmljZXMiXV1bWyJtaXNzX2luZGV4ZXNfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHRyaXBsZXNfZml2ZW1wcltbIm1hdHJpY2VzIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZShxdWludHNbWyJtYXRyaWNlcyJdXVtbIm1pc3NfaW5kZXhlc19ieV90eXBlIl1dKQprbml0cjo6a2FibGUocXVpbnRzX3Rlbm1wcltbIm1hdHJpY2VzIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZShxdWludHNfZml2ZW1wcltbIm1hdHJpY2VzIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCgprbml0cjo6a2FibGUodHJpcGxlc1tbIm1hdHJpY2VzIl1dW1sibWlzc19zZXF1ZW5jZXJfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHRyaXBsZXNfdGVubXByW1sibWF0cmljZXMiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQprbml0cjo6a2FibGUodHJpcGxlc19maXZlbXByW1sibWF0cmljZXMiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQprbml0cjo6a2FibGUocXVpbnRzW1sibWF0cmljZXMiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQprbml0cjo6a2FibGUocXVpbnRzX3Rlbm1wcltbIm1hdHJpY2VzIl1dW1sibWlzc19zZXF1ZW5jZXJfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHF1aW50c19maXZlbXByW1sibWF0cmljZXMiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQpgYGAKClBsb3RzIG9mIHRoaXMgaW5mb3JtYXRpb24KCmBgYHtyIG11dGF0aW9uX2luZGV4X2NvdW50X3Bsb3RzfQp0cmlwbGVzW1sicGxvdHMiXV1bWyJjb3VudHMiXV1bWyJtaXNzX2luZGV4ZXNfYnlfdHlwZSJdXQp0cmlwbGVzX3Rlbm1wcltbInBsb3RzIl1dW1siY291bnRzIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0KdHJpcGxlc19maXZlbXByW1sicGxvdHMiXV1bWyJjb3VudHMiXV1bWyJtaXNzX2luZGV4ZXNfYnlfdHlwZSJdXQoKcXVpbnRzW1sicGxvdHMiXV1bWyJjb3VudHMiXV1bWyJtaXNzX2luZGV4ZXNfYnlfdHlwZSJdXQpxdWludHNfdGVubXByW1sicGxvdHMiXV1bWyJjb3VudHMiXV1bWyJtaXNzX2luZGV4ZXNfYnlfdHlwZSJdXQpxdWludHNfZml2ZW1wcltbInBsb3RzIl1dW1siY291bnRzIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0KYGBgCgpUaGlzIHN1Z2dlc3RzIHRvIG1lIHRoYXQgdGhpcyBpbmZvcm1hdGlvbiBuZWVkcyB0byBiZSBub3JtYWxpemVkIGluIHNvbWUgbW9yZQpzZW5zaWJsZSBmYXNoaW9uLiAgVGh1cyB0aGUgZm9sbG93aW5nOgoKIyMjIE11dGF0aW9ucyBieSBSVCBpbmRleCwgcG9zdCBub3JtYWxpemF0aW9uCgpUaGUgc2FtZSBudW1iZXJzIG1heSBiZSBleHByZXNzZWQgaW4gdGhlIGNvbnRleHQgb2YgdGhlIG51bWJlciBvZiBpbmRleGVzCm9ic2VydmVkIC8gc2FtcGxlIGFuZC9vciBhcyBhIGNwbSBhY3Jvc3Mgc2FtcGxlcy4gIFRodXMgaW4gdGhlIGZpcnN0IGluc3RhbmNlCm9uZSBjYW4gbG9vayBhdCB0aGUgYXBwYXJlbnQgZXJyb3IgcmF0ZSBmb3IgZWFjaCBzYW1wbGUsIGFuZCBpbiB0aGUgc2Vjb25kCmluc3RhbmNlIG9uZSBtYXkgbG9vayBmb3IgcmVsYXRpdmUgY2hhbmdlcyBpbiBhcHBhcmVudCBlcnJvciByYXRlIGFjcm9zcwpzYW1wbGVzLgoKIyMjIyBSZXdyaXRpbmcgdGhlIG1hdHJpY2VzIGFzIGNwbSB0byBhY2NvdW50IGZvciBsaWJyYXJ5IHNpemVzLgoKYGBge3IgbXV0YXRpb25faW5kZXhfbm9ybWFsaXplZCwgcmVzdWx0cz0nYXNpcyd9CmtuaXRyOjprYWJsZSh0cmlwbGVzW1sibm9ybWFsaXplZCJdXVtbIm1pc3NfaW5kZXhlc19ieV90eXBlIl1dKQprbml0cjo6a2FibGUodHJpcGxlc190ZW5tcHJbWyJub3JtYWxpemVkIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZSh0cmlwbGVzX2ZpdmVtcHJbWyJub3JtYWxpemVkIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZShxdWludHNbWyJub3JtYWxpemVkIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZShxdWludHNfdGVubXByW1sibm9ybWFsaXplZCJdXVtbIm1pc3NfaW5kZXhlc19ieV90eXBlIl1dKQprbml0cjo6a2FibGUocXVpbnRzX2ZpdmVtcHJbWyJub3JtYWxpemVkIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCgprbml0cjo6a2FibGUodHJpcGxlc1tbIm5vcm1hbGl6ZWQiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQprbml0cjo6a2FibGUodHJpcGxlc190ZW5tcHJbWyJub3JtYWxpemVkIl1dW1sibWlzc19zZXF1ZW5jZXJfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHRyaXBsZXNfZml2ZW1wcltbIm5vcm1hbGl6ZWQiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQprbml0cjo6a2FibGUocXVpbnRzW1sibm9ybWFsaXplZCJdXVtbIm1pc3Nfc2VxdWVuY2VyX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZShxdWludHNfdGVubXByW1sibm9ybWFsaXplZCJdXVtbIm1pc3Nfc2VxdWVuY2VyX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZShxdWludHNfZml2ZW1wcltbIm5vcm1hbGl6ZWQiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQpgYGAKCiMjIyMgUmV3cml0aW5nIHRoZSBtYXRyaWNlcyBieSBkaXZpZGluZyBieSBhbGwgaW5kZXhlcwoKVGhpcyBJIHRoaW5rIHN0YXJ0cyB0byBhZGRyZXNzIHRoZSBsYXRlciB0ZXh0IGluIHlvdXIgcXVlcnkuCgpgYGB7ciBtdXRhdGlvbl9pbmRleF9ub3JtYWxpemVkX2J5X2NvdW50cywgcmVzdWx0cz0nYXNpcyd9CmtuaXRyOjprYWJsZSh0cmlwbGVzW1sibWF0cmljZXNfYnlfY291bnRzIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZShxdWludHNbWyJtYXRyaWNlc19ieV9jb3VudHMiXV1bWyJtaXNzX2luZGV4ZXNfYnlfdHlwZSJdXSkKCmtuaXRyOjprYWJsZSh0cmlwbGVzW1sibWF0cmljZXNfYnlfY291bnRzIl1dW1sibWlzc19zZXF1ZW5jZXJfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHF1aW50c1tbIm1hdHJpY2VzX2J5X2NvdW50cyJdXVtbIm1pc3Nfc2VxdWVuY2VyX2J5X3R5cGUiXV0pCmBgYAoKIyMjIyBSZXdyaXRpbmcgdGhlIG1hdHJpY2VzIGJ5IGRpdmlkaW5nIGJ5IGFsbCBpbmRleGVzIGFuZCBjcG0KCkkgdGhpbmsgdGhpcyBtaWdodCBwcm92ZSB0byBiZSB3aGVyZSB3ZSBnZXQgdGhlIG1vc3QgbWVhbmluZ2Z1bCByZXN1bHRzLgoKVGhlIG5pY2VzdCB0aGluZyBpbiBpdCBpcyB0aGF0IGFmdGVyIGFjY291bnRpbmcgZm9yIGxpYnJhcnkgc2l6ZXMgYW5kIHRvdGFsCmluZGV4ZXMgb2JzZXJ2ZWQsIHdlIGZpbmFsbHkgc2VlIHRoYXQgdGhlIHNlcXVlbmNlciBlcnJvciBpcyBtb3N0bHkgY29uc2lzdGVudAphY3Jvc3MgYWxsIHNhbXBsZXMgYW5kIG11dGF0aW9uIHR5cGVzIC0tIHdpdGggYSBjb3VwbGUgb2Ygbm90YWJsZSBleGNlcHRpb25zLgoKQnkgdGhlIHNhbWUgdG9rZW4sIGZvciB0aGUgbXV0YXRpb25zIHdoaWNoIF9hcmVfIGlkZW50aWNhbCBmb3IgdGhlIHNlcXVlbmNlciwgd2UKaGF2ZSBzb21lIHdoaWNoIGFyZSBkZWNpZGVkbHkgZGlmZmVyZW50IGZvciB0aGUgbm9uLXNlcXVlbmNlciBkYXRhLiAgVGhlIG1vc3QKbm90YWJsZSBleGFtcGxlcyBJIHRoaW5rIGFyZSBBIHRvIEcgYnV0IF9ub3QgRyB0byBBOyBhbmQgQyB0byBULgoKYGBge3IgbXV0YXRpb25faW5kZXhfY3BtX2J5X2NvdW50cywgcmVzdWx0cz0nYXNpcyd9CmtuaXRyOjprYWJsZSh0cmlwbGVzW1sibm9ybWFsaXplZF9ieV9jb3VudHMiXV1bWyJtaXNzX2luZGV4ZXNfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHRyaXBsZXNfdGVubXByW1sibm9ybWFsaXplZF9ieV9jb3VudHMiXV1bWyJtaXNzX2luZGV4ZXNfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHRyaXBsZXNfZml2ZW1wcltbIm5vcm1hbGl6ZWRfYnlfY291bnRzIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZShxdWludHNbWyJub3JtYWxpemVkX2J5X2NvdW50cyJdXVtbIm1pc3NfaW5kZXhlc19ieV90eXBlIl1dKQprbml0cjo6a2FibGUocXVpbnRzX3Rlbm1wcltbIm5vcm1hbGl6ZWRfYnlfY291bnRzIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCmtuaXRyOjprYWJsZShxdWludHNfZml2ZW1wcltbIm5vcm1hbGl6ZWRfYnlfY291bnRzIl1dW1sibWlzc19pbmRleGVzX2J5X3R5cGUiXV0pCgprbml0cjo6a2FibGUodHJpcGxlc1tbIm5vcm1hbGl6ZWRfYnlfY291bnRzIl1dW1sibWlzc19zZXF1ZW5jZXJfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHRyaXBsZXNfdGVubXByW1sibm9ybWFsaXplZF9ieV9jb3VudHMiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQprbml0cjo6a2FibGUodHJpcGxlc19maXZlbXByW1sibm9ybWFsaXplZF9ieV9jb3VudHMiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQprbml0cjo6a2FibGUocXVpbnRzW1sibm9ybWFsaXplZF9ieV9jb3VudHMiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQprbml0cjo6a2FibGUocXVpbnRzX3Rlbm1wcltbIm5vcm1hbGl6ZWRfYnlfY291bnRzIl1dW1sibWlzc19zZXF1ZW5jZXJfYnlfdHlwZSJdXSkKa25pdHI6OmthYmxlKHF1aW50c19maXZlbXByW1sibm9ybWFsaXplZF9ieV9jb3VudHMiXV1bWyJtaXNzX3NlcXVlbmNlcl9ieV90eXBlIl1dKQpgYGAKCiMjIyBJbmRlbHMgYnkgUlQgaW5kZXgKClRoZSBmb2xsb3dpbmcgYmxvY2tzIHdpbGwgcmVwZWF0IHRoZSBhYm92ZSwgYnV0IGxvb2tpbmcgZm9yIGluc2VydGlvbnMuClRoaXMgZGF0YSBkb2VzIG5vdCBvYnNlcnZlIHN1ZmZpY2llbnQgZGVsZXRpb25zIHRvIG1ha2UgYSBwcm9wZXIgY291bnQgZm9yIHRoZW0uCgpgYGB7ciBpbnNlcnRfaW5kZXhfY291bnQsIHJlc3VsdHM9J2FzaXMnfQprbml0cjo6a2FibGUodHJpcGxlc1tbIm1hdHJpY2VzIl1dW1siaW5zZXJ0X2luZGV4ZXNfYnlfbnQiXV0pCmtuaXRyOjprYWJsZSh0cmlwbGVzX3Rlbm1wcltbIm1hdHJpY2VzIl1dW1siaW5zZXJ0X2luZGV4ZXNfYnlfbnQiXV0pCmtuaXRyOjprYWJsZSh0cmlwbGVzX2ZpdmVtcHJbWyJtYXRyaWNlcyJdXVtbImluc2VydF9pbmRleGVzX2J5X250Il1dKQprbml0cjo6a2FibGUocXVpbnRzW1sibWF0cmljZXMiXV1bWyJpbnNlcnRfaW5kZXhlc19ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c190ZW5tcHJbWyJtYXRyaWNlcyJdXVtbImluc2VydF9pbmRleGVzX2J5X250Il1dKQprbml0cjo6a2FibGUocXVpbnRzX2ZpdmVtcHJbWyJtYXRyaWNlcyJdXVtbImluc2VydF9pbmRleGVzX2J5X250Il1dKQoKa25pdHI6OmthYmxlKHRyaXBsZXNbWyJtYXRyaWNlcyJdXVtbImluc2VydF9zZXF1ZW5jZXJfYnlfbnQiXV0pCmtuaXRyOjprYWJsZSh0cmlwbGVzX3Rlbm1wcltbIm1hdHJpY2VzIl1dW1siaW5zZXJ0X3NlcXVlbmNlcl9ieV9udCJdXSkKa25pdHI6OmthYmxlKHRyaXBsZXNfZml2ZW1wcltbIm1hdHJpY2VzIl1dW1siaW5zZXJ0X3NlcXVlbmNlcl9ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c1tbIm1hdHJpY2VzIl1dW1siaW5zZXJ0X3NlcXVlbmNlcl9ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c190ZW5tcHJbWyJtYXRyaWNlcyJdXVtbImluc2VydF9zZXF1ZW5jZXJfYnlfbnQiXV0pCmtuaXRyOjprYWJsZShxdWludHNfZml2ZW1wcltbIm1hdHJpY2VzIl1dW1siaW5zZXJ0X3NlcXVlbmNlcl9ieV9udCJdXSkKYGBgCgpQbG90cyBvZiB0aGlzIGluZm9ybWF0aW9uCgpgYGB7ciBpbnNlcnRfaW5kZXhfY291bnRfcGxvdHN9CnRyaXBsZXNbWyJwbG90cyJdXVtbImNvdW50cyJdXVtbImluc2VydF9pbmRleGVzX2J5X250Il1dCnRyaXBsZXNfdGVubXByW1sicGxvdHMiXV1bWyJjb3VudHMiXV1bWyJpbnNlcnRfaW5kZXhlc19ieV9udCJdXQp0cmlwbGVzX2ZpdmVtcHJbWyJwbG90cyJdXVtbImNvdW50cyJdXVtbImluc2VydF9pbmRleGVzX2J5X250Il1dCgpxdWludHNbWyJwbG90cyJdXVtbImNvdW50cyJdXVtbImluc2VydF9pbmRleGVzX2J5X250Il1dCnF1aW50c190ZW5tcHJbWyJwbG90cyJdXVtbImNvdW50cyJdXVtbImluc2VydF9pbmRleGVzX2J5X250Il1dCnF1aW50c19maXZlbXByW1sicGxvdHMiXV1bWyJjb3VudHMiXV1bWyJpbnNlcnRfaW5kZXhlc19ieV9udCJdXQpgYGAKCiMjIyBJbnNlcnRpb25zIGJ5IFJUIGluZGV4LCBwb3N0IG5vcm1hbGl6YXRpb24KCiMjIyMgUmV3cml0aW5nIHRoZSBtYXRyaWNlcyBhcyBjcG0gdG8gYWNjb3VudCBmb3IgbGlicmFyeSBzaXplcy4KCmBgYHtyIGluc2VydF9pbmRleF9ub3JtYWxpemVkLCByZXN1bHRzPSdhc2lzJ30Ka25pdHI6OmthYmxlKHRyaXBsZXNbWyJub3JtYWxpemVkIl1dW1siaW5zZXJ0X2luZGV4ZXNfYnlfbnQiXV0pCmtuaXRyOjprYWJsZSh0cmlwbGVzX3Rlbm1wcltbIm5vcm1hbGl6ZWQiXV1bWyJpbnNlcnRfaW5kZXhlc19ieV9udCJdXSkKa25pdHI6OmthYmxlKHRyaXBsZXNfZml2ZW1wcltbIm5vcm1hbGl6ZWQiXV1bWyJpbnNlcnRfaW5kZXhlc19ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c1tbIm5vcm1hbGl6ZWQiXV1bWyJpbnNlcnRfaW5kZXhlc19ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c190ZW5tcHJbWyJub3JtYWxpemVkIl1dW1siaW5zZXJ0X2luZGV4ZXNfYnlfbnQiXV0pCmtuaXRyOjprYWJsZShxdWludHNfZml2ZW1wcltbIm5vcm1hbGl6ZWQiXV1bWyJpbnNlcnRfaW5kZXhlc19ieV9udCJdXSkKCmtuaXRyOjprYWJsZSh0cmlwbGVzW1sibm9ybWFsaXplZCJdXVtbImluc2VydF9zZXF1ZW5jZXJfYnlfbnQiXV0pCmtuaXRyOjprYWJsZSh0cmlwbGVzX3Rlbm1wcltbIm5vcm1hbGl6ZWQiXV1bWyJpbnNlcnRfc2VxdWVuY2VyX2J5X250Il1dKQprbml0cjo6a2FibGUodHJpcGxlc19maXZlbXByW1sibm9ybWFsaXplZCJdXVtbImluc2VydF9zZXF1ZW5jZXJfYnlfbnQiXV0pCmtuaXRyOjprYWJsZShxdWludHNbWyJub3JtYWxpemVkIl1dW1siaW5zZXJ0X3NlcXVlbmNlcl9ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c190ZW5tcHJbWyJub3JtYWxpemVkIl1dW1siaW5zZXJ0X3NlcXVlbmNlcl9ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c19maXZlbXByW1sibm9ybWFsaXplZCJdXVtbImluc2VydF9zZXF1ZW5jZXJfYnlfbnQiXV0pCmBgYAoKIyMjIyBSZXdyaXRpbmcgdGhlIG1hdHJpY2VzIGJ5IGRpdmlkaW5nIGJ5IGFsbCBpbmRleGVzCgpJIHRoaW5rIHRoYXQgdGhlcmUgYXJlIGZldyBlbm91Z2ggaW5zZXJ0aW9uIGV2ZW50cyB0aGF0IHRoaXMgZ2V0cyBhIGJpdCBtZXNzZWQKdXAuICBJIHdpbGwgZG91YmxlIGNoZWNrIHRoZSBsb2dpYyBvZiB0aGlzLCBidXQgdGhhdCBpcyBteSBpbml0aWFsIGd1ZXNzIGdpdmVuCmhvdyBmZXcgaW5zZXJ0aW9ucyBJIHdhcyBzZWVpbmcgd2hlbiByZWFkaW5nIHRoZSBvdXRwdXRzIG1hbnVhbGx5LgpVbmZvcnR1bmF0ZWx5LCB0aGlzIG1lYW5zIHRoYXQgZm9yIHRoZXNlIEkgYWxzbyBjYW5ub3QgcHJvdmlkZSBhIGNwbSBtZWFzdXJlbWVudC4KCmBgYHtyIGluc2VydF9pbmRleF9ub3JtYWxpemVkX2J5X2NvdW50cywgcmVzdWx0cz0nYXNpcyd9CmtuaXRyOjprYWJsZSh0cmlwbGVzW1sibWF0cmljZXNfYnlfY291bnRzIl1dW1siaW5zZXJ0X2luZGV4ZXNfYnlfbnQiXV0pCmtuaXRyOjprYWJsZSh0cmlwbGVzX3Rlbm1wcltbIm1hdHJpY2VzX2J5X2NvdW50cyJdXVtbImluc2VydF9pbmRleGVzX2J5X250Il1dKQprbml0cjo6a2FibGUodHJpcGxlc19maXZlbXByW1sibWF0cmljZXNfYnlfY291bnRzIl1dW1siaW5zZXJ0X2luZGV4ZXNfYnlfbnQiXV0pCmtuaXRyOjprYWJsZShxdWludHNbWyJtYXRyaWNlc19ieV9jb3VudHMiXV1bWyJpbnNlcnRfaW5kZXhlc19ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c190ZW5tcHJbWyJtYXRyaWNlc19ieV9jb3VudHMiXV1bWyJpbnNlcnRfaW5kZXhlc19ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c19maXZlbXByW1sibWF0cmljZXNfYnlfY291bnRzIl1dW1siaW5zZXJ0X2luZGV4ZXNfYnlfbnQiXV0pCgprbml0cjo6a2FibGUodHJpcGxlc1tbIm1hdHJpY2VzX2J5X2NvdW50cyJdXVtbImluc2VydF9zZXF1ZW5jZXJfYnlfbnQiXV0pCmtuaXRyOjprYWJsZSh0cmlwbGVzX3Rlbm1wcltbIm1hdHJpY2VzX2J5X2NvdW50cyJdXVtbImluc2VydF9zZXF1ZW5jZXJfYnlfbnQiXV0pCmtuaXRyOjprYWJsZSh0cmlwbGVzX2ZpdmVtcHJbWyJtYXRyaWNlc19ieV9jb3VudHMiXV1bWyJpbnNlcnRfc2VxdWVuY2VyX2J5X250Il1dKQprbml0cjo6a2FibGUocXVpbnRzW1sibWF0cmljZXNfYnlfY291bnRzIl1dW1siaW5zZXJ0X3NlcXVlbmNlcl9ieV9udCJdXSkKa25pdHI6OmthYmxlKHF1aW50c190ZW5tcHJbWyJtYXRyaWNlc19ieV9jb3VudHMiXV1bWyJpbnNlcnRfc2VxdWVuY2VyX2J5X250Il1dKQprbml0cjo6a2FibGUocXVpbnRzX2ZpdmVtcHJbWyJtYXRyaWNlc19ieV9jb3VudHMiXV1bWyJpbnNlcnRfc2VxdWVuY2VyX2J5X250Il1dKQpgYGAKClRoZSBmb2xsb3dpbmcgaXMgbXkgcHJldmlvdXMgd3JpdGluZyBvZiB0aGlzIHdvcmtzaGVldCB3aGljaCBqdXN0IGR1bXBlZCB0aGUKdmFyaW91cyB0YWJsZXMuCgojIFByaW50IHJhdyB0YWJsZXMKCmBgYHtyIHJhdywgcmVzdWx0cz0nYXNpcyd9CmZvciAodCBpbiAxOmxlbmd0aCh0cmlwbGVzW1sibWF0cmljZXMiXV0pKSB7CiAgdGFibGVfbmFtZSA8LSBuYW1lcyh0cmlwbGVzW1sibWF0cmljZXMiXV0pW3RdCiAgbWVzc2FnZSgiUmF3IHRhYmxlOiAiLCB0YWJsZV9uYW1lLCAiLiIpCiAgcHJpbnQoa25pdHI6OmthYmxlKHRyaXBsZXNbWyJtYXRyaWNlcyJdXVt0XSkpCn0KYGBgCgojIFByaW50IHJhdyBwbG90cwoKYGBge3IgcmF3X3Bsb3RzfQpmb3IgKHQgaW4gMTpsZW5ndGgodHJpcGxlc1tbInBsb3RzIl1dW1sibWF0cmljZXMiXV0pKSB7CiAgbWVzc2FnZSgiUmF3IHRhYmxlOiAiLCB0YWJsZV9uYW1lLCAiLiIpCiAgcHJpbnQodHJpcGxlc1tbInBsb3RzIl1dW1sibWF0cmljZXMiXV1bdF0pCn0KYGBgCgojIFByaW50IG5vcm1hbGl6ZWQgdGFibGVzCgpgYGB7ciBub3JtLCByZXN1bHRzPSdhc2lzJ30KZm9yICh0IGluIDE6bGVuZ3RoKHRyaXBsZXNbWyJtYXRyaWNlc19jb3VudHMiXV0pKSB7CiAgdGFibGVfbmFtZSA8LSBuYW1lcyh0cmlwbGVzW1sibWF0cmljZXNfY291bnRzIl1dKVt0XQogIG1lc3NhZ2UoIk5vcm1hbGl6ZWQgdGFibGU6ICIsIHRhYmxlX25hbWUsICIuIikKICBwcmludChrbml0cjo6a2FibGUodHJpcGxlc1tbIm1hdHJpY2VzX2NvdW50cyJdXVt0XSkpCn0KYGBgCgojIFByaW50IG5vcm1hbGl6ZWQgcGxvdHMKCmBgYHtyIG5vcm1fcGxvdHN9CmZvciAodCBpbiAxOmxlbmd0aCh0cmlwbGVzW1sicGxvdHMiXV1bWyJjb3VudHMiXV0pKSB7CiAgbWVzc2FnZSgiTm9ybWFsaXplZCB0YWJsZTogIiwgdGFibGVfbmFtZSwgIi4iKQogIHByaW50KHRyaXBsZXNbWyJwbG90cyJdXVtbImNvdW50cyJdXVt0XSkKfQpgYGAKCmBgYHtyIHNhdmVtZX0KcGFuZGVyOjpwYW5kZXIoc2Vzc2lvbkluZm8oKSkKbWVzc2FnZShwYXN0ZTAoIlRoaXMgaXMgaHBnbHRvb2xzIGNvbW1pdDogIiwgZ2V0X2dpdF9jb21taXQoKSkpCnRoaXNfc2F2ZSA8LSBwYXN0ZTAoZ3N1YihwYXR0ZXJuPSJcXC5SbWQiLCByZXBsYWNlPSIiLCB4PXJtZF9maWxlKSwgIi12IiwgdmVyLCAiLnJkYS54eiIpCm1lc3NhZ2UocGFzdGUwKCJTYXZpbmcgdG8gIiwgdGhpc19zYXZlKSkKdG1wIDwtIHNtKHNhdmVtZShmaWxlbmFtZT10aGlzX3NhdmUpKQpgYGAKCgpgYGB7ciBsb2FkbWUsIGV2YWw9RkFMU0V9CmxvYWRtZShmaWxlbmFtZT10aGlzX3NhdmUpCmBgYAo=

Counting RT mutations from illumina sequencing data.

atb abelew@gmail.com

2020-03-12

1 Calculating error rates.

1.1 Categorize the data with at least 3 indexes per mutant

1.2 Categorize the data with at least 5 indexes per mutant

2 Questions from Dr. DeStefano

3 Answers

3.1 Recovered mutations of each type

3.1.1 Mutations by RT index

3.1.2 Mutations by RT index, post normalization

3.1.2.1 Rewriting the matrices as cpm to account for library sizes.

3.1.2.2 Rewriting the matrices by dividing by all indexes

3.1.2.3 Rewriting the matrices by dividing by all indexes and cpm

3.1.3 Indels by RT index

3.1.4 Insertions by RT index, post normalization

3.1.4.1 Rewriting the matrices as cpm to account for library sizes.

3.1.4.2 Rewriting the matrices by dividing by all indexes

4 Print raw tables

5 Print raw plots

6 Print normalized tables

7 Print normalized plots