A BRIEF overview of Correlations
The three correlations we will be using are some of the most common (though Kendall is less so).
Pearson Correlation:
- The Pearson product-moment correlation is one of the most commonly used correlations in statistics. It’s a measure of the strength and the direction of a linear relationship between two variables. It relies on four key assumptions (much of this below is taken from //statistics.laerd.com/spss-tutorials/pearsons-product-moment-correlation-using-spss-statistics.php).
Assumption 1:
- Your data is interval or ratio. These types of continous data are important for how the correlation assumes values in variables will be related, and thus ordinal or categorical variable coding won’t work.
Semi-Assumption 2:
- As stated above, Pearson only works with linear data. That means that your two correlated factors have to approximate a line, and not a curved or parabolic shape. It’s not that you can’t use pearson to see if there is a linear relationship in data, it’s just that there are other tests suited to analyzing those different data structures.
Assumption 3:
- Outliers in your data can really throw off a Pearson correlation. More information on that here: //www.purplemath.com/modules/boxwhisk3.htm
Assumption 4:
- The data you are analyzing needs to be normally distributed. This can be done in a couple of ways (Skewness, Kurtosis) but it can also be done in a quick and dirty manner through histograms.
Spearman Correlation
- The nice thing about the Spearman correlation is that relies on nearly all the same assumptions as the pearson correlation, but it doesn’t rely on normality, and your data can be ordinal as well. Thus, it’s a non-parametric test. More on the spearman correlation here, //www.statstutor.ac.uk/resources/uploaded/spearmans.pdf, and on parametric vs. non-parametric here, //www.oxfordmathcenter.com/drupal7/node/246.
Kendall Correlation
- The Kendall correlation is similar to the spearman correlation in that it is non-parametric. It can be used with ordinal or continuous data. It is a statistic of dependence between two variables. A discussion of correlation vs. dependence can be found here, and a comparison of all three of these correlations can be found here,//www.quora.com/Probability-statistics-What-is-the-difference-between-dependence-and-correlation-What-is-the-physical-difference, //www.statisticssolutions.com/correlation-pearson-kendall-spearman/.
Setting up the dataset
- Now let’s simluate a dataset to take a look at how the results of these different kinds of correlatiosn may be affected by different parameters of data. First, we need to install some packages.
- Now, we need to create a dataset. Let’s use the scenario of an entire grade of school children across a districs taking an english test at the beginning of the semster (Test.1), and the end (Test.2). Let’s make the datset correlated at .7 (Pearson). The solution for creating the data this way can be found at: //stackoverflow.com/questions/28416897/r-create-dataset-with-specific-correlation-in-r
-Now we can look out our data in a scatterplot, and also fit a linear trend line, to make sure it looks correlated, and also that the linear trend line looks good.
# Create liniear model to calculate intercept and slope fit <- lm(out$Test.2 ~ out$Test.1, data=out) coef(fit)## (Intercept) out$Test.1 ## 34.68142 0.70000# Plot scatterplot along with regression line ggplot(out, aes(x=Test.1, y=Test.2)) + geom_point() + coord_fixed() + geom_smooth(method='lm')Comparing Correlations
Now we want to check our three different pairwise comparisons and compare their values.
cor(Class.Data$Test.1,out$Test.2, method = c("pearson"))## [1] 0.7cor(Class.Data$Test.1,out$Test.2, method = c("spearman"))## [1] 0.6822871cor(Class.Data$Test.1,out$Test.2, method = c("kendall"))## [1] 0.493584we can see pearson and spearman are roughly the same, but kendall is very much different. That’s because Kendall is a test of strength of dependece (i.e. one could be written as a linear function of the other), whereas Pearson and Spearman are nearly equivalent in the way they correlate normally distributed data. All of these correlations are correct in their result, it’s just that Pearson/Spearman are looking at the data in one way, and Kendall in another.
A better situation for spearman or kendall (but not for pearson) when the data is ORDINAL, in that it is ranked. So let’s transform the test 1 scores into rank scores of how well each classmate did relative to one another.
-And now let’s check the correlations again with the test 1 ranked data and the test 2 raw data:
cor(Class.Rank.1$Test.1.Rank,Class.Rank.1$Test.2, method=c("pearson"))## [1] 0.6871984cor(Class.Rank.1$Test.1.Rank,Class.Rank.1$Test.2, method=c("spearman"))## [1] 0.6822871cor(Class.Rank.1$Test.1.Rank,Class.Rank.1$Test.2, method=c("kendall")) ## [1] 0.493584Here again we can see that pearson and spearman are very similar, though pearson has changed slightly. This is likely due to the granularity of one of the sources of data changing to whole integers instead of the numerous decimal places they had previously. However, we see that spearman and Kendall are exactly the same, as they is not as dependent upon the granularity of the integers.
Let’s transform the second score into a rank as well, just to see how it looks:
Now we can see that Pearson exactly matches spearman, as would be expected since the integers are now whole across the board.
While these data are technically ordinal, what we’ve really done is a transformation from raw scores to rank integers. We should expect these to correlate nearly the same (or exactly the same) as the raw scores since they inherently linked. A different way to better expose the differences between these correlations may be to create a non-normal distribution, which can create problems for the Pearson correlation.
Let’s make a uniform distribution of (hypothetically, as this would likely be normally distributed in real life) the children’s average math scores throughout the year.
And now let’s look at our correlations, with our original Test 2. These correlations are going to be vastly different than our previous correlations.
##now let's do some correlations between the new uniform test scores and the original test 2 scores cor(Class.Uni$Math.Avg,Class.Uni$Test.2, method=c("spearman"))## [1] -0.01241258cor(Class.Uni$Math.Avg,Class.Uni$Test.2, method=c("pearson"))## [1] -0.0342776cor(Class.Uni$Math.Avg,Class.Uni$Test.2, method=c("kendall"))## [1] -0.008621554While in reality it may not be the case that math ability and english (or language, generally) ability are this uncorrelated, in our hypothetical world they are very unrelated. Though pearson and spearman may be close to one another, spearman is reliable in this case because the data is not normally distributed. Again, you can still do a pearson correlation on non-normal data, but it’s not going to be as relaible as a non-parametric test which does not assume normality. On the other hand, we can also see that these data are not linearly dependent upon one another, as the kendall correlation is very low also.
Now lets rank order test 1, turning it into ordinal data, and see what happens
##rank order tests based on test 1 score Math.Uni.Rank <-rank(Class.Uni$Math.Avg, na.last=NA,ties.method="first") ##add the rank order tests to the data frame Class.Uni.Rank<-cbind(Math.Rank=Math.Uni.Rank,Class.Uni) #and check our work View(Class.Uni.Rank) #now lets correlate those ranks with test 2 cor(Class.Uni.Rank$Math.Rank,Class.Uni.Rank$Test.2, method=c("spearman"))## [1] -0.01241258cor(Class.Uni.Rank$Math.Rank,Class.Uni.Rank$Test.2, method=c("pearson"))## [1] -0.02603802cor(Class.Uni.Rank$Math.Rank,Class.Uni.Rank$Test.2, method=c("kendall"))## [1] -0.008621554Now we can see that the correlations have remained basically the same, similar to as when we did this with the normally distributed data. Again, the spearman (for relationship) and kendal (for dependence) are going to be more reliable here than pearson.
Note that this data (since it went to so many decimal points) did not have any ties. When you have data that is originaly in whole integers, the rank function is much more important to be aware of in how it handles ties.
Let’s quickly look at how things might change if those uniform math scores were rounded prior to ranking
#create the rounded values from the original math scores Math.Avg.R <-round(Class.Uni$Math.Avg,digits = 0) #bind those to a new dataset Class.Math.Round<-cbind(Math.Round=Math.Avg.R,Class.Uni) #this viewing is optional #View(Class.Math.Round) #now rank the roudned values Math.Rank.Round <-rank(Math.Avg.R, na.last=NA,ties.method="first") #and bind them to a new dataset Class.Math.RoundRank<-cbind(Math.RoundRank=Math.Rank.Round,Class.Math.Round) #and now view View(Class.Math.RoundRank)We can already see that the ranked math scores that depend upon the whole ingeters might change these correlation values, but let’s check.
cor(Class.Math.RoundRank$Math.RoundRank,Class.Uni.Rank$Test.2, method=c("spearman"))## [1] -0.01240358cor(Class.Math.RoundRank$Math.RoundRank,Class.Uni.Rank$Test.2, method=c("pearson"))## [1] -0.02592213cor(Class.Math.RoundRank$Math.RoundRank,Class.Uni.Rank$Test.2, method=c("kendall"))## [1] -0.008746867These changes aren’t dramatic, but in the rank package, there are 6 different ways to handle tie values. If we change it to, say, “average” from “first”
#create the rounded values from the original math scores Math.Avg.RA <-round(Class.Uni$Math.Avg,digits = 0) #bind those to a new dataset Class.Math.RoundA<-cbind(Math.Round=Math.Avg.RA,Class.Uni) #this viewing is optional #View(Class.Math.Round) #now rank the roudned values Math.Rank.RoundA <-rank(Math.Avg.RA, na.last=NA,ties.method="average") #and bind them to a new dataset Class.Math.RoundRankA<-cbind(Math.RoundRankA=Math.Rank.RoundA,Class.Math.RoundA) #and now view View(Class.Math.RoundRankA) cor(Class.Math.RoundRankA$Math.RoundRankA,Class.Uni.Rank$Test.2, method=c("spearman"))## [1] -0.0124663cor(Class.Math.RoundRankA$Math.RoundRankA,Class.Uni.Rank$Test.2, method=c("pearson"))## [1] -0.02586804cor(Class.Math.RoundRankA$Math.RoundRankA,Class.Uni.Rank$Test.2, method=c("kendall"))## [1] -0.008545671Again, we see that these changes aren’t dramatic, but it shows that even small decisions in how your data is handled can affect your results, even when the basis of your data is the same, and the correlation you use is the same. In other studies, this may greatly impact your interpretations of your data.
Conclusion
Given what we see above, there are a number of things to be aware of before going with the commonly used pearson correlations. Beyond the assumptions, it’s important to know if you are looking for relationship or dependence between variables. It’s also important to be aware what may happen to your correlations if you transform your data into ranked scores (though that was not a huge factor here), or how two different distributions of data from different (in this case subject areas) can impact what statsitic your use. There are a number of different threads across forums discussing the differences between these statsitics (e.g. //stats.stackexchange.com/questions/3943/kendall-tau-or-spearmans-rho) if you have more specific questions regarding how to use these statistics with your data.
It takes dilligence to use the right correlation!
LS0tDQp0aXRsZTogJ0NoYXB0ZXIgMjI6IENvcnJlbGF0aW9uIFR5cGVzIGFuZCBXaGVuIHRvIFVzZSBUaGVtJw0KYXV0aG9yOiAiRGF2aWQgU2FybWVudG8iDQpvdXRwdXQ6DQogIGh0bWxfZG9jdW1lbnQ6DQogICAgdGhlbWU6IGNlcnVsZWFuDQogICAgaGlnaGxpZ2h0OiB0ZXh0bWF0ZQ0KICAgIGZvbnRzaXplOiA4cHQNCiAgICB0b2M6IHRydWUNCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUNCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlDQogICAgdG9jX2Zsb2F0Og0KICAgICAgY29sbGFwc2VkOiBmYWxzZQ0KLS0tDQoNCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQ0Ka25pdHI6Om9wdHNfY2h2bmskc2V0KGVjaG8gPSBUUlVFKQ0KYGBgDQoNCiMgUHVycG9zZSBvZiB0aGlzIGNoYXB0ZXINCi0gSW4gdGhpcyBjaGFwdGVyLCB3ZSBhcmUgZ29pbmcgdG8gY292ZXIgdGhlIHN0cmVuZ3Rocywgd2Vha25lc3NlcywgYW5kIHdoZW4gb3Igd2hlbiBub3QgdG8gdXNlIHRocmVlIGNvbW1vbiB0eXBlcyBvZiBjb3JyZWxhdGlvbnMgKFBlYXJzb24sIFNwZWFybWFuLCBhbmQgS2VuZGFsbCkuIEl0J3MgcGFydCBzdGF0aXN0aWNzIHJlZnJlc2hlciwgcGFydCBSIHR1dG9yaWFsLg0KDQojIEEgQlJJRUYgb3ZlcnZpZXcgb2YgQ29ycmVsYXRpb25zDQoNClRoZSB0aHJlZSBjb3JyZWxhdGlvbnMgd2Ugd2lsbCBiZSB1c2luZyBhcmUgc29tZSBvZiB0aGUgbW9zdCBjb21tb24gKHRob3VnaCBLZW5kYWxsIGlzIGxlc3Mgc28pLg0KIA0KIyMgUGVhcnNvbiBDb3JyZWxhdGlvbjoNCi0gVGhlIFBlYXJzb24gcHJvZHVjdC1tb21lbnQgY29ycmVsYXRpb24gaXMgb25lIG9mIHRoZSBtb3N0IGNvbW1vbmx5IHVzZWQgY29ycmVsYXRpb25zIGluIHN0YXRpc3RpY3MuIEl0J3MgYSBtZWFzdXJlIG9mIHRoZSBzdHJlbmd0aCBhbmQgdGhlIGRpcmVjdGlvbiBvZiBhIGxpbmVhciByZWxhdGlvbnNoaXAgYmV0d2VlbiB0d28gdmFyaWFibGVzLiBJdCByZWxpZXMgb24gZm91ciBrZXkgYXNzdW1wdGlvbnMgKG11Y2ggb2YgdGhpcyBiZWxvdyBpcyB0YWtlbiBmcm9tIGh0dHBzOi8vc3RhdGlzdGljcy5sYWVyZC5jb20vc3Bzcy10dXRvcmlhbHMvcGVhcnNvbnMtcHJvZHVjdC1tb21lbnQtY29ycmVsYXRpb24tdXNpbmctc3Bzcy1zdGF0aXN0aWNzLnBocCkuDQoNCiMjIyBBc3N1bXB0aW9uIDE6DQotIFlvdXIgZGF0YSBpcyBpbnRlcnZhbCBvciByYXRpby4gVGhlc2UgdHlwZXMgb2YgY29udGlub3VzIGRhdGEgYXJlIGltcG9ydGFudCBmb3IgaG93IHRoZSBjb3JyZWxhdGlvbiBhc3N1bWVzIHZhbHVlcyBpbiB2YXJpYWJsZXMgd2lsbCBiZSByZWxhdGVkLCBhbmQgdGh2cyBvcmRpbmFsIG9yIGNhdGVnb3JpY2FsIHZhcmlhYmxlIGNvZGluZyB3b24ndCB3b3JrLg0KDQojIyMgU2VtaS1Bc3N1bXB0aW9uIDI6DQotIEFzIHN0YXRlZCBhYm92ZSwgUGVhcnNvbiBvbmx5IHdvcmtzIHdpdGggbGluZWFyIGRhdGEuIFRoYXQgbWVhbnMgdGhhdCB5b3VyIHR3byBjb3JyZWxhdGVkIGZhY3RvcnMgaGF2ZSB0byBhcHByb3hpbWF0ZSBhIGxpbmUsIGFuZCBub3QgYSBjdXJ2ZWQgb3IgcGFyYWJvbGljIHNoYXBlLiAgSXQncyBub3QgdGhhdCB5b3UgY2FuJ3QgdXNlIHBlYXJzb24gdG8gc2VlIGlmIHRoZXJlIGlzIGEgbGluZWFyIHJlbGF0aW9uc2hpcCBpbiBkYXRhLCBpdCdzIGp1c3QgdGhhdCB0aGVyZSBhcmUgb3RoZXIgdGVzdHMgc3VpdGVkIHRvIGFuYWx5emluZyB0aG9zZSBkaWZmZXJlbnQgZGF0YSBzdHJ1Y3R1cmVzLg0KDQojIyMgQXNzdW1wdGlvbiAzOg0KLSBPdXRsaWVycyBpbiB5b3VyIGRhdGEgY2FuIHJlYWxseSB0aHJvdyBvZmYgYSBQZWFyc29uIGNvcnJlbGF0aW9uLiBNb3JlIGluZm9ybWF0aW9uIG9uIHRoYXQgaGVyZTogaHR0cDovL3d3dy5wdXJwbGVtYXRoLmNvbS9tb2R1bGVzL2JveHdoaXNrMy5odG0NCg0KIyMjQXNzdW1wdGlvbiA0Og0KLSBUaGUgZGF0YSB5b3UgYXJlIGFuYWx5emluZyBuZWVkcyB0byBiZSBub3JtYWxseSBkaXN0cmlidXRlZC4gIFRoaXMgY2FuIGJlIGRvbmUgaW4gYSBjb3VwbGUgb2Ygd2F5cyAoU2tld25lc3MsIEt1cnRvc2lzKSBidXQgaXQgY2FuIGFsc28gYmUgZG9uZSBpbiBhIHF1aWNrIGFuZCBkaXJ0eSBtYW5uZXIgdGhyb3VnaCBoaXN0b2dyYW1zLg0KDQojIyBTcGVhcm1hbiBDb3JyZWxhdGlvbg0KLSBUaGUgbmljZSB0aGluZyBhYm91dCB0aGUgU3BlYXJtYW4gY29ycmVsYXRpb24gaXMgdGhhdCByZWxpZXMgb24gbmVhcmx5IGFsbCB0aGUgc2FtZSBhc3N1bXB0aW9ucyBhcyB0aGUgcGVhcnNvbiBjb3JyZWxhdGlvbiwgYnV0IGl0IGRvZXNuJ3QgcmVseSBvbiBub3JtYWxpdHksIGFuZCB5b3VyIGRhdGEgY2FuIGJlIG9yZGluYWwgYXMgd2VsbC4gVGh2cywgaXQncyBhIG5vbi1wYXJhbWV0cmljIHRlc3QuICBNb3JlIG9uIHRoZSBzcGVhcm1hbiBjb3JyZWxhdGlvbiBoZXJlLCBodHRwOi8vd3d3LnN0YXRzdHV0b3IuYWMudWsvcmVzb3VyY2VzL3VwbG9hZGVkL3NwZWFybWFucy5wZGYsIGFuZCBvbiBwYXJhbWV0cmljIHZzLiBub24tcGFyYW1ldHJpYyBoZXJlLCBodHRwOi8vd3d3Lm94Zm9yZG1hdGhjZW50ZXIuY29tL2RydXBhbDcvbm9kZS8yNDYuDQoNCiMjIEtlbmRhbGwgQ29ycmVsYXRpb24NCi0gVGhlIEtlbmRhbGwgY29ycmVsYXRpb24gaXMgc2ltaWxhciB0byB0aGUgc3BlYXJtYW4gY29ycmVsYXRpb24gaW4gdGhhdCBpdCBpcyBub24tcGFyYW1ldHJpYy4gIEl0IGNhbiBiZSB1c2VkIHdpdGggb3JkaW5hbCBvciBjb250aW51b3VzIGRhdGEuICBJdCBpcyBhIHN0YXRpc3RpYyBvZiBkZXBlbmRlbmNlIGJldHdlZW4gdHdvIHZhcmlhYmxlcy4gIEEgZGlzY3Vzc2lvbiBvZiBjb3JyZWxhdGlvbiB2cy4gZGVwZW5kZW5jZSBjYW4gYmUgZm91bmQgaGVyZSwgYW5kIGEgY29tcGFyaXNvbiBvZiBhbGwgdGhyZWUgb2YgdGhlc2UgY29ycmVsYXRpb25zIGNhbiBiZSBmb3VuZCBoZXJlLGh0dHBzOi8vd3d3LnF1b3JhLmNvbS9Qcm9iYWJpbGl0eS1zdGF0aXN0aWNzLVdoYXQtaXMtdGhlLWRpZmZlcmVuY2UtYmV0d2Vlbi1kZXBlbmRlbmNlLWFuZC1jb3JyZWxhdGlvbi1XaGF0LWlzLXRoZS1waHlzaWNhbC1kaWZmZXJlbmNlLCBodHRwOi8vd3d3LnN0YXRpc3RpY3Nzb2x1dGlvbnMuY29tL2NvcnJlbGF0aW9uLXBlYXJzb24ta2VuZGFsbC1zcGVhcm1hbi8uDQoNCiMgU2V0dGluZyB1cCB0aGUgZGF0YXNldCANCi0gTm93IGxldCdzIHNpbWx1YXRlIGEgZGF0YXNldCB0byB0YWtlIGEgbG9vayBhdCBob3cgdGhlIHJlc3VsdHMgb2YgdGhlc2UgZGlmZmVyZW50IGtpbmRzIG9mIGNvcnJlbGF0aW9zbiBtYXkgYmUgYWZmZWN0ZWQgYnkgZGlmZmVyZW50IHBhcmFtZXRlcnMgb2YgZGF0YS4gIEZpcnN0LCB3ZSBuZWVkIHRvIGluc3RhbGwgc29tZSBwYWNrYWdlcy4NCg0KYGBgIHtyLCBtZXNzYWdlPUZBTFNFLCBlY2hvPVRSVUV9DQojaW5zdGFsbC5wYWNrYWdlcygiTUFTUyIpDQpsaWJyYXJ5KE1BU1MpDQojaW5zdGFsbC5wYWNrYWdlcygiZ2dwbG90MiIpDQpsaWJyYXJ5KGdncGxvdDIpDQojaW5zdGFsbC5wYWNrYWdlcygicm9jb2NvIikNCmxpYnJhcnkocm9jb2NvKQ0KI2luc3RhbGwucGFja2FnZXMoInBzeWNoIikNCmxpYnJhcnkocHN5Y2gpDQojaW5zdGFsbC5wYWNrYWdlcygibHBTb2x2ZSIpDQpsaWJyYXJ5KGxwU29sdmUpDQojaW5zdGFsbC5wYWNrYWdlcygiaXJyIikNCmxpYnJhcnkoaXJyKQ0KI2luc3RhbGwucGFja2FnZXMoIm12dG5vcm0iKQ0KbGlicmFyeShtdnRub3JtKQ0KDQpgYGANCg0KLSBOb3csIHdlIG5lZWQgdG8gY3JlYXRlIGEgZGF0YXNldC4gTGV0J3MgdXNlIHRoZSBzY2VuYXJpbyBvZiBhbiBlbnRpcmUgZ3JhZGUgb2Ygc2Nob29sIGNoaWxkcmVuIGFjcm9zcyBhIGRpc3RyaWNzIHRha2luZyBhbiBlbmdsaXNoIHRlc3QgYXQgdGhlIGJlZ2lubmluZyBvZiB0aGUgc2Vtc3RlciAoVGVzdC4xKSwgYW5kIHRoZSBlbmQgKFRlc3QuMikuIExldCdzIG1ha2UgdGhlIGRhdHNldCBjb3JyZWxhdGVkIGF0IC43IChQZWFyc29uKS4gVGhlIHNvbHV0aW9uIGZvciBjcmVhdGluZyB0aGUgZGF0YSB0aGlzIHdheSBjYW4gYmUgZm91bmQgYXQ6IGh0dHBzOi8vc3RhY2tvdmVyZmxvdy5jb20vcXVlc3Rpb25zLzI4NDE2ODk3L3ItY3JlYXRlLWRhdGFzZXQtd2l0aC1zcGVjaWZpYy1jb3JyZWxhdGlvbi1pbi1yDQoNCmBgYCB7ciwgbWVzc2FnZT1GQUxTRSwgZWNobz1UUlVFfQ0KI1N0ZXAgMSAtIHNldCB0aGUgcGFyYW1ldGVycyBvZiBvdXIgZGF0YXNldCBhbmQgZGF0YQ0KDQojIERlc2lyZWQgQ29ycmVsYXRpb24NCmQuY29yIDwtIDAuNw0KIyBEZXNpcmVkIG1lYW4gb2YgWA0KZC5teCA8LSA4MA0KIyBEZXNpcmVkIHJhbmdlIG9mIFgNCmQucmFuZ2V4IDwtIDIwDQojIERlc2lyZWQgbWVhbiBvZiBZDQpkLm15IDwtIDkwDQojIERlc2lyZWQgcmFuZ2Ugb2YgWQ0KZC5yYW5nZXkgPC0gMjANCg0KIyNTdGVwMg0KIyBDYWxjdWxhdGlvbnMgdG8gY3JlYXRlIG11bHRpcGxpY2F0aW9uIGFuZCBhZGRpdGlvbiBmYWN0b3JzIGZvciBtZWFuIGFuZCByYW5nZSBvZiBYIGFuZCBZDQpteC5mYWN0b3IgPC0gZC5yYW5nZXgvNg0KYWRkeC5mYWN0b3IgPC0gZC5teCAtIChteC5mYWN0b3IqMykNCm15LmZhY3RvciA8LSBkLnJhbmdleS82DQphZGR5LmZhY3RvciA8LSBkLm15IC0gKG15LmZhY3RvciozKQ0KDQojIEdlbmVyYXRlIGRhdGEgLSBmb3IgdGhpcyBleGFtcGxlLCBsZXQncyB0aGluayBvZiB0aGlzIGFzIDYwIHN0dWRlbnRzIChyb3dzKS4gIExldCdzIHNheSB0aGV5IGFsbCB0b29rIGEgdGVzdCBhdCB0aGUgYmVnaW5uaW5nDQojb2YgdGhlIHNlbWVzdGVyLCBhbmQgdGhlbiBhZ2FpbiBhdCB0aGUgZW5kIG9mIHRoZSBzZW1lc3Rlci4gIFRoYXQgd2lsbCBnaXZlIHVzIDIgY29sdW1ucyBvZiBkYXRhLCB3aGljaCBpcyAyIHNjb3JlcyBwZXIgc3R1ZGVudCwNCiN3aXRoIGEgcGVhcnNvbiBjb3JyZWxhdGlvbiBvZiAuODAuICBOb3RlIHRoYXQgeW91IGNhbiBhZGp1c3QgdGhlIHBhcmFtZXRlcnMgYXMgeW91IGxpa2Ugd2l0aCB0aGUgY29kZSBpbiBTdGVwcyAxIGFuZCAyLiAgRm9yIG5vdywNCiN3ZSB3aWxsIGJlIG1ha2luZyBlYWNoIHRlc3Qgc2NvcmUgcm91Z2hseSBub3JtYWxseSBkaXN0cmlidXRlZC4NCg0Kb3V0IDwtIGFzLmRhdGEuZnJhbWUobXZybm9ybSg0MDAsIG11ID0gYygwLDApLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgU2lnbWEgPSBtYXRyaXgoYygxLGQuY29yLGQuY29yLDEpLCBuY29sID0gMiksIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICBlbXBpcmljYWwgPSBUUlVFKSkNCg0KIyBBZGp1c3Qgc28gdGhhdCB2YWx1ZXMgYXJlIHBvc2l0aXZlIGFuZCBpbmNsdWRlIGZhY3RvcnMgdG8gbWF0Y2ggZGVzaXJlZCBtZWFucyBhbmQgcmFuZ2VzDQojKHdlIGRvbid0IHdhbnQgbmVnYXRpdmUgdmFsZXMgb24gYSB0ZXN0IHNjb3JlKQ0KI2FuZCBhbHNvIHJlbmFtZSB0aGVtIHRvIFRlc3QuMSwgYW5kIFRlc3QuIDIgIFdlIHdpbGwgbGVhdmUgIlYxIiAkICJWMiIgaW4gdGhlIGRhdHNzZXQgaW4gY2FzZSB3ZSANCiN3YW50IHRvIGFsdGVyIHRoZSByYW5nZSBvZiB0aGUgZGF0YSBhbmQgdGhlIGNvcnJlbGF0aW9uIGxhdGVyLg0KDQpvdXQkIlRlc3QuMSIgPC0gKG91dCRWMSAtIG1pbihvdXQkVjEpKSpteC5mYWN0b3IgKyBhZGR4LmZhY3Rvcg0Kb3V0JCJUZXN0LjIiIDwtIChvdXQkVjIgLSBtaW4ob3V0JFYyKSkqbXkuZmFjdG9yICsgYWRkeS5mYWN0b3INCg0KIyNJdCBtYXkgYWxzbyBiZSBoZWxwZnVsIHRvIGdpdmUgZWFjaCBzdHVkZW50IGFuIElEIG51bWJlciBpbiBjYXNlIHdlIHdhbnQgdG8gbG9vayBhdCBzcGVjaWZpYyBzdHVkZW50IGRhdGEgbGF0ZXIgb24NCg0KI1RvIGRvIHRoaXMsIHdlIG5lZWQgdG8gY3JlYXRlIGEgdmFyaWFibGUsIG4sIHRoYXQgd2lsbCBhbHdheXMgYWRhcHQgdG8gdGhlIG51bWJlciBvZiBzdWJqZWN0cyB5b3UgaGF2ZSB0byBnaXZlIHRoZW0gYSBzdWJqZWN0IG51bWJlcg0KI2luIGNhc2UgeW91IHdhbnQgdG8gYWx0ZXIgdGhlIG51bWJlciBvZiBzdWJqZWN0cyBpbiB5b3VyIHNpbXVsYXRlZCBkYXRhIHNldA0KDQpuPC1sZW5ndGgob3V0JCJUZXN0LjEiKQ0KDQojbm93IHdlIG5lZWQgdG8gY3JlYXRlIGEgc3ViamVjdCBpZCBjb2x1bW4gDQpTdWIuSWQ8LWMoMTpuKQ0KDQojI2FuZCB0aGVuIHB1dCBpdCBhcyBhIG5ldyBjb2x1bW4gaW4gb3VyIGRhdGEgZnJhbWUgdXNpbmcgdGhlICJjYmluZCIgZnVuY3Rpb24NCkNsYXNzLkRhdGE8LWNiaW5kKElEPVN1Yi5JZCxvdXQpDQoNCiNhbmQgdGhlbiBjaGVjayBvdXIgd29yaw0KVmlldyhDbGFzcy5EYXRhKQ0KDQojV2UgY2FuIGFsc28gbG9vayBhdCBvdXIgaGlzdG9ncmFtcyB0byBtYWtlIHN1cmUgdGhlIGRhdGEgd2l0aGluIG91ciBpbmRpdmlkdWFsIHRlc3RzIGlzIG5vcm1hbGx5IGRpc3RyaWJ1dGVkDQoNCmhpc3Qob3V0JCJUZXN0LjEiKQ0KaGlzdChvdXQkIlRlc3QuMiIpDQoNCmBgYA0KDQotTm93IHdlIGNhbiBsb29rIG91dCBvdXIgZGF0YSBpbiBhIHNjYXR0ZXJwbG90LCBhbmQgYWxzbyBmaXQgYSBsaW5lYXIgdHJlbmQgbGluZSwgdG8gbWFrZSBzdXJlIGl0IGxvb2tzIGNvcnJlbGF0ZWQsIGFuZCBhbHNvIHRoYXQgdGhlIGxpbmVhciB0cmVuZCBsaW5lIGxvb2tzIGdvb2QuDQoNCmBgYHtyLCBtZXNzYWdlPUZBTFNFLCBlY2hvPVRSVUV9DQojIENyZWF0ZSBsaW5pZWFyIG1vZGVsIHRvIGNhbGN1bGF0ZSBpbnRlcmNlcHQgYW5kIHNsb3BlDQpmaXQgPC0gbG0ob3V0JFRlc3QuMiB+IG91dCRUZXN0LjEsIGRhdGE9b3V0KQ0KY29lZihmaXQpDQojIFBsb3Qgc2NhdHRlcnBsb3QgYWxvbmcgd2l0aCByZWdyZXNzaW9uIGxpbmUNCmdncGxvdChvdXQsIGFlcyh4PVRlc3QuMSwgeT1UZXN0LjIpKSArIGdlb21fcG9pbnQoKSArIGNvb3JkX2ZpeGVkKCkgKyBnZW9tX3Ntb290aChtZXRob2Q9J2xtJykNCiMgUHJvZHVjZSBzdW1tYXJ5IHRhYmxlDQpzdW1tYXJ5KG91dCkNCmBgYA0KDQojIENvbXBhcmluZyBDb3JyZWxhdGlvbnMNCk5vdyB3ZSB3YW50IHRvIGNoZWNrIG91ciB0aHJlZSBkaWZmZXJlbnQgcGFpcndpc2UgY29tcGFyaXNvbnMgYW5kIGNvbXBhcmUgdGhlaXIgdmFsdWVzLg0KDQoNCmBgYCB7ciwgbWVzc2FnZT1GQUxTRSwgZWNobz1UUlVFfQ0KY29yKENsYXNzLkRhdGEkVGVzdC4xLG91dCRUZXN0LjIsDQogICAgbWV0aG9kID0gYygicGVhcnNvbiIpKQ0KDQpjb3IoQ2xhc3MuRGF0YSRUZXN0LjEsb3V0JFRlc3QuMiwNCiAgbWV0aG9kID0gYygic3BlYXJtYW4iKSkNCg0KY29yKENsYXNzLkRhdGEkVGVzdC4xLG91dCRUZXN0LjIsDQogICAgbWV0aG9kID0gYygia2VuZGFsbCIpKQ0KYGBgDQoNCi0gd2UgY2FuIHNlZSBwZWFyc29uIGFuZCBzcGVhcm1hbiBhcmUgcm91Z2hseSB0aGUgc2FtZSwgYnV0IGtlbmRhbGwgaXMgdmVyeSBtdWNoIGRpZmZlcmVudC4gIFRoYXQncyBiZWNhdXNlIEtlbmRhbGwgaXMgYSB0ZXN0IG9mIHN0cmVuZ3RoIG9mIGRlcGVuZGVjZSAoaS5lLiBvbmUgY291bGQgYmUgd3JpdHRlbiBhcyBhIGxpbmVhciBmdW5jdGlvbiBvZiB0aGUgb3RoZXIpLCB3aGVyZWFzIFBlYXJzb24gYW5kIFNwZWFybWFuIGFyZSBuZWFybHkgZXF1aXZhbGVudCBpbiB0aGUgd2F5IHRoZXkgY29ycmVsYXRlIG5vcm1hbGx5IGRpc3RyaWJ1dGVkIGRhdGEuICBBbGwgb2YgdGhlc2UgY29ycmVsYXRpb25zIGFyZSBjb3JyZWN0IGluIHRoZWlyIHJlc3VsdCwgaXQncyBqdXN0IHRoYXQgUGVhcnNvbi9TcGVhcm1hbiBhcmUgbG9va2luZyBhdCB0aGUgZGF0YSBpbiBvbmUgd2F5LCBhbmQgS2VuZGFsbCBpbiBhbm90aGVyLg0KDQotIEEgYmV0dGVyICBzaXR1YXRpb24gZm9yIHNwZWFybWFuIG9yIGtlbmRhbGwgKGJ1dCBub3QgZm9yIHBlYXJzb24pIHdoZW4gdGhlIGRhdGEgaXMgT1JESU5BTCwgaW4gdGhhdCBpdCBpcyByYW5rZWQuIFNvIGxldCdzIHRyYW5zZm9ybSB0aGUgdGVzdCAxIHNjb3JlcyBpbnRvIHJhbmsgc2NvcmVzIG9mIGhvdyB3ZWxsIGVhY2ggY2xhc3NtYXRlIGRpZCByZWxhdGl2ZSB0byBvbmUgYW5vdGhlci4NCg0KYGBge3IsIG1lc3NhZ2U9RkFMU0UsZWNobz1UUlVFfQ0KIyNDcmVhdGUgYSByYW5rIGZvciB0ZXN0IG9uZS4gIFNlZSBtb3JlIGFib3V0IHRoZSAicmFuayIgZnVuY3Rpb24gYmVsb3c6DQojP3JhbmsNClRlc3QuMS5SYW5rIDwtcmFuayhDbGFzcy5EYXRhJFRlc3QuMSwgbmEubGFzdD1OQSx0aWVzLm1ldGhvZD0iZmlyc3QiKQ0KDQojIyBBbmQgbWFrZSBhIG5ldyBkYXRhIHNldCB3aXRoIHRoZSByYW5rIHRlc3QgYmFzZWQgb24gdGVzdCAxIHNjb3JlDQpDbGFzcy5SYW5rLjE8LWNiaW5kKFRlc3QuMS5SYW5rPVRlc3QuMS5SYW5rLENsYXNzLkRhdGEpDQoNCiNhbmQgbm93IGNoZWNrIG91ciB3b3JrDQpWaWV3KENsYXNzLlJhbmsuMSkNCmBgYA0KLUFuZCBub3cgbGV0J3MgY2hlY2sgdGhlIGNvcnJlbGF0aW9ucyBhZ2FpbiB3aXRoIHRoZSB0ZXN0IDEgcmFua2VkIGRhdGEgYW5kIHRoZSB0ZXN0IDIgcmF3IGRhdGE6DQpgYGB7cixtZXNzYWdlPUZBTFNFLGVjaG89VFJVRX0NCmNvcihDbGFzcy5SYW5rLjEkVGVzdC4xLlJhbmssQ2xhc3MuUmFuay4xJFRlc3QuMiwNCiAgICBtZXRob2Q9YygicGVhcnNvbiIpKQ0KDQpjb3IoQ2xhc3MuUmFuay4xJFRlc3QuMS5SYW5rLENsYXNzLlJhbmsuMSRUZXN0LjIsDQogICAgbWV0aG9kPWMoInNwZWFybWFuIikpDQoNCmNvcihDbGFzcy5SYW5rLjEkVGVzdC4xLlJhbmssQ2xhc3MuUmFuay4xJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNCi0gSGVyZSBhZ2FpbiB3ZSBjYW4gc2VlIHRoYXQgcGVhcnNvbiBhbmQgc3BlYXJtYW4gYXJlIHZlcnkgc2ltaWxhciwgdGhvdWdoIHBlYXJzb24gaGFzIGNoYW5nZWQgc2xpZ2h0bHkuICBUaGlzIGlzIGxpa2VseSBkdWUgdG8gdGhlIGdyYW51bGFyaXR5IG9mIG9uZSBvZiB0aGUgc291cmNlcyBvZiBkYXRhIGNoYW5naW5nIHRvIHdob2xlIGludGVnZXJzIGluc3RlYWQgb2YgdGhlIG51bWVyb3VzIGRlY2ltYWwgcGxhY2VzIHRoZXkgaGFkIHByZXZpb3VzbHkuIEhvd2V2ZXIsIHdlIHNlZSB0aGF0IHNwZWFybWFuIGFuZCBLZW5kYWxsIGFyZSBleGFjdGx5IHRoZSBzYW1lLCBhcyB0aGV5IGlzIG5vdCBhcyBkZXBlbmRlbnQgdXBvbiB0aGUgZ3JhbnVsYXJpdHkgb2YgdGhlIGludGVnZXJzLg0KDQotIExldCdzIHRyYW5zZm9ybSB0aGUgc2Vjb25kIHNjb3JlIGludG8gYSByYW5rIGFzIHdlbGwsIGp1c3QgdG8gc2VlIGhvdyBpdCBsb29rczoNCg0KYGBge3IsbWVzc2FnZT1GQUxTRSwgZWNobz1UUlVFfQ0KVGVzdC4yLlJhbmsgPC1yYW5rKENsYXNzLlJhbmsuMSRUZXN0LjIsIG5hLmxhc3Q9TkEsdGllcy5tZXRob2Q9ImZpcnN0IikNCg0KQ2xhc3MuUmFuay4yPC1jYmluZChUZXN0LjIuUmFuaz1UZXN0LjIuUmFuayxDbGFzcy5SYW5rLjEpDQoNCmNvcihDbGFzcy5SYW5rLjIkVGVzdC4xLlJhbmssQ2xhc3MuUmFuay4yJFRlc3QuMi5SYW5rLA0KICAgIG1ldGhvZD1jKCJwZWFyc29uIikpDQoNCmNvcihDbGFzcy5SYW5rLjIkVGVzdC4xLlJhbmssQ2xhc3MuUmFuay4yJFRlc3QuMi5SYW5rLA0KICAgIG1ldGhvZD1jKCJzcGVhcm1hbiIpKQ0KDQpjb3IoQ2xhc3MuUmFuay4yJFRlc3QuMS5SYW5rLENsYXNzLlJhbmsuMiRUZXN0LjIuUmFuaywNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNCi0gTm93IHdlIGNhbiBzZWUgdGhhdCBQZWFyc29uIGV4YWN0bHkgbWF0Y2hlcyBzcGVhcm1hbiwgYXMgd291bGQgYmUgZXhwZWN0ZWQgc2luY2UgdGhlIGludGVnZXJzIGFyZSBub3cgd2hvbGUgYWNyb3NzIHRoZSBib2FyZC4NCg0KLSBXaGlsZSB0aGVzZSBkYXRhIGFyZSB0ZWNobmljYWxseSBvcmRpbmFsLCB3aGF0IHdlJ3ZlIHJlYWxseSBkb25lIGlzIGEgdHJhbnNmb3JtYXRpb24gZnJvbSByYXcgc2NvcmVzIHRvIHJhbmsgaW50ZWdlcnMuICBXZSBzaG91bGQgZXhwZWN0IHRoZXNlIHRvIGNvcnJlbGF0ZSBuZWFybHkgdGhlIHNhbWUgKG9yIGV4YWN0bHkgdGhlIHNhbWUpIGFzIHRoZSByYXcgc2NvcmVzIHNpbmNlIHRoZXkgaW5oZXJlbnRseSBsaW5rZWQuICBBIGRpZmZlcmVudCB3YXkgdG8gYmV0dGVyIGV4cG9zZSB0aGUgZGlmZmVyZW5jZXMgYmV0d2VlbiB0aGVzZSBjb3JyZWxhdGlvbnMgbWF5IGJlIHRvIGNyZWF0ZSBhIG5vbi1ub3JtYWwgZGlzdHJpYnV0aW9uLCB3aGljaCBjYW4gY3JlYXRlIHByb2JsZW1zIGZvciB0aGUgUGVhcnNvbiBjb3JyZWxhdGlvbi4NCg0KLSBMZXQncyBtYWtlIGEgdW5pZm9ybSBkaXN0cmlidXRpb24gb2YgKGh5cG90aGV0aWNhbGx5LCBhcyB0aGlzIHdvdWxkIGxpa2VseSBiZSBub3JtYWxseSBkaXN0cmlidXRlZCBpbiByZWFsIGxpZmUpIHRoZSBjaGlsZHJlbidzIGF2ZXJhZ2UgbWF0aCBzY29yZXMgdGhyb3VnaG91dCB0aGUgeWVhci4NCg0KYGBge3IsIG1lc3NhZ2U9RkFMU0UsZWNobz1UUlVFfQ0KIyNDcmVhdGUgdGhlIG5ldyB2YXJpYmxlIHdpdGggYSBub3JtYWwgZGlzdHJpYnV0aW9uLg0KI0xvb2sgbW9yZSBhdCB0aGUgcnVuaWYgZnVuY3Rpb24gaGVyZQ0KIz9ydW5pZg0KDQojc28gbGV0cyBtYWtlIGEgbmV3IHRlc3QgMSB0aGF0IGhhcyBhIHVuaWZvcm0gZGlzdHJpYnV0aW9uIHdpdGggYSByYW5nZSBmcm9tIDUwLTEwMCB1c2luZyB0aGUgInJ1bmlmIiBmdW5jdGlvbiANCg0KTWF0aC5Bdmc8LXJ1bmlmKDQwMCxtaW49NTAsbWF4PTEwMCkNCg0KIyNMZXQncyBjaGVjayB0aGUgc2hhcGUgb2YgdGhlIGRpc3RyaWJ1dGlvbiBhbmQgbm90aWNlIGl0J3Mgbm90IG5vcm1hbA0KaGlzdChNYXRoLkF2ZykgDQoNCiNhbmQgbm93IHB1dCB0aGUgbmV3IHVuaWZvcm4gdGVzdCBpbnRvIHRoZSBkYXRhIHNldCANCkNsYXNzLlVuaTwtY2JpbmQoTWF0aC5Bdmc9TWF0aC5BdmcsQ2xhc3MuUmFuay4yKQ0KDQojYW5kIGNoZWNrIG91ciB3b3JrDQpWaWV3KENsYXNzLlVuaSkNCmBgYA0KDQpBbmQgbm93IGxldCdzIGxvb2sgYXQgb3VyIGNvcnJlbGF0aW9ucywgd2l0aCBvdXIgb3JpZ2luYWwgVGVzdCAyLiAgVGhlc2UgY29ycmVsYXRpb25zIGFyZSBnb2luZyB0byBiZSB2YXN0bHkgZGlmZmVyZW50IHRoYW4gb3VyIHByZXZpb3VzIGNvcnJlbGF0aW9ucy4NCg0KYGBge3IsbWVzc2FnZT1GQUxTRSxlY2hvPVRSVUV9DQoNCiMjbm93IGxldCdzIGRvIHNvbWUgY29ycmVsYXRpb25zIGJldHdlZW4gdGhlIG5ldyB1bmlmb3JtIHRlc3Qgc2NvcmVzIGFuZCB0aGUgb3JpZ2luYWwgdGVzdCAyIHNjb3JlcyANCmNvcihDbGFzcy5VbmkkTWF0aC5BdmcsQ2xhc3MuVW5pJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygic3BlYXJtYW4iKSkNCg0KY29yKENsYXNzLlVuaSRNYXRoLkF2ZyxDbGFzcy5VbmkkVGVzdC4yLA0KICAgIG1ldGhvZD1jKCJwZWFyc29uIikpDQoNCmNvcihDbGFzcy5VbmkkTWF0aC5BdmcsQ2xhc3MuVW5pJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNCldoaWxlIGluIHJlYWxpdHkgaXQgbWF5IG5vdCBiZSB0aGUgY2FzZSB0aGF0IG1hdGggYWJpbGl0eSBhbmQgZW5nbGlzaCAob3IgbGFuZ3VhZ2UsIGdlbmVyYWxseSkgYWJpbGl0eSBhcmUgdGhpcyB1bmNvcnJlbGF0ZWQsIGluIG91ciBoeXBvdGhldGljYWwgd29ybGQgdGhleSBhcmUgdmVyeSB1bnJlbGF0ZWQuICBUaG91Z2ggcGVhcnNvbiBhbmQgc3BlYXJtYW4gbWF5IGJlIGNsb3NlIHRvIG9uZSBhbm90aGVyLCBzcGVhcm1hbiBpcyByZWxpYWJsZSBpbiB0aGlzIGNhc2UgYmVjYXVzZSB0aGUgZGF0YSBpcyBub3Qgbm9ybWFsbHkgZGlzdHJpYnV0ZWQuICBBZ2FpbiwgeW91IGNhbiBzdGlsbCBkbyBhIHBlYXJzb24gY29ycmVsYXRpb24gb24gbm9uLW5vcm1hbCBkYXRhLCBidXQgaXQncyBub3QgZ29pbmcgdG8gYmUgYXMgcmVsYWlibGUgYXMgYSBub24tcGFyYW1ldHJpYyB0ZXN0IHdoaWNoIGRvZXMgbm90IGFzc3VtZSBub3JtYWxpdHkuICBPbiB0aGUgb3RoZXIgaGFuZCwgd2UgY2FuIGFsc28gc2VlIHRoYXQgdGhlc2UgZGF0YSBhcmUgbm90IGxpbmVhcmx5IGRlcGVuZGVudCB1cG9uIG9uZSBhbm90aGVyLCBhcyB0aGUga2VuZGFsbCBjb3JyZWxhdGlvbiBpcyB2ZXJ5IGxvdyBhbHNvLg0KDQpOb3cgbGV0cyByYW5rIG9yZGVyIHRlc3QgMSwgdHVybmluZyBpdCBpbnRvIG9yZGluYWwgZGF0YSwgYW5kIHNlZSB3aGF0IGhhcHBlbnMNCg0KYGBgIHtyLG1lc3NhZ2U9RkFMU0UsZWNobz1UUlVFfQ0KIyNyYW5rIG9yZGVyIHRlc3RzIGJhc2VkIG9uIHRlc3QgMSBzY29yZQ0KTWF0aC5VbmkuUmFuayA8LXJhbmsoQ2xhc3MuVW5pJE1hdGguQXZnLCBuYS5sYXN0PU5BLHRpZXMubWV0aG9kPSJmaXJzdCIpDQoNCg0KIyNhZGQgdGhlIHJhbmsgb3JkZXIgdGVzdHMgdG8gdGhlIGRhdGEgZnJhbWUNCkNsYXNzLlVuaS5SYW5rPC1jYmluZChNYXRoLlJhbms9TWF0aC5VbmkuUmFuayxDbGFzcy5VbmkpDQoNCiNhbmQgY2hlY2sgb3VyIHdvcmsNClZpZXcoQ2xhc3MuVW5pLlJhbmspDQoNCiNub3cgbGV0cyBjb3JyZWxhdGUgdGhvc2UgcmFua3Mgd2l0aCB0ZXN0IDINCmNvcihDbGFzcy5VbmkuUmFuayRNYXRoLlJhbmssQ2xhc3MuVW5pLlJhbmskVGVzdC4yLA0KICAgIG1ldGhvZD1jKCJzcGVhcm1hbiIpKQ0KDQpjb3IoQ2xhc3MuVW5pLlJhbmskTWF0aC5SYW5rLENsYXNzLlVuaS5SYW5rJFRlc3QuMiwNCiAgICBtZXRob2Q9YygicGVhcnNvbiIpKQ0KDQpjb3IoQ2xhc3MuVW5pLlJhbmskTWF0aC5SYW5rLENsYXNzLlVuaS5SYW5rJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNCi0gTm93IHdlIGNhbiBzZWUgdGhhdCB0aGUgY29ycmVsYXRpb25zIGhhdmUgcmVtYWluZWQgYmFzaWNhbGx5IHRoZSBzYW1lLCBzaW1pbGFyIHRvIGFzIHdoZW4gd2UgZGlkIHRoaXMgd2l0aCB0aGUgbm9ybWFsbHkgZGlzdHJpYnV0ZWQgZGF0YS4gIEFnYWluLCB0aGUgc3BlYXJtYW4gKGZvciByZWxhdGlvbnNoaXApIGFuZCBrZW5kYWwgKGZvciBkZXBlbmRlbmNlKSBhcmUgZ29pbmcgdG8gYmUgbW9yZSByZWxpYWJsZSBoZXJlIHRoYW4gcGVhcnNvbi4NCg0KLSBOb3RlIHRoYXQgdGhpcyBkYXRhIChzaW5jZSBpdCB3ZW50IHRvIHNvIG1hbnkgZGVjaW1hbCBwb2ludHMpIGRpZCBub3QgaGF2ZSBhbnkgdGllcy4gIFdoZW4geW91IGhhdmUgZGF0YSB0aGF0IGlzIG9yaWdpbmFseSBpbiB3aG9sZSBpbnRlZ2VycywgdGhlIHJhbmsgZnVuY3Rpb24gaXMgbXVjaCBtb3JlIGltcG9ydGFudCB0byBiZSBhd2FyZSBvZiBpbiBob3cgaXQgaGFuZGxlcyB0aWVzLiANCg0KTGV0J3MgcXVpY2tseSBsb29rIGF0IGhvdyB0aGluZ3MgbWlnaHQgY2hhbmdlIGlmIHRob3NlIHVuaWZvcm0gbWF0aCBzY29yZXMgd2VyZSByb3VuZGVkIHByaW9yIHRvIHJhbmtpbmcNCg0KYGBge3IsbWVzc2FnZT1GQUxTRSxlY2hvPVRSVUV9DQojY3JlYXRlIHRoZSByb3VuZGVkIHZhbHVlcyBmcm9tIHRoZSBvcmlnaW5hbCBtYXRoIHNjb3Jlcw0KTWF0aC5BdmcuUiA8LXJvdW5kKENsYXNzLlVuaSRNYXRoLkF2ZyxkaWdpdHMgPSAwKQ0KI2JpbmQgdGhvc2UgdG8gYSBuZXcgZGF0YXNldA0KQ2xhc3MuTWF0aC5Sb3VuZDwtY2JpbmQoTWF0aC5Sb3VuZD1NYXRoLkF2Zy5SLENsYXNzLlVuaSkNCiN0aGlzIHZpZXdpbmcgaXMgb3B0aW9uYWwNCiNWaWV3KENsYXNzLk1hdGguUm91bmQpDQoNCiNub3cgcmFuayB0aGUgcm91ZG5lZCB2YWx1ZXMNCk1hdGguUmFuay5Sb3VuZCA8LXJhbmsoTWF0aC5BdmcuUiwgbmEubGFzdD1OQSx0aWVzLm1ldGhvZD0iZmlyc3QiKQ0KI2FuZCBiaW5kIHRoZW0gdG8gYSBuZXcgZGF0YXNldA0KQ2xhc3MuTWF0aC5Sb3VuZFJhbms8LWNiaW5kKE1hdGguUm91bmRSYW5rPU1hdGguUmFuay5Sb3VuZCxDbGFzcy5NYXRoLlJvdW5kKQ0KDQojYW5kIG5vdyB2aWV3DQpWaWV3KENsYXNzLk1hdGguUm91bmRSYW5rKQ0KYGBgDQoNCldlIGNhbiBhbHJlYWR5IHNlZSB0aGF0IHRoZSByYW5rZWQgbWF0aCBzY29yZXMgdGhhdCBkZXBlbmQgdXBvbiB0aGUgd2hvbGUgaW5nZXRlcnMgbWlnaHQgY2hhbmdlIHRoZXNlIGNvcnJlbGF0aW9uIHZhbHVlcywgYnV0IGxldCdzIGNoZWNrLg0KDQpgYGB7cixtZXNzYWdlPUZBTFNFLGVjaG89VFJVRX0NCmNvcihDbGFzcy5NYXRoLlJvdW5kUmFuayRNYXRoLlJvdW5kUmFuayxDbGFzcy5VbmkuUmFuayRUZXN0LjIsDQogICAgbWV0aG9kPWMoInNwZWFybWFuIikpDQoNCmNvcihDbGFzcy5NYXRoLlJvdW5kUmFuayRNYXRoLlJvdW5kUmFuayxDbGFzcy5VbmkuUmFuayRUZXN0LjIsDQogICAgbWV0aG9kPWMoInBlYXJzb24iKSkNCg0KY29yKENsYXNzLk1hdGguUm91bmRSYW5rJE1hdGguUm91bmRSYW5rLENsYXNzLlVuaS5SYW5rJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNClRoZXNlIGNoYW5nZXMgYXJlbid0IGRyYW1hdGljLCBidXQgaW4gdGhlIHJhbmsgcGFja2FnZSwgdGhlcmUgYXJlIDYgZGlmZmVyZW50IHdheXMgdG8gaGFuZGxlIHRpZSB2YWx1ZXMuIElmIHdlIGNoYW5nZSBpdCB0bywgc2F5LCAiYXZlcmFnZSIgZnJvbSAiZmlyc3QiDQoNCmBgYHtyLG1lc3NhZ2U9RkFMU0UsZWNobz1UUlVFfQ0KI2NyZWF0ZSB0aGUgcm91bmRlZCB2YWx1ZXMgZnJvbSB0aGUgb3JpZ2luYWwgbWF0aCBzY29yZXMNCk1hdGguQXZnLlJBIDwtcm91bmQoQ2xhc3MuVW5pJE1hdGguQXZnLGRpZ2l0cyA9IDApDQojYmluZCB0aG9zZSB0byBhIG5ldyBkYXRhc2V0DQpDbGFzcy5NYXRoLlJvdW5kQTwtY2JpbmQoTWF0aC5Sb3VuZD1NYXRoLkF2Zy5SQSxDbGFzcy5VbmkpDQojdGhpcyB2aWV3aW5nIGlzIG9wdGlvbmFsDQojVmlldyhDbGFzcy5NYXRoLlJvdW5kKQ0KDQojbm93IHJhbmsgdGhlIHJvdWRuZWQgdmFsdWVzDQpNYXRoLlJhbmsuUm91bmRBIDwtcmFuayhNYXRoLkF2Zy5SQSwgbmEubGFzdD1OQSx0aWVzLm1ldGhvZD0iYXZlcmFnZSIpDQojYW5kIGJpbmQgdGhlbSB0byBhIG5ldyBkYXRhc2V0DQpDbGFzcy5NYXRoLlJvdW5kUmFua0E8LWNiaW5kKE1hdGguUm91bmRSYW5rQT1NYXRoLlJhbmsuUm91bmRBLENsYXNzLk1hdGguUm91bmRBKQ0KDQojYW5kIG5vdyB2aWV3DQpWaWV3KENsYXNzLk1hdGguUm91bmRSYW5rQSkNCg0KY29yKENsYXNzLk1hdGguUm91bmRSYW5rQSRNYXRoLlJvdW5kUmFua0EsQ2xhc3MuVW5pLlJhbmskVGVzdC4yLA0KICAgIG1ldGhvZD1jKCJzcGVhcm1hbiIpKQ0KDQpjb3IoQ2xhc3MuTWF0aC5Sb3VuZFJhbmtBJE1hdGguUm91bmRSYW5rQSxDbGFzcy5VbmkuUmFuayRUZXN0LjIsDQogICAgbWV0aG9kPWMoInBlYXJzb24iKSkNCg0KY29yKENsYXNzLk1hdGguUm91bmRSYW5rQSRNYXRoLlJvdW5kUmFua0EsQ2xhc3MuVW5pLlJhbmskVGVzdC4yLA0KICAgIG1ldGhvZD1jKCJrZW5kYWxsIikpDQpgYGANCg0KQWdhaW4sIHdlIHNlZSB0aGF0IHRoZXNlIGNoYW5nZXMgYXJlbid0IGRyYW1hdGljLCBidXQgaXQgc2hvd3MgdGhhdCBldmVuIHNtYWxsIGRlY2lzaW9ucyBpbiBob3cgeW91ciBkYXRhIGlzIGhhbmRsZWQgY2FuIGFmZmVjdCB5b3VyIHJlc3VsdHMsIGV2ZW4gd2hlbiB0aGUgYmFzaXMgb2YgeW91ciBkYXRhIGlzIHRoZSBzYW1lLCBhbmQgdGhlIGNvcnJlbGF0aW9uIHlvdSB1c2UgaXMgdGhlIHNhbWUuICBJbiBvdGhlciBzdHVkaWVzLCB0aGlzIG1heSBncmVhdGx5IGltcGFjdCB5b3VyIGludGVycHJldGF0aW9ucyBvZiB5b3VyIGRhdGEuDQoNCiMgQ29uY2x1c2lvbg0KDQpHaXZlbiB3aGF0IHdlIHNlZSBhYm92ZSwgdGhlcmUgYXJlIGEgbnVtYmVyIG9mIHRoaW5ncyB0byBiZSBhd2FyZSBvZiBiZWZvcmUgZ29pbmcgd2l0aCB0aGUgY29tbW9ubHkgdXNlZCBwZWFyc29uIGNvcnJlbGF0aW9ucy4gIEJleW9uZCB0aGUgYXNzdW1wdGlvbnMsIGl0J3MgaW1wb3J0YW50IHRvIGtub3cgaWYgeW91IGFyZSBsb29raW5nIGZvciByZWxhdGlvbnNoaXAgb3IgZGVwZW5kZW5jZSBiZXR3ZWVuIHZhcmlhYmxlcy4gSXQncyBhbHNvIGltcG9ydGFudCB0byBiZSBhd2FyZSB3aGF0IG1heSBoYXBwZW4gdG8geW91ciBjb3JyZWxhdGlvbnMgaWYgeW91IHRyYW5zZm9ybSB5b3VyIGRhdGEgaW50byByYW5rZWQgc2NvcmVzICh0aG91Z2ggdGhhdCB3YXMgbm90IGEgaHVnZSBmYWN0b3IgaGVyZSksIG9yIGhvdyB0d28gZGlmZmVyZW50IGRpc3RyaWJ1dGlvbnMgb2YgZGF0YSBmcm9tIGRpZmZlcmVudCAoaW4gdGhpcyBjYXNlIHN1YmplY3QgYXJlYXMpIGNhbiBpbXBhY3Qgd2hhdCBzdGF0c2l0aWMgeW91ciB1c2UuIFRoZXJlIGFyZSBhIG51bWJlciBvZiBkaWZmZXJlbnQgdGhyZWFkcyBhY3Jvc3MgZm9ydW1zIGRpc2N1c3NpbmcgdGhlIGRpZmZlcmVuY2VzIGJldHdlZW4gdGhlc2Ugc3RhdHNpdGljcyAoZS5nLiBodHRwczovL3N0YXRzLnN0YWNrZXhjaGFuZ2UuY29tL3F1ZXN0aW9ucy8zOTQzL2tlbmRhbGwtdGF1LW9yLXNwZWFybWFucy1yaG8pIGlmIHlvdSBoYXZlIG1vcmUgc3BlY2lmaWMgcXVlc3Rpb25zIHJlZ2FyZGluZyBob3cgdG8gdXNlIHRoZXNlIHN0YXRpc3RpY3Mgd2l0aCB5b3VyIGRhdGEuDQoNCkl0IHRha2VzIGRpbGxpZ2VuY2UgdG8gdXNlIHRoZSByaWdodCBjb3JyZWxhdGlvbiEgDQoNCg==