A Pearson product-moment correlation is often used for this type of reliability

A BRIEF overview of Correlations

The three correlations we will be using are some of the most common (though Kendall is less so).

Pearson Correlation:

• The Pearson product-moment correlation is one of the most commonly used correlations in statistics. It’s a measure of the strength and the direction of a linear relationship between two variables. It relies on four key assumptions (much of this below is taken from https://statistics.laerd.com/spss-tutorials/pearsons-product-moment-correlation-using-spss-statistics.php).

Assumption 1:

• Your data is interval or ratio. These types of continous data are important for how the correlation assumes values in variables will be related, and thus ordinal or categorical variable coding won’t work.

Semi-Assumption 2:

• As stated above, Pearson only works with linear data. That means that your two correlated factors have to approximate a line, and not a curved or parabolic shape. It’s not that you can’t use pearson to see if there is a linear relationship in data, it’s just that there are other tests suited to analyzing those different data structures.

Assumption 3:

• Outliers in your data can really throw off a Pearson correlation. More information on that here: http://www.purplemath.com/modules/boxwhisk3.htm

Assumption 4:

• The data you are analyzing needs to be normally distributed. This can be done in a couple of ways (Skewness, Kurtosis) but it can also be done in a quick and dirty manner through histograms.

Spearman Correlation

• The nice thing about the Spearman correlation is that relies on nearly all the same assumptions as the pearson correlation, but it doesn’t rely on normality, and your data can be ordinal as well. Thus, it’s a non-parametric test. More on the spearman correlation here, http://www.statstutor.ac.uk/resources/uploaded/spearmans.pdf, and on parametric vs. non-parametric here, http://www.oxfordmathcenter.com/drupal7/node/246.

Kendall Correlation

• The Kendall correlation is similar to the spearman correlation in that it is non-parametric. It can be used with ordinal or continuous data. It is a statistic of dependence between two variables. A discussion of correlation vs. dependence can be found here, and a comparison of all three of these correlations can be found here,https://www.quora.com/Probability-statistics-What-is-the-difference-between-dependence-and-correlation-What-is-the-physical-difference, http://www.statisticssolutions.com/correlation-pearson-kendall-spearman/.

Setting up the dataset

• Now let’s simluate a dataset to take a look at how the results of these different kinds of correlatiosn may be affected by different parameters of data. First, we need to install some packages.

#install.packages("MASS")
library(MASS)
#install.packages("ggplot2")
library(ggplot2)
#install.packages("rococo")
library(rococo)
#install.packages("psych")
library(psych)
#install.packages("lpSolve")
library(lpSolve)
#install.packages("irr")
library(irr)
#install.packages("mvtnorm")
library(mvtnorm)

• Now, we need to create a dataset. Let’s use the scenario of an entire grade of school children across a districs taking an english test at the beginning of the semster (Test.1), and the end (Test.2). Let’s make the datset correlated at .7 (Pearson). The solution for creating the data this way can be found at: https://stackoverflow.com/questions/28416897/r-create-dataset-with-specific-correlation-in-r

#Step 1 - set the parameters of our dataset and data

# Desired Correlation
d.cor <- 0.7
# Desired mean of X
d.mx <- 80
# Desired range of X
d.rangex <- 20
# Desired mean of Y
d.my <- 90
# Desired range of Y
d.rangey <- 20

##Step2
# Calculations to create multiplication and addition factors for mean and range of X and Y
mx.factor <- d.rangex/6
addx.factor <- d.mx - (mx.factor*3)
my.factor <- d.rangey/6
addy.factor <- d.my - (my.factor*3)

# Generate data - for this example, let's think of this as 60 students (rows).  Let's say they all took a test at the beginning
#of the semester, and then again at the end of the semester.  That will give us 2 columns of data, which is 2 scores per student,
#with a pearson correlation of .80.  Note that you can adjust the parameters as you like with the code in Steps 1 and 2.  For now,
#we will be making each test score roughly normally distributed.

out <- as.data.frame(mvrnorm(400, mu = c(0,0), 
                             Sigma = matrix(c(1,d.cor,d.cor,1), ncol = 2), 
                             empirical = TRUE))

# Adjust so that values are positive and include factors to match desired means and ranges
#(we don't want negative vales on a test score)
#and also rename them to Test.1, and Test. 2  We will leave "V1" $ "V2" in the datsset in case we 
#want to alter the range of the data and the correlation later.

out$"Test.1" <- (out$V1 - min(out$V1))*mx.factor + addx.factor
out$"Test.2" <- (out$V2 - min(out$V2))*my.factor + addy.factor

##It may also be helpful to give each student an ID number in case we want to look at specific student data later on

#To do this, we need to create a variable, n, that will always adapt to the number of subjects you have to give them a subject number
#in case you want to alter the number of subjects in your simulated data set

n<-length(out$"Test.1")

#now we need to create a subject id column 
Sub.Id<-c(1:n)

##and then put it as a new column in our data frame using the "cbind" function
Class.Data<-cbind(ID=Sub.Id,out)

#and then check our work
View(Class.Data)

#We can also look at our histograms to make sure the data within our individual tests is normally distributed

hist(out$"Test.1")

hist(out$"Test.2")

-Now we can look out our data in a scatterplot, and also fit a linear trend line, to make sure it looks correlated, and also that the linear trend line looks good.

# Create liniear model to calculate intercept and slope
fit <- lm(out$Test.2 ~ out$Test.1, data=out)
coef(fit)

## (Intercept)  out$Test.1 
##    34.68142     0.70000

# Plot scatterplot along with regression line
ggplot(out, aes(x=Test.1, y=Test.2)) + geom_point() + coord_fixed() + geom_smooth(method='lm')

# Produce summary table
summary(out)

##        V1                 V2               Test.1          Test.2     
##  Min.   :-2.97985   Min.   :-3.19032   Min.   :70.00   Min.   :80.00  
##  1st Qu.:-0.62519   1st Qu.:-0.68310   1st Qu.:77.85   1st Qu.:88.36  
##  Median : 0.01322   Median : 0.03225   Median :79.98   Median :90.74  
##  Mean   : 0.00000   Mean   : 0.00000   Mean   :79.93   Mean   :90.63  
##  3rd Qu.: 0.69287   3rd Qu.: 0.68248   3rd Qu.:82.24   3rd Qu.:92.91  
##  Max.   : 3.37708   Max.   : 2.67112   Max.   :91.19   Max.   :99.54

Comparing Correlations

Now we want to check our three different pairwise comparisons and compare their values.

cor(Class.Data$Test.1,out$Test.2,
    method = c("pearson"))

## [1] 0.7

cor(Class.Data$Test.1,out$Test.2,
  method = c("spearman"))

## [1] 0.6822871

cor(Class.Data$Test.1,out$Test.2,
    method = c("kendall"))

## [1] 0.493584

• we can see pearson and spearman are roughly the same, but kendall is very much different. That’s because Kendall is a test of strength of dependece (i.e. one could be written as a linear function of the other), whereas Pearson and Spearman are nearly equivalent in the way they correlate normally distributed data. All of these correlations are correct in their result, it’s just that Pearson/Spearman are looking at the data in one way, and Kendall in another.

• A better situation for spearman or kendall (but not for pearson) when the data is ORDINAL, in that it is ranked. So let’s transform the test 1 scores into rank scores of how well each classmate did relative to one another.

##Create a rank for test one.  See more about the "rank" function below:
#?rank
Test.1.Rank <-rank(Class.Data$Test.1, na.last=NA,ties.method="first")

## And make a new data set with the rank test based on test 1 score
Class.Rank.1<-cbind(Test.1.Rank=Test.1.Rank,Class.Data)

#and now check our work
View(Class.Rank.1)

-And now let’s check the correlations again with the test 1 ranked data and the test 2 raw data:

cor(Class.Rank.1$Test.1.Rank,Class.Rank.1$Test.2,
    method=c("pearson"))

## [1] 0.6871984

cor(Class.Rank.1$Test.1.Rank,Class.Rank.1$Test.2,
    method=c("spearman"))

## [1] 0.6822871

cor(Class.Rank.1$Test.1.Rank,Class.Rank.1$Test.2,
    method=c("kendall"))

## [1] 0.493584

• Here again we can see that pearson and spearman are very similar, though pearson has changed slightly. This is likely due to the granularity of one of the sources of data changing to whole integers instead of the numerous decimal places they had previously. However, we see that spearman and Kendall are exactly the same, as they is not as dependent upon the granularity of the integers.

• Let’s transform the second score into a rank as well, just to see how it looks:

Test.2.Rank <-rank(Class.Rank.1$Test.2, na.last=NA,ties.method="first")

Class.Rank.2<-cbind(Test.2.Rank=Test.2.Rank,Class.Rank.1)

cor(Class.Rank.2$Test.1.Rank,Class.Rank.2$Test.2.Rank,
    method=c("pearson"))

## [1] 0.6822871

cor(Class.Rank.2$Test.1.Rank,Class.Rank.2$Test.2.Rank,
    method=c("spearman"))

## [1] 0.6822871

cor(Class.Rank.2$Test.1.Rank,Class.Rank.2$Test.2.Rank,
    method=c("kendall"))

## [1] 0.493584

• Now we can see that Pearson exactly matches spearman, as would be expected since the integers are now whole across the board.

• While these data are technically ordinal, what we’ve really done is a transformation from raw scores to rank integers. We should expect these to correlate nearly the same (or exactly the same) as the raw scores since they inherently linked. A different way to better expose the differences between these correlations may be to create a non-normal distribution, which can create problems for the Pearson correlation.

• Let’s make a uniform distribution of (hypothetically, as this would likely be normally distributed in real life) the children’s average math scores throughout the year.

##Create the new varible with a normal distribution.
#Look more at the runif function here
#?runif

#so lets make a new test 1 that has a uniform distribution with a range from 50-100 using the "runif" function 

Math.Avg<-runif(400,min=50,max=100)

##Let's check the shape of the distribution and notice it's not normal
hist(Math.Avg) 

#and now put the new uniforn test into the data set 
Class.Uni<-cbind(Math.Avg=Math.Avg,Class.Rank.2)

#and check our work
View(Class.Uni)

And now let’s look at our correlations, with our original Test 2. These correlations are going to be vastly different than our previous correlations.

##now let's do some correlations between the new uniform test scores and the original test 2 scores 
cor(Class.Uni$Math.Avg,Class.Uni$Test.2,
    method=c("spearman"))

## [1] -0.01241258

cor(Class.Uni$Math.Avg,Class.Uni$Test.2,
    method=c("pearson"))

## [1] -0.0342776

cor(Class.Uni$Math.Avg,Class.Uni$Test.2,
    method=c("kendall"))

## [1] -0.008621554

While in reality it may not be the case that math ability and english (or language, generally) ability are this uncorrelated, in our hypothetical world they are very unrelated. Though pearson and spearman may be close to one another, spearman is reliable in this case because the data is not normally distributed. Again, you can still do a pearson correlation on non-normal data, but it’s not going to be as relaible as a non-parametric test which does not assume normality. On the other hand, we can also see that these data are not linearly dependent upon one another, as the kendall correlation is very low also.

Now lets rank order test 1, turning it into ordinal data, and see what happens

##rank order tests based on test 1 score
Math.Uni.Rank <-rank(Class.Uni$Math.Avg, na.last=NA,ties.method="first")


##add the rank order tests to the data frame
Class.Uni.Rank<-cbind(Math.Rank=Math.Uni.Rank,Class.Uni)

#and check our work
View(Class.Uni.Rank)

#now lets correlate those ranks with test 2
cor(Class.Uni.Rank$Math.Rank,Class.Uni.Rank$Test.2,
    method=c("spearman"))

## [1] -0.01241258

cor(Class.Uni.Rank$Math.Rank,Class.Uni.Rank$Test.2,
    method=c("pearson"))

## [1] -0.02603802

cor(Class.Uni.Rank$Math.Rank,Class.Uni.Rank$Test.2,
    method=c("kendall"))

## [1] -0.008621554

• Now we can see that the correlations have remained basically the same, similar to as when we did this with the normally distributed data. Again, the spearman (for relationship) and kendal (for dependence) are going to be more reliable here than pearson.

• Note that this data (since it went to so many decimal points) did not have any ties. When you have data that is originaly in whole integers, the rank function is much more important to be aware of in how it handles ties.

Let’s quickly look at how things might change if those uniform math scores were rounded prior to ranking

#create the rounded values from the original math scores
Math.Avg.R <-round(Class.Uni$Math.Avg,digits = 0)
#bind those to a new dataset
Class.Math.Round<-cbind(Math.Round=Math.Avg.R,Class.Uni)
#this viewing is optional
#View(Class.Math.Round)

#now rank the roudned values
Math.Rank.Round <-rank(Math.Avg.R, na.last=NA,ties.method="first")
#and bind them to a new dataset
Class.Math.RoundRank<-cbind(Math.RoundRank=Math.Rank.Round,Class.Math.Round)

#and now view
View(Class.Math.RoundRank)

We can already see that the ranked math scores that depend upon the whole ingeters might change these correlation values, but let’s check.

cor(Class.Math.RoundRank$Math.RoundRank,Class.Uni.Rank$Test.2,
    method=c("spearman"))

## [1] -0.01240358

cor(Class.Math.RoundRank$Math.RoundRank,Class.Uni.Rank$Test.2,
    method=c("pearson"))

## [1] -0.02592213

cor(Class.Math.RoundRank$Math.RoundRank,Class.Uni.Rank$Test.2,
    method=c("kendall"))

## [1] -0.008746867

These changes aren’t dramatic, but in the rank package, there are 6 different ways to handle tie values. If we change it to, say, “average” from “first”

#create the rounded values from the original math scores
Math.Avg.RA <-round(Class.Uni$Math.Avg,digits = 0)
#bind those to a new dataset
Class.Math.RoundA<-cbind(Math.Round=Math.Avg.RA,Class.Uni)
#this viewing is optional
#View(Class.Math.Round)

#now rank the roudned values
Math.Rank.RoundA <-rank(Math.Avg.RA, na.last=NA,ties.method="average")
#and bind them to a new dataset
Class.Math.RoundRankA<-cbind(Math.RoundRankA=Math.Rank.RoundA,Class.Math.RoundA)

#and now view
View(Class.Math.RoundRankA)

cor(Class.Math.RoundRankA$Math.RoundRankA,Class.Uni.Rank$Test.2,
    method=c("spearman"))

## [1] -0.0124663

cor(Class.Math.RoundRankA$Math.RoundRankA,Class.Uni.Rank$Test.2,
    method=c("pearson"))

## [1] -0.02586804

cor(Class.Math.RoundRankA$Math.RoundRankA,Class.Uni.Rank$Test.2,
    method=c("kendall"))

## [1] -0.008545671

Again, we see that these changes aren’t dramatic, but it shows that even small decisions in how your data is handled can affect your results, even when the basis of your data is the same, and the correlation you use is the same. In other studies, this may greatly impact your interpretations of your data.

Conclusion

Given what we see above, there are a number of things to be aware of before going with the commonly used pearson correlations. Beyond the assumptions, it’s important to know if you are looking for relationship or dependence between variables. It’s also important to be aware what may happen to your correlations if you transform your data into ranked scores (though that was not a huge factor here), or how two different distributions of data from different (in this case subject areas) can impact what statsitic your use. There are a number of different threads across forums discussing the differences between these statsitics (e.g. https://stats.stackexchange.com/questions/3943/kendall-tau-or-spearmans-rho) if you have more specific questions regarding how to use these statistics with your data.

It takes dilligence to use the right correlation!

LS0tDQp0aXRsZTogJ0NoYXB0ZXIgMjI6IENvcnJlbGF0aW9uIFR5cGVzIGFuZCBXaGVuIHRvIFVzZSBUaGVtJw0KYXV0aG9yOiAiRGF2aWQgU2FybWVudG8iDQpvdXRwdXQ6DQogIGh0bWxfZG9jdW1lbnQ6DQogICAgdGhlbWU6IGNlcnVsZWFuDQogICAgaGlnaGxpZ2h0OiB0ZXh0bWF0ZQ0KICAgIGZvbnRzaXplOiA4cHQNCiAgICB0b2M6IHRydWUNCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUNCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlDQogICAgdG9jX2Zsb2F0Og0KICAgICAgY29sbGFwc2VkOiBmYWxzZQ0KLS0tDQoNCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQ0Ka25pdHI6Om9wdHNfY2h2bmskc2V0KGVjaG8gPSBUUlVFKQ0KYGBgDQoNCiMgUHVycG9zZSBvZiB0aGlzIGNoYXB0ZXINCi0gSW4gdGhpcyBjaGFwdGVyLCB3ZSBhcmUgZ29pbmcgdG8gY292ZXIgdGhlIHN0cmVuZ3Rocywgd2Vha25lc3NlcywgYW5kIHdoZW4gb3Igd2hlbiBub3QgdG8gdXNlIHRocmVlIGNvbW1vbiB0eXBlcyBvZiBjb3JyZWxhdGlvbnMgKFBlYXJzb24sIFNwZWFybWFuLCBhbmQgS2VuZGFsbCkuIEl0J3MgcGFydCBzdGF0aXN0aWNzIHJlZnJlc2hlciwgcGFydCBSIHR1dG9yaWFsLg0KDQojIEEgQlJJRUYgb3ZlcnZpZXcgb2YgQ29ycmVsYXRpb25zDQoNClRoZSB0aHJlZSBjb3JyZWxhdGlvbnMgd2Ugd2lsbCBiZSB1c2luZyBhcmUgc29tZSBvZiB0aGUgbW9zdCBjb21tb24gKHRob3VnaCBLZW5kYWxsIGlzIGxlc3Mgc28pLg0KIA0KIyMgUGVhcnNvbiBDb3JyZWxhdGlvbjoNCi0gVGhlIFBlYXJzb24gcHJvZHVjdC1tb21lbnQgY29ycmVsYXRpb24gaXMgb25lIG9mIHRoZSBtb3N0IGNvbW1vbmx5IHVzZWQgY29ycmVsYXRpb25zIGluIHN0YXRpc3RpY3MuIEl0J3MgYSBtZWFzdXJlIG9mIHRoZSBzdHJlbmd0aCBhbmQgdGhlIGRpcmVjdGlvbiBvZiBhIGxpbmVhciByZWxhdGlvbnNoaXAgYmV0d2VlbiB0d28gdmFyaWFibGVzLiBJdCByZWxpZXMgb24gZm91ciBrZXkgYXNzdW1wdGlvbnMgKG11Y2ggb2YgdGhpcyBiZWxvdyBpcyB0YWtlbiBmcm9tIGh0dHBzOi8vc3RhdGlzdGljcy5sYWVyZC5jb20vc3Bzcy10dXRvcmlhbHMvcGVhcnNvbnMtcHJvZHVjdC1tb21lbnQtY29ycmVsYXRpb24tdXNpbmctc3Bzcy1zdGF0aXN0aWNzLnBocCkuDQoNCiMjIyBBc3N1bXB0aW9uIDE6DQotIFlvdXIgZGF0YSBpcyBpbnRlcnZhbCBvciByYXRpby4gVGhlc2UgdHlwZXMgb2YgY29udGlub3VzIGRhdGEgYXJlIGltcG9ydGFudCBmb3IgaG93IHRoZSBjb3JyZWxhdGlvbiBhc3N1bWVzIHZhbHVlcyBpbiB2YXJpYWJsZXMgd2lsbCBiZSByZWxhdGVkLCBhbmQgdGh2cyBvcmRpbmFsIG9yIGNhdGVnb3JpY2FsIHZhcmlhYmxlIGNvZGluZyB3b24ndCB3b3JrLg0KDQojIyMgU2VtaS1Bc3N1bXB0aW9uIDI6DQotIEFzIHN0YXRlZCBhYm92ZSwgUGVhcnNvbiBvbmx5IHdvcmtzIHdpdGggbGluZWFyIGRhdGEuIFRoYXQgbWVhbnMgdGhhdCB5b3VyIHR3byBjb3JyZWxhdGVkIGZhY3RvcnMgaGF2ZSB0byBhcHByb3hpbWF0ZSBhIGxpbmUsIGFuZCBub3QgYSBjdXJ2ZWQgb3IgcGFyYWJvbGljIHNoYXBlLiAgSXQncyBub3QgdGhhdCB5b3UgY2FuJ3QgdXNlIHBlYXJzb24gdG8gc2VlIGlmIHRoZXJlIGlzIGEgbGluZWFyIHJlbGF0aW9uc2hpcCBpbiBkYXRhLCBpdCdzIGp1c3QgdGhhdCB0aGVyZSBhcmUgb3RoZXIgdGVzdHMgc3VpdGVkIHRvIGFuYWx5emluZyB0aG9zZSBkaWZmZXJlbnQgZGF0YSBzdHJ1Y3R1cmVzLg0KDQojIyMgQXNzdW1wdGlvbiAzOg0KLSBPdXRsaWVycyBpbiB5b3VyIGRhdGEgY2FuIHJlYWxseSB0aHJvdyBvZmYgYSBQZWFyc29uIGNvcnJlbGF0aW9uLiBNb3JlIGluZm9ybWF0aW9uIG9uIHRoYXQgaGVyZTogaHR0cDovL3d3dy5wdXJwbGVtYXRoLmNvbS9tb2R1bGVzL2JveHdoaXNrMy5odG0NCg0KIyMjQXNzdW1wdGlvbiA0Og0KLSBUaGUgZGF0YSB5b3UgYXJlIGFuYWx5emluZyBuZWVkcyB0byBiZSBub3JtYWxseSBkaXN0cmlidXRlZC4gIFRoaXMgY2FuIGJlIGRvbmUgaW4gYSBjb3VwbGUgb2Ygd2F5cyAoU2tld25lc3MsIEt1cnRvc2lzKSBidXQgaXQgY2FuIGFsc28gYmUgZG9uZSBpbiBhIHF1aWNrIGFuZCBkaXJ0eSBtYW5uZXIgdGhyb3VnaCBoaXN0b2dyYW1zLg0KDQojIyBTcGVhcm1hbiBDb3JyZWxhdGlvbg0KLSBUaGUgbmljZSB0aGluZyBhYm91dCB0aGUgU3BlYXJtYW4gY29ycmVsYXRpb24gaXMgdGhhdCByZWxpZXMgb24gbmVhcmx5IGFsbCB0aGUgc2FtZSBhc3N1bXB0aW9ucyBhcyB0aGUgcGVhcnNvbiBjb3JyZWxhdGlvbiwgYnV0IGl0IGRvZXNuJ3QgcmVseSBvbiBub3JtYWxpdHksIGFuZCB5b3VyIGRhdGEgY2FuIGJlIG9yZGluYWwgYXMgd2VsbC4gVGh2cywgaXQncyBhIG5vbi1wYXJhbWV0cmljIHRlc3QuICBNb3JlIG9uIHRoZSBzcGVhcm1hbiBjb3JyZWxhdGlvbiBoZXJlLCBodHRwOi8vd3d3LnN0YXRzdHV0b3IuYWMudWsvcmVzb3VyY2VzL3VwbG9hZGVkL3NwZWFybWFucy5wZGYsIGFuZCBvbiBwYXJhbWV0cmljIHZzLiBub24tcGFyYW1ldHJpYyBoZXJlLCBodHRwOi8vd3d3Lm94Zm9yZG1hdGhjZW50ZXIuY29tL2RydXBhbDcvbm9kZS8yNDYuDQoNCiMjIEtlbmRhbGwgQ29ycmVsYXRpb24NCi0gVGhlIEtlbmRhbGwgY29ycmVsYXRpb24gaXMgc2ltaWxhciB0byB0aGUgc3BlYXJtYW4gY29ycmVsYXRpb24gaW4gdGhhdCBpdCBpcyBub24tcGFyYW1ldHJpYy4gIEl0IGNhbiBiZSB1c2VkIHdpdGggb3JkaW5hbCBvciBjb250aW51b3VzIGRhdGEuICBJdCBpcyBhIHN0YXRpc3RpYyBvZiBkZXBlbmRlbmNlIGJldHdlZW4gdHdvIHZhcmlhYmxlcy4gIEEgZGlzY3Vzc2lvbiBvZiBjb3JyZWxhdGlvbiB2cy4gZGVwZW5kZW5jZSBjYW4gYmUgZm91bmQgaGVyZSwgYW5kIGEgY29tcGFyaXNvbiBvZiBhbGwgdGhyZWUgb2YgdGhlc2UgY29ycmVsYXRpb25zIGNhbiBiZSBmb3VuZCBoZXJlLGh0dHBzOi8vd3d3LnF1b3JhLmNvbS9Qcm9iYWJpbGl0eS1zdGF0aXN0aWNzLVdoYXQtaXMtdGhlLWRpZmZlcmVuY2UtYmV0d2Vlbi1kZXBlbmRlbmNlLWFuZC1jb3JyZWxhdGlvbi1XaGF0LWlzLXRoZS1waHlzaWNhbC1kaWZmZXJlbmNlLCBodHRwOi8vd3d3LnN0YXRpc3RpY3Nzb2x1dGlvbnMuY29tL2NvcnJlbGF0aW9uLXBlYXJzb24ta2VuZGFsbC1zcGVhcm1hbi8uDQoNCiMgU2V0dGluZyB1cCB0aGUgZGF0YXNldCANCi0gTm93IGxldCdzIHNpbWx1YXRlIGEgZGF0YXNldCB0byB0YWtlIGEgbG9vayBhdCBob3cgdGhlIHJlc3VsdHMgb2YgdGhlc2UgZGlmZmVyZW50IGtpbmRzIG9mIGNvcnJlbGF0aW9zbiBtYXkgYmUgYWZmZWN0ZWQgYnkgZGlmZmVyZW50IHBhcmFtZXRlcnMgb2YgZGF0YS4gIEZpcnN0LCB3ZSBuZWVkIHRvIGluc3RhbGwgc29tZSBwYWNrYWdlcy4NCg0KYGBgIHtyLCBtZXNzYWdlPUZBTFNFLCBlY2hvPVRSVUV9DQojaW5zdGFsbC5wYWNrYWdlcygiTUFTUyIpDQpsaWJyYXJ5KE1BU1MpDQojaW5zdGFsbC5wYWNrYWdlcygiZ2dwbG90MiIpDQpsaWJyYXJ5KGdncGxvdDIpDQojaW5zdGFsbC5wYWNrYWdlcygicm9jb2NvIikNCmxpYnJhcnkocm9jb2NvKQ0KI2luc3RhbGwucGFja2FnZXMoInBzeWNoIikNCmxpYnJhcnkocHN5Y2gpDQojaW5zdGFsbC5wYWNrYWdlcygibHBTb2x2ZSIpDQpsaWJyYXJ5KGxwU29sdmUpDQojaW5zdGFsbC5wYWNrYWdlcygiaXJyIikNCmxpYnJhcnkoaXJyKQ0KI2luc3RhbGwucGFja2FnZXMoIm12dG5vcm0iKQ0KbGlicmFyeShtdnRub3JtKQ0KDQpgYGANCg0KLSBOb3csIHdlIG5lZWQgdG8gY3JlYXRlIGEgZGF0YXNldC4gTGV0J3MgdXNlIHRoZSBzY2VuYXJpbyBvZiBhbiBlbnRpcmUgZ3JhZGUgb2Ygc2Nob29sIGNoaWxkcmVuIGFjcm9zcyBhIGRpc3RyaWNzIHRha2luZyBhbiBlbmdsaXNoIHRlc3QgYXQgdGhlIGJlZ2lubmluZyBvZiB0aGUgc2Vtc3RlciAoVGVzdC4xKSwgYW5kIHRoZSBlbmQgKFRlc3QuMikuIExldCdzIG1ha2UgdGhlIGRhdHNldCBjb3JyZWxhdGVkIGF0IC43IChQZWFyc29uKS4gVGhlIHNvbHV0aW9uIGZvciBjcmVhdGluZyB0aGUgZGF0YSB0aGlzIHdheSBjYW4gYmUgZm91bmQgYXQ6IGh0dHBzOi8vc3RhY2tvdmVyZmxvdy5jb20vcXVlc3Rpb25zLzI4NDE2ODk3L3ItY3JlYXRlLWRhdGFzZXQtd2l0aC1zcGVjaWZpYy1jb3JyZWxhdGlvbi1pbi1yDQoNCmBgYCB7ciwgbWVzc2FnZT1GQUxTRSwgZWNobz1UUlVFfQ0KI1N0ZXAgMSAtIHNldCB0aGUgcGFyYW1ldGVycyBvZiBvdXIgZGF0YXNldCBhbmQgZGF0YQ0KDQojIERlc2lyZWQgQ29ycmVsYXRpb24NCmQuY29yIDwtIDAuNw0KIyBEZXNpcmVkIG1lYW4gb2YgWA0KZC5teCA8LSA4MA0KIyBEZXNpcmVkIHJhbmdlIG9mIFgNCmQucmFuZ2V4IDwtIDIwDQojIERlc2lyZWQgbWVhbiBvZiBZDQpkLm15IDwtIDkwDQojIERlc2lyZWQgcmFuZ2Ugb2YgWQ0KZC5yYW5nZXkgPC0gMjANCg0KIyNTdGVwMg0KIyBDYWxjdWxhdGlvbnMgdG8gY3JlYXRlIG11bHRpcGxpY2F0aW9uIGFuZCBhZGRpdGlvbiBmYWN0b3JzIGZvciBtZWFuIGFuZCByYW5nZSBvZiBYIGFuZCBZDQpteC5mYWN0b3IgPC0gZC5yYW5nZXgvNg0KYWRkeC5mYWN0b3IgPC0gZC5teCAtIChteC5mYWN0b3IqMykNCm15LmZhY3RvciA8LSBkLnJhbmdleS82DQphZGR5LmZhY3RvciA8LSBkLm15IC0gKG15LmZhY3RvciozKQ0KDQojIEdlbmVyYXRlIGRhdGEgLSBmb3IgdGhpcyBleGFtcGxlLCBsZXQncyB0aGluayBvZiB0aGlzIGFzIDYwIHN0dWRlbnRzIChyb3dzKS4gIExldCdzIHNheSB0aGV5IGFsbCB0b29rIGEgdGVzdCBhdCB0aGUgYmVnaW5uaW5nDQojb2YgdGhlIHNlbWVzdGVyLCBhbmQgdGhlbiBhZ2FpbiBhdCB0aGUgZW5kIG9mIHRoZSBzZW1lc3Rlci4gIFRoYXQgd2lsbCBnaXZlIHVzIDIgY29sdW1ucyBvZiBkYXRhLCB3aGljaCBpcyAyIHNjb3JlcyBwZXIgc3R1ZGVudCwNCiN3aXRoIGEgcGVhcnNvbiBjb3JyZWxhdGlvbiBvZiAuODAuICBOb3RlIHRoYXQgeW91IGNhbiBhZGp1c3QgdGhlIHBhcmFtZXRlcnMgYXMgeW91IGxpa2Ugd2l0aCB0aGUgY29kZSBpbiBTdGVwcyAxIGFuZCAyLiAgRm9yIG5vdywNCiN3ZSB3aWxsIGJlIG1ha2luZyBlYWNoIHRlc3Qgc2NvcmUgcm91Z2hseSBub3JtYWxseSBkaXN0cmlidXRlZC4NCg0Kb3V0IDwtIGFzLmRhdGEuZnJhbWUobXZybm9ybSg0MDAsIG11ID0gYygwLDApLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgU2lnbWEgPSBtYXRyaXgoYygxLGQuY29yLGQuY29yLDEpLCBuY29sID0gMiksIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICBlbXBpcmljYWwgPSBUUlVFKSkNCg0KIyBBZGp1c3Qgc28gdGhhdCB2YWx1ZXMgYXJlIHBvc2l0aXZlIGFuZCBpbmNsdWRlIGZhY3RvcnMgdG8gbWF0Y2ggZGVzaXJlZCBtZWFucyBhbmQgcmFuZ2VzDQojKHdlIGRvbid0IHdhbnQgbmVnYXRpdmUgdmFsZXMgb24gYSB0ZXN0IHNjb3JlKQ0KI2FuZCBhbHNvIHJlbmFtZSB0aGVtIHRvIFRlc3QuMSwgYW5kIFRlc3QuIDIgIFdlIHdpbGwgbGVhdmUgIlYxIiAkICJWMiIgaW4gdGhlIGRhdHNzZXQgaW4gY2FzZSB3ZSANCiN3YW50IHRvIGFsdGVyIHRoZSByYW5nZSBvZiB0aGUgZGF0YSBhbmQgdGhlIGNvcnJlbGF0aW9uIGxhdGVyLg0KDQpvdXQkIlRlc3QuMSIgPC0gKG91dCRWMSAtIG1pbihvdXQkVjEpKSpteC5mYWN0b3IgKyBhZGR4LmZhY3Rvcg0Kb3V0JCJUZXN0LjIiIDwtIChvdXQkVjIgLSBtaW4ob3V0JFYyKSkqbXkuZmFjdG9yICsgYWRkeS5mYWN0b3INCg0KIyNJdCBtYXkgYWxzbyBiZSBoZWxwZnVsIHRvIGdpdmUgZWFjaCBzdHVkZW50IGFuIElEIG51bWJlciBpbiBjYXNlIHdlIHdhbnQgdG8gbG9vayBhdCBzcGVjaWZpYyBzdHVkZW50IGRhdGEgbGF0ZXIgb24NCg0KI1RvIGRvIHRoaXMsIHdlIG5lZWQgdG8gY3JlYXRlIGEgdmFyaWFibGUsIG4sIHRoYXQgd2lsbCBhbHdheXMgYWRhcHQgdG8gdGhlIG51bWJlciBvZiBzdWJqZWN0cyB5b3UgaGF2ZSB0byBnaXZlIHRoZW0gYSBzdWJqZWN0IG51bWJlcg0KI2luIGNhc2UgeW91IHdhbnQgdG8gYWx0ZXIgdGhlIG51bWJlciBvZiBzdWJqZWN0cyBpbiB5b3VyIHNpbXVsYXRlZCBkYXRhIHNldA0KDQpuPC1sZW5ndGgob3V0JCJUZXN0LjEiKQ0KDQojbm93IHdlIG5lZWQgdG8gY3JlYXRlIGEgc3ViamVjdCBpZCBjb2x1bW4gDQpTdWIuSWQ8LWMoMTpuKQ0KDQojI2FuZCB0aGVuIHB1dCBpdCBhcyBhIG5ldyBjb2x1bW4gaW4gb3VyIGRhdGEgZnJhbWUgdXNpbmcgdGhlICJjYmluZCIgZnVuY3Rpb24NCkNsYXNzLkRhdGE8LWNiaW5kKElEPVN1Yi5JZCxvdXQpDQoNCiNhbmQgdGhlbiBjaGVjayBvdXIgd29yaw0KVmlldyhDbGFzcy5EYXRhKQ0KDQojV2UgY2FuIGFsc28gbG9vayBhdCBvdXIgaGlzdG9ncmFtcyB0byBtYWtlIHN1cmUgdGhlIGRhdGEgd2l0aGluIG91ciBpbmRpdmlkdWFsIHRlc3RzIGlzIG5vcm1hbGx5IGRpc3RyaWJ1dGVkDQoNCmhpc3Qob3V0JCJUZXN0LjEiKQ0KaGlzdChvdXQkIlRlc3QuMiIpDQoNCmBgYA0KDQotTm93IHdlIGNhbiBsb29rIG91dCBvdXIgZGF0YSBpbiBhIHNjYXR0ZXJwbG90LCBhbmQgYWxzbyBmaXQgYSBsaW5lYXIgdHJlbmQgbGluZSwgdG8gbWFrZSBzdXJlIGl0IGxvb2tzIGNvcnJlbGF0ZWQsIGFuZCBhbHNvIHRoYXQgdGhlIGxpbmVhciB0cmVuZCBsaW5lIGxvb2tzIGdvb2QuDQoNCmBgYHtyLCBtZXNzYWdlPUZBTFNFLCBlY2hvPVRSVUV9DQojIENyZWF0ZSBsaW5pZWFyIG1vZGVsIHRvIGNhbGN1bGF0ZSBpbnRlcmNlcHQgYW5kIHNsb3BlDQpmaXQgPC0gbG0ob3V0JFRlc3QuMiB+IG91dCRUZXN0LjEsIGRhdGE9b3V0KQ0KY29lZihmaXQpDQojIFBsb3Qgc2NhdHRlcnBsb3QgYWxvbmcgd2l0aCByZWdyZXNzaW9uIGxpbmUNCmdncGxvdChvdXQsIGFlcyh4PVRlc3QuMSwgeT1UZXN0LjIpKSArIGdlb21fcG9pbnQoKSArIGNvb3JkX2ZpeGVkKCkgKyBnZW9tX3Ntb290aChtZXRob2Q9J2xtJykNCiMgUHJvZHVjZSBzdW1tYXJ5IHRhYmxlDQpzdW1tYXJ5KG91dCkNCmBgYA0KDQojIENvbXBhcmluZyBDb3JyZWxhdGlvbnMNCk5vdyB3ZSB3YW50IHRvIGNoZWNrIG91ciB0aHJlZSBkaWZmZXJlbnQgcGFpcndpc2UgY29tcGFyaXNvbnMgYW5kIGNvbXBhcmUgdGhlaXIgdmFsdWVzLg0KDQoNCmBgYCB7ciwgbWVzc2FnZT1GQUxTRSwgZWNobz1UUlVFfQ0KY29yKENsYXNzLkRhdGEkVGVzdC4xLG91dCRUZXN0LjIsDQogICAgbWV0aG9kID0gYygicGVhcnNvbiIpKQ0KDQpjb3IoQ2xhc3MuRGF0YSRUZXN0LjEsb3V0JFRlc3QuMiwNCiAgbWV0aG9kID0gYygic3BlYXJtYW4iKSkNCg0KY29yKENsYXNzLkRhdGEkVGVzdC4xLG91dCRUZXN0LjIsDQogICAgbWV0aG9kID0gYygia2VuZGFsbCIpKQ0KYGBgDQoNCi0gd2UgY2FuIHNlZSBwZWFyc29uIGFuZCBzcGVhcm1hbiBhcmUgcm91Z2hseSB0aGUgc2FtZSwgYnV0IGtlbmRhbGwgaXMgdmVyeSBtdWNoIGRpZmZlcmVudC4gIFRoYXQncyBiZWNhdXNlIEtlbmRhbGwgaXMgYSB0ZXN0IG9mIHN0cmVuZ3RoIG9mIGRlcGVuZGVjZSAoaS5lLiBvbmUgY291bGQgYmUgd3JpdHRlbiBhcyBhIGxpbmVhciBmdW5jdGlvbiBvZiB0aGUgb3RoZXIpLCB3aGVyZWFzIFBlYXJzb24gYW5kIFNwZWFybWFuIGFyZSBuZWFybHkgZXF1aXZhbGVudCBpbiB0aGUgd2F5IHRoZXkgY29ycmVsYXRlIG5vcm1hbGx5IGRpc3RyaWJ1dGVkIGRhdGEuICBBbGwgb2YgdGhlc2UgY29ycmVsYXRpb25zIGFyZSBjb3JyZWN0IGluIHRoZWlyIHJlc3VsdCwgaXQncyBqdXN0IHRoYXQgUGVhcnNvbi9TcGVhcm1hbiBhcmUgbG9va2luZyBhdCB0aGUgZGF0YSBpbiBvbmUgd2F5LCBhbmQgS2VuZGFsbCBpbiBhbm90aGVyLg0KDQotIEEgYmV0dGVyICBzaXR1YXRpb24gZm9yIHNwZWFybWFuIG9yIGtlbmRhbGwgKGJ1dCBub3QgZm9yIHBlYXJzb24pIHdoZW4gdGhlIGRhdGEgaXMgT1JESU5BTCwgaW4gdGhhdCBpdCBpcyByYW5rZWQuIFNvIGxldCdzIHRyYW5zZm9ybSB0aGUgdGVzdCAxIHNjb3JlcyBpbnRvIHJhbmsgc2NvcmVzIG9mIGhvdyB3ZWxsIGVhY2ggY2xhc3NtYXRlIGRpZCByZWxhdGl2ZSB0byBvbmUgYW5vdGhlci4NCg0KYGBge3IsIG1lc3NhZ2U9RkFMU0UsZWNobz1UUlVFfQ0KIyNDcmVhdGUgYSByYW5rIGZvciB0ZXN0IG9uZS4gIFNlZSBtb3JlIGFib3V0IHRoZSAicmFuayIgZnVuY3Rpb24gYmVsb3c6DQojP3JhbmsNClRlc3QuMS5SYW5rIDwtcmFuayhDbGFzcy5EYXRhJFRlc3QuMSwgbmEubGFzdD1OQSx0aWVzLm1ldGhvZD0iZmlyc3QiKQ0KDQojIyBBbmQgbWFrZSBhIG5ldyBkYXRhIHNldCB3aXRoIHRoZSByYW5rIHRlc3QgYmFzZWQgb24gdGVzdCAxIHNjb3JlDQpDbGFzcy5SYW5rLjE8LWNiaW5kKFRlc3QuMS5SYW5rPVRlc3QuMS5SYW5rLENsYXNzLkRhdGEpDQoNCiNhbmQgbm93IGNoZWNrIG91ciB3b3JrDQpWaWV3KENsYXNzLlJhbmsuMSkNCmBgYA0KLUFuZCBub3cgbGV0J3MgY2hlY2sgdGhlIGNvcnJlbGF0aW9ucyBhZ2FpbiB3aXRoIHRoZSB0ZXN0IDEgcmFua2VkIGRhdGEgYW5kIHRoZSB0ZXN0IDIgcmF3IGRhdGE6DQpgYGB7cixtZXNzYWdlPUZBTFNFLGVjaG89VFJVRX0NCmNvcihDbGFzcy5SYW5rLjEkVGVzdC4xLlJhbmssQ2xhc3MuUmFuay4xJFRlc3QuMiwNCiAgICBtZXRob2Q9YygicGVhcnNvbiIpKQ0KDQpjb3IoQ2xhc3MuUmFuay4xJFRlc3QuMS5SYW5rLENsYXNzLlJhbmsuMSRUZXN0LjIsDQogICAgbWV0aG9kPWMoInNwZWFybWFuIikpDQoNCmNvcihDbGFzcy5SYW5rLjEkVGVzdC4xLlJhbmssQ2xhc3MuUmFuay4xJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNCi0gSGVyZSBhZ2FpbiB3ZSBjYW4gc2VlIHRoYXQgcGVhcnNvbiBhbmQgc3BlYXJtYW4gYXJlIHZlcnkgc2ltaWxhciwgdGhvdWdoIHBlYXJzb24gaGFzIGNoYW5nZWQgc2xpZ2h0bHkuICBUaGlzIGlzIGxpa2VseSBkdWUgdG8gdGhlIGdyYW51bGFyaXR5IG9mIG9uZSBvZiB0aGUgc291cmNlcyBvZiBkYXRhIGNoYW5naW5nIHRvIHdob2xlIGludGVnZXJzIGluc3RlYWQgb2YgdGhlIG51bWVyb3VzIGRlY2ltYWwgcGxhY2VzIHRoZXkgaGFkIHByZXZpb3VzbHkuIEhvd2V2ZXIsIHdlIHNlZSB0aGF0IHNwZWFybWFuIGFuZCBLZW5kYWxsIGFyZSBleGFjdGx5IHRoZSBzYW1lLCBhcyB0aGV5IGlzIG5vdCBhcyBkZXBlbmRlbnQgdXBvbiB0aGUgZ3JhbnVsYXJpdHkgb2YgdGhlIGludGVnZXJzLg0KDQotIExldCdzIHRyYW5zZm9ybSB0aGUgc2Vjb25kIHNjb3JlIGludG8gYSByYW5rIGFzIHdlbGwsIGp1c3QgdG8gc2VlIGhvdyBpdCBsb29rczoNCg0KYGBge3IsbWVzc2FnZT1GQUxTRSwgZWNobz1UUlVFfQ0KVGVzdC4yLlJhbmsgPC1yYW5rKENsYXNzLlJhbmsuMSRUZXN0LjIsIG5hLmxhc3Q9TkEsdGllcy5tZXRob2Q9ImZpcnN0IikNCg0KQ2xhc3MuUmFuay4yPC1jYmluZChUZXN0LjIuUmFuaz1UZXN0LjIuUmFuayxDbGFzcy5SYW5rLjEpDQoNCmNvcihDbGFzcy5SYW5rLjIkVGVzdC4xLlJhbmssQ2xhc3MuUmFuay4yJFRlc3QuMi5SYW5rLA0KICAgIG1ldGhvZD1jKCJwZWFyc29uIikpDQoNCmNvcihDbGFzcy5SYW5rLjIkVGVzdC4xLlJhbmssQ2xhc3MuUmFuay4yJFRlc3QuMi5SYW5rLA0KICAgIG1ldGhvZD1jKCJzcGVhcm1hbiIpKQ0KDQpjb3IoQ2xhc3MuUmFuay4yJFRlc3QuMS5SYW5rLENsYXNzLlJhbmsuMiRUZXN0LjIuUmFuaywNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNCi0gTm93IHdlIGNhbiBzZWUgdGhhdCBQZWFyc29uIGV4YWN0bHkgbWF0Y2hlcyBzcGVhcm1hbiwgYXMgd291bGQgYmUgZXhwZWN0ZWQgc2luY2UgdGhlIGludGVnZXJzIGFyZSBub3cgd2hvbGUgYWNyb3NzIHRoZSBib2FyZC4NCg0KLSBXaGlsZSB0aGVzZSBkYXRhIGFyZSB0ZWNobmljYWxseSBvcmRpbmFsLCB3aGF0IHdlJ3ZlIHJlYWxseSBkb25lIGlzIGEgdHJhbnNmb3JtYXRpb24gZnJvbSByYXcgc2NvcmVzIHRvIHJhbmsgaW50ZWdlcnMuICBXZSBzaG91bGQgZXhwZWN0IHRoZXNlIHRvIGNvcnJlbGF0ZSBuZWFybHkgdGhlIHNhbWUgKG9yIGV4YWN0bHkgdGhlIHNhbWUpIGFzIHRoZSByYXcgc2NvcmVzIHNpbmNlIHRoZXkgaW5oZXJlbnRseSBsaW5rZWQuICBBIGRpZmZlcmVudCB3YXkgdG8gYmV0dGVyIGV4cG9zZSB0aGUgZGlmZmVyZW5jZXMgYmV0d2VlbiB0aGVzZSBjb3JyZWxhdGlvbnMgbWF5IGJlIHRvIGNyZWF0ZSBhIG5vbi1ub3JtYWwgZGlzdHJpYnV0aW9uLCB3aGljaCBjYW4gY3JlYXRlIHByb2JsZW1zIGZvciB0aGUgUGVhcnNvbiBjb3JyZWxhdGlvbi4NCg0KLSBMZXQncyBtYWtlIGEgdW5pZm9ybSBkaXN0cmlidXRpb24gb2YgKGh5cG90aGV0aWNhbGx5LCBhcyB0aGlzIHdvdWxkIGxpa2VseSBiZSBub3JtYWxseSBkaXN0cmlidXRlZCBpbiByZWFsIGxpZmUpIHRoZSBjaGlsZHJlbidzIGF2ZXJhZ2UgbWF0aCBzY29yZXMgdGhyb3VnaG91dCB0aGUgeWVhci4NCg0KYGBge3IsIG1lc3NhZ2U9RkFMU0UsZWNobz1UUlVFfQ0KIyNDcmVhdGUgdGhlIG5ldyB2YXJpYmxlIHdpdGggYSBub3JtYWwgZGlzdHJpYnV0aW9uLg0KI0xvb2sgbW9yZSBhdCB0aGUgcnVuaWYgZnVuY3Rpb24gaGVyZQ0KIz9ydW5pZg0KDQojc28gbGV0cyBtYWtlIGEgbmV3IHRlc3QgMSB0aGF0IGhhcyBhIHVuaWZvcm0gZGlzdHJpYnV0aW9uIHdpdGggYSByYW5nZSBmcm9tIDUwLTEwMCB1c2luZyB0aGUgInJ1bmlmIiBmdW5jdGlvbiANCg0KTWF0aC5Bdmc8LXJ1bmlmKDQwMCxtaW49NTAsbWF4PTEwMCkNCg0KIyNMZXQncyBjaGVjayB0aGUgc2hhcGUgb2YgdGhlIGRpc3RyaWJ1dGlvbiBhbmQgbm90aWNlIGl0J3Mgbm90IG5vcm1hbA0KaGlzdChNYXRoLkF2ZykgDQoNCiNhbmQgbm93IHB1dCB0aGUgbmV3IHVuaWZvcm4gdGVzdCBpbnRvIHRoZSBkYXRhIHNldCANCkNsYXNzLlVuaTwtY2JpbmQoTWF0aC5Bdmc9TWF0aC5BdmcsQ2xhc3MuUmFuay4yKQ0KDQojYW5kIGNoZWNrIG91ciB3b3JrDQpWaWV3KENsYXNzLlVuaSkNCmBgYA0KDQpBbmQgbm93IGxldCdzIGxvb2sgYXQgb3VyIGNvcnJlbGF0aW9ucywgd2l0aCBvdXIgb3JpZ2luYWwgVGVzdCAyLiAgVGhlc2UgY29ycmVsYXRpb25zIGFyZSBnb2luZyB0byBiZSB2YXN0bHkgZGlmZmVyZW50IHRoYW4gb3VyIHByZXZpb3VzIGNvcnJlbGF0aW9ucy4NCg0KYGBge3IsbWVzc2FnZT1GQUxTRSxlY2hvPVRSVUV9DQoNCiMjbm93IGxldCdzIGRvIHNvbWUgY29ycmVsYXRpb25zIGJldHdlZW4gdGhlIG5ldyB1bmlmb3JtIHRlc3Qgc2NvcmVzIGFuZCB0aGUgb3JpZ2luYWwgdGVzdCAyIHNjb3JlcyANCmNvcihDbGFzcy5VbmkkTWF0aC5BdmcsQ2xhc3MuVW5pJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygic3BlYXJtYW4iKSkNCg0KY29yKENsYXNzLlVuaSRNYXRoLkF2ZyxDbGFzcy5VbmkkVGVzdC4yLA0KICAgIG1ldGhvZD1jKCJwZWFyc29uIikpDQoNCmNvcihDbGFzcy5VbmkkTWF0aC5BdmcsQ2xhc3MuVW5pJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNCldoaWxlIGluIHJlYWxpdHkgaXQgbWF5IG5vdCBiZSB0aGUgY2FzZSB0aGF0IG1hdGggYWJpbGl0eSBhbmQgZW5nbGlzaCAob3IgbGFuZ3VhZ2UsIGdlbmVyYWxseSkgYWJpbGl0eSBhcmUgdGhpcyB1bmNvcnJlbGF0ZWQsIGluIG91ciBoeXBvdGhldGljYWwgd29ybGQgdGhleSBhcmUgdmVyeSB1bnJlbGF0ZWQuICBUaG91Z2ggcGVhcnNvbiBhbmQgc3BlYXJtYW4gbWF5IGJlIGNsb3NlIHRvIG9uZSBhbm90aGVyLCBzcGVhcm1hbiBpcyByZWxpYWJsZSBpbiB0aGlzIGNhc2UgYmVjYXVzZSB0aGUgZGF0YSBpcyBub3Qgbm9ybWFsbHkgZGlzdHJpYnV0ZWQuICBBZ2FpbiwgeW91IGNhbiBzdGlsbCBkbyBhIHBlYXJzb24gY29ycmVsYXRpb24gb24gbm9uLW5vcm1hbCBkYXRhLCBidXQgaXQncyBub3QgZ29pbmcgdG8gYmUgYXMgcmVsYWlibGUgYXMgYSBub24tcGFyYW1ldHJpYyB0ZXN0IHdoaWNoIGRvZXMgbm90IGFzc3VtZSBub3JtYWxpdHkuICBPbiB0aGUgb3RoZXIgaGFuZCwgd2UgY2FuIGFsc28gc2VlIHRoYXQgdGhlc2UgZGF0YSBhcmUgbm90IGxpbmVhcmx5IGRlcGVuZGVudCB1cG9uIG9uZSBhbm90aGVyLCBhcyB0aGUga2VuZGFsbCBjb3JyZWxhdGlvbiBpcyB2ZXJ5IGxvdyBhbHNvLg0KDQpOb3cgbGV0cyByYW5rIG9yZGVyIHRlc3QgMSwgdHVybmluZyBpdCBpbnRvIG9yZGluYWwgZGF0YSwgYW5kIHNlZSB3aGF0IGhhcHBlbnMNCg0KYGBgIHtyLG1lc3NhZ2U9RkFMU0UsZWNobz1UUlVFfQ0KIyNyYW5rIG9yZGVyIHRlc3RzIGJhc2VkIG9uIHRlc3QgMSBzY29yZQ0KTWF0aC5VbmkuUmFuayA8LXJhbmsoQ2xhc3MuVW5pJE1hdGguQXZnLCBuYS5sYXN0PU5BLHRpZXMubWV0aG9kPSJmaXJzdCIpDQoNCg0KIyNhZGQgdGhlIHJhbmsgb3JkZXIgdGVzdHMgdG8gdGhlIGRhdGEgZnJhbWUNCkNsYXNzLlVuaS5SYW5rPC1jYmluZChNYXRoLlJhbms9TWF0aC5VbmkuUmFuayxDbGFzcy5VbmkpDQoNCiNhbmQgY2hlY2sgb3VyIHdvcmsNClZpZXcoQ2xhc3MuVW5pLlJhbmspDQoNCiNub3cgbGV0cyBjb3JyZWxhdGUgdGhvc2UgcmFua3Mgd2l0aCB0ZXN0IDINCmNvcihDbGFzcy5VbmkuUmFuayRNYXRoLlJhbmssQ2xhc3MuVW5pLlJhbmskVGVzdC4yLA0KICAgIG1ldGhvZD1jKCJzcGVhcm1hbiIpKQ0KDQpjb3IoQ2xhc3MuVW5pLlJhbmskTWF0aC5SYW5rLENsYXNzLlVuaS5SYW5rJFRlc3QuMiwNCiAgICBtZXRob2Q9YygicGVhcnNvbiIpKQ0KDQpjb3IoQ2xhc3MuVW5pLlJhbmskTWF0aC5SYW5rLENsYXNzLlVuaS5SYW5rJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNCi0gTm93IHdlIGNhbiBzZWUgdGhhdCB0aGUgY29ycmVsYXRpb25zIGhhdmUgcmVtYWluZWQgYmFzaWNhbGx5IHRoZSBzYW1lLCBzaW1pbGFyIHRvIGFzIHdoZW4gd2UgZGlkIHRoaXMgd2l0aCB0aGUgbm9ybWFsbHkgZGlzdHJpYnV0ZWQgZGF0YS4gIEFnYWluLCB0aGUgc3BlYXJtYW4gKGZvciByZWxhdGlvbnNoaXApIGFuZCBrZW5kYWwgKGZvciBkZXBlbmRlbmNlKSBhcmUgZ29pbmcgdG8gYmUgbW9yZSByZWxpYWJsZSBoZXJlIHRoYW4gcGVhcnNvbi4NCg0KLSBOb3RlIHRoYXQgdGhpcyBkYXRhIChzaW5jZSBpdCB3ZW50IHRvIHNvIG1hbnkgZGVjaW1hbCBwb2ludHMpIGRpZCBub3QgaGF2ZSBhbnkgdGllcy4gIFdoZW4geW91IGhhdmUgZGF0YSB0aGF0IGlzIG9yaWdpbmFseSBpbiB3aG9sZSBpbnRlZ2VycywgdGhlIHJhbmsgZnVuY3Rpb24gaXMgbXVjaCBtb3JlIGltcG9ydGFudCB0byBiZSBhd2FyZSBvZiBpbiBob3cgaXQgaGFuZGxlcyB0aWVzLiANCg0KTGV0J3MgcXVpY2tseSBsb29rIGF0IGhvdyB0aGluZ3MgbWlnaHQgY2hhbmdlIGlmIHRob3NlIHVuaWZvcm0gbWF0aCBzY29yZXMgd2VyZSByb3VuZGVkIHByaW9yIHRvIHJhbmtpbmcNCg0KYGBge3IsbWVzc2FnZT1GQUxTRSxlY2hvPVRSVUV9DQojY3JlYXRlIHRoZSByb3VuZGVkIHZhbHVlcyBmcm9tIHRoZSBvcmlnaW5hbCBtYXRoIHNjb3Jlcw0KTWF0aC5BdmcuUiA8LXJvdW5kKENsYXNzLlVuaSRNYXRoLkF2ZyxkaWdpdHMgPSAwKQ0KI2JpbmQgdGhvc2UgdG8gYSBuZXcgZGF0YXNldA0KQ2xhc3MuTWF0aC5Sb3VuZDwtY2JpbmQoTWF0aC5Sb3VuZD1NYXRoLkF2Zy5SLENsYXNzLlVuaSkNCiN0aGlzIHZpZXdpbmcgaXMgb3B0aW9uYWwNCiNWaWV3KENsYXNzLk1hdGguUm91bmQpDQoNCiNub3cgcmFuayB0aGUgcm91ZG5lZCB2YWx1ZXMNCk1hdGguUmFuay5Sb3VuZCA8LXJhbmsoTWF0aC5BdmcuUiwgbmEubGFzdD1OQSx0aWVzLm1ldGhvZD0iZmlyc3QiKQ0KI2FuZCBiaW5kIHRoZW0gdG8gYSBuZXcgZGF0YXNldA0KQ2xhc3MuTWF0aC5Sb3VuZFJhbms8LWNiaW5kKE1hdGguUm91bmRSYW5rPU1hdGguUmFuay5Sb3VuZCxDbGFzcy5NYXRoLlJvdW5kKQ0KDQojYW5kIG5vdyB2aWV3DQpWaWV3KENsYXNzLk1hdGguUm91bmRSYW5rKQ0KYGBgDQoNCldlIGNhbiBhbHJlYWR5IHNlZSB0aGF0IHRoZSByYW5rZWQgbWF0aCBzY29yZXMgdGhhdCBkZXBlbmQgdXBvbiB0aGUgd2hvbGUgaW5nZXRlcnMgbWlnaHQgY2hhbmdlIHRoZXNlIGNvcnJlbGF0aW9uIHZhbHVlcywgYnV0IGxldCdzIGNoZWNrLg0KDQpgYGB7cixtZXNzYWdlPUZBTFNFLGVjaG89VFJVRX0NCmNvcihDbGFzcy5NYXRoLlJvdW5kUmFuayRNYXRoLlJvdW5kUmFuayxDbGFzcy5VbmkuUmFuayRUZXN0LjIsDQogICAgbWV0aG9kPWMoInNwZWFybWFuIikpDQoNCmNvcihDbGFzcy5NYXRoLlJvdW5kUmFuayRNYXRoLlJvdW5kUmFuayxDbGFzcy5VbmkuUmFuayRUZXN0LjIsDQogICAgbWV0aG9kPWMoInBlYXJzb24iKSkNCg0KY29yKENsYXNzLk1hdGguUm91bmRSYW5rJE1hdGguUm91bmRSYW5rLENsYXNzLlVuaS5SYW5rJFRlc3QuMiwNCiAgICBtZXRob2Q9Yygia2VuZGFsbCIpKQ0KYGBgDQoNClRoZXNlIGNoYW5nZXMgYXJlbid0IGRyYW1hdGljLCBidXQgaW4gdGhlIHJhbmsgcGFja2FnZSwgdGhlcmUgYXJlIDYgZGlmZmVyZW50IHdheXMgdG8gaGFuZGxlIHRpZSB2YWx1ZXMuIElmIHdlIGNoYW5nZSBpdCB0bywgc2F5LCAiYXZlcmFnZSIgZnJvbSAiZmlyc3QiDQoNCmBgYHtyLG1lc3NhZ2U9RkFMU0UsZWNobz1UUlVFfQ0KI2NyZWF0ZSB0aGUgcm91bmRlZCB2YWx1ZXMgZnJvbSB0aGUgb3JpZ2luYWwgbWF0aCBzY29yZXMNCk1hdGguQXZnLlJBIDwtcm91bmQoQ2xhc3MuVW5pJE1hdGguQXZnLGRpZ2l0cyA9IDApDQojYmluZCB0aG9zZSB0byBhIG5ldyBkYXRhc2V0DQpDbGFzcy5NYXRoLlJvdW5kQTwtY2JpbmQoTWF0aC5Sb3VuZD1NYXRoLkF2Zy5SQSxDbGFzcy5VbmkpDQojdGhpcyB2aWV3aW5nIGlzIG9wdGlvbmFsDQojVmlldyhDbGFzcy5NYXRoLlJvdW5kKQ0KDQojbm93IHJhbmsgdGhlIHJvdWRuZWQgdmFsdWVzDQpNYXRoLlJhbmsuUm91bmRBIDwtcmFuayhNYXRoLkF2Zy5SQSwgbmEubGFzdD1OQSx0aWVzLm1ldGhvZD0iYXZlcmFnZSIpDQojYW5kIGJpbmQgdGhlbSB0byBhIG5ldyBkYXRhc2V0DQpDbGFzcy5NYXRoLlJvdW5kUmFua0E8LWNiaW5kKE1hdGguUm91bmRSYW5rQT1NYXRoLlJhbmsuUm91bmRBLENsYXNzLk1hdGguUm91bmRBKQ0KDQojYW5kIG5vdyB2aWV3DQpWaWV3KENsYXNzLk1hdGguUm91bmRSYW5rQSkNCg0KY29yKENsYXNzLk1hdGguUm91bmRSYW5rQSRNYXRoLlJvdW5kUmFua0EsQ2xhc3MuVW5pLlJhbmskVGVzdC4yLA0KICAgIG1ldGhvZD1jKCJzcGVhcm1hbiIpKQ0KDQpjb3IoQ2xhc3MuTWF0aC5Sb3VuZFJhbmtBJE1hdGguUm91bmRSYW5rQSxDbGFzcy5VbmkuUmFuayRUZXN0LjIsDQogICAgbWV0aG9kPWMoInBlYXJzb24iKSkNCg0KY29yKENsYXNzLk1hdGguUm91bmRSYW5rQSRNYXRoLlJvdW5kUmFua0EsQ2xhc3MuVW5pLlJhbmskVGVzdC4yLA0KICAgIG1ldGhvZD1jKCJrZW5kYWxsIikpDQpgYGANCg0KQWdhaW4sIHdlIHNlZSB0aGF0IHRoZXNlIGNoYW5nZXMgYXJlbid0IGRyYW1hdGljLCBidXQgaXQgc2hvd3MgdGhhdCBldmVuIHNtYWxsIGRlY2lzaW9ucyBpbiBob3cgeW91ciBkYXRhIGlzIGhhbmRsZWQgY2FuIGFmZmVjdCB5b3VyIHJlc3VsdHMsIGV2ZW4gd2hlbiB0aGUgYmFzaXMgb2YgeW91ciBkYXRhIGlzIHRoZSBzYW1lLCBhbmQgdGhlIGNvcnJlbGF0aW9uIHlvdSB1c2UgaXMgdGhlIHNhbWUuICBJbiBvdGhlciBzdHVkaWVzLCB0aGlzIG1heSBncmVhdGx5IGltcGFjdCB5b3VyIGludGVycHJldGF0aW9ucyBvZiB5b3VyIGRhdGEuDQoNCiMgQ29uY2x1c2lvbg0KDQpHaXZlbiB3aGF0IHdlIHNlZSBhYm92ZSwgdGhlcmUgYXJlIGEgbnVtYmVyIG9mIHRoaW5ncyB0byBiZSBhd2FyZSBvZiBiZWZvcmUgZ29pbmcgd2l0aCB0aGUgY29tbW9ubHkgdXNlZCBwZWFyc29uIGNvcnJlbGF0aW9ucy4gIEJleW9uZCB0aGUgYXNzdW1wdGlvbnMsIGl0J3MgaW1wb3J0YW50IHRvIGtub3cgaWYgeW91IGFyZSBsb29raW5nIGZvciByZWxhdGlvbnNoaXAgb3IgZGVwZW5kZW5jZSBiZXR3ZWVuIHZhcmlhYmxlcy4gSXQncyBhbHNvIGltcG9ydGFudCB0byBiZSBhd2FyZSB3aGF0IG1heSBoYXBwZW4gdG8geW91ciBjb3JyZWxhdGlvbnMgaWYgeW91IHRyYW5zZm9ybSB5b3VyIGRhdGEgaW50byByYW5rZWQgc2NvcmVzICh0aG91Z2ggdGhhdCB3YXMgbm90IGEgaHVnZSBmYWN0b3IgaGVyZSksIG9yIGhvdyB0d28gZGlmZmVyZW50IGRpc3RyaWJ1dGlvbnMgb2YgZGF0YSBmcm9tIGRpZmZlcmVudCAoaW4gdGhpcyBjYXNlIHN1YmplY3QgYXJlYXMpIGNhbiBpbXBhY3Qgd2hhdCBzdGF0c2l0aWMgeW91ciB1c2UuIFRoZXJlIGFyZSBhIG51bWJlciBvZiBkaWZmZXJlbnQgdGhyZWFkcyBhY3Jvc3MgZm9ydW1zIGRpc2N1c3NpbmcgdGhlIGRpZmZlcmVuY2VzIGJldHdlZW4gdGhlc2Ugc3RhdHNpdGljcyAoZS5nLiBodHRwczovL3N0YXRzLnN0YWNrZXhjaGFuZ2UuY29tL3F1ZXN0aW9ucy8zOTQzL2tlbmRhbGwtdGF1LW9yLXNwZWFybWFucy1yaG8pIGlmIHlvdSBoYXZlIG1vcmUgc3BlY2lmaWMgcXVlc3Rpb25zIHJlZ2FyZGluZyBob3cgdG8gdXNlIHRoZXNlIHN0YXRpc3RpY3Mgd2l0aCB5b3VyIGRhdGEuDQoNCkl0IHRha2VzIGRpbGxpZ2VuY2UgdG8gdXNlIHRoZSByaWdodCBjb3JyZWxhdGlvbiEgDQoNCg==

What is Pearson product

Is Pearson correlation reliability or validity?

What kind of data are suitable for a Pearson's product moment correlation?

What is Pearson moment correlation in research?