9 – Correlation and Regression

Introduction

Once the best estimates are determined, both statistically and epidemiologically, hypotheses about the association between a single mean, proportion, or rate and a fixed value, typically a standard or goal, or between two or more means, proportions, or rates can be tested.

9.1 Measures of Association

Measures of association encompass various coefficients that gauge the strength of a relationship, described in multiple ways. The term “association” in these measures denotes the strength of association where at least one variable is dichotomous, typically nominal or ordinal. These measures define the strength of the linear relationship in terms of monotonicity degree, based on counting various types of pairs in a relationship.

Correlation

Correlation, defined by various experts, implies the degree of association between two or more variables. Notable definitions include:

L.R. Connor: “If two or more quantities vary in sympathy so that movements in one tend to be accompanied by corresponding movements in other(s), then they are said to be correlated.”
A.M. Tuttle: “Correlation is an analysis of covariation between two or more variables.”
Croxton and Cowden: “When the relationship is quantitative, the appropriate statistical tool for discovering and measuring the relationship and expressing it in a brief formula is known as correlation.”
Ya Lun Chou: “Correlation analysis attempts to determine the ‘degree of relationship’ between variables.”

Correlation Coefficient

The correlation coefficient is a numerical measure of the degree of association between two or more variables.

Scope of Correlation Analysis

Correlation between variables indicates that they either increase or decrease together, or an increase (or decrease) in one is accompanied by a corresponding decrease (or increase). However, correlation analysis does not answer questions regarding cause-and-effect relationships between variables.

One Variable Affecting Another: Correlation may indicate a high association between variables but does not reveal causation. Additional information is required to determine causal relationships.
Mutual Interaction: Variables may act upon each other, making it challenging to discern which is independent.
External Influences: Outside factors may influence both variables, leading to a correlation without a clear cause-and-effect relationship.
Pure Chance: High correlation values may occur by coincidence, without any underlying relationship between variables.

Understanding the nuances of correlation analysis is crucial for accurately interpreting the strength and direction of relationships between variables.

9.1.1 Scatter Diagram

A scatter diagram, a scatter plot or scatter graph, represents the relationship between two continuous variables. It consists of a series of data points plotted on a two-dimensional graph, with one variable on the horizontal axis (x-axis) and the other on the vertical axis (y-axis). Each data point represents the value of both variables for a single observation or individual.

Components of a Scatter Diagram

Horizontal Axis (x-axis): Represents one of the variables being studied. It is typically the independent variable or the variable that is manipulated or controlled in an experiment or study.
Vertical Axis (y-axis): Represents the other variable being studied. It is typically the dependent variable or the variable observed or measured to assess the effect of changes in the independent variable.
Data Points: Each data point on the scatter diagram represents the values of both variables for a specific observation or individual in the dataset. The values of the variables it represents determine its position.
Trend Line: In some scatter diagrams, a trend line may be added to show the general direction or trend of the relationship between the variables. The trend line can help identify patterns or correlations in the data.

Interpretation of Scatter Diagrams

Scatter diagrams are helpful for visually identifying relationships, patterns, or correlations between two variables. By examining the distribution of data points on the graph, researchers can make observations and conclude the nature of the relationship between the variables. Here are some common patterns that may be observed in scatter diagrams:

Positive Linear Relationship: Data points are clustered around a straight line that slopes upward from left to right. This indicates that the other variable also tends to increase as one variable increases.
Negative Linear Relationship: Data points are clustered around a straight line that slopes downward from left to right. This indicates that the other variable tends to decrease as one variable increases.
No Relationship: Data points are scattered randomly across the graph, with no apparent pattern or trend. This suggests that there is no systematic relationship between the variables.
Curvilinear Relationship: Data points form a curved pattern on the graph, indicating that the relationship between the variables is not linear. The direction and shape of the curve provide insights into the nature of the relationship.

Uses of Scatter Diagrams

Exploratory Data Analysis: Scatter diagrams are often used in exploratory data analysis to visually inspect the relationships between variables before conducting more formal statistical analyses.
Correlation Analysis: Scatter diagrams can help assess the strength and direction of the relationship between two variables, which helps determine whether a correlation exists and, if so, the nature of the correlation.
Prediction: In some cases, scatter diagrams can be used to make predictions about one variable based on the values of another variable. This is particularly useful when the variables have a strong and consistent relationship.

Scatter diagrams are powerful tools for visualizing relationships between two continuous variables. By plotting data points on a graph and examining the patterns and trends, researchers can gain insights into the relationship between the variables and make informed decisions based on their findings.

9.1.2 Karl Pearson’s Coefficient of Linear Correlation

Karl Pearson’s Coefficient of Linear Correlation, often referred to as Pearson’s Correlation Coefficient (r), is a statistical measure that quantifies the strength and direction of a linear relationship between two continuous variables. It is one of the most widely used methods in statistics to assess the correlation between variables.

The Formula for Pearson’s Correlation Coefficient (r)

The Pearson’s Correlation Coefficient (r) is calculated using the following formula:

Where:

xᵢ: Individual data points for variable X
yᵢ: Individual data points for variable Y
\overline{x}: Mean of the X values
\overline{y}: Mean of the Y values
∑: Summation over all data points
The numerator represents the sum of the products of the deviations of each pair of values from their respective means.
The denominator is the square root of the product of the sum of squared deviations for both x and y.

This formula measures the strength and direction of the linear relationship between two variables.

Step-by-Step Calculation Example:

Consider a dataset with two variables, X and Y, as shown below:

X	Y
2	3
3	4
5	2
7	5
9	7

Step 1: Calculate the Mean of X and Y

To find the mean of X and Y, sum the values and divide by the number of data points (5 in this case):

pearsons

Step 2: Subtract the Mean from Each Data Point

Next, subtract the mean from each data point to get the deviations for X and Y:

X	Y
2	3	-3.2	-1.2
3	4	-2.2	-0.2
5	2	-0.2	-2.2
7	5	1.8	0.8
9	7	3.8	2.8

Step 3: Multiply the Deviations for Each Pair

Now, multiply the deviations for each corresponding pair of X and Y:

X	Y
2	3	3.84
3	4	0.44
5	2	0.44
7	5	1.44
9	7	10.64

Step 4: Sum of the Products

Now, calculate the sum of the products:

Step 5: Square the Deviations and Sum Them

Next, square the deviations for both X and Y, and then sum them:

X		Y	$^{2}$
2	10.24	3	1.44
3	4.84	4	0.04
5	0.04	2	4.84
7	3.24	5	0.64
9	14.44	7	7.84

Sum of squared deviations for X:

Sum of squared deviations for Y:

Step 6: Calculate Pearson’s Correlation Coefficient (r)

Now, use the formula for Pearson’s Correlation Coefficient:

pearsons

So, Pearson’s Correlation Coefficient (r) is approximately 0.762.

Interpretation of Pearson’s Correlation Coefficient:

The value of r ranges from -1 to 1. Here’s how to interpret it:

Value of r	Interpretation
r = 1	Perfect positive linear correlation. As X increases, Y increases.
r = -1	Perfect negative linear correlation. As X increases, Y decreases.
r = 0	No linear correlation. Changes in X do not predict changes in Y.
0 < r < 1	Positive linear correlation. The closer r is to 1, the stronger the positive correlation.
-1 < r < 0	Negative linear correlation. The closer r is to -1, the stronger the negative correlation.

Assumptions of Pearson’s Correlation Coefficient

For the Pearson’s Correlation Coefficient to be valid, certain assumptions must be met:

Linearity: The relationship between the variables should be linear. This means that a change in one variable should result in a proportional change in the other variable.
Bivariate Normality: The data should follow a normal distribution. While Pearson’s correlation can still be calculated with non-normal data, the interpretation may be less reliable.
Homogeneity of Variance (Homoscedasticity): The variance of one variable should be roughly the same at all levels of the other variable.
Continuous Variables: Pearson’s correlation is only appropriate for continuous variables (interval or ratio scale).

Uses of Pearson’s Correlation Coefficient

Pearson’s Correlation Coefficient is widely used in various fields to measure the strength of a linear relationship between two variables:

In Finance: To assess the relationship between the returns of different assets, helping in portfolio diversification.
In Medicine: To explore the relationship between drug dosage and patient response.
In Psychology: To examine the correlation between test scores and certain psychological traits.
In Economics: To analyze the relationship between variables such as income and expenditure, or supply and demand.

Explanation of Pearson’s Correlation Coefficient with Table and Formulation

To further understand Pearson’s correlation, let’s consider a more detailed example with the following data:

X	Y
1	2
2	4
3	6
4	8
5	10

Step-by-Step Calculation:

1.Mean of X and Y:
X‾=1+2+3+4+5 / 5=3Y‾=2+4+6+8+10 / 5=6

2. Deviation from the mean:

X	(xi−X‾)	Y
1	-2	2	-4
2	-1	4	-2
3	0	6	0
4	1	8	2
5	2	10	4

3.Product of deviations:

X	Y	(xi−X‾)×(yi−Y‾)
1	2	8
2	4	2
3	6	0
4	8	2
5	10	8

4. Sum of products:

∑(xi−X‾)×(yi−Y‾)=20

5. Sum of squares:

X	(xi−X‾)2	Y	(yi−Y‾)2
1	4	2	16
2	1	4	4
3	0	6	0
4	1	8	4
5	4	10	16

Sum of squares:

∑(xi−X‾)2=10

∑(yi−Y‾)2=40

6. Calculate $r$ :

PEARSONS

Since r = 1, this indicates a perfect positive linear correlation between X and Y.

Conclusion

Karl Pearson’s Coefficient of Linear Correlation is a powerful tool for measuring the strength and direction of a linear relationship between two continuous variables. By following the formula and ensuring the assumptions are met, one can gain valuable insights into the relationship between variables, aiding in decision-making across various fields. Understanding how to interpret and calculate Pearson’s r is essential for anyone with statistical data.

9.1.3 Properties of Coefficient of Correlation

The coefficient of correlation, such as Pearson’s correlation coefficient (r), measures the strength and direction of the linear relationship between two continuous variables.

Range of Values

The correlation coefficient (r) always falls between -1 and +1, inclusive:
- r = +1: Indicates a perfect positive linear relationship.
- r = -1: Indicates a perfect negative linear relationship.
- r = 0: Indicates no linear relationship between the variables.
Symmetry

The correlation coefficient is symmetric, meaning:

This shows that switching the variables X and Y does not affect the value of the correlation coefficient.

2. Independence of Units

The coefficient of correlation is independent of the units of measurement. The correlation coefficient remains the same whether variables are measured in inches, meters, or any other unit.

3. Interpretation

The sign of the correlation coefficient (r) indicates the direction of the relationship: positive (+) or negative (−).
The magnitude of r indicates the strength of the relationship, with values closer to 1 or -1 indicating stronger correlations.

Table Representation

Consider a dataset with two continuous variables, X and Y as shown in the table below:

Observation	X	Y
1	10	15
2	20	25
3	30	35
4	40	45
5	50	55

Formulation of Pearson’s Correlation Coefficient (r)

To calculate Pearson’s Correlation Coefficient r, follow these steps:

Calculate the Mean of X and Y

First, calculate the mean of each variable

$X = n \sum X and Y = n \sum Y$

For example, with the data:

X	Y
10	15
20	25
30	35
40	45
50	55

X‾=10+20+30
+40+ 505=30

Y‾=15+25+35+45+555=35

Calculate the Deviation from the Mean

For each observation, subtract the mean from the corresponding values of X and Y:

xi−X‾ and yi−Y‾

Observation	X	Y	xi−X‾	yi−Y‾
1	10	15	-20	-20
2	20	25	-10	-10
3	30	35	0	0
4	40	45	10	10
5	50	55	20	20

3. Calculate the Product of Deviations for Each Observation

Multiply the deviations from the mean for each pair of X and Y: (xi−X‾) × (yi−Y‾)

Observation	xi−X‾	yi−Y‾	Product of Deviations
1	-20	-20	400
2	-10	-10	100
3	0	0	0
4	10	10	100
5	20	20	400

Calculate the Squared Deviations from the Mean for Each Variable

Square the deviations from the mean for each variable:

(xi−X‾)2 and (yi−Y‾)2

Observation	(xi−X‾)2	(yi−Y‾)2
1	400	400
2	100	100
3	0	0
4	100	100
5	400	400

5. Sum All the Products of Deviations and Squared Deviations

Calculate the sum of the products of deviations and the sum of the squared deviations:

∑(xi−X‾)(yi−Y‾)=400+100+0+100+400=1000

∑(xi−X‾)2=400+100+0+100+400=1000

∑(yi−Y‾)2=400+100+0+100+400=1000

Calculate Pearson’s Correlation Coefficient (r)

To calculate $r$ , use the following formula:

Conclusion

The Pearson’s Correlation Coefficient $r$ for this dataset is 1, indicating a perfect positive linear correlation between X and Y. This means that as X increases, Y also increases perfectly linearly.

9.1.4 Merits and Limitations of Coefficient of Correlation

Merits of Coefficient of Correlation

Quantification of Relationship: The correlation coefficient provides a numerical measure of the strength and direction of the linear relationship between two variables. This allows researchers to quantify the extent to which changes in one variable are associated with changes in another.
Comparability: It enables comparisons between different pairs of variables, regardless of the units of measurement. This makes assessing relationships between variables measured in other scales or units easier.
Identification of Patterns: Correlation coefficients help identify patterns and trends in data. By examining the sign and magnitude of the correlation coefficient, researchers can determine whether variables are positively, negatively, or not correlated.
Predictive Value: A strong correlation coefficient suggests that changes in one variable can be used to predict changes in another. This predictive value is valuable in various fields like finance, economics, and social sciences.
Statistical Testing: The coefficient of correlation is used in hypothesis testing to determine the significance of the relationship between variables. This allows researchers to assess the reliability of their findings and draw valid conclusions from the data.

Limitations of Coefficient of Correlation

Limited to Linear Relationships: The coefficient of correlation measures only linear relationships between variables. It may not accurately capture non-linear relationships, leading to misinterpretation if the relationship is not linear.
Influence of Outliers: Outliers or extreme values in the data can significantly affect the correlation coefficient, leading to misleading conclusions about the relationship between variables.
Dependence on Sample Size: The reliability of the correlation coefficient depends on the sample size. Small sample sizes may produce unreliable estimates of the actual population correlation.
Directionality vs. Causality: Correlation does not imply causation. A significant correlation between two variables does not necessarily mean one causes the other. Other factors must be considered to establish causality, and further research must be conducted.
Sensitive to Scale: The correlation coefficient is sensitive to changes in the scale or variables’ measurement units. Transforming the units of measurement can alter the correlation coefficient’s magnitude, affecting the relationship’s interpretation.

9.1.5 Probable Error of r

Probable Error of r

The probable error of $r$ (Pearson’s correlation coefficient) is a statistical measure that indicates the variability or uncertainty associated with the estimated value of the correlation coefficient. It provides insight into the range within which the true population correlation coefficient will likely fall.

Formula for Probable Error of r

The formula to calculate the Probable Error (P.E.) of Pearson’s Correlation Coefficient is:

Pearsons

Where:

is the Pearson’s Correlation Coefficient.
is the number of observations in the dataset.

Step-by-Step Calculation

Assuming we have already calculated $r$ and know the number of observations :

Determine and For example, let’s assume (from previous calculations) and
Substitute Values into the Formula

3. Perform the Calculations

Conclusion:

The Probable Error (P.E.) of Pearson’s Correlation Coefficient $r$ estimates the potential error in the correlation coefficient due to random sampling. In this example, the Probable Error is approximately 0.126. This value helps assess the reliability of the correlation coefficient in representing the true relationship between the variables.

9.1.6 Correlation in a Bivariate Frequency Distribution

Consider the following bivariate frequency distribution, where and are two variables and represents the frequency for each pair of and alues:

X	Y	Frequency (f)
2	3	4
3	5	6
4	6	5
5	7	7
6	8	3

Objective: Calculate Pearson’s Correlation Coefficient $r$ for the given bivariate frequency distribution.

Solution:

Step 1: Calculate the Mean of X and Y

To find the means X‾ and Y‾ :

a. Calculate the Mean of X:

X‾=∑(X×f)∑f

Calculate

∑(X×f):

∑(X×f)=(2×4)+(3×6)+(4×5)+(5×7)+(6×3)=8+18+20+35+18=99

Calculate ∑f :

∑f=4+6+5+7+3=25

X‾=99 / 25=3.9

Calculate the Mean of Y:

Y‾=∑(Y×f)∑f }

Calculate

∑(Y×f):

∑(Y×f)=(3×4)+(5×6)+(6×5)+(7×7)+(8×3)=12+30+30+49+24=145

Y‾=14525=5.8

Step 2:Calculate the Product of X and Y with Frequency

Calculate

X×Y×f :

X	Y	f	X×Y×f
2	3	4	2 × 3 × 4 = 24
3	5	6	3 × 5 × 6 = 90
4	6	5	4 × 6 × 5 = 120
5	7	7	5 × 7 × 7 = 245
6	8	3	6 × 8 × 3 = 144

Step 3: Calculate the Sum of

∑(X×Y×f)=24+90+120+245+144=623

Step 4: Calculate the Sum of Frequencies (n)

n=∑f=25

Step 5: Calculate the Sum of Squares for X and Y

a. Calculate

X2×f :

X	f	X2×f
2	4	4 × 2² = 16
3	6	6 × 3² = 54
4	5	5 × 4² = 80
5	7	7 × 5² = 175
6	3	3 × 6² = 108

b. Calculate

Y2×f :

Y	f	Y2×f
3	4	4 × 3² = 36
5	6	6 × 5² = 150
6	5	5 × 6² = 180
7	7	7 × 7² = 343
8	3	3 × 8² = 192

Sum of X2×f:

∑(X2×f)=16+54+80+175+108=433

Sum of Y2×f:

∑(Y2×f)=36+150+180+343+192=901

Step 6: Calculate Pearson’s Correlation Coefficient (r)

Use the formula:

Substitute values:

Calculate ∑X and ∑Y

∑X=(2×4)+(3×6)+(4×5)+(5×7)+(6×3)=8+18+20+35+18=99

∑Y=(3×4)+(5×6)+(6×5)+(7×7)+(8×3)=12+30+30+49+24=145

pearsons

Conclusion

The Pearson’s Correlation Coefficient for the given bivariate frequency distribution is approximately 0.977, indicating a very strong positive linear correlation between and .

Spearman’s rank correlation coefficient is a non-parametric measure of correlation between two variables. It assesses how well the relationship between two variables can be described using a monotonic function. In other words, it measures the strength and direction of association between the ranks of two variables.

Formula:

Spearman’s Rank Correlation Coefficient

ρrhois calculated using the formula:

Where:

is the difference between the ranks of each pair of values.
is the number of observations.

Problem:

Consider the following dataset with two variables, X and Y. Calculate Spearman’s Rank Correlation Coefficient

ρrho.

X	Y
10	20
20	30
30	40
40	50
50	60

Solution:

Step 1: Assign Ranks

First, assign ranks to the values of

X and Y. If the values are already in ascending order, their ranks will be their positions.

X	Y	Rank (X)	Rank (Y)
10	20	1	1
20	30	2	2
30	40	3	3
40	50	4	4
50	60	5	5

Step 2: Calculate Differences

Next, calculate the difference between the ranks of $X$ and $Y$ , denoted as $d$ , and then square the differences

X	Y	Rank (X)	Rank (Y)	d = Rank(X) – Rank(Y)
10	20	1	1	1 – 1 = 0
20	30	2	2	2 – 2 = 0
30	40	3	3	3 – 3 = 0
40	50	4	4	4 – 4 = 0
50	60	5	5	5 – 5 = 0

Step 3: Calculate

Sum the squared differences:

∑d2=0+0+0+0+0=0

Step 4: Calculate Spearman’s Rank Correlation Coefficient

Substitute the values into the formula:

Spearman's Rank Correlation

Conclusion

The Spearman’s Rank Correlation Coefficient ρ for the given dataset is 1, indicating a perfect positive rank correlation between X and. This means that as X increases, Y increases perfectly consistently.

9.1.8 Case of Tied Ranks

In some scenarios, when ranking data, we might encounter tied ranks. Tied ranks occur when two or more observations have the same value. In such cases, we assign them the average rank of the tied values.

Formulation:

When calculating Spearman’s rank correlation coefficient (ρ) in the case of tied ranks, we use the following adjusted formula:

Spearman's Rank Correlation

Example:

Let’s consider an example to illustrate the case of tied ranks and how to adjust the formula for Spearman’s rank correlation coefficient.

Problem:

Suppose we have the following data representing the ranks of two variables, X and Y, for a sample of students:

Student	X(Hours Spent Studying)	Y(Exam Scores)
A	5	7
B	3	4
C	2	3
D	4	6
E	1	2
F	1	2
G	1	2

Calculate the Spearman’s rank correlation coefficient for the given data, considering the tied ranks for students E, F, and G.

Solution:

We’ll adjust the ranks for the tied values and then proceed with calculating ∑ d2 and ρ. Let’s continue with the detailed solution.

Step 1: Assign Ranks

First, we assign ranks to the data for both variables X and Y. If there are tied values, we assign them the average rank.

Student	X(Hours Spent Studying)	Rank of X	Y (Exam Scores)	Rank of Y
A	5	4	7	5
B	3	3	4	3
C	2	2	3	2
D	4	4	6	4
E	1	1.5	2	1.5
F	1	1.5	2	1.5
G	1	1.5	2	1.5

Step 2: Calculate Differences

Next, we calculate the differences between the adjusted ranks of corresponding observations.

Student	Difference in Ranks ()
A	$4 - 5 = - 1$
B	$3 - 3 = 0$
C	$2 - 2 = 0$
D	$4 - 4 = 0$
E	$1.5 - 1.5 = 0$
F	$1.5 - 1.5 = 0$
G	$1.5 - 1.5 = 0$

Step 3: Calculate

Now we sum the squared differences:

Step 4: Calculate Spearman’s Rank Correlation Coefficient

Substitute the values into the formula:

Spearman's Rank Correlation

Conclusion

The Spearman’s Rank Correlation Coefficient, ρ for the dataset with tied ranks, is approximately 0.982, indicating a strong positive correlation between and . Despite the presence of ties, the correlation remains high, reflecting a robust association between the two variables.

9.1.9 Limitations of Rank Correlation

While rank correlation coefficients such as Spearman’s rank correlation coefficient and Kendall’s tau are valuable tools for assessing the relationship between variables, they also have certain limitations:

Assumption of Monotonicity: Rank correlation coefficients measure the strength and direction of association based on the ranks of the data rather than the actual values. They assume that the relationship between variables is monotonic, meaning that as one variable increases, the other variable either consistently increases or decreases. If the relationship is not monotonic, rank correlation coefficients may not accurately reflect the true relationship.
Sensitivity to Outliers: Rank correlation coefficients are less sensitive to outliers compared to parametric correlation coefficients like Pearson’s correlation coefficient. However, extreme outliers can still affect the ranking process and skew the results.
Limited Sensitivity to Non-Linear Relationships: Rank correlation coefficients primarily capture linear relationships between variables. They may not adequately capture non-linear relationships, such as U-shaped or inverted U-shaped relationships, leading to potentially misleading interpretations.
Limited Interpretability: Rank correlation coefficients provide information about the strength and direction of association but do not provide insights into the magnitude of the relationship or the degree of change in one variable associated with a unit change in the other variable. This can limit their interpretability in specific contexts.
Sample Size Requirements: Compared to parametric correlation coefficients, rank correlation coefficients may require larger sample sizes to achieve sufficient statistical power. In small samples, rank correlation coefficients may be less reliable and prone to fluctuations.

Despite these limitations, rank correlation coefficients remain valuable tools in statistical analysis, particularly when the assumptions of parametric correlation coefficients are not met or when dealing with non-parametric data. Considering the data’s specific context and characteristics, it’s essential to consider these limitations and interpret the results of rank correlation analyses cautiously.

9.1.10 Coefficient of Correlation by Concurrent Deviation Method

The Concurrent Deviation Method is a technique used to calculate the correlation coefficient by examining the direction of deviations from the mean of paired data. This method is simpler than calculating the Pearson correlation coefficient and is especially helpful for understanding correlation in datasets.

Formula

The formula for calculating the coefficient of correlation is:

Coefficient of Correlation

Steps to Calculate the Coefficient of Correlation

1.Calculate the Mean of and :

Coefficient of Correlation

2. Compute the Deviations from the Mean:

For each data point, calculate the deviation from the mean for and :

Coefficient of Correlation

3. Calculate the Product of Deviations:

Multiply the deviations of $x$ and $y$ for each pair of data points:

Coefficient of Correlation

4. Compute the Sum of the Products of Deviations:

Sum the products of deviations:

Coefficient of Correlation

5. Calculate the Sum of Squared Deviations for and :

Coefficient of Correlation

6. Substitute Values into the Formula:

Use the values calculated to find the coefficient of correlation:

Coefficient of Correlation

Example Calculation

Data:


1	2
2	3
3	4
4	5
5	6

Step-by-Step Calculation:

1. Calculate the Mean:

Coefficient of Correlation

2. Compute Deviations from the Mean:


1	2	-2	-2
2	3	-1	-1
3	4	0	0
4	5	1	1
5	6	2	2

3. Calculate the Product of Deviations:

4. Calculate the Sum of Squared Deviations:

$\sum (x_{i} - x)^{2} = (- 2)^{2} + (- 1)^{2} + 0^{2} + 1^{2} + 2^{2} = 4 + 1 + 0 + 1 + 4 = 10$

5. Calculate the Coefficient of Correlation:

Coefficient of Correlation

Conclusion

The Coefficient of Correlation calculated using the Concurrent Deviation Method is 1, indicating a perfect positive linear correlation between and . This means that as increases, also increases in a perfectly consistent manner.

Advantages:

Relatively simple method.
Suitable for calculating correlation coefficients between two variables with interval or ratio-level data.
Provides insights into the relationship between variables without making assumptions about their distribution.

Limitations:

Outliers may influence it in the data.
Assumes a linear relationship between variables, which may not always hold true.
It requires calculating means and standard deviations, which can be time-consuming for large datasets.

9.2 Multiple Correlation

Multiple correlation is a statistical technique used to examine the relationship between one dependent variable and two or more independent variables simultaneously. It extends the concept of correlation from two variables to multiple variables, allowing for a more comprehensive analysis of the relationships among variables in a dataset.

Formula:

The formula to calculate multiple correlation (R) between a dependent variable Y and multiple independent variables X1,X2,…,Xn is typically derived from the coefficient of determination (R^2) in multiple regression analysis. It can be expressed as:

Multiple Correlation

Interpretation:

The multiple correlation coefficient (R) represents the strength and direction of the linear relationship between the dependent variable and the set of independent variables collectively. It indicates the proportion of variability in the dependent variable that can be explained by the independent variables taken together.

Example:

Suppose we have a dataset containing information about students’ exam scores (Y) as the dependent variable and their hours spent studying (X1), attendance in class (X2), and previous GPA (X3) as independent variables. By calculating the multiple correlation coefficient, we can assess how well the combination of these independent variables predicts the students’ exam scores.

Advantages:

Provides a more comprehensive understanding of the relationships among multiple variables.
Allows for the assessment of the unique contribution of each independent variable to the dependent variable while controlling for the effects of other variables.
Enables the development of more accurate predictive models compared to single-variable analyses.

Limitations:

Assumes a linear relationship between the dependent and independent variables, which may not always hold true.
Requires careful consideration of potential multicollinearity among independent variables, which can affect the interpretation of results.
May become complex and challenging to interpret when dealing with a large number of independent variables.

9.3 Partial Correlation

Partial correlation is a statistical technique used to measure the strength and direction of the linear relationship between two variables while controlling for the effects of one or more additional variables. It allows researchers to assess the association between two variables of interest after removing the influence of other variables that may be confounding the relationship.

Formula:

The formula to calculate the partial correlation coefficient (rXY.Z) between two variables, X and Y, while controlling for a third variable, Z is given as follows:

Interpretation:

The partial correlation coefficient measures the strength and direction of the relationship between variables and after removing the influence of variable . It indicates how much of the correlation between and is unique to their relationship, independent of the influence of $Z$ .

Example:

Suppose we want to examine the relationship between students’ exam scores (X) and their hours spent studying (Y) while controlling for the effect of attendance in class (Z). By calculating the partial correlation coefficient

rXY.Z, we can assess the association between exam scores and study hours, adjusting for the influence of class attendance.

Advantages:

It helps to clarify the relationship between variables by isolating the specific association of interest.
It allows researchers to account for the effects of confounding variables, providing more accurate estimates of the actual relationship.
Provides insights into the unique contribution of each variable to the relationship of interest.

Limitations:

Assumes a linear relationship between variables, which may not always hold true.
Requires careful selection of control variables to ensure accurate interpretation of results.
It may become computationally complex when analyzing relationships among multiple variables simultaneously.

9.4 Regression Analysis

Regression analysis is a statistical method used to examine the relationship between one dependent variable and one or more independent variables. It aims to model the relationship between the variables and make predictions about the dependent variable based on the values of the independent variables. Regression analysis is widely used in various fields, including economics, finance, psychology, and epidemiology, among others.

Types of Regression Analysis:

Simple Linear Regression: One independent variable and one dependent variable are in simple linear regression. The relationship between the variables is modelled using a straight line.
Multiple Linear Regression: Multiple linear regression involves more than one independent variable and one dependent variable. It allows for examining the combined effect of multiple predictors on the dependent variable.
Polynomial Regression: Polynomial regression is used when the relationship between the variables is not linear but can be approximated by a polynomial function.
Logistic Regression: Logistic regression is used when the dependent variable is binary or categorical. It models the probability of the occurrence of an event based on one or more independent variables.

Key Concepts:

Dependent Variable: The variable that is being predicted or explained by the independent variables. It is denoted as
Independent Variables: The variables used to predict or explain the dependent variable. They are denoted as
Regression Equation: The equation representing the relationship between the dependent and independent variables. It takes the form:

Y=β0+β1X1+β2X2+…+βnXn+ϵ

Where:

$Y$ Is the dependent variable.
are the independent variables.
are the regression coefficients, representing the effect of each independent variable on the dependent variable.
is the error term, representing the difference between the observed and predicted values of the dependent variable.

4. Regression Coefficients: The coefficients

β0,β1,…,βn represent the change in the dependent variable for a one-unit change in the corresponding independent variable, holding all other variables constant.

Applications:

Predicting sales based on advertising expenditure.
Estimating the impact of education on income.
Analyzing the relationship between temperature and crop yield.
Predicting the likelihood of disease based on risk factors.

Advantages:

Provides insights into the relationship between variables.
Allows for the prediction of future outcomes.
Helps in identifying important predictors or factors influencing the dependent variable.

Limitations:

Assumes a linear relationship between variables, which may not always hold true.
Requires the independence of observations and the absence of multicollinearity among independent variables.
May not account for all factors influencing the dependent variable, leading to model misspecification.

9.4.1 Simple Regression

Simple linear regression is a statistical method used to model the relationship between one independent variable (X) and one dependent variable (Y). It assumes that the relationship between the variables can be approximated by a straight line. The goal of simple linear regression is to find the best-fitting line that minimizes the difference between the observed values of the dependent variable and the values predicted by the regression model.

Formula:

The equation for simple linear regression is given by:

Y=β0+β1X+ϵ

Where:

is the dependent variable.
is the independent variable.
is the intercept, representing the value of Y when X is zero.
is the slope coefficient, representing the change in Y for a one-unit change in X
is the error term, representing the difference between the observed and predicted values of Y

The coefficients β0 and β1 are estimated from the data using least squares regression, which minimizes the sum of the squared differences between the observed and predicted values of Y

Example:

Suppose we have a dataset containing information about students’ hours spent studying ( X ) and their exam scores ( Y). We want to use simple linear regression to predict students’ exam scores based on the number of hours they study.

Steps to Perform Simple Linear Regression:

Data Collection: Gather data on the independent variable (X and Y) from a sample of observations.
Data Exploration: Examine the relationship between X and Y visually using a scatter plot to determine if a linear relationship exists.
Model Estimation: Estimate the coefficients $β0beta_0$ and β1beta_1using least squares regression.
Model Evaluation: Evaluate the goodness of fit of the regression model using measures such as the coefficient of determination (
Prediction: Use the fitted regression model to make predictions about the dependent variable () based on new values of the independent variable ().

Graphical Representation:

Below is an example of a scatter plot illustrating the relationship between the independent variable (X) and the dependent variable (Y):

The best-fitting line through the data points represents the regression line, which is determined by the coefficients

β0 and β1 estimated from the regression model.

regression

Application of Formula:

Suppose we have a dataset containing information about students’ hours spent studying (X) and their exam scores (Y). We want to use simple linear regression to predict students’ exam scores based on the number of hours they study.

Using the formula for simple linear regression, we estimate the coefficients β0 and β1 from the data. Once we have the estimated coefficients, we can plug in new values of X to predict the corresponding values of Yusing the regression equation.

For example, if a student studies for 5 hours ( X=5), we can use the regression equation to predict their exam score (Y) as follows:

Y=β0^+β1^×5

where

β0^and β1^ are the estimated coefficients obtained from the regression model.

Advantages:

Provides a simple and interpretable model for understanding the relationship between variables.
Allows for prediction of the dependent variable based on the independent variable.
Can be easily extended to incorporate more complex relationships using polynomial regression or other techniques.

Limitations:

Assumes a linear relationship between the variables, which may not always hold true.
Sensitive to outliers and influential observations that can affect the estimated coefficients and predictions.
Does not account for the influence of other variables on the dependent variable, which may lead to omitted variable bias.

9.4.2 Correlation Coefficient and the two Regression Coefficients

Correlation Coefficient

The correlation coefficient is a statistical measure that quantifies the strength and direction of the linear relationship between two variables. It indicates how closely the data points in a scatter plot cluster around a straight line. The correlation coefficient, denoted by r, ranges from -1 to +1:

If , it indicates a perfect positive linear relationship, meaning that as one variable increases, the other variable also increases proportionally.
If , it indicates a perfect negative linear relationship, meaning that as one variable increases, the other variable decreases proportionally.
If , it indicates no linear relationship between the variables.

Regression Coefficients

Regression coefficients are parameters that represent the relationship between the independent variable(s) and the dependent variable in a regression model. In simple linear regression, there are two regression coefficients:

Intercept (): The intercept represents the value of the dependent variable when the independent variable is zero. It determines the point where the regression line intersects the y-axis.
Slope (): The slope coefficient represents the change in the dependent variable for a one-unit change in the independent variable. It determines the steepness of the regression line.

Relationship between Correlation Coefficient and Regression Coefficients

Correlation Coefficient (): The correlation coefficient measures the strength and direction of the linear relationship between two variables, regardless of their roles as independent or dependent variables.
Regression Coefficients ( and β1): The regression coefficients quantify the relationship between the independent and dependent variables within a regression model. They represent how changes in the independent variable(s) affect the dependent variable.

Interpretation:

In simple linear regression, the correlation coefficient () is equal to the product of the slope coefficient ( β1) and the ratio of the standard deviations of the dependent and independent variables.r=Cov(X,Y) / σX×σY=β1×σX / σY
The sign of the correlation coefficient () is the same as the sign of the slope coefficient (β1), indicating the direction of the relationship.
The magnitude of the correlation coefficient () reflects the strength of the linear relationship, while the slope coefficient (β1) indicates the rate of change in the dependent variable for a one-unit change in the independent variable.

While both correlation coefficient and regression coefficients provide valuable insights into the relationship between variables, they serve different purposes: correlation coefficient quantifies the strength and direction of the relationship, while regression coefficients describe the specific impact of independent variables on the dependent variable within a regression model.

Let’s consider a hypothetical scenario where we have data on students’ study hours and their corresponding exam scores. We want to explore the relationship between study hours (independent variable) and exam scores (dependent variable) using correlation coefficient and regression coefficients.

Here’s a table representing the data:

Study Hours (X)	Exam Scores (Y)
2	60
3	70
4	75
5	80
6	85
7	90
8	95
9	100

Now, let’s calculate the correlation coefficient (r) and regression coefficients (β0 and β1) based on this data.

Calculations:

Correlation Coefficient ( $)$ :

Using the given data:

n=8

∑X=2+3+4+5+6+7+8+9=44

∑Y=60+70+75+80+85+90+95+100=655

∑X2=22+32+42+52+62+72+82+92=204

∑Y2=602+702+752+802+852+902+952+1002=53,350

∑XY=(2∗60)+(3∗70)+(4∗75)+(5∗80)+(6∗85)+(7∗90)+(8∗95)+(9∗100)=5,880

Now, plug these values into the correlation coefficient formula to calculate r.

Regression Coefficients ( and β1):

Regression Coefficients

Conclusion:

Once we have calculated the correlation coefficient and regression coefficients, we can interpret their values to understand the relationship between study hours and exam scores. The correlation coefficient will indicate the strength and direction of the relationship, while the regression coefficients will provide specific information about the impact of study hours on exam scores, including the intercept and slope of the regression line.

9.4.3 Non-parametric Regression

Non-parametric regression is a statistical method used to model the relationship between variables without making explicit assumptions about the functional form of the relationship. Unlike parametric regression models, which assume a specific functional form (such as linear or polynomial), non-parametric regression allows the data to dictate the shape of the relationship.

Key Features of Non-parametric Regression:

Flexibility: Non-parametric regression is highly flexible and can capture complex relationships between variables without imposing a predetermined functional form. This makes it suitable for modeling relationships that may not follow a linear or polynomial pattern.
No Assumptions: Unlike parametric regression models, non-parametric regression does not assume a specific distribution or form for the underlying relationship. It allows the data to speak for itself and does not require the researcher to specify a particular model structure.
Local Estimation: Non-parametric regression often employs local estimation techniques, such as kernel smoothing or local polynomial regression. These methods estimate the relationship between variables based on data points in the vicinity of the point of interest, rather than fitting a single global function to the entire dataset.
Robustness: Non-parametric regression methods are often more robust to outliers and deviations from model assumptions compared to parametric regression models. They can adapt to irregularities and heteroscedasticity in the data without sacrificing performance.
Interpretability: Non-parametric regression models may offer less straightforward interpretation compared to parametric models, especially when dealing with high-dimensional data or complex relationships. However, they can still provide valuable insights into the underlying patterns in the data.

Techniques Used in Non-parametric Regression:

Kernel Smoothing: This technique involves assigning weights to nearby data points based on their distance from the point of interest. The weighted average of these points is used to estimate the value of the dependent variable at that point.
Local Polynomial Regression: In local polynomial regression, a low-degree polynomial is fitted to a small subset of data points centered around the point of interest. This allows the regression function to vary smoothly across the range of the independent variable.
Splines: Splines are flexible non-parametric regression techniques that use piecewise polynomials to approximate the relationship between variables. They offer a compromise between flexibility and interpretability.
Smoothing Splines: Smoothing splines use penalized regression techniques to balance goodness of fit and smoothness of the estimated curve. They automatically adjust the degree of smoothing based on the data.

Advantages of Non-parametric Regression:

Flexibility to capture complex relationships without assuming a specific functional form.
Robustness to outliers and deviations from model assumptions.
Ability to adapt to irregularities and heteroscedasticity in the data.

Limitations of Non-parametric Regression:

Generally requires larger sample sizes compared to parametric regression models.
May be computationally intensive, especially for large datasets or high-dimensional problems.
Interpretation of results may be less straightforward compared to parametric models.

Non-parametric regression methods do not typically rely on explicit mathematical formulas like those in parametric regression models. Instead, they utilize algorithms and techniques that allow the data to determine the shape of the relationship between variables. However, some non-parametric regression techniques involve specific mathematical operations for estimation. Here are a few commonly used formulas in non-parametric regression:

1. Kernel Smoothing Formula:

Kernel Smoothing Formula:

2. Local Polynomial Regression Formula:

f^(x)=β0(x)+β1(x)(x−x0)+β2(x)(x−x0)2+…+βd(x)(x−x0)d Where:

$f^(x)$ is the estimated function at point x.
is the point of interest.
are the local polynomial coefficients, which are estimated using weighted least squares.
is the degree of the polynomial.

3. Smoothing Splines Formula:

f^(x)=∑j=1nβjBj(x)+λ∫[f^′′(t)]2dt

Where:

$f^(x)$ is the estimated function at point x.
are the coefficients of the basis functions Bj(x).
is the smoothing parameter, controlling the trade-off between smoothness and goodness of fit.
The last term is the penalty term that penalizes the curvature of the estimated function.

These formulas represent just a few examples of non-parametric regression techniques. Each method may have variations in formula depending on the specific implementation and algorithm used. The goal of non-parametric regression is to estimate the underlying relationship between variables without imposing a predetermined functional form.

Curriculum

Research Methodology

9 – Correlation and Regression

Introduction

9.1 Measures of Association

Correlation

Correlation Coefficient

Scope of Correlation Analysis

9.1.1 Scatter Diagram

Components of a Scatter Diagram

Interpretation of Scatter Diagrams

Uses of Scatter Diagrams

The Formula for Pearson’s Correlation Coefficient (r)

Step-by-Step Calculation Example:

Step 1: Calculate the Mean of X and Y

Step 2: Subtract the Mean from Each Data Point

Step 3: Multiply the Deviations for Each Pair

Step 4: Sum of the Products

Step 5: Square the Deviations and Sum Them

Step 6: Calculate Pearson’s Correlation Coefficient (r)

Interpretation of Pearson’s Correlation Coefficient:

Uses of Pearson’s Correlation Coefficient

Explanation of Pearson’s Correlation Coefficient with Table and Formulation

Step-by-Step Calculation:

9.1.3 Properties of Coefficient of Correlation

Range of Values

Symmetry

Calculate Pearson’s Correlation Coefficient (r)

Conclusion

9.1.4 Merits and Limitations of Coefficient of Correlation

Merits of Coefficient of Correlation

Limitations of Coefficient of Correlation

9.1.5 Probable Error of r

Probable Error of r

Formula for Probable Error of r

Step-by-Step Calculation

9.1.6 Correlation in a Bivariate Frequency Distribution

Step 1: Calculate the Mean of X and Y

To find the means X‾ and Y‾ :

Step 2:Calculate the Product of X and Y with Frequency

Step 3: Calculate the Sum of X×Y×f

∑(X×Y×f)=24+90+120+245+144=623

Step 4: Calculate the Sum of Frequencies (n)

Step 5: Calculate the Sum of Squares for X and Y

Step 6: Calculate Pearson’s Correlation Coefficient (r)

Conclusion

9.1.7 Spearman’s Rank Correlation

Problem:

Solution:

Step 1: Assign Ranks

Step 2: Calculate Differences

Step 3: Calculate ∑d2

Step 4: Calculate Spearman’s Rank Correlation Coefficient ρrho

Conclusion

9.1.8 Case of Tied Ranks

Formulation:

Example:

Problem:

Solution:

Step 1: Assign Ranks

Step 2: Calculate Differences

Step 3: Calculate ∑d2

Step 4: Calculate Spearman’s Rank Correlation Coefficient ρ

Conclusion

9.1.9 Limitations of Rank Correlation

9.1.10 Coefficient of Correlation by Concurrent Deviation Method

Formula

Steps to Calculate the Coefficient of Correlation

Example Calculation

Step-by-Step Calculation:

Conclusion

Limitations:

9.2 Multiple Correlation

Formula:

Interpretation:

Example:

Advantages:

Limitations:

9.3 Partial Correlation

Formula:

Step 3: Calculate the Sum of

Step 3: Calculate

Step 4: Calculate Spearman’s Rank Correlation Coefficient

Step 3: Calculate

Step 4: Calculate Spearman’s Rank Correlation Coefficient