How To Transform Data That Is Not Normally Distributed

What to do with not normally distributed Information

Normal Distribution data is required for many statistical tools that assume normality. This folio gives some information most how to bargain with not usually distributed data.

Footstep 1

Do normally check Anderson Darling normality test with a high p value you lot can assume normality of the information. Develve assumes a p value above 0.10 every bit normally distributed. Develve is on the rubber side some people say that 0.05 is plenty to assume normality.

Stride ii

Find out why the data is possible not usually distributed.

Mixture of various distributions

Samples from different batches
Samples from dissimilar dates
Samples form different mold cavities

Try to sort the data in subgroups. This is possible in the DOE mode in Develve.

Example

In this example the data is sorted on the two production lines i and 2 and after sorting the data of the both product lines are normally distributed Column B and C, and the original data is in column A.

Data file
More than data how to sort data in subgroups see hither.

Example 2

Sometimes the indication of a mixture of 2 different distributions is not clearly visible in the histogram but when looking to the normally plot there is a bend in line (meet graph below).

Data file

Farthermost values (outliers)

Too many outliers will consequence in not normality. If the outliers are special causes it wise to filter these data points. Only be enlightened in unremarkably distributed data-set you tin can expect some outliers. In normally distributed information a outlier is non always caused by a special cause.

When filtering the information you should analysis and explain why you can remove these outliers.

Example

In the instance in column B is the filtered data and in column C are the outliers and in cavalcade A is the original data. After filtering the information is usually distributed.

Data file

Drift in measurement system

Wait to the Fourth dimension graph.

Data file

Cases that are not solvable by rearranging the information.

Sorted data

The data prepare is only a office of all the data and all the data outside the tolerance borders is filtered.

Data file
On from left to right: the original data, without the data to a higher place the tolerance border, information without min max tolerance and merely data higher up the upper tolerance.

This can happen when analyzing

Field returns
Line rejects
Information without the rejects

Information is shut to zero or a other limit

Information close to the zero or the optimum will tend to skew to the left.

Low resolution of the measurement

Due low resolution of the measurement the information is rounded to the nearest digit. This leads to data that the data is grouped in small sets see graph. To solve this endeavour to increase the measurement resolution. Utilise the histogram or the individual dot plot see if there is a rounding effect in the data.

Data file

Data is following an other distribution

Lifetime data is oft non normal distributed (clothing out). This data is oftentimes following the Weibull or Lognormal distribution. For this data use Weibull analysis.
Data is close to cypher or a other limit
Proportional data

Example

Utilise the Distribution plumbing equipment role Tools=>Distribution fitting. The graph with the highest Correlation coefficient (r²) is the best fitting distribution.

Data file

Step 3

If the example is not solvable by rearranging the data there are two options. Transform data or use a examination that is not based on a normally assumption.

Transform

With the Box-Cox transformation it is possible to transform non normal distributed data to a more than normal distributed data-set see Box-Cox transformation.

Test not based on normal assumption