How to Find Outliers Meaning, Formula & Examples

It’s important to carefully identify potential outliers in your dataset and deal with them in an appropriate manner for accurate results. By spotting and delivering the correct treatment of outliers, analysts can make sensible decisions and describe their data clearly. Keeping an eye on outliers with proper detection methods is a thoughtful way to make various industry analysis and research claim solid.

• If you are interested in learning more about Statistics and the basics of Data Science, check out this free 8hour University course on freeCodeCamp’s YouTube channel.
• These errors can include typos, incorrect measurements, or unintended mutations of the dataset.
• I give an example of a very simple dataset and how to calculate the interquartile range, so you can follow along if you want.
• Non-parametric statistical tests perform better for these data.
• So, let’s see what each of those does and break down how to find their values in both an odd and an even dataset.

The Interquartile Range (IQR) is the distance between the first and third quartile. Subtract the first quartile from the third quartile to find the interquartile range. From the co-founder of MasterClass, earn transferable college credits from the University of Pittsburgh (a top 50 global school). The world’s best online college courses for 50% less than a traditional college.

This is similar to the choice you’re faced with when dealing with missing data. Your outliers are any values greater than your upper fence or less than your lower fence. You can use software to visualise your data with a box plot, or a box-and-whisker plot, so you can see the data distribution at a glance. This type of chart highlights minimum and maximum values (the range), the median, and the interquartile range for your data. The average is much lower when you include the outlier compared to when you exclude it. Your standard deviation also increases when you include the outlier, so your statistical power is lower as well.

True outliers

It’s best to remove outliers only when you have a sound reason for doing so. This is a simple way to check whether you need to investigate certain data points before using more sophisticated methods. In the same dataset, a mild outlier would fall between 20 and 35. Data points that are moderately different from the rest of the data, falling between 1.5 to 3 times the IQR from the quartiles. One popular method is to declare an observation to be an outlier if it has a value 1.5 times greater than the IQR or 1.5 times less than the IQR.

The lower fence is the boundary around the first quartile. You can convert extreme data points into z scores that tell you how many standard deviations away they are from the mean. Outliers can be handled by removing them, transforming data, or using robust statistical methods that minimize their impact. An outlier is a data point that lies outside the overall pattern of a dataset, significantly differing from other observations.

This means that a data point needs to fall more than 1.5 times the Interquartile range below the first quartile to be considered a low outlier. I give an example of a very simple dataset and how to calculate the interquartile range, so you can follow along if you want. An outlier is the data point of the given sample, observation, or distribution that shall lie outside the overall pattern. A commonly used rule says that one will consider a data point an outlier if it has more than 1.5 IQR below the first quartile or above the third quartile. There are many methods to identify outliers, this outlier calculator uses the following methods.

How to Find Outliers Using the Interquartile Range

They may also use regression, hypothesis testing, and Z-scores to identify outliers. Values that lie in a normal distribution’s extreme right and left tails can be considered outliers. You can use Z-scores to identify outliers in a normal distribution. If you apply the outlier formula, any value in a normal distribution with a Z-score above 2.68 or below -2.68 should be considered an outlier.

You can choose from several methods to detect outliers depending on your time and resources. The outliers are those points in the dataset that are outliers formula very far from other data points. The interquartile range, often abbreviated IQR, is the difference between the 25th percentile (Q1) and the 75th percentile (Q3) in a dataset. To find Q1, you need to take the average of the 2nd and 3rd values of the data set.

There is a non-fiction book ‘Outliers’ written by Malcolm Gladwell that debuted as the number one on the best seller books of the New York Times. Here, Malcolm describes outliers as people with exceptional intelligence, large fortunes, and who are different from the usual set of people. Natural variations in samples can sometimes result in outliers. If a study accidentally obtains an item or person that is not from the target population, it can lead to unusual values in the dataset.

Data Entry Errors

Further, let us apply the Turkey rule to find the outlier. We can now observe how the outlier creates a variation in the mean value of the data. Follow these steps to use the outlier formula in Excel, Google Sheets, Desmos, or R. The first step is to sort the values in ascending numerical order,from smallest to largest number. First, we need to arrange data in ascending order to find the median. Many computer programs highlight an outlier on a chart with an asterisk, and these will lie outside the bounds of the graph.

What Do Subsets Mean in Statistics?

Now, if somebody takes an average of these values, it will be 28.25, but 75% of the observations lie below 7. Hence, one would be an incorrect decision regarding the observations of this sample. It’s a tricky procedure because it’s often impossible to tell the two types apart for sure. Deleting true outliers may lead to a biased dataset and an inaccurate conclusion. Just like with missing values, the most conservative option is to keep outliers in your dataset. Keeping outliers is usually the better option when you’re not sure if they are errors.

How to calculate Q1 in an odd dataset

We’ll use a sample data set containing just 10 data points for this example. The outlier formula — also known as the 1.5 IQR rule — is a rule of thumb used for identifying outliers. Outliers are extreme values that lie far from the other values in your data set. With a large sample, outliers are expected and more likely to occur.

An example is the marks scored by the students in which the student gaining a 100 mark (full marks) is an outlier, which cannot be removed from the dataset. The mean of the data set is sensitive to outliers, so removing an outlier can dramatically change the value of the mean. To calculate to upper and lower quartiles in an even dataset, you keep all the numbers in the dataset (as opposed to in the odd set you removed the median). Some outliers represent natural variations in the population, and they should be left as is in your dataset. If you have a small dataset, you may also want to retain as much data as possible to make sure you have enough statistical power.