Measurement System Analysis (MSA) overview

In this post, we will talk about Measurement System Analysis, commonly referred to as MSA. We will also look at which Measurement System Analysis you should perform depending on the type of your Lean Six Sigma project data. And we will talk about the important terms in an MSA along-with the pre-work needed before you can do a measurement system analysis. Read on.

In my previous posts, I spoke about what variation in the processes is and how you can identify the same. I also spoke in details regarding the types of variation in the process, specially about Measurement System variation. Please click on the links to read regarding these topics (opens in a new tab) to get the context. I will build on from there.

Introduction to Measurement System Analysis

We established the steps to ensure measurement system variation does not get into our Lean Six Sigma process data in the post regarding measurement system variation. As stated, we need to define the data requirement precisely, establish a robust data collection plan, build data collection templates and train the operators. The next step is to collect data using these operators, plans and templates. However, before you go full guns blazing on data collection, take some time and test your plan by doing Measurement System Analysis (MSA). It is a strongly recommended step to get optimum results from your Lean Six Sigma project and you should not ignore it.

Measurement System Analysis results will tell you if your way of data collection is good enough. In other words, it will tell you the possibility of measurement system variation creeping into your data. Lower this possibility, the better data you will have.

To perform MSA, you will collect sample data using the already established data collection plan and templates, the calibrated measurement equipment and the trained operators. You should collect data for 10 parts or transactions as a standard practice. You should use 3 operators and each operator should have minimum of 2 trials for each part.

There are various studies which show that increasing the number of parts, operators or trials do not have any statistically significant impact on your conclusions. It does, however, significantly increase your efforts. Hence, stick to the 10 parts, 3 operators and 2 trials rule for the time being.

Let us also look at a scenario where you will need to do an MSA. An example works better for understanding.

Short case study to better understand MSA

Assume you are in the business of manufacturing wooden planks. You have specific requirements around the length, width, height, color, smoothness, surface texture and other such attributes. However, the most important parameter for you is the length, which has to be 150 centimeters. Second important attribute or metric is the combination of color, smoothness and surface tension. The quality control associates of your factory determines conformance to this attribute by physically inspecting the finished product.

To summarize, you have 2 key metrics or CTQs (critical to quality). First is length. Lets call second as Acceptability. Length will be measured in centimeters whereas Acceptability will be measured as Yes or No.

You wish to measure these metrics to track your performance. Accordingly, you have already created the data collection plan and put together the templates required to collect this data. You have also trained everyone in your factory on the procedure. As the next step, you need to do Measurement System Analysis and test your measurement system.

Pre-work before Measurement System Analysis

Before you can actually go to Minitab (or any other statistical analysis software which can perform MSA), you will need to do some preparation.

1. Select the parts for MSA

First, you need to select the 10 parts or units that you will use for Measurement System Analysis. These parts should be selected in such a way that your actual process variation is appropriately represented in this sample. Do not pick 10 consecutive planks coming out of the production line. Such sample will not give you any meaningful insights on your measurement system. Rather, pick planks from across the day, produced at different time of the day or maybe from multiple days. Maybe, 1 from Monday morning and another from Friday evening. One from before the lunch break another from after the lunch break. Randomize it as much as you can. You get what I mean, right!

Related Post : What are the measures of Variation?

2. Number the selected parts

Second, number these parts. Put a unique identifier for each plank which only you know. You will need to collect multiple data points for each plank and hence its important and critical to be able to identify each plank.

3. Identify operators

Third, identify 3 operators who will participate in this MSA. They should be from the group of people who help you collect data and calculate your metrics on a regular basis.

4. Select measuring equipment

Fourth, select the measurement equipment that you will use for this MSA. For measuring the length of the plank, you will have a measuring tape. Ensure that the same measuring tape is used for the entire MSA study. However, for measuring Acceptability, you wont need a measuring equipment. As stated earlier, acceptability of a plank is decided by the operator who physically inspect the plank. So, the three operators that you selected will suffice.

5. Create data collection template

Fifth, put together the template required to collect data for measurement system analysis. You will need 2 templates, one for length and another for acceptability. It is a simple template. It should be able to record the metric for each plank, by operator and by trial. Below is a picture of how this template should look like.

Now you are ready to do the measurement system analysis for length and acceptability of your wooden planks. However, before you jump there, you should understand a few important terms and concepts related to MSA.

Important terms to know for Measurement System Analysis

To understand and interpret MSA results correctly, you need to know a few terms. Lets take some time and talk about these briefly.

1. Type of data

The Measurement System Analysis test that you will perform depends on the type of the data that you have.

You will do an Attribute Gage R&R if you have discrete data set. In our scenario, the data collected for acceptability of the wooden plank will always have either Yes or No. This is binary discrete data type. So, for MSA on Acceptability, you will do the Attribute Gage R&R.

Length data of the wooden plank is continuous data. Hence, you will do continuous Gage R&R on this data set. Continuous Gage R&R is also commonly referred to as just Gage R&R.

2. Accuracy (or Bias)

Accuracy or Bias refers to the the difference between what your operators record versus the actual or true value of the part / transaction.

In an MSA, you compare each measured value of each part with a defined standard or true value. You measure the difference of each such value from the standard. You need to define such standard or true value for each part before you perform the MSA. This has to be as defined by the customer or your subject matter expert.

Back to our case study. Acceptability is essentially what your customer will accept as a good wooden plank. That is your gold standard. You should get one of your customers or vendor to evaluate all the selected 10 planks. Record the observations on if it is an acceptable plank or not. This becomes your true value or standard.

And for length, get your most experienced SME to measure each of the 10 selected planks and record the readings. You can strive to be more accurate and get the lengths measured by a more precise digital instrument. These readings are the true values or standard for length.

Ensure that you keep the results to yourself and do not share the same with the operators till the end of MSA.

Accuracy is influenced by linearity, resolution and stability. Let us also understand what these terms mean.

By the way, do check out the Certified Lean Six Sigma Black Belt Handbook – it is one of the most essential guide for anyone trying to get certified as LSS Black belt or in general wants to understand LSS and improve processes. – check it out here.

i. Linearity

Simply put, linearity refers to the consistency of inaccuracy (or Bias) in your measurement.

Lets assume you have a plank with true length of 150 cm. However, every operator in each of his/her trial reports the length as 155 cm consistently. These measurements are off-course not accurate. They have a bias of 5 cm. However, this bias or inaccuracy, is consistent. Which means that the bias is linear. A perfectly linear bias is good for a measurement system because it is easy to correct the same by some re-calibration of the measuring equipment. But, if the bias is non-linear, it becomes a problem and is a bit more difficult to resolve.

Best case scenario is perfect linearity between your measured and true values with zero bias. Next best case is perfect linearity with constant bias between measured and true values. Everything else is a problem.

ii. Resolution

Resolution refers to the lowest unit of measurement on your measuring instrument. In other words, it is the ability of your measurement equipment to efficiently differentiate between measurement values.

For example, a normal measuring tape will have meters, centimeters and millimeters marked on it. The smallest unit at which this tape can measure is millimeters. That is the resolution of your measurement equipment.

However, it does not mean that this tape can easily differentiate between any increase or decrease in the length of the plank in increments of millimeter. The operator might still read it as 150.2 cm or 150.4 cm for an actual reading of 150.3 cm. This ability (or inability) to effectively differentiate between any incremental increase or decrease is referred to as sensitivity.

Your measurement instrument needs to have a resolution of atleast 1/10th of your tolerance limits. What does this mean? Lets say that for the wooden plank, your acceptable tolerance limit is of +/- 1 cm. This means, a plank with length of 149 cm to 151 cm will still be acceptable to your customers. So, a 1/10th of 1 cm resolution means the measuring tape you use should have millimeters marked on it. If the tolerance limit is +/- 5 millimeters, then you will need to use a scale which can differentiate in decimeters and not just millimeters.

iii. Stability

Stability refers to the data which do not have any variation due to special causes. Please refer to the posts on variation to understand special cause variation and common cause variation. For you to get the correct results from the MSA, your data should be stable, without any special cause variation.

3. Precision

Precision refers to the ability of your measurement system, to produce the same result (accurate or inaccurate), for the same part, measured using the same measurement instrument, multiple times.

For example, lets say I measure plank no 1 with a measuring tape today and record the length as 150.2 cm. Next time, maybe after a couple of hours, I am again measuring the same plank with the same measuring tape again. This time, I should be able to record the length as 150.2 cm again, right! For that matter, even if you measure the same part with the same measuring tape, you should be able to record the length as 150.2 cm. If you or I come up with a different length reading the second time, we have a precision problem in our measurement system.

Precision is influenced by Repeatability and Reproducibility of your measurement system, that is, by your measurement equipment and your operators.

i. Repeatability

Repeatability refers to the ability of the same operator to produce the same results for the same part measured using the same measuring equipment multiple times. If an operator is not able to record the same length in multiple trials for the same part, you have a repeatability issue. You will need to further coach and train the operator on how to read the length.

ii. Reproducibility

Reproducibility refers to the ability of the different operators to produce the same results for the same part measured using the same measuring equipment multiple times. If the lengths measured by different operators for the same part with the same measuring tape are not the same, you have a reproducibility issue. You will need to further calibrate your operators such that all of them have the same understanding of how to read the length.

Great! Now you understand what Measurement System Analysis or MSA is, why it needs to be performed and its importance. You also know about all important terms and concepts related to MSA and the pre-work that you need to do before doing an MSA.

As stated earlier, you will do an Attribute gage R&R for variable or discrete data and a continuous gage R&R for continuous data set. Please read through my detailed posts on discrete gage r&r and continuous gage r&r to know the steps for performing these tests using the same data from our case study and to understand how to interpret the test results.

Also, do read my short post on Precision and Accuracy. These are the 2 concepts that merits a detailed post and need to be understood well in your Lean Six Sigma journey. Because, in most projects, you will either end up improving the accuracy or the process or the precision of the process. Sometimes both.

Do let me know your thoughts on this post in the comments below.

Sachin Naik

Passionate about improving processes and systems | Lean Six Sigma practitioner, trainer and coach for 14+ years consulting giant corporations and fortune 500 companies on Operational Excellence | Start-up enthusiast | Change Management and Design Thinking student | Love to ride and drive

Read More about Sachin