Attribute Gage R&R – A Step by Step approach

In this post, we will look at the step by step approach to perform Attribute Gage R&R. It is also referred to as Discrete Gage R&R. We will also explore the results of Attribute Gage R&R and how to interpret these results.

In my earlier post, we looked at what Measurement System Variation means, how it can creep into your process data and what steps you should take to ensure that your data is free of such variation. We also looked at an overview of Measurement System Analysis (MSA). MSA is done to ensure that your measurement system is robust, precise and accurate. Please click on the links to read these posts (opens in new tab). We will build on the examples we discussed in these posts.

When you have Discrete data set, you perform Attribute Gage R&R. In our example, the Acceptability data set of the wooden planks is discrete. Hence, we will use this data to perform Attribute Gage R&R. If your data set consists of continuous data, you will need to perform Continuous Gage R&R. Please remember, it is extremely important to do the correct MSA depending on the type of your data. Else, these tests will be of no use.

Quick recall. Acceptability as a metric is the combination of color, smoothness, surface texture and other desired attributes of the wooden plank. Your quality control associates determine the conformance to these attributes by physically inspecting the wooden plank and decide on acceptability (Yes or No).

Key Assumptions

Before we do the Attribute Gage R&R, you should have completed all the pre-work as discussed in Measurement System Analysis (MSA) overview post. Which means, you have selected the correct parts for performing MSA, you have numbered the parts, you have identified the operators for this test, you have selected the right measuring equipment and you have the data collection template ready. Below is a simple data collection template that we will use.

We will also assume that you already have the standard or accurate values for each of these 10 planks, as confirmed by your SME or one of your customer. (These steps are discussed at length in Measurement System Analysis Overview post).

We will use 10 selected parts, which will be measured / evaluated by 3 operators, each of whom will do 2 trials for each part.

Steps to do Attribute Gage R&R

Step 1 : Operator A evaluates each of the 10 selected wooden planks and decides if it’s acceptable or not. He records his findings in the first column – Operator A – Trial 1.

Step 2 : Give these 10 parts to operator B. He records his findings in the 3rd column – Operator B – Trial 1.

Step 3 : Operator C repeats the same exercise and records his findings on acceptability of the wooden planks in fifth column – Operator C – trial 1.

Once the above 3 steps are done, your data collection template will look somewhat similar to this;

Step 4 : Repeat step 1 to 3 in the same order with the same parts and same operators for trial 2. Ensure that you do not let the operators see each others results as well as their own trial 1 results while doing this exercise as it may influence their decisions.

Your data collection template now looks like below;

Step 5 : Now you have the results of both the trails for each operator for all 10 parts. Include the standard or accurate response in the last column. You get this standard or accurate response through your customer or your most experienced subject matter expert.

Attribute Gage R&R – Test

Now that you have the required data with you, its time to do the test.

You can use any statistical analysis software, or an MSA macro enabled template that you can easily download from the internet, to do the test. I will use MiniTab, one of the preferred statistical analysis software for Lean Six Sigma practitioners. However, the results and the takeaways will stay the same irrespective of what software / template you use.

You need to structure the the data in a specific format to use it in MiniTab. You need to put this whole data into 4 columns. The first column is for Part numbers, second for Operator names and third for results. The fourth column should have the standard values for each part. It will look similar to the image below. Copy this data set into Minitab worksheet.

The MiniTab test for Attribute Gage R&R is called Attribute Agreement Analysis. The path for this test in MiniTab is as shown below – 

Stat > Quality Tools > Attribute Agreement Analysis.

Attribute Agreement Analysis Command Box

Once you click on the above test, the following command box appears.

In the command box, you see 4 fields to provide inputs to MiniTab. These are the inputs that MiniTab will use to analyze the data that you have collected and give you the results.

The first field is for “Attribute Column”. This refers to the column where you have the Acceptability results stored. This is your Results column. To provide the input, click on the text box in front of Attribute Column. All the eligible column names will appear in the white box on left hand side of the command box. Double click on “Results”. It will appear in the text box next to Attribute Column. Illustration below;

Similarly, provide input to MiniTab for Samples, which is your ‘Parts’ column and for Appraisers which is your ‘Operator’ column. Additionally, select the ‘Standard’ column for Known Standard/ attribute field. Once you provide the correct input in all the fields, click on OK to run the test.

Lets look at the results of Attribute Agreement Analysis and see how to interpret the same.

By the way, do check out the Certified Lean Six Sigma Black Belt Handbook – it is one of the most essential guide for anyone trying to get certified as LSS Black belt or in general wants to understand LSS and improve processes. – check it out here.

Attribute Gage R&R – Results

You get MiniTab test results in 2 forms, a textual output and a graphical output. Below are both the results;

Do not get confused or overwhelmed by so many numbers and statistics thrown at you by MiniTab. What you need to check in these results is the Percent Agreements. As a ground rule, everything that has an Agreement of 90% or more is good, everything that has an Agreement of less than 90% is bad.

Simply put, for you to accept your measurement system, the within appraiser, between appraiser and appraiser vs standard agreements have to be 90% or above. In such cases you conclude that you have the right measurement system in place and proceed with your data collection. Let us look at each of these parameters;

Within Appraisers

The first block in the text output talks about Within Appraiser agreement assessment. This is a comparison of the results of each trial of a single operator and shows if the operators are able to repeat their own results across multiple trials. This is called Repeatability.

As you can see in the below results, Appraiser A and B repeats his results for 9 out of 10 parts across trials. Hence it shows as 90%. Appraiser C is able to repeat his results across all trails for all parts and hence the assessment agreement is at 100%.

Overall, all appraiser assessment agreement stands at 90% or more and hence is acceptable. This means, your measurement system does not have Repeatability issues. If any one or more percentages drop below 90%, we will conclude that our measurement system has Repeatability issues and the MSA will be rejected. In such cases, we will need to further coach the appraiser on correctly assessing the parts for Acceptability and redo the MSA.

Each Appraiser Vs Standard

In this section, the output for each appraiser across trials is compared with the Standard response and agreement %age is displayed. This is Accuracy of the Measurement System.

In our results, Appraiser A and B provides the same results as the Standard for 8 out of 10 parts, in all trails (80%). Appraiser C, however, provides the same response as the Standard for all 10 parts across all trails. This means, Appraiser C is able to correctly identify the planks with conformance to Acceptability whereas Appraisers A and B fail to do so in 20% of the cases.

Since the agreement percentage drops below 90% for 2 appraisers, we will reject this measurement system, correct any disconnects that Appraisers A and B have and redo the MSA till the percentage reach above 90%.

Between Appraisers

This is the comparison of results of each part across all operators and all trials. Ideally, all appraisers should give the same results for each part across all trials. This is called Reproducibility.

Related Post : What are the measures of Variation?

As you can see, out of 10 parts, all the appraisers were able to match the results only for 8 parts in all the trials. This is below 90% and hence is an issue. We have Reproducibility problem in our Measurement System. This needs to be corrected before we can use this measurement system to collect data.

All Appraisers Vs Standard

The previous comparison was between each appraiser vs standard whereas this test compares the result of all appraiser collectively with the standard. That means, it will show a particular part as matched only when all appraisers across all trails give the same result for the part as the standard.

In the below result, the above is true only for 8 out of 10 parts, 80% matched. Since this is below 90%, the measurement system is not good to proceed and needs to be corrected.

What if the Attribute Gage R&R is not acceptable

As discussed earlier, if any of the above assessment agreement %age is below 90%, we say that the measurement system is not good. Or the MSA failed. What do we do in such cases?

Well, is quite simple and obvious. At the highest level, there are just 2 reasons why the MSA can fail. Either the equipment used is not correctly calibrated or the operators are not able to read the equipment correctly and precisely.

For Attribute Gage R&R, the issue mostly lies with the operators not able to evaluate / calculate the required attributes correctly. What you need to do is further refine the operation definition of the said attribute (in this case, Acceptability of the wooden plank) and make is more robust for everyone to understand. The definition should be crystal clear leaving no doubt about the metric in the operators mind.

Next, coach the operators. Ensure that they all are on the same page with regards to the attribute, what is acceptable, what is not. Once this is done, repeat the whole MSA again and check the agreement percentage. You will need to keep repeating this process till the agreement percentage for all assessments above reach 90% or more.

That is all about Attribute Gage R&R.

In the next post, we will look at Continuous Gage R&R.

Thank you for reading. Do let me know what you think about this post or if you have any questions / suggestions in the comment section below. Happy to hear from you.

Sachin Naik

Passionate about improving processes and systems | Lean Six Sigma practitioner, trainer and coach for 14+ years consulting giant corporations and fortune 500 companies on Operational Excellence | Start-up enthusiast | Change Management and Design Thinking student | Love to ride and drive

Read More about Sachin