This is a question we struggle with regularly. How do we measure color? What is too much variation?
It helps to quantify, but you have a real problem when the definition of what makes a quality part (or service) is a moving target.
We have been producing and validating products for a customer using a "Form, Fit, Function" approach. This has been the approach since we took on the business over two years ago. Now we are validating design changes requested by the customer, the parts meet our historical standards. We sent samples, which were approved. Now we have gone to production and our production runs are being rejected. The customer is now inspecting these parts, not just to whether they fit their assemblies, but to the engineering drawings. The target has shifted. Our definition of quality is no longer the same as theirs.
Our problem stems from a lack of communication between our engineering group and the customer. We are not defining our customer requirements or expectations at the outset. We are relying on our historical knowledge of the parts to set our standards. Sometimes it is working, other times it is not.
We recognize the gap and are redefining our validation process. We have a series of meetings scheduled with the customer to discuss the process and the help we will need from them in defining what will constitute a quality product for them in the future. After all this whole endeavor comes back to what the customer wants. Hopefully the meetings will go well and we won't lose the business.
This improvement will be painful for us. We are going to scrap thousands of dollars in product. We are going to admit we had flaws in our process. But we are going to fix them. We are going to improve. We will institute these changes, we will review them for more problems, and we will act to fix those problems. Continuous improvement is the feedback loop for our processes, our products, and our lives.