Q. 190 In the DMADV roadmap of Lean Six Sigma, the last phase is Verify/ Validate. Explain the difference in Verification and Validation using examples from different domains/ industries.

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening as per Indian Standard Time.  
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

 

In DMADV, focus is on new product/service design, unlike for existing product/service in DMAIC, during the last phase of DMADV, verification of design is performed and whether the design is capable of meeting needs of the customer is validated.

 

Numerous pilot runs will be required to validate and verify the design outcomes.  Major aspect of this phase to check whether all metrics which are designed are performing as expected. Conformance to Specification.

 

Some of the common used tools in verify phase includes Control charts, control plans, Flagging, Poka Yoke, check sheets, SOP’s and work instructions.  

 

Software Application Design:

In a new design viewpoint,

Verification is whether Software Application developed in right way &

Validation is whether Right Software Application is being produced

 

In simple terms, verification is checking whether the application works perfectly without any errors/bugs and validation is checking whether the application is meeting the requirement and expectation

Verification	Validation
Application and design review, code walk through, code inspection	Black Box and White box testing
It is static testing	It is dynamic testing
Performed first	Usually performed post verification
Verification done without software execution	Validation done with software execution

 

Automotive Manufacturing:

Reference to a gearbox manufacturing, as per the new design in DMADV process, in actual manufacturing high level steps include preforming, annealing, machining, producing teeth, shaving, grinding and inspection.

Here verification is, comparing the gearbox to design requirement of material, dimension, tolerance etc., that is all specs are verified

Whereas, in validation, post inspection assembling gearbox and doing a dry run, test it to check whether it runs as expected.

 

Verification	Validation
Done during development, review and inspection, production and scaleup	Usually done before scaleup and after the actual production
Random inspection can be done for verification	Stringent checks are done during validation

 

Validation can be done directly by skipping verification in some scenarios, especially when we are not able to measure component outcomes or when cost of verification is very high.

 

Medical Devices:

Verification usually done on the design: design input, process and the output. It is done by test, inspections and analysis.

Validation is checking whether the intended need of the medical device is met

Source: U.S. Food and Drug Administration (FDA)

Verification undertakes activities which demonstrate in a logical, progressive and traceable manner that the-

1. design satisfy the requirements of the customer;
2. Subsequent stages of design satisfy the requirement of the earlier stages of designs; and 
3. Future stage of realization which can be validated against the design requirement .

The verification method/s can be -

1. Analysis of design (FEA Modelling )
2. Simulations
3. Hand calculation
4. RAM (Reliability availability maintainability )Studies -(Prediction & Modelling)
5. Participation in an observation and questioning role in design reviews
6. Bench-marking against international best practices where appropriate 

Through objective evidence, validation is used for confirming that the requirements for an intended use or application have been fulfilled. Should be carried out with joint involvement of customer and developers. 
 

Many requirement could be validated in Factory acceptance tests (FAT) while quite a few may requirement may have to demonstrated during product acceptance testing. 

A Validation plan should be subjected to approval on test strategy; technique and measures used ; test and/or analyses used and how the result will be reported; management of deviation from actual to test results, conditions and constraints derived from deviation, plan for how this will be considered in next life-cycle tasks of validation.

 

The Above is typical used and leverage in railway domain project life cycle.

DMADV – Define, Measure, Analyse, Design and Verify/Validate

 

Dictionary Meaning:-

Verification - The process of establishing the truth, accuracy, or validity of something

e.g. – Before processing a loan disbursement, accounts officer verifies the details mentioned in the system with the documents provided by the customer.

Validation - The action of making or declaring something legally or officially acceptable

e.g. – In excel we put validation formula so that only given values will be accepted by cell otherwise it will give error. It means the text is validate and accepted.

 

Difference between Verification and Validation

 

Verification	Validation
Verification can be done via physical checking of one thing with another or through Gemba visit	Validation can be done through tools and techniques. Mostly done in IT industry to test the final product
Verification is done to check the accuracy of the product	Validation is done to check the accurate performance of the product
Verification is done before validation	Validation is done post verification

 

Domain	Verification	Validation
Service Industry	Checking whether a Quality Control Procedure is implemented	Checking whether the Quality Control Procedure is rightly implemented i.e. procedure rightly identifies the errors
Software	Understanding whether input specifications have resulted in product output i.e. product should have 10 modules with 3 sub-modules - whether all modules are developed would be verification - Feature reviews, high-level checks, simple checks	Understanding whether output  results in meeting the expectations i.e. whether the developed 10 modules with 3 sub-modules works are desired providing output meeting the expectations
Automotive Industry	Assembly inspection of a car assembly on the different features such as finished product, dimensions, colour texture, weight etc.,	Carrying out road test for the agility, strength, performance, crash strength etc.,

Verification and Validation are used interchangeably and often considered as same. However the two are different.

 

Validation simply means – are you making the right thing?
(Source: https://en.wikipedia.org/wiki/Verification_and_validation)

 

Verification simply means – are you making it right?
(Source: https://en.wikipedia.org/wiki/Verification_and_validation)

 

The same difference is also highlighted in the definitions of the two as provided by PMBOK

"Validation - The assurance that a product, service, or system meets the needs of the customer and other identified stakeholders. It often involves acceptance and suitability with external customers. Contrast with verification."
(Source: https://en.wikipedia.org/wiki/Verification_and_validation)

"Verification - The evaluation of whether or not a product, service, or system complies with a regulation, requirement, specification, or imposed condition. It is often an internal process. Contrast with validation."
(Source: https://en.wikipedia.org/wiki/Verification_and_validation)

 

Simple example to illustrate the difference between the two

Voice of Customer: Customer wants to have a hot cup of coffee

Validation: Customer would accept the coffee basis below specifications

1.     It is hot (research suggests that temp. should be close to 96 degree Celsius)

2.     Right amount of sugar (say 5 gm)

3.     Right amount of milk (say 200 ml)

4.     Right strength of coffee (say strong)

These specifications are the needs of the customers and the coffee will only be accepted only if these specifications are met. These specifications only talk about what is the right product. It does not talk about how the coffee will be made to ensure that these specifications are met

Verification: The process of brewing the coffee should be designed in a way which ensures that the above 4 specifications are met. Let us assume that we are using a coffee machine to make this coffee. If the machine keeps the temperature around 96 C, adds just the right amount of sugar, milk and coffee, then we would say that the machine is verified to provide the right coffee to the customer. The verification could either be done during development (of coffee machine) or during the production (like QC check).

 

Other Examples

1.     Maneuvering Characteristics Augmentation System (MCAS) is a software to control the plane from stalling and is the software which is argued to be the reason behind the 2 fatal crashes of Boeing 737 Max planes (one for Lion Air and the other for Ethiopian Airlines).

Verification: When the software was developed, it was put through hundreds of hours of analysis, laboratory testing, verification in a simulator and two test flights, including an in-flight certification test with Federal Aviation Administration (FAA) representatives on board as observers (Source: https://www.boeing.com/commercial/737max/737-max-software-updates.page). This is where the software was verified as per the guidelines provided by Boeing.

Validation: Even though verified, it is not acceptable or let me say validated by the airlines or by the aviation regulator as it fails to meet a key customer requirement of SAFETY (the most important requirement in aviation)

 

2.     NASA has an independent verification and validation facility which was set up after the Challenger accident. This facility is set up as an independent verification and validation facility. Reason for challenge accident – failure of O-rings. These O-rings were designed to work at high temperatures and design required each hole (in the rocket motor) to have 2 O-rings. The manufacturing process verified that the O-rings are prepared as per the design specifications. The fabrication process verified that each hole has 2 O-rings in the motor. However, the validation i.e. customer acceptance failed which resulted in the accident. The reason for passed verification but failed validation what the incorrect design specifications for O-rings.

 

3.     Software Development (Agile method). Let us a say a new web-page is developed which captures the demographic data of the user (details like name, age, address, pin code etc.)

Verification: This is like unit testing. This would include the following things

a.     Code written as per the agreed upon standard

b.     Individual sprint testing to check that all fields are coded perfectly and the overall page is working fine

Validation: This is when the new web-page is being integrated with the existing system and released to the customer or in a duplicate replica. This is more like User Acceptance Testing (UAT). This would include things like

a.     User navigation to page

b.     User experience on the page

c.     Overall system working fine

 

4. Satellite phone developed by Motorola. Motorola at one point of time was the front runner in the field of telecommunications devices. They had a vision of making the satellite phone a household thing. The phone was a verified product (isn't is obvious?). Motorola is the pioneer of Lean Six Sigma. However, even this verified product failed on validation as the customers did not make a beeline to buy the product (unlike what we see for Iphones)

As per system engineering, a product or a system being developed will have following levels (similar to what APQP also prescribes)

1. System Level

2. Sub-system Level 

3. Component Level

 

Verification is done at all the 3 levels however validation is only done at the system level as the customer is going to use the overall system (with all its parts).

 

From the above it is clear that verification is a more internal process while validation is a more external process (i.e. involves the customer).

One could link verification to the process width or the control limits and validation to specification width or the specification limits. Process (manufacturing or services) could well be stable or verifiable, but still not be capable or validated. Best case for a process is to be both verifiable and validated.

Benchmark Six Sigma Expert View by Venugopal R

The terms ‘Verification’ and ‘Validation’ have been popularized in Quality Management systems from the days of introduction of the ISO 9000 standards. However they have been indistinct for many people, and continues to be so even today. This may be because if one refers a normal English dictionary, these terms could appear as synonym to each other.

In the industrial world, be it manufacturing, services or IT, any Product, Process or Design has to be defined using specifications. The evaluations that help us to ascertain that the Product, Process of Design meet the ‘specified’ requirements is termed as ‘Verification’.

 

However, once we create a Product, Design, or a Process as per the specifications, we still need to ensure that it is capable of performing under real time circumstances, also referred to as field performance. The assessment process to ensure the performance under actual conditions under which it is intended to perform is termed as ‘Validation’

 

As mentioned, these terms can apply to Design, Product or Process. Let us discuss each with example.

Imagine a packaging company was entrusted with the task of developing a special packing for packing a complex and sensitive medical equipment that has to be transported to a long distance. The packaging company can come up with a design and perform ‘verification’ on the design to ensure whether all the requirements have been adequately captured by the design. The company may then proceed to ‘validate’ the design by creating prototypes and ensuring that the intended purpose is served when the design is put to real use. Feedback from the design validation is used to carry out necessary corrections on the design.

 

In a similar manner, when a process is setup, say for assembly of a TV, there will be a process lay out, specification of the equipment, resource requirements and so on. It will have to be ‘verified’ whether the process has been set-up as per the specified requirements. However, we need to perform trial runs of the process and ‘validate’ whether the entire assembling process performs to deliver the desired outputs.

 

Hopefully, now it will be clear how the verification and validation apply to a Product. A product, once build gets ‘verified’ using certain evaluations and tests to ensure whether it complies to Product specification and regulatory requirements. For example a manufactured refrigerator undergoes various verification such as visual, measurements, electrical tests etc. However, it will further have to be validated for its performance under actual usage conditions. Accelerated tests are used by simulating field conditions. Other validation methods include subjecting products to actual field conditions using monitored trials.

 

Moving to software, let’s take an example of a software used for credit administration in a bank. The software developer subjects it to code verification and also perform verification tests using specified test cases. However, installing the software in the actual workplace and getting it used by the employees who execute the process, and seeing whether it performs as required by the business is the actual ‘validation’ for the software.

 

In brief, verification is always necessary before we get into validation. What could have been identified through verification, should ideally not get identified during validation, since the correction process will prove costlier.

The winning answer is that of Asif as multiple examples from different industries are provided to explain the distinction between validation and verification. The graphics, esp from medical devices - drive the point home. The answer could have been strengthened by providing the definition of the two terms upfront.

 

For clear distinction between Verification and Validation, refer to answers provided by Swapnil and Natwar. For distinction between the two terms explained with examples (not with definition), refer to Praveen Kumar's answer.

 

For a full rounded view, read through Benchmark Expert view provided by Venugopal R.