Q 209. Regression testing is a type of software testing that confirms that a recent program or code change has not adversely affected existing features. As a software program becomes bigger, the need for regression testing creates a huge inventory. What are the methods used in the IT world to maintain speed while not losing out on effectiveness? Explain in simple words.

 

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time.  
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

 

It is an Incremental Validation Technique, advantageous especially in agile development environment.In regression testing, we re-run previously performed test cases to validate working of current functionalities.This is performed mainly to test the code changes (enhancements), has any impact over the existing features.

 

Regression testing is necessary because, any changes to the existing code can throw erroneous output and might cause the software to work inaccurately.  

It is a software testing type usually done in regular intervals, specifically post bug fixing (error correction), enhancements (after adding a new feature to an existing software), code optimization, environment change and fixing performance issue.     

 

For instance, in the below referred example Instagram added dark mode to its existing application.

Irrespective of the update/release. whether it is minor/major/patch fix, regression testing is performed. On an average 1-4 weeks of regression testing is performed before releasing to production Environment based on the complexity of the application/system. 

However, we can optimize and make the testing effective by formulating and following a good regression testing strategy.

 

Comprehensive techniques include:

• Retest all
• Regression test selection &
• Prioritization of test cases

 

Below are the different types of regression testing.

Types of Regression Testing
SN	Types	Performed when
1	Corrective	When no changes introduced to application specification
2	Re-test all	Re using all test cases
3	Selective	Testing for specific module / sub set
4	Progressive	Changes to specification + New test cases created
5	Complete	Multiple changes are performed
6	Partial	When new code is added to existing code
7	Unit	Unit testing phase (Code Isolation), dependencies blocked

 

In QA Process, regression testing is a significant step.

However, it can be Complex, Tedious (executing test again and again) and Time-consuming.

Challenge is to have wide test coverage with minimal execution of test cases.

 

For Rapid and Effective testing, strategy is used in selecting the test type. 

Best Practices and some recommended steps for effective testing follows:

• Primarily, Maintain and make amendments to the test cases in the regression test suite. Amendments can include adding new test cases, removing outdated test cases and modifying expected test results.
• Further categorizing test cases in the test suite for effective regression testing, viz., Reusable, Re-testable and Obsolete categories.  
• Based on the bug report, identify vital few problematic areas to prioritize testing.
• Focusing on functionalities that are commonly and frequently used by the users and selecting appropriate test case can make the testing effective.
• We might miss certain scenarios in test cases. So, It is recommended not to forget/ignore Random testing
• Using cross team QA’s, testers to perform regression testing

 

Having an effective regression strategy can help organization in saving time and efforts invested in Quality testing.

Benchmark Six Sigma Expert View by Venugopal R

The term ‘Regress’ means ‘to return to a former or less developed state’. Regression testing is the terminology used in software testing to test whether there is any change in the software functionalities that used to be performing well, before introducing a change. Those of us who have been associated with software quality directly or indirectly are likely to have experienced loss or deterioration of existing functions of a software after being subjected to some change, be it a bug fix or an up-gradation.

 

Looking back, similar situation prevails in manufacturing and other products as well. I had once got the AC condenser replaced in my car after which the AC was functioning perfectly. However, after a day, I realized that some of my dashboard indicators, that used to work properly, had stopped working. Most of you would have faced similar issues.

 

Though the term  ‘Regression testing’ is associated with software testing the need for similar testing has been prevalent in all fields. As LSS professionals, we are often torn between choosing between “Effectiveness and Speed”. How do we know whether any of previous functionalities would be adversely impacted by a change in a software? How do we decide what functionality could have got impacted and how much variability needs to be considered for evaluation?

 

Since we do not know the ‘unknown’, the blind decision could be to test everything i.e. Retest all. While this could be very expensive and time consuming, it still may not guarantee that it will identify any regression issues that may exist. For instance, not all test cases that were prepared earlier may be re-usable. Some would have become ‘obsolete’. Moreover, it could be possible that those test cases had been developed without considering any interactive impact of the new change.

 

Hence even if we can afford to go for a ‘Retest all’, it may not provide us the required protection. SW developing organizations and divisions strive to find those potential components and areas that are most vulnerable for being impacted by the change and ensure effective evaluation of those relevant functionalities – coupled with contingency plans. Based on my experiences, I share a few thoughts below and look forward to see many more from other professionals.

 

Test automation

Test automation is a very fundamental approach that may be adopted. This is useful especially when we re-run same test cases involving multiple scenarios. There are numerous tools available, depending upon the need.. like Selenium, Ranorex, QTP, to name a few.

 

Keep client in the loop

Where the SW product is for a client organization, and the change is requested by the customer,  it is a common practice to keep the customers involved during the development and UAT planning and execution for the change. Many clients with long term tie-up, insist that for all changes, even when not initiated by client, they need to be kept involved throughout the change process. It may so happen that certain tests would pass in the developer’s environment but may fail within the customer’s environment.

 

Keep ‘Roll Back’ options

Despite best attention, in the event of any unexpected regression issue that crops up only after going live, the developer should have ensured to have the option to 'switch back' to the previous version until the issue is fixed. However, this may not be practically possible for all cases – for instance, if the change is linked with a change that is done at the client side as well.

 

Prioritising Test Cases

Selecting the test cases and having the confidence of not missing out on an important one is a very tricky decision and should not be done without cross functional involvement. A good understanding of test cases that detect frequent defects, functionalities that have high user visibility and sensitivity, test cases related to core functionalities, Integration test cases are important inputs for the deciding the priorities. FMEA will be a useful tool to structure, document and prioritize the profusion of knowledge and experience.

 

Test Driven Development

It is a legacy approach to subject previous set of test cases as part of ‘regression testing’. However as discussed earlier, it may not be adequately effective always. Parallel efforts to develop appropriate tests along with the code development for the change would help in identifying relevant test cases. Integrate regression testing with exploratory testing used in Agile development.

 

Mitigation options other than tests

Having in the development approach, keeping in mind that changes could be expected. With more focus on ‘Agile’ development, customers want to keep the options for change requests very open. System architecture may focus on building more smaller, independent components. This will make the effect of changes more isolated than the case of common coding and hence the test cases could be focused and limited.

 

Version and Change controls

Well defined and complied version and change controls will help to ensure that the access to changes in codes are well controlled, authorized and undergo required levels of approvals… thus preventing the possibilities of impairing unwanted sections inadvertently or due to ignorance.

 

The success of optimizing the efforts and resources for Regression testing is achieved with a combination of Cross Functional Involvement, Software Quality Management Systems, and Proactive developmental prudence.

Winner today is Mohamed Asif for his excellent answer.

 

Please read great expert write up by Venugopal as well.