Friends,

 

Would appreciate if someone can offer a BPO example (let's a say a call evaluation process) where Attribute Gage Study (Analytic Method) can be applied. The example offered in Minitab 15 is'nt striking enough......help will be appreciated....Gud times...

 

Kiran Varri

 

If you can describe any measurement that is being made in the case of any BPO (say a call evaluation process) - that is explain us what is being measured and how it is being measured, then I can help you in explaining how the attribute Gage R&R study will help in determining the reliability of the measurement system.

Dear Bala,

 

Thank you for the assistance. I would take a situation where 20 Call Quality agents are auditing 45 calls every day. A Quality Analyst from the team acts as the Control. He/She audits randomly few calls of the Auditors and gives them the feedback for better uniformity(calibration) of the audit process. Each Evaluation form is scored for 100 points based on Verification/Resolution/Clarity/Tone/Upselling e.t.c

 

Could we take this as an example...

Dear Kiran,

 

One way of doing an attribute G&R is as follows: Choose about 20-40 calls. The quality analyst rates these calls. The response of the quality analyst can be used as the standard. Give the same calls to the quality agents (2 or more) for them to rate. This would be trial 1. After a few days ask the same quality agents to rate the same set of calls (trial 2).

 

The quality agents response would be used to check the repeatability and reproducibility. It is similar to call calibration, however, the objective here is to validate the existing measurement system. If the MSA is acceptable, then you can continue with your analysis. Else, you would have to fix the measurement system before any meaningful analysis.

 

Hope this is useful.

Hi Kiran,

You raised a very good point and the gauge R&R is a very important aspect in a six sigma project. Now we have to conduct MSA. In MSA three things are very important

 

1:- What to measure?

2:- How to measure?

3:- How to ensure that what we measured is correct is right?

 

In a call centre environment (call monitoring) we have discrete X and discrete Y. To measure discrete data we have to do Effectiveness and Efficiency. So if you talking about the call quality data then the first thing you need to check is the effectiveness of the data by monitoring few call as per pre decided sample size. The moment you got the data now you have to gauge the efficiency. Now the point is how you can ensure the efficiency of this data, for this you have to put the golden rule for those evaluated calls. For example call calibration scores or the client evaluation scores. Hope it will help you. Please feel free for any clarification.

 

Thanks,

Ajeet

 

 

 

Hi Kiran,

 

The Attribute R&R Analytical is used when your response is Binary. Pls choose attribute agreement analysis (Stat/Quality Tools/Attribute agreement analysis).

10 operators with 3 trials and 20 calls will be 600 readings. If it is 20 operators you need 1200 readings. I am not sure how doable is this?

 

Also the study should be conducted without the Quality operators knowing that this is being done otherwise we are introducing a bias. Thanks

Srinivas MBB

Hi Kiran

This is a very relevant discussion going on.

 

Here are my drops in the ocean:

 

1. The first element is to decide which parameter is being measure.  If a rating on say a five-point scale is being given, it will be ordinal data and hence fall under Attribute R&R. If the measure is only good/bad, then its binomial data and hence also fall under Attribute R&R. However, if scores (numbers) are being given in a parameter based on the evaluation, then the overall score will fall under Variable R&R.

 

2. Let us take only one element of the call monitoring i.e., the quality analyst is checking whether the resolution took place or not based on the conversation. So this is pass/fail and hence binomial.

 

3. For the above scenario, you should take minimum 30 calls. Lower samples reduce the resolution. e.g., with 10 samples, every sample going wrong impacts overall score by 10%. Typically 2 trials per sample per analyst and 3 analysts would be taken.  Here one must note that if there are less than three analysts for that team, only those should be taken. For larger numbers, a thumb rule of 20% is taken. Also, please note, as the number of analysts will increase, the overall score will have higher opportunity of going down.

 

4. Let say you have taken 30 calls, 2 trials each and 3 analysts. Since this is a R&R exercise, the same sample should be available for the analysts for evaluation. This could be achieved by recording such calls and then playing the call for each reading of an analyst.  It should be attempted that the calls are played at some gap or specific info in the call should be removed which could create bias from memory rather than fresh evaluation.

 

5. Each analyst listens to a call and marks pass/fail as per his/her evaluation.

 

6. Once the complete data is captured, it is analysed through software like excel or Minitab.

 

7. The challenge here could be to understand the Standard. This typically is achieved through

a) having an expert who is considered as the final authority from a knowledge perspective

checking with the customer, which most people avoid

c) post the data collection, all analysts come together and deliberate till they reach a consensus on the result (pass/fail), which is then treated as the standard

It is not mandatory to have a standard. However, this helps in accuracy check i.e., calibration of the system.

 

Trust this helps.

Regards

Hi Kiran,

What I would suggest is that put a metrics for call evaluation process. Ask your Call Quality Analyst to rate this call in 1 to 5 points and ask them to jude the same. By this way they will come out valuable points (feedbacks). The same feedback should be conveyed to call agent and try to monitor his call for nearly one week surely you can find an improvemnt in that.

Muragan I am agreeing with your point u r talking about the efficiency; however could u please through some light on efficeness. If u dint mind and sorry for  the spell mistake......

Dear Kiran

 

Attribute Gage Study (Analytic Method) (pulldown menu Stat->Quality Tools-> Gage Study->Attribute Gage Study) is for continuous data. Here the reference values are provided in the form of continuous data and the outcome is binary (go/no-go). This method should not be used for quality evaluation. For quality evaluation, you will need to use Attribute Agreement Analysis (pull down menu Stat-> Quality Tools-> Attribute Agreement Analysis)