Q 257. Reporting bias leads to incorrect data analysis. What are the different types of reporting biases and how can an organization safeguard itself from reporting bias?

 

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time. Questions launched on Tuesdaya are open till Friday and questions launched on Friday are open till Tuesday. 
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term

Reporting Bias:

         Reporting bias means “distortion in the dissemination of details in dissertation” i.e misinterpretation or suppression of information or findings in the research/publication due to the influence of nature, location, time frame, Cultural background, conflicts or by detailing only one part of the research.

Reporting Bias in Research/ Academics:

In academics/ research, Researcher tends to hide the undesirable results and always tries to undercover the unlikely results with errors. Unlikely results are even suppressed believing that the desirable results are most trustworthy by forgetting the fact that all possible outcomes have their own probability.

Reporting Bias in Survey:

When we are taking up a survey on the brand preference among the customer, The sample customers are taken from the zone where they glorify the Particular brand or the surveyor questions may lead to promote the particular brand due to the direction of the desired result.

Reporting bias is either intentional or unintentional, either natural or driven by the preferred results. However, it is important to address and avoid the reporting bias with concern of future interpretation. Most of the political and marketing surveys have some biases which seems to be obvious, but it will not be always obvious and tends to occur.

 

Reporting Bias leads to incorrect data analysis:

         Data Analysis, Data driven decision making and empirical or Analytical research depends on the facts of the data been reported or the outcome of research. Most of the organization are facing these challenges while making decision due to data bias (as an outcome of reporting bias). 

Bias in data challenges the decision making and makes any organization to fail in their decision. Even with most of the best intentions, reporting bias likely to be presenting and pervasive. Considering that decision makers and data analytics should be aware and put their best effort in order to eliminate or diminish the effect of Bias in their analysis 

 

 

Some types of Reporting Bias:

 

1.   Publication Bias:

Researcher fail to submit or document their reporting which makes distortion in prediction. Most of the researcher’s work in finding the vaccination of COVID-19 is not published due to failure or null finding. Because of that reason, progress of the research will be overestimated.

 

2.   Time Lag Bias:

 

Delay in reporting due to constraints in publication or an article published after a year from experimentation may bring out totally different scenario. Due to intervention of new things, research results with time lag may not be valid. Research conducted on street foods preference become obsolete because of intervention of COVID-19.

  

3.   Duplication Bias:

 

Multiple publication of the same desired results by the same author or different author will make the analyst to overestimate the effect on the results. Due to multiple publication of the effect of Hydroxychloroquine on COVID-19, Many people started believing HCQ is proven vaccination (the failures are not in open source – Citation bias).

  

4.   Location Bias:

 

Research findings published in one location may not be applicable for the other location due to the difference in characteristics or behavior. Deceased rate of COVID-19 in Italy will not be same as UK.

 

5.   Citation Bias:

 

Researches prefer positive results to publish than the null or negative results.

Success of HCQ is published more than the failure of same. It results in opinion of HCQ as a better vaccine for COVID-19

 

6.   Language Bias:

 

Studies published in different languages are neglected in the systematic review. Most of the time researchers don’t know that language bias is existing in their process. It has to be taken care on locating and assessing the relevant non-english publications.

 

Role of organization to safeguard itself from reporting bias/ data bias:

         Business leaders /Organization should be aware of the impact of Reporting bias in their future goals. Since we have various forms of bias in our research or data collection process, there should be complete transparency and should be even addressed to the researcher/ surveyor on possible shortcomings in the process.

         We should ensure that we have the quality information without bias, so the organizational system should know the source of data. It is very important to decide on few things which are mentioned below for the data collection.

a.   Where the data collected

b.   When the data collected

c.    How the data collected

d.   From whom and by whom the data collected

e.   How long the data will be valid

f.    What criteria should be given

Considering all those factors, data collection should be as generic as possible inorder to minimize the effect of influencing factors and should have open mindset on all possible outcomes which will ensure that there is no misleading decisions and safeguard the organization from the risks of reporting bias. 

 

 

 

Bias is the difference between the measured values against the actual value. Bias can be positive or negative. If the difference between the measured values and the actual value is positive then you have positive bias. If the difference between the measured values and the actual values is negative then you have negative bias.

 

Three types of bias can be distinguished: information bias, selection bias, and confounding

 

 

 

Reporting bias means that only a selection of results are included in any analysis, which typically covers only a fraction of relevant evidence. This can lead to inappropriate decisions (for example, prescribing ineffective or harmful drugs), resource waste and misguided future research.

 

Types of Reporting Bias

1.      Citation bias: basing your analysis on studies that you find in the citations of other studies.

2.      Language bias: ignoring studies not published in your native Language.

3.      Location bias: certain reports or studies are harder to find than others. For example, studies that are published in journals might be indexed higher in databases.

4.      Duplicate publication bias: studies that are published in more than one place might get more weighting than other studies.

5.      Outcome reporting bias: selective reporting of certain outcomes, such as outcomes that paint a company in a good light.

6.     Publication bias: studies with positive findings are more likely to be published — and published faster — than studies with negative findings or no significant findings.

7.      Time lag bias: some studies take years to be published, especially if they show no effect or have unwanted results. Studies that are positive or newsworthy are published much faster.

 

Reporting Bias can occur in the life cycle of many research and are as follows:

Reporting Bias are widespread phenomena in the medical literature. So, the organisations took some preventive measures to safeguard itself from reporting bias. Transparency is the most important action to safeguard health research. 

Tips to avoid different types of bias during a trial are given below:

 

 

1. Clearly define risk and outcome, preferably with validated methods and standardize data collection.

2. Samples should originate from the same general population.

3. Standardize interviewers interaction.

4. Use prospective studies and avoid using historical controls.

5. Use objective data sources whenever possible.

6. Carefully design the plan for lost to follow up.

7. Clearly define exposure prior to study.

8. Validate measures as primary as primary outcome.

9. Consider cluster stratification to minimize variability.

10. Register trial with an accepted trial register.

11.  Unknown confounders can only be controlled with randomization.

 

 

 

 

 

Edited April 30, 20206 yr by Satyajit Das
spelling mistake

Reporting bias happens when a researcher’s report is influenced by the nature and significance of the results. This sometimes defeat the objective of research.

1.       Multiple publication bias: This happens when the results of a study are published multiple times most especially when they are favourable. Data is also sometimes duplicated within the same study. Studies should always be subjected to rigorous reviews to avoid this kind of bias.

2.       Location bias: This happens when pulling results from different databases. Studies stored on some databases are already influenced by some other bias, extracting secondary data from such database will introduce location bias into our study. Studies should span through all relevant databases to remove this bias.

3.       Citation bias: We sometimes look into the reference section of a research to get more research results about a particular subject. Studies have shown that researchers sometimes select references that confirm their biased judgement of the subject. Hence, analysing citations from the reference list of one study may introduce citation bias. Selection of research materials from a single reference list should be discouraged address this.

4.       Language bias: We find it more comfortable to consider results from studies that are reported in a language that we are familiar with. The implication of this is that our assessment will not cover studies in other languages which leads to bias against data in studies conducted in other languages. Translation software can be employed to solve this problem.            

5.       Outcome reporting bias: Results are sometimes filtered based on how favourable they are to our expectations. The conclusions from such studies are therefore misleading because it was done only to justify subjective conclusions. To avoid this kind of bias, sampling procedures and sample selection should be well reviewed, live data reporting can also be adopted.

 

Reporting bias  means - selective  reporting of data/ information or selective ignoring or under reporting  of  genuine information.

Types of Reporting biases and their prevention

Publication bias:

Causes: Influenced by research findings

            :  Published  studies are only included

            :  Positive results more likely published

Prevention:- Both published and unpublished studies need to be included - Research should not be directed to a particular outcome-  Peer authors involved in similar studies should be contacted.

Time lag bias

 Causes:- Findings with significant or positive results much earlier published, than studies with negative results or null  hypothesis. Early positive results  of studies are promptly reported, whose long-term results may be actually negative.

Prevention: Both positive and negative results need to be published without any discrimination.

 

Multiple (duplicate) publication bias:

 

Causes: Same data published in multiple studies, in different resources as slightly modified versions of the same study. More duplication is found in positive or significant results.

Prevention: - Plagiarism check, linking the results of the studies.

 

Location bias:

Causes:- Studies may be published  in  journals  with higher indexing in databases, leading to difficultly in tracing. Data may be from a particular location, or group with more chances of desirable result.

Prevention:  Data to be taken from multiple locations and  representative groups. Data  to be published in literature, which are easier to retrieve.

 

Citation bias:

Citation bias : Citing  of studies with more positive results than negative results. Citation of positive findings may be done for funding purpose.

Prevention: Citation should include  studies with both supporting and contradictory findings.

Language bias: 

Causes: Studies only in particular language, especially English. Non-English studies with positive results are published, where as negative results are not reported.

Prevention: All language studies with positive and negative findings should be reported.

 

Knowledge reporting bias 

Causes:  Studies depicting various facts regarding particular action, property, about a  particular class of individuals is not reflective of real world frequency and leads to knowledge bias.

Prevention:  Information about a small group of individuals should not be generalized to the larger population or the entire world.

Outcome reporting bias:

Causes: The outcome of studies are often influenced by the expected outcome, that leads to erroneous reporting of study results.

Prevention:  True results should be reported, never altered for the sake of reporting the desired outcomes.

 

Prevention of reporting bias in organizational context:

 

-  Reporting of only positive findings and suppressing the flaws in  an organization, leads to reporting  bias and should be prevented.

- Employees should be given free hand to report about any  issues  concerning management, technical and personal issues to the  higher management level personals.

- Business research studies and surveys should be conducted regarding possible ways of increasing the market share, revenue generation, possible research and development, and the reporting  of results of  these studies should not be influenced by any kind of bias due to any person or any  particular pre-decided outcome. 

 

 

 

Q 257. Reporting bias leads to incorrect data analysis. What are the different types of reporting biases and how can an organization safeguard itself from reporting bias?

 

Bias:

Bias is a belief science and engineering for self and inconsistent or unequal weight against a single or individual, or a group. This is purely an unfair sampling of a population for any function in any industry that does not give any accurate results.

Biased Outcome & Reporting:

Biased outcome and Reporting may lead to incorrect data analysis in any organization. Also, biased outcome may lead to negative or positive consequences in Diversity & Inclusiveness (Harassment & Bullying, Discrimination), COI (Conflict of Interest), ABC (Anti Bribery and Corruption), Ethics and Behavior, Regulatory Compliance, Auditing, Communication, Change Management and People Development in large organization. It will also lead to huge time/cost spending in near future in the same organization to recover from biased risk.

My personal view about Bias: Biased story means that you are supporting or motivating one side of an issue or event. Other than Politics, Biased outcome may lead to negative or positive consequences and it will not give any value to the bottom line.

Unbiased Reporting:

We need to focus on Unbiased and it means you are telling the exact facts & measures and reporting the actuals that allowing Superior or leadership to make a correct decision.

Type of Bias:

-         Cognitive Bias: Their own Perceptions or Their Own Behavior.

-         Conflict of Interest: COI (Conflict of Interest) is when a Person or Group or Association has intersecting interests. The respective person or individual & associations are incompatible and It creates the risk.

-         Statistical Bias: Incorrect data collection and sample selection.

-         Prejudices: Bias and Prejudice are closed related. Prejudice is prejudgment, unfavorable judgement to person or people with respect to their own limitations.

Contextual Biases: The following Subjectivity may bias the evaluation outcome.

-         Biases in academia

-         Biases in law enforcement

-         Biases in media

-         Other contexts: Educational Bias, Inducive bias, Insider Trading and Match fixing.

-         Implicit Bias or Unconscious Bias

-         Explicit Bias or Conscious Bias

Safeguarding Organization from Reporting Bias:

-         Engagement, Collaboration, Respect and Performance (Leadership Goals)

-         Equity and Equality: Treat everyone in equal manner with respect.

-         Assign Observer in Team meetings.

-         Assign a Moderator and Recorder: The Moderator or Recorder is an unbiased representative of any initiative and the feedback process.

-         Conduct the focus group meetings.

-         Accurate Data collection, Specialized skills in Statistical Sampling methods.

-         Metrics Action Plan, Field Work, Audit Planning, Auditing and Audit Reporting.

-         Regression Testing.

-         Simulation, Regression, Drill Activities.

-         Survey and Questionnaire (Open and Closed ended questions)

Scenario 1: The following simple matrix example shows the overall effort for various HSE/Cyber Security incidents. (Unbiased Reporting or Safeguarding Organization from Reporting Bias)

No	Incident No	Incident Category	Incident Assigned to	Data Collection	Data Accuracy	Lessons Learnt	QA/QC	RM Approval	Status
1	1	Tier 2	HSE/Cyber Security	Yes	Yes	Updated V1.0	Yes	Yes	PASS
2	2	Tier 3	HSE/Cyber Security	Yes	Yes	Updated V1.0	Yes	Yes	PASS
3	3	Tier 4	HSE/Cyber Security	Yes	Yes	Updated V1.0	Yes	Yes	PASS

 

Safeguarding Organization from Reporting Bias: Communicating about safety and cyber security incidents is key to a safe and healthful workplace. We must take the time to discuss safety issues with employees of all levels to get a thorough, unbiased gauge of the conditions. Everyone participation, evaluation, and feedback should always be encouraged.

 

 

Scenario 2: The following simple matrix example shows the overall effort for various DR - Disaster Recovery Drills. (Unbiased Reporting or Safeguarding Organization from Reporting Bias)

No	Simulation	Disaster Recovery	Function	Moderator	Lessons Learnt	DR Procedure Document	QA/QC	RM Approval	Status
1	1	DR Drill 1	IT	Yes	Yes	Updated V1.0	Yes	Yes	PASS
2	2	DR Drill 2	IT	Yes	Yes	Updated V1.1	Yes	Yes	PASS
3	3	DR Drill 3	IT	Yes	Yes	Updated V1.2	Yes	Yes	PASS

 

Safeguarding Organization from Reporting Bias: DR Drill simulation is key to a safe and healthful workplace for longer run. We must take the time to simulate DR Drill activities and ensure our learnings updated with high accuracy, unbiased gauge of the conditions. Everyone participation in simulation activities, evaluation, and feedback should always be encouraged.

 

With respect to Safeguarding Organization from Reporting Bias with above recommendations or practices, the greater performance along with high quality will be achieved.   

 

Reference:

 

https://diversity.ucsf.edu/resources/unconscious-bias

https://en.wikipedia.org/wiki/Bias

 

 

Thanks and Regards,

Senthilkumar Ganesan,

Email: [email protected]

Mobile: +91-7598124052

 

Bias.docx

Benchmark Six Sigma Expert View by Venugopal R

 

Benchmark Six Sigma Expert View by Venugopal RAlthough the term “Reporting Bias” and its categorization is popularly related to Epidemiology,  some of them would be relevant for other businesses as well. Reporting bias is also referred to as ‘selective reporting’, where certain information tend to get reported dominantly, advertently or inadvertently. Such bias is common, especially in reporting of scientific matter and clinical trials.

Below are various types of ‘reporting biases’, their definitions and some general thoughts on how an organization can safeguard against such biases.

 

1.       Citation bias

Basing the report from other articles and reports, has the risk of providing only ‘one side of the story’. There could also be a tendency to report the ‘positive outcomes’ for a study and not focusing on ‘negative’ aspects.

Tips to safeguard: Ensure the practice of quoting the sources of references for citations. Corroboration by multiple references should be insisted to obtain realistic picture of the findings being reported.

 

2.       Language bias

This is a possibility when reporting needs to be done in multiple languages. In an organization context, while a corporate report would be released in English, there would be need to translate into regional languages for the benefit of all levels of employees. Or the need could be due to the presence of the organization present in multiple geographies. A bias in such situation could be intentional or unintentional.

Tips to safeguard: Translations may be subjected to review using unbiased translators along with a SME to ensure that the message does not get biased intentionally or other wise.

 

3.       Duplicate reporting bias

Duplicate reporting could happen when the same topic is reported multiple times by same source or different sources. This can result in incorrect exaggeration of certain results or duplicate accounting of certain benefits.

Tips to safeguard: Have defined authorities for reporting of specified topics. If any one else has inputs on the same topic, they need to forward to the designated authority to ensure that the chances of duplication are avoided.

 

4.       Selective Reporting bias

This is a very common type of bias, where some outcomes are reported and some are omitted, depending upon the nature of the results. Such bias could be introduced with vested motives to skew the interpretation in favor of the expectation of the author.

Tips to safeguard: Provide equal opportunity of representation by all stakeholders for the given report. Any interpretation and related actions need to be taken up only after verifying that the facts are represented in full. Involvement of key stakeholders before final interpretation is important.

 

5.       Time-lag bias

This bias could be related to some of the earlier discussed bias, for instance, the selective reporting bias. A positive outcome may get reported faster and negative outcome may get reported after a significant time gap. This could result in incorrect actions being taken based on partial interpretation of the situation. For example, a product launch based only on the success outcomes of a new product trial reported without timely mention of certain potential risks, could result in serious reputation damages.

Tips to safeguard: Follow a balanced set of business reporting parameters as far as possible and we need to know whether observations on all parameters have been considered with equal attention and reported on time, before taking major decisions

 

For any organization, reporting methodologies for various business activities need to be pre-planned and reporting standards established, taking into account the potential reporting biases. To the extent possible, build-in logical validations that could throw up suspected biases, and for the rest, the organisational discipline needs to be set in place to minimize errors due to reporting biases.

Reporting bias, is basically introducing bias in the way a subject is reported. This could be under reporting or selectively revealing things.

 

Reporting bias occurs when the dissemination of research findings is influenced by the nature and direction of the results, for instance in systematic reviews.

 

In daily life, a classic example is in News reporting. The same news event can be projected in either ways (positive or negative is also depending on the bias of the viewer) by channels that are typically branded right or left. Depending on the nature of the event, it becomes a glass half full or glass half empty. For example, in the current Covid situation, while one set might focus on controlled spread, the other set minimizes that coverage and focused more on economic fallout.

 

Also there is a time bias. Almost always negative news are reported faster than positive news.

Bias also seeps in language, location, skin color etc.

 

Statistics is another key area. Classic one is "correlation doesn’t mean causality", yet a lot of reporting happens on correlating something to imply something.

4 out of 5 dentists recommend this product - could effectively mean only a sample of 5 and 4 reported positive. Selectively projecting numbers is a classic case that happens everywhere - financial reporting, operational reporting etc.

 

One way to minimize reporting bias, is to pool results from similar but disparate sources. Its necessary to get pieces of corroboration from multiple sources.

 

Some Reference sites

The website https://catalogofbias.org/ provides a catalogue of bias in health services

Media bias https://www.washingtontimes.com/news/2016/nov/8/mainstream-media-maligned-10-examples-blatant-bias/

Cognitive bias https://www.businessinsider.com/cognitive-biases-2015-10#the-affect-heuristic-describes-how-humans-sometimes-make-decisions-based-on-emotion-1

 

 

Reporting bias means partial information is being considered for the final analysis. It indicates that the sample data used for further studies are not the accurate representation of the population. There can be various reasons for Reporting bias – convenience, lack of knowledge, restrictions etc. This results in the inappropriate conclusion of the sample data due to the under-reporting of the information available.

The types of reporting bias which leads to incorrect data analysis are:

A.  Convenience bias: There can be reporting bias due to cherry-picking done by the Internal QA team in the BPO Voice process. The major reason for cherry-picking by QA could be to meet their respective pre-defined weekly target of monitoring calls.

          i.      This can be curbed by designing an effective sampling plan along with guidelines defined, e.g., minimum call length.

         ii.      Randomize the sample selection approach, e.g., Randomizer in excel.

B.  Lack of Knowledge Bias: When the evaluators are not calibrated or there is a gauge difference due to lack of knowledge on evaluating the calls, this leads to incorrect data analysis.

          i.      Define guidelines to evaluate the calls

          ii.     Rating sheet should be simple & easy to understand

        iii.      Maker–Checker approach implemented

        iv.      Periodic (weekly) calibration session to be conducted to measure the GRR score and design an action plan for bottom performers

C.  System driven bias: In the case of automation, specific rules are predefined. These predefined rules should be revisited frequently to validate its applicability in the current situation. E.g., Logic set in SAS when a new product is launched; system calculation can be relaxed; however, as the process matures, the logics should be revisited.

          i.      Create SOP to define the frequency to revisit the system calculation

         ii.      Period internal audits to be conducted

D.  Judgment bias: The Selection of a candidate is influenced by certain pre-notions, which are not statistically validated. E.g., judgment bias exists in finalizing a woman candidate under the second career category, reflecting the disqualification of a candidate due to judgment.

          i.      Management to define clear criteria in the requisition for shortlisting the candidate

         ii.     To capture the summary of Selection & Rejection of a candidate from the interviewer and evaluate it

        iii.     Rejected candidates to be contacted & obtain their feedback on the interview process.  

 

 

Although medical writers have acknowledged the problem of reporting biases for over a century, it was not until the second half of the 20th century that researchers began to investigate root and size of the problem of the reporting biases. Depending on the outcomes, the decision to publish or not the research findings whether positive or negative.

Over the last 20 years, enough evidences have been collected that speaks that failure of publishing research studies which includes clinical trials and testing reports and their effectiveness is pervasive. Almost 90% of all failures to publish is due to the failure of researcher to submit. Rest of the failures are due to the rejection by journals.

Reporting Bias covers a range of different types of bias acting like an umbrella. Community is using biases since 17^th century for hundreds of years. Since then, different definitions of reporting bias have been proposed like “Selective revelation or suppression of information”, “when the dissemination of research findings is influenced by the nature and direction of results.”

The reporting bias of research comes to existence when the data or direction of significant results influence how the research is being reported. The definition of reporting biases is an element of the presented information from research due to the selective disclosure or withholding of information by parties involved with regards to the topic selected for study and the design, conduct, analysis, or dissemination of study methods, findings or both.

Researchers have define seven types of reporting biases including

1-Publication Bias

2-Time-lag bias

3-Multiple or duplicate publication bias

4-Location bias

5-Citation Bias

6-Language Bias

7-Outcome reporting bias

 

These seven types of reporting biases can be understood with the help of below diagram of a lifecycle of a research

Example of reporting bias :- When a review conducted by researchers, it was found that reporting bias is a widespread phenomenon in the medical literature. The researchers identified reporting biases in 50 types of pharmacological, surgical, diagnostic and preventative interventions which included the withholding of study data or the active attempt of manufacturers to suppress the publication of findings.

Later review conducted that compared the results of random trials under controlled environment with the subsequent peer-reviewed journal articles. Then it was found that there were discrepancies between prespecified and reported outcomes. 13% of trials introduced a new outcome in the published articles compared with those specified in the registered protocols.

Another study found considerable inconsistency in the reporting of adverse events when comparing sponsors databases with study protocols. Out of 22 studies, it was found that in 14, the number of adverse events in the sponsor’s database differed from the published articles by 20% and more. When more detailed information for interventions was analyzed for trials, 55% of the previous risk of bias assessments were reclassified from ‘low’ risk of bias to ‘high’

Trials and systematic reviews are used by clinicians and policymakers to develop evidence-based guidelines and make decisions about treatment or prevention of health problems. When the evidence base available to clinicians, policymakers or patients is incomplete or distorted, healthcare decisions and recommendations are made on biased evidence.

The first safety analysis of the largest study found a 79% greater risk of death or serious cardiovascular event in one treatment group compared with the other. This information was not disclosed by the manufacturer and the trial continued. The cardiovascular risk associated was obscured in several ways. A number of significant conflicts of interest among board members were undisclosed, and not made public while the trial was in progress or when it was published. This serves a classic example of mis-reporting and under-reporting.

If benefits are over-reported and harms are under-reported, clinicians, patients and the public will have a false sense of security about the safety of treatments. This results in unnecessary suffering and death which will misguides future research.

Preventive Steps: Transparency is the most important action that can save from under-reporting, over-reporting or mis-reporting and can avoid bigger loses in medical industry.

Pre-study: The results of prospectively registered trials are significantly more likely to be published than those of unregistered trials. Prospective registration of all clinical trials should be required, and encouraged for other study designs, by journal editors, regulators, research ethics committees, funders, and sponsors. 

During the study: Open science practices, such as making unidentified data and analytical code publicly available through digital platforms which aids reproducibility, prevents duplication, reduces waste, accelerates innovation, identifies errors and prevents reporting biases.

Post-study: Reporting guidelines such as CONSORT can help guide researchers to improve their reporting of randomized trials.

The most obvious evidences come from medical fields of publication bias from studies of research which are identified at the funding stage or at the time of approval of ethics. These studies have shown that “positive findings” is the principal factor associated with subsequent publication, researchers say that the reason they don't write up and submit reports of their research for publication is generally due to their disinterest in the outcomes.

Even the researchers who have published their results initially as the conference abstracts do not want to publish in full just because they are not satisfied with the results. This happens because of the data which is presented in abstracts is preliminary or interim result. And may not be reliable of what was found once all data were collected and analyzed. In addition, abstracts are often not accessible to the public through journals, or easily accessed databases. Many are published in conference programs, conference proceedings and are made available only to meeting registrants.

The main reason of failure to publish is negative or irrelevant findings. Controlled trials those are published in full are more likely to be positive results. Publication bias lead to underestimate the adverse effects of treatment, which in turn can lead doctors and decision makers to believe a treatment is more useful than it is.

It is now well-established that publication bias with more favorable efficacy results is associated with the source of funding for studies that would not otherwise be explained through usual risk of bias assessments.

Reporting bias means any analysis outcome achieved based on any fractional data or bias data which will give you wrong data interpretation and leads to inappropriate decisions.

Below are different types of bias reporting.

1> Outliers - Which will leads to wrong data interpretation. Outliers need to be separate it out from other data points & analyze the root cause to overcome it during analysis.

2> Confirmation - This leads to wrong data interpretation due to proving of something historical is right. To avoid this type of Bias need to take consent of  other people to finalize data interpretation and decision.

3> Selection bias - This leads to wrong outcome if sample selection is bias. To avoid this we need to follow proper sampling method during data collection.

4> Confounding bias - This leads to again wrong data interpretation & outcome due to influence of dependent and independent variables.To avoid this samples should be selected randomized. 

There are varied forms of reporting bias depending on the context, motivation and objectives of provider of report and consumers of report, which impacts/influences the quality of decision supposed to be taken basis the information shared. In any organizational context here are varies types of reporting bias 

 

1. Over-reporting : Providing over optimistic view consciously like by marketing on potential of certain campaign or by R&D on market potential or performance of certain product

 

2. Under-reporting : Providing pessimistic view consciously to influence certain decisions e.g., sales pipeline and hence sales target; capacity required to execute a project or timelines required to execute a project 

 

3. Delayed-reporting : Consciously delaying the information flow to upwards or downwards to manage impact of information (e.g., issues in operations )

 

4. Convenience - reporting : Reporting driven by availability of information to solve for immediate information needs. 

 

5. Confirmation - reporting :Consciously driving the data collection and subsequent analysis to prove/disprove already laid hypothesis as per individual preference

 

Such situations can be tackled via following approaches

 

1. Independent agency/set up for data collection (e.g., companies engage independent survey agencies to measure their CSAT)

2. Deploying scientific sampling techniques - Increasing sample size, Increase duration of sample collection, Ensuring representative sampling  of situation on ground or reality

3. Cross validation of analysis of reports via a triangulation mechanism (e.g., sales increased - did inventory reduced or production increased, did invoicing happened on time, did cash got collected from customers - were there any spike in returns)  

Some of the reporting bias types are bellowed mentioned,

 

1) Citation Bias
2) Language Bias
3) Location Bias
4) Duplicate Publication Bias
5) Outcome Reporting Bias
6) Publication Bias
7) Time lag bias

 

Some of the pre-cautions below mention which may help any organization to safe guard itself from reporting bias. 

 

1) Avoid single person based decision making. Should apply team based approach.
2) By not ignoring outliers, addressing them through proper root cause analysis
3) Accurate sample selection methods, competent sample selection team and validation.
4) Before implanting any important decision, should use simulation models technique wherever possible to get validate outcomes. 

In a typical services industry scenario, reporting biases are seen across operations where depending on the metric type (HTB or LTB) we see reporting bias which would make sure that we are meeting the specified targets or getting the acceptable outcome or keeping stakeholders happy.

These are typically seen where data is recorded manually and as the input for analysis is a biased set of data, the outcome of the analysis is ought to be faulty.

 

The best way to negate reporting bias is to automate the reports wherever it is feasible, and having measures to check the sanctity of the data if manual intervention is unavoidable.

 

Another example would be "exit polls", where if a certain demography of section is avoided to showcase the outcome is a classic case of reporting bias.

Reporting bias, as the name indicates is “An inclination for a pre-conceived outcome” or “Manipulation of inputs / tweaking factors while keeping personal interests at the forefront”. Can also be called as the difference between “Selective Revealing Vs Actual Truth”

 

Factors leading to bias – First step in order to safeguard organization from reporting bias is to make an assessment of various factors which might lead to bias depending on the business type. Needless to mention, every business type may have completely different factors leading to reporting bias (e.g. A Manufacturing set-up Vs a Service set-up will have different factors / degrees and so on). An example of various factors in News industry are mentioned -

a)      Advertising Bias - When stories are selected to please advertisers.

b)      Political Bias – Tweaking news in favour of any particular political party.

c)      Coverage Bias – Increasing visibility / coverage of less relevant content with respect to other important content.

d)      Gatekeeping bias - When stories are selected or deselected, sometimes on ideological grounds (Also referred to as agenda bias), when the focus is on political actors and whether they are covered based on their preferred policy issues.

e)      False Balance - When an issue is presented as even sided, despite disproportionate amounts of evidence.

 

Steps to safeguard viewer’s interest (by avoid any Reporting biases)-

a)       Vet the publisher’s credibility.

b)      Pay attention to quality and timeliness

c)       Check the sources and citations

d)      Perform reverse searches for sources and images (sample based – to confirm about the correctness of critical information)

e)      Check if the information is available on other channels / sites / social platforms (or if the channel is into coverage bias).

 

Bias is the incorrect information or data than the true information or data.

if the information or data is incorrect which will result to lead the wrong conclusions and decisions.

Reporting bias is a nothing but hiding of information than actual conditions.

for example:  before joining any organization medical checkup is mandatory in now a days and will follow the same . while medical investigation doctor will ask few questions like for smoking and drinking but patient will not tell actuals and  will hide the actual information.

Types of reporting Bias:

there are seven types of reporting biases.

1. publication bias : during the R&D studies, will conduct so many trials to develop the product but in majority of cases will not research more on the negative studies and will not report the negatives and will not perform the investigation for negative impacts and only they will mention the positives only.

2.time log bias: time taken for research and trails testing will not mention accurate for the product development . stability studies data will not include in the product launch. they will consider the time for lunching is trials to manufacturing.

3.multiple publication bias; by changing small content in process, filing the dmf for process.

4.location bias

5.citation bias

6.laungage bias

7.knowledge reporting bias

 

All published answers are correct and great. There are 4 winners for this question - Shashikant Adlakha, Satyajit Das, Senthilkumar G and Mala Pulickel for clearly highlighting how reporting bias should be avoided.

 

Also review the answer provided by Mr Venugopal R, Benchmark Six Sigma's in-house expert.