Sachin Tanwar

What is Quantile-Based Analysis?

Imagine you have a pile of rocks. You want to understand how big the rocks are but just looking at the biggest and smallest ones won't tell you the whole story. Quantile-based analysis is like sorting the rocks into five equal groups based on their size. It helps you understand the distribution of rock sizes, not just the extremes.
 
Let review a real-world example. Let's say we're looking at salaries in a medium-sized BPO. Instead of just saying "the average salary is $75,000," quantile analysis helps us see the full picture. Here's a simple salary quantile breakdown:
 
Quantile Salary Range What It Tells Us 20th Percentile $55,000 20% of employees earn at or below this 40th Percentile $65,000 40% of employees earn at or below this 60th Percentile $80,000 60% of employees earn at or below this 80th Percentile $110,000 80% of employees earn at or below this  
Now post evaluating the above spread, we can see that average salary might be $75,000, but most people are not exactly earning that, some are way high, and some are way too low. This is a classic example.
 
Quantile analysis is a sophisticated technique that provides many benefits such as:
It doesn't fall apart in the face of extreme values in the same way a mean does It reveals the actual real-world distribution, not just a single number It detects inequalities or patterns that may be concealed by the averages Even though it's inevitable. Disabilities include:
Needs a satisfactory amount of data to be meaningful Might be hard to communicate to those who only like simple numbers Has to be done quite frequently by software generating statistical data to make exact calculations Suppose for example that you're a city planner who wants to know where people can buy homes at various prices. The mean may tell you "$300,000," whereas the quantiles can show you that:
20% of them are the only ones who can afford houses that are under $200,000 80% are the ones who will not buy any property beyond the price of $450,000 This allows you to learn about housing inequality in a different, perhaps more comprehensive way compared with the usual method using entire datasets.
 
The analysis of quantiles is like X-ray vision in the fields of data science and research. It allows you to look past the superficial numbers and get a handle on the information that is really being communicated.
 
Always remember: Number will tell the stories, but quantiles will help you read between the lines. 

Named Entity Recognition systems are used to identify specific entities in the text, such as people, places, or organizations. More often than not, though, these systems are constrained by ambiguity. A word can denote more than one meaning, so that ambiguity can occur when the system is not certain about the proper meaning for a particular context.

Strategies for Handling Ambiguity:

Contextual Analysis:  NER systems take into account the words around a potential ambiguous term to dissect what that term means. Consider the word "Orange", which could refer to either a fruit or a company providing logistics support. If it is surrounded with words like "Warehouse" or "inventory," it is more likely to be identified as the technology company.

Gazetteers: They are list of entities along with their types. If a word can be found in a gazetteer, then the system is more likely to identify it as the listed entity.

Machine Learning - Advanced NER makes use of machine learning algorithms to learn for large, labeled datasets of text. Machine learning identifies patterns and relationships that will allow the system to make better predictions.

Techniques for Improving Accuracy:

Quality of Training Data: Quality of the training data is critical. If the noisy and inconsistent data are fed to the system, it will most certainly produce incorrect results.

Feature engineering: building informative features can enable the system to have a better appreciation of the context in which a word is being used. As such, it could be essential to include features like whether it is part of speech, whether it has been capitalized, and distance from other entities.

Ensemble Methods: The accuracy of a number of multiple NER systems can be enhanced by combining these together. These different systems have their strengths and weaknesses, and by combining them, errors from individual systems are decreased.

Domain Knowledge: If the domain is medicine or law, then the addition of domain knowledge helps them to understand the nuances of language.

By employing these strategies and techniques, NER systems can become more accurate and reliable in real-world applications.

The illusion of control is a psychological phenomenon where people overestimate their ability to influence or predict outcomes. In simpler terms, it's believing you have more power over events than you actually do. While a certain degree of confidence is essential for leadership, an excessive sense of control can be detrimental to an organization's progress.
Negative Impacts on Business Excellence:
Impact Area Description Decision Making Overconfidence can lead to hasty decisions without thorough analysis, ignoring potential risks and alternative options. Risk Management Underestimating uncertainties and relying solely on internal capabilities can expose the organization to unforeseen challenges. Innovation Excessive control can stifle creativity and experimentation, hindering the development of new ideas and approaches. Team Dynamics A controlling leadership style can demotivate employees, reduce collaboration, and hinder knowledge sharing. Organizational Learning The tendency to attribute successes to personal efforts and failures to external factors can impede learning from mistakes and adapting to change. Few Remedial Measure a firm can take:
Promote a Culture of Questioning: Encourage open dialogue, challenge assumptions, and foster a culture where diverse perspectives are valued. Develop Robust Risk Management Systems: Implement comprehensive processes to identify, assess, and mitigate risks. Foster a Learning Organization: Create an environment that encourages experimentation, failure, and continuous improvement. Empower Employees: Delegate authority, provide autonomy, and build trust to enhance employee engagement and innovation. Enhance Decision-Making Processes: Establish structured decision-making frameworks that involve multiple stakeholders and consider various scenarios. Seek External Perspectives: Bring in external experts or consultants to provide fresh insights and challenge internal assumptions. By recognizing the potential pitfalls of the illusion of control and taking proactive steps to address it, organizations can enhance their decision-making, risk management, and overall performance, ultimately driving business excellence.
 
In essence, while a sense of control is important, it's crucial to balance it with humility, open-mindedness, and a willingness to adapt to changing circumstances.

The clustering illusion, or Texas Sharpshooter Fallacy, can seriously affect how organizations make decisions based on data. It happens when patterns or clusters in data are perceived as meaningful when they're actually due to random chance or unrelated factors. This can lead to misguided strategies and poor decisions.

Impact on Data-Driven Decision Making:
False Patterns: Imagine a marketing team analyzing customer reviews for a new product. They notice a cluster of positive reviews and assume it signifies overall success. However, this cluster might be random noise or influenced by other factors. Relying solely on this cluster could lead to misguided resource allocation.
  Overconfidence: When we spot clusters, we tend to become overconfident in our predictions. Organizations might base critical decisions on these perceived patterns, ignoring other relevant information. For instance, a sudden spike in website traffic during a specific hour could lead to an erroneous conclusion about peak user engagement.
  Resource Allocation: Organizations may allocate resources disproportionately based on perceived clusters. For instance, a sales team might focus on a specific customer segment due to a recent sales spike, neglecting other segments that could yield better long-term results.
An example:

A retail chain analyzes customer purchases and notices a correlation between people buying peanut butter and diapers. They launch a marketing campaign promoting these products together, assuming parents always buy them at once. Turns out, it was just a coincidence. People buy both products frequently, but not necessarily together. The clustering illusion led to a potentially wasteful marketing campaign.

How can we avoid it?

Beware of cherry-picking: Don't focus only on data that supports your initial hunch. Look at the bigger picture and consider alternative explanations.

Statistical significance is your friend: Don't jump to conclusions based on small samples. Use statistical tests to see if the patterns you see are likely due to chance.

Seek diverse perspectives: Discuss your analysis with colleagues from different departments. A fresh set of eyes can help spot potential biases in your interpretation.

Focus on the "why" behind the data: Don't just see patterns, understand the reasons behind them. Investigate further before making big decisions.

In Conclusion, by being aware of the clustering illusion and taking these steps, we can ensure our data-driven decisions hit the real bullseye – sustainable success for the organization.

You know how sometimes our choices change just because of the way something is said or shown to us? That’s called the Framing Effect. It’s like when a restaurant menu says "95% fat-free" instead of "contains 5% fat"—the first one just sounds better, right? But how can we make sure we’re making decisions based on the real facts, not just clever wording.

Here are some methods to avoid this behavior and have data driven decision making:

Double-Check the Numbers: Always look at the raw data. If someone says, "This strategy improved sales by 30%," find out what the numbers were before and after. For instance, "Our sales went from $100,000 to $130,000." It’s clearer and gives you a real picture.

Ask for Both Sides: Try to see the same information framed in different ways. If a project is presented as having a "90% success rate," ask what the failure rate is. Sometimes, hearing "10% failure rate" can change your perspective and help you make a more balanced decision.

Use a Consistent Framework: Develop a standard method for evaluating information. Whether it’s a spreadsheet or a checklist, having a consistent process ensures that you’re comparing apples to apples. For example, if you’re deciding on a vendor, always look at cost, quality, and delivery time in that order.

Critical Thinking: Always question the context. Why is this information being presented this way? Is there an agenda? For example, if a report highlights how much time a new software saves but doesn’t mention the cost, dig deeper. Maybe it saves time but at a high expense.

Discuss with Others: Get opinions from different people. Different perspectives can highlight biases you might have missed. For example, discuss a potential business strategy with both the finance and marketing teams to get a well-rounded view.

A real-life example:

Imagine you’re at a team meeting, and your boss, Ravi, presents two options for a new marketing campaign. He says, "Option A has a 70% chance of success, and Option B has a 30% chance of failure." They sound different, right? But they’re actually the same. To avoid the framing effect, focus on the underlying data, like past campaign performances, budget requirements, and potential ROI.

By following above mentioned steps, you’ll be better equipped to make decisions based on solid data, no matter how or who framed the information. It’s all about looking past the surface and digging into the real details.

The Bandwagon Effect is when people start adopting a product or behavior just because it's becoming popular. This can definitely mess with logical decision making. Instead of thinking things through and making choices based on facts and analysis, people might just go with the flow. It's like everyone's hopping on a train because it's getting crowded, not because it's the best or safest option.

1. Impact on Logical Decision Making:
The Bandwagon Effect can distort logical reasoning in several ways: Herding Behavior: People may follow the crowd without critically evaluating the merits of an option. This can lead to poor decisions. Social Proof: The popularity of a product or idea becomes a heuristic for quality or correctness, bypassing rigorous evaluation. Fear of Missing Out (FOMO): Individuals fear being left behind, prompting them to join the bandwagon even if it contradicts their initial judgment. Confirmation Bias: Once a choice gains momentum, people seek evidence that supports it, reinforcing their decision.  
2. Mitigating the Bandwagon Effect: Organizations can take proactive steps to minimize the impact of the Bandwagon Effect:
Critical Thinking Training:
Encourage employees to think critically and independently. Provide training on logical reasoning, cognitive biases, and decision-making. Teach them to evaluate options based on objective criteria rather than popularity. Diverse Perspectives:
Foster an organizational culture that values diverse viewpoints. Encourage dissent and constructive disagreement during decision-making processes. Diverse perspectives can counteract herd behavior. Evidence-Based Decision-Making:
Base decisions on data, research, and evidence rather than following trends. Conduct thorough analyses, including cost-benefit assessments and risk evaluations. Delayed Judgment:
Encourage employees to pause before making decisions. Implement mechanisms (such as decision-making committees) that allow time for reflection and evaluation. Transparency and Accountability:
Clearly communicate the rationale behind decisions. Hold decision-makers accountable for their choices. Transparency reduces blind conformity. Independent Audits:
Periodically review decisions and their outcomes. Assess whether the organization fell prey to the Bandwagon Effect. Adjust strategies accordingly.  
Real-World Examples:
Investment Bubbles: The Dot-com bubble of the late 1990s and the housing bubble in the mid-2000s are classic examples of the Bandwagon Effect. Investors followed the crowd, leading to unsustainable valuations. Fashion Trends: Fashion industries thrive on the Bandwagon Effect. Consumers often buy what’s in vogue, regardless of practicality or personal preference. Remember that while trends and popularity can provide useful information, critical thinking and independent analysis are essential for making sound decisions. Organizations that actively promote these skills can avoid blindly following the bandwagon and make more informed choices.

You know how some stores have both physical shops and online websites? That's the Bricks and Clicks thing. It's like having the best of both worlds.

Think about it like this: You can walk into a store, touch the stuff, try it out, and get help from real people. That's the "bricks" part. But then, if you're feeling lazy or just prefer shopping in your pajamas, you can hop online and buy the same stuff. That's the "clicks" part.

Benefits of Brick and Clicks in Retail Industry:

Expanded Reach and Market Growth: Offering both physical and online options allows retailers to reach a larger audience. Omnichannel clients often have a 30% higher lifetime value than single-channel shoppers.

Improved Access and Convenience: Customers may select how and where they interact with the company, whether it's online or in-store.

Enhanced Customer Experiences: The seamless transition between channels creates a holistic experience. Customers, for example, can place an order online and pick it up in store.
Improved Inventory Management: Unified inventory systems eliminate stockouts and excess inventory.

Higher Revenue and Lower Overhead: Using online shopping data, retailers can optimise their physical sites while lowering lease costs.

Few Companies Examples: 

Let's start with Walmart. I take it that they have physical storefronts all around? However, they're also very popular online. They have an online store where you can place orders and pick them up in person or have them delivered to your home. They seem to have combined the greatest aspects of both worlds.

And there's Ikea, the furniture company, as you may know. Most likely, you've visited one of their enormous stores. They are more than simply the showroom experience, though. You can place orders by visiting their website. They even offer door-to-door delivery services. Thus, you're covered whether you're perusing in-store or surfing the web.

Sephora. They specialise in beauty and skincare, and they provide a great loyalty programme. It works whether you shop online or in-store. You get points either way. They seem to want to reward you regardless of how you shop.

Simply put, these stores demonstrate how combining physical and digital shopping can be transformative. They're all about providing options and ensuring a positive experience, whether you're tapping away on your phone or wandering through the aisles. And you know what? It's working for them, increasing revenue and keeping customers satisfied.

Sample Size in Control Charts: 
Sample size for control charts is chosen based on factors like process variation, stability, and resource constraints.

If the sample size is too low:
It may miss important process changes Estimates of process variation may be unreliable If the sample size is too high:
It can be costly and time-consuming The chart may overreact to small changes, leading to not meaningful variations Finding the right balance of samples [variable] and range [Time frame] is key for effective monitoring and control. 

Attaching my practice control chart image, where the sample count [ Week on week Quality Scores ] was enough to determine my process health.