Q234. The most frequent English word occurs approximately twice as often as the second most frequent word, three times as often as the third most frequent word, and so on. This is a pattern seen in certain phenomena and was popularised by George Kingsley Zipf. Explain other applications of Zipf’s law and its practical utility. 

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term

Zipf’s  law as proposed by American linguist George Kingsley Zipf, states that frequency of any word in a language relates inversely with the rank of that word in frequency table. Frequency is the number of time, the word is appearing in the sample or a text .

-The most commonly evident word in a corpus suppose has a  frequency f

-The second most appearing word , will have a  frequency around f/2

-The third most appearing word would have frequency nearly  f/3

-The fourth most appearing word would have frequency around f/4

Zipf’s  law can be applied in many other data and rankings such as:

- Poulation ranks of cities in various countries

 -Corporation size

 -Income rankings of richest persons

 -Ranks or number of people watching the same TV channel

  -Temprature trends over recent years

  -Facebook likes of favourite teams

 - Number of citations to papers 

 - Number of hits on web sites

- Copies of books sold in the US

- Telephone calls received

- Magnitude of earthquakes 

- Diameter of moon  craters

-  Intensity of solar flares

-  Intensity of wars

- Net worth of Americans 

 -Frequency of family names

The level of fit between the  data and Zipf’s distribution, can be tested by Kolmogorov-Smirnov test and then it  be compared with the fits to alternative distributions, like  lognormal, exponential distribution.

Zipf’s Law is more or less in compliance   with one of the most widely acclaimed economical and statistical  principles ‘Pareto Principal’. The Zipf distribution is also labelled as  the discrete Pareto distribution,  as it includes primarily the discrete data and deals with   frequency and rankings.

The  Pareto principle states that 20% of the invested input  accounts for 80% of output. 20 % of work-related input yields 80% of the results. Similarly Zipf’s law accounts for the fact that few of the words, only first 20% of words, accounts for 80% frequency of entire corpus.

The probability mass function(pmf) of the Zipf distribution is

 F(x)= C/x^s  , C-  Calculated Constant, x=1, 2, 3-------------------n

S- Value of exponent, characterizing the distribution

 

The winner is Shashikant Adlakha for his excellent answer.