Lecture 4: Empirical Laws

[@pawanchoure1289]

Zipf's law is an empirical law, formulated using mathematical statistics, named after the linguist George Kingsley Zipf, who first proposed it. Zipf's law states that given a large sample of words used, the frequency of any word is inversely proportional to its rank in the frequency table

[@pawanchoure1289]

Heaps' law means that as more instance text is gathered, there will be diminishing returns in terms of discovery of the full vocabulary from which the distinct terms are drawn. ... Heaps' law has been observed also in single-cell transcriptomes considering genes as the distinct objects in the "vocabulary".

[@pawanchoure1289]

Rank - The rank of a word can be defined as the position occupied, for having the highest number of occurrence in the given document or corpus. A word with the highest frequency will have the highest rank

[@pratham.malviya]

21:45 i think it should be f2=3f1 as f2 is 150 and f1 is 50 that's why

[@rishabhbandi3368]

Its correct. the rank r1, r2 is 50 and 150 respectively. since frequency is inversely proportional to rank, hence we can say roughly f1 = 3f2 (because r2=3f1).

[@rahuldebdas2374]

You used the concept of structure of a sentence in several places. I am wondering is it structure or semantics of the sentence?

[@pawanchoure1289]

type , token , type-token ratio

[@thiruveeran1001]

i cant understand how frequency is inversely proportional to rank. its directly proportional na

[@samklopp3172]

It's inversely proportional, higher the frequency gets rank 1, 4000 frequency got 1, 2000 got 2, hence inversely proportional.

[@atharva-naik]

We are talking about numerical value of rank, so rank 1 is lower than rank 2

[@chandan174saha]

How can u write f1=3f2, while its clearly 3f1=f2. Please correct me but i think the equation you shown wasn't correct.

[@samklopp3172]

f1 is frequency, 50 is the rank of that word.f1 *50 = f2 *150. That's how you get the relation. Hope it is clear now.

[@louerleseigneur4532]

Thanks a lot sir