N P(ai) = 1 i=1 (Probability Distribution) Given or Developed fundamental theorem Shannon’s Entropy: N = - P(ai) log2 [P(ai)] (p.22) H i=1 Minimum (theoretical) bit rate at which the group of symbols can be transmitted, # of bits/symbol. Huffman code is a VWL code. Very close to entropy. Practical code. Contributes to compression. P (a1) 1/2 P (a2) 1/8
N P(ai) = 1 i=1 (Probability Distribution) Given or Developed Shannon’s fundamental theorem Entropy: N H = - P(ai) log2 [P(ai)] (p.22) i=1 Minimum (theoretical) bit rate at which the group of symbols can be transmitted, # of bits/symbol. Huffman code is a VWL code. Very close to entropy. Practical code. Contributes to compression. P (a1) 1/2 P (a2) 1/8
Entropynew.docx