Top: Science: Mathematics: Applied: Cybernetics: K1 and K2 - General Cybernetics: Information Theory



[ history ]

Description

Information Theory deals with the statistics of signals and information processing. Founded in the "midcentury" (of the 20th century) by scientists like Claude Shannon, Warren Weaver and Norbert Wiener this branch of cybernetics (reffered to as Information Theory or Communication Theory) provides a methodology for a statistical description of signals and communication processes.

Reference: Shannon, C. E. und W. Weaver (1949, 1998). The Mathematical Theory of Communication. Urbana and Chicago, University of Illinois Press.


[ history ]

Information Content

The uncertainty (and hence the information content) of a random event i can quantitatively be described as negative logarithm of its probability with

I = - ld pi

where ld denotes the dual logarithm as ld M = lg M / lg 2 = ln M / ln 2


[ history ]

Shannon's Equation

The quantity of information produced by an ergodic source is given with

H = - K Sigma i=1 n (pi ld pi).

H is referred to as the entropy of the information source.


[ history ]

Redundancy

Difference of actually transfered information and the minimum required information for carrying the semantics. Positive redundancy is able to reduce the effect of errors. 



 All text is available under the terms of the GNU Free Documentation License. (See Copyright Policy for details.)
© Open-Site Foundation, Inc.
Hosted by Android Technologies, Inc. the medical robotics news source.
Visit our sister sites dmoz.org | mozilla.org | chefmoz.org | musicmoz.org