Minimum description length (Information theory) (Topical Term)

Work cat.: 2004055932: Advances in minimum description length, 2005: p. 3 ("The minimum description length (MDL) principle is a relative recent method for inductive inference that provides a generic solution to the model selection problem. MDL is based on the following insight: any regularity in the data can be used to compress that data, that is, to describe it using fewer symbols than the number of symbols needed to describe the data literally. The more regularities there are, the more the data can be compressed.") p. 66 ("MDL shares some ideas with the minimum message length (MML) principle which predates MDL by ten years... Just as in MDL, MML chooses the hypothesis minimizing the code length of the data. But the codes that are used are quite different from those in MDL. First of all, in MML one always uses two-part codes, so that MML automatically selects both a model family and parameter values.")

Wikipedia, Aug. 16, 2005 (minimum description length: "a formalization of Occam's Razor in which the best hypothesis for a given set of data is the one that leads to the largest compression of the data. MDL is important in information theory and learning theory... MDL was not the first attempt to do hypothesis selection through minimizing description length; as early as 1968 Wallace and Boulton pioneered a related concept called Minimum Message Length (MML). MDL was introduced by Jorma Rissanen in 1978; it differs from MML in several ways, most notably (at least in most of J. Rissanen's early MLD papers) in its extensive use of one-part rather than two-part codes.")

TR-AnTOB Op 07.06.2020