About: Information bottleneck method     Goto   Sponge   NotDistinct   Permalink

An Entity of Type : yago:WikicatDataClusteringAlgorithms, within Data Space : dbpedia.demo.openlinksw.com associated with source document(s)
QRcode icon
http://dbpedia.demo.openlinksw.com/describe/?url=http%3A%2F%2Fdbpedia.org%2Fresource%2FInformation_bottleneck_method&invfp=IFP_OFF&sas=SAME_AS_OFF&graph=http%3A%2F%2Fdbpedia.org&graph=http%3A%2F%2Fdbpedia.org

The information bottleneck method is a technique in information theory introduced by Naftali Tishby, Fernando C. Pereira, and William Bialek. It is designed for finding the best tradeoff between accuracy and complexity (compression) when summarizing (e.g. clustering) a random variable X, given a joint probability distribution p(X,Y) between X and an observed relevant variable Y - and self-described as providing "a surprisingly rich framework for discussing a variety of problems in signal processing and learning".

AttributesValues
rdf:type
rdfs:label
  • Information bottleneck method (en)
  • 信息瓶颈 (zh)
rdfs:comment
  • 信息瓶颈（英語：information bottleneck）是信息论中的一种方法，由、费尔南多·佩雷拉（Fernando C. Pereira）与于1999年提出。对于一随机变量，假设已知其与观察变量之间的联合概率分布。此时，当需要概括（聚类）时，可以通过信息瓶颈方法来分析如何最优化地平衡准确度与复杂度（数据压缩）。该方法的应用还包括分布聚类（distributional clustering）与降维等。此外，信息瓶颈也被用于分析深度学习的过程。 信息瓶项方法中运用了互信息的概念。假设压缩后的随机变量为，我们试图用代替来预测。此时，可使用以下算法得到最优的： 其中与分别为与之间、以及与之间的互信息，可由计算得到。则表示拉格朗日乘数。 (zh)
  • The information bottleneck method is a technique in information theory introduced by Naftali Tishby, Fernando C. Pereira, and William Bialek. It is designed for finding the best tradeoff between accuracy and complexity (compression) when summarizing (e.g. clustering) a random variable X, given a joint probability distribution p(X,Y) between X and an observed relevant variable Y - and self-described as providing "a surprisingly rich framework for discussing a variety of problems in signal processing and learning". (en)
foaf:depiction
  • http://commons.wikimedia.org/wiki/Special:FilePath/BottleCateg_1.jpg
dcterms:subject
Wikipage page ID
Wikipage revision ID
Link from a Wikipage to another Wikipage
Link from a Wikipage to an external page
sameAs
dbp:wikiPageUsesTemplate
thumbnail
has abstract
  • The information bottleneck method is a technique in information theory introduced by Naftali Tishby, Fernando C. Pereira, and William Bialek. It is designed for finding the best tradeoff between accuracy and complexity (compression) when summarizing (e.g. clustering) a random variable X, given a joint probability distribution p(X,Y) between X and an observed relevant variable Y - and self-described as providing "a surprisingly rich framework for discussing a variety of problems in signal processing and learning". Applications include distributional clustering and dimension reduction, and more recently it has been suggested as a theoretical foundation for deep learning. It generalized the classical notion of minimal sufficient statistics from parametric statistics to arbitrary distributions, not necessarily of exponential form. It does so by relaxing the sufficiency condition to capture some fraction of the mutual information with the relevant variable Y. The information bottleneck can also be viewed as a rate distortion problem, with a distortion function that measures how well Y is predicted from a compressed representation T compared to its direct prediction from X. This interpretation provides a general iterative algorithm for solving the information bottleneck trade-off and calculating the information curve from the distribution p(X,Y). Let the compressed representation be given by random variable . The algorithm minimizes the following functional with respect to conditional distribution : where and are the mutual information of and , and of and , respectively, and is a Lagrange multiplier. (en)
  • 信息瓶颈（英語：information bottleneck）是信息论中的一种方法，由、费尔南多·佩雷拉（Fernando C. Pereira）与于1999年提出。对于一随机变量，假设已知其与观察变量之间的联合概率分布。此时，当需要概括（聚类）时，可以通过信息瓶颈方法来分析如何最优化地平衡准确度与复杂度（数据压缩）。该方法的应用还包括分布聚类（distributional clustering）与降维等。此外，信息瓶颈也被用于分析深度学习的过程。 信息瓶项方法中运用了互信息的概念。假设压缩后的随机变量为，我们试图用代替来预测。此时，可使用以下算法得到最优的： 其中与分别为与之间、以及与之间的互信息，可由计算得到。则表示拉格朗日乘数。 (zh)
prov:wasDerivedFrom
page length (characters) of wiki page
foaf:isPrimaryTopicOf
is Link from a Wikipage to another Wikipage of
is foaf:primaryTopic of
Faceted Search & Find service v1.17_git139 as of Feb 29 2024


Alternative Linked Data Documents: ODE     Content Formats:   [cxml] [csv]     RDF   [text] [turtle] [ld+json] [rdf+json] [rdf+xml]     ODATA   [atom+xml] [odata+json]     Microdata   [microdata+json] [html]    About   
This material is Open Knowledge   W3C Semantic Web Technology [RDF Data] Valid XHTML + RDFa
OpenLink Virtuoso version 08.03.3330 as of Mar 19 2024, on Linux (x86_64-generic-linux-glibc212), Single-Server Edition (378 GB total memory, 67 GB memory in use)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2024 OpenLink Software