Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The Minimal Hoppe-Beta Prior Distribution for Directed Acyclic Graphs and Structure Learning
KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematical Statistics.ORCID iD: 0000-0003-1489-8512
KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematical Statistics.ORCID iD: 0000-0002-6886-5436
(English)Manuscript (preprint) (Other academic)
Abstract [en]

The main contribution of this article is a new prior distribution over directed acyclic graphs intended for structured Bayesian networks, where the structure is given by an ordered block model. That is, the nodes of the graph are objects which fall into categories or blocks; the blocks have a natural ordering or ranking. The presence of a relationship between two objects is denoted by a directed edge, from the object of category of lower rank to the object of higher rank. The models considered here were introduced in Kemp et al. [7] for relational data and extended to multivariate data in Mansinghka et al. [12].

We consider the situation where the nodes of the graph represent random variables, whose joint probability distribution factorises along the DAG. We use a minimal layering of the DAG to express the prior. We describe Monte Carlo schemes, with a similar generative that was used for prior, for finding the optimal a posteriori structure given a data matrix and compare the performance with Mansinghka et al. and also with the uniform prior. 

Keyword [en]
Graphical models, Bayesian networks, structure learning, DAG prior
National Category
Probability Theory and Statistics
Research subject
Applied and Computational Mathematics
Identifiers
URN: urn:nbn:se:kth:diva-180327OAI: oai:DiVA.org:kth-180327DiVA: diva2:892655
Note

QC 20160524

Available from: 2016-01-11 Created: 2016-01-11 Last updated: 2017-09-15Bibliographically approved
In thesis
1. Bayesian structure learning in graphical models
Open this publication in new window or tab >>Bayesian structure learning in graphical models
2016 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis consists of two papers studying structure learning in probabilistic graphical models for both undirected graphs anddirected acyclic graphs (DAGs).

Paper A, presents a novel family of graph theoretical algorithms, called the junction tree expanders, that incrementally construct junction trees for decomposable graphs. Due to its Markovian property, the junction tree expanders are shown to be suitable for proposal kernels in a sequential Monte Carlo (SMC) sampling scheme for approximating a graph posterior distribution. A simulation study is performed for the case of Gaussian decomposable graphical models showing efficiency of the suggested unified approach for both structural and parametric Bayesian inference.

Paper B, develops a novel prior distribution over DAGs with the ability to express prior knowledge in terms of graph layerings. In conjunction with the prior, a search and score algorithm based on the layering property of DAGs, is developed for performing structure learning in Bayesian networks. A simulation study shows that the search and score algorithm along with the prior has superior performance for learning graph with a clearly layered structure compared with other priors.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2016. viii, 19 p.
Series
TRITA-MAT-A, 2015:16
Keyword
Bayesian statistics, graphical models, Bayesian networks, Markov networks, structure learning
National Category
Probability Theory and Statistics
Research subject
Applied and Computational Mathematics
Identifiers
urn:nbn:se:kth:diva-179852 (URN)978-91-7595-832-3 (ISBN)
Presentation
2016-01-28, Rum 3418, Instititionen för matematik, Lindstedtsvägen 25, Kungliga Tekniska Högskolan, Stockholm, 14:00 (English)
Opponent
Supervisors
Note

QC 20160111

Available from: 2016-01-11 Created: 2016-01-04 Last updated: 2016-01-11Bibliographically approved
2. Bayesian inference in probabilistic graphical models
Open this publication in new window or tab >>Bayesian inference in probabilistic graphical models
2017 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis consists of four papers studying structure learning and Bayesian inference in probabilistic graphical models for both undirected and directed acyclic graphs (DAGs).

Paper A presents a novel algorithm, called the Christmas tree algorithm (CTA), that incrementally construct junction trees for decomposable graphs by adding one node at a time to the underlying graph. We prove that CTA with positive probability is able to generate all junction trees of any given number of underlying nodes. Importantly for practical applications, we show that the transition probability of the CTA kernel has a computationally tractable expression. Applications of the CTA transition kernel are demonstrated in a sequential Monte Carlo (SMC) setting for counting the number of decomposable graphs.

Paper B presents the SMC scheme in a more general setting specifically designed for approximating distributions over decomposable graphs. The transition kernel from CTA from Paper A is incorporated as proposal kernel. To improve the traditional SMC algorithm, a particle Gibbs sampler with a systematic refreshment step is further proposed. A simulation study is performed for approximate graph posterior inference within both log-linear and decomposable Gaussian graphical models showing efficiency of the suggested methodology in both cases.

Paper C explores the particle Gibbs sampling scheme of Paper B for approximate posterior computations in the Bayesian predictive classification framework. Specifically, Bayesian model averaging (BMA) based on the posterior exploration of the class-specific model is incorporated into the predictive classifier to take full account of the model uncertainty. For each class, the dependence structure underlying the observed features is represented by a distribution over the space of decomposable graphs. Due to the intractability of explicit expression, averaging over the approximated graph posterior is performed. The proposed BMA classifier reveals superior performance compared to the ordinary Bayesian predictive classifier that does not account for the model uncertainty, as well as to a number of out-of-the-box classifiers.

Paper D develops a novel prior distribution over DAGs with the ability to express prior knowledge in terms of graph layerings. In conjunction with the prior, a stochastic optimization algorithm based on the layering property of DAGs is developed for performing structure learning in Bayesian networks. A simulation study shows that the algorithm along with the prior has superior performance compared with existing priors when used for learning graph with a clearly layered structure.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2017. 21 p.
Series
TRITA-MAT-A, 2017:03
Keyword
Graphical models, Bayesian inference, predictive classification, decomposable graphs
National Category
Probability Theory and Statistics
Research subject
Applied and Computational Mathematics
Identifiers
urn:nbn:se:kth:diva-214542 (URN)978-91-7729-525-9 (ISBN)
Public defence
2017-10-11, F3, Lindstedtsvägen 26, Stockholm, 13:00 (English)
Opponent
Supervisors
Note

QC 20170915

Available from: 2017-09-15 Created: 2017-09-14 Last updated: 2017-09-15Bibliographically approved

Open Access in DiVA

fulltext(558 kB)34 downloads
File information
File name FULLTEXT01.pdfFile size 558 kBChecksum SHA-512
1e9edc419a46f60b894410653a2d0c4fd633b5eb54e2c487209e6438b0a50bb98dbc769841ad1d1a0bad2664055deadfa101fb882664cdf9e90fde38686789d7
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Koski, Timo J. T.Noble, John M.Rios, Felix L.
By organisation
Mathematical Statistics
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 34 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 104 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf