Change search
Refine search result
1 - 15 of 15
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1. Dang, Khue-Dung
    et al.
    Quiroz, Matias
    Kohn, Robert
    Minh-Ngoc, Tran
    Villani, Mattias
    Stockholm University, Faculty of Social Sciences, Department of Statistics. Linköping University, Sweden; ARC Centre of Excellence for Mathematical and Statistical Frontiers (ACEMS), Australia.
    Hamiltonian Monte Carlo with Energy Conserving Subsampling2019In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 20, p. 1-31, article id 100Article in journal (Refereed)
    Abstract [en]

    Hamiltonian Monte Carlo (HMC) samples efficiently from high-dimensional posterior distributions with proposed parameter draws obtained by iterating on a discretized version of the Hamiltonian dynamics. The iterations make HMC computationally costly, especially in problems with large data sets, since it is necessary to compute posterior densities and their derivatives with respect to the parameters. Naively computing the Hamiltonian dynamics on a subset of the data causes HMC to lose its key ability to generate distant parameter proposals with high acceptance probability. The key insight in our article is that efficient subsampling HMC for the parameters is possible if both the dynamics and the acceptance probability are computed from the same data subsample in each complete HMC iteration. We show that this is possible to do in a principled way in a HMC-within-Gibbs framework where the subsample is updated using a pseudo marginal MH step and the parameters are then updated using an HMC step, based on the current subsample. We show that our subsampling methods are fast and compare favorably to two popular sampling algorithms that use gradient estimates from data subsampling. We also explore the current limitations of subsampling HMC algorithms by varying the quality of the variance reducing control variates used in the estimators of the posterior density and its gradients.

  • 2.
    Dang, Khue-Dung
    et al.
    Univ New South Wales, Australia; ARC Ctr Excellence Math and Stat Frontiers ACEMS, Australia.
    Quiroz, Matias
    ARC Ctr Excellence Math and Stat Frontiers ACEMS, Australia; Univ Technol Sydney, Australia.
    Kohn, Robert
    Univ New South Wales, Australia; ARC Ctr Excellence Math and Stat Frontiers ACEMS, Australia.
    Minh-Ngoc, Tran
    ARC Ctr Excellence Math and Stat Frontiers ACEMS, Australia; Univ Sydney, Australia.
    Villani, Mattias
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Arts and Sciences. ARC Ctr Excellence Math and Stat Frontiers ACEMS, Australia; Stockholm Univ, Sweden.
    Hamiltonian Monte Carlo with Energy Conserving Subsampling2019In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 20, article id 1Article in journal (Refereed)
    Abstract [en]

    Hamiltonian Monte Carlo (HMC) samples efficiently from high-dimensional posterior distributions with proposed parameter draws obtained by iterating on a discretized version of the Hamiltonian dynamics. The iterations make HMC computationally costly, especially in problems with large data sets, since it is necessary to compute posterior densities and their derivatives with respect to the parameters. Naively computing the Hamiltonian dynamics on a subset of the data causes HMC to lose its key ability to generate distant parameter proposals with high acceptance probability. The key insight in our article is that efficient subsampling HMC for the parameters is possible if both the dynamics and the acceptance probability are computed from the same data subsample in each complete HMC iteration. We show that this is possible to do in a principled way in a HMC-within-Gibbs framework where the subsample is updated using a pseudo marginal MH step and the parameters are then updated using an HMC step, based on the current subsample. We show that our subsampling methods are fast and compare favorably to two popular sampling algorithms that use gradient estimates from data subsampling. We also explore the current limitations of subsampling HMC algorithms by varying the quality of the variance reducing control variates used in the estimators of the posterior density and its gradients.

  • 3. Ekdahl, Magnus
    et al.
    Koski, Timo
    Bounds for the loss in probability of correct classification under model based approximation2006In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 7, p. 2449-2480Article in journal (Refereed)
    Abstract [en]

    In many pattern recognition/classification problem the true class conditional model and class probabilities are approximated for reasons of reducing complexity and/or of statistical estimation. The approximated classifier is expected to have worse performance, here measured by the probability of correct classification. We present an analysis valid in general, and easily computable formulas for estimating the degradation in probability of correct classification when compared to the optimal classifier. An example of an approximation is the Naive Bayes classifier. We show that the performance of the Naive Bayes depends on the degree of functional dependence between the features and labels. We provide a sufficient condition for zero loss of performance, too.

  • 4.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Jordan, Michael I.
    University of Calif Berkeley, CA 94720 USA; University of Calif Berkeley, CA 94720 USA.
    Schon, Thomas B.
    Uppsala University, Sweden.
    Particle Gibbs with Ancestor Sampling2014In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 15, p. 2145-2184Article in journal (Refereed)
    Abstract [en]

    Particle Markov chain Monte Carlo (PMCMC) is a systematic way of combining the two main tools used for Monte Carlo statistical inference: sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC). We present a new PMCMC algorithm that we refer to as particle Gibbs with ancestor sampling (PGAS). PGAS provides the data analyst with an off-the-shelf class of Markov kernels that can be used to simulate, for instance, the typically high-dimensional and highly autocorrelated state trajectory in a state-space model. The ancestor sampling procedure enables fast mixing of the PGAS kernel even when using seemingly few particles in the underlying SMC sampler. This is important as it can significantly reduce the computational burden that is typically associated with using SMC. PGAS is conceptually similar to the existing PG with backward simulation (PGBS) procedure. Instead of using separate forward and backward sweeps as in PGBS, however, we achieve the same effect in a single forward sweep. This makes PGAS well suited for addressing inference problems not only in state-space models, but also in models with more complex dependencies, such as non-Markovian, Bayesian nonparametric, and general probabilistic graphical models.

  • 5. Lindsten, Fredrik
    et al.
    Jordan, Michael I.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control.
    Particle Gibbs with ancestor sampling2014In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 15, p. 2145-2184Article in journal (Refereed)
    Abstract [en]

    Particle Markov chain Monte Carlo (PMCMC) is a systematic way of combining the two main tools used for Monte Carlo statistical inference: sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC). We present a new PMCMC algorithm that we refer to as particle Gibbs with ancestor sampling (PGAS). PGAS provides the data analyst with an off-the-shelf class of Markov kernels that can be used to simulate, for instance, the typically high-dimensional and highly autocorrelated state trajectory in a state-space model. The ancestor sampling procedure enables fast mixing of the PGAS kernel even when using seemingly few particles in the underlying SMC sampler. This is important as it can significantly reduce the computational burden that is typically associated with using SMC. PGAS is conceptually similar to the existing PG with backward simulation (PGBS) procedure. Instead of using separate forward and backward sweeps as in PGBS, however, we achieve the same effect in a single forward sweep. This makes PGAS well suited for addressing inference problems not only in state-space models, but also in models with more complex dependencies, such as non-Markovian, Bayesian nonparametric, and general probabilistic graphical models.

  • 6. Lisitsyn, Sergey
    et al.
    Widmer, Christian
    Garcia, Fernando J. Iglesias
    KTH, School of Computer Science and Communication (CSC).
    Tapkee: An Efficient Dimension Reduction Library2013In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 14, p. 2355-2359Article in journal (Refereed)
    Abstract [en]

    We present Tapkee, a C++ template library that provides efficient implementations of more than 20 widely used dimensionality reduction techniques ranging from Locally Linear Embedding (Roweis and Saul, 2000) and Isomap (de Silva and Tenenbaum, 2002) to the recently introduced Barnes-Hut-SNE (van der Maaten, 2013). Our library was designed with a focus on performance and flexibility. For performance, we combine efficient multi-core algorithms, modern data structures and state-of-the-art low-level libraries. To achieve flexibility, we designed a clean interface for applying methods to user data and provide a callback API that facilitates integration with the library. The library is freely available as open-source software and is distributed under the permissive BSD 3-clause license. We encourage the integration of Tapkee into other open-source toolboxes and libraries. For example, Tapkee has been integrated into the codebase of the Shogun toolbox (Sonnenburg et al., 2010), giving us access to a rich set of kernels, distance measures and bindings to common programming languages including Python, Octave, Matlab, R, Java, C#, Ruby, Perl and Lua. Source code, examples and documentation are available at http://tapkee.lisitsyn.me.

  • 7.
    Nilsson, Roland
    et al.
    Linköping University, The Institute of Technology. Linköping University, Department of Physics, Chemistry and Biology, Computational Biology.
    Peña, Jose M.
    Linköping University, Department of Computer and Information Science, Database and information techniques.
    Björkegren, Johan
    Computional Medicine group KI.
    Tegnér, Jesper
    Linköping University, The Institute of Technology. Linköping University, Department of Physics, Chemistry and Biology, Computational Biology.
    Consistent feature selection for pattern recognition in polynomial time2007In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 8, p. 589-612Article in journal (Refereed)
    Abstract [en]

    We analyze two different feature selection problems: finding a minimal feature set optimal for classification (MINIMAL-OPTIMAL) vs. finding all features relevant to the target variable (ALL-RELEVANT). The latter problem is motivated by recent applications within bioinformatics, particularly gene expression analysis. For both problems, we identify classes of data distributions for which there exist consistent, polynomial-time algorithms. We also prove that ALL-RELEVANT is much harder than MINIMAL-OPTIMAL and propose two consistent, polynomial-time algorithms. We argue that the distribution classes considered are reasonable in many practical cases, so that our results simplify feature selection in a wide range of machine learning tasks.

  • 8.
    Parviainen, Pekka
    et al.
    KTH, Centres, Science for Life Laboratory, SciLifeLab. KTH, Centres, SeRC - Swedish e-Science Research Centre.
    Farahani, H. S.
    Lagergren, Jens
    KTH, Centres, Science for Life Laboratory, SciLifeLab. KTH, Centres, SeRC - Swedish e-Science Research Centre.
    Learning bounded tree-width Bayesian networks using integer linear programming2014In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 33, p. 751-759Article in journal (Refereed)
    Abstract [en]

    In many applications one wants to compute conditional probabilities given a Bayesian network. This inference problem is NP-hard in general but becomes tractable when the network has low tree-width. Since the inference problem is common in many application areas, we provide a practical algorithm for learning bounded tree-width Bayesian networks. We cast this problem as an integer linear program (ILP). The program can be solved by an anytime algorithm which provides upper bounds to assess the quality of the found solutions. A key component of our program is a novel integer linear formulation for bounding tree-width of a graph. Our tests clearly indicate that our approach works in practice, as our implementation was able to find an optimal or nearly optimal network for most of the data sets.

  • 9.
    Parviainen, Pekka
    et al.
    KTH, School of Computer Science and Communication (CSC). KTH, Centres, Science for Life Laboratory, SciLifeLab.
    Koivisto, Mikko
    Finding Optimal Bayesian Networks Using Precedence Constraints2013In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 14, p. 1387-1415Article in journal (Refereed)
    Abstract [en]

    We consider the problem of finding a directed acyclic graph (DAG) that optimizes a decomposable Bayesian network score. While in a favorable case an optimal DAG can be found in polynomial time, in the worst case the fastest known algorithms rely on dynamic programming across the node subsets, taking time and space 2(n), to within a factor polynomial in the number of nodes n. In practice, these algorithms are feasible to networks of at most around 30 nodes, mainly due to the large space requirement. Here, we generalize the dynamic programming approach to enhance its feasibility in three dimensions: first, the user may trade space against time; second, the proposed algorithms easily and efficiently parallelize onto thousands of processors; third, the algorithms can exploit any prior knowledge about the precedence relation on the nodes. Underlying all these results is the key observation that, given a partial order P on the nodes, an optimal DAG compatible with P can be found in time and space roughly proportional to the number of ideals of P, which can be significantly less than 2(n). Considering sufficiently many carefully chosen partial orders guarantees that a globally optimal DAG will be found. Aside from the generic scheme, we present and analyze concrete tradeoff schemes based on parallel bucket orders.

  • 10.
    Rai, Akshara
    et al.
    Carnegie Mellon Univ, Sch Comp Sci, Robot Inst, Pittsburgh, PA 15213 USA..
    Antonova, Rika
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Meier, Franziska
    Univ Washington, Paul G Allen Sch Comp Sci Engn, Seattle, WA 98195 USA..
    Atkeson, Christopher G.
    Carnegie Mellon Univ, Sch Comp Sci, Robot Inst, Pittsburgh, PA 15213 USA..
    Using Simulation to Improve Sample-Efficiency of Bayesian Optimization for Bipedal Robots2019In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 20, article id 49Article in journal (Refereed)
    Abstract [en]

    Learning for control can acquire controllers for novel robotic tasks, paving the path for autonomous agents. Such controllers can be expert-designed policies, which typically require tuning of parameters for each task scenario. In this context, Bayesian optimization (BO) has emerged as a promising approach for automatically tuning controllers. However, sample-efficiency can still be an issue for high-dimensional policies on hardware. Here, we develop an approach that utilizes simulation to learn structured feature transforms that map the original parameter space into a domain-informed space. During BO, similarity between controllers is now calculated in this transformed space. Experiments on the ATRIAS robot hardware and simulation show that our approach succeeds at sample-efficiently learning controllers for multiple robots. Another question arises: What if the simulation significantly differs from hardware? To answer this, we create increasingly approximate simulators and study the effect of increasing simulation-hardware mismatch on the performance of Bayesian optimization. We also compare our approach to other approaches from literature, and find it to be more reliable, especially in cases of high mismatch. Our experiments show that our approach succeeds across different controller types, bipedal robot models and simulator fidelity levels, making it applicable to a wide range of bipedal locomotion problems.

  • 11.
    Taghia, Jalil
    et al.
    KTH, School of Electrical Engineering (EES), Communication Theory.
    Leijon, Arne
    KTH, School of Electrical Engineering (EES), Communication Theory.
    Bayesian Recursive Blind Source SeparationIn: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928Article in journal (Other academic)
    Abstract [en]

    We consider the problem of blind source separation (BSS) of convolutive mixtures in underdeterminedscenarios, where there are more sources to estimate than recorded signals. This problemhas been intensively studied in the literature. Many successful methods relay on batch processingof previously recorded signals, and hence are only best suited for noncausal systems. This paperaddresses the problem of online BSS. To realize this, we develop a Bayesian recursive framework.The proposed Bayesian framework allows incorporating prior knowledge in a coherentway, and therecursive learning allows to combine information gained from the current observation with all informationfromthe previous observations. Experiments using live audio recordings show promisingresults.

  • 12. Van Belle, Vanya
    et al.
    Pelckmans, Kristiaan
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control.
    Suykens, Johan A. K.
    Van Huffel, Sabine
    Learning transformation models for ranking and survival analysis2011In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 12, p. 819-862Article in journal (Refereed)
  • 13.
    Vasiloudis, Theodore
    et al.
    RISE - Research Institutes of Sweden, ICT, SICS.
    Morales, G. D.
    ISI Foundation, Italy.
    Bostrom, H.
    KTH Royal Institute of Technology, Sweden.
    Quantifying Uncertainty in Online Regression Forests2019In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 20Article in journal (Refereed)
    Abstract [en]

    Accurately quantifying uncertainty in predictions is essential for the deployment of machine learning algorithms in critical applications where mistakes are costly. Most approaches to quantifying prediction uncertainty have focused on settings where the data is static, or bounded. In this paper, we investigate methods that quantify the prediction uncertainty in a streaming setting, where the data is potentially unbounded. We propose two meta-algorithms that produce prediction intervals for online regression forests of arbitrary tree models; one based on conformal prediction, and the other based on quantile regression. We show that the approaches are able to maintain specified error rates, with constant computational cost per example and bounded memory usage. We provide empirical evidence that the methods outperform the state-of-the-art in terms of maintaining error guarantees, while being an order of magnitude faster. We also investigate how the algorithms are able to recover from concept drift.

  • 14.
    Vasiloudis, Theodore
    et al.
    RISE SICS, Stockholm, Sweden..
    Morales, Gianmarco De Francisci
    ISI Fdn, Turin, Italy..
    Boström, Henrik
    KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Software and Computer systems, SCS.
    Quantifying Uncertainty in Online Regression Forests2019In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 20, p. 1-35, article id 155Article in journal (Refereed)
    Abstract [en]

    Accurately quantifying uncertainty in predictions is essential for the deployment of machine learning algorithms in critical applications where mistakes are costly. Most approaches to quantifying prediction uncertainty have focused on settings where the data is static, or bounded. In this paper, we investigate methods that quantify the prediction uncertainty in a streaming setting, where the data is potentially unbounded. We propose two meta-algorithms that produce prediction intervals for online regression forests of arbitrary tree models; one based on conformal prediction, and the other based on quantile regression. We show that the approaches are able to maintain specified error rates, with constant computational cost per example and bounded memory usage. We provide empirical evidence that the methods outperform the state-of-the-art in terms of maintaining error guarantees, while being an order of magnitude faster. We also investigate how the algorithms are able to recover from concept drift.

  • 15. Yun, S. -Y
    et al.
    Proutiere, Alexandre
    KTH, School of Electrical Engineering (EES), Automatic Control. INRIA, France.
    Community detection via random and adaptive sampling2014In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 35, p. 138-175Article in journal (Refereed)
    Abstract [en]

    In this paper, we consider networks consisting of a finite number of non-overlapping communities. To extract these communities, the interaction between pairs of nodes may be sampled from a large available data set, which allows a given node pair to be sampled several times. When a node pair is sampled, the observed outcome is a binary random variable, equal to 1 if nodes interact and to 0 otherwise. The outcome is more likely to be positive if nodes belong to the same communities. For a given budget of node pair samples or observations, we wish to jointly design a sampling strategy (the sequence of sampled node pairs) and a clustering algorithm that recover the hidden communities with the highest possible accuracy. We consider both non-adaptive and adaptive sampling strategies, and for both classes of strategies, we derive fundamental performance limits satisfied by any sampling and clustering algorithm. In particular, we provide necessary conditions for the existence of algorithms recovering the communities accurately as the network size grows large. We also devise simple algorithms that accurately reconstruct the communities when this is at all possible, hence proving that the proposed necessary conditions for accurate community detection are also sufficient. The classical problem of community detection in the stochastic block model can be seen as a particular instance of the problems consider here. But our framework covers more general scenarios where the sequence of sampled node pairs can be designed in an adaptive manner. The paper provides new results for the stochastic block model, and extends the analysis to the case of adaptive sampling.

1 - 15 of 15
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf