Change search
ReferencesLink to record
Permanent link

Direct link
Overproduce-and-Select: The Grim Reality
University of Borås, School of Business and IT. (CSL@BS)
University of Borås, School of Business and IT. (CSL@BS)
University of Borås, School of Business and IT. (CSL@BS)
2013 (English)Conference paper (Refereed)
Abstract [en]

Overproduce-and-select (OPAS) is a frequently used paradigm for building ensembles. In static OPAS, a large number of base classifiers are trained, before a subset of the available models is selected to be combined into the final ensemble. In general, the selected classifiers are supposed to be accurate and diverse for the OPAS strategy to result in highly accurate ensembles, but exactly how this is enforced in the selection process is not obvious. Most often, either individual models or ensembles are evaluated, using some performance metric, on available and labeled data. Naturally, the underlying assumption is that an observed advantage for the models (or the resulting ensemble) will carry over to test data. In the experimental study, a typical static OPAS scenario, using a pool of artificial neural networks and a number of very natural and frequently used performance measures, is evaluated on 22 publicly available data sets. The discouraging result is that although a fairly large proportion of the ensembles obtained higher test set accuracies, compared to using the entire pool as the ensemble, none of the selection criteria could be used to identify these highly accurate ensembles. Despite only investigating a specific scenario, we argue that the settings used are typical for static OPAS, thus making the results general enough to question the entire paradigm.

Place, publisher, year, edition, pages
IEEE , 2013.
Keyword [en]
Ensembles, Neural networks, Overproduce-and-select, Data mining, Machine Learning
National Category
Computer Science Computer and Information Science
URN: urn:nbn:se:hb:diva-7054ISI: 000335317800008Local ID: 2320/12920ISBN: 10.1109/CIEL.2013.6613140OAI: diva2:887761
IEEE Symposium on Computational Intelligence and Ensemble Learning (CIEL), 16-19 April 2013 , Singapore


Swedish Foundation for

Strategic Research through the project High-Performance Data

Mining for Drug Effect Detection (ref. no. IIS11-0053)

Available from: 2015-12-22 Created: 2015-12-22

Open Access in DiVA

fulltext(199 kB)12 downloads
File information
File name FULLTEXT01.pdfFile size 199 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Johansson, UlfLöfström, TuveBoström, Henrik
By organisation
School of Business and IT
Computer ScienceComputer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 12 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 7 hits
ReferencesLink to record
Permanent link

Direct link