Abstract
A spectral analysis of a Boolean function is proposed for ap- proximating the decision boundary of an ensemble of classifiers, and an in- tuitive explanation of computing Walsh coefficients for the functional ap- proximation is provided. It is shown that the difference between first and third order coefficient approximation is a good indicator of optimal base classifier complexity. When combining Neural Networks, experimental re- sults on a variety of artificial and real two-class problems demonstrate un- der what circumstances ensemble performance can be improved. For tuned base classifiers, first order coefficients provide performance similar to ma- jority vote. However, for weak/fast base classifiers, higher order coefficient approximation may give better performance. It is also shown that higher order coefficient approximation is superior to the Adaboost logarithmic weighting rule when boosting weak Decision Tree base classifiers.