On the construction of the relevance vector machine based on Bayesian Ying-Yang harmony learning

Dansong Cheng, Minh Nhut Nguyen, Junbin Gao, Daming Shi

    Research output: Contribution to journalArticlepeer-review

    5 Citations (Scopus)
    5 Downloads (Pure)

    Abstract

    Tipping's relevance vector machine (RVM) applies kernel methods to construct basis function networks using a least number of relevant basis functions. Compared to the well-known support vector machine (SVM), the RVM provides a better sparsity, and an automatic estimation of hyperparameters. However, the performance of the original RVM purely depends on the smoothness of the presumed prior of the connection weights and parameters. Consequently, the sparsity is actually still controlled by the choice of kernel functions and/or kernel parameters. This may lead to severe underfitting or overfitting in some cases. In this research, we explicitly involve the number of basis functions into the objective of the optimization procedure, and construct the RVM by maximizing the harmony function between "hypothetical" probability distribution in the forward training pathway and "true" probability distribution in the backward testing pathway, using Xu's Bayesian Ying-Yang (BYY) harmony learning technique. The experimental results have shown that our proposed methodology can achieve both the least complexity of structure and goodness of fit to data.
    Original languageEnglish
    Pages (from-to)173-179
    Number of pages7
    JournalNeural Networks
    Volume48
    DOIs
    Publication statusPublished - Dec 2013

    Grant Number

    • DP130100364

    Fingerprint

    Dive into the research topics of 'On the construction of the relevance vector machine based on Bayesian Ying-Yang harmony learning'. Together they form a unique fingerprint.

    Cite this