Sparse model construction using coordinate descent optimization

Xia Hong, Yi Guo, Sheng Chen, Junbin Gao

Research output: Book chapter/Published conference paperConference paper

1 Citation (Scopus)

Abstract

We propose a new sparse model construction method aimed at maximizing a model's generalisation capability for a large class of linear-in-the-'parameters models. The coordinate descent optimization algorithm is employed with a modifiedl1- penalized least squares cost function in order to estimate a single parameter and its regularization parameter simultaneously based on the leave one out mean square error (LOOMSE). Our original contribution is to derive a closed form of optimal LOOMSE regularization parameter for a single term model, for which we show that the LOOMSE can be analytically computed without actually splitting the data set leading to a very simple parameter estimation method. We then integrate the new results within the coordinate descent optimization algorithm to update model parameters one at the time for linear-in-the-parameters models. Consequently a fully automated procedure is achieved without resort to any other validation data set for iterative model evaluation. Illustrative examples are included to demonstrate the effectiveness of the new approaches.
Original languageEnglish
Title of host publicationDSP 2013
Subtitle of host publication18th Proceedings
Place of PublicationUnited States
PublisherInstitute of Electrical and Electronics Engineers
Pages1-6
Number of pages6
ISBN (Electronic)9781467358057
DOIs
Publication statusPublished - 2013
EventInternational Conference on Digital Signal Processing (DSP) - Santorini, Greece, Greece
Duration: 01 Jul 201303 Jul 2013

Conference

ConferenceInternational Conference on Digital Signal Processing (DSP)
CountryGreece
Period01/07/1303/07/13

Fingerprint

Mean square error
Cost functions
Parameter estimation

Grant Number

  • DP130100364

Cite this

Hong, X., Guo, Y., Chen, S., & Gao, J. (2013). Sparse model construction using coordinate descent optimization. In DSP 2013: 18th Proceedings (pp. 1-6). United States: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICDSP.2013.6622775
Hong, Xia ; Guo, Yi ; Chen, Sheng ; Gao, Junbin. / Sparse model construction using coordinate descent optimization. DSP 2013: 18th Proceedings. United States : Institute of Electrical and Electronics Engineers, 2013. pp. 1-6
@inproceedings{cf3ed21c15494befbcecc75eee07667d,
title = "Sparse model construction using coordinate descent optimization",
abstract = "We propose a new sparse model construction method aimed at maximizing a model's generalisation capability for a large class of linear-in-the-'parameters models. The coordinate descent optimization algorithm is employed with a modifiedl1- penalized least squares cost function in order to estimate a single parameter and its regularization parameter simultaneously based on the leave one out mean square error (LOOMSE). Our original contribution is to derive a closed form of optimal LOOMSE regularization parameter for a single term model, for which we show that the LOOMSE can be analytically computed without actually splitting the data set leading to a very simple parameter estimation method. We then integrate the new results within the coordinate descent optimization algorithm to update model parameters one at the time for linear-in-the-parameters models. Consequently a fully automated procedure is achieved without resort to any other validation data set for iterative model evaluation. Illustrative examples are included to demonstrate the effectiveness of the new approaches.",
keywords = "Cross validation, Lasso, Leave one out errors, Linear-in-the-parameters model, Regular-ization",
author = "Xia Hong and Yi Guo and Sheng Chen and Junbin Gao",
note = "Imported on 03 May 2017 - DigiTool details were: publisher = United States: Institute of Electrical and Electronics Engineers, 2013. Grant ID (550a) = DP130100364. Event dates (773o) = 1-3 July 2013; Parent title (773t) = International Conference on Digital Signal Processing (DSP).",
year = "2013",
doi = "10.1109/ICDSP.2013.6622775",
language = "English",
pages = "1--6",
booktitle = "DSP 2013",
publisher = "Institute of Electrical and Electronics Engineers",

}

Hong, X, Guo, Y, Chen, S & Gao, J 2013, Sparse model construction using coordinate descent optimization. in DSP 2013: 18th Proceedings. Institute of Electrical and Electronics Engineers, United States, pp. 1-6, International Conference on Digital Signal Processing (DSP), Greece, 01/07/13. https://doi.org/10.1109/ICDSP.2013.6622775

Sparse model construction using coordinate descent optimization. / Hong, Xia; Guo, Yi; Chen, Sheng; Gao, Junbin.

DSP 2013: 18th Proceedings. United States : Institute of Electrical and Electronics Engineers, 2013. p. 1-6.

Research output: Book chapter/Published conference paperConference paper

TY - GEN

T1 - Sparse model construction using coordinate descent optimization

AU - Hong, Xia

AU - Guo, Yi

AU - Chen, Sheng

AU - Gao, Junbin

N1 - Imported on 03 May 2017 - DigiTool details were: publisher = United States: Institute of Electrical and Electronics Engineers, 2013. Grant ID (550a) = DP130100364. Event dates (773o) = 1-3 July 2013; Parent title (773t) = International Conference on Digital Signal Processing (DSP).

PY - 2013

Y1 - 2013

N2 - We propose a new sparse model construction method aimed at maximizing a model's generalisation capability for a large class of linear-in-the-'parameters models. The coordinate descent optimization algorithm is employed with a modifiedl1- penalized least squares cost function in order to estimate a single parameter and its regularization parameter simultaneously based on the leave one out mean square error (LOOMSE). Our original contribution is to derive a closed form of optimal LOOMSE regularization parameter for a single term model, for which we show that the LOOMSE can be analytically computed without actually splitting the data set leading to a very simple parameter estimation method. We then integrate the new results within the coordinate descent optimization algorithm to update model parameters one at the time for linear-in-the-parameters models. Consequently a fully automated procedure is achieved without resort to any other validation data set for iterative model evaluation. Illustrative examples are included to demonstrate the effectiveness of the new approaches.

AB - We propose a new sparse model construction method aimed at maximizing a model's generalisation capability for a large class of linear-in-the-'parameters models. The coordinate descent optimization algorithm is employed with a modifiedl1- penalized least squares cost function in order to estimate a single parameter and its regularization parameter simultaneously based on the leave one out mean square error (LOOMSE). Our original contribution is to derive a closed form of optimal LOOMSE regularization parameter for a single term model, for which we show that the LOOMSE can be analytically computed without actually splitting the data set leading to a very simple parameter estimation method. We then integrate the new results within the coordinate descent optimization algorithm to update model parameters one at the time for linear-in-the-parameters models. Consequently a fully automated procedure is achieved without resort to any other validation data set for iterative model evaluation. Illustrative examples are included to demonstrate the effectiveness of the new approaches.

KW - Cross validation

KW - Lasso

KW - Leave one out errors

KW - Linear-in-the-parameters model

KW - Regular-ization

U2 - 10.1109/ICDSP.2013.6622775

DO - 10.1109/ICDSP.2013.6622775

M3 - Conference paper

SP - 1

EP - 6

BT - DSP 2013

PB - Institute of Electrical and Electronics Engineers

CY - United States

ER -

Hong X, Guo Y, Chen S, Gao J. Sparse model construction using coordinate descent optimization. In DSP 2013: 18th Proceedings. United States: Institute of Electrical and Electronics Engineers. 2013. p. 1-6 https://doi.org/10.1109/ICDSP.2013.6622775