Wavelet decomposition reconstructs a signal by a series of scaled and translated wavelets. Incorporating discrete wavelet decomposition theory with neural network techniques, wavelet networks have recently emerged as a powerful tool for many applications in the field of signal processing, such as data compression and function approximation. In this paper, four contributions are claimed: (1) From the point of view of machine learning, we analyse and construct wavelet network to achieve the compact representation of a signal. (2) A new algorithm of constructing wavelet network is proposed. The orthogonal least square (OLS) is employed to prune the wavelet network. (3) Our experiments on speech signal processing results show that the wavelet network pruned by OLS achieves the best approximation and prediction capabilities among the representative speech processing techniques. (4) Our proposed methodology has been successfully applied to speech synthesis for a talking head to read web texts.