[精品论文]A New LocalPCA SOM Algorithm.doc
《[精品论文]A New LocalPCA SOM Algorithm.doc》由会员分享,可在线阅读,更多相关《[精品论文]A New LocalPCA SOM Algorithm.doc(20页珍藏版)》请在三一办公上搜索。
1、精品论文A New Local PCA SOM AlgorithmDong Huang Zhang Yi and Xiaorong PuComputational Intelligence Laboratory, School of Computer Science andEngineering, University of Electronic Science and Technology of China, Chengdu610054, P. R. China.E-mail: donnyhuang, zhangyi, puxiaor.AbstractThis paper proposes
2、a Local PCA-SOM algorithm. The new competition measure is computational efficient, and implicitly incorporates the Mahalanobis distance and the reconstruction error. The matrix inversion or PCA decomposition for each data input is not needed as compared to the previous models. Moreover, the local da
3、ta distribution is completely stored in the covariance matrix instead of the pre-defined numbers of the principal components. Thus, no priori information of the optimal principal subspace is required. Experiments on both the synthesis data and a pattern learning task are carried out to show the perf
4、ormance of the proposed method.Key words: Neural Networks; Unsupervised Learning; Local Principal ComponentAnalysis; Self-Organizing Mapping1 IntroductionPrincipal component analysis (PCA) has been wildly used in dimension re- duction of multi-variate data, data compression, pattern recognition and
5、sta- tistical analysis. A PCA algorithm is designed to approximate the original high-dimensional pattern space by a low-dimensional subspace spanned by the principal eigenvectors of the data covariance matrix. In this way, the data distribution can be represented and reconstructed by the principal e
6、igenvec- tors and their corresponding eigenvalues. However, PCA is globally linear and1 This work was supported by National Science Foundation of China under Grant60471055 and Specialized Research Fund for the Doctoral Program of Higher Edu- cation under Grant 20040614017.Preprint submitted to Elsev
7、ier Science 28 September 20071inefficient for data distribution with non-linear dependencies 1. Several ex- tensions have been suggested to overcome this problem. These extensions fall into two main categories: the global nonlinear approach and locally linear de- scriptions. The former includes the
8、Principal Curves 2 and Kernel PCA 3. We focus on the latter approaches in which the data distribution is represented by a collection of Local PCA units 78.When describing a data distribution with a mixture model, the question arises what kind of units should the mixture contain. In Gaussian mixture
9、models 7, the local iso-density surface of a uniform Gaussian unit is a sphere. And a local PCA unit corresponds to a multivariate Gaussian with an ellipsoid iso-density surface. Despite its greater complexity, the local PCA is favorable over a sphere unit for the following reasons. An ellipsoid can
10、 describe a local structure for which many spheres are needed. Furthermore, data distributions are usually constrained locally to subspaces with fewer dimensions than the space of the training data. Thus, there are directions in which the distribution has locally zero variance (or almost zero becaus
11、e of noise). An ellipsoid unit representation can extend its minor components into the additional noise dimensions. But the computational cost increases over-proportionally with the number of principal components needed.In the recent work NGAS-PCA 4, the local PCA model is based on soft competition
12、scheme of Neural Gas algorithm and the RRLSA 10 online PCA learning in each local unit. The novelty of the NGAS-PCA is that its dis- tance measure for neuron competition is the combination of a normalized Mahalanobis distance and the squared reconstruction error. Note that storing the principal basi
13、s vectors for every unit involves rather complicated updating procedures as in NGAS-PCA and ASSOM 9. Another recent model is called the PCA-SOM 5 which proposes an alternative way by storing the local in- formation in the covariance matrix. This is directly derived from on statistics theories and ha
14、s great advantages over the ASSOM model both in computa- tion burden and reliability of the result. However, PCA-SOM only uses the reconstruction error in the principal subspace in its neuron competition. For this reason, PCA-SOM still need the PCA decomposition or updating with predefined numbers o
15、f the principal components when each training data is presented.In this paper, we propose a new computational efficient Local PCA algorithm that combines the advantages of NGAS-PCA and PCA-SOM. Each unit is associated with its mean vector and covariance matrix. The new competition measure implicitly
16、 incorporate the reconstruction error and distance between the input data and the unit center. In the proposed algorithm, the extra up- dating step of the principal subspaces is eliminated from the data distribution learning process. In addition, the local distribution of the input data is com- plet
17、ely stored in the covariance matrix instead of the predefined numbers of8精品论文the principal components. One potential application of the proposed model is the nonlinear pattern learning and recalling 11. In most models of associative memory, memories are stored as attractive fixed points at discrete
18、locations in state space. Discrete attractors may not be appropriate for patterns with con- tinuous variability 13. For example, the images of a three-dimensional object from different viewpoints 12. After the training process, the data distribution is represented by a collection of local linear uni
19、ts. No priori information for the optimal principal subspace is needed for the pattern representation.The rest of this paper is organized as follows. In section 2, the proposed compe- tition measure and neuron updating method are formulated. Section 3 presents the simulation results and discussions.
20、 Section 4 briefly reviews the NGAS- PCA and PCA-SOM algorithms. And their relations to the proposed method are also discussed in details in this section. Finally, the paper is concluded in Section 5.2 The Local PCA modelIn this section, we present the detailed formulation of the Local PCA models. T
21、he following notations are used throughout the paper. The input data setis X = xi |xi Rd , i = 1, , M , where d is the data dimension. For each local unit, there are the center of the unit cj and the covariance matrixCj where j = 1, , N . The principal subspace of Cj is characterized by W (p) = w1,
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- 精品论文 精品论文A New Local PCA SOM Algorithm 精品 论文
链接地址:https://www.31ppt.com/p-5189796.html