model = ccl_learnv_ncl(X, Y, model) Learn the nullspace component of the input data X
0001 function model = ccl_learnv_ncl(X, Y, model) 0002 % model = ccl_learnv_ncl(X, Y, model) 0003 % 0004 % Learn the nullspace component of the input data X 0005 0006 % Input: 0007 % 0008 % X Observed states 0009 % Y Observed actions of the form Y = A(X)'B(X) + N(X) F(X) where N and F 0010 % are consistent across the dataset X 0011 % 0012 % Output: 0013 % 0014 % model Model that prodicts the nullspace component N(X)F(X) 0015 0016 0017 0018 0019 % CCL: A MATLAB library for Constraint Consistent Learning 0020 % Copyright (C) 2007 Matthew Howard 0021 % Contact: matthew.j.howard@kcl.ac.uk 0022 % 0023 % This library is free software; you can redistribute it and/or 0024 % modify it under the terms of the GNU Lesser General Public 0025 % License as published by the Free Software Foundation; either 0026 % version 2.1 of the License, or (at your option) any later version. 0027 % 0028 % This library is distributed in the hope that it will be useful, 0029 % but WITHOUT ANY WARRANTY; without even the implied warranty of 0030 % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 0031 % Lesser General Public License for more details. 0032 % 0033 % You should have received a copy of the GNU Library General Public 0034 % License along with this library; if not, write to the Free 0035 % Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. 0036 0037 % calculate the basis functions of the training data 0038 BX = model.phi(X) ; 0039 0040 % learn an initial model using a simple parametric model 0041 model = learn_model_dir ( model, BX, Y ) ; 0042 0043 % learn the model by minimising the proposed step-1 method 0044 obj = @(W) ccl_obj_ncl ( model, W, BX, Y); % setup the learning objective function 0045 options = optimset( 'Jacobian','on', 'Display', 'notify',... % options for the optimisation 0046 'MaxFunEvals',1e9, 'MaxIter', 1000,... 0047 'TolFun',1e-9, 'TolX',1e-9,... 0048 'Algorithm', 'levenberg-marquardt'); 0049 model.w = lsqnonlin(obj, model.w, [], [], options ); % use the non-linear optimiser to solve obj_ncl_reg 0050 end 0051 0052 function model = learn_model_dir ( model, BX, U ) 0053 % model = learn_model_dir ( model, BX, U ) 0054 % 0055 % Direct policy learning approach 0056 % 0057 % Input: 0058 % model Model related parameters 0059 % BX High demensionality of the input data 0060 % 0061 % Output: 0062 % model Learnt model 0063 0064 HS = eye(model.num_basis); 0065 g = BX * U' ; 0066 H = BX * BX'; 0067 0068 % do eigen-decomposition for inversion 0069 [V,D] = eig( H + 1e-8*HS); 0070 ev = diag(D); 0071 ind = find( ev > 1e-8); 0072 V1 = V(:,ind); 0073 pinvH1 = V1 * diag(ev(ind).^-1)*V1'; 0074 model.w = (pinvH1 * g)' ; 0075 end