Tuesday, October 4, 2016

SVM with Gaussian Kernel: How to find the best training parameters C and sigma that give lowest cross-validation error?

This code find the best C and sigma based on given cross-validation set (Xval, yval):

function [C, sigma] = FindBestSVMParameters(X, y, Xval, yval)

C_vec = [0.01 0.03 0.1 0.3 1 3 10 30]';
sigma_vec = [0.01 0.03 0.1 0.3 1 3 10 30]';
ErrorMatrix = zeros(8, 8);

for i = 1:length(C_vec)
  C = C_vec(i);

  for j = 1:length(sigma_vec)
    sigma = sigma_vec(j);
    model= svmTrain(X, y, C, @(x1, x2) gaussianKernel(x1, x2, sigma)); 
    predictions = svmPredict(model, Xval);
    ErrorMatrix(i,j) = mean(double(predictions ~= yval)); %prediction error rate    
  end
end

% surf(C_vec, sigma_vec, ErrorMatrix);

[i,j] = find(ismember(ErrorMatrix, min(ErrorMatrix(:))));
C = C_vec(i);
sigma = sigma_vec(j);

end
----------------------------------------------------------------------------------
function sim = gaussianKernel(x1, x2, sigma)
% returns a radial basis function (RBF) kernel between x1 and x2

% Ensure that x1 and x2 are column vectors
x1 = x1(:);
x2 = x2(:);
sim = exp( - ((x1 - x2)' * (x1 - x2)) / (2*sigma*sigma) );

end

No comments:

Post a Comment