This chapter will be teaching us about systematically improving your learning algorithm. The videos for this week will teach you how to tell when a learning algorithm is doing poorly, and describe the ‘best practices’ for how to ‘debug’ your learning algorithm and go about improving its performance.

grad = 1/m*\sum_{i=1}^{m}*((h_\theta(x^i)-y^i)*x^i)+\frac{\lambda}{m}\theta_j;

for j=0,\lambda=0;

2.3 cost coding

1 2 3 4 5 6 7 8 9 10 11

function [J, grad] = linearRegCostFunction(X, y, theta, lambda) % your code here H = X* theta; len_theta = length(theta); J = 1/(2*m)*sum((H-y).^2)+lambda/(2*m)*sum(theta(2:len_theta).^2);

grad = 1/m*(X'*(H-y))+ lambda/m * [0;theta(2:len_theta)]; %-end--%

grad = grad(:); end

2.4 Fitting linear regression

其中 lambda = 0

3. Bias-variance

3.1 Learning curves for Polynomial Regression

The data is non-linear, so this will not give a great fit. lambda = 0

function [error_train, error_val] = ... learningCurve(X, y, Xval, yval, lambda) % LEARNINGCURVE Generates the train and cross validation set errors needed to plot a learning curve

% In this function, you will compute the train and test errors for dataset sizes from 1 up to m. In practice, when working with larger datasets, you might want to do this in larger intervals. % Number of training examples m = size(X, 1); m_Xval = size(Xval,1); % You need to return these values correctly error_train = zeros(m, 1); error_val = zeros(m, 1);

% your code here

for i = 1:m X_train = X(1:i,:); y_train = y(1:i); [theta_curve] = trainLinearReg([ones(i, 1) X_train],y_train,lambda); [J, grad] = linearRegCostFunction([ones(i, 1) X_train],y_train,theta_curve,0); error_train(i) = J; [J, grad] = linearRegCostFunction([ones(m_Xval, 1) Xval],yval,theta_curve,0); error_val(i) = J; end % code end

3.3 结果

4 Polynomial regression

4.1 Feature Mapping for Polynomial Regression

You should now complete `polyFeatures` to map each example into its powers

function [X_poly] = polyFeatures(X, p) %POLYFEATURES Maps X (1D vector) into the p-th power % [X_poly] = POLYFEATURES(X, p) takes a data matrix X (size m x 1) and % maps each example into its polynomial features where % X_poly(i, :) = [X(i) X(i).^2 X(i).^3 ... X(i).^p]; %

% You need to return the following variables correctly. X_poly = zeros(numel(X), p);

% ====================== YOUR CODE HERE ====================== % Instructions: Given a vector X, return a matrix X_poly where the p-th % column of X contains the values of X to the p-th power. %

for i = 1:p X_poly(:,i) = X.^i; end % ==================================================

end

4.3 Learning Curve for Polynomial Regression

result:

4.4 Validation for Selecting Lambda

You will now implement `validationCurve` to test various values of lambda on a validation set. You will then use this to select the "best" lambda value.

function [lambda_vec, error_train, error_val] = ... validationCurve(X, y, Xval, yval) %VALIDATIONCURVE Generate the train and validation errors needed to %plot a validation curve that we can use to select lambda

% Selected values of lambda (you should not change this) lambda_vec = [0 0.001 0.003 0.01 0.03 0.1 0.3 1 3 10]';

% ====================== YOUR CODE HERE ======================

m = size(X, 1); m_val = size(Xval,1);

for i = 1:length(lambda_vec) lambda = lambda_vec(i); [theta_curve] = trainLinearReg([ones(m, 1) X],y,lambda); [error_train(i), grad] = linearRegCostFunction([ones(m, 1) X],y,theta_curve,0); [error_val(i), grad] = linearRegCostFunction([ones(m_val, 1) Xval],yval,theta_curve,0); end