Andrew-机器学习(第六周作业)

By Mac小兔

Summary

This chapter will be teaching us about systematically improving your learning algorithm. The videos for this week will teach you how to tell when a learning algorithm is doing poorly, and describe the ‘best practices’ for how to ‘debug’ your learning algorithm and go about improving its performance.

0. 题目

第六周作业的题目:点击查看

1. Loading and Visualizing Data

2. Regularized Linear Regression

1
2
3
4
5
6
7
8
theta = [1 ; 1];
J = linearRegCostFunction([ones(m, 1) X], y, theta, 1);
fprintf(['Cost at theta = [1 ; 1]: %f '...
'\n(this value should be about 303.993192)\n'], J);
fprintf('Program paused. Press enter to continue.\n');
pause;

任务: Compute cost and gradient for regularized linear,fulfill linearRegCostFunction.m

2.1 regression cost formula

1
J = \frac{1}{2m} *\sum_{i=1}^{m}((H_\theta(x^i)-y^{i})^2)+\lambda/(2m)*\sum_{j=1}^{n}(\theta(2:length_\theta)^2);

2.2 regression gradient formula

1
2
3
grad = 1/m*\sum_{i=1}^{m}*((h_\theta(x^i)-y^i)*x^i)+\frac{\lambda}{m}\theta_j;
for j=0,\lambda=0;

2.3 cost coding

1
2
3
4
5
6
7
8
9
10
11
function [J, grad] = linearRegCostFunction(X, y, theta, lambda)
% your code here
H = X* theta;
len_theta = length(theta);
J = 1/(2*m)*sum((H-y).^2)+lambda/(2*m)*sum(theta(2:len_theta).^2);
grad = 1/m*(X'*(H-y))+ lambda/m * [0;theta(2:len_theta)];
%-end--%
grad = grad(:);
end

2.4 Fitting linear regression

其中 lambda = 0

linear regression

3. Bias-variance

3.1 Learning curves for Polynomial Regression

The data is non-linear, so this will not give a great fit. lambda = 0

you should implement the learningCurve function.

3.2 Coding

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
function [error_train, error_val] = ...
learningCurve(X, y, Xval, yval, lambda)
% LEARNINGCURVE Generates the train and cross validation set errors needed to plot a learning curve
% In this function, you will compute the train and test errors for dataset sizes from 1 up to m. In practice, when working with larger datasets, you might want to do this in larger intervals.
% Number of training examples
m = size(X, 1);
m_Xval = size(Xval,1);
% You need to return these values correctly
error_train = zeros(m, 1);
error_val = zeros(m, 1);
% your code here
for i = 1:m
X_train = X(1:i,:);
y_train = y(1:i);
[theta_curve] = trainLinearReg([ones(i, 1) X_train],y_train,lambda);
[J, grad] = linearRegCostFunction([ones(i, 1) X_train],y_train,theta_curve,0);
error_train(i) = J;
[J, grad] = linearRegCostFunction([ones(m_Xval, 1) Xval],yval,theta_curve,0);
error_val(i) = J;
end
% code end

3.3 结果

结果1

结果2

4 Polynomial regression

4.1 Feature Mapping for Polynomial Regression

You should now complete `polyFeatures` to map each example into its powers

4.2 coding

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
function [X_poly] = polyFeatures(X, p)
%POLYFEATURES Maps X (1D vector) into the p-th power
% [X_poly] = POLYFEATURES(X, p) takes a data matrix X (size m x 1) and
% maps each example into its polynomial features where
% X_poly(i, :) = [X(i) X(i).^2 X(i).^3 ... X(i).^p];
%
% You need to return the following variables correctly.
X_poly = zeros(numel(X), p);
% ====================== YOUR CODE HERE ======================
% Instructions: Given a vector X, return a matrix X_poly where the p-th
% column of X contains the values of X to the p-th power.
%
for i = 1:p
X_poly(:,i) = X.^i;
end
% ==================================================
end

Normalized Training Example

4.3 Learning Curve for Polynomial Regression

result:

Polynomial Regression Fit lambda =0

Polynomial Regression Learning Curve (lambda = 0)

4.4 Validation for Selecting Lambda

You will now implement `validationCurve` to test various values of lambda on a validation set. You will then use this to select the "best" lambda value.

4.5 coding

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
function [lambda_vec, error_train, error_val] = ...
validationCurve(X, y, Xval, yval)
%VALIDATIONCURVE Generate the train and validation errors needed to
%plot a validation curve that we can use to select lambda
% Selected values of lambda (you should not change this)
lambda_vec = [0 0.001 0.003 0.01 0.03 0.1 0.3 1 3 10]';
% ====================== YOUR CODE HERE ======================
m = size(X, 1);
m_val = size(Xval,1);
for i = 1:length(lambda_vec)
lambda = lambda_vec(i);
[theta_curve] = trainLinearReg([ones(m, 1) X],y,lambda);
[error_train(i), grad] = linearRegCostFunction([ones(m, 1) X],y,theta_curve,0);
[error_val(i), grad] = linearRegCostFunction([ones(m_val, 1) Xval],yval,theta_curve,0);
end
% =========================================================================
end

4.6 result

different lambda error value

different lambda error value

Link.