Machine Learning 第四波编程作业 - Neural Networks: Learning

仅列出核心代码：

1.sigmoidGradient.m

h = 1.0 ./ (1.0 + exp(-z));
g = h.*(1 - h);

2.randInitializeWeights.m

epsilon_init = 0.12;
W = rand(L_out, 1 + L_in)*2*epsilon_init - epsilon_init;

3.nnCostFunction.m

% cost function

A1 = X;
A1 = [ones(m, 1), A1];
Z2 = A1 * Theta1.';
A2 = sigmoid(Z2);
A2 = [ones(m, 1), A2];
Z3 = A2 * Theta2.';
A3 = sigmoid(Z3);
H = A3;
Y = zeros(m, num_labels);
for ind = 1:m
Y(ind, y(ind)) = 1;
end
K = num_labels;
Jk = zeros(K, 1);

for k =1:K

Jk(k) = ( -Y(:, k).' *log(H(:, k)) )-( (1 - Y(:, k)).' * log(1-H(:, k)) );

end
J = sum(Jk)/m;

J = J + ( lambda/(2*m) )*( sum(sum(Theta1(:, 2:end).^2))+sum(sum(Theta2(:, 2:end).^2)) );

% Unroll gradients

delta3 = A3 - Y;
delta2 = delta3*Theta2.* (A2.*(1-A2));
delta2 = delta2(:, 2: end);
Delta2 = zeros(size(delta3, 2), size(A2, 2));
Delta1 = zeros(size(delta2, 2), size(A1, 2));
for i=1:m
Delta2 = Delta2 + delta3(i, :).' * A2(i, :);
Delta1 = Delta1 + delta2(i, :).' * A1(i, :);
end
Theta1_grad = Delta1/m;
Theta1_grad(:, 2:end) = Theta1_grad(:, 2:end) + Theta1(:, 2:end)*(lambda/m);
Theta2_grad = Delta2/m;
Theta2_grad(:, 2:end) = Theta2_grad(:, 2:end) + Theta2(:, 2:end)*(lambda/m);
grad = [Theta1_grad(:) ; Theta2_grad(:)];
end

课程地址：https://www.coursera.org/course/ml

五月记——重新上路

这个月即将过去。

刚刚开完组会，趁着我糟糕的记忆还没有忘记，赶紧写下一些东西。而且，我意识到自己已经好久没有做月记了。

42 = 4(0905:0909)+8(0909:1005)+12(1005:1105)+8(1105:1201)+10(1201:1211)；

上面这个等式是我半年前深刻意识到的事实，42不是终极答案，它是我必须偿还的债务。

这让我一度无法承受，几近崩溃。

好在现在情况似乎有了转机，我终于确定了未来的研究方向，并且这次确确实实不会再改变了，一方面是因为开题，一方面是因为——我真的耗不起了。

关于现在的研究课题，一部分基于应用，一部分偏重数学理论，看上去蛮适合博士阶段的研究工作，而且应用部分的初步建模已经完成，至于理论部分，现在正在啃，我相信自己能啃下，但问题在于时间，我是否可以及时地做出量化的成果？

想起最近看的一本书：《有了博士学位还不够》。

书中很不客气地批判了那种对于科研的理想主义气质，作者以一个过来人的的口吻，指点那些有志于学术生涯的后生，要想在学术界立足，必须要做一些「常规工作」，比如定期汇总工作形成论文，与同事积极沟通互相帮助，做一次漂亮的报告，花心思修饰一份申请……看上去就像一份求职宝典中包含的内容，而作者也毫不避讳对这种实用主义科研态度的坚定立场。他说得很不客气：如果不这样，那么你在学术圈将举步维艰。

当然这是站在美国的科研环境得出的结论，我想国内的情况又有所不同，比如作者说如果想尽快拿到终身教职，与其在大学中从助理教授一点点做起，不如先进入工业界或者政府实验室，因为后者的科研氛围更单纯，不会被诸如授课或者其他杂事占去时间，而且实验室的薪资水平落差不大，前期更是比高校优厚不少，等到有了学术声誉之时再进入高校拿终身教职则更为稳妥。这个理由在国内是完全不成立的，首先国内的高校的相对更封闭，外来的和尚不一定好念经，此外国内的高校其实对所有的教师都可视为「终身制」，至少没有美国高校普遍的七年不升正教授就走人的「潜规则」。

但是，作者的观点还是很有启发性，让人意识到所谓学术，并非什么阳春白雪的事情，科研人员也都是普通人类，在其他领域存在的问题在学术界一样存在：待遇、前景、风险、心态。

考虑到学术界是一个人才辈出的地方，你的周围都是些绝顶聪明的头脑，如果想在这样的环境下生存，当然需要提前做足准备——一个博士学位当然是远远不够的。

今天的讨论也给了我类似的感觉，那就是对于普通的科研人员来说，仅仅有工作量还不够，一定要把工作展示出来，让大家都看得到，这才靠谱，至少别人会觉得你靠谱，这是获得学术声誉的必要途径。

说道学术声誉，老板也说到国外学界对此的重视程度异乎寻常得高，哪怕自己吃点亏，也不能让声誉受损，带着这样的心态，才会有那些高标准的学术成果做出来吧。

搞科研绝对不是一件轻松的事，但是这份工作的意义也值得你去为之皓首穷经。

这个月还有一件事必须一提，那就是在Coursera上跟了一门课，斯坦福Andrew Ng的机器学习，几周学下来，收获很大，尤其是在建立数学模型上面，获得了一些颇有教益的新知，下个月哪怕再忙，也要把这门课坚持听下去。

此外，老板今天说到他的大学同学有三分之一都在国外，我想到在我认识的同学、朋友中，很大一部分也在国外，而且这一部分人通常是成绩优秀、能力较强的那一类。我再一次感觉到，无论是亲眼所见，还是客观数据，都表明搞科研（甚至普通生活）的最佳环境不是在国内，所以，不论以何种方式，走出去都是我值得为之努力的一件事。

多说无益，从下个月开始，要在已经建立的模型上，多做仿真，积累数据，尽快形成工作量，尽快出paper。

不管怎样，必须要给自己一个坚持下去的信念。

It's never too late.

Machine Learning 第三波编程作业 – Multi-class Classification and Neural Networks

仅列出核心代码：

1.lrCostFunction.m

h = sigmoid(X * theta);   %   h_theta(X) : m*1
%   Cost func
J = (-log(h.')*y - log(ones(1, m) - h.')*(ones(m, 1) - y)) / m ...
    +(lambda/(2*m)) * sum(theta(2:end).^2);

%   Gradient
grad(1) = (X(:, 1).' * (h - y)) /m;

grad(2:end) = (X(:, 2:end).' * (h - y)) /m ...
    + (lambda/m) * theta(2:end);

2.oneVsAll.m

options = optimset('GradObj', 'on', 'MaxIter', 50);
initial_theta = zeros(size(X, 2), 1);
for c = 1:num_labels
    [all_theta(c, :)] = fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)),...
        initial_theta, options);   
end

3.predictOneVsAll.m

[~, p] = max(X * all_theta.', [], 2);

4.predict.m

X = [ones(size(X), 1), X]; % Add ones to the X data matrix

X1 = sigmoid(X * Theta1.');
X1 = [ones(size(X1), 1), X1]; % Add ones to the X1 data matrix

[~, p] = max(X1 * Theta2.', [], 2);

课程地址：https://www.coursera.org/course/ml

Machine Learning 第二波编程作业 – Logistic Regression

仅列出核心代码：

1.plotData.m

ind1 = find(y==1); ind0 = find(y==0);
plot(X(ind1, 1), X(ind1, 2), 'k+','LineWidth', 2, 'MarkerSize', 7);
plot(X(ind0, 1), X(ind0, 2), 'ko', 'MarkerFaceColor', 'y', 'MarkerSize', 7);

2.sigmoid.m

g = 1 ./ (ones(size(z)) + exp(-z));

3.costFunction.m

h = sigmoid(X * theta); % h_theta(X) : m*1
J = (-log(h.')*y - log(ones(1, m) - h.')*(ones(m, 1) - y)) / m;
grad = (X.' * (h - y)) /m;

4.predict.m

h = sigmoid(X * theta);
p = (h >= 0.5);

5.costFunctionReg.m

h = sigmoid(X * theta); % h_theta(X) : m*1
% Cost func
J = (-log(h.')*y - log(ones(1, m) - h.')*(ones(m, 1) - y)) / m ...
+(lambda/(2*m)) * sum(theta(2:end).^2);

% Gradient
grad(1) = (X(:, 1).' * (h - y)) /m;

grad(2:end) = (X(:, 2:end).' * (h - y)) /m ...
+ (lambda/m) * theta(2:end);

课程地址：https://www.coursera.org/course/ml

Machine Learning 第一波编程作业 - Linear Regression

仅列出核心代码：

1.computeCost

J = sum((X*theta-y).^2)/(2*m);

2.gradientDescent

theta = theta - (1/m)*alpha*(X.'*(X*theta-y));

3.featureNormalize

mu = mean(X);
sigma = std(X);
X_norm = (X - ones(size(X, 1), 1) * mu) ./ (ones(size(X, 1), 1) * sigma);

4.computeCostMulti

J = (X * theta - y).' * (X * theta - y) / (2*m);

5.gradientDescentMulti

theta = theta - (1/m)*alpha*(X.'*(X*theta-y));

6.normalEqn

theta = inv(X.' * X) * X.' * y;

课程地址：https://www.coursera.org/course/ml

《A Beginner's Guide to Irrational Behavior》课程论文

China is a labor-populous country, known as the "factory of the world". Some enterprises e.g. the Apple Intel etc., all set OEM factories located in China. I live in Chengdu, Sichuan in China. There is a huge factory with 110,000 workers, Foxconn. Two years ago, employee suicided in Foxconn Shenzhen factory frequently. During a short period(four months), twelve workers jumped from the top of building. As in such a tragic way to end their lives, it shocked the society. Soon, the news attracted attention of the media of China and abroad.

What's the reason of Foxconn's tragedy? Many experts (most of them were psychologists) had made a deep analysis. Some said that Foxconn put too much pressure onto its workers and the cost of living in such a big city in China is high. So that made a very obviously contrast between the two incentives. As a result of such effect, the workers' emotion inevitably lead to go extremely. On the other hand, there were some reports to disclose Foxconn industrial park's management, which was very mechanical, totally inhumanness. Among workers, there were also competitive relationships. It is difficult to establish friendship, unable to find a sense of belonging. Therefore, over time, those workers will be more and more unhappy....

In my opinion, one thing is certain: the workers were hard to feel a sense of accomplishment or happiness. Dan Ariely etc. (D Ariely, 2008) and Michael Norton etc. (M Norton, 2012), made a statement of a unanimous conclusion in their papers, which is: "a sense of identity" is important to everyone. This sense of identity comes from both the completion of the work and the affirmation of the results of the work from others. As "IKEA Effect" indicated, a doubly cherish would be put on the fruit of the task which need someone tried hard to achieve. And such endeavor would better be "systematically", for example, a furniture assembled by someone's own power. But if it is over-specialized, such as the division of labor(A Smith, 1776), there wouldn't be the similar effect. Foxconn's employees just like the products of socialization which makes the extreme division of labor. They are like a big car bed's bearing, working repeatedly  for times. They do not know what is the specific role of such assembly work, day after day, year after year. Where is the "meaning"? Whether it is to assemble the the Legos or assemble iPhone, people must be able to genuineness of seeing the fruits of their own labor. And that will take the initiative to stimulate a strong desire to finish their jobs. Otherwise, do not say to love the work, maybe they will feel tired of living for a long term.

Of course, as complicated "toys" as the iPhone, it is impossible to expect one person to assemble it correctly, and also the efficiency will be too low. But we can bring a sense of accomplishment to workers through other ways. For instance, in each specific part of the assembly, set a visualization object identifies which part of the mobile phone the workers' job belong to. Or in every step-assembly, set a graphical phone showing more completely, for every workers, a virtual mobile phone was assembled after several hours. And then, they would touched the goal of completion and feel some kind meaning. The principle of such mechanism is similar with the electric toothbrush's handle with smiley -- although it seems a little bit silly, sometimes it's very effective.

References:

Ariely, D., Kamenica, E. & Prelec, D. (2008). Man’s Search for Meaning: The Case of Legos. Journal of Economic Behavior and Organization, 67, 671-677.

Norton, M. I., Mochon, D., & Ariely, D. The IKEA Effect: When Labor Leads to Love. Harvard Business School Marketing Unit Working Paper, (11-091).

Smith A. The Wealth of Nations (1776)[J]. New York: Modern Library, 1937, 740.

---

最后得了7.5分。
一个比较长的评语的是：

I commend you on your use of English, which is generally clear in terms of meaning. You describe a problem of worker despair, leading to suicide. In paragraph 2, you refer to "a very obviously contrast between the two incentives." A clear statement of the two incentives and related behavior would be helpful. In addition, your assertion that inhumane treatment and lack of meaning in work were causative factors in the suicides. Your argument would be much stronger if you gave examples to work conditions to support these assertions.

想想我自己给别人打得分也都很高，瞬间淡定了。