programming assignment logistic regression week 3

• 🌐 All Sites
• _APDaga DumpBox
• _APDaga Tech
• _APDaga Invest
• _APDaga Videos
• 🗃️ Categories
• _Free Tutorials
• __Python (A to Z)
• __Internet of Things
• __Coursera (ML/DL)
• __HackerRank (SQL)
• __Interview Q&A
• _Artificial Intelligence
• __Machine Learning
• __Deep Learning
• _Internet of Things
• __Raspberry Pi
• __Coursera MCQs
• __Linkedin MCQs
• __Celonis MCQs
• _Handwriting Analysis
• __Graphology
• _Investment Ideas
• _Open Diary
• _Troubleshoots
• _Freescale/NXP
• 📣 Mega Menu
• _Logo Maker
• _Youtube Tumbnail Downloader
• 🕸️ Sitemap

Coursera: Machine Learning (Week 3) [Assignment Solution] - Andrew NG

Recommended Machine Learning Courses: Coursera: Machine Learning    Coursera: Deep Learning Specialization Coursera: Machine Learning with Python Coursera: Advanced Machine Learning Specialization Udemy: Machine Learning LinkedIn: Machine Learning Eduonix: Machine Learning edX: Machine Learning Fast.ai: Introduction to Machine Learning for Coders

• ex2.m - Octave/MATLAB script that steps you through the exercise
• ex2 reg.m - Octave/MATLAB script for the later parts of the exercise
• ex2data1.txt - Training set for the first half of the exercise
• ex2data2.txt - Training set for the second half of the exercise
• submit.m - Submission script that sends your solutions to our servers
• mapFeature.m - Function to generate polynomial features
• plotDecisionBoundary.m - Function to plot classifier's decision boundary
• [*] plotData.m - Function to plot 2D classification data
• [*] sigmoid.m - Sigmoid Function
• [*] costFunction.m - Logistic Regression Cost Function
• [*] predict.m - Logistic Regression Prediction Function
• [*] costFunctionReg.m - Regularized Logistic Regression Cost
• Video - YouTube videos featuring Free IOT/ML tutorials

plotData.m :

Sigmoid.m :, costfunction.m :, check-out our free tutorials on iot (internet of things):.

predict.m :

Costfunctionreg.m :, 61 comments.

how could you do this please explain me...

What explanation you want? Please be more specific.

How can i download these files?

You can copy the the code from above code sections.

Hi Akshay, Please may I have theses files as well: ex2.m ex2 reg.m ex2data1.txt ex2data2.txt submit.m mapFeature.m plotDecisionBoundary.m

You can get those files from Coursera assignments. I don't have those with me now.

can you please tell me what you did by this grad = (1/m)* (X'*(h_x-y));

its the simplified version of derivative term d/d0*j0 which we call gradient. check the formula once and you will understand it

this means:- take the transpose of feature matrix X(i.e X') and multiply it with the difference of matrices h_x and y i.e the matrix with sigmoid outputs and the result matrix(y). Finally multiply the end product with 1/m , where m is the number of training examples. This is the vectorized implementation of the code that's actually way more lengthier to implement using loops.

Hi, can you please explain the predict function?

In this gradient decent the number of iteration are not specified so how is the gradient decent working? can someone please explain?

I used the exact code at the end but I'm still getting 65/100 not able to figure out the reason

Did you figure out the reason yet?

Hi !! why didn't you use sum() function for grad even why formula contains that ?

sum() is used for the summation in the formula. But here while coding for grad computation: grad = (1/m)* (X'*(h_x-y)); Here We are doing matrix multiplication which itself consist of "sum of product". So, no need of external sum function. Please try to do it on paper by yourself, you will get clear idea. Thanks

we have learned that Z= theta transpose X then why are using Z=X multiplied by theta in the above codes ?

When we are calculating z(small z) for a single sample, then it is z=theta' * x. (here small x) But When you do the same computation for all the samples at the same time then we call it as Z (Capital Z). Z = X * theta. (Here Capital X) Try to do it using pen-paper, you will get clear understanding.

Hii, thanks for your help mr. Akshay. I had this one doubt about predict.m function: I tried coding for predict function in the following way: h_x = sigmoid(X*theta); if (0<=h_x<0.5) p=0; elseif (0.5<=h_x<=1) p=1; endif I know I did it in a long way but the accuracy that I am getting 60.00. Your code gave me the accuracy 89.00. Can you please help me understand what's wrong with this and what's the exact difference between your code and mines'?

P is a matrix with dimensions m x 1. Solution: You can put your code in a "for" loop and check the value of each element in h_x and accordingly set the value of each element in p. It will work.

hey bro it says z not defined why???

Hi, I think you are doing this assignment in Octave and that's why you are facing this issue. Chethan Bhandarkar has provided solution for it. Please check it out: https://www.apdaga.com/2018/06/coursera-machine-learning-week-2.html?showComment=1563986935868#c4682866656714070064 Thanks

I have copy the exact code for plotData.m , and all the others program worked very well but I am still getting 70/100. Can you tel what's the problem ?

Can you tell me , how can I run "ex2" script in console ?

hi I want to clarify few things from you, I have read in regression, these are few important points which have not been covered in andrew ng regression topic, how to find how significant your variable is, significance of p value and R^2 (R-square) values. I would like to know more about them. kindly share some sources.

HI, The line code reg_term = (lambda/(2*m)) * sum(theta(2:end).^2); in costFunctionReg function, can you explain more about this part theta(2:end) , what does it mean and how did you deduce it,

sir,please explain me predict.m function I used for i=1:size(X,1) if sigmoid(X*theta)>=0.5 p=sigmoid(X*theta); end as well as, h_x = sigmoid(X*theta); for i=1:size(X,1) if (0<=h_x<0.5) p=0; elseif (0.5<=h_x<=1) p=1; end but i am getting 40 accuracy it is working only with your code.why sir?

Hi there, I am trying the the same code as yours of sigmoid() function but each time it is getting an error saying that 'z' undefined near line 6 column 18 error: called from sigmoid at line 6 column 5 what to do please help me out..

Hello Akshay, It'd be great if you kindly share the code for "fminunc" in this week's files(wherever needed), coz i don't understand that particular function well, neither did i get its solution anywhere else on internet.

Hi Ankit, Sorry but I don't have the code for "fminunc".

grad(2:end) = (1/m)* (X(:,2:end)'*(h_x-y))+(lambda/m)*theta(2:end); can u please explain this..

Hey it says my plot is empty can someone help?

I am facing this type of problem in matlab , what can i do ? how to fix that n where ?? 'fminunc' requires Optimization Toolbox. Error in ex2 (line 99) fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);

In sigmoid error in line 6 (the preallocated value assigned to variable 'g' might be unused) what should i do

How's value of 'g' is unused. 'g' is nothing but output of sigmoid function. If you are getting some msg, it must be warning not error. So, don't worry about it, keep it as it is. (But I don't think you should get any kind of warning like this). line 6, is called initialization of variable.

Hi Akshay can you please explain why we use this X(:,2:end) and theta(2:end) instead of plain X and theta??

It's because as per the theory in videos, We don't apply regularization on theta_0. Regularization is applied from theta_1 onwards. and that's why 2 gradients. 1st corresponding to theta_0 and other for theta_1 onwards.

And also why use two gradents?

Good day sir, im new in this course...i could not fully understand the assignment in week 3...as i enter my code...i think still in error..

please explain the predict function

Predict function is fairly simple. You have implemented your gradient and now you just have to predict whether the answer will be 1 or 0... So, what will you do is check for the result > 0.5. If it is above the 0.5, then prediction will be true (1), otherwise false (0)

@Hassan Ashas Thank you very much for your explanation.

costfuntion is not returning the scalar value, it is returning the 1*100 matrix.

Opening and closing brackets are not matching you code. NOTE: check the brackets are "2*m" YOUR CODE: reg_term = (lambda/2*m)) * sum(theta(2:end).^2); WORKING CODE: reg_term = (lambda/(2*m)) * sum(theta(2:end).^2);

Hello Akshay, While computing cost function I am getting so many outputs

You should only get [J, grad] as a output of costFunction & costFunctionReg.

Error - theta may not be defined , predict function

hi i have a doubt i took theta as [zeros(n+1),1] it is giving me 0 and i cant submit the assignment can you specify initial value of theta and theta and values of X. i am totally confused

nothing is working here every time it is showing >> plotData error: 'y' undefined near line 14 column 12 error: called from plotData at line 14 column 5 >>

J = (1 / m) * sum ((- y. * Log (h_x)) - ((1-y). * Log (1-h_x))) the log representation in this equation means ln isn't it? So, shouldn't we write it as log (1-h_x) / log (10).

I made it this way: function [J, grad] = costFunctionReg(theta, X, y, lambda) %COSTFUNCTIONREG Compute cost and gradient for logistic regression with regularization % J = COSTFUNCTIONREG(theta, X, y, lambda) computes the cost of using % theta as the parameter for regularized logistic regression and the % gradient of the cost w.r.t. to the parameters. % Initialize some useful values m = length(y); % number of training examples % You need to return the following variables correctly J = 0; grad = zeros(size(theta)); % ====================== YOUR CODE HERE ====================== % Instructions: Compute the cost of a particular choice of theta. % You should set J to the cost. % Compute the partial derivatives and set grad to the partial % derivatives of the cost w.r.t. each parameter in theta [J, grad] = costFunction(theta, X, y); feats = theta(2:end); J = J + lambda / (2 * m) * (feats' * feats); grad(2:end) = grad(2:end) + lambda / m * feats; % ============================================================= end

My question is about the solved subroutine 'plotDecisionBoundary.m' Line 20 : plot_y I didn't understand the definition of this Infact how this particular code helped to plot the decision boundary! Please explain..

so in cost function grad is basically you doing gradient descent right? but what is the use of 1/m? i'm really confused sorry

While calculating cost function, we are doing sum (summation) operation over 'm' samples. And then dividing it by 'm' in order to scale the output (as a scaling factor).

Muje 55 marks hi aa rahe he mane code bhi sahi likha he phir bhi...logistic regression cost and regularised logistic regression gradient dono me 0 marks he..

i really confused in assignment, i enjoyed all the stuff that prof.Ng doing buat why it turns out to become nightmare when im face the programming assignment? In the cosfunctionreg.m why you put grad(1) = (1/m)* (X(:,1)'*(h_x-y)); whats this mean? grad(2:end) = (1/m)* (X(:,2:end)'*(h_x-y))+(lambda/m)*theta(2:end); what grad(2:end) mean?

These 2 lines are for calcuating gradient with regularization. since we don't add regularization term to 1st entry. (we have to write 2 seperate lines of code for it)

Hi dear Akshay. I'm trying to submit week 3 assignment but I keep seeing the error: !! Submission failed: unexpected error: Error: File: costFunctionReg.m Line: 22 Column: 3 Invalid expression. Check for missing or extra characters. Can you help me out?

I am getting a syntax error in exercise "CostfunctionReg.m" at grad(1) = (1/m)* (X(:,1)'*(h_x-y)); please tell me why am i getting this error. yes i am running it in octave but please don't tell me to go through the another link . please just tell me the issue.

!! Submission failed: Index exceeds array bounds. Function: getResponse LineNumber: 132of submitWithConfiguration

Here in the cost function though y and log(h_x) both have the same dimensions (mx1), how the dot product is possible between them?

We are not doing the dot product of y and log(h_x) while calculating cost function. Multiplication represented by dot astrix (.*) means element wise multiplication in matlab. Eg. -y.*log(h_x) Please check the code once again.

Our website uses cookies to improve your experience. Learn more

Contact form

Week 3 - Logistic Regression

Download template here

The following chunk will set up your document. Run it, then ignore it.

If the system prompts you to install a package, or gives you a “package not found” error, simply run install.packages("packagename") once to install it.

The data set

We will be using the flights data set from the nycflights13 package. nycflights13 is an R data package containing all out-bound flights from NYC.

We will build a classification model that sees if any given flight is delayed or not. Furthermore, let us trim down the number of variables we are working with. Lastly, let us select to only work with flights taken place during the first month.

now that we have performed some cleaning, will we proceed to perform a train-test split.

we will use the training data set for visual exploratory data analysis to reinforce the idea that we don’t touch the testing data set.

We can look at many things with this data set. What we want to look at is how any of the variables relate to delay .

We see a varied amount of flights throughout the day. This makes sense, no one wants to leave early or late from the airport.

We see some strong artifacts in the time of scheduled departure. Most flights leave on a multiple of 5 which we confirm below.

By combining hour and minute we can look at how much the different flights have departure delays. There are some really long delays in here!

If we color the points by delay we see that it appears that most of the delayed arrivals happen because of a delayed departure.

Let’s begin with a logistic model. We will look at how dep_delay and distance affects delay .

Our first step is to establish which model(s) we want to try on the data.

For now, this is just a logistic model.

To establish the model, we need to determine which R package it comes from (the “engine”) and whether we are doing regression or classification .

(These functions come from the tidymodels package that we loaded in the setup chunk.)

Next, we will fit the model to our data:

Let’s check out the output of this model fit:

the coefficients are shown as log-odds terms. We could also get this information using tidy()

setting exponentiate = TRUE , gives us the odds instead of log-odds.

We can also take a look at how well the model is doing. By using augment() we can generate predictions, and conf_mat() and autoplot() let us create a confusion matrix and visualize it.

Experiment with using some of the other predictors in your model. Are the answers surprising? Evaluate your models with conf_mat() and accuracy() .

Once you have a model you like, predict on the test data set and calculate the performance metric. Compare to the performance metrics you got for the training data set.

Logistic Regression with a Neural Network mindset

In this post, we will build a logistic regression classifier to recognize cats. This is the summary of lecture "Neural Networks and Deep Learning" from DeepLearning.AI. (slightly modified from original assignment)

May 11, 2022 • Chanseok Kang • 17 min read

Python   Coursera   DeepLearning.AI

Information from dataset

Reshape dataset, general architecture of the learning algorithm, initializing parameters, forward and backward propagation, optimization, merge all functions into a model, choice of learning rate.

First, let's run the cell below to import all the packages that you will need during this assignment.

• numpy is the fundamental package for scientific computing with Python.
• h5py is a common package to interact with a dataset that is stored on an H5 file.
• matplotlib is a famous library to plot graphs in Python.
• PIL and scipy are used here to test your model with your own picture at the end.

You are given a dataset ("data.h5") containing:

You will build a simple image-recognition algorithm that can correctly classify pictures as cat or non-cat.

We added "_orig" at the end of image datasets (train and test) because we are going to preprocess them. After preprocessing, we will end up with train_set_x and test_set_x (the labels train_set_y and test_set_y don't need any preprocessing).

Each line of your train_set_x_orig and test_set_x_orig is an array representing an image. You can visualize an example by running the following code. Feel free also to change the index value and re-run to see other images.

We want to find out how many data do we have, and what shape each image have. Remember that train_set_x_orig is a numpy-array of shape (m_train, num_px, num_px, 3).

For convenience, you should now reshape images of shape (num_px, num_px, 3) in a numpy-array of shape (num_px $*$ num_px $*$ 3, 1). After this, our training (and test) dataset is a numpy-array where each column represents a flattened image. There should be m_train (respectively m_test) columns.

Reshape the training and test data sets so that images of size (num_px, num_px, 3) are flattened into single vectors of shape (num_px $*$ num_px $*$ 3, 1).

A trick when you want to flatten a matrix X of shape (a,b,c,d) to a matrix X_flatten of shape (b$*$c$*$d, a) is to use:

To represent color images, the red, green and blue channels (RGB) must be specified for each pixel, and so the pixel value is actually a vector of three numbers ranging from 0 to 255.

One common preprocessing step in machine learning is to center and standardize your dataset, meaning that you substract the mean($\mu$) of the whole numpy array from each example, and then divide each example by the standard deviation($\sigma$) of the whole numpy array. But for picture datasets, it is simpler and more convenient and works almost as well to just divide every row of the dataset by 255 (which is the maximum value of a pixel channel).

Let's standardize our dataset.

What you need to remember:

Common steps for pre-processing a new dataset are:

• Figure out the dimensions and shapes of the problem (m_train, m_test, num_px, ...)
• Reshape the datasets such that each example is now a vector of size (num_px * num_px * 3, 1)
• "Standardize" the data

It's time to design a simple algorithm to distinguish cat images from non-cat images.

You will build a Logistic Regression, using a Neural Network mindset. The following Figure explains why Logistic Regression is actually a very simple Neural Network!

Mathematical expression of the algorithm :

For one example $x^{(i)}$: $$z^{(i)} = w^T x^{(i)} + b \tag{1}$$ $$\hat{y}^{(i)} = a^{(i)} = sigmoid(z^{(i)})\tag{2}$$ $$ \mathcal{L}(a^{(i)}, y^{(i)}) = - y^{(i)} \log(a^{(i)}) - (1-y^{(i)} ) \log(1-a^{(i)})\tag{3}$$

The cost is then computed by summing over all training examples: $$ J = \frac{1}{m} \sum_{i=1}^m \mathcal{L}(a^{(i)}, y^{(i)})\tag{6}$$

Building the parts of our algorithm

The main steps for building a Neural Network are:

• Define the model structure (such as number of input features)
• Initialize the model's parameters
• Calculate current loss (forward propagation)
• Calculate current gradient (backward propagation)
• Update parameters (gradient descent)

You often build 1-3 separately and integrate them into one function we call model() .

we need to implement implement sigmoid() . As you've seen in the figure above, you need to compute $$sigmoid(z) = \frac{1}{1 + e^{-z}}$$ for $z = w^T x + b$ to make predictions.

Now we need to implement parameter initialization in the cell below. You have to initialize w as a vector of zeros.

Now that your parameters are initialized, you can do the "forward" and "backward" propagation steps for learning the parameters. Note that, Forward Propagation:

• You compute $A = \sigma(w^T X + b) = (a^{(1)}, a^{(2)}, ..., a^{(m-1)}, a^{(m)})$
• You calculate the cost function: $J = -\frac{1}{m}\sum_{i=1}^{m}(y^{(i)}\log(a^{(i)})+(1-y^{(i)})\log(1-a^{(i)}))$

Here are the two formulas you will be using:

$$ \frac{\partial J}{\partial w} = \frac{1}{m}X(A-Y)^T\tag{7}$$ $$ \frac{\partial J}{\partial b} = \frac{1}{m} \sum_{i=1}^m (a^{(i)}-y^{(i)})\tag{8}$$

You have initialized your parameters. and also able to compute a cost function and its gradient. Now, you want to update the parameters using gradient descent.

The previous function will output the learned w and b. We are able to use w and b to predict the labels for a dataset X. Implement the predict() function. There are two steps to computing predictions:

Calculate $\hat{Y} = A = \sigma(w^T X + b)$

Convert the entries of a into 0 (if activation <= 0.5) or 1 (if activation > 0.5), stores the predictions in a vector Y_prediction . If you wish, you can use an if / else statement in a for loop (though there is also a way to vectorize this).

What to remember:

You've implemented several functions that:

• Initialize (w,b)
• Computing the cost and its gradient
• Updating the parameters using gradient descent
• Use the learned (w,b) to predict the labels for a given set of examples

You will now see how the overall model is structured by putting together all the building blocks (functions implemented in the previous parts) together, in the right order.

Comment : Training accuracy is close to 100%. This is a good sanity check: your model is working and has high enough capacity to fit the training data. Test accuracy is 70%. It is actually not bad for this simple model, given the small dataset we used and that logistic regression is a linear classifier.

Also, you see that the model is clearly overfitting the training data. Later in this specialization you will learn how to reduce overfitting, for example by using regularization. Using the code below (and changing the index variable) you can look at predictions on pictures of the test set.

Let's also plot the cost function and the gradients.

Interpretation : You can see the cost decreasing. It shows that the parameters are being learned. However, you see that you could train the model even more on the training set. Try to increase the number of iterations in the cell above and rerun the cells. You might see that the training set accuracy goes up, but the test set accuracy goes down. This is called overfitting.

Further analysis

Let's analyze it further, and examine possible choices for the learning rate $\alpha$.

Reminder : In order for Gradient Descent to work you must choose the learning rate wisely. The learning rate $\alpha$ determines how rapidly we update the parameters. If the learning rate is too large we may "overshoot" the optimal value. Similarly, if it is too small we will need too many iterations to converge to the best values. That's why it is crucial to use a well-tuned learning rate.

Let's compare the learning curve of our model with several choices of learning rates. Run the cell below. This should take about 1 minute. Feel free also to try different values than the three we have initialized the learning_rates variable to contain, and see what happens.

What to remember from this assignment:

• Preprocessing the dataset is important.
• You implemented each function separately: initialize(), propagate(), optimize(). Then you built a model().
• Tuning the learning rate (which is an example of a "hyperparameter") can make a big difference to the algorithm.

Bibliography:

• http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/
• https://stats.stackexchange.com/questions/211436/why-do-we-normalize-images-by-subtracting-the-datasets-image-mean-and-not-the-c

Instantly share code, notes, and snippets.

mGalarnyk / machineLearningWeek3Quiz1.md

• Download ZIP
• Star ( 2 ) 2 You must be signed in to star a gist
• Fork ( 2 ) 2 You must be signed in to fork a gist
• Embed Embed this gist in your website.
• Share Copy sharable link for this gist.
• Clone via HTTPS Clone using the web URL.
• Learn more about clone URLs
• Save mGalarnyk/a890c3d6bb42923cb0543c7d7809cb77 to your computer and use it in GitHub Desktop.

Machine Learning Week 3 Quiz 1 (Logistic Regression) Stanford Coursera

Github repo for the Course: Stanford Machine Learning (Coursera) Quiz Needs to be viewed here at the repo (because the image solutions cant be viewed as part of a gist)

chukwurah498 commented Sep 3, 2020

There ought to be an explanation why you felt these are answers. While I agree on some , there must be explanation as the instructor does.

Sorry, something went wrong.

Coursera - Predictive Modeling Analytics - Week 3

Matt girard.

This document will walk through the Week 3 assignment for the Coursera Predictive Modeling Analytics course. This course focuses on classification using Logistic Regression.

The following libraries are used:

Reading and Preparing Data

Per the assignment, we will continue working with the customer rewards program dataset. The code below reads in the data and creates the two binary dummy columns as called for by the assignment.

Training initial model

Next, we train a Logistic Regression model using Reward as the target variable and our dummy grocery and discount variables as our predictors. Note that family is set to "binomial" meaning that our output/target is between 0 and 1.

Confusion Matrix

To display our confusion matrix, we use the predict function to get probability predictions for our data, then use ifelse to create discrete predictions that align with our desired binary output.

For this, we use a standard 0.5 as our cutoff.

Data Partitioning

The second part of the assignment calls for data partition using a 60/40 split for training and testing sets respectively.

Note: Due to variations in how caret or R’s random number generator work compared to XLMiner, you will need to use XLMiner to get the exact answers required to pass the assignment’s automated grader.

Partitioning can be done using a variety of methods, in a commented section I use R’s sample.int function to generate a vector of random indecies that I can use to subset my data, using the size parameter to limit the numbers selected to match the desired split size. We negate this set using the - operator to get our testing set.

An easier method is to use the createDataPartition method offered by caret which allows you to specify the percent using the p argument, as well as a vector to partition by. In this case I used Reward , which will ensure that the distribution of Reward in our training and testing sets is the same as the original.

To check distributions among the different datasets I create a function, get_dist , to aid in prepapring the summary.

I then plot the results using ggplot. Alternatively, we can print to console using dcast.

Both methods show that our target variable distirbution is the same across all datasets.

We then train our model using the training dataset and display the summary.

With the model trained, we get predictions for our test set and create a confusion matrix.

As the assignment suggests, we can modify our cutoff to be 0.3 instead of the standard 0.5. However, this ends up giving us all positive predictions.

Deep-Learning-Specialization

Coursera deep learning specialization, neural networks and deep learning.

In this course, you will learn the foundations of deep learning. When you finish this class, you will:

• Understand the major technology trends driving Deep Learning.
• Be able to build, train and apply fully connected deep neural networks.
• Know how to implement efficient (vectorized) neural networks.
• Understand the key parameters in a neural network’s architecture.

Week 1: Introduction to deep learning

Be able to explain the major trends driving the rise of deep learning, and understand where and how it is applied today.

• Quiz 1: Introduction to deep learning

Week 2: Neural Networks Basics

Learn to set up a machine learning problem with a neural network mindset. Learn to use vectorization to speed up your models.

• Quiz 2: Neural Network Basics
• Programming Assignment: Python Basics With Numpy
• Programming Assignment: Logistic Regression with a Neural Network mindset

Week 3: Shallow neural networks

Learn to build a neural network with one hidden layer, using forward propagation and backpropagation.

• Quiz 3: Shallow Neural Networks
• Programming Assignment: Planar Data Classification with Onehidden Layer

Week 4: Deep Neural Networks

Understand the key computations underlying deep learning, use them to build and train deep neural networks, and apply it to computer vision.

• Quiz 4: Key concepts on Deep Neural Networks
• Programming Assignment: Building your Deep Neural Network Step by Step
• Programming Assignment: Deep Neural Network Application

Course Certificate

[Syllabus subject to change until August 28th, 2024].

Students in Stat 131A are expected to have read the syllabus in its entirety by the second week of the course.

Course Details 🥗

Description 🔎.

Stat 131A is a upper-division course that follows Data 8 or STAT 20. The course will teach a broad range of statistical methods that are used to solve data problems, including group comparisons, standard parametric statistical models, multivariate data visualization, multiple linear regression and classification, classification and regression trees, and random forests. Students will be introduced to the widely used R statistical language and they will obtain hands-on experience in implementing a range of statistical methods on numerous real world datasets.

In short, Stat 131A will provide you with a Swiss army knife of foundational statistical methods to use for data science projects!

Lectures 🧑‍🏫

MWF 2-3pm @ Morgan 101

Lab 101: Tuesday and Thursday, 11am-12pm @ Evans 330

Lab 102: Tuesday and Thursday, 4-5pm @ Evans 344

You should only be signed up for one lab group.

Office hours (OH) 🗓️

OH times and locations are tentative.

• Note: There are no office hours in the first week of the course!

Josh’s Office Hours:

• Tuesdays 5-6pm @ Evans 421
• Thursdays 5-6pm @ Evans 421

Van’s Office Hours:

• Wednesdays 10am-12pm @ Evans 434
• Fridays 9am-11am on Zoom
• For security reasons, Zoom links for OH are posted on bcourses .

15-minute coffee chats with Josh on the phone (experimental!):

• One slot every weekday at 9am.
• Designed for individual advising, not logistical concerns or coursework help.
• For example, we can talk about career plans or life advice.
• Please limit to no more than one chat per month.
• Book at this link

We may add or reschedule OH if needed.

• Please create a private Ed post if you have conflicts with all of the scheduled OH.

Coming to office hours does not send a signal that you are behind or need extra help. In fact, the students who come to OH are often the most successful in the course.

• OH is a great opportunity to discuss not only topics directly related to the course, but also anything else that’s on your mind.
• We also welcome questions about career trajectories and research opportunities at UC Berkeley and beyond.
• Keep in mind that you do not need to come to office hours with an agenda. Listening in is welcomed and encouraged!
• Finally, attending and participating in office hours is a great way to set yourself up for a terrific letter of recommendation. This is true for most courses!
• If you don’t already, I highly recommend that you attend the instructors’ office hours in other classes from time to time.

Study groups 👥

We encourage you to work together in groups to solidify your understanding of the course material.

If you would like assistance forming a study group, please complete the study group form by Monday, September 2nd at 11:59pm PT.

Our goal is to form the study groups ASAP, so students can begin discussing the first homework assignment.

Concurrent enrollment and auditing 👂

Concurrent enrollment students wishing to register for the class should fill out this Google Form to give me information about their previous coursework so we can assess whether they have satisfied the pre-requisites.

Students who wish to audit the course can also fill out the top portion of this form to get their email added to bcourses as a guest. Only name and email is needed for auditors.

Course platforms 🖥️

bcourses will only be used for secure course material, like exam solutions, grades, and office hour Zoom links.

All other course materials will be posted on the public course homepage .

Assignments should be submitted via Gradescope .

All course communication will take place via Ed .

The only acceptable large-language model (LLM) for use in Stat 131a is PingPong .

• Unless otherwise indicated, all other LLMs (e.g., ChatGPT) are prohibited and considered cheating in this course.
• Enrolled students will receive an invitation to PingPong via email shortly after the semester begins.

Grades are calculated as follows:

• Lecture attendance and participation: 10%
• Homework: 20%
• Final project: 15%
• Midterm 1 (during class): 10%
• Midterm 2 (during class): 15%

Grades will not be curved.

• In other words, there is no limit to the proportion of students with an A, B, etc. You are incentivized to help each other learn and succeed.
• You are guaranteed an A if you score 93% or higher, an A- if 90% or higher, a B+ if 87% or higher, a B if 83% or higher, and so on.
• A+ grades are awarded rarely, and only for truly exceptional performance.
• Grade cutoffs may be adjusted downward at the end of the semester, but this is not guaranteed.

See attendance policy below for an opportunity to earn up to two percentage points of extra credit.

Lecture technology policy ❌ 👩‍💻 \(~\) ✅📱

Most lectures will consist of an interactive problem-solving session, followed by a hands-on demo or coding session.

Laptops and tablets with attached keyboards are not allowed during the problem-solving session , though you are permitted to use a tablet to take handwritten notes.

If you need to use technology for accessibility reasons, the previous bullet does not apply to you.

Laptop use is permitted (and encouraged!) during the hands-on demo and coding sessions.

Phones are allowed during lecture. It is preferable to use a phone to submit conceptual questions and neighbor discussion answers during lecture.

This article explains why we have the laptop policy. Long story short, laptop use can negatively impact the learning of nearby students (i.e., this policy does not punish you; the policy prevents you from punishing others).

The course staff reserves the right to reduce your lecture attendance grade for violating the technology policy.

Lecture recordings 🎥

Lectures will be recorded automatically.

• The course staff cannot guarantee audio or video quality.
• Lecture recordings are posted on bcourses .

Labs and office hours are not recorded.

The homework assignments may occasionally ask you to watch additional recordings to supplement the lecture material (e.g., if we run out of time covering an essential topic).

Attendance and participation ✋

In-person lecture attendance is mandatory.

• It is critically important to practice learning in a live setting.
• Difficulty with paying attention in live meetings is a common hurdle for new grads.

Lecture attendance is a substantial component of your grade.

• Lecture cannot be attended remotely.
• You are allowed three unexcused lecture absences. Each additional absence will impact your lecture attendance grade.

If you cannot attend a lecture due to an extenuating circumstance, please complete the lecture attendance excusal form before the lecture starts.

• This form can be completed months, weeks, or days in advance of lecture.

Acceptable extenuating circumstances include:

• Illness. DO NOT come to class if you are sick! Even a sniffle!
• Personal emergencies.
• Important life events (e.g., weddings)
• Pre-planned collegiate athletic events in which you are a participant.
• This list is not exhaustive. If you think an absence should be excused, complete this form and explain your reasoning. We cannot guarantee that your absence will be excused, but we will be reasonable.

Concept checks ✅

We will use in-class concept checks and neighbor discussions to track attendance.

• Concept checks are not graded.
• Concept checks are answered via this form .
• Submitting a concept check outside of standard lecture time is considered cheating and an honor code violation. We will use your seat number and submission time to validate that your responses were entered during lecture time. We reserve the right to photograph the lecture hall to verify attendance.

Neighbor discussions 🗣️

In addition to concept checks, there may be one or more neighbor discussions during each lecture.

• Neighbor discussion answers are submitted via this form .
• Neighbor discussion answers are not graded.

To encourage discussion among all classmates, we will award up to two percentage points of extra credit for having a variety of neighbors.

• The students with the highest number of unique neighbors will receive the full two percentage points of extra credit.
• Everyone else will receive, at the minimum, a fraction of extra credit proportional to their number of unique neighbors.
• For example, if you sit next to the same person all semester, you can receive full participation credit for neighbor discussions, but you will very little extra credit.
• The extra credit policy will only take effect if at least one student has spoken to at least 20 unique neighbors over the course of the semester.
• As above, submitting a neighbor discussion answer outside of standard lecture time is considered cheating and an honor code violation.

There are 6 homework assignments planned, though the exact number may change.

• Homework will be a combination of computational exercises and data analysis using the computer, as well as conceptual questions.
• Homework assignments are weighted equally.

HW is generally due every other week.

• Homework assignments will be posted to the course website at least one week before the HW deadline.
• All homework assignments will be submitted via Gradescope and are due by 11:59 pm of the due date.

We will not drop your lowest-scoring homework assignment.

• Instead, we will raise your lowest homework score to 80% of its maximum score, regardless of what you actually scored.
• We will not change the grade of your lowest-scoring homework assignment if its score is above 80%.

Poorly organized assignments will be docked points at the discretion of the grader.

• It is critical to have empathy for the person who will be reviewing your work, whether a member of the course staff, another student providing feedback, or your future manager.

You are allotted five slip days for labs and homework assignments.

• Each slip day adds 24 hours to the deadline.
• Slip days cannot be used on the final project or exams.
• Slip days are intended to account for unexpected delays, like minor illness or homework overload.
• There is no extra credit awarded for unused slip days.
• You cannot use partial slip days.

You are allowed to use, at most, two slip days per assignment.

• In other words, assignments will not be accepted more than 48 hours after the original due date.
• This policy ensures that we can grade all assignments in a timely fashion.

If you plan to use slip days, do not contact the course staff.

• We will automatically account for slip days when calculating grades.

Extensions will only be granted if required by a Letter of Accommodation (LoA), or in extraordinary circumstances (e.g., medical emergencies).

During lab sessions, a GSI will review conceptual material and help you work through lab coding assignments.

• Lab sections meet twice a week.
• There are no lab sections the first week of classes.

We plan to have 12 lab assignments.

• Each assignment will teach you how to perform the analyses shown in class using R .
• Labs are intended to be finished or mostly finished during section.
• HW assignments may build on the exercises covered in lab.

Lab assignments are generally due on Mondays at 11:59 pm and should be submitted via Gradescope .

• Labs are graded on completion, not correctness.

While there is no Week 1 lab, there is a self-paced Lab 0 for you to work through independently. This lab is not graded.

Before the first lab section on Tuesday of Week 2, work through Lab 0 .

You do not have to turn in this lab, but we will assume that you have worked through this lab and understand the code.

If you have questions about this lab, please come to office hours or post on Ed.

Final project 📊

The final project will be due on the last day of reading week, Friday December 13 .

More details on the final project will be provided later in the semester.

The first midterm is scheduled for Wednesday October 9 and will take place during lecture in Morgan 101.

The second midterm is scheduled for Wednesday November 13 and will take place during lecture in Morgan 101.

The final exam will take place Thursday December 19, 3-6pm (scheduled by the registrar). Location TBD.

All exams are cumulative, with emphasis on more recent material.

• To acknowledge maturation over the course of the semester, exams are increasingly weighted in your course grade.
• If you do not perform well on the first midterm, you can still do very well in 131a!

If you cannot attend an exam due to an extenuating circumstance, please contact the course staff ASAP to determine whether your circumstance qualifies for a make-up exam.

Textbooks and resources 📖

Everything you need to know for Stat 131a will be covered in lectures, labs, and assignments.

• It is possible to do very well in Stat 131a without ever referring to an outside textbook or resource.

However, most of the course material is covered by the online textbook developed specifically for 131A.

• You can find the textbook here .

The StatQuest YouTube Channel is an excellent resource.

• StatQuest provides videos on many of the topics we will cover in class. The instructor is very entertaining!

If you would like some additional optional reading, you can try the following books.

• Theory Meets Data by Ani Adhikari. This is the online book for STAT 88 that covers introductory probability at the level of Stat 20.
• R for Data Science , by Garrett Grolemund and Hadley Wickham. This is a free online book that covers the tidyverse set of R packages.
• The Statistical Sleuth: A Course in Methods of Data Analysis by Ramsey and Schafer
• Introductory Statistics with R by Peter Dalgaard

None of these books covers all of the topics we will cover in 131A, nor do they necessarily have the same perspective and focus as this class. But for those students wanting some additional structure or R assistance, these books may be helpful and should be at the right level for this class.

Stat 20 and Data 8 are similar courses, but each covers some subjects that the other does not. While we will cover these topics in class, you may find the following useful background if you are seeing them for the first time (more to follow):

• Computational and Inferential Thinking: The Foundations of Data Science , by Ani Adhikari and John DeNero – chapters 11-13.

This is the online book used by Data 8. These chapters introduce hypothesis testing using only resampling ideas, ideas which are not necessarily covered in Stat 20.

Policy on Large Language Models (LLMs) 💬

LLMs (e.g., ChatGPT) are becoming increasingly essential in the workplace.

• To that end, the use of LLMs is not only permitted in this course, but encouraged.
• Use this course as an opportunity to learn where LLMs are most useful, and where they fall short.

Potential uses of LLMs in Stat 131A:

• Generating practice quiz questions
• Explaining course concepts
• Helping you code

Unless otherwise indicated, you can only use the course-approved LLM PingPong for help on labs, homework, and the final project. .

• Furthermore, if you use PingPong to help you, you must submit a PDF of the relevant PingPong conversation along with your assignment.
• You are responsible for understanding every line of code that you submit in 131A. Exams may ask you to explain specific lines of code used in lab and HW solutions. So, don’t use LLMs to write code without taking time to understanding the code.
• Of course, LLMs cannot be used on exams.

It is often easy to spot default LLM text output.

• If you copy and paste answers directly from PingPong, be warned that your grader may interpret your answer as lacking in effort and you may lose points on the assignment.
• Take the time to understand and paraphrase the information LLMs provide you.

If you find an especially interesting use case of an LLM for any component of the course, please share it with the course staff! We are excited to hear what you find.

Course communication 🗣️

We use the Ed platform to manage course questions and discussion, and to make announcements.

In general, do not email the course staff.

• Exception: You are welcome to email individual members of the course staff if you have a private concern that you do not want shared with the entire course staff.

Please post publicly when possible.

• Public posts benefit many more students than private posts.
• We may ask you to change your private post to a public post if the answer could be of use to other students.
• You are always allowed to remain anonymous!

If you include code in your Ed post, please use the code editing fonts :

Standard font is hard to read:

── Attaching packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 1.3.0 ── ✓ ggplot2 3.3.2 ✓ purrr 0.3.4 ✓ tibble 3.0.3 ✓ dplyr 1.0.2 ✓ tidyr 1.1.2 ✓ stringr 1.4.0 ✓ readr 1.3.1 ✓ forcats 0.5.0 ── Conflicts ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ── x dplyr::filter() masks stats::filter() x dplyr::lag() masks stats::lag()

# here’s my plot code

x <- ggplot(df) + geom_point(aes(x = year, y = count))

Code font is easier to read:

Computing environment 🖥️

The official course materials use the R programming language.

• As in Data 8 and Stat 20, labs and assignments will be distributed via DataHub .

You do not need to know anything about R to take this course.

• We will provide resources for you to learn everything you need to know.

The concepts taught in this course are language-agnostic.

• In other words, everything you learn in this class can be readily implemented using a combination of other tools (e.g., Python , SQL , etc.).
• Note that LLMs are an excellent aid for translating your knowledge across different programming language and software.

Weekly topics 🥗

The following is a rough and optimistic guideline for the material we will cover in the semester.

• The actual topics may vary as the semester goes along.
• It is likely that we will proceed more slowly than this schedule indicates.
• Relevant sections of the textbook are linked for each week, though you are only responsible for the material we cover in lecture, lab, or HW assignments.

Week 1. Principles of visualization.

Week 2. Boxplots and histograms. Discrete and continuous distributions. 2.1 , 2.3 .

Week 3. Probability. Bayes’ theorem. Naive Bayes algorithm. 2.2 .

Week 4. Sampling distributions. Bootstrapping. 2.4 .

Week 5. Confidence intervals. Parametric hypothesis testing. 3 .

Week 6. Non-parametric hypothesis testing. Type I and II errors. Power. Study design. Multiple testing. 3 .

Week 7. Midterm 1 (Weeks 1-5). Linear regression. 4.1-4.3 , 6

Week 8. More linear regression. Feature generation. Transformations. 6

Week 9. Cross-validation. Bias-variance tradeoff. Logistic regression. Classification error metrics. 7 7.5

Week 10. Buffer for Week 1-9 catch-up.

Week 11. Intro to non-parametric methods. Kernel density estimation (KDE). LOESS. 2.5 . 4.4-4.5

Week 12. Midterm 2 (Weeks 1-10). Principle components analysis (PCA). Clustering. 5.4

Week 13. Decision trees. Random Forests. 8

Week 14. Buffer for Week 10-13 catch-up.

Week 15. Most likely, catch-up and review. If time permits, intro to causal inference.

Academic Honesty Policy 👍

Homework and projects must be completed independently, with the following exceptions:

• You may discuss specific issues/questions you have about the homework at a high level, but you must not sit down and do the assignment jointly.
• Giving advice about code or coding tips is also not cheating, but you can not directly share code with other classmates.

For exams, cheating includes, but is not limited to, using electronic materials in an exam beyond that allowed, copying off another person’s exam or quiz, allowing someone to copy off of your exam or quiz, and having someone take an exam or quiz for you.

Requesting, obtaining, and/or using solutions from previous years or from the internet or other sources, if such happen to be available, is considered cheating.

In fairness to students who put in an honest effort, cheaters will be harshly treated.

• Any evidence of cheating will result in a score of zero (0) on the entire assignment or examination, and perhaps a failing grade in the class.
• We will always report incidences of cheating to the Office of Student Conduct, which may administer additional punishment.

Accommodations 💙

UC Berkeley is committed to creating a learning environment that meets the needs of its diverse student body including students with disabilities.

• If you anticipate or experience any barriers to learning in this course, please feel welcome to discuss your concerns with Josh, whether after class, in office hours, via Ed, or via email.

If you already have a Letter of Accommodation, please open a private Ed post ASAP and attach your LoA.

• We can accommodate you more easily if you provide this information early in the semester.
• We cannot guarantee that last-minute requests for accommodation will be provided.

If you have a disability, or think you may have a disability, you can work with the Disabled Students’ Program (DSP) to determine any accommodations you may need to have equal access in this course.

• The Disabled Students’ Program (DSP) is the campus office responsible for authorizing disability-related academic accommodations, in cooperation with the students themselves and their instructors.
• You can find more information about the DSP application process here .
• Josh is available if you have any questions or concerns about your accommodations.
• In the event of a disagreement, the proper procedure is for you to work with your DSP Specialist and your DSP Specialist to work with Josh toward a resolution.

Accessible DS education for all ⭐

In support of our commitment to making Data Science education inviting, engaging, and respectful for people of diverse identities, backgrounds, experiences, and perspectives, I want to relay the following three items from the Data Science Undergraduate Studies (DSUS):

Device Lending options

Students can access device lending options through the Student Technology Equity Program (STEP) program .

Data Science Student Climate

Data Science Undergraduate Studies faculty and staff are committed to creating a community where every person feels respected, included, and supported. We recognize that incidents may happen, sometimes unintentionally, that run counter to this goal. There are many things we can do to try to improve the climate for students, but we need to understand where the challenges lie. If you experience a remark, or disrespectful treatment, or if you feel you are being ignored, excluded or marginalized in a course or program-related activity, please speak up. Consider talking to your instructor, but you are also welcome to contact Executive Director Christina Teller at [email protected] or report an incident anonymously through this online form .

Community Standards

Ed is a formal, academic space. Posts in this forum must relate to the course and be in alignment with Berkeley’s Principles of Community and the Berkeley Campus Code of Student Conduct. We expect all posts to demonstrate appropriate respect, consideration, and compassion for others. Please be friendly and thoughtful; our community draws from a wide spectrum of valuable experiences. Posts that violate these standards will be removed.