Jue Guo
← All courses

CSE 474/574 · Undergraduate and graduate

Introduction to Machine Learning

A first course in machine learning for upper-level undergraduates and graduate students, developed carefully from the mathematics up and implemented in Python.

The course covers supervised learning (linear and logistic regression, support vector machines, decision trees, and neural networks), unsupervised learning (clustering and dimensionality reduction), model evaluation, regularization, and the bias–variance trade-off. Students implement the core algorithms from scratch with NumPy and SciPy, and learn to read and reproduce results from the machine-learning literature.

Every module has a full set of notes, linked from the outline below, one per chapter of Bishop: the models and derivations in plain language, NumPy code written from scratch with its real output, figures, and exercises.

What the course covers

  1. Foundations
  2. Week 1 Introduction: curve fitting, probability, and decisionsOverfitting and regularization, Bayes' theorem, cross-validation, the curse of dimensionality, decision theory, and information theory.
  3. Week 2 Probability distributionsBeta, Dirichlet, and conjugate priors; the Gaussian in depth; the exponential family; kernel density and nearest neighbors.
  4. Linear models, neural networks, and kernels
  5. Week 3 Linear models for regressionLeast squares and ridge regression, the bias–variance trade-off, Bayesian linear regression, and the evidence approximation.
  6. Week 4 Linear models for classificationFisher's discriminant, the perceptron, generative classifiers and naive Bayes, logistic regression with IRLS, and the Laplace approximation.
  7. Week 5 Neural networksBackpropagation built from scratch, the Hessian, regularization and early stopping, and mixture density networks.
  8. Week 6 Kernel methods and Gaussian processesDual representations, valid kernels, Nadaraya–Watson, and Gaussian processes for regression and classification.
  9. Week 7 Sparse kernel machines: SVMs and RVMsMaximum margins, soft margins and the hinge loss, support vector regression, and the relevance vector machine.
  10. Week 8 Midterm examCovers modules 01–07.
  11. Graphical models and approximate inference
  12. Week 9 Graphical modelsBayesian networks, d-separation, Markov random fields, and the sum-product and max-sum algorithms.
  13. Week 10 Mixture models and EMK-means, Gaussian mixtures, and the EM algorithm as coordinate ascent on a lower bound.
  14. Week 11 Approximate inferenceMean-field variational inference, the variational Gaussian mixture, local variational bounds, and expectation propagation.
  15. Week 12 Sampling methodsRejection and importance sampling, Metropolis–Hastings, Gibbs and slice sampling, and Hamiltonian Monte Carlo.
  16. Latent structure, sequences, and ensembles
  17. Week 13 Continuous latent variablesPCA, probabilistic PCA and factor analysis, kernel PCA, ICA, and autoencoders.
  18. Week 14 Sequential dataHidden Markov models with forward–backward, Baum–Welch, and Viterbi; the Kalman filter and particle filters.
  19. Week 15 Combining models; project presentationsBagging, AdaBoost, decision trees, and mixtures of experts.
  20. Finals Cumulative final exam and final project reportThe final exam emphasizes modules 08–14.

How the course runs

Three lecture hours a week, about one chapter per week. The exercises at the end of each module are good practice for the problem sets and the exams: try each derivation yourself before checking it against the notes, and change the code to see what breaks. Office hours are posted on the LMS.

  • Problem sets (40%) — five sets, 8% each, mixing derivations with implementation, released in weeks 2, 4, 6, 9, and 11 and due a week later. A short math diagnostic (HW0) in week 1 helps you find gaps in the prerequisites early.
  • Midterm exam (15%) — in week 8, covering modules 01–07.
  • Final exam (20%) — cumulative, with emphasis on modules 08–14.
  • Final project (25%) — research-style: a proposal (week 5), a milestone (week 10), a draft (week 14), a presentation (week 15), and a final report.

The midterm date is announced a week ahead, based on the pace of the course, and the schedule may change with the pace and performance of the class. Grades use an absolute scale with no curve: 95–100 A, 90–94 A−, 85–89 B+, 80–84 B, 75–79 B−, 70–74 C+, 65–69 C, 60–64 C−, 55–59 D, below 55 F.

Getting set up

The code in the notes needs Python 3 with NumPy (version 2.0 or newer) and SciPy; nothing else. Any recent Python installation, or a free Google Colab notebook in the browser, runs every example. Paste the cells of a module into a notebook in order and run them: every random number comes from a seeded generator, so your output should match the notes, apart from cells that time the code and differences in the last printed digit.

Course policies

Late work

  • Three late days in total across all problem sets, no questions asked; each extends a deadline by 24 hours.
  • Once late days are used up, late submissions lose 20% per day for up to three days, and receive no credit after that.
  • Project deadlines are firm; late days do not apply.

Collaboration

Discussing problem sets with classmates at the level of ideas is encouraged, but your code and write-ups must be your own. Helping someone debug an error message is fine; sharing solutions is not. Pair work is welcome on the final project, as long as the report makes each member's contribution clear.

AI and LLM use

  • Allowed without disclosure: autocomplete-style help (syntax lookups, translating code between libraries), debugging individual error messages, and reading recommendations.
  • Allowed with disclosure: explanations of mathematical concepts you are stuck on, and suggested approaches after you have thought the problem through. A one-line note in your submission is enough.
  • Not allowed: generating problem-set solutions or large blocks of project code with an LLM. Undisclosed LLM-generated work is an academic-integrity violation.

When in doubt, disclose. Disclosure costs you nothing; non-disclosure can cost you the course.