Jue Guo
← All courses

CSE 455/555 · Undergraduate and graduate

Introduction to Pattern Recognition

Statistical pattern classification from first principles — Bayes decision theory, density estimation, nearest neighbors, linear machines, neural networks, trees, and clustering — built up carefully and implemented in Python.

The course follows Duda, Hart & Stork's Pattern Classification, one chapter per session. It asks the questions every working classifier has to answer: what the best possible decision is and how close we can get to it, how to learn densities and boundaries from a finite sample, how to measure and compare error rates without fooling ourselves, and what can be learned when there are no labels at all. The notes implement each method in NumPy and test it against theory, brute force, or simulation, and the exercises ask you to extend that code.

Each module's notes, linked from the outline below, give the ideas and derivations in plain language, NumPy code written from scratch with its real output, figures, and exercises.

What the course covers

Twelve sessions over six weeks. Each of the first ten covers one module and has a short online quiz on it, due before class; the last two go to the final projects. No exams.

  1. Decision theory and estimation
  2. Session 1 Introduction: machine perception and the design cycleThe stages of a recognition system, features and costs, and why training error is not the error that matters.
  3. Session 2 Bayesian decision theoryRisk and the Bayes rule, minimax and Neyman–Pearson, Gaussian discriminants, error bounds, and ROC curves.
  4. Session 3 Maximum-likelihood and Bayesian parameter estimationMaximum-likelihood and Bayesian learning, the curse of dimensionality, PCA and Fisher discriminants, EM, and hidden Markov models.
  5. Nonparametric and discriminative methods
  6. Session 4 Nonparametric techniquesParzen windows, k-nearest neighbors, the Cover–Hart error bounds, editing and condensing, and tangent distance.
  7. Session 5 Linear discriminant functionsPerceptron and relaxation rules with their convergence proofs, MSE and Ho–Kashyap, linear programming, and SVMs.
  8. Session 6 Multilayer neural networksBackpropagation from scratch, outputs as posterior probabilities, practical training, and second-order methods.
  9. Search, trees, and the theory of learning
  10. Session 7 Stochastic methodsSimulated annealing, Boltzmann learning, genetic algorithms, and genetic programming.
  11. Session 8 Nonmetric methods: trees, strings, and grammarsCART, ID3, and C4.5; edit distance and string matching; grammars and parsing.
  12. Session 9 Algorithm-independent machine learningNo free lunch, bias and variance, the bootstrap, bagging and boosting, cross-validation, and combining classifiers.
  13. Learning without labels
  14. Session 10 Unsupervised learning and clusteringMixtures and EM, k-means, clustering criteria, hierarchical clustering, ICA, MDS, and self-organizing maps.
  15. Sessions 11–12 Final-project talksEach student presents their question, the evidence, and what they would try next.

How the course runs

Two sessions a week; office hours are announced at the start of the term, and the schedule may shift with the pace of the class. Read the module's notes, and ideally the matching chapter, before class: a short online quiz on it is due before the session, which then works through the key derivations and code together. Class time goes to each chapter's core sections; the sections DHS marks as starred are covered in the notes for reading. The exercises at the end of each module are good preparation for the quizzes: try each derivation yourself before checking it against the notes, and change the code to see what breaks.

  • Pre-class quizzes (50%) — a short online quiz on each module, released 48 hours before the session and due 30 minutes before it starts; the lowest two are dropped.
  • Participation (15%) — working through derivations and code in class, and questions and discussion.
  • Final project (35%) — described below; the last two sessions are the project talks.

Grades use an absolute scale with no curve: 95–100 A, 90–94 A−, 85–89 B+, 80–84 B, 75–79 B−, 70–74 C+, 65–69 C, 60–64 C−, 55–59 D, below 55 F.

Final project

One project for each student: take a real classification problem, build several classifiers for it from scratch, and answer a sharp question about why they perform as they do, using the tools of the whole course. Implement at least three classifiers from different modules in NumPy — for example a Gaussian Bayes classifier, a nearest-neighbor rule, and a linear machine, network, or tree — estimate each one's error with cross-validation and a confidence interval, and compare them on the same folds with a paired difference and its standard error (module 09). Then pick one question and answer it with evidence: does performance peak as features are added, does editing the training set help nearest neighbors, when does boosting beat bagging, how far is each classifier from a bound on the Bayes error, such as the Cover–Hart bounds computed from the nearest-neighbor error? The project is written up as a short research paper with reproducible code. Proposal due in week 3, draft in week 5, and the paper and a 5-minute talk in the final two sessions.

Getting set up

The code in the notes needs Python 3 with NumPy (version 2.0 or newer) and SciPy; nothing else. Any recent Python installation, or a free Google Colab notebook in the browser, runs every example. Paste the cells of a module into a notebook in order and run them: every random number comes from a seeded generator, so your output should match the notes, up to rounding in the last digit.

Course policies

Collaboration

Discussing the modules and the project with classmates at the level of ideas is encouraged, but your code and paper must be your own. Helping someone debug an error message is fine; sharing code or results is not.

AI and LLM use

  • Allowed without disclosure: autocomplete-style help (syntax lookups, translating code between libraries), debugging individual error messages, and reading recommendations.
  • Allowed with disclosure: explanations of mathematical concepts you are stuck on, and suggested approaches after you have thought the problem through. A one-line note in your submission is enough.
  • Not allowed: generating quiz answers or large blocks of project code or text with an LLM. Undisclosed LLM-generated work is an academic-integrity violation.

When in doubt, disclose. Disclosure costs you nothing; non-disclosure can cost you the course.