Course Description
Mathematics and AI offers an exploration of the intersection between mathematics and artificial intelligence (AI). Covering state-of-the-art machine learning techniques and their mathematical foundations, this course aims to provide students with both a broad theoretical understanding and practical skills. The syllabus starts with a brief review of the history of AI, and current limits and issues. This is followed by an introduction to statistical learning in a supervised setting and a deeper dive on neural networks and their applications with some references to current mathematical research. The syllabus continues with an overview of unsupervised learning methods and their applications in feature selection. It concludes with student's presentations of their final projects.
Prerequisites: Math 13, Math 20, and Math 22 or advanced placement/ instructor override. Familiarity with at least one programming language. Python preferred.
Instructor: Alice Schwarze (alice.c.schwarze@Dartmouth.edu)
Classes: (2) MWF 2:10 - 3:15 and x-hour Th 1:20 - 2:10
Textbooks and other materials
-
Introduction to Statistical Learning with Applications in Python by Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, Jonathan Taylor
available on the book's website -
Data-Driven Science and Engineering by Steven Brunton and Nathan Kutz
videos available on the book's website - Geometry of Deep Learning by Jong Chul Ye
-
Artificial Intelligence With an Introduction to Machine Learning by Richard Neapolitan and Xia Jiang
available via Dartmouth Libraries (link) - Limits of AI - Theoretical, Practical, Ethical by Klaus Mainzer and Reinghard Kahle
Syllabus
The following is a tentative schedule for the course. Please check back regularly for updates as the term progresses.
| Date | Lecture | Text | Keywords |
|---|---|---|---|
| Thu Jun 20 | No class | ||
| Fri Jun 21 | Artificial intelligence: Ideas and their evolution | Turing test, Dartmouth workshop, expert systems, strong AI, weak AI, artificial general intelligence (AGI), explainable AI (XAI), responsible AI | |
| Mon Jun 24 | Representing knowledge | tables, functions, frames, knowledge graphs, causal networks, directed acyclic graphs, Bayesian networks, Markov random fields (MRFs) | |
| Wed Jun 26 | Formalizing reason | logical programming, propositional logic, first-order logic, fuzzy logic | |
| Thu Jun 27 | Linear regression (or: Why vanilla is the best flavor) | linear regression, gradient descent, mean squared error | |
| Fri Jun 28 | Regression and classification | logistic regression, k-nearest neighbors (KNN), linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), naive Bayes, 1-hot encoding | |
| Mon Jul 1 | Resampling and validation | Crossvalidation, bootstrap, data leakage | |
| Wed Jul 3 | Feature selection (or: Why sometimes less is more) | subset selection, shrinkage, dimension reduction, principal component regression (PCR) | |
| Thu Jul 4 | No class on Independence Day. | ||
| Fri Jul 5 | Regularization | ridge regression, lasso | |
| Mon Jul 8 | Basis functions for regression | step functions, splines, radial basis functions (RBFs), generalized additive models (GAMs) | |
| Wed Jul 10 | Decision trees | regression trees, classification trees, tree ensemble methods, bagging, boosting, random forests, Bayesian additive regression trees (BART) | |
| Thu Jul 11 | Support vector machines | maximum-margin models, hard margin, soft margin, VC theory, nonlinear kernels | |
| Fri Jul 12 | Kernel methods | kernel trick, kernel ridge regression, reproducing kernel Hilbert spaces, representer theorems | |
| Mon Jul 15 | Introduction to neural networks: Perceptron and beyond | perceptron, multi-class perceptron, universal approximation theorems, ReLU, softmax | |
| Wed Jul 17 | Neural network architectures and neural coding | feed-forward neural network, deep learning, encoder, decoder | |
| Thu Jul 18 | Training and regularizing neural networks | backpropagation, stochastic gradient descent, Adam, drop out | |
| Fri Jul 19 | Transfer learning | teacher-student learning, multitask learning | |
| Mon Jul 22 | Forecasting and prediction | Taken's theorem, time-delayed embedding, recurrent neural networks (RNNs), reservoir computing | |
| Wed Jul 24 | Natural language processing | structured prediction, text classification, bag of words, self-supervised learning, word embeddings | |
| Thu Jul 25 | Natural language processing (continued) | long-term short-term memory, attention, transformer, generative pre-trained transformers (GPTs) | |
| Fri Jul 26 | Image generation and more transfer learning | general adversial networks (GANs), contrastive language-image pre-training (CLIP), DALL-E, diffusion | |
| Mon Jul 29 | Project proposals | ||
| Wed Jul 31 | Representation learning | latent space, autoencoders, restricted Boltzmann machines (RBMs) | |
| Thu Aug 1 | No class | ||
| Fri Aug 2 | Principal component analysis | principal component analysis (PCA), matrix factorizations, Hebbian learning | |
| Mon Aug 5 | Project updates | ||
| Wed Aug 7 | The topology of data | self-organizing maps (SOMs), competitive learning, topological data analysis (TDA) | |
| Thu Aug 8 | No class | ||
| Fri Aug 9 | Clustering | k-means clustering, hierarchical clustering | |
| Mon Aug 12 | Project updates | ||
| Wed Aug 14 | Network analysis | centrality measures, community detection, modularity maximization, belief propagation | |
| Thu Aug 15 | No class | ||
| Fri Aug 16 | Matrix completion | Low rank matrix completion, high rank matrix completion, link prediction, recommender systems | |
| Mon Aug 19 | Final project presentations | ||
| Wed Aug 21 | Final project presentations |
General Information
In person lectures and office hours
Lectures and office hours will generally be held in person. From time to time, it may be announced in class or on CANVAS that lectures or office hours will be conducted via zoom. Individual appointments with your instructor may be held remotely via zoom, especially those made for late afternoon.
Grading
The course grade will be based upon on
- weekly in-class quizzes (total 50 points),
- weekly homework problems (total 100 points),
- class participation, which may involve presenting a blog post or research paper in class, (total 100 points),
- and a final coding project to be completed by the final week of classes (total 250 points). This project can be a group project.
Academic Honor Principle
For quizzes and all other assessments, Dartmouth's Academic Honor Principle will be upheld. Please be advised of especially
- Quizzes: Giving and/or receiving assistance during an examination violates the Academic Honor Principle.
- Homework and course projects: Collaboration is both permitted and encouraged, but it is a violation of the honor code for someone to provide the answers for you.
Student Religious Observances
Some students may wish to take part in religious observances that fall during this academic term. Should you have a religious observance that conflicts with your participation in the course, please come speak with your instructor before the end of the second week of the term to discuss appropriate accommodations.