CS 429/529 Machine Learning

Course Information

Class time: 11:00 AM - 12:15 PM

Classroom: Dane Smith Hall 229

Prerequisites: Fluent in at least one object oriented language and one numerical computing environment or library

Preferred: Background in Statistics and Linear Algebra

Piazza link: https://piazza.com/unm/spring2023/cs429529

UNM Canvas: https://canvas.unm.edu/

Appointments: https://calendly.com/trilce-estrada

Class gallery: 2020

Course Description

As intelligent systems become pervasive and data production grows at a rate never seen before, the ability to understand, analyze, and automatically learn from data is becoming a crucial technical skill. In this course we cover principles and practice of Machine Learning, that is, systems that improve performance on specific tasks through experience. The course balances theory and practice to provide students with the fundamentals of statistical learning as well as with hands-on experience building predictive systems. Specifically, the course includes a set of projects, where students are required to perform data cleaning and preprocessing, envision a suitable learning approach, and evaluate their solution.

Core Topics

Overview and taxonomy of machine learning: supervised, unsupervised, semi-supervised, and reinforcement learning. Applications and case studies. Inductive learning and inductive bias. Decision trees. Overfitting, underfitting, and model selection. Principles of bayesian learning. Logistic and linear regression. Dimensionality reduction. Support vector machines and kernels. Neural networks. Introduction to modern neural architectures. Clustering. Ensembles. Bias-variance tradeoff.

Course Objectives

At the end of this course, the student will become familiar with fundamental concepts of Machine Learning and modern analytics; will be competent in recognizing challenges faced by ML approaches and applications, and will be able to propose working solutions for them. The student will be able to discuss how Machine Learning impacts business intelligence, scientific discovery, and our day-to-day life.

Text Books



  • Deep Learning by Ian Goodfellow, Yoshua Begio, Aaron Courville. ISBN-10: 0262035618 | ISBN-13: 978-0262035613 (optional)
  • Machine Learning by Tom Mitchell. ISBN-10: 0070428077 | ISBN-13: 978-0070428072 (optional)
  • Pattern Recognition and Machine Learning by Christopher Bishop. ISBN-10: 0387310738 | ISBN-13: 978-0387310732 (optional)
  • Coursework

    Participation

    Participation is the barometer of the class. Based on it I can determine if the pace of the course is too fast or too slow, it helps me to spot pitfalls and misconceptions, and it helps you to reinforce the material you learned.

    The student can expect to have simple exercises frequently. Some of these daily assignments will be done in groups specified by the instructor and they will account for the participation grade of the course. Make up assignments will be allowed only if the instructor or TA were informed of a documented absence before the quiz took place.

    Participation accounts for 15% of your final grade and won't be given for granted. You are required to participate either in class or electronically (through Piazza).

    Quizzes

    Readings in this syllabus are mandatory, you can expect short individual quizzes every week. Those quizzes will be graded and they account for the bulk of your final grade. Quizzes are done first thing in the morning if you are late 5 minutes that may mean that you missed the quiz. If you have an excused absence and missed a quiz, you can make up for it by writing a 2-page summary of the reading. The assigned grade to that assignment will be the average of your current quiz grade. If your absence is unexcused, you cannot make up that quiz.

    Quizzes are open notes but only handwritten notes are allowed.

    Students in the 529 section will be graded with all -1 (lowest quiz), students in the 429 section will be graded on all -2 (lowest quiz grades)

    Projects

    Projects and homework reinforce what you learned in class by materializing abstract concepts into practical problems. Every project usually consists of a programming assignment. Tentative schedule is as follows:

    Project 1. release date Jan 30, due date Feb 20 (Structured data)

    Project 2. release date Feb 25, due date Mar 24 (Text)

    project 3. release date Mar 26, due date Apr 16 (Audio/Images)

    project 4. release date Apr 16, due date May 12 (AI generated art)

    You can discuss projects with other classmates but all the code have to be written by you. Any student suspected of plagiarizing code will be prosecuted according to the University guidelines.

    Project reports and code (No datasets) have to be uploaded to Canvas, solution key has to be uploaded through Kaggle. Late projects will be accepted only within the following policy: Every student has 4 free days to be used at their own discretion across the multiple projects. Once a student has used all of his/her free days, no other late project will be accepted

    Projects can be made in teams as follows: 2 graduate students, OR 3 to 4 undergraduate students

    Grading

    Grades will be based on your earned points, following this grade scale. You need to get the specified number of points or more to obtain the grade from the same column. Scores will be rounded to the closest integer value.

    Incomplete can be assigned only for a documented medical reason. Change of grade to CR/NC after the semester deadline will be granted ONLY under special, documented extenuating circumstances.

    A (95), A- (90), B+ (87), B (83), B- (80) C+ (77), C (73), C- (70), D+ (67), D (63), D- (60), F (le 60)

    Participation20%

    Quizzes40%

    Projects40%

    Policies

    Academic Honesty

    Unless otherwise specified, you must write/code your own homework assignments. You cannot use the web to find answers to any assignment. If you do not have time to complete an assignment, it is better to submit your partial solutions than to get answers from someone else. Cheating students will be prosecuted according to University guidelines. Students should get acquainted with their rights and responsibilities as explained in the Student Code of Conduct

    Dean of Student's Academic Integrity/Honesty
    Student code of conduct

    Any and all acts of plagiarism will result in an immediate dismissal from the course and an official report to the dean of students.

    Instances of plagiarism include, but are not limited to: downloading code and snippets from the Internet without explicit permission from the instructor and/or without proper acknowledgment, citation, or license use; using code from a classmate or any other past or present student; quoting text directly or slightly paraphrasing from a source without proper reference; any other act of copying material and trying to make it look like it is yours.

    Note that dismissal from the class means that the student will be dropped with an F from the course.

    The best way of avoiding plagiarism is to start your assignments early. Whenever you feel like you cannot keep up with the course material, your instructor is happy to find a way to help you. Make an appointment or come to office hours, but DO NOT plagiarize; it is not worth it!.

    Attendance

    Attendance to class is expected (read mandatory) and note taking encouraged. Important information (about exams, assignments, projects, policies) may be communicated only during lecture time. We may also cover additional material (not available in the book or in slides) during the lecture.

    If you miss a lecture, you should find what material was covered and if any announcement was made. If you have unexcused absences, this may result in participation points being deducted. Excused absences include sickness, attending conferences, job interviews, and similar. Even if your absence is excused, it is your responsibility to find out what material you missed. The professor is happy to answer specific questions regarding the lecture, but cannot go through all of the missed material on a one-to-one basis.

    Excused absences have to be notified to the TA and instructor (through a piazza private post) at least 24hrs in advance, sickness has to be justified with a doctor's note

    Communication

    In order to facilitate interaction between students and to promote a broader participation, I created a Piazza group. Use the Piazza public group to ask general questions about homework, exams, projects, and lectures. You can also paste small snippets of code to clarify an idea. Students are encouraged to answer each others questions. Recall that your thoughtful participation in this forum accounts through your final grade. Use Piazza private posts to ask for excused absences and other personal matters. Always cc the class TA in those cases. Piazza is a discussion forum for the class and members are expected to conduct themselves with respect by posting comments and replies only in the context of the course.

    Feedback

    I value student's opinions regarding the course and I will take them into consideration to make this course as exciting and engaging as possible. Thus, through the semester I will ask students formal and informal feedback. Formal feedback includes short surveys on my teaching effectiveness, preferred teaching methods, and the pace of the class. Informal feedback will be in the form of polls or in-class questions regarding learning preferences. You can also leave anonymous feedback in the form of a note in my departmental mailbox, under my office door, or using this form. Remember that it is in the best interest of the class if you bring up to my attention if something is not working properly (e.g the pace of the class is too slow, the projects are boring, my teaching style is not effective) so that I can make the corrective steps.

    ADA

    In accordance with University Policy 2310 and the Americans with Disabilities Act (ADA), academic accommodations may be made for any student who notifies the instructor of the need for an accommodation. If you have a disability, either permanent or temporary, contact Accessibility Resource Center at 277-3506 for additional information.

    Schedule

    Topic

    Readings

    Introduction Mitchell:CH1
    CART Mitchell:CH3 QUIZ
    Probability review Goodfellow:CH3 QUIZ optional Bishop:CH1,CH2
    Bayesian vs MLE Approach Tutorial on MLE estimation QUIZ
    optional Bishop:CH1,CH2
    Naive Bayes Tom Mitchell's book chapter QUIZ
    Logistic Regression Tom Mitchell's book chapter QUIZ
    optional Bishop:CH3
    Linear Regression Simple Linear Regression QUIZ
    Bias-Variance tradeoff, Ensembles Bishop:CH14
    Support Vector Machines A users guide to SVMs QUIZ
    Perceptrons and Neural Networks Goodfellow: CH6 QUIZ
    Regularization, activation and loss functions Goodfellow: CH7 QUIZ
    Optimization strategies and dealing with overfitting Goodfellow: CH8 QUIZ
    Convolutional Neural Networks Goodfellow: CH9 QUIZ
    DL for Data representation, Autoencoders Goodfellow:CH14
    Dimensionality reduction, PCA, SVD PCA tutorial by Lindsay Smith QUIZ
    Optional Bishop: CH12
    Transfer Learning
    Sequence modeling: RNNs and LSTMs Goodfellow:CH10
    Generative Adversarial Networks Goodfellow:CH20
    Bayesian Networks: inference and construction Bishop: CH8 QUIZ
    Instance-Based Learning Mitchell:CH8 QUIZ
    Unsupervised Learning: Clustering and Gaussian mixture models Bishop: CH9 QUIZ

    Please refer to Canvas > Course Information, for updated schedule, readings, and resources