Dark Mode
Light Mode

Instructor

Danna Gurari (first name pronounced “dah-nah”, similar to “Donna”; last name rhymes with Ferrari)

If easier, feel free to just call me Dr. G.

Course Support

Teaching Assistants

– Everley Tseng: Yu-Yun.Tseng@colorado.edu

– Nick Cooper: Nicholas.Cooper-1@colorado.edu

Grader

– Sruthi Sampath Kumar

Syllabus

You can find more details about the course in the syllabus.

Course Platforms

All course materials and communications will take place in Canvas and Piazza (also linked to in Canvas).

Textbooks

The recommended readings for this course are freely available online and are as follows:

Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville

Deep Learning for NLP and Speech Recognition by Uday Kamath, John Liu, and James Whitaker (can be downloaded for free when connected to the CU network or VPN)

Office Hours

Below are the available time slots for office hours. To attend, please sign-up on the spreadsheet shared in Canvas.

– Mondays: 11am-12pm

– Tuesdays: 8-9am

– Wednesdays: 11am-12pm

– Thursdays: 9:30-11am

– Fridays at 9:30-11am

Acknowledgements

We are fortunate to receive the generous support of a Google Cloud Education Grant to provide us with state-of-the-art computing resources.

DateTopics
(lecture slides hyperlinked)
Assigned Readings
(due prior to class)
Assignments
(due prior to class and posted on Canvas)
Wed, Jan 17Course Introduction
Mon, Jan 22Artificial NeuronsCh. 1 of Goodfellow book; Ch. 1-2.5 and 4-4.2 of Kamath book
Wed, Jan 24Feedforward Neural NetworksCh. 6-6.4 of Goodfellow book
Mon, Jan 29Neural Network Training: Bottom-Up PerspectiveCh. 4-5.1, 5.9-5.10, and 6.5-6.6 of Goodfellow book; Ch. 4.3-4.42 of Kamath bookProblem Set 1
Wed, Jan 31Neural Network Training: Top-Down PerspectiveCh. 5.2-5.6 of Goodfellow book
Mon, Feb 5Neural Network Training: ImplementationCh. 8-8.6 of Goodfellow book; Ch. 4.4.3 of Kamath book
Wed, Feb 7RegularizationCh. 7 of Goodfellow book; Ch. 4.5 of Kamath bookLab Assignment 1
Mon, Feb 12Convolutional Neural Networks Ch. 9-9.5 and 9.10-9.11 of Goodfellow book; (Optional) Ch. 6-6.3 of Kamath book
Wed, Feb 14Introduction to Computer Vision, Image ClassificationCh. 12.2 of Goodfellow book; Ch. 6.5-6.6 of Kamath bookProblem Set 2
Mon, Feb 19Image Classification Continued, Representation Learning, and Fine-Tuning (Guest Lecturer: Everley Tseng)Ch. 12-12.1 of Goodfellow book
Wed, Feb 21Object DetectionCh. 9.6-9.7 of Goodfellow book; Faster R-CNN
Mon, Feb 26Image-to-Image Translation, Semantic SegmentationFully Convolutional Networks for Semantic SegmentationLab Assignment 2
Wed, Feb 28Recurrent Neural NetworksCh. 10 of Goodfellow book; (Optional) Ch. 7.1-7.4 of Kamath book
Mon, Mar 4Introduction to Natural Language Processing, Neural Word EmbeddingsCh. 12.4 of Goodfellow book; Ch. 5.1-5.5 of Kamath bookProblem Set 3
Wed, Mar 6Introduction to AttentionCh. 7.5 and 9-9.2.5 of Kamath book
Mon, Mar 11Transformers OverviewCh. 9.2.6-9.2.10 of Kamath book
Wed, Mar 13Popular TransformersBERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLab Assignment 3
Mon, Mar 18Foundation Models and Prompts
Wed, Mar 20Multimodal LearningProblem Set 4
Mon, Mar 25No Class (Spring Break)
Mon, Mar 27No Class (Spring Break)
Mon, Apr 1Practical Systems-Level Development Challenges
Wed, Apr 3Self-Supervised Learning
Mon, Apr 8Model Compression
Wed, Apr 10Efficient LearningFinal Project Outline
Mon, Apr 15Responsible Deep Learning
Wed, Apr 17Responsible Deep Learning
Mon, Apr 22Deep Learning in Industry (Guest: Dr. Mehrnoosh Sameki, Responsible AI Tools Lead at Microsoft)
Wed, Apr 24Deep Learning in Industry (Guest: Dr. Suyog Jain, Research Scientist at Meta)
Mon, Apr 29Course SummaryFinal Project Presentation
Wed, May 1Final Project PresentationPeer Evaluation
Tue, May 7No Class (Final Exam Week)Final Project Report

Overview

The goal of this project is for you to develop your skills in conducting and communicating original work involving deep learning. This is an opportunity for you to enhance your expertise on a topic you feel passionate about, as it will be a self-designed project. The only requirement is that your project includes a deep learning component with analysis.

Your final project will constitute 30% of your total class grade. Your project grade will be calculated as follows:

Assignment% of Final Project Grade
Project outline20% 
Final project presentation20% 
Peer evaluation10% 
Final project report50% 

Project Outline

The project outline should map out the entire project. You are strongly encouraged to work with a partner but can work alone with approval from the teaching assistant.

You will be expected to submit:

  • A detailed project outline as a PDF that is 3-4 pages long (excluding references). Only one member of the group is required to submit the PDF, but both group members are welcome to submit the PDF to avoid any potential submission issues.

When choosing your topic, general guidelines are to:

  • Choose a problem you have an idea for how to solve
  • Choose a problem someone else cares about
  • Choose a problem that is not yet solved (review current literature!)
  • Choose a problem that you can objectively evaluate by tying it to a task

For the PDF submission, please create this using a latex template from a mainstream conference or journal, such as ECCV. A great latex paper editor is Overleaf. The outline should include each of the following:

  • Title for your project
  • [Section 1] Introduction
    • Paragraph 1: Explain the motivation for your work; e.g., Why anyone should care? What are the desired benefits?
    • Paragraph 2: Explain why existing solutions work is inadequate for the motivated problem; e.g., Is there a gap in the literature? Is there a weakness in existing approaches?
    • Paragraph 3: Explain what you are proposing, what is new about your idea, and why you believe this solution will be better than previous solutions; e.g., Are you asking a new question, offering a greater understanding of a research problem, establishing a new methodology to solve a problem, building a new software tool, or offering greater understanding about existing methods/tools?
  • [Section 2] Related Work
    • Identify 2-4 related topics. Then, for each topic, cite 2-4 related papers (must include the bibliography). Finally, for each cluster of related works, give a 1-2 sentence explanation describing the key difference(s) of your proposed idea to the cluster of prior works. One way to format each topic is as follows:

      Topic:
      Reference 1
      Reference 2
      Reference 3
      Reference 4

      Our work is different from these works because…
  • [Section 3] Methods – describe the implementation of your proposed idea (e.g., features, algorithm(s), training overview) so that:
    • A reader could reproduce your set-up
    • A reader could understand why you made your design decisions
  • [Section 4] Experiments – describe 1-2 experiments or analyses you plan to conduct in order to demonstrate/validate the target contribution(s) of your work. Your description should be detailed enough so that a reader could reproduce it. Your description should include the following for each experiment:
    • Main purpose: 1-3 sentence high level explanation
    • Evaluation Metric(s): which ones will you use and why?
  • Bibliography: this must be formatted correctly.

Please note that your proposed project is not a binding contract. You will continue to update and improve it as you learn more from your readings and/or feedback.

Final Project Presentation

The final project presentation should be submitted as a zip file which include two parts:

  • Text document indicating a URL to a 4 minute recorded video
  • PDF document of a poster

Both deliverables should convey the following:

  • Motivate the problem your work is designed to solve.
  • (Very) briefly explain what other solutions are available and why they are not suitable.
  • Demo your idea, approach, and key design decisions.
  • Highlight key findings from your experiments and offer insights into what your work has taught us. Focus on finding 1-3 punchlines that explain why your work is exciting/valuable.

Please design your video for an audience who has not taken the class (i.e., lay audience). In other words, your mom, dad, friend, or a potential employer should be able to watch it and understand what you did and why what you did is valuable. You must submit a URL (either public or unlisted) that links to your video in your Canvas submission. To support inclusion of your presentation in our online forum, provide video links to either YouTube or Vimeo. Please verify viewing permissions are set properly before submission or your grade will be penalized.

Your poster should provide a concise framework for you to communicate about your project to a lay audience. It can be as simple as a summary of the content presented in your video in one PDF. If you would like to work from a template, the following are suggested by popular conferences for computer vision material: CVPRNeurIPS, and ECCV.

On the last day of class, we will gather in an online forum so that students can present their work and learn about other students’ works. We will use the online platform, Gather, where every team’s video and poster will be posted. During this class session, students will be allowed to both stay in their designated spaces to present their poster/videos while answering any questions as well as to visit other students’ designated locations to learn about their work by reviewing their posters and/or videos.

Peer Evaluation

You will evaluate the presentation for one third of the projects in the course at the link shared by the course staff. The evaluations will be done during the day of the last class meeting. The evaluations that you do for other students’ projects will not affect your own grade, except that you will be penalized if you do not complete an evaluation (following the requirements) for each assigned group. You do not need to complete an evaluation for your own project.

Final Project Report

  • Title
  • Abstract – one paragraph summary of your paper describing the motivation, problem, conducted experiments, and experimental findings
  • [Section 1] Introduction (as needed, improve upon the material from your project outline)
  • [Section 2] Related Work (as needed, improve upon the material from your project outline; if you have not already, you should remove the bulleted structure you used in the initial proposal and instead have a paragraph form)
  • [Section 3] Methods (as needed, improve upon the material from your project outline)
  • [Section 4] Experiments
    • As needed, improve upon the material from your project outline
    • Report the experimental results, what general trends are observed, and insights/speculations into why your results may be turning out the way they are.
    • Also include at least one paragraph explaining what questions are not fully answered by your experiments and natural next steps for this direction of research.
  • [Section 5] Conclusions
    • Summarize in one paragraph what is the main take-away point from your work.
    • Add a final paragraph discussing any potential ethical implications of your project (e.g., fairness, accountability, transparency, privacy, social impact, etc).
  • [Section 6] Bibliography

This document is a revision of your project outline. Consequently you only need to expand on the content submitted from the project outline.