Danna Gurari

---

Date	Topics (lecture slides hyperlinked)	Assigned Readings (due prior to class)	Assignments (due prior to class and posted on Canvas)
Mon, Aug 26	Course Introduction
Wed, Aug 28	Background: Evolution and Fundamentals of Modern Computer Vision
Mon, Sep 2	No Class (Labor Day)
Wed, Sep 4	Object Recognition: Dataset Challenges and CNNs
Mon, Sep 9	Object Recognition: CNNs
Wed, Sep 11	Object Recognition: Transformers
Mon, Sep 16	Scene, Fine-Grained, and Attribute Classification
Wed, Sep 18	Semantic Segmentation
Mon, Sep 23	Object Detection
Wed, Sep 25	Instance Segmentation
Mon, Sep 30	Panoptic Segmentation
Wed, Oct 2	Object Tracking
Mon, Oct 7	Vision and Language: Image Captioning and Visual Question Answering
Wed, Oct 9	Foundation Models and Prompts
Mon, Oct 14	TBD
Wed, Oct 16	TBD
Mon, Oct 21	TBD
Wed, Oct 23	TBD
Mon, Oct 28	TBD
Wed, Oct 30	TBD
Mon, Nov 4	TBD
Wed, Nov 6	TBD
Mon, Nov 11	TBD
Wed, Nov 13	TBD
Mon, Nov 18	TBD
Wed, Nov 20	TBD
Mon, Nov 25	No Class (Fall Break)
Wed, Nov 27	No Class (Fall Break)
Mon, Dec 2	Efficient Computer Vision
Wed, Dec 4	Responsible Computer Vision
Mon, Dec 9	Responsible Computer Vision and Course Summary
Wed, Dec 11	Final Project Presentations
Wed, Dec 18	No Class (Final Exam Week)

Overview

The goal of the student-led lectures is for you to develop your skills in analyzing and presenting cutting-edge research in computer vision. Each student team will lead one lecture on one computer vision topic.

This effort will constitute 30% of your total class grade. Your grade for this effort will be calculated as follows:

Assignment	% of Final Project Grade
Candidate paper proposal	10%
Presentation review	40%
Lecture	50%

Candidate Paper Proposal

For the paper proposal, you and your teammate you should:

• Identify 4-6 candidate papers that were recently published at a premiere computer vision conference (e.g., CVPR, ICCV, ECCV) on your topic. Two of these papers will be assigned as readings, with one needing to be about a specific dataset challenge (optional reading) and the other about a computer vision model (required reading).
• Select a 30 minute time slot that works for all group members to meet with the instructor to discuss which papers to cover during the presentation. Available time slots will be posted via Canvas. This meeting must be at least two weeks prior to your first lecture.
• Email the candidate papers to the instructor at least 48 hours in advance of the meeting.

Presentation Review

The presentation review is a chance to receive feedback on the slide deck for your lecture and resolve any open questions. Expected content for each lecture is outlined in the next section. For this presentation review, you will be expected to:

• Select a 30 minute time slot that works for all group members to meet with the instructor to review the lecture slides for both lectures. Available time slots will be posted via Canvas. This meeting must be at least one week prior to your first lecture.
• Email the lecture slides to the instructor at least 24 hours in advance of the meeting.

Lecture

Each lecture should take about 50 minutes and consist of two parts. The first portion should: (i) define the problem, (ii) motivate the practical importance of solving this problem with a computer vision solution (i.e., applications that can/do benefit society), (iii) describe 1-2 datasets used to track progress on this problem, and (iv) describes metric(s) used to evaluate the performance of computer vision models. The second portion should introduce at least one computer vision model, covering: (i) its claimed novelty, (ii) mechanisms used to validate the claims, and (iii) open technical questions/problems. Then, the lecture will conclude with a facilitated class discussion about the lecture topic, organized by the instructor around the questions and discussion points submitted by all students. Students can incorporate materials from outside sources in their presentations (for example, content from the paper’s authors or slides), but proper credit MUST be given.

Overview

The goal of this project is for you to develop your skills in conducting and communicating original work involving deep learning. This is an opportunity for you to enhance your expertise on a topic you feel passionate about, as it will be a self-designed project. The only requirement is that your project includes a deep learning component with analysis.

Your final project will constitute 30% of your total class grade. Your project grade will be calculated as follows:

Assignment	% of Final Project Grade
Project proposal
Project outline	20%
Final project presentation	20%
Peer evaluation	10%
Final project report	50%

---

Project Proposal

Project Outline

The project outline should map out the entire project. You are strongly encouraged to work with a partner but can work alone with approval from the teaching assistant.

You will be expected to submit:

• A detailed project outline as a PDF that is 3-4 pages long (excluding references).

When choosing your topic, general guidelines are to:

• Choose a problem you have an idea for how to solve
• Choose a problem someone else cares about
• Choose a problem that is not yet solved (review current literature!)
• Choose a problem that you can objectively evaluate by tying it to a task

For the PDF submission, please create this using a latex template from a mainstream conference or journal, such as ECCV. A great latex paper editor is Overleaf. The outline should include each of the following:

• Title for your project
• [Section 1] Introduction
  • Paragraph 1: Explain the motivation for your work; e.g., Why anyone should care? What are the desired benefits?
  • Paragraph 2: Explain why existing solutions work is inadequate for the motivated problem; e.g., Is there a gap in the literature? Is there a weakness in existing approaches?
  • Paragraph 3: Explain what you are proposing, what is new about your idea, and why you believe this solution will be better than previous solutions; e.g., Are you asking a new question, offering a greater understanding of a research problem, establishing a new methodology to solve a problem, building a new software tool, or offering greater understanding about existing methods/tools?
• [Section 2] Related Work
  • Identify 2-4 related topics. Then, for each topic, cite 2-4 related papers (must include the bibliography). Finally, for each cluster of related works, give a 1-2 sentence explanation describing the key difference(s) of your proposed idea to the cluster of prior works. One way to format each topic is as follows:
    
    Topic:
    Reference 1
    Reference 2
    Reference 3
    Reference 4
    
    Our work is different from these works because…
• [Section 3] Methods – describe the implementation of your proposed idea (e.g., features, algorithm(s), training overview) so that:
  • A reader could reproduce your set-up
  • A reader could understand why you made your design decisions
• [Section 4] Experimental Design – describe 1-2 experiments or analyses you plan to conduct in order to demonstrate/validate the target contribution(s) of your work. Your description should be detailed enough so that a reader could reproduce it. Your description should include the following for each experiment:
  • Main purpose: 1-3 sentence high level explanation
  • Evaluation Metric(s): which ones will you use and why?
• Bibliography: this must be formatted correctly.

Please note that your proposed project is not a binding contract. You will continue to update and improve it as you learn more from your readings and/or feedback.

Final Project Presentation

The final project presentation submission will be a:

• PDF document of a poster

It should convey the following:

• Motivate the problem your work is designed to solve.
• (Very) briefly explain what other solutions are available and why they are not suitable.
• Demo your idea, approach, and key design decisions.
• Highlight key findings from your experiments and offer insights into what your work has taught us. Focus on finding 1-3 punchlines that explain why your work is exciting/valuable.

Your poster should provide a concise framework for you to communicate about your project to a lay audience. In other words, your mom, dad, friend, or a potential employer should be able to see it and understand what you did and why what you did is valuable. If you would like to work from a template, the following are suggested by popular conferences for computer vision material: CVPR, NeurIPS, and ECCV.

On the last day of class, all students will present their work as well as learn about other students’ works. When pitching your work, you will have 1-2 minutes to pitch to the class the material in the poster, which will be projected for everyone to see.

Peer Evaluation

You will evaluate the poster presentation from every person in the course at the link shared by the instructor. The evaluations will be done during the day of the last class meeting. The evaluations that you do for other students’ projects will not affect your own grade, except that you will be penalized if you do not complete an evaluation (following the requirements) for every person (excluding your own).

Final Project Report

For the final project submission, you should submit a final report that is at least 7 pages long (excluding references). It should include each of the following:

• Title
• Abstract – one paragraph summary of your paper describing the motivation, problem, conducted experiments, and experimental findings
• [Section 1] Introduction (improve upon the material from your project outline)
• [Section 2] Related Work (improve upon the material from your project outline; if you have not already, you should remove the bulleted structure you used in the initial proposal and instead have a paragraph form)
• [Section 3] Methods (improve upon the material from your project outline)
• [Section 4] Experiments
  • Improve upon the material from your project outline
  • Report the experimental results, what general trends are observed, and insights/speculations into why your results may be turning out the way they are.
  • Also include at least one paragraph explaining what questions are not fully answered by your experiments and natural next steps for this direction of research.
• [Section 5] Conclusions
  • Summarize in one paragraph what is the main take-away point from your work.
  • Add a final paragraph discussing any potential ethical implications of your project (e.g., fairness, accountability, transparency, privacy, social impact, etc).
• [Section 7] Bibliography