ECE 590: Generative AI: Foundations, Applications, and Safety (Spring 2025)

---

Instructor

Neil Gong
neil.gong@duke.edu

Teaching Assistant

Yuqi Jia
yuqi.jia@duke.edu

Lectures

Time: MoWe 3:05PM - 4:20PM.
Location: Hudson Hall 115A

Office Hours

Time: Tursday 9:00AM - 10:00AM.
Location: 413 Wilkinson Building

Tentative Schedule

01/08    Course overview (Slides)

01/13    Transformer (Slides)

• Attention Is All You Need

01/15    Transformer (Slides)

01/20    Holiday

01/22    Representation learning (Slides)

• Learning Transferable Visual Models From Natural Language Supervision
• Auto-Encoding Variational Bayes

01/27    Image generation (Slides)

• Denoising Diffusion Probabilistic Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Optional: Classifier-Free Diffusion Guidance
• Optional: Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction
• Optional: Imagic: Text-Based Real Image Editing with Diffusion Models

01/29    Safety guardrails for image generation models (Slides)

• Erasing Concepts from Diffusion Models
• Optional: SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models
• Optional: Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models

02/03    Jailbreaking safety guardrails of image generation models (Slides)

• SneakyPrompt: Jailbreaking Text-to-image Generative Models
• Optional: Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?
• Speakers: Chengyang Zhou and Michael (Zeyu) Li

02/05    AI-generated image detection (Slides)

• Towards Universal Fake Image Detectors that Generalize Across Generative Models
• HiDDeN: Hiding Data With Deep Networks
• Optional: Leveraging Frequency Analysis for Deep Fake Image Recognition
• Optional: Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust
• Optional: The Stable Signature: Rooting Watermarks in Latent Diffusion Models
• Optional: Watermark-based Attribution of AI-Generated Content
• Speakers: Hung Anh Vu, Steven Seiden, and Zini Yang

02/10    Robustness of AI-generated image detectors (Slides)

• Towards Deep Learning Models Resistant to Adversarial Attacks
• Evading Watermark based Detection of AI-Generated Content
• Optional: A Transfer Attack to Image Watermarks
• Speakers: Anika Mitra and Adam Kosinski

02/12    Robust AI-generated image detectors (Slides)

• Certifiably Robust Image Watermark
• Optional: Certified Adversarial Robustness via Randomized Smoothing

02/17    LLM pre-training and alignment

• Fine-Tuning Language Models from Human Preferences
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• Optional: Multi-modal LLM pre-training and alignment
• Optional: BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• Speakers: Peter Yang, Mobasserul Haque, and Dhaval Potdar

02/19    LLM agent

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• ReAct: Synergizing Reasoning and Acting in Language Models
• Speakers: Qinsi Wang and Ming-Yu Chung

02/24    Prompt injection attacks

• Formalizing and Benchmarking Prompt Injection Attacks and Defenses
• Speakers: Jason Wang and Reachal Wang

02/26    Defenses against prompt injection attacks (Slides)

• StruQ: Defending Against Prompt Injection with Structured Queries
• Aligning LLMs to Be Robust Against Prompt Injection
• Optional: Jatmo: Prompt Injection Defense by Task-Specific Finetuning
• Guest speaker: Sizhe Chen, UC Berkeley

03/03    Jailbreak attacks to LLM

• Universal and Transferable Adversarial Attacks on Aligned Language Models
• Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
• Optional: Jailbreaking Black Box Large Language Models in Twenty Queries
• Optional: AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs
• Speakers: Zhendong Zhang, Vivian Zhang, and Ming Yin

03/05    Defenses against jailbreak attacks

• Safety Alignment Should Be Made More Than Just a Few Tokens Deep
• Optional: GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis
• Optional: SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding
• Speakers: Yuchen Jiang, Zedian Shao, and Yangchenchen Jin

03/10    Spring recess

03/12    Spring recess

03/17    AI-generated text detection: passive detectors (Slides)

• DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

03/19    AI-generated text detection: watermarks (Slides)

• A Watermark for Large Language Models
• Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding
• Optional: Scalable watermarking for identifying large language model outputs

03/24    Robustness of AI-generated text detectors (Slides)

• Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
• Optional: Are AI-Generated Text Detectors Robust to Adversarial Perturbations?
• Speakers: Austin Phillips, Osama Ahmed, and Ryan Devries

03/26    Hallucination

• Towards Mitigating Hallucination in Large Language Models via Self-Reflection
• Visual Hallucinations of Multi-modal Large Language Models
• Speakers: Yanming Xiu

03/31    Data-use auditing: passive methods

• Membership Inference Attacks against Machine Learning Models
• Detecting Pretraining Data from Large Language Models
• Optional: Membership Inference Attacks From First Principles

04/02    Data-use auditing: proactive methods

• A General Framework for Data-Use Auditing of ML Models
• Optional: Radioactive data: tracing through training

04/07    Audio generation and safety issues

• Audioldm: Text-to-audio generation with latent diffusion models
• Optional: Proactive Detection of Voice Cloning with Localized Watermarking
• Speakers: Hao-Lun Hsu, Jiwoo Kim, and Naman Saxena

04/09    Video generation and safety issues

• Make-a-video: Text-to-video generation without text-video data
• Make it move: controllable image-to-video generation with text descriptions
• Optional: LVMark: Robust Watermark for latent video diffusion models
• Speakers: Hengfan Zhang, Yupu Wang, and Haocheng Ni

04/14    Project presentation

04/16    Project presentation

Prerequisite

ECE 580 or 687D or Computer Science 371 or graduate standing.

Course Description

Generative AI is revolutionizing content creation by enabling machines to generate text, images, videos, music, and even code. In this course, we will discuss foundations, applications, and safety and security of generative AI.

Class Format

The class is structured around paper reading, lectures, discussions, and projects. Each lecture will focus on a specific topic, with students expected to read the suggested papers and submit their comments to a designated email address by the end of the day before the lecture. Students will be required to lead a lecture on a chosen topic, complete a class project, present their project, and write a project report. Groups of up to three students can be formed for both the lecture and the class project.

Deadlines

Reading assignments

• Sunday and Tuesday 11:59pm. Send comments to ecegenerativeai@gmail.com. Please send your comments to all papers in a single email thread.

Choosing a topic for lecture

• A group sends three preferred dates to ecegenerativeai@gmail.com by 11:59pm, 01/25.

Class project

• 02/01: project proposal due.
• 03/15: milestone report due.
• 04/14, 04/16: project presentation.
• 04/27: final project report due.

Grading Policy

50% project
25% reading assignment
10% class participation
15% class presentation