OVERVIEW:

- Introduction, Feature extraction - edge, texture, motion (optical flow), others

- Image Segmentation

- Registration

- Tracking & Sequential Detection - Kalman filter & Extensions, Particle filter, Change Detection

- Object Recognition (Image Classification)

- Shape Analysis - Representations, Distances, Segmentation, Tracking

- 3D Scene Reconstruction

   - Image formation models (camera models)

   - Structure from Motion

   - Structure from Stereo (if time permits)

   - Shape representations in 3D (if time permits)

 

DETAILED SYLLABUS:  I intend to give basics of each problem, not necessarily state of the art.  The goal of the class will be to also learn the underlying techniques from Signal Processing / Math. I will keep putting pointers to latest research for those interested.

 

Introduction & Feature extraction                                                                3-4 classes                                                      

Edge Detection                        

Texture                                    

Motion - Optical Flow

 

Segmentation                                                                                                 4 classes

Histogram Mode Finding

Clustering - K-means, Expectation Maximization, Clustering on Intensity and Location

Energy Minimization for Contour Estimation, "Snakes"

Region growing

Contour Tracing after edge filtering

 

Registration                                                                                                   3 classes

             Hough Transform

             Learning Affine parameters as a Least Mean Square Error estimation problem, Weighted LS, Recursive LS

 Learning Scaled Euclidean parameters - Landmark Shape Analysis

"Deformotion": Joint Segmentation and Registration

 

Recognition - Face/Object/Activity                                                                 3 classes

            Likelihood Ratio Testing

            PCA, LDA, Subspace LDA,    Kernel methods, Discriminant EM

            Support Vector Machines

            PCNSA

 

Tracking and Change Detection                                                                   3-4 classes

            Kalman Filter

            Extended Kalman Filter, Gaussian Sum Filter

            Particle Filter

            Change Detection

 

Shape Analysis - Landmark shape & Continuous curves                           4-5 classes, maybe more

            Landmark Shape - Procrustes distance, tangent coordinates

            Classification & Registration

            Continuous curves:

Finite dim - Fourier descriptors & B-splines

                        Infinite dim - Level Set Representation

                        Segmentation using level sets - edge & region based

            Shape tracking

           

3-D Reconstruction                                                                                        3 classes

            Image Formation models (Camera models)

            Structure from Motion

            Stereo

 

One or two guest lectures by other faculty working on Image Analysis

 

EVALUATION: All 3 components below will carry almost equal weight. We will decide after the first class.

1 mid-term exam         

2 projects                    

 

PROJECTS: One project will require algorithm implementation and testing. The second one can be reading and analyzing algorithms or implementing and analyzing algorithms. You can choose which one to do when. Also note: EVERYONE will do a DIFFERENT project. Two projects can be on the same topic, but specific problems will be different.

 

Possible Ideas:

Implementation:

  1. Tracking algorithms - Compare & contrast EKF, GSF, PF, Multiple mode tracker and others.  Choose 2-3 tracking algorithms and choose one problem (say Structure from Motion or 2D Shape Tracking or any problem from your research area) to analyze the algorithms on.  Discuss advantages and disadvantages of each.

 

  1. Classification - Choose one problem domain - Face recognition or Object recognition or Activity recognition from video or something from your research. Implement PCA, LDA, SVMs, PCNSA and compare. Try to develop a combination of these methods that works better.

 

  1. Landmark shape tracking - EKF or PF or perfect observations (no tracking)

 

  1. 2D (and possibly 3D) landmark shape sequence simulator

 

  1. From your research?

 

Analysis Topics:

            Shape Matching, shape distances

            3D Shape representations

            Solving for Correspondences

Particle filtering algorithms

More to come

 

 

REFERENCES: I will keep telling you which chapter to use from which book or will post notes.

  1. Milan Sonka et al: Image Processing, Analysis and Computer Vision
  2. Tekalp: Digital Video Processing
  3. A.K. Jain Digital Image Processing, Chapter 9
  4. Horn: Robot Vision
  5. Forsyth & Ponce: Computer Vision: A Modern Approach

For specific topics:

  1. Tracking:                       Kailath, Sayed, Hassibi: Linear Estimation
  2. Landmark Shape:          Dryden & Mardia:  Statistical Shape Analysis    
  3. Level Set Methods:       Guillermo Sapiro: Geometric Partial Differential Equations and Image Analysis

 

Very detailed reference:

  1. Faugeras: Three-Dimensional Computer Vision

 

General reference (used for EE528 in Spring 2005)

  1. Gonzalez & Woods: Digital Image Processing, There is also a MATLAB version of it

 

Also:      MATLAB help for Image and Video Processing