ACM ACM ACM

ACM216
Markov Chain, Discret Stochastic Processes and Applications

Announcements

  • Feb 05, 2008.
    • Homework 2 is posted.
  • Jan 29, 2008.
    • Solution 1 is posted.
  • Jan 17, 2008.
    • Homework 1 is posted.
  • Jan 9, 2008.
    • If you did not register for the class, please send me an email. I will put your email address mannually into the mailing list.
    • Course website is created! All announcements regarding the course will be posted here.

Instructor
Houman Owhadi
Office Hours: Tue 2:30pm - 3:30pm, right after the lecture
Location: 302 Firestone
Email: owhadi@caltech.edu

TA
Mulin Cheng
Office Hours: Fri 4pm - 5pm
Location: SFL study room 229
Email: mulinch@caltech.edu

Prerequisites
ACM/EE116 Introduction to stochastic Processes and Modeling

Schedule
Classes are scheduled from 1:00pm - 2:30pm on Tuesdays and Thursdays at Firestone 306.

Grading
Active participation in class: 10% (There will be "in class" exercises.)
Homework: 90% (There will be four or five problem sets)
NO Midterm or Final Exam

Syllabus
Additional materials will be posted as the course proceedes.

  1. Markov Models.
  2. Transition matrices and Markov Chains.
  3. Kernels and Markov chains on arbitrary spaces.
  4. Finite Markov Chains.
  5. Markov Chains on countable state spaces.
  6. Simulations with Markov Chains.
  7. MCMC algortithms.
  8. Simulated annealing.
  9. The Propp-Wilson algorithm.
  10. Sandwiching.
  11. Rate of convergence.

Primary Text
Finite Markov Chains and Algorithmic Applications (Olle Hagstrom).
This thin and inexpensive book supplies a nice and up-to-date introduction to Markov Chain, algorithms and applications.

Other Suggested Texts
The lectures will not closely follow these texts: they are given only as suggestions. The first lecture notes gives a rush, but nice and rigorous introduction to probability theory. If you are not familiar with that or you want to refresh your mind, it is a good starting point.

  1. Chapter 2 of Lawrence C. Evans' lecture notes, available online: http://math.berkeley.edu/~evans/SDE.course.pdf
  2. Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (P. Bremaud).
  3. Probability and Random processes (G. R. Grimmett and D. R. Stirzaker).

Homework
Homework and solutions will be posted here. Hand in homework at the class on the specified due date. Points will be deducted from late homeworks unless prior arrangements have been made with the TAs. There is a fifteen-percentage penalty per day. That is, 15% for one day, 30% for two days, 45% for three days and etc.

Consult any materials you like and feel free to work with other students, but be sure to you understand and write up everything you turn in.

  Available Day Due Day
#1 Thur, Jan 17, 2008
Homework
(.pdf) (.html)		 Tues, Jan 29, 2008
Solution
(.pdf) (.html)
#2 Tues, Feb 05, 2008
Homework
(.pdf) (.html)		 Thur, Feb 14, 2008
Solution
(.pdf) (.html)
#3 Thur, Mar 06, 2008
Homework
(.pdf) (.html)		 12:00pm

Mon, Mar 17, 2008
Solution

top


Last Updated: Feb 05, 2008