ACM 118, Fall 2009 Methods in Applied Statistics and Data Analysis

Basics:

Professor: Dr. Joel A. Tropp
E-mail: jtropp * acm.caltech.edu
Office: Annenberg 307
Office Hours: W 1:00--2:30 pm

Lectures: Guggenheim 101, TTh 1:00--2:25 pm
Recitation: Annenberg 105, F 3:00--4:00 pm
Textbook: Montgomery, Peck, and Vining, Introduction to Linear Regression Analysis, 4th ed., Wiley, 2006.
Website: http://www.acm.caltech.edu/~jtropp/courses/acm118-f09/

I gladly accept appointments---but only via email with advance notice of at least one business day. Time management, you know.

Teaching Assistants:

Assistant: John Bruer
E-mail: john.bruer * gmail.com
Office Hours: Annenberg 313, M 1:00--2:30 pm

Assistant: Michael McCoy
E-mail: michael.b.mccoy * gmail.com
Office Hours: Annenberg 313, Th 9:00--10:30 am

Prerequisites:

A complete calculus sequence, including multivariate calculus, and a strong background in linear algebra (e.g., Math 1abc). Introductory probability and statistics (e.g., Math 2ab). Exposure to basic concepts of programming (e.g., CS 1). This preparation is necessary.

Background and goals:

Introduction to fundamental ideas and techniques of statistical modeling, with an emphasis on conceptual understanding and on the analysis of real data sets. Assignments will draw on data analysis problems in various science and engineering fields.

Content:

We will cover the following topics:

1. Simple linear regression (least squares estimation, analysis of residuals)
2. Inferences about model parameters, multiple linear regression
3. Analysis of variance, comparison of models, model selection
4. Assessing goodness-of-fit, outliers, influential observations
5. Collinearity and rank-deficiency, singular value decomposition
6. Regularization (truncated SVD, ridge regression)
7. Choosing regularization parameters (generalized cross-validation, L-curve)
8. Principal component analysis
9. Linear discriminant analysis
10. Resampling methods and the bootstrap

Homework:

There will be approximately seven homework sets. Expect to spend 10 hours on each one. Assignments will consist of R programming, statistical computations, data analysis, written explanations, and the occasional proof. You are encouraged to discuss homework problems with other students, but you must prepare and turn in your own assignment independently. There is a separate handout on homework format.

We will have a weekly recitation during which you can ask questions about assignments or lectures. I also plan to provide solution guides. It is a violation of the honor code to use solution guides from current or previous terms when preparing your work.

Homework will (usually) be assigned on Wednesday and will come due the following Wednesday at 5 pm. Assignments will be returned the next Tuesday at the end of class. There will be no assignment due during the midterm exam week. On-time assignments should be delivered to the ACM 118 Mailbox in the lobby of Firestone Labs.

Due dates are strict. Late assignments must be delivered during regular business hours to Sheila Shull (217 Firestone), who will record the date the assignment is turned in. You will (automatically) receive three grace days over the course of the term. After you have consumed these grace days, each late assignment will receive zero credit.

At my discretion, extensions may be granted for academic reasons (e.g., conference trips) provided at least one week notice. I provide extensions for documented medical or psychological reasons. I will never grant an extension after an assignment comes due, unless you have proof of a genuine emergency. The teaching assistants are not empowered to grant extensions.

Exams:

The course will conclude with a take-home final exam. The exam will be due at noon on Wednesday, December 9. Other policies will be stated clearly before the exam.

Grading:

The course grade will be determined by a weighted average of the homework grade (70%) and the exam grade (30%). There is no fixed distribution of letter grades. Each student with a total score of 90% (resp., 80%, 70%, 60%) is guaranteed a final grade of A (resp., B or higher, C or higher, D or higher). I usually adjust these boundaries downward (i.e., in the favorable direction).

Grade review policy:

If you have a question or concern about an assignment grade, please discuss it first with the teaching assistant who graded your assignment. If you are unable to resolve the problem, you may attend my office hours or schedule an appointment. For your privacy, I will not discuss grades in the classroom.

Resources:

Introductory textbooks on applied statistics and data analysis on reserve:

• Draper and Smith, Applied Regression Analysis, 3rd ed., Wiley, 1998.
• Efron and Tibshirani, An Introduction to the Bootstrap, Chapman and Hall, 1994.
• Johnson and Wichern, Applied Multivariate Statistical Analysis, 6th ed., Prentice Hall, 2007.
• Manly, Multivariate Statistical Methods: A Primer, 3rd ed., Chapman and Hall, 2005.
• Ramsey and Schafer, The Statistical Sleuth: A Course in Methods of Data Analysis, 2nd ed., Duxbury, 2002.
• Ryan, Modern Regression Analysis, 2nd ed., Wiley, 2009.
• Venables and Ripley, Modern Applied Statistics with S, 4th ed., Springer, 2002.
• Weisberg, Applied Linear Regression, 3rd ed., Wiley, 2005.

• http://cran.r-project.org/doc/manuals/R-intro.pdf
• http://www.cyclismo.org/tutorial/R/

• http://www.gnu.org/software/octave
• http://www.mathworks.com/
• http://www.cyclismo.org/tutorial/matlab/
• N. Higham and D. Higham, Matlab Guide, SIAM, 2000.

Schedule (subject to change at any moment)

Handouts, etc.

• Syllabus
• Homework Guide
• H. Wainer, "How to present data badly," Amer. Statistician, Vol. 38, No. 2, pp. 137-147, May 1984.
• Class height data and R
• F. Galton, "Regression toward mediocrity in anthropological stature," J. Anthrop. Inst., Vol. 15, pp. 246-273, 1886.
• Predicting snowfall with linear regression
• Model assessment and outliers

Data

• Class height data set
• stat-grades.csv
• penn-votes.txt
• oakland.txt
• life.csv
• swls.csv
• mortality.txt
• o-rings.csv
• ford-firestone.csv