Stochastic Processes I

Department: 
MATH
Course Number: 
4221
Hours - Lecture: 
3
Hours - Lab: 
0
Hours - Recitation: 
0
Hours - Total Credit: 
3
Typical Scheduling: 
Typically every fall semester

Simple random walk and the theory of discrete time Markov chains

Prerequisites: 

MATH 3215 or MATH 3225 or MATH 3235 or MATH 3670 or MATH 3770 or ISYE 3770 or CEE 3770

Course Text: 

At the level of Introduction to Stochastic Processes, Lawler, 2nd edition or Introduction to Probability Models, Ross, 10th edition

Topic Outline: 
  • Simple random walk
  • Applications of weak law and central limit theorem
  • Reflection principle and combinatorial approach
  • Techniques including difference equations and generating functions
  • Gambler's ruin and expected gain problems
  • Markov Chains
    • Conditional probability and conditional expectation
    • Renewal theory with limit theorems
    • Markov chains using renewal theory
    • Finite state space and matrix approach
    • Countable state spaces with examples and applications
    • Absorption probabilities
    • Sojourn times, expected duration, etc.
    • Limiting and stationary distributions
    • Reversibility and applications
  • Introduction to continuous state, discrete time, Markov processes Applications to IFS