Stochastic Processes I

Course Number: 
Hours - Lecture: 
Hours - Lab: 
Hours - Recitation: 
Hours - Total Credit: 
Typical Scheduling: 
Typically every fall semester

Simple random walk and the theory of discrete time Markov chains


MATH 3215 or MATH 3225 or MATH 3235 or MATH 3670 or MATH 3770 or ISYE 3770 or CEE 3770

Course Text: 

At the level of Introduction to Stochastic Processes, Lawler, 2nd edition or Introduction to Probability Models, Ross, 10th edition

Topic Outline: 
  • Simple random walk
  • Applications of weak law and central limit theorem
  • Reflection principle and combinatorial approach
  • Techniques including difference equations and generating functions
  • Gambler's ruin and expected gain problems
  • Markov Chains
    • Conditional probability and conditional expectation
    • Renewal theory with limit theorems
    • Markov chains using renewal theory
    • Finite state space and matrix approach
    • Countable state spaces with examples and applications
    • Absorption probabilities
    • Sojourn times, expected duration, etc.
    • Limiting and stationary distributions
    • Reversibility and applications
  • Introduction to continuous state, discrete time, Markov processes Applications to IFS