- You are here:
- Home

Department:

MATH

Course Number:

4221

Hours - Lecture:

3

Hours - Lab:

0

Hours - Recitation:

0

Hours - Total Credit:

3

Typical Scheduling:

Typically every fall semester

Simple random walk and the theory of discrete time Markov chains

Prerequisites:

Course Text:

At the level of *Introduction to Stochastic Processes*, Lawler, 2nd edition or *Introduction to Probability Models*, Ross, 10th edition

Topic Outline:

- Simple random walk
- Applications of weak law and central limit theorem
- Reflection principle and combinatorial approach
- Techniques including difference equations and generating functions
- Gambler's ruin and expected gain problems
- Markov Chains
- Conditional probability and conditional expectation
- Renewal theory with limit theorems
- Markov chains using renewal theory
- Finite state space and matrix approach
- Countable state spaces with examples and applications
- Absorption probabilities
- Sojourn times, expected duration, etc.
- Limiting and stationary distributions
- Reversibility and applications

- Introduction to continuous state, discrete time, Markov processes Applications to IFS