Stochastic Processes II

Department: 
MATH
Course Number: 
6762
Hours - Lecture: 
3
Hours - Lab: 
0
Hours - Recitation: 
0
Hours - Total Credit: 
3
Typical Scheduling: 
Every spring semester

Continuous time Markov chains. Uniformization, transient and limiting behavior. Brownian motion and martingales. Optional sampling and convergence. Modeling of inventories, finance, flows in manufacturing and computer networks. (Also listed as ISyE 6762)

Prerequisites: 

MATH 6761

Course Text: 

At the level of Kulkarni, Modeling and Analysis of Stochastic Systems, and Karlin and Taylor, A First Course in Stochastic Processes

Topic Outline: 
  • Continuous Time Markov Chains (CTMC)
    • Markov property
    • Sample path property
    • Birth-death process
    • Embedded DTMC
    • Chapman-Kolmogorov equations
    • Transient probabilities
    • Transience and recurrence criterion
    • Limiting behavior
    • Stationary distribution
    • Network of queues
    • Reversibility
  • Random Walks, Martingales, and Brownian motions
    • Simple random walk as DTMC
    • Definition of martingales
    • The optional sampling theorem
    • Martingales associated with random walks
    • Hitting probabilities
    • Expected hitting times
    • Connection with renewal process
    • Brownian motions
    • Martingales associated with a Brownian motion
    • Hitting times of a Brownian motion
    • Expected hitting times
    • Connection with renewal process
    • Brownian motions
    • Martingales associated with a Brownian motion
    • Hitting times of a Brownian motion
    • Functional strong law of large numbers
    • Functional central limit theorems for random walks and renewal processes
    • One dimensional reflecting Brownian motion
    • Approximate analysis of G/G/1 queue and other systems