This course introduces basic concepts and applications of stochastic processes, as well as one more advanced topic.

Two lectures per week for 10 weeks and fortnightly tutorials.

STOCHASTIC PROCESSES STATS4024

120-minute, end-of-course examination (100%)

To provide an introduction to properties of stochastic processes in a variety of settings, including those in i) discrete time with a finite state space; ii) discrete time with a countable state space; iii) continuous time with a countable state space. One more advanced topic will be included.

By the end of this course students will be able to:

■ describe the Gambler's Ruin problem and derive simple properties of it, such as the probability of ruin and the expected duration of a game;

■ define a random walk and calculate some of its properties, such as the probability that the walk is at a specific location at a specific time, the probability of first return, the expected time to first return;

■ describe and identify homogeneous Markov chains (in discrete time with a finite state space), and use the transition matrix to establish some of their basic properties;

■ explain what limiting and stationary distributions are and find them in simple cases;

■ establish whether states are absorbing, periodic, persistent, transient or ergodic in simple cases;

■ define a homogeneous Poisson process, establish the distribution of inter-arrival times and use the Poisson process in applications such as to explore the behaviour of queues;

■ define and derive reliability and hazard functions and calculate the expected number of renewals in a renewal process;

■ define and derive properties of a more advanced stochastic process, such as a continuous-time finite state-space Markov process.