Stochastic Processes
Introduction
1
Introduction to R and RStudio
1.1
Getting access to R and RStudio
1.2
R basics
1.2.1
Data structures, vectors, matrices
1.3
Installing and using packages in R
1.4
Random variables
1.5
Basics of programming, R scripts
1.6
Importing datasets into R
Summary
2
Counting, sets, and probability basics
2.1
Ticket-in-a-box model of probability
2.1.1
Sampling from a box of tickets
2.2
Relative area model of probability
2.2.1
One dimension
2.2.2
Two or more dimensions
2.3
Sample Spaces and Events
2.3.1
Sample space
2.3.2
Events
2.4
Set Operations
2.5
Venn Diagrams
2.6
Counting, permutations, and combinations
2.6.1
Multiplication rule
2.6.2
\(n^k\)
2.6.3
Factorials
2.6.4
Permutation
2.6.5
Combination
2.6.6
Summary of counting
2.6.7
Special case: with replacement, order doesn’t matter: the multiset
2.7
Probability
2.7.1
Equally likely outcomes
2.7.2
General probability theory
2.7.3
Independence
2.7.4
Conditional probability
2.7.5
Partitions
2.7.6
Baye’s Theorem
3
Random variables and distributions
3.1
Expectation and variance
3.2
Joint distributions
3.3
Independence of random variables
3.4
Discrete: Bernoulli, binomial, geometric, Poisson
3.4.1
Bernoulli
3.4.2
Binomial
3.4.3
Geometric
3.4.4
Poisson
3.5
Continuous: Uniform, exponential, normal
3.5.1
Uniform
3.5.2
Exponential
3.5.3
Normal
Summary
4
Conditional expectation
4.1
Conditional distributions
4.2
Conditional expectation
4.3
(Random) conditional expectation
4.3.1
Total expectation
Summary
5
Intro Stochastic Processes
Summary
6
Random walks
6.1
The simple symmetric random walk (SSRW)
6.1.1
Definition of a random walk
6.2
Distribution of
\(X_n\)
6.3
Shift invariance & memorylessness
6.4
Reflection principle
6.5
Maximum state reached
6.5.1
Maximum over infinite sample paths for
\(p<1/2\)
6.6
Hitting times
6.6.1
Hitting time for state
\(1\)
6.6.2
Hitting time for other states
6.7
Return time to state
\(0\)
Summary
7
Limit theorems
7.1
Inequalities
7.2
Law of large numbers
7.3
Central limit theorem
7.4
Borel-Cantelli
8
Markov Chains
8.1
Graph of a Markov chain
8.2
Classification of states
8.3
Distribution at time
\(n\)
8.4
Simulating a Markov chain in R
8.5
Return times and hitting probabilities
8.6
Limiting probabilities
8.6.1
Limiting distributions
8.7
Stationary distributions
8.7.1
Stationary distributions for reducible chains
8.7.2
R code for finding stationary distributions
8.8
Probability of absorption & time to absorption
8.9
Gambler’s ruin
Summary
9
Branching process
9.1
Offspring distribution
9.2
Survival and extinction
10
Poisson processes
11
Queues
12
Continuous-time Markov processes
13
Brownian motion
Published with bookdown
Math 423 Stochastic Processes Course Notes
Chapter 12
Continuous-time Markov processes
[to be continued]
examples:
radioactive decay
chemical reactions
mean field ODE derivations