Correcting violations of independence with dependent models
In my previous article discussing Maximum Likelihood Estimation of Parameters for Random Variables, we acted as a hospital risk manager, senior doctor statistician, data science nurse (I still have literally no idea who could be in control of this) and developed a straightforward probability model to estimate our risk of not having enough beds to accommodate latest patients. To perform this we made the next assumptions:
- Assume all patients checked in to the hospital will try same day
- Assume patients checked in every day are independent of each other
Though impractical, these assumptions enabled us to model the variety of patients in a given day as a Poisson random variable (see Common Random Variables) which has a well-defined distribution function we may use to estimate the probability of being unable to accommodate a brand new patient.
A fast digression — in fact, the Poisson random variable is parameterized by lambda which models the expectation and variance of patients on a given day. We spent a majority of the previous article discussing the best statistical method to estimate this parameter given a set of observed data using the strategy of maximum likelihood estimation. In case you are unfamiliar with this method of selecting an estimator for our parameter estimates I encourage you to examine out that article as we will even be applying the concept herein.
Incorrect assumptions overestimate or underestimate risk or probabilities depending on the violation. For instance, assuming everyone checks in and checks out on the identical day underestimates risk because it is probably going some patients will stay overnight.
This text might be broken up into the next sections.
- Introduction to Markov Chains and Transition Matrices
- Computing Arbitrary Step Transition Probabilities
- Redefining and Solving the Original Problem as a Markov Chain
- Absorbing States
- Maximum Likelihood Estimation of Transition Probabilities
- Calibrating a Transition Matrix to Data and…