Negative multinomial distribution

Content sourced from Wikipedia, licensed under CC BY-SA 3.0.

Negative multinomial distribution: a short, easy guide

What it is
- The negative multinomial distribution generalizes the negative binomial to m possible “success” outcomes.
- Imagine an urn with m+1 colors: outcomes 1, 2, ..., m and a special “failure” color 0.
- You draw repeatedly until the failure color has appeared x0 times. The counts of the other colors X1, X2, ..., Xm before stopping follow the negative multinomial distribution.

Parameters
- x0 > 0: the stopping threshold for the failure color (must be a positive integer in many formulations).
- p = (p1, p2, ..., pm): probabilities of the m success colors.
- p0 = 1 − (p1 + p2 + ... + pm): probability of the failure color.

Support
- Each xi ∈ {0, 1, 2, ...} for i = 1, ..., m.

Probability model (PMF)
- The joint PMF is
P(X1 = x1, ..., Xm = xm) = Γ(x0 + x1 + ... + xm) / Γ(x0) × p0^x0 × ∏_{i=1}^m (p_i^{x_i} / x_i!)
- Here Γ is the Gamma function. If x0 is a positive integer, Γ(x0) = (x0 − 1)!, so you can write it with factorials.

What the numbers mean
- The mean (for each i = 1,...,m):
E[Xi] = x0 × p_i / p0
- The variance and covariance:
Var(Xi) = x0 × p_i × (p0 + p_i) / p0^2
Cov(Xi, Xj) = x0 × p_i × p_j / p0^2 for i ≠ j
- Correlation between Xi and Xj (i ≠ j):
ρ_{ij} = sqrt( p_i p_j / [(p0 + p_i)(p0 + p_j)] )

Key properties
- If m = 1, the negative multinomial reduces to the familiar negative binomial distribution.
- Marginals: If you combine some of the m categories into a new category, the distribution stays negative multinomial with updated probabilities that still sum to 1.
- Sum property: If X1 and X2 are independent negative multinomial random vectors with the same p and radii r1, r2, then X1 + X2 ~ NM(r1 + r2, p).
- Conditional distribution: Given the counts for some categories, the remaining counts follow a negative multinomial with adjusted parameters.

Intuition and interpretation
- It’s an urn-style counting model: you repeatedly draw with fixed probabilities for m+1 colors, stopping only when color 0 has appeared x0 times. The numbers of the other colors before stopping are the X1,...,Xm.

Optional forms
- Moment-generating function (MGF) and characteristic function exist and have specific closed forms, but the essential takeaways are the mean, variance, and the PMF above.

Summary
- Negative multinomial distributes a multivariate count of events before a fixed number of a separate stopping event occurs.
- It generalizes the negative binomial and includes useful marginal, conditional, and aggregation properties for multivariate count data.