Characteristic function (probability theory)
In probability theory and statistics, the characteristic function of any realvalued random variable completely defines its probability distribution. If a random variable admits a probability density function, then the characteristic function is the Fourier transform of the probability density function. Thus it provides an alternative route to analytical results compared with working directly with probability density functions or cumulative distribution functions. There are particularly simple results for the characteristic functions of distributions defined by the weighted sums of random variables.
In addition to univariate distributions, characteristic functions can be defined for vector or matrixvalued random variables, and can also be extended to more generic cases.
The characteristic function always exists when treated as a function of a realvalued argument, unlike the momentgenerating function. There are relations between the behavior of the characteristic function of a distribution and properties of the distribution, such as the existence of moments and the existence of a density function.
IntroductionEdit
The characteristic function provides an alternative way for describing a random variable. Similar to the cumulative distribution function,
(where 1_{{X ≤ x}} is the indicator function — it is equal to 1 when X ≤ x, and zero otherwise), which completely determines the behavior and properties of the probability distribution of the random variable X, the characteristic function,
also completely determines the behavior and properties of the probability distribution of the random variable X. The two approaches are equivalent in the sense that knowing one of the functions it is always possible to find the other, yet they provide different insights for understanding the features of the random variable. However, in particular cases, there can be differences in whether these functions can be represented as expressions involving simple standard functions.
If a random variable admits a density function, then the characteristic function is its dual, in the sense that each of them is a Fourier transform of the other. If a random variable has a momentgenerating function , then the domain of the characteristic function can be extended to the complex plane, and
 ^{[1]}
Note however that the characteristic function of a distribution always exists, even when the probability density function or momentgenerating function do not.
The characteristic function approach is particularly useful in analysis of linear combinations of independent random variables: a classical proof of the Central Limit Theorem uses characteristic functions and Lévy's continuity theorem. Another important application is to the theory of the decomposability of random variables.
DefinitionEdit
For a scalar random variable X the characteristic function is defined as the expected value of e^{itX}, where i is the imaginary unit, and t ∈ R is the argument of the characteristic function:
Here F_{X} is the cumulative distribution function of X, and the integral is of the Riemann–Stieltjes kind. If a random variable X has a probability density function f_{X}, then the characteristic function is its Fourier transform with sign reversal in the complex exponential,^{[2]}^{[3]} and the last formula in parentheses is valid. Q_{X}(p) is the inverse cumulative distribution function of X also called the quantile function of X.^{[4]} This convention for the constants appearing in the definition of the characteristic function differs from the usual convention for the Fourier transform.^{[5]} For example, some authors^{[6]} define φ_{X}(t) = Ee^{−2πitX}, which is essentially a change of parameter. Other notation may be encountered in the literature: as the characteristic function for a probability measure p, or as the characteristic function corresponding to a density f.
GeneralizationsEdit
The notion of characteristic functions generalizes to multivariate random variables and more complicated random elements. The argument of the characteristic function will always belong to the continuous dual of the space where the random variable X takes its values. For common cases such definitions are listed below:
 If X is a kdimensional random vector, then for t ∈ R^{k}
 where is the transpose of the matrix ,
 If X is a k × pdimensional random matrix, then for t ∈ R^{k×p}
 where is the trace operator,
 If X is a complex random variable, then for t ∈ C ^{[7]}
 where is the complex conjugate of and is the real part of the complex number ,
 If X is a kdimensional complex random vector, then for t ∈ C^{k} ^{[8]}
 where is the conjugate transpose of the matrix ,
 If X(s) is a stochastic process, then for all functions t(s) such that the integral converges for almost all realizations of X ^{[9]}
ExamplesEdit
Distribution  Characteristic function φ(t) 

Degenerate δ_{a}  
Bernoulli Bern(p)  
Binomial B(n, p)  
Negative binomial NB(r, p)  
Poisson Pois(λ)  
Uniform (continuous) U(a, b)  
Uniform (discrete) DU(a, b)  
Laplace L(μ, b)  
Normal N(μ, σ^{2})  
Chisquared χ^{2}_{k}  
Cauchy C(μ, θ)  
Gamma Γ(k, θ)  
Exponential Exp(λ)  
Geometric Gf(p) (number of failures) 

Geometric Gt(p) (number of trials) 

Multivariate normal N(μ, Σ)  
Multivariate Cauchy MultiCauchy(μ, Σ)^{[10]} 
Oberhettinger (1973) provides extensive tables of characteristic functions.
PropertiesEdit
 The characteristic function of a realvalued random variable always exists, since it is an integral of a bounded continuous function over a space whose measure is finite.
 A characteristic function is uniformly continuous on the entire space
 It is nonvanishing in a region around zero: φ(0) = 1.
 It is bounded: φ(t) ≤ 1.
 It is Hermitian: φ(−t) = φ(t). In particular, the characteristic function of a symmetric (around the origin) random variable is realvalued and even.
 There is a bijection between probability distributions and characteristic functions. That is, for any two random variables X_{1}, X_{2}, both have the same probability distribution if and only if .
 If a random variable X has moments up to kth order, then the characteristic function φ_{X} is k times continuously differentiable on the entire real line. In this case
 If a characteristic function φ_{X} has a kth derivative at zero, then the random variable X has all moments up to k if k is even, but only up to k – 1 if k is odd.^{[11]}
 If X_{1}, ..., X_{n} are independent random variables, and a_{1}, ..., a_{n} are some constants, then the characteristic function of the linear combination of the X_{i} 's is
 One specific case is the sum of two independent random variables X_{1} and X_{2} in which case one has
 The tail behavior of the characteristic function determines the smoothness of the corresponding density function.
 Let the random variable be the linear transformation of a random variable . The characteristic function of is . For random vectors and (where A is a constant matrix and B a constant vector), we have .^{[12]}
ContinuityEdit
The bijection stated above between probability distributions and characteristic functions is sequentially continuous. That is, whenever a sequence of distribution functions F_{j}(x) converges (weakly) to some distribution F(x), the corresponding sequence of characteristic functions φ_{j}(t) will also converge, and the limit φ(t) will correspond to the characteristic function of law F. More formally, this is stated as
 Lévy’s continuity theorem: A sequence X_{j} of nvariate random variables converges in distribution to random variable X if and only if the sequence φ_{Xj} converges pointwise to a function φ which is continuous at the origin. Then φ is the characteristic function of X.^{[13]}
This theorem is frequently used to prove the law of large numbers, and the central limit theorem.
Inversion formulaeEdit
There is a onetoone correspondence between cumulative distribution functions and characteristic functions, so it is possible to find one of these functions if we know the other. The formula in the definition of characteristic function allows us to compute φ when we know the distribution function F (or density f). If, on the other hand, we know the characteristic function φ and want to find the corresponding distribution function, then one of the following inversion theorems can be used.
Theorem. If characteristic function φ_{X} is integrable, then F_{X} is absolutely continuous, and therefore X has a probability density function. In the univariate case (i.e. when X is scalarvalued) the density function is given by
In the multivariate case it is
where is the dotproduct.
The pdf is the Radon–Nikodym derivative of the distribution μ_{X} with respect to the Lebesgue measure λ:
Theorem (Lévy).^{[note 1]} If φ_{X} is characteristic function of distribution function F_{X}, two points a < b are such that {x  a < x < b} is a continuity set of μ_{X} (in the univariate case this condition is equivalent to continuity of F_{X} at points a and b), then
 If X is scalar:
 This formula can be restated in a form more convenient for numerical computation as ^{[14]}
 For a random variable bounded from below one can obtain by taking such that Otherwise, if a random variable is not bounded from below, the limit for gives , but is numerically impractical.^{[14]}
 If X is a vector random variable:
Theorem. If a is (possibly) an atom of X (in the univariate case this means a point of discontinuity of F_{X} ) then
 If X is scalar:
 If X is a vector random variable:^{[15]}
Theorem (GilPelaez).^{[16]} For a univariate random variable X, if x is a continuity point of F_{X} then
where the imaginary part of a complex number is given by .The integral may be not Lebesgueintegrable; for example, when X is the discrete random variable that is always 0, it becomes the Dirichlet integral.
Inversion formulas for multivariate distributions are available.^{[17]}
Criteria for characteristic functionsEdit
The set of all characteristic functions is closed under certain operations:
 A convex linear combination (with ) of a finite or a countable number of characteristic functions is also a characteristic function.
 The product of a finite number of characteristic functions is also a characteristic function. The same holds for an infinite product provided that it converges to a function continuous at the origin.
 If φ is a characteristic function and α is a real number, then , Re(φ), φ^{2}, and φ(αt) are also characteristic functions.
It is well known that any nondecreasing càdlàg function F with limits F(−∞) = 0, F(+∞) = 1 corresponds to a cumulative distribution function of some random variable. There is also interest in finding similar simple criteria for when a given function φ could be the characteristic function of some random variable. The central result here is Bochner’s theorem, although its usefulness is limited because the main condition of the theorem, nonnegative definiteness, is very hard to verify. Other theorems also exist, such as Khinchine’s, Mathias’s, or Cramér’s, although their application is just as difficult. Pólya’s theorem, on the other hand, provides a very simple convexity condition which is sufficient but not necessary. Characteristic functions which satisfy this condition are called Pólyatype.^{[18]}
Bochner’s theorem. An arbitrary function φ : R^{n} → C is the characteristic function of some random variable if and only if φ is positive definite, continuous at the origin, and if φ(0) = 1.
Khinchine’s criterion. A complexvalued, absolutely continuous function φ, with φ(0) = 1, is a characteristic function if and only if it admits the representation
Mathias’ theorem. A realvalued, even, continuous, absolutely integrable function φ, with φ(0) = 1, is a characteristic function if and only if
for n = 0,1,2,..., and all p > 0. Here H_{2n} denotes the Hermite polynomial of degree 2n.
Pólya’s theorem. If is a realvalued, even, continuous function which satisfies the conditions
 ,
 is convex for ,
 ,
then φ(t) is the characteristic function of an absolutely continuous distribution symmetric about 0.
UsesEdit
Because of the continuity theorem, characteristic functions are used in the most frequently seen proof of the central limit theorem. The main technique involved in making calculations with a characteristic function is recognizing the function as the characteristic function of a particular distribution.
Basic manipulations of distributionsEdit
Characteristic functions are particularly useful for dealing with linear functions of independent random variables. For example, if X_{1}, X_{2}, ..., X_{n} is a sequence of independent (and not necessarily identically distributed) random variables, and
where the a_{i} are constants, then the characteristic function for S_{n} is given by
In particular, φ_{X+Y}(t) = φ_{X}(t)φ_{Y}(t). To see this, write out the definition of characteristic function:
The independence of X and Y is required to establish the equality of the third and fourth expressions.
Another special case of interest for identically distributed random variables is when a_{i} = 1/n and then S_{n} is the sample mean. In this case, writing X for the mean,
MomentsEdit
Characteristic functions can also be used to find moments of a random variable. Provided that the n^{th} moment exists, the characteristic function can be differentiated n times and
For example, suppose X has a standard Cauchy distribution. Then φ_{X}(t) = e^{−t}. This is not differentiable at t = 0, showing that the Cauchy distribution has no expectation. Also, the characteristic function of the sample mean X of n independent observations has characteristic function φ_{X}(t) = (e^{−t/n})^{n} = e^{−t}, using the result from the previous section. This is the characteristic function of the standard Cauchy distribution: thus, the sample mean has the same distribution as the population itself.
The logarithm of a characteristic function is a cumulant generating function, which is useful for finding cumulants; some instead define the cumulant generating function as the logarithm of the momentgenerating function, and call the logarithm of the characteristic function the second cumulant generating function.
Data analysisEdit
Characteristic functions can be used as part of procedures for fitting probability distributions to samples of data. Cases where this provides a practicable option compared to other possibilities include fitting the stable distribution since closed form expressions for the density are not available which makes implementation of maximum likelihood estimation difficult. Estimation procedures are available which match the theoretical characteristic function to the empirical characteristic function, calculated from the data. Paulson et al. (1975) and Heathcote (1977) provide some theoretical background for such an estimation procedure. In addition, Yu (2004) describes applications of empirical characteristic functions to fit time series models where likelihood procedures are impractical.
ExampleEdit
The gamma distribution with scale parameter θ and a shape parameter k has the characteristic function
Now suppose that we have
with X and Y independent from each other, and we wish to know what the distribution of X + Y is. The characteristic functions are
which by independence and the basic properties of characteristic function leads to
This is the characteristic function of the gamma distribution scale parameter θ and shape parameter k_{1} + k_{2}, and we therefore conclude
The result can be expanded to n independent gamma distributed random variables with the same scale parameter and we get
Entire characteristic functionsEdit
This section needs expansion. You can help by adding to it. (December 2009) 
As defined above, the argument of the characteristic function is treated as a real number: however, certain aspects of the theory of characteristic functions are advanced by extending the definition into the complex plane by analytical continuation, in cases where this is possible.^{[19]}
Related conceptsEdit
Related concepts include the momentgenerating function and the probabilitygenerating function. The characteristic function exists for all probability distributions. This is not the case for the momentgenerating function.
The characteristic function is closely related to the Fourier transform: the characteristic function of a probability density function p(x) is the complex conjugate of the continuous Fourier transform of p(x) (according to the usual convention; see continuous Fourier transform – other conventions).
where P(t) denotes the continuous Fourier transform of the probability density function p(x). Likewise, p(x) may be recovered from φ_{X}(t) through the inverse Fourier transform:
Indeed, even when the random variable does not have a density, the characteristic function may be seen as the Fourier transform of the measure corresponding to the random variable.
Another related concept is the representation of probability distributions as elements of a reproducing kernel Hilbert space via the kernel embedding of distributions. This framework may be viewed as a generalization of the characteristic function under specific choices of the kernel function.
See alsoEdit
 Subindependence, a weaker condition than independence, that is defined in terms of characteristic functions.
 Cumulant, a term of the cumulant generating functions, which are logs of the characteristic functions.
NotesEdit
ReferencesEdit
CitationsEdit
 ^ Lukacs (1970) p. 196
 ^ Statistical and Adaptive Signal Processing (2005)
 ^ Billingsley (1995)
 ^ Shaw, W. T.; McCabe, J. (2009). "Monte Carlo sampling given a Characteristic Function: Quantile Mechanics in Momentum Space". arXiv:0903.1592 [qfin.CP].CS1 maint: ref=harv (link)
 ^ Pinsky (2002)
 ^ Bochner (1955)
 ^ Andersen et al. (1995, Definition 1.10)
 ^ Andersen et al. (1995, Definition 1.20)
 ^ Sobczyk (2001, p. 20)
 ^ Kotz et al. p. 37 using 1 as the number of degree of freedom to recover the Cauchy distribution
 ^ Lukacs (1970), Corollary 1 to Theorem 2.3.1
 ^ "Joint characteristic function". www.statlect.com. Retrieved 7 April 2018.
 ^ Cuppens (1975, Theorem 2.6.9)
 ^ ^{a} ^{b} Shepard, N.G. (1991a)
 ^ Cuppens (1975, Theorem 2.3.2)
 ^ Wendel, J.G. (1961)
 ^ Shephard (1991a,b)
 ^ Lukacs (1970), p. 84
 ^ Lukacs (1970, Chapter 7)
SourcesEdit
 Andersen, H.H., M. Højbjerre, D. Sørensen, P.S. Eriksen (1995). Linear and graphical models for the multivariate complex normal distribution. Lecture Notes in Statistics 101. New York: SpringerVerlag. ISBN 9780387945217.CS1 maint: uses authors parameter (link)
 Billingsley, Patrick (1995). Probability and measure (3rd ed.). John Wiley & Sons. ISBN 9780471007104.
 Bisgaard, T. M.; Sasvári, Z. (2000). Characteristic functions and moment sequences. Nova Science.
 Bochner, Salomon (1955). Harmonic analysis and the theory of probability. University of California Press.
 Cuppens, R. (1975). Decomposition of multivariate probabilities. Academic Press.
 Heathcote, C.R. (1977). "The integrated squared error estimation of parameters". Biometrika. 64 (2): 255–264. doi:10.1093/biomet/64.2.255.CS1 maint: ref=harv (link)
 Lukacs, E. (1970). Characteristic functions. London: Griffin.
 Kotz, Samuel; Nadarajah, Saralees (2004). Multivariate T Distributions and Their Applications. Cambridge University Press.
 Oberhettinger, Fritz (1973). "Fourier Transforms of Distributions and their Inverses: A Collection of Tables". Academic Press. Cite journal requires
journal=
(help)CS1 maint: ref=harv (link)  Paulson, A.S.; Holcomb, E.W.; Leitch, R.A. (1975). "The estimation of the parameters of the stable laws". Biometrika. 62 (1): 163–170. doi:10.1093/biomet/62.1.163.CS1 maint: ref=harv (link)
 Pinsky, Mark (2002). Introduction to Fourier analysis and wavelets. Brooks/Cole. ISBN 9780534376604.
 Sobczyk, Kazimierz (2001). Stochastic differential equations. Kluwer Academic Publishers. ISBN 9781402003455.
 Wendel, J.G. (1961). "The nonabsolute convergence of GilPelaez' inversion integral". The Annals of Mathematical Statistics. 32 (1): 338–339. doi:10.1214/aoms/1177705164.CS1 maint: ref=harv (link)
 Yu, J. (2004). "Empirical characteristic function estimation and its applications". Econometrics Reviews. 23 (2): 93–1223. doi:10.1081/ETC120039605. S2CID 9076760.CS1 maint: ref=harv (link)
 Shephard, N. G. (1991a). "From characteristic function to distribution function: A simple framework for the theory". Econometric Theory. 7 (4): 519–529. doi:10.1017/s0266466600004746.
 Shephard, N. G. (1991b). "Numerical integration rules for multivariate inversions". J. Statist. Comput. Simul. 39 (1–2): 37–46. doi:10.1080/00949659108811337.
External linksEdit
 "Characteristic function", Encyclopedia of Mathematics, EMS Press, 2001 [1994]