Crosscorrelation
In signal processing, crosscorrelation is a measure of similarity of two series as a function of the displacement of one relative to the other. This is also known as a sliding dot product or sliding innerproduct. It is commonly used for searching a long signal for a shorter, known feature. It has applications in pattern recognition, single particle analysis, electron tomography, averaging, cryptanalysis, and neurophysiology.
The crosscorrelation is similar in nature to the convolution of two functions. In an autocorrelation, which is the crosscorrelation of a signal with itself, there will always be a peak at a lag of zero, and its size will be the signal energy.
In probability and statistics, the term crosscorrelations is used for referring to the correlations between the entries of two random vectors and , while the correlations of a random vector are considered to be the correlations between the entries of itself, those forming the correlation matrix (matrix of correlations) of . If each of and is a scalar random variable which is realized repeatedly in temporal sequence (a time series), then the correlations of the various temporal instances of are known as autocorrelations of , and the crosscorrelations of with across time are temporal crosscorrelations.
Furthermore, in probability and statistics the definition of correlation always includes a standardising factor in such a way that correlations have values between −1 and +1.
If and are two independent random variables with probability density functions and , respectively, then the probability density of the difference is formally given by the crosscorrelation (in the signalprocessing sense) ; however this terminology is not used in probability and statistics. In contrast, the convolution (equivalent to the crosscorrelation of and ) gives the probability density function of the sum .
Contents
Crosscorrelation of deterministic signalsEdit
For continuous functions and , the crosscorrelation is defined as:^{[1]}^{[2]}^{[3]}


(Eq.1) 
which is equivalent to
where denotes the complex conjugate of , and is the displacement, also known as lag (a feature in at occurs in at ).
Similarly, for discrete functions, the crosscorrelation is defined as:^{[4]}^{[5]}


(Eq.2) 
which is equivalent to
 .
ExplanationEdit
As an example, consider two real valued functions and differing only by an unknown shift along the xaxis. One can use the crosscorrelation to find how much must be shifted along the xaxis to make it identical to . The formula essentially slides the function along the xaxis, calculating the integral of their product at each position. When the functions match, the value of is maximized. This is because when peaks (positive areas) are aligned, they make a large contribution to the integral. Similarly, when troughs (negative areas) align, they also make a positive contribution to the integral because the product of two negative numbers is positive.
With complexvalued functions and , taking the conjugate of ensures that aligned peaks (or aligned troughs) with imaginary components will contribute positively to the integral.
In econometrics, lagged crosscorrelation is sometimes referred to as crossautocorrelation.^{[6]}^{:p. 74}
PropertiesEdit
 The crosscorrelation of functions and is equivalent to the convolution (denoted by ) of and . That is:
 If is a Hermitian function, then
 If both and are Hermitian, then .
 .
 Analogous to the convolution theorem, the crosscorrelation satisfies
 where denotes the Fourier transform, and an again indicates the complex conjugate of , since . Coupled with fast Fourier transform algorithms, this property is often exploited for the efficient numerical computation of crosscorrelations ^{[7]} (see circular crosscorrelation).
 The crosscorrelation is related to the spectral density (see Wiener–Khinchin theorem).
 The crosscorrelation of a convolution of and with a function is the convolution of the crosscorrelation of and with the kernel :
 .
Definition for periodic signalsEdit
If and are both continuous periodic functions of period , the integration from to is replaced by integration over any interval of length :
which is equivalent to
Crosscorrelation of random vectorsEdit
DefinitionEdit
For random vectors and , each containing random elements whose expected value and variance exist, the crosscorrelation matrix of and is defined by^{[8]}^{:p.337}


(Eq.3) 
and has dimensions . Written componentwise:
The random vectors and need not have the same dimension, and either might be a scalar value.
ExampleEdit
For example, if and are random vectors, then is a matrix whose th entry is .
Definition for complex random vectorsEdit
If and are complex random vectors, each containing random variables whose expected value and variance exist, the crosscorrelation matrix of and is defined by
where denotes Hermitian transposition.
Crosscorrelation of stochastic processesEdit
In time series analysis and statistics, the crosscorrelation of a pair of random process is the correlation between values of the processes at different times, as a function of the two times. Let be a pair of random processes, and be any point in time ( may be an integer for a discretetime process or a real number for a continuoustime process). Then is the value (or realization) produced by a given run of the process at time .
Crosscorrelation functionEdit
Suppose that the process has means and and variances and at time , for each . Then the definition of the crosscorrelation between times and is^{[8]}^{:p.392}


(Eq.4) 
where is the expected value operator. Note that this expression may be not defined.
Crosscovariance functionEdit
Subtracting the mean before multiplication yields the crosscovariance between times and :^{[8]}^{:p.392}


(Eq.5) 
Note that this expression is not welldefined for alltime series or processes, because the mean may not exist, or the variance may not exist.
Definition for widesense stationary stochastic processEdit
Let represent a pair of stochastic processes that are jointly widesense stationary. Then the Crosscovariance function and the crosscorrelation function are given as follows.
Crosscorrelation functionEdit


(Eq.6) 
or equivalently
Crosscovariance functionEdit


(Eq.7) 
or equivalently
where and are the mean and standard deviation of the process , which are constant over time due to stationarity; and similarly for , respectively. indicates the expected value. That the crosscovariance and crosscorrelation are independent of is precisely the additional information (beyond being individually widesense stationary) conveyed by the requirement that are jointly widesense stationary.
The crosscorrelation of a pair of jointly wide sense stationary stochastic processes can be estimated by averaging the product of samples measured from one process and samples measured from the other (and its time shifts). The samples included in the average can be an arbitrary subset of all the samples in the signal (e.g., samples within a finite time window or a subsampling^{[which?]} of one of the signals). For a large number of samples, the average converges to the true crosscorrelation.
NormalizationEdit
It is common practice in some disciplines (e.g. statistics and time series analysis) to normalize to crosscorrelation function to get a timedependent Pearson correlation coefficient. However, in other disciplines (e.g. engineering) the normalization is usually dropped and the terms "crosscorrelation" and "crosscovariance" are used interchangeably.
The definition of the normalized crosscorrelation of a stochastic process is
 .
If the function is welldefined, its value must lie in the range , with 1 indicating perfect correlation and −1 indicating perfect anticorrelation.
For jointly widesense stationary stochastic processes, the definition is
 .
The normalization is important both because the interpretation of the autocorrelation as a correlation provides a scalefree measure of the strength of statistical dependence, and because the normalization has an effect on the statistical properties of the estimated autocorrelations.
PropertiesEdit
Symmetry propertyEdit
For jointly widesense stationary stochastic processes, the crosscorrelation function has the following symmetry property:^{[9]}^{:p.173}
Respectively for jointly WSS processes:
Time delay analysisEdit
Crosscorrelations are useful for determining the time delay between two signals, e.g., for determining time delays for the propagation of acoustic signals across a microphone array.^{[10]}^{[11]}^{[clarification needed]} After calculating the crosscorrelation between the two signals, the maximum (or minimum if the signals are negatively correlated) of the crosscorrelation function indicates the point in time where the signals are best aligned; i.e., the time delay between the two signals is determined by the argument of the maximum, or arg max of the crosscorrelation, as in
Terminology in image processingEdit
Zeronormalized crosscorrelation (ZNCC)Edit
For imageprocessing applications in which the brightness of the image and template can vary due to lighting and exposure conditions, the images can be first normalized. This is typically done at every step by subtracting the mean and dividing by the standard deviation. That is, the crosscorrelation of a template, with a subimage is
 .
where is the number of pixels in and , is the average of and is standard deviation of .
In functional analysis terms, this can be thought of as the dot product of two normalized vectors. That is, if
and
then the above sum is equal to
where is the inner product and is the L² norm.
Thus, if and are real matrices, their normalized crosscorrelation equals the cosine of the angle between the unit vectors and , being thus if and only if equals multiplied by a positive scalar.
Normalized correlation is one of the methods used for template matching, a process used for finding incidences of a pattern or object within an image. It is also the 2dimensional version of Pearson productmoment correlation coefficient.
Normalized crosscorrelation (NCC)Edit
NCC is similar to ZNCC with the only difference of not subtracting the local mean value of intensities:
Nonlinear systemsEdit
Caution must be applied when using cross correlation for nonlinear systems. In certain circumstances, which depend on the properties of the input, cross correlation between the input and output of a system with nonlinear dynamics can be completely blind to certain nonlinear effects.^{[12]} This problem arises because some quadratic moments can equal zero and this can incorrectly suggest that there is little "correlation" (in the sense of statistical dependence) between two signals, when in fact the two signals are strongly related by nonlinear dynamics.
See alsoEdit
ReferencesEdit
 ^ Bracewell, R. "Pentagram Notation for Cross Correlation." The Fourier Transform and Its Applications. New York: McGrawHill, pp. 46 and 243, 1965.
 ^ Papoulis, A. The Fourier Integral and Its Applications. New York: McGrawHill, pp. 244–245 and 252253, 1962.
 ^ Weisstein, Eric W. "CrossCorrelation." From MathWorldA Wolfram Web Resource. http://mathworld.wolfram.com/CrossCorrelation.html
 ^ Rabiner, L.R.; Schafer, R.W. (1978). Digital Processing of Speech Signals. Signal Processing Series. Upper Saddle River, NJ: Prentice Hall. pp. 147–148. ISBN 0132136031.
 ^ Rabiner, Lawrence R.; Gold, Bernard (1975). Theory and Application of Digital Signal Processing. Englewood Cliffs, NJ: PrenticeHall. p. 401. ISBN 0139141014.
 ^ Campbell; Lo; MacKinlay (1996). The Econometrics of Financial Markets. NJ: Princeton University Press. ISBN 0691043019.
 ^ Kapinchev; Bradu; Barnes; Podoleanu (2015). "GPU Implementation of CrossCorrelation for Image Generation in Real Time". ICSPCS 2015. doi:10.1109/ICSPCS.2015.7391783.
 ^ ^{a} ^{b} ^{c} Gubner, John A. (2006). Probability and Random Processes for Electrical and Computer Engineers. Cambridge University Press. ISBN 9780521864701.
 ^ Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 9783319680743
 ^ Rhudy, Matthew; Brian Bucci; Jeffrey Vipperman; Jeffrey Allanach; Bruce Abraham (November 2009). "Microphone Array Analysis Methods Using CrossCorrelations". Proceedings of 2009 ASME International Mechanical Engineering Congress, Lake Buena Vista, FL. doi:10.1115/IMECE200910798.
 ^ Rhudy, Matthew (November 2009). "Real Time Implementation of a Military Impulse Classifier". University of Pittsburgh, Master's Thesis.
 ^ Billings, S. A. (2013). Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and SpatioTemporal Domains. Wiley. ISBN 9781118535561.
Further readingEdit
 Tahmasebi, Pejman; Hezarkhani, Ardeshir; Sahimi, Muhammad (2012). "Multiplepoint geostatistical modeling based on the crosscorrelation functions". Computational Geosciences. 16 (3): 779–797. doi:10.1007/s1059601292871.