A Generalization of Wiener Optimum Filtering and Prediction Citation Beutler, Fredrick Joseph (1957) A Generalization of Wiener Optimum Filtering and Prediction. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/D8FV-4255. https://resolver.caltech.edu/CaltechETD:etd-07082004-135353 Abstract This work generalizes the Wiener-Kolmogorov theory of optimum linear filtering and prediction of stationary random inputs. It is assumed that signal and noise have passed through a random device before being available for filtering and prediction. A random device is a unit whose behavior depends on an unknown parameter for which an a priori probability distribution is given. Use of representation theorems and a Hilbert space structure make it possible to present the mathematical theory without the ambiguities encountered in engineering derivations. This approach also leads to a proof of the essential identity between the operator solution and a realizable lumped parameter filter. A number of engineering applications are cited. A few of these are worked out in some detail to illustrate the optimization procedure. Item Type: Thesis (Dissertation (Ph.D.)) Subject Keywords: (Engineering Science and Mathematics) Degree Grantor: California Institute of Technology Division: Engineering and Applied Science Major Option: Engineering Minor Option: Mathematics Thesis Availability: Public (worldwide access) Research Advisor(s): De Prima, Charles R. Thesis Committee: Unknown, Unknown Defense Date: 1 January 1957 Record Number: CaltechETD:etd-07082004-135353 Persistent URL: https://resolver.caltech.edu/CaltechETD:etd-07082004-135353 DOI: 10.7907/D8FV-4255 Default Usage Policy: No commercial reproduction, distribution, display or performance rights in this work are provided. ID Code: 2828 Collection: CaltechTHESIS Deposited By: Imported from ETD-db Deposited On: 13 Jul 2004 Last Modified: 13 Oct 2023 18:47 Thesis Files Preview PDF (Beutler_fj_1957.pdf) - Final Version See Usage Policy. 1MB Repository Staff Only: item control page

A GENERALIZATION OF WIENER OPTIMUM FILTERING AND PREDICTION Thesis by Frederick Joseph Beutler In Partial Fulfillment of the Requirements For the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California 1957 - ACKNOWLEDGMENTS _ The author wishes to express his gratitude to Professor C. R. DePrima who has.gone far beyond his role as thesis advisor to aid the author. His inspirational teaching, constant encourage- ment, and invaluable suggestions have enabled the author to com- plete this work successfully. Thanks are also due the many other members of the Institute staff with whom the author has been in contact. Both formal instruction and informal discussions have contributed greatly in the preparation of the thesis. The understanding, patience, and self-denial of the author's wife have made it possible for him to initiate and continue his graduate studies. Finally, the author is grateful for the financial assistance provided by the International Business Machines Corporation in the form of a fellowship grant. ABSTRACT _ This work generalizes the Wiener-~Kolmogorov theory of optimum linear filtering and prediction of stationary random inputs. It is assumed that signal and noise have passed through a random device before being available for filtering and prediction. A ran- dom device is a unit whose behavior depends on an unknown para- meter for which an a priori probability distribution is given. Use of representation theorems and a Hilbert space structure make it possible to present the mathematical theory without the am- biguities encountered in engineering derivations. This approach also leads to a proof of the essential identity between the operator solution and a realizable lumped parameter filter. A number of engineering applications are cited. A few of these are worked out in some detail to illustrate the optimization procedure. PART II Ii IV VI VII TABLE OF CONTENTS TITLE Introduction Mathematical Foundations Generalized Optimum Filters and the Wiener Problem Derivation of the Stable, Realizable Filter Solution General Prediction and Filtering The Random Gain Amplifier Predicting and Lagging Filters An Example: Random Delay Time References PAGE 10 21 25 37 39 41 45 54 -4- INTRODUCTION During the past three decades the science of control and communication systems has evolved from a poorly understood art to an important branch of engineering practice. One reason for this upsurge is the development of a substantial body of theory, enabling the engineer to perform effective analysis and synthesis. One extensively explored aspect of control and communi- cation engineering is the behavior of constant coefficient systems, i.e., systems which relate aninput x(t) to an output y(t) through the relation n > sSbe]- > 3 [+0 | on where the a; and b are real constants, and n >m. A large number of physical systems can be assumed to act according to 0.4 over their normal operating range. The design of constant coefficient systems to perform con- trol or communication tasks is largely done by one of two methods. The transient response approach places requirements on the system output in response to a step (or ramp) input. For example, the overshoot may be limited to a given figure, or some part of the transition from one steady state to a new steady state output ac- complished within a specified time. Frequency response design emphasizes the magnitude and phase relationship of output and input sinusoids. Upper and lower bounds are placed on both mag- nitude and phase; the design is considered successful if the response remains within these bounds. -2- Both of the above methods require that the design meet criteria which are established intuitively, and which base the per- formance of the system on the response to a single type of input. It is therefore no surprise that the classical design techniques are often inadequate. Many systems must deal with a wide variety of inputs, so that design according to a specific type of input may result in poor performance with other inputs*. Furthermore, the input may be corrupted by random noise, so that the system must perform some indicated operation on the signal while at the same time rejecting the noise. Lastly, the system may be required to predict future values of a random signal, when noise may again be present. Certain design problems involving random inputs were (1) attacked independently (and simultaneously) by Kolmogorov and Wiener”), It is desirable to discuss their objectives as a pre- liminary to an explanation of the aims of this paper. In order to give the design problem its analytical setting it is necessary to define a quantitative measure of performance. Consider a signal x(t) plus noise n(t) as the input of a physically realizable system, and call the system output w(t). Define an ideal operator which acts on x(t) to give y(t). This operator is determined by the desired relationship of y(t) to x(t). An illus- tration is the prediction operator which has output y(t +h) for input x(t) where h > 0 is specified. The error, € (t), is *This and the preceding comments are not valid for all control sys- tems. For instance, the rocket fuel shutoff mechanism of a,missile is expected to respond to one step input during its life. Therefore, the transient response to a step input is the only meaningful per- formance criterion. ~3- taken to be the difference between the actual and desired outputs, that is, & (t) = w(t) - y(t). The error can also be described in terms ofa flow chart, viz: n(t) physically w(t x(t)-—— realizable E(t) system (0. 2) _»| ideal y(t) operator The engineer wishes to choose that realizable system which minimizes the error € (t). This type of optimization is feasible only if noise and signal are explicitly known beforehand, for € (t) (and thus the optimum system) depends on the input. In most instances the exact system input is unknown at the time the system is being designed. However, the input may be ex- pected to belong to a class of possible inputs. Suppose that the re- lative frequency with which the inputs in this class will be realized in practice can be estimated or measured. The optimum realizable system might then represent a compromise design which is most heavily influenced by those possible inputs most frequently en- countered. In that case, the design objective is to minimize the error averaged over the various inputs which the system may meet in actual use. ‘The Wiener-Kolmogorov theory assumes that the average of x(t)x(t +h) over all possible inputs is known. Furthermore, this average does not depend on +, a property referred to as ~4. stationarity*. A similar requirement is placed on n(t). Then the average of € (t) E(t +h) is likewise independent of t, and (with h = 0) the expectation (or average) of Ey]? does not depend on t.. The expectation of |é (|? is generally called the mean square error. The Kolmogorov-Wiener problem may now be stated as follows: find that physically realizable system which minimizes the mean square error. Of course, the minimization might have been performed with respect to some other criterion. However, there are two reasons for the choice of the mean square error as the standard. In the first place, the mean square error possesses physical significance as a measure of error power Or energy. Secondly, this criterion has the distinct mathematical advantage that the theory does not involve the probability distributions of x(t) or x(t)x(t +h); only the second moments (covariances) enter into the problem**, A principal objective of this thesis is to present a generali- zation of Wiener optimum filtering and prediction. Instead of the flow chart 0.2 applicable to the Wiener problem, the following *To avoid a (trivial) deterministic version of this problem it is required that for at least one possible x(t) + n(t) ‘the future cannot be determined precisely from the past. In addition, Wiener de- mands that x(t) be ergodic. This means that for each possible input the average over t of x(t)x(t + h) must be equal to the en- semble average discussed in the text. This latter assumption is not needed if the derivations of this paper are used in place of those employed by Wiener. **While probability distributions do not play a role here, itis in- _teresting to observe that if x(t) is a Gaussian process the linear filter which is obtained is the best of all possible filters, linear or nonlinear. Otherwise, a nonlinear filter may give even better performance relative to the mean square error criterion than the optimum linear filter. See Reference (3), pp. 74-78. more general diagram is considered: physically . realizable w(t, ¥; Bh + E(t, y, #) system (O. 3) x(t) ideal y(t) , operator The difference between this diagram and 0.2 evidently lies in the addition of h(t, y) and l{t,y). Here h(t, y) is a weighting function so that z(t, y) = {ox - T) h(t, y)dt (O. 4) oO The parameter y is taken to be a random variable whose proba- bility distribution is known. The physical interpretation of h(t, y) is that the random process x(t) is acted upon by a device which is itself random*. The output of this device constitutes the input to the system which is to be optimized with respect to the mean square error. The mean square error is now obtained by averaging | E(t, y. BR) \* over y and yp as well as all possible inputs x(t) and n(t). The above discussion applies to l{t,p) in analogous fashion, except that I(t,p) acts on the noise n(t) instead of the signal x(t). Several engineering examples will serve to clarify the idea of a random weighting function, and to indicate some of the possi- ble applications of the optimization problem depicted by 0.3. eee cee eg ete nde ee tne eet tee et ce ie at et ee ee a ee a ang eee Se ee mee ee ee ee ee ee ee ne * h(t, y}) may also be a delta function representing a random gain or scale factor device. ~6- Suppose that it is desired to design a phonograph. Ideally, this instrument would be equipped with compensation for each of the recording characteristics employed by the various record man- ufacturers. But commercial policies often permit only a single fixed compensation network to be used. This should be designed to represent an optimum compromise among the different recording characteristics. If the optimum is that network which minimizes the average error power the work of this thesis is applicable. Each recording characteristic is an h(t, Y;)+ The probability that yY = Y; can be taken as the number of records sold having charac- teristic i divided by the total number of records sold. Further- more, estimates are available for the frequency spectrum of music. These estimates describe the statistical properties of x(t). The phonograph design example illustrates a larger class of applications. It is typical of a control or data transmission systems that corrective feedback cannot be applied to input and output devices. The compensation then takes the form of a net- work in series with the input and/or output device. The design of such a network is doubly difficult because the characteristics of the input and output elements in question will vary from unit to unit, and may also change with age or use. The statistical para- meters of the input and output devices can be determined from a large number of laboratory tests. Then h(t, y) and the distri- bution of y will be known. The ideal operator is determined from the over-all purpose of the control or data transmission -7- system. Lf a mean square error criterion is adopted, and the system input is stationary, the methods of this thesis are appli- cable. The new techniques of this thesis can also be applied to communication systems in which the error is considered to be the difference between the input to the transmitter and the output of the receiver. It is conventional practice to attach a network filter to the receiver output to reduce the average error power to the lowest value possible. In ground-to-ground microwave systems multipath problems are created by ground reflections. More simply, the transmitted signal arrives not only by a direct line of sight path, but also via multiple ground reflection paths of varying lengths. The received signal is shifted in phase, and there is cancellation of reinforcement which affects the signal amplitude. In the case of an amplitude modulated communication system* the effect is as if the signal had been passed through a filter h(t, y) whose Fourier transform is , + F(w, y). Here the first term is due to the direct transmission path, and the second expresses the aggregate of the reflections in terms of the para- meter y. y depends on the terrain over the transmission path, the transmission distance, and the height of the transmitter and receiver above ground. These factors are not known in a mobile communication complex, nor is it easy to calculate F(w, y) once they are measured. However, controlled experiments can deter- *xAmplitude modulation is specified because it is a linear process with sidebands whose amplitude and phase vary directly with those of the input signal. These requirements must be met if linear analysis is to be used. ~8- mine Flo, y) under various sets of conditions. The probability that the mobile system achieves a particular Fw, y) ona given occasion is assigned in advance an a priori grounds from a know- ledge of the tactical mission of the system. The problem is then of the type which has been solved in this thesis, assuming again that signal and noise are stationary with known statistical para- meters. Of course, the illustrations provided in this introduction encompass but a small part of the engineering problems which can be handled through use of the methods of the thesis. For instance, further examples are offered (and worked out in some detail) in sections five to seven of this paper. It should not be inferred that the present work follows Wiener's derivations in arriving at the generalized results. In- stead, representation theorems and a Hilbert space structure are employed to resolve some of the ambiguities in Wiener's work*, The operator solution which appears in the frequency domain often lacks an interpretation in the time domain, and cer- tainly cannot be synthesed by known network techniques. The writer shows that a filter can be constructed so that the mean square error is as close as desired to that of the operator so- . lution. In fact, it is possible to find a lumped parameter filter having this property. *The Hilbert space approach has been exploited by Doob (Re- ference (3)) to obtain a rigorous treatment of Wiener-Kolmogorov prediction and filtering. -9- A second point, neglected by engineering texts dealing with Wiener filtering, is that the factorization of the frequency spectrum (see section 5) is not unique. The uniqueness condition is stated here, and a time domain equivalent given. Still other matters of interest are the limiting cases of long time prediction and lag, as discussed in section 6, Lastly, section 7 is concerned with the generalized prediction and filtering theory as applied to processes which are not themselves stationary, but which have stationary increments. ~10- SECTION I MATHEMATICAL F OUNDATIONS The optimum filter problem we shall treat was discussed and motivated in the introductory section of this paper. In order to proceed to the solution of the problem we must express it in more precise terms. In that way, we are able to explore the widest conditions of validity under which the solution holds, and to show that this solution is unique and does yield the minimum mean square error. Certain basic definitions and concepts will be used without further explanation.: For example, such notions as probability space, expectation, and stationary (wide sense) process are dis- aissed elsewhere in great detail, Any exposition of these in this paper would be redundant and lengthy. The reader is advised to consult Reference (3) for the necessary background information. Let | x(t) be a process which is wide sense stationary and continuous in the mean. Such a process has a spectral repre- sentation of the form 4 jot x(t) = { dX (w) (1. 1) = € w -@ The integral is defined in the sense of a "limit in mean", usually written lim. (see Reference (3), Chap. IX and pp. 527-8). Here X(w) is a process with orthogonal increments such that iN | 2 E}X(A,-) ~ XOY |? : | dA (w) (1.2) dy ~11- We shall restrict ourselves to absolutely continuous spectral dis- tribution functions. That is, there exists a spectral density $e) such that N [ 6 (a) da = A() <@ (1.3) Evidently, PloeLy - We shall also assume that “Log (w) ——7— dw > - (1.4) 1+ ~00 It may be shown (Reference (3), p. 584) that if (and only if) 1.4 does not hold,the entire future and past of x(t) “may be exactly de- termined from the behavior of x(t) over any t interval of some specified length. In other words, 1.4 is satisfied unless x(t) is a trivial process from the viewpoint of prediction and filtering theory. The output which we desire to obtain from our filter will be denoted by y(t). y(t) is related to x(t) by an operator u(w). We define y(t) = fore dX (wo) (4.5) -a lam where the integral is understood to be l.i.m. The existence of the integral 1.5 is equivalent to the finiteness of the covariance of y(t), or B| yi? - iE: (2) wo) | © a (1.6) ~{2- as shown in Reference (3), p. 534. Since the notion of an operator is basic to this paper, a few clarifying examples will be offered at this point. Suppose that ufw) = e°*, Then ao 1 iw(t + a) . (t) = —— dX(w) = x(t +a) (1.7) y = [ ®) x a so that e* is regarded as a prediction or lag operator according to whether a is positive or negative. Consider next u(w) = iw. _ . x(t +h) - x(t) ,_ dx In that case we have y(t) = }ei;m. hr (= arp): Asa third example, assume that u(w) is the Fourier transform of an L, function U(t). There follows cs) a rs) y(t) = | ulwettax(w) = — et ax(a)f et? TU(rart 27 b on \27 = | 0 (41. 8) @ oe) jn oo = _! [ vee lt-Naxy) = x(t -7)U(t)dt \27 Jeo | {2 on the interchange of integrals being valid according to Reference (3), p. 431, i.e., by a direct application of Fubini's theorem. As is seen from the above examples, the operation u(w) is more general than that of prediction, differentiation, or appli- cation of an ordinary filter to x(t). Infact, any u(w) satisfying 1.6 is admissible, whether or not it admits of an interpretation in the time domain. The process available to our optimum filter is yet a third process, called z(t,y). The z process is created by subjecting -13- x(t) to an operator depending on a random variable y, y being defined on a probability field independent of that of x(t). It will be assumed that this operator has meaning in the time domain. In fact, we begin by specifying that h(t, y) be jointly measurable on the space of t and y, andthat h(t, y) = 0 onthe product set of t<0 and y with the possible exception of a set of product measure zero. We demand, furthermore, that oo 2 E / | nit, y) | at < 2 (1.9) and ° sf [nit |* at < (4. 10) ° It follows from 1.9 and the Schwarz inequality that z [| ms fa s(z J [aon Pe Me ws (1.44) () ° / Therefore, h(t, y)e L, with probability one, so that the L, Fourier transform of h(t, y) exists with probability one; this transform is written H(w,y) - We shall have need to speak of | H(w, y) | and E | E(w, v| 2 in the future. It is convenient, therefore, to define H(w) and G(w) by Hw) = B| Hl, » | and G{w) = E| He, |? - These expec- tations always exist, and indeed Parseval's theorem together with 1.10 gives eo G(w) dw << (1.42) = 00 In consequence of 1.29 G(w) is everywhere bounded so that ~14- / P (a) Glo) doe | (1. 13) “® Furthermore, 1.8 may be applied to h(t, y) to yield z(t, y) = te | Hw, ye? ax) = [e-n0ee v8 (1. 44) This means that a h(t, y) is the weighting function of a realiz- able, stable filter (with probability one), in accordance with the criteria to be presented near the end of this section. Let r(w) be an operator applied to z(t, y). If Po) Glo) rl) | “dao < (1. 15) we have a new process, w(t, y;r) which is represented by t iwt w(t,y;r) = Jw, y) r(w)e dX(w) (1. 16) Ts am -o the integral again being defined in the l.i.m. sense. In the terminology of the engineer, r(w) describes the oper- ation of a device with input z(t, y) and output w(t,y;r). Thus, since the desired outputis y(t) (see 1.5), it makes sense to speak of an error E(t,ysr) = w(t,y) - y(t) (1. 17) The error is again a stochastic process which is wide sense stationary and continuous in the mean (Reference (3), p.534). Then -~i5- a > we may denote the mean square value of €(t,y3r) by € (1); i.e@., & (r) E | E (t, yir)| é (4. 18) We find that £7 (2) e| [Fea] serv - wc]? (4. 19) where BE is the expectation on y. 1.19 may be rewritten oF g(r) = [Be] o0/-001" - H(o)r(w)ae) - (1. 20) H(w)r(w)u(w) + joc]? Jo The latter form is justified by Fubini's theorem. The application of the theorem to the first and third term of 1.20 is clear. For the middle terms the Schwarz inequality gives | H(o) £ G(w) so that a second application of the tnequality yields [ iermerener ee (foro r(o) | © dus /2 « (, [3 (2) Glo) | w(w) | © dew It is convenient to denote the class of all r(w) satisfying 1/2 (1.214) 1.15 by */. As will be seen below, we may find a unique re # which minimizes the mean square error defined by 1.18. However, the engineer thinks in terms of another class, so that ultimately -~16- we shall minimize the mean square error over a class -y, the class of stable, realizable filters. In what follows, we shall give my a mathematically precise meaning. A filter is defined by a weighting function, say K(t), which relates a filter input x(t) to a filter output y(t) by wo y(t) = [ x- tT) K(r)dr (1.22) oo if the integral has meaning. A filter is said to be realizable if that filter is capable of acting only on signals received in the past, and not on signals which are yet to arrive in the future. Thus, a filter is defined to be realizable if and only if its weighting function K(t) = O fora.e. t<0O. Lastly, a filter is stable if every essentially bounded (measurable) input to the filter results in an essentially bounded (measurable) output. We have Theorem 1: A filter is stable if and only if its weighting function is in Ly ,» thatis [ls < | (1. 23) Proof: Using the notation of 1.22, let x(t)<¢ Na.e. Then oa | y(t) | <N [ | K(e) | dt which is bounded if K(t)eL,~. Con- versely, suppose that K(t)}éL, . If x(t)=N, we have 3a J s-oxinan = N / K(t)dr - vf | K(n)fer (1.24) 00 A X-A where A = {¢ | K(t) 2 0} - Atleast one of the two terms on the ~17- right hand side of 1.24 is infinite because K(t)éL, - Thus K(t) cannot represent a stable filter. We shall study the relationship between m and # further. For this purpose it is useful to utilize well-known properties of Hilbert spaces. We may define an inner product (r,s) = E] w(t, y3 r)w(ts y3 s) = J Rierctey=taraTma0 (1.25) where r and s are any two elements of 9 - The inner product is seen to be a symmetric bilinear functional on *% . Also, is complete by the Riesz-Fisher theorem. In short, the space 5 A with inner product defined by 1.25 is a Hilbert space. The null element of the Hilbert space ¥ deserves mention. If Il = || = 0, we have [ Brrcrer| stew = 0 (1. 26) Dl) > 0 awe. is assured by 1.4. G(w) is non-negative, since by definition G(w) = e| Hl, | . In section three, we shall see that [ Log GMs <a so that G(w) >» Oa.e. It follows that 1.26 holds only if r(w) = Oa.e.; hence, two operators in we are equiva- lent if and only if they are equal a.e. The mean square error may be rewritten as 2 2 €7(r) = (1. 27) a ul Te G from 1.20. We shall not develop 1.27 further at present, but only -18- note that e-(r) is minimized if r(wo) = ae) : (1.28) Let r' be any other element of we, Then e7(r') - €*(x) = jet \° (1.29) Thus, if the choice of operators is to be restricted by some sub- space Jc}, the mean square error is minimized by the element in .f which is the projection of reM% on vA . The above discussion reveals how the optimization is to be performed with respect to ™ . Indeed, we need only show that WLC w. From 1.8 and the properties of L, Fourier transforms, an element of a) has an operator representation which is bounded and uniformly continuous. Now we have Lemma 1: Every essentially bounded measurable operator be- longs to We. Proof: According to 1.3, Pw)e Ly - Then 1,15 is satisfied for an essentially bounded operator if G(w) is shown to be bounded. We write for the absolute value of G(w) 2 2 °c | h(t, ye fat ° 2m | Glw)| = E £ E [ [pe y) | at (1.30) which proves the lemma. The projection operator of ¥ on m is difficult to find by A direct means. Instead, we shall define a subspace 77] » project Ye -19- on 97 in an obvious fashion, and then show that wi = 77 . Quite often, the engineer wishes to synthesize an optimum filter which is not only in 7 » but which can be constructed of lumped parameter elements (resistors, capacitors, inductors). A filter of this type is characterized by a rational Fourier transform. The class of such filters will be called #7’ . We have already de- manded that I'am. Now if, in addition, Fy? = wy , the engi- neer knows that he will be able to synthesize a lumped parameter filter whose performance (as measured by the corresponding mean square error) will approach that element of Wy which we shall de- termine to be optimum. Accordingly, we shall prove | Theorem 2: Let Plo) Glo) be essentially bounded. Then Mm = Wr. Proof: Let %” be the subspace generated by the family of oper- 1 (4 = iw)? } * fF (1 + iw)??? Let 4 be the family of step functions in (0,°); we shall think in ators terms of Fourier transforms of the step functions. Then it will be shown that %"c M/ch c gc mm; so that also ” = a aa (1.31) . \n It is evident that 4”c%’, since (poeper is the rational Fourier transform of an L, function which is zero for a.e. nega- tive argument. By definition, we also have nay. ena ee ee ne ee ee te Be eden de eke ete end * These are the well-known Laguerre polynomials. ~20- Given ¢€ > 0, anda K(t)« ™ ,» there exists an S(t) « Le such that k(w) - s(w)| < [Is - S(t) | dt < € (4.32) where k(w) and s(w) are Fourier transforms of K(t) and S(t), respectively. If G(w) < M (see 1.30) there follows Jk- = I <MéE“ [Ser (1.33) a Lastly, we prove that Say". The set { (1 ~ iw)” {Fo (1 + iw) is a complete orthonormal set in L, norm. Therefore, (since ihc L,) yh” is dense in & with respect to L, norm. Because’ Pe)Go) is essentially bounded the density argument holds in XZ norm also; hence de i's -241- SECTION II _ GENERALIZED OPTIMUM FILTERS -AND THE WIENER PROBLEM In this section we shall study some relationships between the filter problem of Wiener, and the problem presented in the pre- ceding portions of this paper. It should be apparent that our problem reduces to the Wiener problem if h(t, y) is a known (rather than a random) function, i.e., a function not dependent on y. On the other hand, if h(t, y) is not exactly known, it might be expected that the mean square error becomes larger, and the optimum filter solution differs from the Wiener solution. If the mean square error 1.27 is rewritten in the form ea) = e-Fe e (2.4) in which Eo (the greatest lower bound mean square error) is j - [Boje 2fe- 1 |x) 1? “ae |e 29 (2. 2) we observe that a new error term foreign to Wiener filtering arises, uH i° G Cob (ce and that this term depends only on the nature of h(t, y) . It will be demonstrated that EoD = 0 if and only if h(t, y) is a sure function as defined below. In that case, the optimum element of XE pecomes ee ee ee ee ee *If h(t, y) is not a sure function, we always have [x(w)] < ¢ |-wer (oo) a.e. except on sets where u(w) = 0. -~22- We shall now demonstrate the validity of the above state- ments. A sure function is defined by the existence ofa (t) such that h(t, y) = A(t) (2.4) except on a set whose product measure in t and y is zero. An equivalent statement is given by Lemma 2: Let h(t, y) be a sure function according to 2.4. If Hw) is the Fourier transform of h(t), we have H(w, y) = H(w) (2.5) except on a set whose product measure in w and y is zero. Con- versely, if 2.5 holds as stated, h(t, y) is a sure function with H(t) having FH (us) as its Fourier transform. Proof: For every 1 fe. ~ivt 1 fe. -iwt ow H(a, y) = Ae [attri dt = A. [eye dt = H(w) (2.6) {20 a \2r ° where the middle integrals are equal except fora y set of measure zero. Note that this y set does not depend on the choice of w , for with probability one | [> y) - Xo | e wt at For the converse, 1.10 and Parseval's relation yields < Jlren -Fit)|at = 0 (2.7) ° p [fats Bee 2 at = p [|x ~ Hw) 2 bes = 0 (2. 8) 3 = OD _23- which can be true only if h(t, y) = h(t) (2.9) except on a set of product measure zero. An application of the first half of the lemma shows that h(t) indeed has F(«) for its Fourier transform. We can now show how our optimum filtering reduces to the Wiener case when h(t, y) is a sure function. This is accomplished by Theorem 3: | H(w) | a G(w) a.e. if and only if h(t, y) is a sure function. Proof: If h(t, y) is a sure function we use the preceding lemma to write Go) = ElHw.y|* = [He |* = |=] eo | EE y) = |He)| (2. 10) which holds for awe. w. Conversely, if | (0 | ? = Gw) ae. eo 2 . i E]H(o,y) - Bl Hl, » | 0 (2. 11) 00 so that ile, y) = B] Hw, wv] = Ble) (2. 12) except on a product set of measure zero. ~24- Corollary: A sufficient condition that Eb = 0 is that h(t, y) be a sure function. The condition is also necessary if | w(o) | > 0 aee. The proof of the corollary is obvious. It need only be added that G(w) > 0 awe., a statement that will be proved in section three. ~25- SECTION III DERIVATION OF THE STABLE, REALIZABLE FILTER SOLUTION It was stated in section one that the optimization with re- spect to re (the subspace in we generated by the class of stable, realizable filters) would be accomplished by projecting the optimum element in A on 7H . In this section we shall determine a pro- jection operator appropriate to this purpose, so that the engineer can find an explicit solution in terms of Mm . The following theorem plays a central role in the work of \ this section: Theorem 4: Given a function F(w) such that 1. Fi(w) 2 O ace. (3. 1) 2. F(w)eL, (3.2) ~ . 3. log F(w) dw > - oc (3.3) 1 +0 00 there exists an F,(o) such that 1. [F,)| = {F(o) a.e. (3.4) 2. F, () is the L, Fourier transform of a function f(t) with f(t) = 0 for a.e. t < 0, and (3.5) co . [aoe%* dt # 0 forall o< 0. 00 co 1 log F(w) g 0 1 -t 3. log| => f(t)e ‘dt |} = > o (3.6) {2a 2 1 +0 oO Oo - 4. If f(t} = 0 for a.e. tin [90,5], then 6=0. (3.7) -26- Furthermore, the F , () specified by 3.4, 3.5, and 3.6, is a.e. unique up to a complex constant of modulus unity. Proof: Let Rw) = {F(@) 20. Then R(w)eL, , and satisfies 3.3 if F(w) does so. 3.3 implies the existence of oo o log R(x) M(z) = -4 dx o < 0 (3.8) 2 fe? tt (x-w) with z=w+io. Since 3, 8is the Poisson integral applied to the lower half plane, X(z) is harmonic in that half plane. The function F(z) may be defined by Fy) = exp] Me) + inte) |v <0 (3.9) where p(z) is the conjugate of (z). Paley and Wiener have shown* that for all o < 0 . “ c [ [eyo-i0 [Pos < [Poe < 00 (3. 10) The fact that Fy (2) is regular in the lower half plane, together with 3.10 means that there exists an f(t)eL, » i(t)=Ofora.e. t <0, such that oo Fj(o tic) = — [reeves (3.44) ou . and indeed en ee ee ee a ee ee ee ee ee * Reference (4), p. 18, (eq. 8.06) ** Reference (4), pp. 8 - 9 _27- F jo) = lism. £(t)e 7? at (3. 12) an where 3.10 enables us to define F,() = Liem. Fy + ic) (3. 13) o> Oo Thus F,() satisfies 3.5. The theorem of Fatou* applied to the half plane yields lim dw+ic) = log R(w) a.e. (3. 14) “op O so that lim | Fy + io) | = [F@) a.e. (3. 15) o> 0 — If 3.13 and 3.15 are compared, we obtain | Fy (| = (Fe) (3. 16) which verifies 3.4. To prove 3.6, we note that the construction 3.8 and 3.9 defines w(z) up to an arbitrary constant. This constant is now chosen in such a manner that p(-i) = 0. If we set w=0, o=-i, 3.9 becomes so ty 1 log F(w) F,(-i) = vo = 5 dw (3.17) 1+ - oe which, in combination with 3.11 gives 3.6 as the result. em ee me ee ee ee et ee ee ee ee ee ee * For the unit circle, this theorem may be found in Reference (5), Vol. II, pp. 147 ff. ~28- To prove uniqueness, suppose F, (0) also satisfies 3.4, 3.5, and 3.6. Then 3.11 defines a F, (2) regular and non-zero (see 3.5) in the lower half plane, and such that a aon Fy(o +is)| “aw < | F (or 2 dye (3. 18) Fj(o) = lism. F (wo + ic) (3.19) o— O The last two equations follow from the work of Paley and Wiener*. According to a theorem of Krylote!®), a F, (2) possessing the above properties has a representation F(z) = e° 7797 p(z)G(z) (3. 20) where a¢ 0, c is real, and D(z) and G(z) are given by oo j 1+xz lo F x D(z) = exp = — g Fis) dx (3.24) '- 00 and co Giz) = exp o | ~+*4. asx) (3. 22) -oo respectively. Here S(x) is monotone increasing and bounded. The ‘corresponding representation for log | F (2)| is then ao ~ -o log [FO | -ac + — - dx T 2 o + (x - w) oO log | F(z) | n (3. 23) 1 | o(f +x%) dS(x) + 5 3 Nw ee eee ee mt me me ee me me cee mee mee ce ee me me ee ere me mee ee ee ee ee ee ee ee ee oe * Reference (4), theorem V. -29- The second term on the right hand side of 3.23 is equal to log |r, (2) by 3.4 and 3.8, while the first and third terms are negative or zero. Hence log |Fyc| < log | F,(z)| (3. 24) But 3.6 and 3.11 imply that F,(-i) = F, (-i) - Therefore, the maxi- mum modulus theorem applied to log | Fy (2) | - log | F(z) | gives |F,(2| = JF, (3. 25) from which EF, (4) = eoF (2) (3.26) Then also F,(o) = eoF (w) (3.27) by virtue of 3.13. Only 3.7 remains to be demonstrated. Let f(t) =0 for a.e. t in [9, 5 ]- Then f(t) = = f(t +6) satisfies 3.4 and 3.5, and we have Bj(z) = e°"F, (2) (3. 28) by 3.11. This gives log | F, (z)| = -5y + log | F ,(2)| (3.29) with y <0. If the Kryloff representation'©) is applied to F(z) aA tog | F(z) < log |F (2) (3.30) the proof being identical with that leading to 3.24. It is evident that -~30- 3.29 and 3.30 are contradictory unless 6=0. This completes the proof of the theorem. If a function meets the conditions placed on F(w) by theorem 4, that function will be called factorable. Its factor is the (unique) function satisfying 3.4, 3.5, and 3.6. Theorem 4 is readily applied to Pl) - That Ol) is a.e. non-negative follows from the nondecreasing nature of A(w) as re- quired by t.2. The assumption that x(t} is continuous in the mean leads to Hire L, - Lastly, 1.4 is exactly condition 3 of the theorem. Thus, Pio) is factorable; we shall denote its factor by Y (e) . It is necessary that not only dw) but also G(w) be fac- torable. We note that G(w) =E | Hw, y) |? >O0O, andthat Giw)e L, by 1.12. Then G(w) is factorable if ao 128 GO) ay 5 - 60 (3.34) 1+ Lo] We shall assume 3.31 to be true, deferring its proof for the moment. The factor of G(w) will be called G,(w) . Consider now the product Pw) Glo) . Since G(w) is bounded according to 1.30, this product is in L, - Obviously, PeyGw) is a.e. non-negative. Furthermore, 1+ 1 +o Lue log BwiGe) 4, | [Bag pisces GW) gy >- oo (3.32) because of 1.4 and 3.31. Then Plo) Glo) is factorable. A straight- forward computation shows that QW, )G,) satisfies 3.4, 3.5 and ~31- 3.6, where Ye) and G, (w) are as defined above. Therefore, UY w)G, (e) is the (unique) factor of HoGw) . We now return to the proof of 3.31. We have Theorem 5: Let h(t, y) be defined as in section one. Let 1.9 and 1.10 hold. Then 3.31 is satisfied. Proof: By 1.9, Fubini's theorem applies to h(t, y) in the sense that co he] Hw) = - EB | h(t, ye*@tae = LE fel n(t, y) | eo tat (3.33) \ 20 0 {2m - o Thus, H(w) is the Fourier transform of a function zero for a.e. t <0. Also, 3.33 and the Schwarz inequality yield the result B| Hl, w | Then G(w)eL, (by 1.12) means that H(w)eL, - In short, Hw) 2 { E(w) | 7 = £ E pe | = Gv) (3. 34) satisfies the conditions of a theorem of Paley and Wiener* which is now used to assert that oa log | H(w)| 2 1 +w dw >- co (3.35) Finally, we use 3.34 to write co oo log Gla) do > 2 | 208lHOM 9, 2 co (3. 36) ol to o 1 +o which completes the proof of the theorem. * Reference (4), pp. 16-20 -32- The factorization which we have constructed is used to define a . mn a new subspace Py. . We shall define ea as the subspace (in y, a2 ) of all operators of the form A t l.i.m. | P(t)e tat J 20 A—>@ Oo p(w) = P(t)eL, (3.37) YW IG, ) We see that an element of the above type is in x4 . Furthermore, the fact that 97 is closed in y we norm is equivalent to the fact that the class of L, functions vanishing for a.e. negative argument . A is closed in L,(-« » +@) norm. Therefore, 77 is indeed a sub- space in xz . The subspace A is of great importance, for it is the sub- space of all operators involving only the past of the input process. More practically, we have Theorem 6: Let 7 be the subspace (in “Ly generated by oper- ators of the type feieh, h> 0} . Then a G H =H (3. 38) Proof: The proof of this theorem is omitted, since our theorem forms a part of a theorem of Doob. The reader is referred to Re- ference (3), pp. 586-587 (theorem 5.2). We now assert that 3.37 is a representation of operators in an . The validity of this statement is a result of N — Theorem 7: ‘™ = 771 . Proof: It suffices to prove that ™ = real (see theorem 6). If WA is the family of step functions (in the time domain) defined in ~33- theorem 1, we have mod by that theorem. But also, any step function is in L so that Le mm. Therefore* zal = _L<- . a To show that NaH, we consider eg iwh h>O. There exists a step function S_(t) with the Fourier transform e7iwh sin wa Dn - Now A l|e*e _ eo twh sin oa | 2 < Ho) Glo) | 1 - sina 2 des cA -A (3.39) $2 + PiBerciene rove) ea which can be made as small as desired by choosing A sufficiently ~s — — large, andthen a sufficiently small. Hence, 7 Oude = 7H . ww Conversely, a H. For we observe that ke m has the form <o 1 ~iwt kiw) = =e K(t)e dt (3.40) V ow | so that k(w) is the limit of linear combinations of operators tah, h >0O. Given any finite w interval and any & >0, we can find a linear combination Ene, et O<t, << t, <+St cm, such that n | kK) - doce | < € (3.41) over the chosen finite interval. At the same time, the c; are — * 774) = aL has already been proved by theorem 1; however, that proof required (w) to be essentially bounded. ~34. such that for every w n ; n oo Ee ; y c,et5| < | (0) | + > Is 2 fc a (3.42) 4 ° 1 & An argument similar to 3.39 completes the proof that Me . The representation 3.37 of operators in ~ makes it easy to project A on Za) . In fact, Theorem 8: The projection of re Ww on m is given by P(r) = plo) = . x Te = PP Se W, ()G, (&) (3.43) A B l.i.m. et} itm. YW, (p)G, (p) (pe do | dt A-° B-o fe) -B Proof: It is convenient to call R(t) the inverse L, Fourier transform of YY, (©) G, (w)r(o) . Then 3.43 becomes A p(w) = leism. R(t)e?* at (3.44) 1 Y2r Yi W)G,o) Awe oO We also define a q(w) such that r(w) = p(w) + q(w) . This operator is given by R(t)e2@* at (3.45) 4 . Q(o) = lLi.m. Var Y )G, ) Ae Jd, To show that p(w) is the desired projection, we prove that p and q is the orthogonal decomposition of r with pe and A aA qe™m+ . That pe ™m is obvious from the definition 3.37. For -35- q , we shall demonstrate that q is orthogonal to every element of e274 . Given any ke there exists a K(t)eL,, K(t) = 0 for t <0, such that A k(w) 1 l.i.m. / K(t)e dt (3.46) iw YG, o) Are d Also, we write q(w) as A q(w) ! l.i.m. Q(tye Fat (3.47) ee &, (w)G,(w) Are where now Q(t) = (3. 48) R(t) t <0 An application of Parseval's relation may be used to show that (k.q) = | K(yQHat - (3.49) But for a.e. t at least one of K(t) and Q(t) are zero. It follows that (k,q) =0, as was to be shown. If we project the optimum operator in Ks (as given by 1.28 —_ i.e. rw) = we fo)uto) ) on ari the optimum operator in TH is found to be* A (w) = 1 1. -iw t eS Fa BCG) ye J ° | (p)E(p)u(p) Ee [ en it we [a (3. 50) G1 (ey) ee ee a ee a ee ee er rr ere rere en een ee eee * That the optimum operator in ‘A is unique, a.e. is a consequence of the unique nature of an orthogonal decomposition. See Reference (7). -36- in accordance with the preceding theorem. If this operator is used, the corresponding mean square error is computed to be Ew) - Eo, = |p-z |]? = fie-a--]}? = fal]? eso by 1.29 and theorem 8. We substitute for Eon from 2.2 and evaluate | q | to obtain 2 _ 2 ls(0) |? Emin = 7 - eT(0) | (3. 52) (P)H(p)u(p) . v A> oo Gta in which we have adopted the notation €* . = inf E “ey | kem This completes the solution of the filter problem in the ab- sence of noise inputs. The more general case of an input consisting of both signal and noise is treated in the next section of this paper. ~37- SECTION IV GENERAL PREDICTION AND FILTERING We shall now consider filtering and prediction of inputs con- sisting of both signal and noise when each of these is subjected toa random operator. The random operators applied to the signal and noise may be the same or different, correlated or uncorrelated. Let random operator H(w,y) be applied to the signal, and M(w,p) be applied to the noise. Then the input to our filter is oo 1 Z(t, y,~) = == H(w, y)dX_(w) + Mw, p)dX_ (w) (4. 1) en _ s n where the subscript s refers to signal and n refers to noise. The integrals, of course, are l.i.m. as before. Let the signal and noise inputs x(t) and x<_(t) be uncor- related, and ascribe to each of them the properties of x(t) in section one. The total spectral density of the input to the filter is then Pe) = Geo) 6,0) + Le), () (4.2) in which Li{w) = E | Mio, ) \° . We. wish iw) to be factorable, XY 1) being its factor. dw) meets conditions one and two of theorem 4. To satisfy con- dition three, it is necessary and sufficient that at least one of $5) and $e) satisfy this condition. Physically, this means that either signal or noise must be a non-deterministic process if signal plus noise is to be non-deterministic. ~38- As before, we define a Hilbert space BL . The inner product is now (r,s) = | Plw)r(a\sfo)du (4. 3) The work of sections two and three now follows with little, if any, modification. The optimum operator in nh becomes A _ 4 . -iwt BE ESS (4. 4) B (pP)H(p)u(p) l.i.m. / $, oP ao | at Boe XA with a corresponding mean square error oo 2 min ~ (0) | we) | 1 - Pe) cho ™ (4. 5) , © O,H@nw) 4, | + za l.i.m. e dw dt Bo -B {(e) “oO We remark that a = 771 = ra remains unchanged; the manifolds of operators do not depend on the input process. Of course, appropriate simplifications of the above equations yield the conventional Wiener filter formulae, or the filter derived in the previous section. -39- SECTION V THE RANDOM GAIN AMPLIFIER The operator H(w, y}) has been made subject to certain assumptions (in section one) which exclude an important class of applications. We refer here toan H(w,y) of the form H(w, y) = ¥ . (5. 1) which has the physical interpretation that the signal (or input) is passed through an amplifier with random gain, or that the signal is derived from a transducer with a random scale factor. Actually, operators such as 5.1 cause little theoretical dif- ficulty. The restrictions of section one had the purpose of insuring that E | H(w, y) | 2 be factorable, and that the process into the filter be of finite mean square. These two requirements are satisfied if E ly |? < co ., For convenience, we eliminate the case y= 0 with probability one as trivial. Now let Ey = ¥ , (5.2) and Ely|° = y" (5. 3) The general case of section four is not much simplified. Consider, however, the somewhat simpler assumption that both signal and noise are passed through the same random gain amplifier. Thenin M w -40- k(w) = SE kyle) (5.4) 2 where ky, (o) is the ordinary Wiener filter, viz. ,(o)u(p) ky (w) - it g wpa eltP ap (5.5) W 27 xX 4 ) a Yl) 5 the integrals being 1.i.m. where the factor X ,) constitutes ‘the factorization of $.() + (0) . The mean square error corresponding to this optimum filter is 20 12 Rin $ (0) € in = | 500 jr | 1 - ve fF.) + ?) ceo (5. 6) dy 2 + ly fase ef ay | at 2 oo %1©) We note that if y=0 the optimum operation becomes k(w) = 0, and the mean square error is J Fetorfator|2as - This is in accordance with the intuitive idea that if a number is equally likely to be positive or negative we guess zero if the penalty for an incor- rect guess is the square of the difference between the actual number and the guess. -41- SECTION VI PREDICTING AND LAGGING FILTERS One of the most important applications of optimum filter theory lies in predicting the value of a random input process x(t) (perhaps corrupted by noise) over some prediction time we shall call a. This type of prediction has been used where some component or natural phenomena introduces a lag between per- ception of x(t) and decisions made or action applied as a result. Typical examples include anti-aircraft gun computers which must account for the motion of the target while the shell is on its way, | and an automatic landing technique to be used in connection with an aircraft carrier in a stormy sea. At the other extreme, lagging filters are useful also. Such a filter has the advantage that as the lag between output and input is increased the mean square error is lessened. For many pur- poses, alag is quite acceptable and the delay in obtaining the out- put is gladly exchanged for enhanced accuracy. We shall give two examples where lagging filters are appropriate. On missile test flights performance data is telemetered to a ground station where it is reduced at some later date from a tape recording made during the flight. If the playback process involves a (reasonable) delay time no harm is done. The same argument may be applied to the playback of recorded music. In fact, a delay or lagging filter is applicable wherever the data in question need not be available instantaneously. ~42- It was stated in 1.7 and the text of section one that ; lwa ujo) = € (6. 1) constitutes a lead or lag operator. If a is positive, u(w) cor- responds to lead or prediction. Analogously, u(w) is a delay or lag operator whenever a< 08 The physical interpretation of u(w) as in 6.1 is that the filter output y(t) should lead or lag the random process x(t) by a time interval | a | » the choice of lead or lag being determined by the signofa. If u(w) is given by 6.1, the minimum mean square error in W is called € nin!) » and the corresponding operation in w will be denoted by k_(w) . These are given from 4.5 and 4.4 by ” 2 2 (w) | H(w) Ein!) ,() [ ~ i | | cho J. PG) + > W)LO) ° 2 (6. 2) Hoy 2 4 oH ” i(t + a)w dt {20 C{Y and oo re) , (o)H(p)—«z ko) = s : et at D6(eVROY elt +a)Pa. (6.3) a -@ -We may now assert 2 Theorem it: E nin’? is a monotone nondecreasing function of a. ee ee ee er eee a ee eee ei ee ee eee eee eer * Our concept of a lag or delay may differ from that sometimes _found in electrical engineering usage. We mean a time delay, i.e., a delay independent of frequency. The corresponding phase delay is then proportional to frequency. ~43- Also 2 ,() | He) | int] Ec) | = $.(o) 1 - dw (6. 4) |. (Glo) + Oo) Lo) and oo 2 = eu] Ernin() | = | $,(ode (6.5) -o Proof: From the work of sections one and three, the mean square error in question may be expressed as < S 3 min) = glib + | kot | (6. 6) where r(w)eH, is 6 WH W ea OG) + BLO) and © 2 2 Bo [1 | Ey b,(o 1 - dw (6. 8) 8 | a $ (WIG) + $ (LW) The theorem is proved if we show that | r-k, |" is monotone and 0 (6.9) u im ols - ¥a I eel IP = PP (6.10) i It is convenient to rewrite 6.3 as — ” 6 _(p)He) k () = s—2 eta, f 8 a 27 % ,() 5 - Ze) oP ap (6. 11) ~44- by a change of variable t+a=r. The integrals in 6.11 are to be taken l.i.m. In the same sense, the substitution of 6.7 into 6.14 yields | eg iety ee (p)E(p) et tp k_&) = rw) oe AO) dp (6. 12) from which a : | = - a |“ = | Re) |“ ae (6. 13) -~a R(t) being the inverse Fourier transform in L, of % , (o) x (0) . It is clear that || r-k, | is monotone. In addition, it follows from 6.13 that a Lim | r-k ||" | Rio | * at = 0 (6. 14) oe -os -B and oo oo 24. | Te 2 lima || r-k aie | R() | dt = i | % ode) | dw = | r | ““ ™ - (6.15) thereby completing the proof of the theorem. A supplementary result of 6.15 is that Lin | Kk, | = 0 (6. 16) or vi [et | = 0 : (6.17) This result, of course, is also true in the case of the conventional Wiener filter. -45- SECTION VII AN EXAMPLE: RANDOM DELAY TIME Every communication system involves delay between the transmission and reception of messages. Inevitably, the information is propagated through some medium so that there is a lag directly proportional to the distance between transmitter and receiver, and inversely proportional to the propagation velocity. Two persons conversing together experience such a delay without inconvenience. Over larger distances, and in more critical applications, the propagation delay can greatly reduce the value of the data received. In that case, Wiener filter theory yields the re- sult that the best estimate (in the mean square sense) of the present data is achieved by passing the delayed data through a Wiener opti- mum prediction filter whose prediction time is exactly equal to the propagation delay. The transmission range and therefore the delay time is likely to vary where mobile equipment is used, or where a target is tracked. A variable prediction time filter is then called for. However, range measurements may be difficult, or a fixed pre- diction filter may be preferable for reasons of simplicity or economy. The optimum fixed prediction filter must then represent a compromise over the transmission ranges which will be en- countered in practice. | The optimum fixed prediction filter referred to above is easily derived by the methods of this paper. Let y be the pro- pagation time (delay), and let F(y) be its probability distribution -46- function. Then ‘Hiv, y) = ey y 20 (7. 1) from which Ho) = i eFY ary) = Ue) (7.2) where ‘? (w) is the characteristic function of F(y) , as generally defined. We also have, by an obvious calculation Gt) = 1 (7.3) If there is no noise present the optimum operator in mR is 09 rv.) kk(w) = SaTAOE [eo [sore dp (7. 4) with the integrals being defined 1.i.m. The mean square error becomes ES % 2 mas BoLs = [to]? Je (7.5) 2 dt + 52 | | [ WZ (o)#(p)e" Pap If the operation 7.4 were not applied to the signal the error due to the delay is calculated to be 2 oa eo —€ (1) = ESE - jroo]? fo + | Be) =P (2) | (7.6) which must be larger than 7.5; this difference in mean square -47- error is e . 2 * > (1)- Ew = [ e) (ew) |" do (7.7) 00 where k({w) is given by 7.4. It should be noted that k(w) = 1 a.e. if and only if ‘P (w) ® 1, that is, the delay is zero with probability one. A classical example of Wiener optimum prediction may be described as follows: an anti-aircraft battery is equipped with a radar tracking system which determines the position and velocity of enemy aircraft. In one dimension, the aircraft velocity is stationary with spectral density 1 +o The time required for the shell to attain the altitude of the aircraft is y (y is fixed for the present) . If the anti-aircraft shell is fired at time t we must be able to predict the aircraft position at time t+y. In other words, we require a prediction filter to com- pensate as nearly as possible for the delay y inherent in the system. Unfortunately, the aircraft position z(t) is not a stationary process, so that the techniques of this paper are not immediately applicable. However, we shall show that z(t +y) - z(t) is stationary. It suffices to work with z(t +y) - z(t) since the radar measures z(t) directly and exactly. Then our best estimate of z(t + y) is obviously attained by adding the best estimate of * The appropriateness of this form of spectral density is justified in Reference (8), pp. 300 - 304. _48- z(t + y) - z(t) to the known value of z(t). In what follows, our choice of notation will be consistent (insofar as possible) with the notation used in previous sections. Accordingly, let x(t) be the stationary process of aircraft velocity. To investigate the properties of z(t +y) - z(t), let - {ar -y¢t <0 h(t) = (7. 8) 0 t > 0 It is clear that h(t)eL, - Indeed, h(t) has the L, Fourier trans- form Lt - ete H(w) = ————— (7.9) tw so that some stationary process, say s(t), has the representation s(t) = —e | Hlw)aX(u) (7. 10) QT co the integral being defined l.i.m. We also have from 1.8 eo s(t) = & x(t - r)h(r)ar (7. 41) x(t - tT)dr = a(t + y) - a(t) <a In this application, the input available to the optimum filter is the velocity, x(t). The filter output will be denoted by w(t) , as in the first section. The optimum operator in %L is designated by r*(w) , and the optimum operator in ‘” is called k*(w) . Then E(tsk*) = w(t) - [ 2 +y)- ate) | (7. 12) -49- w(t) having the representation a (tt) = == k*(w) dX(w) (7. 13) WO SE [ srewe The mean square error becomes e (Ic**) - [a (co) | H(o) ~ ko) 1% deo (7. 14) and this error is minimized by oo © . . _ i -iwt i -e YP itp o -oo The formula 7.15 is identical with that of Wiener. If the spectral density Plo) is rational the inner integral in 7.15 may be evaluated by a contour integration in the upper half plane. The result is the sum of terms of the form tS e7 ant » so that the second integration can be done by parts or through use of readily available integral tables. In our case, Plu) = 2 so that the above procedure yields k*(o) = 1-e (7. 16) We note particularly that for small y the use of the approximation 1-e Ya y implies that when the prediction interval is short the best estimate of z(t + y) - a(t) is obtained by assuming that the aircraft continues at its present velocity x(t) throughout the prediction interval y. ~50- The filter 7.15 is indeed optimum if it can be assumed that all enemy aircraft will fly at the altitude corresponding to shell flight time y. Since the enemy is not expected to be so obliging, we. must take into account the possibility that aircraft will come in at various altitudes. This situation can be dealt with by providing an adjustable filter which varies in accordance with the flight time of the shell as calculated from the aircraft altitude measured by the radar set. However, a fixed filter may be preferable for reasons indicated at the beginning of this section. Such a filter attempts to predict z(t + y) - z(t) where the aircraft altitude and thus y are random variables. In analogy to 7.9 _ Lyw Hlw,y) = —3S— (7. 17) and as in 7.14 2 “ ; € (k) = ¥ Pl) fH(o, y) - k(w) deo (7. 18) where u denotes expectation with respect to y. If y assumes some given value with probability one, 7.18 reduces to 7. 14 and the optimum filter is given by 7.15 as has also been shown by Wiener. In the more general case (where the aircraft may fly at more than one altitude) we suppose that y is associated with the probability distribution function F(y). The only restriction placed on y and F(y) are that y 2 0 with probability one, and that F(y) is such that Y dF(y) < © (7.19) ~51- for some 6 >0. It is convenient to rewrite 7.18 as 2 - 42 2 € (k) = [ Gere - += Fis) dw + E wip (7.20) in which 2 © Dw) Ew = ca E yoo? fa (7. 24) Here ‘Pf (w) is again the characteristic function belonging to F(y) . The computation of 7.20 and 7.21 is quite similar to the work of section three. But in addition it is also necessary to demonstrate the integrability of the integrands of 7.20 and 7.21. Since | $Y (w) | <1 the only question remaining is that of the behavior of 2 1 -% (o) and 4 Hol near w = O Let w w ° ea : free = m (7.22) =] 2 2 Then At) and A -hwol are integrable, for in the vicinity of w =0, the expansion 1 +86 Yio) = 1 + iwm + Ow ) (7.23) is valid*. It should be noted that the condition 7.19 is a sufficient but not necessary one for 7.20 and 7.21 to exist. For example, the Couchy distribution has no finite first moment; yet the characteristic 1 - ‘P(w) 5 is bounded near w=0. Of course, function is such that *This is a well known theorem in probability theory. See Reference (9), p- 199. ~52- any distribution of y is acceptable if, for any A>0O, A / Pw). oa ew [e) we . Proceeding as in section three, we see that the element in — H which minimizes the mean square error 7.20 is given by Ke) = ETH) [: wae 7B (o) A= #O) 45 Fle) oi ap (7.24) -~m% the integrals being taken as l.i.m. If the aircraft velocity spectrum Pw) = ar, as was assumed in the calculation of the conventional Wiener filter 7.16 the optimum operator 7.24 specializes to oo k(o) = 1-%) = 1- e” YdF(y) (7. 25) ° This result is easily established by contour integration of the first integral in 7.24. The upper half plane is used since the integral along a sufficiently large semi-circle converges to zero. In this regard it should be noted that (2) = eV aR(y) (7. 26) is regular and bounded in the upper half plane*, It is instructive to note the increase in the mean square error when the Wiener filter k*(w) is used in place of the k(w) of 7.24. For the latter we obtain * This fact may be established by a modification of a proof found in Reference (10), p. 8. -53- En - [2 [ -[t09? ]os [e wlt-Pe] - 4, am e da (7.27) 2 dt ° n 1 27 = OD which is analogous to 3.47 and is derived in an identical manner. Further calculations then lead to an equation like 1.28, or 2- 2 “ E(ky)- E = | Plo) | }*(o) ~ lw) [* de ™. (7.28) | “yg LP - ev | itu iw 2 e dt oO 1 ~ Or oe In the particular case Pio) = aw) we have k*¥(w) and k(w) from 7.16 and 7. 24, respectively. It is seen that 7.28 becomes 2 - 2 -a ee - e YdF(y) 2 Ese - € w)dw 7.29 in x | Pe (7. 29) where the Wiener filter k*(w) has been designed on the assumption that y=a. If the mean square error is defined as in this paper the Wiener filter is seen to be optimum if and only if y = a with probability one. Then also k(w) = k*(w) a.e. 10. -54- REFERENCES Kolmogorov, Bull. ‘Acad. Sci. U.S.S.R. Ser. Math. (1941), 5, pp. 5-14 Wiener, "Extrapolation, Interpolation, and Smoothing of Stationary Time Series", Wiley and Sons (1949) Doob, "Stochastic Processes", Wiley and Sons, Inc. (1953) Paley and Wiener, "Fourier Transforms in the Complex Domain", AMS Colloquium Publication Volume XIX, American Mathematical Society (1934) Bieberbach, "Lehrbuch der Funktionentheorie" (2 volumes), Chelsea Publishing Co. (1945) Kryloff, Rec. Math. (Mat. Sbornik) N. S. (1939), 6, p. 106 Halimos, "Introduction to Hilbert Space and the Theory of Spectral Multiplicity", Chelsea Publishing Co. (1951) James, Nichols, and Phillips, "Theory of Servomech- anisms", M.I.T. Radiation Laboratory Series No. 25, McGraw-Hill Book Co., Inc. (1947} Loeve, "Probability Theory", Van Nostrand Co., Inc. (1955) Bochner and Chandrasekharan, "Fourier Transforms", Annals of Mathematics Studies No. 19, Princeton Uni- versity Press (1949