In my last post I motivated Fourier's series and how they naturally arose out of boundary value problems for partial differential equations. If you recall, I pointed out a fundamental problem with Fourier series, namely that they implicitly assume that one is working on a finite domain. This came from the fact that we were considering the heat equation on a finite domain. What if instead we were concerned with how temperature would change over an infinite domain? Clearly, the separation of variables technique is useless. We can easily rule out the case of \( \lambda < 0 \) because either one of the exponentials diverges at \(\pm\infty\), thus so does \( u \) and this is wholly unphysical. The case of \( \lambda = 0 \) has the same issues. The case of \( \lambda > 0 \) has different issues however. In this case, the solutions are sines and cosines. Such functions cannot obey any sort of boundary condition at infinity since they are oscillatory in nature. Thus we must discard the separation of variables technique and thus Fourier series. At an intuitive level, this means that on the whole line, we should expect that spatial and temporal variables get mixed.
It should not be unexpected that we cannot use Fourier series since in the formula for the coefficients of the Fourier series appears the length of the interval; on an infinite interval, the Fourier coefficients do not make immediate sense. One can however consider what happens as the length of the interval over which the Fourier series is defined goes to infinity. What will emerge from this is known as the Fourier transform which will be the subject of this blog post. From Fourier series I will make my way to the Fourier transform and from there I will make my way to the short-time Fourier transform.
It should not be unexpected that we cannot use Fourier series since in the formula for the coefficients of the Fourier series appears the length of the interval; on an infinite interval, the Fourier coefficients do not make immediate sense. One can however consider what happens as the length of the interval over which the Fourier series is defined goes to infinity. What will emerge from this is known as the Fourier transform which will be the subject of this blog post. From Fourier series I will make my way to the Fourier transform and from there I will make my way to the short-time Fourier transform.
In (2) in my last post, I considered a one-sided interval but if we wish to consider what happens to Fourier series on the whole real line, we should adapt this to include the two-sided interval \([-L,L]\). This can be done with a simple modification:
$$ f(x) \sim \sum_{n=-\infty}^{\infty} \left(\frac{1}{2L} \int_{-L}^L f(y) e^{-i\frac{n\pi}{L}y} dy \right)e^{i\frac{n\pi}{L}x}.$$
Note that I am not claiming that $f$ and its Fourier series are the same in the previous expression. I am nearly saying that $f$ is in some way related to the Fourier series on the right. Given some conditions on $f$, we could say that they are indeed equal or perhaps equal "almost everywhere" (i.e. they disagree on a fairly negligible set).
We can clearly see that the wavelength \( \lambda_n \) of the waves is given by \( \lambda_n = \frac{2L}{n} \) and the wavenumber \( k_n \) is given by \( k_n = \frac{n\pi}{L} \). From this it follows that \( \frac{1}{2L} = \frac{k_{n+1}-k_n}{2\pi} \). We will write \( \Delta k = k_{n+1}-k_n \) and note that as \( L \rightarrow\infty, \Delta k\rightarrow 0 \).
(Those who know Fourier transform theory should see things start to come together at this point.)
Putting the previous pieces together we have
This all holds on a finite domain but we are interested in the case of $L\rightarrow\infty$ and so we will take this limit. At this point, some liberty will be taken with mathematics but it can all be made very precise. This is only to motivate the Fourier transform.
$$f(x) \sim \lim_{L\rightarrow\infty}\frac{1}{\sqrt{2\pi}}\sum_{n=-\infty}^{\infty}\left(\frac{1}{\sqrt{2\pi}}\int_{-L}^Lf(y)e^{-ik_ny}dy\right)e^{ik_nx}\Delta k.$$
The outer sum looks very similar to a Riemann sum if we make the substitution
$$F_L(k) = \frac{1}{\sqrt{2\pi}}\int_{-L}^Lf(y)e^{-iky}dy.$$
We then have that
$$f(x) \sim \lim_{L\rightarrow\infty}\frac{1}{\sqrt{2\pi}}\sum_{n=-\infty}^{\infty}F_L(k_n)\Delta k.$$
Since \( k_n = \frac{n\pi}{L} \) and \( n \) ranges over all integers, we have that as \( L\rightarrow\infty \), the continuous variable \( k \) corresponding to the Riemann sum above ranges over all of \( \mathbb{R} \). Thus
$$f(x) \sim\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty}F_{\infty}(k) dk = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\left(\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}f(y)e^{-iky}dy\right)e^{ikx}dk.$$
The last set of equalities is a bit loose mathematically but there are ways to assure convergence which I shall not discuss here. From the discrete sums in Fourier series we have now established continuous sums (integrals) in the Fourier transform. The part within the parenthesis is known as the Fourier transform of \( f \) and the integral outside of it (with the coefficient out front) is known as the inverse Fourier transform. Taking the place of the Fourier coefficients is the Fourier transform of \( f \); this gives a very clear interpretation for the Fourier transform of \( f \): it is a measure of how prevalent a certain wavenumber is within your function. With this, we make the following definition
Definition. The Fourier transform of a function $f$, when it exists (i.e. when the integral below makes sense), is given by
$$\mathcal{F}f(k) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{-ikx}f(x)\, dx.$$
An alternative notation for the Fourier transform is $\hat{f}$. I prefer the former notation since it keeps the operator flavor one might like. One may notice that I did not state what kind of properties the function $f$ must satisfy for its Fourier transform to make sense.
Instead of discussing the technicalities of the Fourier transform, I will give some elementary properties with an eye towards future work.
Suppose the Fourier transform of $xf$ and $f'$ make sense and $L_xf(y) = f(y-x)$, then
\begin{align*}
\mathcal{F}(f')(k) &= ik\mathcal{F}f(k) \\
\mathcal{F}(xf)(k) &=-i \mathcal{F}f'(k) \\
\mathcal{F}(L_xf)(k) &= e^{-ikx}\mathcal{F}f(k).
\end{align*}
Two nontrivial properties of the Fourier transform is how it behaves with regards to convolutions and how it interacts with the function $e^{-x^2}$.
Suppose $f,g$ have finite integral over the whole real line, then define
$$(f*g)(x) = \int_{-\infty}^{\infty} f(y)g(x-y)\,dy.$$
It is not hard to check that
$$\mathcal{F}(f*g)(k) = \sqrt{2\pi}\mathcal{F}f(k)\mathcal{F}g(k).$$
(To see this, just apply a clever form of $1$ and interchange the integrals.) Let's now consider the Fourier transform of $e^{-x^2}$. Let $f$ be defined by
$$f(k) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-ikx} e^{-x^2}\,dx.$$
Then
$$f'(k) = -\frac{i}{\sqrt{2\pi}} \int_{-\infty}^{\infty} xe^{-ikx} e^{-x^2}\,dx.$$
However we can recognize that $-xe^{-x^2}$ is simply $\frac{1}{2} \left(e^{-x^2}\right)'$ and so
$$f'(k) = \frac{i}{2\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-ikx} \left(e^{-x^2}\right)'\,dx.$$
This is just the derivative of the Fourier transform of $e^{-x^2}$ and so by our above property,
$$f'(k) = -\frac{k}{2}\int_{-\infty}^{\infty} e^{-ikx} e^{-x^2}\,dx = -\frac{k}{2} f(k).$$
The solution to this differential equation is nothing more than $f(k) = e^{-k^2/4}$.
With this in mind, we can solve the heat equation on the whole real line. Let's consider the heat equation given by
$$ \frac{\partial u}{\partial t} = \kappa\frac{\partial^2 u}{\partial x^2}\tag{1}$$
subject to the initial condition $u(x,0) = \phi(x)$. Just like we employed Fourier series to solve the heat equation on a finite domain, we can employ the Fourier transform to solve the heat equation on the whole real line. Taking the Fourier transform of (1) with respect to $x$ gives the equation
$$ \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} e^{-ikx} \frac{\partial u}{\partial t}(x,t)\,dx = \kappa \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-ikx} \frac{\partial^2 u}{\partial x^2}(x,t)\,dx. $$
Making use of the properties of the Fourier transform and by pulling the $\frac{\partial}{\partial t}$ out of the integral on the left hand side, this gives the equation
$$ \frac{\partial}{\partial t}\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-ikx} u(x,t)\,dx = \kappa (-ik)^2\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{-ikx}u(x,t)\,dx.$$
Equivalently, if we let $v(k,t) = \mathcal{F}u(k,t)$, then
$$ \frac{\partial v}{\partial t} = - \kappa k^2 v.$$
Dividing both sides by $v$ gives
$$\frac{1}{v}\frac{\partial v}{\partial t} = -\kappa k^2.$$
Or equivalently,
$$\frac{\partial}{\partial t} \log v = -\kappa k^2.$$
Partially integrating both sides gives that
$$\log v = -\kappa k^2 t + f(k).$$
for some as-of-yet unknown $f$. Or equivalently,
$$v(k,t) = g(k)e^{-\kappa k^2t}.$$
Thus far, we have no information about $g$. However by noting that $v(k,0) = \mathcal{F}u(k,0)$, this says that
$$v(k,0) = \mathcal{F}\phi(k) = g(k).$$
Thus we have a "nice" expression for $v$:
$$v(k,t) = \mathcal{F}\phi(k)e^{-\kappa k^2t}.$$
We can realize $e^{-\kappa k^2 t}$ as being nothing more than the Fourier transform of $\frac{1}{\sqrt{2\kappa t}} e^{-x^2/(4\kappa t)}.$ (Check this yourself!) Thus,
$$v(k,t) = \mathcal{F}\phi(k)\mathcal{F}\left(\frac{1}{\sqrt{2\kappa t}} e^{-x^2/(4\kappa t)}\right).$$
By our handy-dandy "convolution theorem" above, we can realize this as
$$v(k,t) = \frac{1}{\sqrt{4\pi\kappa t}}\mathcal{F}(\phi*e^{-x^2/(4\kappa t)}).$$
Since $v(k,t) = \mathcal{F}u(k,t)$, we see that by inverting the Fourier transform,
$$u(x,t) = \frac{1}{\sqrt{4\pi \kappa t}} \int_{-\infty}^{\infty} \phi(y) e^{-(x-y)^2/(4\kappa t)}\,dy.\tag{2}$$
Here, I have assumed something about the Fourier transform that I did not prove but it is true. The Fourier transform has the property that if $\mathcal{F}u = \mathcal{F}v$, then $u=v$. This is definitely not true for any arbitrary function but it happens to hold true for the Fourier transform (under certain assumptions). Thus the general solution to the heat equation is of the form (2); given the initial conditions and diffusion constant $\kappa$, the heat distribution at any later time and location can be immediately solved by (2). This concludes our foray into Fourier transforms. This is woefully alack of rigor but to keep the post relatively readable for the uninitiated reader, the rigor had to be foregone. A well-versed reader however would know under what assumptions each of our hypotheses hold.
$$ f(x) \sim \sum_{n=-\infty}^{\infty} \left(\frac{1}{2L} \int_{-L}^L f(y) e^{-i\frac{n\pi}{L}y} dy \right)e^{i\frac{n\pi}{L}x}.$$
Note that I am not claiming that $f$ and its Fourier series are the same in the previous expression. I am nearly saying that $f$ is in some way related to the Fourier series on the right. Given some conditions on $f$, we could say that they are indeed equal or perhaps equal "almost everywhere" (i.e. they disagree on a fairly negligible set).
We can clearly see that the wavelength \( \lambda_n \) of the waves is given by \( \lambda_n = \frac{2L}{n} \) and the wavenumber \( k_n \) is given by \( k_n = \frac{n\pi}{L} \). From this it follows that \( \frac{1}{2L} = \frac{k_{n+1}-k_n}{2\pi} \). We will write \( \Delta k = k_{n+1}-k_n \) and note that as \( L \rightarrow\infty, \Delta k\rightarrow 0 \).
(Those who know Fourier transform theory should see things start to come together at this point.)
Putting the previous pieces together we have
$$f(x) \sim \frac{1}{\sqrt{2\pi}}\sum_{n=-\infty}^{\infty}\left(\frac{1}{\sqrt{2\pi}}\int_{-L}^Lf(y)e^{-ik_ny}dy\right)e^{ik_nx}\Delta k.$$
This all holds on a finite domain but we are interested in the case of $L\rightarrow\infty$ and so we will take this limit. At this point, some liberty will be taken with mathematics but it can all be made very precise. This is only to motivate the Fourier transform.
$$f(x) \sim \lim_{L\rightarrow\infty}\frac{1}{\sqrt{2\pi}}\sum_{n=-\infty}^{\infty}\left(\frac{1}{\sqrt{2\pi}}\int_{-L}^Lf(y)e^{-ik_ny}dy\right)e^{ik_nx}\Delta k.$$
The outer sum looks very similar to a Riemann sum if we make the substitution
$$F_L(k) = \frac{1}{\sqrt{2\pi}}\int_{-L}^Lf(y)e^{-iky}dy.$$
We then have that
$$f(x) \sim \lim_{L\rightarrow\infty}\frac{1}{\sqrt{2\pi}}\sum_{n=-\infty}^{\infty}F_L(k_n)\Delta k.$$
Since \( k_n = \frac{n\pi}{L} \) and \( n \) ranges over all integers, we have that as \( L\rightarrow\infty \), the continuous variable \( k \) corresponding to the Riemann sum above ranges over all of \( \mathbb{R} \). Thus
$$f(x) \sim\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty}F_{\infty}(k) dk = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\left(\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}f(y)e^{-iky}dy\right)e^{ikx}dk.$$
The last set of equalities is a bit loose mathematically but there are ways to assure convergence which I shall not discuss here. From the discrete sums in Fourier series we have now established continuous sums (integrals) in the Fourier transform. The part within the parenthesis is known as the Fourier transform of \( f \) and the integral outside of it (with the coefficient out front) is known as the inverse Fourier transform. Taking the place of the Fourier coefficients is the Fourier transform of \( f \); this gives a very clear interpretation for the Fourier transform of \( f \): it is a measure of how prevalent a certain wavenumber is within your function. With this, we make the following definition
Definition. The Fourier transform of a function $f$, when it exists (i.e. when the integral below makes sense), is given by
$$\mathcal{F}f(k) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{-ikx}f(x)\, dx.$$
An alternative notation for the Fourier transform is $\hat{f}$. I prefer the former notation since it keeps the operator flavor one might like. One may notice that I did not state what kind of properties the function $f$ must satisfy for its Fourier transform to make sense.
Instead of discussing the technicalities of the Fourier transform, I will give some elementary properties with an eye towards future work.
Suppose the Fourier transform of $xf$ and $f'$ make sense and $L_xf(y) = f(y-x)$, then
\begin{align*}
\mathcal{F}(f')(k) &= ik\mathcal{F}f(k) \\
\mathcal{F}(xf)(k) &=-i \mathcal{F}f'(k) \\
\mathcal{F}(L_xf)(k) &= e^{-ikx}\mathcal{F}f(k).
\end{align*}
Two nontrivial properties of the Fourier transform is how it behaves with regards to convolutions and how it interacts with the function $e^{-x^2}$.
Suppose $f,g$ have finite integral over the whole real line, then define
$$(f*g)(x) = \int_{-\infty}^{\infty} f(y)g(x-y)\,dy.$$
It is not hard to check that
$$\mathcal{F}(f*g)(k) = \sqrt{2\pi}\mathcal{F}f(k)\mathcal{F}g(k).$$
(To see this, just apply a clever form of $1$ and interchange the integrals.) Let's now consider the Fourier transform of $e^{-x^2}$. Let $f$ be defined by
$$f(k) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-ikx} e^{-x^2}\,dx.$$
Then
$$f'(k) = -\frac{i}{\sqrt{2\pi}} \int_{-\infty}^{\infty} xe^{-ikx} e^{-x^2}\,dx.$$
However we can recognize that $-xe^{-x^2}$ is simply $\frac{1}{2} \left(e^{-x^2}\right)'$ and so
$$f'(k) = \frac{i}{2\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-ikx} \left(e^{-x^2}\right)'\,dx.$$
This is just the derivative of the Fourier transform of $e^{-x^2}$ and so by our above property,
$$f'(k) = -\frac{k}{2}\int_{-\infty}^{\infty} e^{-ikx} e^{-x^2}\,dx = -\frac{k}{2} f(k).$$
The solution to this differential equation is nothing more than $f(k) = e^{-k^2/4}$.
With this in mind, we can solve the heat equation on the whole real line. Let's consider the heat equation given by
$$ \frac{\partial u}{\partial t} = \kappa\frac{\partial^2 u}{\partial x^2}\tag{1}$$
subject to the initial condition $u(x,0) = \phi(x)$. Just like we employed Fourier series to solve the heat equation on a finite domain, we can employ the Fourier transform to solve the heat equation on the whole real line. Taking the Fourier transform of (1) with respect to $x$ gives the equation
$$ \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} e^{-ikx} \frac{\partial u}{\partial t}(x,t)\,dx = \kappa \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-ikx} \frac{\partial^2 u}{\partial x^2}(x,t)\,dx. $$
Making use of the properties of the Fourier transform and by pulling the $\frac{\partial}{\partial t}$ out of the integral on the left hand side, this gives the equation
$$ \frac{\partial}{\partial t}\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-ikx} u(x,t)\,dx = \kappa (-ik)^2\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{-ikx}u(x,t)\,dx.$$
Equivalently, if we let $v(k,t) = \mathcal{F}u(k,t)$, then
$$ \frac{\partial v}{\partial t} = - \kappa k^2 v.$$
Dividing both sides by $v$ gives
$$\frac{1}{v}\frac{\partial v}{\partial t} = -\kappa k^2.$$
Or equivalently,
$$\frac{\partial}{\partial t} \log v = -\kappa k^2.$$
Partially integrating both sides gives that
$$\log v = -\kappa k^2 t + f(k).$$
for some as-of-yet unknown $f$. Or equivalently,
$$v(k,t) = g(k)e^{-\kappa k^2t}.$$
Thus far, we have no information about $g$. However by noting that $v(k,0) = \mathcal{F}u(k,0)$, this says that
$$v(k,0) = \mathcal{F}\phi(k) = g(k).$$
Thus we have a "nice" expression for $v$:
$$v(k,t) = \mathcal{F}\phi(k)e^{-\kappa k^2t}.$$
We can realize $e^{-\kappa k^2 t}$ as being nothing more than the Fourier transform of $\frac{1}{\sqrt{2\kappa t}} e^{-x^2/(4\kappa t)}.$ (Check this yourself!) Thus,
$$v(k,t) = \mathcal{F}\phi(k)\mathcal{F}\left(\frac{1}{\sqrt{2\kappa t}} e^{-x^2/(4\kappa t)}\right).$$
By our handy-dandy "convolution theorem" above, we can realize this as
$$v(k,t) = \frac{1}{\sqrt{4\pi\kappa t}}\mathcal{F}(\phi*e^{-x^2/(4\kappa t)}).$$
Since $v(k,t) = \mathcal{F}u(k,t)$, we see that by inverting the Fourier transform,
$$u(x,t) = \frac{1}{\sqrt{4\pi \kappa t}} \int_{-\infty}^{\infty} \phi(y) e^{-(x-y)^2/(4\kappa t)}\,dy.\tag{2}$$
Here, I have assumed something about the Fourier transform that I did not prove but it is true. The Fourier transform has the property that if $\mathcal{F}u = \mathcal{F}v$, then $u=v$. This is definitely not true for any arbitrary function but it happens to hold true for the Fourier transform (under certain assumptions). Thus the general solution to the heat equation is of the form (2); given the initial conditions and diffusion constant $\kappa$, the heat distribution at any later time and location can be immediately solved by (2). This concludes our foray into Fourier transforms. This is woefully alack of rigor but to keep the post relatively readable for the uninitiated reader, the rigor had to be foregone. A well-versed reader however would know under what assumptions each of our hypotheses hold.
