Fourier Series

Introduction

The central idea of Fourier series is that a wide class of periodic functions can be represented as an infinite sum of sine and cosine functions. This remarkable result, developed by Joseph Fourier in his 1822 work "Théorie analytique de la chaleur" (The Analytical Theory of Heat) while studying heat conduction, has profound implications across mathematics, physics, engineering, and modern computer science.

Consider a function \(f: \mathbb{R} \to \mathbb{R}\) that is periodic with period \(2L\), meaning \(f(x + 2L) = f(x)\) for all \(x \in \mathbb{R}\). Under suitable conditions—such as \(f\) being piecewise smooth or of bounded variation—the Fourier series of \(f\) is: \[ f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty} \left( a_n \cos\left(\frac{n\pi x}{L}\right) + b_n \sin\left(\frac{n\pi x}{L}\right) \right) \] where the Fourier coefficients are given by: \[ \begin{align*} a_0 &= \frac{1}{L} \int_{-L}^{L} f(x) \, dx \\\\ a_n &= \frac{1}{L} \int_{-L}^{L} f(x)\cos\left(\frac{n\pi x}{L}\right) \, dx, \quad n \geq 1 \\\\ b_n &= \frac{1}{L} \int_{-L}^{L} f(x)\sin\left(\frac{n\pi x}{L}\right) \, dx, \quad n \geq 1 \end{align*} \] The constant term \(\frac{a_0}{2}\) represents the average value of the function over one period. Each term in the series represents a harmonic component: the \(n\)-th term oscillates \(n\) times over one period of \(f\). This decomposition reveals the frequency content of the signal—a perspective that proves invaluable in both theoretical analysis and practical computation.

While Fourier series may seem purely theoretical, the discrete computational methods we'll explore in Part 15 are actively used in today's technology:

• Audio and image compression: MP3, AAC, and JPEG use discrete cosine transforms (closely related to Fourier series) to compress data by identifying which frequency components humans are least sensitive to
• Speech recognition and synthesis: Modern speech AI systems (Siri, Alexa, Google Assistant) use Fourier-based spectrograms to convert audio waveforms into features that neural networks can process
• Music information retrieval: Applications like Shazam identify songs by analyzing their frequency signatures using Fast Fourier Transforms (FFT)
• Time series forecasting: Machine learning models for financial data, weather prediction, and sensor data often use Fourier features to capture periodic patterns (daily, weekly, seasonal cycles)

Orthogonality of Trigonometric Functions

The key to computing Fourier coefficients lies in the orthogonality of trigonometric functions. We define an inner product on the space of functions over \([-L, L]\) by: \[ \langle f, g \rangle = \int_{-L}^{L} f(x)g(x) \, dx \]
The trigonometric functions satisfy the following orthogonality relations: \[ \begin{align*} \int_{-L}^{L} \cos\left(\frac{m\pi x}{L}\right)\cos\left(\frac{n\pi x}{L}\right) \, dx &= \begin{cases} 0 & \text{if } m \neq n \\ L & \text{if } m = n \neq 0 \\ 2L & \text{if } m = n = 0 \end{cases} \\\\ \int_{-L}^{L} \sin\left(\frac{m\pi x}{L}\right)\sin\left(\frac{n\pi x}{L}\right) \, dx &= \begin{cases} 0 & \text{if } m \neq n \\ L & \text{if } m = n \neq 0 \\ \end{cases} \\\\ \int_{-L}^{L} \cos\left(\frac{m\pi x}{L}\right)\sin\left(\frac{n\pi x}{L}\right) \, dx &= 0 \end{align*} \] where \(m\) and \(n\) are any nonnegative integers.
Note that when \(m = n = 0\), \(\cos(0) = 1\), giving us the constant function whose self-inner product is \(2L\).

These relations show that the set \(\left\{1, \cos\left(\frac{\pi x}{L}\right), \sin\left(\frac{\pi x}{L}\right), \cos\left(\frac{2\pi x}{L}\right), \sin\left(\frac{2\pi x}{L}\right), \ldots\right\}\) forms an orthogonal system for the space of square-integrable functions on \([-L, L]\). The Riesz-Fischer theorem guarantees that this system is also complete, meaning any square-integrable function can be approximated arbitrarily well by finite linear combinations of these functions. This makes it an orthogonal basis for \(L^2[-L, L]\).

This orthogonality is analogous to the orthogonality of vectors in Euclidean space, but now we're working in an infinite-dimensional function space. Just as we can decompose a vector into components along orthogonal basis vectors, we can decompose a periodic function into components along orthogonal trigonometric functions.

Fourier Coefficients

Using the orthogonality relations, we can derive formulas for the Fourier coefficients by taking inner products of \(f\) with each basis function.

To find \(a_0\), integrate both sides of the Fourier series: \[ \begin{align*} \int_{-L}^{L} f(x) \, dx &= \int_{-L}^{L} \frac{a_0}{2} \, dx + \sum_{n=1}^{\infty} \left( a_n \int_{-L}^{L} \cos\left(\frac{n\pi x}{L}\right) \, dx + b_n \int_{-L}^{L} \sin\left(\frac{n\pi x}{L}\right) \, dx \right) \\\\ &= L a_0 \end{align*} \] Therefore: \[ a_0 = \frac{1}{L} \int_{-L}^{L} f(x) \, dx \]
To find \(a_n\) for \(n \geq 1\), multiply both sides by \(\cos\left(\frac{m\pi x}{L}\right)\) and integrate: \[ \begin{align*} \int_{-L}^{L} f(x)\cos\left(\frac{m\pi x}{L}\right) \, dx &= \int_{-L}^{L} \frac{a_0}{2}\cos\left(\frac{m\pi x}{L}\right) \, dx \\\\ &\quad + \sum_{n=1}^{\infty} \left( a_n \int_{-L}^{L} \cos\left(\frac{n\pi x}{L}\right)\cos\left(\frac{m\pi x}{L}\right) \, dx + b_n \int_{-L}^{L} \sin\left(\frac{n\pi x}{L}\right)\cos\left(\frac{m\pi x}{L}\right) \, dx \right) \\\\ &= L a_m \end{align*} \] Therefore: \[ a_n = \frac{1}{L} \int_{-L}^{L} f(x)\cos\left(\frac{n\pi x}{L}\right) \, dx, \quad n \geq 1 \]
Similarly, multiplying by \(\sin\left(\frac{m\pi x}{L}\right)\) and integrating gives: \[ b_n = \frac{1}{L} \int_{-L}^{L} f(x)\sin\left(\frac{n\pi x}{L}\right) \, dx, \quad n \geq 1 \]
Note: The formulas can be unified as: \[ a_n = \frac{1}{\pi} \int_{-\pi}^{\pi} f(x)\cos(nx) \, dx, \quad n \geq 0 \] where for \(n = 0\), we have \(\cos(0) = 1\), giving \(a_0 = \frac{1}{\pi}\int_{-\pi}^{\pi} f(x) \, dx\). The appearance of \(\frac{a_0}{2}\) in the series (rather than \(a_0\)) is a convention that makes the formula symmetric.

These formulas allow us to compute the Fourier series representation of any suitable periodic function.

Complex Exponential Form

Assume that \(f(x)\) is continuous and periodic, \(f(-L) = f(L)\). Using Euler's formula: \[ e^{i\theta} = \cos(\theta) + i\sin(\theta) \] we can express trigonometric functions as: \[ \cos(\theta) = \frac{e^{i\theta} + e^{-i\theta}}{2}, \quad \sin(\theta) = \frac{e^{i\theta} - e^{-i\theta}}{2i} \] We can rewrite the Fourier series: \[ \begin{align*} f(x) &= \frac{a_0}{2} + \sum_{n=1}^{\infty} \left( a_n \cos\left(\frac{n\pi x}{L}\right) + b_n \sin\left(\frac{n\pi x}{L}\right) \right) \\\\ &= \frac{a_0}{2} + \sum_{n=1}^{\infty} a_n \left( \frac{e^{in\pi x/L} + e^{-in\pi x/L}}{2} \right) + \sum_{n=1}^{\infty} b_n \left( \frac{e^{in\pi x/L} - e^{-in\pi x/L}}{2i} \right) \\\\ &= \frac{a_0}{2} + \frac{1}{2}\sum_{n=1}^{\infty} (a_n - ib_n) e^{in\pi x/L} + \frac{1}{2}\sum_{n=1}^{\infty} (a_n + ib_n) e^{-in\pi x/L} \end{align*} \] Now we reindex the first summation by replacing \(n\) by \(-n\), and since cosine is even and sine is odd, we get: \[ \begin{align*} f(x) &= \frac{a_0}{2} + \frac{1}{2}\sum_{n= -1}^{-\infty} (a_{(-n)} - i b_{(-n)}) e^{\frac{-i n\pi x}{L}} + \frac{1}{2}\sum_{n=1}^{\infty} (a_n + ib_n) e^{\frac{-i n\pi x}{L}} \\\\ &= \frac{a_0}{2} + \frac{1}{2}\sum_{n= -1}^{-\infty} (a_n + i b_n) e^{\frac{-i n\pi x}{L}} + \frac{1}{2}\sum_{n=1}^{\infty} (a_n + ib_n) e^{\frac{-i n\pi x}{L}} \\\\ \end{align*} \] Here, we define \(c_0 = \frac{a_0}{2}\) and \(c_n = \frac{1}{2}(a_n + ib_n)\), then we obtain the complex form of the Fourier series of \(f(x)\).

Derivation using Orthogonality:
A complex function \(\phi(x)\) is orthogonal to another complex function \(\psi(x)\) over an interval \(a \leq x \leq b\) if \[ \int_a^b \overline{\phi}\psi \, dx = 0 \] where \(\overline{\phi}\) is the complex conjugate of \(\phi\).

For \(-\infty < n < \infty\), the eigenfunctions \(e^{\frac{-i n \pi x}{L}}\) can be verified to form an orthogonal set by following integration \[ \int_{-L}^L \left(\overline{e^{\frac{- i m\pi x}{L}}}\right) e^{\frac{ - i n\pi x}{L}} \, dx = \begin{cases} 0 & \text{if } m \neq n \\ 2L & \text{if } m = n \end{cases} \] because \(\left(\overline{e^{\frac{- i m\pi x}{L}}}\right) = e^{\frac{i m\pi x}{L}} \).

Here, we multiply the Equation (1) by \(e^{\frac{i m \pi x}{L}}\) and integrate from \(-L\) to \(L\): \[ \int_{-L}^L f(x) e^{\frac{i m \pi x}{L}} \, dx = \sum_{n= -\infty}^{\infty} c_n \int_{-L}^L e^{\frac{-i n \pi x}{L}} e^{\frac{i m \pi x}{L}} \, dx. \] Using the complex orthogonality condition, only the \(m = n\) term survives, and thus we obtain the complex Fourier coefficients: \[ \begin{align*} &\int_{-L}^L f(x) e^{\frac{i m \pi x}{L}} \, dx = 2Lc_m \\\\ &\Longrightarrow c_m = \frac{1}{2L} \int_{-L}^{L} f(x)e^{\frac{i m\pi x}{L}} \, dx. \end{align*} \]

Notation / Sign Convention

Convention used in this text (Mathematical / PDE form):
We adopt the following complex-exponential convention for the Fourier series: \[ \boxed{ f(x) = \sum_{n=-\infty}^{\infty} c_n e^{-\frac{i n\pi x}{L}}, \qquad c_n = \frac{1}{2L}\int_{-L}^{L} f(x)\,e^{+\frac{i n\pi x}{L}}\,dx } \] This convention is standard in mathematical analysis and the theory of partial differential equations. Note the opposite signs in the exponentials: negative in the series, positive in the coefficient integral. This choice has several mathematical advantages:

Mathematical properties:
The basis functions \(e^{-\frac{i n\pi x}{L}}\) are eigenfunctions of the derivative operator: \[ \frac{d}{dx}\,e^{-\frac{i n\pi x}{L}} = -\frac{i n\pi}{L}\,e^{-\frac{i n\pi x}{L}}, \] which makes the eigenvalue \(-\frac{i n\pi}{L}\) align naturally with the negative definite nature of the Laplacian in PDEs.

These basis functions satisfy the orthogonality relation: \[ \int_{-L}^{L} e^{-\frac{i n\pi x}{L}}\,e^{+\frac{i m\pi x}{L}}\,dx = \begin{cases} 2L, & m = n, \\[4pt] 0, & m \neq n. \end{cases} \] The conjugate relationship \(\overline{e^{-\frac{i n\pi x}{L}}} = e^{+\frac{i n\pi x}{L}}\) ensures that multiplying the series by \(e^{+\frac{i m\pi x}{L}}\) and integrating isolates the coefficient \(c_m\) directly.

Parseval's identity (energy conservation):
With the normalized inner product \(\langle f, g \rangle = \frac{1}{2L}\int_{-L}^{L} f(x)\,\overline{g(x)}\,dx\), the complex exponential system forms an orthonormal basis of \(L^2[-L, L]\), yielding: \[ \frac{1}{2L}\int_{-L}^{L} |f(x)|^2\,dx = \sum_{n=-\infty}^{\infty} |c_n|^2. \] This shows that the transformation \(f \mapsto \{c_n\}\) is unitary, preserving the \(L^2\) norm (energy) of the function.

Relation to Engineering and Physics convention:
In engineering, physics, and signal processing, the opposite sign convention is typically used: \[ \boxed{ f(x) = \sum_{n=-\infty}^{\infty} \tilde{c}_n e^{+\frac{i n\pi x}{L}}, \qquad \tilde{c}_n = \frac{1}{2L}\int_{-L}^{L} f(x)\,e^{-\frac{i n\pi x}{L}}\,dx } \] The two conventions are related by the simple transformation \(\tilde{c}_n = c_{-n}\). Since this is just a re-indexing, all mathematical properties (orthogonality, completeness, Parseval's identity) remain valid in both conventions.

The same sign distinction appears in the Fourier transform:

Both conventions are mathematically equivalent and internally consistent. The choice depends on the field and application:

• Mathematics/PDEs: Prefer negative sign in synthesis for cleaner eigenvalue formulas
• Engineering/Physics: Prefer positive sign in synthesis to match physical wave propagation

Parseval's Identity

Notation: Throughout this section, \(|\cdot|\) denotes the complex modulus. For a complex number \(z = x + iy\), we have \(|z|^2 = x^2 + y^2\). For real numbers, this reduces to the ordinary absolute value.

Parseval's identity relates the total energy of a signal in the time domain to its energy in the frequency domain. For a function \(f\) with Fourier series coefficients \(a_n\) and \(b_n\), it states: \[ \frac{1}{L}\int_{-L}^{L} |f(x)|^2 \, dx = \frac{a_0^2}{2} + \sum_{n=1}^{\infty} (a_n^2 + b_n^2) \]
In the complex exponential form, this becomes: \[ \frac{1}{2L}\int_{-L}^{L} |f(x)|^2 \, dx = \sum_{n=-\infty}^{\infty} |c_n|^2 \]
This identity is a generalization of the Pythagorean theorem to infinite-dimensional function spaces. The left side represents the "energy" or total power of the signal, while the right side shows that this energy is distributed across the frequency components.

Parseval's identity has important applications in signal processing and machine learning:

• Energy conservation: No energy is lost when transforming between time and frequency domains
• Data compression: Truncating small Fourier coefficients removes minimal energy, allowing efficient compression (used in JPEG, MP3)
• Feature selection: Identifies which frequency components contain most signal energy
• Noise filtering: Energy concentrated in few coefficients suggests signal; energy spread across all frequencies suggests noise

Convergence Properties

A fundamental question in Fourier analysis is: when does the Fourier series actually converge to the function \(f\)? Several types of convergence are relevant:

1. Pointwise Convergence:
If \(f\) is periodic and of bounded variation on \([-L, L]\), then at every point \(x\), the Fourier series converges to: \[ \frac{f(x^+) + f(x^-)}{2} \] where \(f(x^+) = \lim_{h \to 0^+} f(x+h)\) and \(f(x^-) = \lim_{h \to 0^-} f(x+h)\) denote the right and left limits at \(x\). At points of continuity, this equals \(f(x)\). Functions of bounded variation include most functions encountered in practice, such as piecewise smooth functions and piecewise monotone functions.

2. Mean Square (L²) Convergence:
For any square-integrable function \(f \in L^2[-L, L]\) (the space of functions where \(\int_{-L}^{L} |f(x)|^2 \, dx < \infty\)), the Fourier series converges in the mean square sense: \[ \lim_{N \to \infty} \int_{-L}^{L} \left|f(x) - \left(\frac{a_0}{2} + \sum_{n=1}^{N} \left(a_n\cos\left(\frac{n\pi x}{L}\right) + b_n\sin\left(\frac{n\pi x}{L}\right)\right)\right)\right|^2 dx = 0 \] This completeness property of the trigonometric system in \(L^2[-L, L]\) is most naturally stated and proved using Lebesgue integration theory, which provides the proper framework for understanding these function spaces in modern analysis. This result is fundamental because:

• It holds for every square-integrable function, even highly discontinuous ones.
• It justifies truncating Fourier series for approximation (basis for compression algorithms).
• It connects directly to Parseval's identity, which we proved earlier.
• It provides the theoretical foundation for frequency-domain methods in machine learning.