Fourier Series

Introduction

The central idea of Fourier series is that a wide class of periodic functions can be represented as an infinite sum of sine and cosine functions. This remarkable result, developed by Joseph Fourier in his 1822 work "Théorie analytique de la chaleur" (The Analytical Theory of Heat) while studying heat conduction, has profound implications across mathematics, physics, engineering, and modern computer science.

Consider a function \(f: \mathbb{R} \to \mathbb{R}\) that is periodic with period \(2L\), meaning \(f(x + 2L) = f(x)\) for all \(x \in \mathbb{R}\). Under suitable conditions—such as \(f\) being piecewise smooth or of bounded variation—the Fourier series of \(f\) is: \[ f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty} \left( a_n \cos\left(\frac{n\pi x}{L}\right) + b_n \sin\left(\frac{n\pi x}{L}\right) \right) \] where the Fourier coefficients are given by: \[ \begin{align*} a_0 &= \frac{1}{L} \int_{-L}^{L} f(x) \, dx \\\\ a_n &= \frac{1}{L} \int_{-L}^{L} f(x)\cos\left(\frac{n\pi x}{L}\right) \, dx, \quad n \geq 1 \\\\ b_n &= \frac{1}{L} \int_{-L}^{L} f(x)\sin\left(\frac{n\pi x}{L}\right) \, dx, \quad n \geq 1 \end{align*} \] The constant term \(\frac{a_0}{2}\) represents the average value of the function over one period. Each term in the series represents a harmonic component: the \(n\)-th term oscillates \(n\) times over one period of \(f\). This decomposition reveals the frequency content of the signal—a perspective that proves invaluable in both theoretical analysis and practical computation.

While Fourier series may seem purely theoretical, the discrete computational methods we'll explore in Part 15 are actively used in today's technology:

• Audio and image compression: MP3, AAC, and JPEG use discrete cosine transforms (closely related to Fourier series) to compress data by identifying which frequency components humans are least sensitive to
• Speech recognition and synthesis: Modern speech AI systems (Siri, Alexa, Google Assistant) use Fourier-based spectrograms to convert audio waveforms into features that neural networks can process
• Music information retrieval: Applications like Shazam identify songs by analyzing their frequency signatures using Fast Fourier Transforms (FFT)
• Time series forecasting: Machine learning models for financial data, weather prediction, and sensor data often use Fourier features to capture periodic patterns (daily, weekly, seasonal cycles)

Orthogonality of Trigonometric Functions

The key to computing Fourier coefficients lies in the orthogonality of trigonometric functions. We define an inner product on the space of functions over \([-L, L]\) by: \[ \langle f, g \rangle = \int_{-L}^{L} f(x)g(x) \, dx \]
The trigonometric functions satisfy the following orthogonality relations: \[ \begin{align*} \int_{-L}^{L} \cos\left(\frac{m\pi x}{L}\right)\cos\left(\frac{n\pi x}{L}\right) \, dx &= \begin{cases} 0 & \text{if } m \neq n \\ L & \text{if } m = n \neq 0 \\ 2L & \text{if } m = n = 0 \end{cases} \\\\ \int_{-L}^{L} \sin\left(\frac{m\pi x}{L}\right)\sin\left(\frac{n\pi x}{L}\right) \, dx &= \begin{cases} 0 & \text{if } m \neq n \\ L & \text{if } m = n \neq 0 \\ \end{cases} \\\\ \int_{-L}^{L} \cos\left(\frac{m\pi x}{L}\right)\sin\left(\frac{n\pi x}{L}\right) \, dx &= 0 \end{align*} \] where \(m\) and \(n\) are any nonnegative integers.
Note that when \(m = n = 0\), \(\cos(0) = 1\), giving us the constant function whose self-inner product is \(2L\).

These relations show that the set \(\left\{1, \cos\left(\frac{\pi x}{L}\right), \sin\left(\frac{\pi x}{L}\right), \cos\left(\frac{2\pi x}{L}\right), \sin\left(\frac{2\pi x}{L}\right), \ldots\right\}\) forms an orthogonal system for the space of square-integrable functions on \([-L, L]\). The Riesz-Fischer theorem guarantees that this system is also complete, meaning any square-integrable function can be approximated arbitrarily well by finite linear combinations of these functions. This makes it an orthogonal basis for \(L^2[-L, L]\).

This orthogonality is analogous to the orthogonality of vectors in Euclidean space, but now we're working in an infinite-dimensional function space. Just as we can decompose a vector into components along orthogonal basis vectors, we can decompose a periodic function into components along orthogonal trigonometric functions.

Fourier Coefficients

Using the orthogonality relations, we can derive formulas for the Fourier coefficients by taking inner products of \(f\) with each basis function.

To find \(a_0\), integrate both sides of the Fourier series: \[ \begin{align*} \int_{-L}^{L} f(x) \, dx &= \int_{-L}^{L} \frac{a_0}{2} \, dx + \sum_{n=1}^{\infty} \left( a_n \int_{-L}^{L} \cos\left(\frac{n\pi x}{L}\right) \, dx + b_n \int_{-L}^{L} \sin\left(\frac{n\pi x}{L}\right) \, dx \right) \\\\ &= L a_0 \end{align*} \] Therefore: \[ a_0 = \frac{1}{L} \int_{-L}^{L} f(x) \, dx \]
To find \(a_n\) for \(n \geq 1\), multiply both sides by \(\cos\left(\frac{m\pi x}{L}\right)\) and integrate: \[ \begin{align*} \int_{-L}^{L} f(x)\cos\left(\frac{m\pi x}{L}\right) \, dx &= \int_{-L}^{L} \frac{a_0}{2}\cos\left(\frac{m\pi x}{L}\right) \, dx \\\\ &\quad + \sum_{n=1}^{\infty} \left( a_n \int_{-L}^{L} \cos\left(\frac{n\pi x}{L}\right)\cos\left(\frac{m\pi x}{L}\right) \, dx + b_n \int_{-L}^{L} \sin\left(\frac{n\pi x}{L}\right)\cos\left(\frac{m\pi x}{L}\right) \, dx \right) \\\\ &= L a_m \end{align*} \] Therefore: \[ a_n = \frac{1}{L} \int_{-L}^{L} f(x)\cos\left(\frac{n\pi x}{L}\right) \, dx, \quad n \geq 1 \]
Similarly, multiplying by \(\sin\left(\frac{m\pi x}{L}\right)\) and integrating gives: \[ b_n = \frac{1}{L} \int_{-L}^{L} f(x)\sin\left(\frac{n\pi x}{L}\right) \, dx, \quad n \geq 1 \]
Note: The formulas can be unified as: \[ a_n = \frac{1}{\pi} \int_{-\pi}^{\pi} f(x)\cos(nx) \, dx, \quad n \geq 0 \] where for \(n = 0\), we have \(\cos(0) = 1\), giving \(a_0 = \frac{1}{\pi}\int_{-\pi}^{\pi} f(x) \, dx\). The appearance of \(\frac{a_0}{2}\) in the series (rather than \(a_0\)) is a convention that makes the formula symmetric.

These formulas allow us to compute the Fourier series representation of any suitable periodic function.

Complex Exponential Form

Assume that \(f(x)\) is continuous and periodic, \(f(-L) = f(L)\). Using Euler's formula: \[ e^{i\theta} = \cos(\theta) + i\sin(\theta) \] we can express trigonometric functions as: \[ \cos(\theta) = \frac{e^{i\theta} + e^{-i\theta}}{2}, \quad \sin(\theta) = \frac{e^{i\theta} - e^{-i\theta}}{2i} \] We can rewrite the Fourier series: \[ \begin{align*} f(x) &= \frac{a_0}{2} + \sum_{n=1}^{\infty} \left( a_n \cos\left(\frac{n\pi x}{L}\right) + b_n \sin\left(\frac{n\pi x}{L}\right) \right) \\\\ &= \frac{a_0}{2} + \sum_{n=1}^{\infty} a_n \left( \frac{e^{\frac{i n\pi x}{L}} + e^{\frac{-i n\pi x}{L}}}{2} \right) + \sum_{n=1}^{\infty} b_n \left( \frac{e^{\frac{i n\pi x}{L}} - e^{\frac{-i n\pi x}{L}}}{2i} \right) \\\\ &= \frac{a_0}{2} + \frac{1}{2}\sum_{n=1}^{\infty} (a_n - i b_n) e^{\frac{i n\pi x}{L}} + \frac{1}{2}\sum_{n=1}^{\infty} (a_n + ib_n) e^{\frac{-i n\pi x}{L}} \\\\ &= \frac{a_0}{2} + \frac{1}{2}\sum_{n= -1}^{-\infty} (a_{(-n)} - i b_{(-n)}) e^{\frac{-i n\pi x}{L}} + \frac{1}{2}\sum_{n=1}^{\infty} (a_n + ib_n) e^{\frac{-i n\pi x}{L}} \\\\ &= \frac{a_0}{2} + \frac{1}{2}\sum_{n= -1}^{-\infty} (a_n + i b_n) e^{\frac{-i n\pi x}{L}} + \frac{1}{2}\sum_{n=1}^{\infty} (a_n + ib_n) e^{\frac{-i n\pi x}{L}} \\\\ \end{align*} \] Here, we define \(c_0 = \frac{a_0}{2}\) and \(c_n = \frac{1}{2}(a_n + ib_n)\), then we obtain the complex form of the Fourier series of \(f(x)\).

Derivation using Orthogonality:
A complex function \(\phi(x)\) is orthogonal to another complex function \(\psi(x)\) over an interval \(a \leq x \leq b\) if \[ \int_a^b \overline{\phi}\psi \, dx = 0 \] where \(\overline{\phi}\) is the complex conjugate of \(\phi\).

For \(-\infty < n < \infty\), the eigenfunctions \(e^{\frac{-i n \pi x}{L}}\) can be verified to form an orthogonal set by following integration \[ \int_{-L}^L \left(\overline{e^{\frac{- i m\pi x}{L}}}\right) e^{\frac{ - i n\pi x}{L}} \, dx = \begin{cases} 0 & \text{if } m \neq n \\ 2L & \text{if } m = n \end{cases} \] because \(\left(\overline{e^{\frac{- i m\pi x}{L}}}\right) = e^{\frac{i m\pi x}{L}} \).

Here, we multiply the Equation (1) by \(e^{\frac{i m \pi x}{L}}\) and integrate from \(-L\) to \(L\): \[ \int_{-L}^L f(x) e^{\frac{i m \pi x}{L}} \, dx = \sum_{n= -\infty}^{\infty} c_n \int_{-L}^L e^{\frac{-i n \pi x}{L}} e^{\frac{i m \pi x}{L}} \, dx. \] Using the complex orthogonality condition, only the \(m = n\) term survives, and thus we obtain the complex Fourier coefficients: \[ \begin{align*} &\int_{-L}^L f(x) e^{\frac{i m \pi x}{L}} \, dx = 2Lc_m \\\\ &\Longrightarrow c_m = \frac{1}{2L} \int_{-L}^{L} f(x)e^{\frac{i m\pi x}{L}} \, dx. \end{align*} \]

Notation / Sign Convention

Convention used in this text (Mathematical / PDE form):
We adopt the following complex-exponential convention for the Fourier series: \[ \boxed{ f(x) = \sum_{n=-\infty}^{\infty} c_n e^{-\frac{i n\pi x}{L}}, \qquad c_n = \frac{1}{2L}\int_{-L}^{L} f(x)\,e^{+\frac{i n\pi x}{L}}\,dx } \] This convention is standard in mathematical analysis and the theory of partial differential equations. The basis functions \(e^{-\frac{i n\pi x}{L}}\) are eigenfunctions of the derivative operator: \[ \frac{d}{dx}\,e^{-\frac{i n\pi x}{L}} = -\frac{i n\pi}{L}\,e^{-\frac{i n\pi x}{L}}, \] and satisfy the orthogonality relation \[ \int_{-L}^{L} e^{-\frac{i n\pi x}{L}}\,e^{+\frac{i m\pi x}{L}}\,dx = \begin{cases} 2L, & m = n, \\[4pt] 0, & m \neq n. \end{cases} \] Multiplying the series by \(e^{+\frac{i m\pi x}{L}}\) and integrating isolates \(c_m\) immediately.

Parseval (energy) identity:
With the normalized inner product \(\langle f, g \rangle = \frac{1}{2L}\int_{-L}^{L} f\,\overline{g}\,dx\), the trigonometric system forms an orthonormal basis of \(L^2[-L, L]\), yielding \[ \frac{1}{2L}\int_{-L}^{L} |f(x)|^2\,dx = \sum_{n=-\infty}^{\infty} |c_n|^2. \] Thus, this convention defines a unitary correspondence between the function \(f\) and its Fourier coefficients \(\{c_n\}\).

Relation to Engineering and Physics convention:
In engineering, physics, and signal processing, the opposite exponential signs are often chosen: \[ f(x) = \sum_{n=-\infty}^{\infty} \tilde{c}_n e^{+\frac{i n\pi x}{L}}, \qquad \tilde{c}_n = \frac{1}{2L}\int_{-L}^{L} f(x)e^{-\frac{i n\pi x}{L}}\,dx. \] These are mathematically equivalent through the re-indexing \(\tilde{c}_n = c_{-n}\), so all orthogonality and Parseval relations remain valid.

The same distinction appears in the Fourier transform on the real line:

Summary: Both conventions are internally consistent and interchangeable by \(n \mapsto -n\) (or \(\xi \mapsto -\xi\)). The mathematical form emphasizes operator symmetry and eigenfunctions, while the engineering form emphasizes physical wave direction and phase. This text follows the mathematical convention.

Parseval's Identity

Parseval's identity relates the total energy of a signal in the time domain to its energy in the frequency domain. For a function \(f\) with Fourier series coefficients \(a_n\) and \(b_n\), it states: \[ \frac{1}{L}\int_{-L}^{L} |f(x)|^2 \, dx = \frac{a_0^2}{2} + \sum_{n=1}^{\infty} (a_n^2 + b_n^2) \]
In the complex exponential form, this becomes: \[ \frac{1}{2L}\int_{-L}^{L} |f(x)|^2 \, dx = \sum_{n=-\infty}^{\infty} |c_n|^2 \]
This identity is a generalization of the Pythagorean theorem to infinite-dimensional function spaces. The left side represents the "energy" or total power of the signal, while the right side shows that this energy is distributed across the frequency components.

Parseval's identity has important applications in signal processing and machine learning:

• Energy conservation: No energy is lost when transforming between time and frequency domains
• Data compression: Truncating small Fourier coefficients removes minimal energy, allowing efficient compression (used in JPEG, MP3)
• Feature selection: Identifies which frequency components contain most signal energy
• Noise filtering: Energy concentrated in few coefficients suggests signal; energy spread across all frequencies suggests noise

Convergence Properties

A fundamental question in Fourier analysis is: when does the Fourier series actually converge to the function \(f\)? Several types of convergence are relevant:

1. Pointwise Convergence:
The Dirichlet-Jordan theorem states that if \(f\) is of bounded variation on \([-L, L]\) (which includes most functions encountered in practice, such as piecewise smooth functions), then the Fourier series converges pointwise at every point \(x\) to: \[ \frac{f(x^+) + f(x^-)}{2} \] where \(f(x^+)\) and \(f(x^-)\) denote the right and left limits at \(x\). At points of continuity, this equals \(f(x)\).

2. Mean Square (L²) Convergence:
For any square-integrable function \(f \in L^2[-L, L]\) (the space of functions where \(\int_{-L}^{L} |f(x)|^2 \, dx < \infty\)), the Fourier series converges in the mean square sense: \[ \lim_{N \to \infty} \int_{-L}^{L} \left|f(x) - \left(\frac{a_0}{2} + \sum_{n=1}^{N} \left(a_n\cos\left(\frac{n\pi x}{L}\right) + b_n\sin\left(\frac{n\pi x}{L}\right)\right)\right)\right|^2 dx = 0 \] This is guaranteed by the Riesz-Fischer theorem, which establishes the completeness of the trigonometric system in \(L^2[-L, L]\). The L² convergence result is most naturally stated and proved using Lebesgue integration theory, which provides the proper framework for understanding these function spaces in modern analysis. This result is fundamental because:

• It holds for every square-integrable function, even highly discontinuous ones.
• It justifies truncating Fourier series for approximation (basis for compression algorithms).
• It connects directly to Parseval's identity, which we proved earlier.
• It provides the theoretical foundation for frequency-domain methods in machine learning.