Sunday, November 3, 2019

Sampling theorem, communication

The sampling theorem lays the very fundamentals of communication theory. It reveals the dimensionality of time & band-limited signal space, which in principle sets the limit on  the rate of communication (or the capacity)

Not only that, it also has its ultimate significance in the practical design of digital communication system, the concept of Inter-Symbol Interference (ISI)

Nyquist criteria

Suppose we'd like to transmit/modulate a sequence $\{a_0, a_1, ..., a_n\}$, the sampling theorem basically assures us that if we use the waveform of
\[ \sum a_n {\rm sinc}(t-n/2W) \]
and if the receiver knows exactly where to re-sample the waveform at the period of $1/2W$, the whole sequence could be recovered precisely (no noise presents).  This is mainly because the sinc function has the following property:
\[ {\rm sinc}(t-n/2W) = \left\{ \begin{array}{lr}
1 & \mbox{, when } t=n/2W \\
0 & \mbox{, when } t \neq n/2W \\ \end{array} \right. \]
which often is referred as no ISI condition, or met the Nyquist criteria.  The convolution kernel of ${\rm sinc}$ function here is usually referred as pulse shaping filter.

We know that sinc function is not the only function to satisfy Nyquist criteria, for example, ${\rm sinc}({2Wt \over M}) $ where $M$ is integer do as well.  However, in communication, they are not efficient enough (the data rate is reduced).  If we picture the no ISI condition in frequency domain, it may be more insightful.

The Fourier transform of ${\rm sinc}(2Wt)$ looks like:
and if we sample at $t=0, 1/2W,....n/2W$, the Fourier transform of the resulted sequence is:
which is the Fourier transform of impulse $\delta_n$.  It means if we sample the waveform at the right timing, the whole chain (modulation & sampling) is just:
\[ a_n * \delta_n \]

Now, if we allow the spectrum of $f(t)$ to be extended beyond W a bit, but put the constraint on the expansion that,  after aliasing,  the whole spectrum will  be purely flat, it should satisfy the Nyquist criteria as well.

One of the famous function has this property is raised cosine, which is the most commonly used pulse shaping filter in digital communication.  Notice that in communication system, the shaping filter is usually divided into to two equal parts, one in transmitter and one in receiver, namely, root raised cosine filters. The rationale behind this separation is the matched filter theory, which can maximize the signal to noise ration (SNR) at sampling point. It also simplifies the analysis of the digital system, as the noise will be filtered by the matched filter and its power will be proportional to the nominal bandwidth W, thus the system performance will be independent of the rate.

Nyquist filter bank

In signal processing, often we hear the name of Nyquist filter bank. It is the very same property (above) of the prototype filter gives the name. And it is this property make the synthesis extremely easy: just add all the samples from the filters together.

Tuesday, October 15, 2019

Singal Processing 101, Sampling

Although often deemed as simple and easy to understand, Shannon's sampling theorem is absolute the most important pillar that underpins the whole modern digital world.  It is never too much to emphasize this!

Signal which is band limited by $W$ can be fully determined by discrete samples with interval less than $\frac{1}{2W}$ seconds:

\[f(t)=\sum_{n=-\infty}^{+\infty}f({n \over 2W})\frac{\sin\pi(2Wt-n)}{\pi(2Wt-n)} \]

This is the very bridge connects the math living your computer program to physical world ! The theory has quite a few interesting points worth to discuss. We will get them later, for now, let us get an impression of this ${\rm sinc}(2Wt)$ function.

Two features of this function are very important
1. $h(n/2W) =0$ for $n \neq 0$
2. Alternate sign of peaks, but decrease slowly.

On ADC sampling rate

Sampling theorem says if we align each sample at the peak, we can get full signal only if the signal change slow enough (bandwidth is less than W). In real application, it means if the ADC sampling rate is fast enough (at least 2W), the samples we get retain all the information of the signal.  A natural question would be: whether sampling rate 2W would be enough ?

Practically, the sampling rate should be strictly greater than 2W.  There are two important factors limiting on how slow an ADC sampling rate can be.  First, the anti-aliasing filter and Second, the latency / history we need to reproduce the signal.

Suppose the signal is very clean and has nothing beyond W on the spectrum, and if the sampling rate is 2W, we have to use sinc(t) to reconstruct the signal. However, sinc(t) decays two slowly and in order to get a value accurately, we have to use large amount of previous and future samples, which typically becomes impractical.

In addition, the analog anti-aliasing filter usually plays a more important role on selecting the sampling rate, as often, the signal is not clean enough, there are a lot of interference component on the spectrum beyond W.   Following picture show the digital spectrum if there is only a second order Butterworth filter and the sampling rate is around 4W:
The second highest peak is the folded image component from a down converter which is at 22.7MHz, the sampling rate is 5MHz.

The choice of analog filter depends a quite a lot of system constrains, like the phase linearity, stop band performance, pass band performance, and implementation cost etc.  But choose the sampling rate to be at least 4W gives an octave of decay at the image frequency (sampling rate).

Higher sampling rate will relax the requirement on analog anti-aliasing filter, and help to reduce the quantization noise, or equally, quantization levels (by oversampling and decimation). But chips start to be expensive beyond 1MHz.

On aliasing 

Sampling a continuous signal $f(t)$ at a cycle of 1/2W seconds means that the spectrum of $f(t)$, $\hat{f}(f)$ is repeated every 2W Hz.   If $f(t)$ is band limited to W, it will look like this:
 However, if the spectrum of $f(t)$ exceeds W, it will look like
as we can see, no matter what filtering is applied to the signal, there is no way to reverse the alias.

As aforementioned, there might be a lot of interference on the spectrum along with the useful signal, e.g, image component of the mixer (frequency multiplier), 2nd order harmonics, and 3rd order harmonics, appropriate filtering will help to remove/attenuate these unwanted components, so that the digital spectrum is not polluted by aliasing.

Following two figures shows how anti-aliasing filter affects the output of ADC

As we can see, ensure sufficient attenuation on interference/unwanted signal is key for a clean digital signal.  Increase the sampling rate will relax the design constrains on this filter.

On bandpass sampling

Because of the spectrum of digital signal will repeat every 2W Hz, (cycle with 2W), sampling $\sin((2kW+\omega_0)t)$ with sampling rate 1/2W will produce the exact same data as sampling  $\sin(\omega_0t)$. Above figure also illustrates this: the gray unwanted signal has a very high frequency, but after sampling, it ends at the position (on the spectrum) as if we sample a signal lower than the blue signal.

This sparks a very interesting fact: for a narrow band passband signal, by carefully select the sampling rate, direct sampling the passband signal is the same as sampling the down converted signal.  But direct sampling does not need the mixer and the low pass anti-aliasing filter. Instead a bandpass filter at higher frequency is required, which is easier and cheap to implement.

Assignment: Use simpleFE to build a bandpass sampling example.