Signal (information theory)

In information theory, a signal is a flow of information. Most signals of interest can be modeled as functions of time or position.

The domain of a function can be a scalar or a vector. Likewise, the range can be scalar or vector, with either real or complex values.

1 Analog and digital signals
- 1.1 Discretization
- 1.2 Quantization
2 Examples of signals
3 Frequency analysis
4 Entropy
5 Some philosophical questions
6 See also

Analog and digital signals

The two main types of signals are analog and digital. In short, the difference between them is that digital signals are discrete and quantized, as defined below, while analog signals possess neither property.

Discretization

Main article: Discrete signal

One of the fundamental distinctions between different types of signals is between continuous and discrete time. In the mathematical abstraction, the domain of a continuous-time (CT) signal is the set of real numbers (or some interval thereof), whereas the domain of a discrete-time signal is the set of integers (or some interval). What these integers represent depends on the nature of the signal.

DT signals often arise via sampling of CT signals. For instance, sensors output data continuously, but since a continuous stream is impossible to record, a discrete signal is used as an approximation. Computers and other digital devices are restricted to discrete time.

Quantization

Main article: Quantization (signal processing)

If a signal is to be represented as a sequence of numbers, it is impossible to maintain arbitrarily high precision – each number in the sequence must have a finite number of digits. As a result, the values of such a signal are restricted to belong to a finite set; in other words, it is quantized.

Examples of signals

Motion. One can conceive of a signal representing the motion of a particle – say, a mote of dust, through some suitable space. The domain of a motion signal is one-dimensional (time), and the range is generally three-dimensional.

Sound. Since a sound is a vibration of a medium (such as air), a sound signal associates a pressure value to every value of time. In the real world, sound signals are analog.

Compact discs (CDs). CDs contain discrete signals representing sound, recorded at 44,100 samples per second. Each sample contains data for a left and right channel (since CDs are recorded in stereo).

Pictures. An picture assigns a color value to each of a set of points. Since the points lie on a plane, the domain is two-dimensional. If the picture is a physical object, such as a painting, it's a continuous signal. If the picture a digital image, it's a discrete signal. It's often convenient to represent color as the sum of the intensities of three primary colors, so that the signal is vector-valued with dimension three.

Videos. A video is a series of images. A point in a video is identified by its position (two-dimensional) and by the time at which it occurs, so a video signal has a three-dimensional domain.

Frequency analysis

Main article: Frequency domain

It is remarkably useful to analyze the frequency spectrum of a signal. This technique is applicable to all signals, both continuous and discrete. For instance, if a signal is passed through an LTI system, the frequency spectrum of the resulting output signal is the product of the frequency spectrum of the original input signal and the frequency response of the system.

Entropy

Another important propery of a signal (actually, of a statistically defined class of signals) is its entropy or information content.

Some philosophical questions

While analog signals exist on paper, they do not exist in reality. This is the result of Planck time, Planck length, and Planck energy units. In other words, all reality-based signals are digital signals but with extremely small quantization levels. In treating a signal as an analog signal it is for mathematical purposes or for simplicity's sake by considering the extremely small quantizations as negligible. This realization that analog signals are only theoretical is one not usually made and so assuming such quantizations as negligible is not one to quibble over except in theoretical arguments.

A second view: The original author (above) raises a very interesting point, but this author feels that his/her depiction of quantization effects at the Planck and Heisenberg level is a bit of a misrepresentation. Microscopic, Nano-scopic, and Pico-scopic signals (such as the sound waves resulting from the orderly transmission of a vibration through the atmosphere or other fluid type substance, be it liquid, gas or other) are indeed only idealized as mathematically continuous functions in the sense of the real numbers. But, neither are they digital signals in the modern sense of sampled data with characteristic sampling rates, bit depths and associated engineering baggage necessary for the adequate perception, conceptualization and processing of such a form of a real world signal.

The important point to note here is that of data granularity. Digitized signals have obvious granularity, and the first author attempts to draw a parallel with the granularity described by quantum physics. One important distinction to consider is the uncertainty inherent in quantum mechanics which we have from Heisenberg, as well as the non-linear dynamics and other mechanisms of chaotic and complex systems theories, which we have from statistical mechanics (thermodynamics and its cousins). Computer digital signals attempt as a general rule to be as precise as possible, ideally with zero variation and no ambiguity in any of the measured features of the data. While this is proveably akin to a dream at some resolutions, at higher resolutions, the effects can be ignored, as the first author states. In between, however, lies the realm of noise effects. Brownian motion is a famous example of this.

If the granularity of the reality which one measures is finer than the granularity of one's instruments, then one can map the system to the continuous real numbers, and its higher dimensional forms ( 2D, 3D spaces and higher). But keep the point in mind that these mathematical systems are indeed mere approximations to the real system (even if one has a good theoretical framework to describe the system!), and one should always be aware of the underlying dogma of one's assumptions, most especially the context in which these assumptions are reasonably valid.