Now, we will show that Information Theory provides the language
necessary to describe the metrization procedure in detail.
It is possible to introduce Information Theory axiomatically by a
suitable generalization of the axioms[16] in Feinstein.[17] But
to simplify the discussion here, we will use the less elegant but
equivalent method of defining certain definite integrals. The
probability density distribution p is defined from the cumulative
probability distribution P by
P(X′) = ∫X′_{measurable ⊂ X} p(x)dx. (1)
Then the information rate H is defined as
H(X) = -∫ₓp(x) ln κ p(x)dx (2)
where kappa has (carries) the units of X. Finally,
the channel rate R is defined as
R(⨀Xᵢ) = ΣH(Xᵢ) - H(X), (3)
I I
where X is the denumerable[18] cartesian product space
X = ⨂Xᵢ. (4)
I
[16] Feinstein uses his axioms only in finite space X; _i.e._, card(X)
< K₀.
[17] Feinstein, A., “Foundations of Information Theory,” New York, New
York: McGraw-Hill, 1958.
[18] If I is infinite, certain precautions have to be exercised.
Next, we define the angle Θ
|Θ(⨀Xᵢ)| = sin⁻¹_e_^{-R(⨀Xᵢ)} (5)
I
and the norm
|X| = κ(2π_e_)⁻¹ᐟ² _e_^{(HX)}. (6)
Now, if[19] a statistically independent basis; _i.e._, one for which
κ
R(⨀Xᵢ) ≡ constant, (7)
I
can be provided in terms of one-dimensional components; _i.e._, none
of them can be decomposed further, then it is just the usual problem
of diagonalization of a symmetric matrix by means of a congruence
transformation to provide an orthogonal coordinate system. Furthermore,
for uniqueness, we arrange the spectrum in decreasing order. Then,
by means of the Radon Nikodym theorem applied to each of these
one-dimensional axes, the probability distribution may be made; _e.g._,
Gaussian, if desired. Thus, we obtain the promised orthogonal Euclidean
space.
[19] This “if” is the catch that makes all methods of metrization of a
space of dimensionality higher than one impractical, except the method
of successive projections upon unit spheres centered at the center of
gravity. The method of using that nilpotent projection operator is
described in the companion paper(see footnote page 65).
Channel
At this time we can state the remaining additional condition required
that a decomposition be unique. The index space I has to be partitioned
into exactly two parts, say I′ and I″; _i.e._,
I′ ∪ I″ = I (8)
I′ ∩ I″ = φ,
such that
dim(X′) = dim(X″), (9)
where
X′ = ⨂Xᵢ (10)
I′
X″ = ⨂Xᵢ.
I″
Public-domain text, read in full here on John Shaqi.
Reviews
Reviews
No reviews yet
Be the first to share your thoughts on this work.
Join the Discussion
Join the discussion
Sign in to leave a comment or review.
Sign InorCreate an account