Superadditivity of Some Functionals Associated to Jensen's Inequality for Convex Functions on Linear Spaces with Applications

Superadditivity of Some Functionals Associated to Jensen's Inequality for Convex Functions on Linear Spaces with Applications

This is the Published version of the following publication

Dragomir, Sever S (2009) Superadditivity of Some Functionals Associated to Jensen's Inequality for Convex Functions on Linear Spaces with Applications.

Research report collection, 12 (2).

The publisher’s official version can be found at

Note that access to this version may require subscription.

Downloaded from VU Research Repository https://vuir.vu.edu.au/17728/

TO JENSEN’S INEQUALITY FOR CONVEX FUNCTIONS ON LINEAR SPACES WITH APPLICATIONS

S.S. DRAGOMIR

Abstract. Some new results related to Jensen’s celebrated inequality for con- vex functions de…ned on convex sets in linear spaces are given. Applications for norm inequalities in normed linear spaces andf-divergences in Information Theory are provided as well.

1. Introduction

The Jensen inequality for convex functions plays a crucial role in the Theory of Inequalities due to the fact that other inequalities such as the generalised triangle inequality, the arithmetic mean-geometric mean inequality, Hölder and Minkowski inequalities, Ky Fan’s inequality etc. can be obtained as particular cases of it.

LetC be a convex subset of the linear space X andf a convex function on C:

IfIdenotes a …nite subset of the setNof natural numbers,xi2C; pi 0fori2I andPI :=P

i2Ipi>0;then

(1.1) f 1

PI

X

i2I

pixi

! 1 PI

X

i2I

pif(xi); is well known in the literature asJensen’s inequality.

We introduce the following notations (see also [16]):

F(C;R) : =the linear space of all real functions on C;

F⁺(C;R) : =ff 2F(C;R) :f(x)>0 for allx2Cg; Pf(N) : =fI N: I is …niteg;

J(R) : = p=fpigi2N; pi2Rare such thatPI 6= 0for allI2Pf(N) ; and

J⁺(R) : =fp2J(R) :pi 0 for alli2Ng; J (C) : = x=fx_igi2N:x_i2C for alli2N and

Conv(C;R) := the cone of all convex functions de…ned onC;

respectively.

Date: May 07, 2009.

1991Mathematics Subject Classi…cation. 26D15; 94.

Key words and phrases. Convex functions, Jensen’s inequality, Norms, Meanf-deviation,f- Divergences.

1

In [16] the authors considered the following functional associated with the Jensen inequality:

(1.2) J(f; I; p; x) :=X

i2I

p_if(x_i) P_If 1 P_I

X

i2I

p_ix_i

!

where f 2 F(C;R); I 2 Pf(N); p 2 J⁺(R); x 2 J (C). They established some quasi-linearity and monotonicity properties and applied the obtained results for norm and means inequalities.

The following result concerning the properties of the functionalJ(f; I; ; x)as a function of weights holds (see [16, Theorem 2.4]):

Theorem 1. Let f 2Conv(C;R); I 2Pf(N)andx2J (C): (i) If p; q2J⁺(R)then

(1.3) J(f; I; p+q; x) J(f; I; p; x) +J(f; I; q; x) ( 0) i.e., J(f; I; ; x)is superadditive onJ⁺(R);

(ii) Ifp; q2J⁺(R)withp q;meaning that pi qi for eachi2N, then (1.4) J(f; I; p; x) J(f; I; q; x) ( 0)

i.e., J(f; I; ; x)is monotonic nondecreasing on J⁺(R).

The behavior of this functional as anindex set function is incorporated in the following (see [16, Theorem 2.1]):

Theorem 2. Let f 2Conv(C;R); p2J⁺(R)andx2J (C): (i) If I; H2Pf(N) withI\H =?;then

(1.5) J(f; I[H; p; x) J(f; I; p; x) +J(f; H; p; x) ( 0) i.e., J(f; ; p; x)is superadditive as an index set function onPf(N);

(ii) IfI; H 2Pf(N)with H I;then

(1.6) J(f; I; p; x) J(f; H; p; x) ( 0)

i.e., J(f; ; p; x)is monotonic nondecreasing as an index set function onP_f(N): As pointed out in [16], the above Theorem 2 is a generalisation of the Vasi´c- Mijalkovi´c result for convex functions of a real variable obtained in [26] and therefore creates the possibility to obtain vectorial inequalities as well.

For applications of the above results to logarithmic convex functions, to norm inequalities, in relation with the arithmetic mean-geometric mean inequality and with other classical results, see [16].

Motivated by the above results, we introduce in the present paper a more general functional, establish its main properties and use it for some particular cases that provide inequalities of interest. Applications for norm inequalities in normed linear spaces andf-divergences in Information Theory are provided as well.

2. Some Superadditivity Properties for the Weights We consider the more general functional

(2.1) D(f; I; p; x; ) :=P_I

"

1 P_I

X

i2I

p_if(x_i) f 1 P_I

X

i2I

p_ix_i

!#

;

where f 2 Conv(C;R); I 2 P_f(N); p 2 J⁺(R); x 2 J (C) and : [0;1) ! R is a function whose properties will determine the behavior of the functionalD as follows. Obviously, for (t) =t we recapture fromD the functional J considered in [16].

First of all we observe that, by Jensen’s inequality, the functional D is well de…ned andpositive homogeneous in the third variable, i.e.,

D(f; I; p; x; ) = D(f; I; p; x; ); for any >0andp2J⁺(R):

The following result concerning the superadditivity and monotonicity of the func- tionalD as a function of weights holds:

Theorem 3. Let f 2 Conv(C;R); I 2 P_f(N) and x 2 J (C): Assume that : [0;1)!Ris monotonic nondecreasing and concave where is de…ned.

(i) If p; q2J⁺(R)then

(2.2) D(f; I; p+q; x; ) D(f; I; p; x; ) +D(f; I; q; x; ) i.e., D is superadditive as a function of weights;

(ii) If p; q 2 J⁺(R) with p q; meaning that pi qi for each i 2 N and : [0;1)![0;1)then

(2.3) D(f; I; p; x; ) D(f; I; q; x; ) ( 0) i.e., D is monotonic nondecreasing as a function of weights.

Proof. (i). Letp; q2J⁺(R):By the convexity of the functionf onC we have 1

P_I+Q_I X

i2I

(pi+qi)f(xi) f 1 P_I+Q_I

X

i2I

(pi+qi)xi

! (2.4)

= P_{I P}¹

I

P

i2Ip_if(x_i) +Q_{I Q}¹

I

P

i2Iq_if(x_i) PI+QI

f 0

@P_{I P}¹

I

P

i2Ip_ix_i +Q_{I Q}¹

I

P

i2Iq_ix_i PI+QI

1 A

PI 1 PI

P

i2Ipif(xi) +QI 1 QI

P

i2Iqif(xi) PI+QI

PIf _P¹

I

P

i2Ipixi +QIf _Q¹

I

P

i2Iqixi

PI+QI

= PI

h 1 PI

P

i2Ipif(xi) f _P¹

I

P

i2Ipixi

i

P_I+Q_I +

QI

h 1 QI

P

i2Iqif(xi) f _Q¹_I P

i2Iqixi

i

P_I+Q_I :

Since is monotonic nondecreasing and concave, then by (2.4) we have

"

1 PI+QI

X

i2I

(p_i+q_i)f(x_i) f 1 PI+QI

X

i2I

(p_i+q_i)x_i

!#

PI

h 1 PI

P

i2Ipif(xi) f _P¹

I

P

i2Ipixi

i

P_I +Q_I +

Q_I h

1 QI

P

i2Iq_if(x_i) f _Q¹

I

P

i2Iq_ix_i i

P_I+Q_I ;

which, by multiplication withP_I+Q_I >0produces the desired result (2.2).

(ii). Ifp q;then by (i) we have

D(f; I; p; x; ) = D(f; I;(p q) +q; x; )

D(f; I; p q; x; ) +D(f; I; p; x; ) D(f; I; p; x; )

sinceD(f; I; p q; x; ) 0:

Corollary 1. Let f 2 Conv(C;R); I 2 Pf(N) and x 2 J (C): Assume that : [0;1)![0;1)is monotonic nondecreasing and concave where is de…ned.

If there exists the numbersM m 0 such thatM q p mq; then we have M Q_I

"

1 QI

X

i2I

q_if(x_i) f 1 QI

X

i2I

q_ix_i

!#

(2.5)

PI

"

1 PI

X

i2I

pif(xi) f 1 PI

X

i2I

pixi

!#

mQI

"

1 Q_I

X

i2I

qif(xi) f 1 Q_I

X

i2I

qixi

!#

:

In particular

M m

"

1 QI

X

i2I

q_if(x_i) f 1 QI

X

i2I

q_ix_i

!#

(2.6)

"

1 PI

X

i2I

pif(xi) f 1 PI

X

i2I

pixi

!#

m M

"

1 Q_I

X

i2I

qif(xi) f 1 Q_I

X

i2I

qixi

!#

:

Now, if we denote by

S(1) := p2J⁺(R) :pi 1for alli2N ; then we can state the following result as well:

Corollary 2. Let f 2 Conv(C;R); I 2 P_f(N) and x 2 J (C): Assume that : [0;1) ! [0;1) is monotonic nondecreasing and concave where is de…ned.

Then we have the bound (2.7) sup

p2S(1)

( P_I

"

1 PI

X

i2I

p_if(x_i) f 1 PI

X

i2I

p_ix_i

!#)

=card(I)

"

1 card(I)

X

i2I

f(xi) f 1 card(I)

X

i2I

xi

!#

;

wherecard(I)denotes the cardinal of the …nite set I:

Remark 1. If we consider the concave and monotonic increasing function (t) = lnt and assume that f 2 Conv(C;R) and x 2 J (C) are selected such that

1 PI

P

i2Ip_if(x_i) > f _P¹

I

P

i2Ip_ix_i for any I 2 P_f(N) with card(I) 2 and p2J⁺(R)(notice that is enough to assume thatf is strictly convex andx is not constant) then by the superadditivity of the functional

D(f; I; p; x; ln) : =P_Iln

"

1 PI

X

i2I

p_if(x_i) f 1 PI

X

i2I

p_ix_i

!#

= lnK(f; I; p; x) where

(2.8) K(f; I; p; x) :=

"

1 PI

X

i2I

pif(xi) f 1 PI

X

i2I

pixi

!#PI

we deduce thatK(f; I; ; x)is supermultiplicative, i.e., it satis…es the property (2.9) K(f; I; p+q; x) K(f; I; p; x)K(f; I; q; x)

for any p; q2J⁺(R):

The proof is obvious by the monotonicity and the positive homogeneity of the functionalD(f; I; ; x; ln):

Notice that the inequality (2.9) has been obtain in a di¤ erent way by Agarwal &

Dragomir in [1].

Another important example of concave and monotonic increasing function is (t) =t^swith s2(0;1]:In this situation the functional

(2.10) Ds(f; I; p; x) :=

"

P_I^{s 1}X

i2I

pif(xi) P_I^sf 1 PI

X

i2I

pixi

!#s

0 is superadditive and monotonic nondecreasing as a function of the weightsp:

It might be useful for applications to observe that the superadditivity property is translated into the following version of the Jensen’s inequality

(2.11)

"

(PI+QI)^{s 1}X

i2I

(pi+qi)f(xi) (PI +QI)^sf P

i2I(pi+qi)xi

P_I+Q_I

#s

"

P_I^{s 1}X

i2I

pif(xi) P_I^sf 1 P_I

X

i2I

pixi

!#s

+

"

Q^s_I ¹X

i2I

q_if(x_i) Q^s_If 1 Q_I

X

i2I

q_ix_i

!#s

( 0);

wherep; q2J⁺(R):

The property of monotonicity provides the following double inequality for p; q2 J⁺(R)such thatM q p mq andM m 0 :

M

"

Q^s_I ¹X

i2I

q_if(x_i) Q^s_If 1 Q_I

X

i2I

q_ix_i

!#s

(2.12)

"

P_I^{s 1}X

i2I

p_if(x_i) P_I^sf 1 P_I

X

i2I

p_ix_i

!#s

m

"

Q^s_I ¹X

i2I

q_if(x_i) Q^s_If 1 QI

X

i2I

q_ix_i

!#s

:

This inequality has the following equivalent form

M^1=s

"

Q^s_I ¹X

i2I

qif(xi) Q^s_If 1 Q_I

X

i2I

qixi

!#

(2.13)

P_I^{s 1}X

i2I

p_if(x_i) P_I^sf 1 PI

X

i2I

p_ix_i

!

m^1=s

"

Q^s_I ¹X

i2I

qif(xi) Q^s_If 1 QI

X

i2I

qixi

!#

:

Finally, from the Corollary 2 we also have the bound

(2.14) sup

p2S(1)

(

P_I^{s 1}X

i2I

pif(xi) P_I^sf 1 PI

X

i2I

pixi

!)

= [card(I)]^{s 1}X

i2I

f(xi) [card(I)]^sf 1 card(I)

X

i2I

xi

! :

For a function : (0;1)!(0;1)we consider now the functional (2.15) D(f; I; p; x; ; )

:=X

i2I

(pi)

"

P 1

i2I (pi) X

i2I

(pi)f(xi) f 1 P

i2I (pi) X

i2I

(pi)xi

!#

where f 2Conv(C;R); I 2Pf(N); p2J⁺(R); x2J (C): Now, if we denote by (p)the sequencef (pi)gi2N, then we observe that

D(f; I; p; x; ; ) =D(f; I; (p); x; ): The following result may be stated:

Corollary 3. Let f 2Conv(C;R); I 2Pf(N) and x2 J (C): Assume that : [0;1)![0;1)is monotonic nondecreasing and concave. If : (0;1)!(0;1) is concave, then D(f; I; ; x; ; )is also concave onJ⁺(R):

Proof. Utilising the properties of monotonicity, superadditivity and positive homo- geneity of the functionalD(f; I; ; x; )we have successively

D(f; I; tp+ (1 t)q; x; ; ) = D(f; I; (tp+ (1 t)q); x; ) D(f; I; t (p) + (1 t) (q); x; )

D(f; I; t (p); x; ) +D(f; I;(1 t) (q); x; )

= tD(f; I; (p); x; ) + (1 t)D(f; I; (q); x; )

= tD(f; I; p; x; ; ) + (1 t)D(f; I; p; x; ; ) for anyp; q2J⁺(R)andt2[0;1];which proves the statement.

3. Some Superadditivity Properties for the Index

The following result concerning the superadditivity and monotonicity of the func- tionalD as an index set function holds:

Theorem 4. Let f 2 Conv(C;R); p 2 J⁺(R) and x 2 J (C): Assume that : [0;1)!Ris monotonic nondecreasing and concave where is de…ned.

(i) If I; H2P_f(N) withI\H =?;then

(3.1) D(f; I[H; p; x; ) D(f; I; p; x; ) +D(f; H; p; x; ) i.e., D(f; ; p; x; )is superadditive as an index set function on Pf(N);

(ii) IfI; H 2Pf(N)with H I and : [0;1)![0;1);then (3.2) D(f; I; p; x; ) D(f; H; p; x; ) ( 0)

i.e.,D(f; ; p; x; )is monotonic nondecreasing as an index set function onPf(N): Proof. (i). LetI; H 2P_f(N)withI\H =?: By the convexity of the functionf onCwe have

1 P_I[H

X

k2I[H

p_kf(x_k) f 1 P_I[H

X

k2I[H

p_kx_k

! (3.3)

= PI 1

PI

P

i2Ipif(xi) +PH 1 PH

P

j2Hpjf(xj) P_I+P_H

f 0

@PI 1 PI

P

i2Ipixi +PH 1 PH

P

j2Hpjxj

PI+PH

1 A

PI 1 PI

P

i2Ipif(xi) +PH 1 PH

P

j2Hpjf(xj) P_I+P_H

PIf _P¹

I

P

i2Ipixi +PHf _P¹

H

P

j2Hpjxj

PI+PH

= PI

h1 PI

P

i2Ipif(xi) f _P¹

I

P

i2Ipixi

i

PI +PH

+ P_Hh

1 PH

P

j2Hp_jf(x_j) f _P¹

H

P

j2Hp_jx_j i PI+PH

:

Since is monotonic nondecreasing and concave, then by (3.3) we have

"

1 PI[H

X

k2I[H

pkf(xk) f 1 PI[H

X

k2I[H

pkxk

!#

PI

h1 PI

P

i2Ipif(xi) f _P¹

I

P

i2Ipixi

i

PI+PH

+ P_H h

1 PH

P

j2Hp_jf(x_j) f _P¹

H

P

j2Hp_jx_j i PI+PH

;

which, by multiplication withPI+PH>0 produces the desired result (3.2).

(ii). IfI; H2Pf(N)withH I;then D(f; I; p; x; ) =D(f;(InH)[H; p; x; )

D(f; InH; p; x; ) +D(f; H; p; x; ) D(f; H; p; x; ) ( 0) sinceD(f; InH; p; x; ) 0:

For the special case I = I_n := f1; :::; ng we write D_n(f; p; x; ) instead of D(f; I_n; p; x; );i.e.,

(3.4) D_n(f; p; x; ) =P_n

"

1 P_n

Xn i=1

p_if(x_i) f 1 P_n

Xn i=1

p_ix_i

!#

wherePn=PIn=Pn

i=1pi>0:

The following particular case is of interest:

Corollary 4. Let f 2 Conv(C;R); p 2 J⁺(R) and x 2 J (C): Assume that : [0;1) ! [0;1) is monotonic nondecreasing and concave where is de…ned.

Then

(3.5) max

IvIn

D(f; I; p; x; ) =D_n(f; p; x; ) 0;

(3.6) D_n(f; p; x; )

1 maxi<j n (pi+pj) pif(xi) +pjf(xj)

p_i+p_j f pixi+pjxj

p_i+p_j 0

and

(3.7) Dn(f; p; x; ) Dn 1(f; p; x; ) ::: D2(f; p; x; ) 0:

The proof is obvious by the monotonicity property of the functionalD(f; ; p; x; ) as an index set function.

If we use the superadditivity property, then we can state the following result as well:

Corollary 5. Let f 2 Conv(C;R); p 2 J⁺(R) and x 2 J (C): Assume that : [0;1)!Ris monotonic nondecreasing and concave where is de…ned. Then

(3.8) P2n

"

1 P2n

X2n i=1

pif(xi) f 1 P2n

X2n i=1

pixi

!#

Xn i=1

p2i

"

Pn1

i=1p_2i Xn i=1

p2if(x2i) f 1 Pn

i=1p_2i Xn i=1

p2ix2i

!#

+ Xn i=1

p_{2i 1}

"

Pn 1

i=1p_{2i 1} Xn i=1

p_{2i 1}f(x_{2i 1}) f 1 Pn

i=1p_{2i 1} Xn i=1

p_{2i 1}x_{2i 1}

!#

and

(3.9) P_2n+1

"

1 P_2n+1

2n+1X

i=1

p_if(x_i) f 1 P_2n+1

2n+1X

i=1

p_ix_i

!#

Xn i=1

p_2i

"

Pn1

i=1p2i

Xn i=1

p_2if(x_2i) f 1 Pn

i=1p2i

Xn i=1

p_2ix_2i

!#

+ Xn i=1

p2i+1

"

Pn 1

i=1p2i+1

Xn i=1

p2i+1f(x2i+1) f 1 Pn

i=1p2i+1

Xn i=1

p2i+1x2i+1

!#

:

Remark 2. If we consider the functional de…ned in (2.7), namely

K(f; I; p; x) :=

"

1 PI

X

i2I

p_if(x_i) f 1 PI

X

i2I

p_ix_i

!#PI

then by Theorem 4 we have that

(3.10) K(f; I[H; p; x) K(f; I; p; x) K(f; H; p; x)

for anyI; H 2Pf(N) with I\H =? meaning that the functionalK(f; ; p; x)is supermultiplicative as an index set mapping.

This fact obviously imply the following multiplicative inequalities of interest:

(3.11)

"

1 P2n

X2n i=1

p_if(x_i) f 1 P2n

X2n i=1

p_ix_i

!#^P2n

"

Pn1

i=1p_2i Xn i=1

p2if(x2i) f 1 Pn

i=1p_2i Xn i=1

p2ix2i

!#^Pn_i=1p2i

"

Pn 1

i=1p2i 1

Xn i=1

p_{2i 1}f(x_{2i 1}) f 1 Pn

i=1p2i 1

Xn i=1

p_{2i 1}x_{2i 1}

!#^Pni=1p2i 1

and wheref 2Conv(C;R); p2J⁺(R)andx2J (C): Moreover, if we consider the functional de…ned in (2.10) by

D_s(f; I; p; x) :=

"

P_I^{s 1}X

i2I

p_if(x_i) P_I^sf 1 P_I

X

i2I

p_ix_i

!#s

0

wheres2(0;1]and introduce the associated functional (3.12) Fs(f; I; p; x) :=P_I^{s 1}X

i2I

pif(xi) P_I^sf 1 PI

X

i2I

pixi

!

;

then by denoting

(3.13) Fs;n(f; p; x) :=Fs(f; In; p; x) =P_n^{s 1} Xn

i=1

pif(xi) P_n^sf 1 Pn

Xn i=1

pixi

!

whereI_n=f1; :::; ng;we have that the sequencefF_s;n(f; p; x)gn 2is nondecreasing and the following bounds are valid

(3.14) max

IvIn

Fs(f; I; p; x) =Fs;n(f; p; x) and

(3.15) F_s;n(f; p; x)

1 maxi<j n

(p_if(x_i) +p_jf(x_j)

(p_i+p_j)^{1 s} (pi+pj)^sf p_ix_i+p_jx_j pi+pj

) 0:

4. Applications for Norm Inequalities

Let (X;k k) be a real or complex normed linear space. It is well known that the function fp : X ! R; fp(x) = kxk^p; p 1 is convex on X: Assume that p= (p1; :::; pn)andq= (q1; :::; qn)are probability distributions with allqj nonzero.

In [11] we obtained the following re…nements of the generalised triangle inequality:

(4.1) max

1 i n

pi

qi

2 4

Xn j=1

qjkxjk^p Xn j=1

qjxj p3

5 Xn j=1

pjkxjk^p Xn j=1

pjxj p

1mini n

p_i q_i

2 4

Xn j=1

q_jkx_jk^p Xn j=1

q_jx_j

p3 5 ( 0)

and

(4.2) max

1 i nfp_ig 2 4

Xn j=1

kx_jk^p n^{1 p} Xn j=1

x_j

p3 5

Xn j=1

p_jkx_jk^p Xn j=1

p_jx_j

p

1mini nfpig 2 4

Xn j=1

kxjk^p n^{1 p} Xn j=1

xj p3

5 ( 0)

for allp 1:

We remark that, forp= 1one may get out of the previous results the following inequalities that are intimately related with the generalised triangle inequality in normed spaces:

(4.3) max

1 i n

pi

q_i 2 4

Xn j=1

q_jkx_jk Xn j=1

q_jx_j 3 5

Xn j=1

p_jkx_jk Xn j=1

p_jx_j

1mini n

pi

q_i 2 4

Xn j=1

qjkxjk Xn j=1

qjxj

3

5 ( 0);

(4.4) max

1 i nfp_ig 2 4

Xn j=1

kx_jk Xn j=1

x_j 3 5

Xn j=1

p_jkx_jk Xn j=1

p_jx_j

1mini nfpig 2 4

Xn j=1

kxjk Xn j=1

xj

3

5 ( 0):

If in (4.4) we take pj := 1

kx_jk= Xn k=1

1

kx_kk with xj6= 0 for allj2 f1; :::; ng; then, by rearranging the inequality, we get the result:

(4.5) max

1 j nfkxjkg 2 4n

Xn j=1

xj

kx_jk 3 5

Xn j=1

kxjk Xn j=1

xj

1minj nfkxjkg 2 4n

Xn j=1

xj

kxjk 3 5:

We note that the inequality (4.5) has been obtained in a di¤erent way by M.

Kato. K.-S. Saito & T. Tamura in [17] where an analysis of the equality case for strictly convex spaces has been performed as well.

We can state the following result that provides a generalization of the inequality (4.1) above:

Proposition 1. Let (X;k k) be a normed linear space,x= (x₁; :::; x_n)ann-tuple of vectors inX; p= (p₁; :::; p_n)andq= (q₁; :::; q_n)are probability distributions with all qj nonzero. If t 1 and : [0;1)![0;1) is monotonic nondecreasing and concave where is de…ned, then we have

(4.6) max

1 i n

p_i qi

2 4

Xn i=1

qikxik^t Xn i=1

qixi t3

5 2

4 Xn i=1

p_ikx_ik^t Xn

i=1

p_ix_i

t3 5

1mini n

pi

q_i 2 4

Xn i=1

q_ikx_ik^t Xn i=1

q_ix_i

t3 5

and, in particular,

(4.7) n max

1 i nfp_ig 2 4n ¹

Xn i=1

kx_ik^t n ^t Xn i=1

x_i

t3 5 2

4 Xn i=1

pikxik^t Xn i=1

qixi t3

5

n min

1 i nfpig 2 4n ¹

Xn i=1

kxik^t n ^t Xn i=1

xi t3

5:

The proof follows from Corollary 1 and the details are omitted.

Now, if p = (p₁; :::; p_n) are positive weights with P_n =Pn

i=1p_i >0 and x= (x₁; :::; x_n)is ann-tuple of vectors inX;then by de…ning the functional

(4.8) D_n(t;k k; p; x; ) =P_n 2 4P_n¹

Xn i=1

p_ikx_ik^t P_n^t Xn i=1

p_ix_i

t3 5

we can state the following result as well:

Proposition 2. If t 1 and : [0;1)![0;1) is monotonic nondecreasing and concave where is de…ned, then we have

(4.9) D_n(t;k k; p; x; )

1 maxi<j n

(

(pi+pj)

"

pikxik^t+pjkxjk^t pi+pj

pixi+pjxj

pi+pj

t#) 0 and

(4.10) D_n(t;k k; p; x; ) D_{n 1}(t;k k; p; x; ) ::: D₂(t;k k; p; x; ) 0:

The proof follows from Corollary 4 and the details are omitted.

5. Applications for f-Divergences

Given a convex functionf : [0;1)!R, thef divergence functional

(5.1) I_f(p; q) =

Xn i=1

q_if pi

q_i ;

was introduced by Csiszár [3]-[4] as a generalized measure of information, a “dis- tance function”on the set of probability distribution Pⁿ: The restriction here to discrete distributions is only for convenience, similar results hold for general distri- butions. As in Csiszár [3]-[4] , we interpret unde…ned expressions by

f(0) = lim

t!0+f(t); 0f ⁰₀ = 0;

0f ^a₀ = lim

"!0+"f ^a_" =a lim

t!1 f(t)

t ; a >0:

The following results were essentially given by Csiszár and Körner [5].

Proposition 3. (Joint Convexity) If f : [0;1)! R is convex, then I_f(p; q) is jointly convex inpandq.

Proposition 4. (Jensen’s inequality) Letf : [0;1)!Rbe convex. Then for any p; q2[0;1)ⁿ with Pn:=

Pn i=1

pi>0,Qn:=

Pn i=1

qi>0, we have the inequality

(5.2) I_f(p; q) Q_nf P_n

Qn

: If f is strictly convex, equality holds in (5.2) i¤

p1

q₁ = p2

q₂ =:::= pn

q_n: It is natural to consider the following corollary.

Corollary 6. (Nonnegativity) Letf : [0;1)!Rbe convex and normalized, i.e.,

(5.3) f(1) = 0:

Then for anyp; q2[0;1)ⁿ withPn=Qn, we have the inequality

(5.4) I_f(p; q) 0:

If f is strictly convex, equality holds in (5.4) i¤

p_i=q_i for all i2 f1; :::; ng.

In particular, if p; q are probability vectors, then Corollary 6 shows that, for strictly convex and normalizedf : [0;1)!Rthat

(5.5) I_f(p; q) 0 andI_f(p; q) = 0 i¤p=q.

We now give some examples of divergence measures in Information Theory which are particular cases off divergences.

Kullback-Leibler distance ([20]). The Kullback-Leibler distance D(; ) is de…ned by

D(p; q) :=

Xn i=1

pilog pi

q_i : If we choosef(t) =tlnt,t >0, then obviously

If(p; q) =D(p; q):

Variational distance(l₁ distance). Thevariational distance V(; )is de…ned by

V (p; q) :=

Xn i=1

jpi qij: If we choosef(t) =jt 1j,t2[0;1), then we have

I_f(p; q) =V(p; q):

Hellinger discrimination ([2]). The Hellinger discrimination is de…ned by p2h²(; ), whereh²(; )is given by

h²(p; q) :=1 2

Xn i=1

(pp_i pq_i)²: It is obvious that iff(t) =¹₂ p

t 1 ², then I_f(p; q) =h²(p; q):

Triangular discrimination ([24]). We de…ne triangular discrimination be- tweenpandqby

(p; q) = Xn i=1

jpi qij² p_i+q_i : It is obvious that iff(t) =^(t_t+1¹⁾²,t2(0;1), then

I_f(p; q) = (p; q): Note thatp

(p; q)is known in the literature as the Le Cam distance.

2 distance. We de…ne the ² distance (chi-square distance) by D ²(p; q) :=

Xn i=1

(p_i q_i)² qi

:

It is clear that iff(t) = (t 1)²,t2[0;1), then If(p; q) =D ²(p; q):

Rényi’s divergences ([23]). For 2Rn f0;1g;consider (p; q) :=

Xn i=1

p_iq_i¹ :

It is obvious that iff(t) =t (t2(0;1));then If(p; q) = (p; q):

Rényi’s divergences R (p; q) := ₍¹ ₁₎ln [ (p; q)] have been introduced for all real orders 6= 0; 6= 1(and continuously extended for = 0and = 1) in [21], where the reader may …nd many inequalities valid for these divergences, without, as well as with, some restrictions forpandq:

For other examples of divergence measures, see the paper [18] and the books [21]

and [25], where further references are given.

For a functionf : (0;1)!Rwe denote byf^# the function de…ned on(0;1) by the equationf^#(x) :=f ¹_x :With this notation we have

(5.6) I_f#(p; q) = Xn i=1

q_if^# p_i q_i =

Xn i=1

q_if q_i p_i :

By the use of Corollary 1 we can state the following result for thef-divergences.

Proposition 5. Letf : [0;1)!Rbe convex and normalized andp; qtwo probabil- ity distributions such thatR:= max_i2f1;:::;ngn

pi

qi

o

<1andr:= min_i2f1;:::;ngn

pi

qi

o

>

0: If : [0;1)![0;1)is monotonic nondecreasing and concave where is de…ned, then we have

(5.7) R I_f#(p; q) f D ²(q; p) + 1 [I_f(q; p)]

r I_f^#(p; q) f D ²(q; p) + 1 :

Proof. Utilising the inequality (2.5) we have (5.8) R

" _n X

i=1

q_if q_i pi

f Xn i=1

q_i² pi

!# " _n X

i=1

p_if q_i pi

f(1)

#

r

" _n X

i=1

qif q_i pi

f Xn i=1

q²_i pi

!#

: Since

Xn i=1

q²_i

p_i =D ²(q; p) + 1;

then by (5.8) we deduce the desired result (5.7).

Finally, by the use of Corollary 4 we also have the following lower bound for the f-divergence:

Proposition 6. Let f : [0;1)!R be convex and normalized andp; q two proba- bility distributions. If : [0;1)![0;1)is monotonic nondecreasing and concave where is de…ned, then we have:

(5.9) [I_f(q; p)]

1 maxi<j n

8<

:(pi+pj) 2 4p_if _p^qi

i +p_jf ^q_p^j

j

p_i+p_j f qi+qj

p_i+p_j 3 5

9=

; 0:

Remark 3. If one chooses di¤ erent examples of convex functions generating the particular divergences mentioned at the beginning of the section, that one can obtain various inequalities of interest. However the details are not presented here.

References

[1] R. P. Agarwal and S. S. Dragomir, The property of supermultiplicity for some classical inequalities and applications.Comput. Math. Appl.35(1998), no. 6, 105–118.

[2] R. Beran, Minimum Hellinger distance estimates for parametric models, Ann. Statist., 5 (1977), 445-463.

[3] I. Csiszár, Information-type measures of di¤erences of probability distributions and indirect observations,Studia Sci. Math. Hung.,2(1967), 299-318.

[4] I. Csiszár, Information-type measures of di¤erence of probability functions and indirect ob- servations,Studia Sci. Math. Hungar,2(1967), 299-318.

[5] I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems, Academic Press, New York, 1981.

[6] S.S. Dragomir, An improvement of Jensen’s inequality, Bull. Math. Soc. Sci. Math.

Roumanie,34(82) (1990), No. 4, 291-296.

[7] S.S. Dragomir, Some re…nements of Ky Fan’s inequality,J. Math. Anal. Appl.,163(2) (1992), 317-321.

[8] S.S. Dragomir, Some re…nements of Jensen’s inequality,J. Math. Anal. Appl.,168(2) (1992), 518-522.

[9] S.S. Dragomir, A further improvement of Jensen’s inequality, Tamkang J. Math., 25(1) (1994), 29-36.

[10] S.S. Dragomir, A new improvement of Jensen’s inequality,Indian J. Pure and Appl. Math., 26(10) (1995), 959-968.

[11] S.S. Dragomir, Bounds for the normalised Jensen functional.Bull. Austral. Math. Soc.74 (2006), no. 3, 471–478.

[12] S.S. Dragomir, A re…nement of Jensen’s inequality with applications forf-divergence mea- sures, Preprint,Res. Rep. Coll.10(2007), Supp., Article 15.

[13] S.S. Dragomir, Re…nements of the Jensen integral inequality,in preparation.

[14] S.S. Dragomir and N.M. Ionescu, Some converse of Jensen’s inequality and applications.Rev.

Anal. Numér. Théor. Approx.23(1994), no. 1, 71–78. MR1325895 (96c:26012).

[15] S.S. Dragomir and C.J. Goh, A counterpart of Jensen’s discrete inequality for di¤erential- ble convex mappings and applications in information theory, Mathl. Comput. Modelling, 24(1996), No. 2, 1-11.

[16] S.S. Dragomir, J. Peµcari´c and L.E. Persson, Properties of some functionals related to Jensen’s inequality,Acta Math. Hung.,70(1-2) (1996), 129-143.

[17] M. Kato, K.-S. Saito and T. Tamura, Sharp triangle inequality and its reverse in Banach spaces.Math. Inequal. Appl.10(2007), no. 2, 451–460.

[18] J.N. Kapur, A comparative assessment of various measures of directed divergence,Advances in Management Studies,3 (1984), No. 1, 1-16.

[19] S. Kullback,Information Theory and Statistics,J. Wiley, New York, 1959.

[20] S. Kullback and R.A. Leibler, On information and su¢ ciency, Annals Math. Statist., 22 (1951), 79-86.

[21] F. Liese and I. Vajda,Convex Statistical Distances,Teubner Verlag, Leipzig, 198

[22] J. Peµcari´c and S.S. Dragomir, A re…nements of Jensen inequality and applications, Studia Univ. Babe¸s-Bolyai, Mathematica,24(1) (1989), 15-19.

[23] A. Renyi, On measures of entropy and information, Proc. Fourth Berkeley Symp. Math.

Statist. Prob., Vol. 1, University of California Press, Berkeley, 1961.

[24] F. Topsoe, Some inequalities for information divergence and related measures of discrimina- tion,Res. Rep. Coll. RGMIA,2(1) (1999), 85-98.7.

[25] I. Vajda,Theory of Statistical Inference and Information,Kluwer, Boston, 1989.

[26] P. M. Vasi´c and µZ. Mijajlovi´c, On an index set function connected with Jensen inequality.

Univ. Beograd. Publ. Elektrotehn. Fak. Ser. Mat. Fiz.No.544–576(1976), 110–112.

Research Group in Mathematical Inequalities & Applications, School of Engineering

& Science, Victoria University, PO Box 14428, Melbourne City, MC 8001, Australia.

E-mail address: sever.dragomir@vu.edu.au

URL:http://www.staff.vu.edu.au/rgmia/dragomir/