Lecture-04: Random Variable

Lecture-04: Random Variable

1 Random Variable

Definition 1.1 (Random variable). Consider a probability space(Ω,F,P). Arandom variableX:Ω→R is a real-valued function from the sample space to real numbers, such that for eachx∈Rthe event

AX(x)≜{ω∈Ω:X(ω)⩽x}={X⩽x}=X⁻¹(−∞,x] =X⁻¹(Bx)∈F.

We say that the random variableXisF-measurable.

Remark1. Recall that the setA_X(x)is always a subset of sample spaceΩfor any mappingX:Ω→_{R, and} A_X(x)∈_Fis an event whenXis a random variable.

Example 1.2 (Constant function). Consider a mappingX:Ω→ {c} ⊆_Rdefined on an arbitrary prob- ability space(_Ω,F,P), such thatX(ω) =cfor all outcomesω∈Ω. We observe that

AX(x) =X⁻¹(Bx) =

(∅, x<c, Ω, x⩾c.

That isAX(x)∈Ffor all event spaces, and henceXis a random variable and measurable for all event spaces.

Example 1.3 (Indicator function). For an arbitrary probability space (_Ω,F,P) and an event A∈F, consider the indicator function1A:Ω→[0, 1]. Letx∈_{R, and}Bx= (−_∞,x], then it follows that

AX(x) =1⁻¹_A (Bx) =







Ω, x⩾1, A^c, x∈[0, 1),

∅, x<0.

That is,A_X(x)∈_Ffor allx∈R, and hence the indicator function1Ais a random variable.

Remark2. Since any outcomeω∈Ωis random, so is the real valueX(ω).

Remark3. Probability is defined only for events and not for random variables. The events of interest for random variables are thelower level setsA_X(x) ={ω:X(ω)⩽x}=X⁻¹(Bx)for any realx.

Remark4. Consider a probability space(_Ω,F,P)and a random variableX:Ω→_Rthat isGmeasurable for G⊆_{F. IfG}⊆_{H, then}Xis alsoHmeasurable.

1.1 Distribution function for a random variable

Definition 1.4. For anFmeasurable random variableX:Ω→_Rdefined on the probability space(_Ω,F,P), we can associate adistribution function(CDF)F_X:R→[0, 1]such that for allx∈_R,

FX(x)≜P(AX(x)) =P({X⩽x}) =P◦X⁻¹(−_∞,x] =P◦X⁻¹(Bx).

1

Example 1.5 (Constant random variable). LetX:Ω→ {c} ⊆_Rbe a constant random variable defined on the probability space(_Ω,F,P). The distribution function is a right-continuous step function atcwith step-value unity. That is,FX(x) =_1[c,∞)(x). We observe thatP({X=c}) =1.

Example 1.6 (Indicator random variable). For an indicator random variable1A:Ω→ {0, 1}defined on a probability space(_Ω,F,P)and an eventA∈F, we have

FX(x) =







1, x⩾1,

1−P(A), x∈[0, 1), 0, x<0.

Lemma 1.7 (Properties of distribution function). The distribution function FXfor any random variable X sat- isfies the following properties.

1. The distribution function is monotonically non-decreasing in x∈R.

2. The distribution function is right-continuous at all points x∈R.

3. The upper limit islimx→∞FX(x) =1and the lower limit islimx→−∞FX(x) =0.

Proof. LetXbe a random variable defined on the probability space(_Ω,F,P)_.

1. Letx₁,x₂∈_Rsuch thatx₁⩽x₂. Then for anyω∈Ax₁, we haveX(ω)⩽x₁⩽x₂, and it follows that ω∈Ax₂. This implies thatAx₁ ⊆Ax₂. The result follows from the monotonicity of the probability.

2. For anyx∈R, consider any monotonically decreasing sequencex∈_R^N such that limnxn =x0. It follows that the sequence of events Axn =X⁻¹(−_∞,xn]∈_F:n∈_N, is monotonically decreasing and hence limn∈NAxn =∩_n∈NAxn =Ax₀. The right-continuity then follows from the continuity of probability, since

FX(x0) =P(Ax₀) =P(lim

n∈NAxn) = lim

n∈NP(Axn) =lim

x_n↓xF(xn).

3. Consider a monotonically increasing sequencex∈_R^Nsuch that limnxn=_{∞, then}(Axn∈_F:n∈_N) is a monotonically increasing sequence of sets and limnAxn =∪_n∈NAxn=Ω. From the continuity of probability, it follows that

xlimn→∞FX(xn) =lim

n P(Axn) =P(lim

n Axn) =P(Ω) =1.

Similarly, we can take a monotonically decreasing sequence x∈_R^N such that limnxn=−_{∞, then} (Axn∈F:n∈N)is a monotonically decreasing sequence of sets and limnAxn=∩_n∈NAxn=∅. From the continuity of probability, it follows that limxn→−∞FX(xn) =0.

Remark5. If two realsx1<x2thenFX(x1)⩽FX(x2)with equality if and only ifP{(x1<X⩽x2}) =0. This follows from the fact thatAx₂ =Ax₁∪X⁻¹(x1,x2].

1.2 Event space generated by a random variable

Definition 1.8 (Event space generated by a random variable). Let X:Ω→R be anFmeasurable ran- dom variable defined on the probability space(Ω,F,P). The smallest event space generated by the events AX(x) =X⁻¹(Bx) =X⁻¹(−∞,x]forx∈Ris called theevent space generatedby this random variableX, and denoted byσ(X)≜^σ({AX(x):x∈R}).

2

Remark 6. The event space generated by a random variable is the collection of the inverse of Borel sets, i.e. σ(X) =X⁻¹(B):B∈_B(_R) . This follows from the fact thatAX(x) =X⁻¹(Bx)and the inverse map respects countable set operations such as unions, complements, and intersections. That is, ifB∈_B(_R) = σ({Bx:x∈_R}), then X⁻¹(B)∈ σ({AX(x):x∈_R}). Similarly, if A∈σ(X) =σ({AX(x):x∈_R}), then A=X⁻¹(B)for someB∈σ({Bx:x∈R}).

Example 1.9 (Constant random variable). LetX:Ω→ {c} ⊆_Rbe a constant random variable defined on the probability space(Ω,F,P). Then the smallest event space generated by this random variable is σ(X) ={∅,Ω}.

Example 1.10 (Indicator random variable). Let 1A be an indicator random variable defined on the probability space (Ω,F,P)and event A∈F, then the smallest event space generated by this random variable isσ(X) =σ({∅,A^c,Ω}) ={∅,A,A^c,Ω}.

1.3 Discrete random variables

Definition 1.11 (Discrete random variables). If a random variableX:Ω→_X⊆_Rtakes countable values on real-line, then it is called adiscrete random variable. That is, the range of random variableXis countable, and the random variable is completely specified by theprobability mass function

PX(x) =P({X=x}), for allx∈X.

Example 1.12 (Bernoulli random variable). For the probability space(Ω,F,P), theBernoulli random variableis a mappingX:Ω→ {0, 1}andPX(1) =p. We observe that Bernoulli random variable is an indicator for the eventA≜X⁻¹{1}, andP(A) =p. Therefore, the distribution functionFXis given by

FX= (1−p)1_[0,1)+1_[1,∞).

Lemma 1.13. Any discrete random variable is a linear combination of indicator function over a partition of the sample space.

Proof. For a discrete random variableX:Ω→X⊂Ron a probability space(Ω,F,P), the rangeXis count- able, and we can define eventsEx≜{ω∈Ω:X(ω) =x} ∈Ffor eachx∈X. Then the mutually disjoint sequence of events(Ex∈F:x∈X)partitions the sample spaceΩ. We can write

X(ω) =

∑

x∈X

x1Ex(ω).

Definition 1.14. Any discrete random variableX:Ω→_X⊆_Rdefined over a probability space(_Ω,F,P), with finite range is called a simple random variable.

Example 1.15 (Simple random variables). LetXbe a simple random variable, thenX=_∑x∈Xx1_AX(x) where(AX(x) =X⁻¹{x} ∈_F:x∈_X)is a finite partition of the sample spaceΩ. Without loss of gener- ality, we can denoteX={x₁, . . . ,xn}wherex₁⩽. . .⩽xn. Then,

X⁻¹(−∞,x] =







Ω, x⩾xn,

∪ⁱ_j=1A_X(x_j), x∈[x_i,x_i+1),i∈[n−1],

∅, x<x1.

Then the smallest event space generated by the simple random variableXis{∪x∈SA_X(x):S⊆_X}.

3

1.4 Continuous random variables

Definition 1.16. For a continuous random variableX, there existsdensity function fX :R→[0,∞)such that

FX(x) = Z x

−∞fX(u)du.

Example 1.17 (Gaussian random variable). For a probability space(_Ω,F,P),Gaussian random vari- ableis a continuous random variableX:Ω→_Rdefined by its density function

f_X(x) = √¹ 2πσexp

−(x−µ)² 2σ²

, x∈_R.

4