Extension to the closure of a subgroup system

Chapter VI: Constructing new P -schemes from old ones

6.4 Extension to the closure of a subgroup system

Proof. LetG be a primitive permutation group. Seress [Ser96] provedb(G) ≤ 4 when Gis solvable. More generally, it was shown in [GSS98] that there exists a functiong(·)such thatb(G)≤g(d)ifGdoes not involveAlt(d). The theorem then follows from Lemma 2.5.

Remark. The scheme conjectures in this section are formulated in terms of discrete- ness ofP-schemes and are used for complete factorization. One can also formulated conjectures in terms of inhomogeneity and use them for proper factorization. We leave the details to the reader. To establish reductions between these conjectures (in terms of inhomogeneity rather than discreteness), one needs to restrict to families oftransitivepermutation groups as transitivity is required in Corollary 6.2.

Proof. ConsiderH ∈ (P_cl)_cl. WriteH = u_Pcl(H)andH = u_P(H). We show thatH ∈ P_clandu_P(H) =H⁰⁰.

We first verify thatH⁰⁰is normal inH. By definition, we knowH⁰ is normal inH. Then for anyg ∈H, we have

H⁰⁰ =uP(H⁰) = uP(gH⁰g⁻¹) = guP(H⁰)g⁻¹ =gH⁰⁰g⁻¹. SoH⁰⁰is normal inH.

Next we show that H⁰⁰ is the unique maximal element in P subject to H⁰⁰ ⊆ H. Assume to the contrary that there exists an elementU ( H⁰⁰ inP ⊆ P_cl that is a subgroup ofH. AsH⁰is the unique maximal element inP_clsubject toH⁰ ⊆H, we haveU ⊆ H⁰. Furthermore, asH⁰⁰ is the unique maximal element inP subject to H⁰⁰ ⊆H⁰, we haveU ⊆H⁰⁰, contradicting the assumptionU (H⁰⁰.

By definition, we haveH ∈ P_cl anduP(H) =H⁰⁰.

We show that aP-scheme can always be extended to aP_cl-scheme where antisym- metry and strong antisymmetry are preserved.

Lemma 6.10. Let P be a subgroup system over a group G and let C = {C_H : H ∈ P}be aP-scheme. There exists a uniqueP_cl-schemeC⁰ ={C_H⁰ : H ∈ P_cl} extendingC (i.e.,C_H⁰ =C_H forH ∈ P), given by

C_H⁰ ={π_u(H),H(B) :B ∈C_u(H)}.

Moreover, ifC is antisymmetric (resp. strongly antisymmetric), so isC⁰. AndC⁰ is not discrete onH ∈ P_clifC is antisymmetric and not discrete onu(H).

Proof. We haveu(H)∈ P ⊆ P_cl forH ∈ P_cl. It follows from Lemma 2.3 thatC⁰ as defined above is the only possible one extendingC.

Then we check that C⁰ is indeed well defined, i.e., for H ∈ P_cl, the set C_H⁰ = {π_u(H),H(B) : B ∈C_u(H)}is indeed a partition ofH\G. For two blocksB₁, B₂ ∈ C_u(H), we prove thatπ_u(H),H(B₁)and π_u(H),H(B₂)are either identical or disjoint.

Suppose there existu(H)g₁ ∈B₁andu(H)g₂ ∈B₂ satisfyingπ_u(H),H(u(H)g₁) = π_u(H),H(u(H)g₂), i.e., Hg₁ = Hg₂. Then g₂g₁⁻¹ ∈ H ⊆ N_G(u(H)). Note that c_u(H),g

2g⁻¹₁ (u(H)g₁) = u(H)g₂. So by invariance of C, we havec_u(H),g

2g₁⁻¹(B₁) = B₂. Then by Lemma 2.2, we have

πu(H),H(B2) = πu(H),H ◦c_u(H),g2g⁻¹

1 (B1) = c_H,g2g⁻¹

1 ◦πu(H),H(B1) = πu(H),H(B1)

as desired. So C is well defined. Moreover, we haveu(H) = H for H ∈ P. It follows thatC⁰ does extendC.

Next we show that C⁰ is a P_cl-scheme. For H, H⁰ ∈ P_cl with H ⊆ H⁰, we have u(H) ⊆ H⁰ and hence u(H) ⊆ u(H⁰) by the unique maximality of u(H⁰). By transitivity of projections (see Lemma 2.2), the following diagram commutes:

u(H)\G u(H⁰)\G

H\G H⁰\G

π_u(H),u(H0)

πu(H),H π_u(H0),H0

π_H,H0

To show compatibility, consider y, y⁰ ∈ H\G lying in the same block B ∈ C_H⁰ . Choose B˜ ∈ C_u(H) satisfying π_u(H),H( ˜B) = B and choose x, x⁰ ∈ B˜ satisfying π_u(H),H(x) = y, π_u(H),H(x⁰) = y⁰. By compatibility of C, the ele- ments π_u(H),u(H⁰₎(x) and π_u(H),u(H⁰₎(x⁰) lie in the same block of C_u(H⁰₎. Then π_u(H⁰_),H⁰ ◦π_u(H),u(H⁰₎ maps x and x⁰ into the same block of C_H⁰ 0 by the defini- tion of C⁰. By commutativity of the diagram above and the factsπu(H),H(x) = y, π_u(H),H(x⁰) = y⁰, we see thatπ_H,H⁰(y)andπ_H,H⁰(y⁰)lie in the same block ofC_H⁰ 0. SoC⁰ is compatible.

For regularity, let B be a block of C_H⁰ . Then π_H,H⁰(B) is contained in a unique blockB⁰ofC_H⁰ 0 by compatibility ofC⁰. LiftB to a blockB˜ ∈C_u(H)alongπ_u(H),H, and letB˜⁰ = π_u(H),u(H⁰₎( ˜B) ∈ C_u(H⁰₎. By regularity of C, the map π_u(H),u(H⁰₎|B˜ : B˜ → B˜⁰ has constant degree, i.e., the number of preimages |(π_u(H),u(H⁰₎|B˜)⁻¹(y)|

is independent of the choices of y ∈ B˜⁰. We show that π_u(H),H|B˜ (and similarly π_u(H)⁰_,H⁰|B˜⁰) also has constant degree. Consider y, y⁰ ∈ B. As π_u(H),H( ˜B) = B, there exists x, x⁰ ∈ B˜ satisfying π_u(H),H(x) = y and π_u(H),H(x⁰) = y⁰. Note that all the elements in(π_u(H),H|B˜)⁻¹(y)(resp. (π_u(H),H|B˜)⁻¹(y⁰)) are of the form c_u(H),g(x)(resp. c_u(H),g(x⁰)) for someg ∈ HsinceH ⊆ N_G(u(H)). And we have c_u(H),g(x) ∈ B˜ iffc_u(H),g(x⁰) ∈ B˜ for g ∈ H by invariance of C. It follows that

|(π_u(H),H|B˜)⁻¹(y)|=|(π_u(H),H|B˜)⁻¹(y⁰)|. Soπ_u(H),H|B˜ (and similarlyπ_u(H)⁰_,H⁰|B˜⁰) has constant degree. Thenπ_H,H⁰|_B also has constant degree by the commutativity of the diagram above. SoC⁰is regular.

For invariance, note that for H, H⁰ ∈ P_cl with H⁰ = gHg⁻¹, we have u(H⁰) =

gu(H)g . And the following diagram commutes by Lemma 2.2:

u(H)\G u(H⁰)\G

H\G H⁰\G

cu(H),g

π_u(H),H π_u(H0),H0

cH,g

For a block B of C_H⁰ , lift it to a block B˜ of C_u(H). Then c_H,g(B) = π_u(H⁰_),H⁰ ◦ c_u(H),g( ˜B) by the commutativity of the diagram above. Note that c_u(H),g( ˜B) is a block of C_u(H⁰₎ by invariance of C. So c_H,g(B) is a block of C_H⁰ 0 by definition.

ThereforeC⁰ is invariant.

Now assume C⁰ is not strongly antisymmetric and we prove that C is not either.

By definition, there exists a nontrivial permutation τ = σ_k◦ · · · ◦σ₁ of a block B ∈ C_H⁰ for some H ∈ Pcl such that each σi : Bi−1 → Bi is a map of the form cHi−1,g|Bi−1, πHi−1,Hi|Bi−1, or (πHi,Hi−1|Bi)⁻¹, and Bi ∈ C_H⁰ _i, Hi ∈ Pcl, B =B₀ =B_k,H =H₀ =H_k(see Definition 2.7). By the two diagrams above, we can lift eachB_itoB˜_i ∈C_u(Hi)for0≤i≤kand lift eachσ_ito a mapσ˜_i : ˜Bi−1 →B˜_i of the formc_u(Hi−1),g|B˜i−1, π_{u(Hi−1),u(Hi)}|B˜i−1, or (π_{u(Hi),u(Hi−1)}|B˜i)⁻¹ respectively, i.e., π_u(Hi),Hi( ˜B_i) = B_i and σ_i ◦ π_{u(Hi−1),Hi−1}|B˜i−1 = π_u(Hi),Hi|B˜i ◦σ˜_i.4 Then

˜

τ := ˜σ_k ◦ · · · ◦σ˜₁ is a map from B˜₀ to B˜_k lifting τ. Note that π_u(H),H( ˜B₀) = π_u(H),H( ˜B_k) = B. So c_u(H),g( ˜B_k) = ˜B₀ for someg ∈ H. By composing τ˜with c_u(H),g(and noting thatc_H,g is the identity map), we may assumeB˜_k= ˜B₀. Soτ˜is a permutation ofB˜0. Moreover τ˜is nontrivial since it liftsτ. SoC is not strongly antisymmetric. The proof for antisymmetry is the same except that we only consider mapsτ that are conjugations.

Finally, to prove the last claim, assume C is antisymmetric and C⁰ is discrete on H ∈ P_cl. We prove thatC is discrete onu(H). Consider distinct elementsx, x⁰ ∈ u(H)\Gand lety =π_u(H),H(x), y⁰ =π_u(H),H(x⁰). Ify 6= y⁰, they are in different blocks ofC_H⁰ and hencex, x⁰are in different blocks ofC_u(H)by the definition ofC_H⁰ . So assume y = y⁰. Then x = u(H)g, x⁰ = u(H)g⁰ for some g, g⁰ ∈ Gsatisfying Hg = Hg⁰, i.e.,g⁰g⁻¹ ∈ H ⊆ NG(u(H)). Asx⁰ =c_u(H),g⁰_g⁻¹(x), the elementsx andx⁰ are in different blocks of C_u(H) by antisymmetry of C. SoC is discrete on u(H), as desired.

4For the case thatσi = (πH_i,Hi−1|B_i)⁻¹, we liftπH_i,Hi−1|B_i toπ_{u(Hi),u(Hi−1)}|B˜i. AsC is antisymmetric, bothπ_{u(Hi−1),Hi−1}|B˜i−1andπ_u(Hi),Hi|B˜iare bijective. Soπ_{u(Hi),u(Hi−1)}|B˜iis also bijective and its inverse is well defined.

Recall that for a subgroup systemP over a finite groupG, we letP₊={H :H ⊆ H ⊆ N_G(H⁰), H⁰ ∈ P}which is also a subgroup system overG(see Section 4.4).

ClearlyP_cl ⊆ P₊. We show that equality holds ifP isjoin-closed.

Lemma 6.11. LetP be a subgroup system that is join-closed, i.e.,hH, H⁰i ∈ P for allH, H⁰ ∈ P. ThenP_cl =P₊.

Proof. Consider H ∈ P+. We prove H ∈ Pcl by verifying the conditions in Definition 6.5.

Choose a maximal elementH⁰ ∈ Psubject toH⁰ ⊆H. Such an element exists by the definition ofP₊. We first show thatH⁰ is unique. Assume to the contrary that there exists another maximal elementH⁰⁰ ⊆H inP different fromH⁰. ThenhH⁰, H⁰⁰i) H⁰is also a subgroup ofHand lies inPby join-closedness, contradicting maximality ofH⁰. SoH⁰ is unique.

Next we proveH⁰ is normal inH. Assume to the contrary that there existsg ∈ H such that gH⁰g⁻¹ 6= H⁰. As gH⁰g⁻¹ ⊆ gHg⁻¹ = H and gH⁰g⁻¹ ∈ P, the join hH⁰, gH⁰g⁻¹i)H⁰ is also a subgroup ofHand lies inP by join-closedness, again contradicting maximality ofH⁰.

As an application, we consider a system of stabilizers with respect to the natural action of a symmetric group or an alternating group.

Lemma 6.12. LetSbe a finiteG-set whereGisSym(S)orAlt(S)acting naturally on S. Let P = P_m be the corresponding system of stabilizers of depth m, where m <|S|/2. ThenP⁰ :=P ∪ {G}is join-closed.

Proof. NoteP⁰ ={G_T : 0≤T ≤m}. LetT andT⁰be subsets ofSof cardinality at mostm. We show thathG_T, G_T⁰i ∈ P⁰. Obviously we havehG_T, G_T⁰i ⊆G_T∩T⁰. First assume G = Sym(S). We have G_T ∼= Sym(S −T), G_T⁰ ∼= Sym(S −T⁰) and G_T∩T⁰ ∼= Sym(S −(T ∩ T⁰)) by restricting to the subsets S − T, S −T⁰ and S −(T ∩T⁰) respectively. The group Sym(S −(T ∩T⁰)) is generated by transpositions(x y) withx, y ∈ S−(T ∩T⁰). We claim that every such(x y)is contained inhG_T, G_T⁰i. This is obvious ifxandyare both inS−T orS−T⁰. So we assumex∈T −T⁰andy ∈T⁰−T. Asm <|S|/2, the setS−(T ∪T⁰)is not empty. Pickz ∈S−(T ∪T⁰). Then(x y) = (y z)(x z)(y z)⁻¹ ∈ hG_T, G_T⁰isince y, z ∈S−T andx, z ∈S−T⁰. SohG_T, G_T⁰i=G_T∩T⁰ ∈ P⁰.

Next assumeG= Alt(S). If|S| ≤4, one can directly verify thathG_T, G_T⁰iequals G, G_T or G_T⁰. So assume |S| ≥ 5. Note that G_T∩T⁰ ∼= Alt(S −(T ∩T⁰)) is generated by 3-cycles (x y z) with x, y, z ∈ S −(T ∩T⁰). We claim that every such(x y z)is contained inhG_T, G_T⁰i. This is obvious ifx, y, zare all inS−T or S−T⁰. So we assumex, y ∈T−T⁰andz ∈T⁰−T (the other cases are symmetric).

Pickw∈S−(T ∪T⁰)and let(w z u)be a3-cycle for someu∈S−T − {z, w}. Then(x y z) = (w z u)(x y w)(w z u)⁻¹ ∈ hG_T, G_T⁰isincew, z, u ∈S−T and x, y, w ∈S−T⁰. So againhG_T, G_T⁰i=G_T∩T⁰ ∈ P⁰.

Corollary 6.5. LetSbe a finiteG-set whereGisSym(S)orAlt(S)acting naturally on S. Let P = P_m be the corresponding system of stabilizers of depth m, where m <|S|/2. ThenP_cl =P₊.

Proof. Let P⁰ = P ∪ {G}. Then by Lemma 6.12, we have P₊ ⊆ P₊⁰ = P_cl⁰ = P_cl∪ {G}. IfG∈ P, we haveP_cl∪ {G}=P_cland henceP₊ ⊆ P_cl. On the other hand, ifG6∈ P, none of the groups inP is normal inG, and henceG6∈ P+. So we still haveP+⊆ Pcl.

Remark. The condition m < |S|/2 is necessary: suppose |S| ≥ 6 is even and let m = |S|/2. Partition S into S1 and S2 of the same cardinality m. When G = Sym(S)(resp. G = Alt(S)), the subgroup hGS1, GS2iis the product of two copies of the symmetric group (resp. alternating group) of degreem. It is a proper subgroup ofGbut stabilizes no element ofS. ThereforehG_S1, G_S2i 6∈ P_m∪ {G}. Indeed, we havehG_S1, G_S2i ∈(P_m)₊−(P_m)_clsinceG_S1 ⊆ hG_S1, G_S2i ⊆N_G(G_S1) whereas bothG_S1 andG_S2 are maximal among subgroups ofhG_S1, G_S2iinP_m. Lemma 4.16 now follows from Lemma 6.10 and Corollary 6.5.

We also consider the caseG= GL(V)with the natural action on a vector spaceV. Lemma 6.13. LetV be a finite dimensional vector space over a finite fieldF. Let P =P_mbe the system of stabilizers of depthmwith respect to the natural action of G:= GL(V)onS :=V − {0}, wherem <dim_FV. ThenP_cl =P₊.

Proof. ConsiderH ∈ P₊ and we prove that H ∈ P_cl. Choose H⁰ ∈ P such that H⁰ ⊆ H ⊆N_G(H⁰). It suffices to show thatH⁰ is the unique maximal element in P subject toH⁰ ⊆ H. Assume to the contrary that there exists another maximal elementH⁰⁰ ⊆ H inP. Asm < dimFV, we have H⁰ = GV⁰ andH⁰⁰ =GV⁰⁰ for

some proper linear subspacesV , V ofV. AsH 6⊆ H, we haveV 6⊆ V . Also note thatV −(V⁰∪V⁰⁰)6=∅since

|V⁰∪V⁰⁰|=|V⁰|+|V⁰⁰| − |V⁰∩V⁰⁰|<2|V|/|F| ≤ |V|.

Pickv ∈ V⁰ −V⁰⁰ andv⁰ ∈V −(V⁰ ∪V⁰⁰). Choose g ∈H⁰⁰ =G_V⁰⁰ sendingv to v⁰ which is possible sincev, v⁰ 6∈V⁰⁰. Asg ∈H⁰⁰⊆H ⊆N_G(H⁰) =N_G(G_V⁰), we have^gV⁰ =V⁰. But^gv =v⁰ 6∈V⁰, and we get a contradiction.

Lemma 4.16 now follows from Lemma 6.10 and Lemma 6.13.