The Organisation of the Small Groups Library This chapter describes how the Small Groups library is organised. It includes information on the determination of the groups, on the encoding used to store them, and on the algorithms used by the identification routines. It is aimed at readers interested in the internals of the library; it is not needed in order to use it.
Layers The Small Groups library is organised in 11 layers. Each layer contains the groups of certain orders: Layer 1 the orders with at most 3 prime factors; that is, the orders of type p, p^2, pq, p^3, pq^2 and pqr. Layer 2 all remaining orders at most 1000 except 512 and 768. Layer 3 all remaining orders of type 2^n \cdot p with n \leq 8 and p an arbitrary odd prime. Layer 4 the orders 7^4 and 5^5 and all remaining orders of type q^n \cdot p with q^n dividing 3^6, 5^5 or 7^4 and p a prime different from q. Layer 5 all remaining orders at most 2000 except 512, 1024, 1152, 1536 and 1920. Layer 6 the orders 1152 and 1920. Layer 7 the order 512. Layer 8 the order 1536. Layer 9 the remaining groups of order p^4, p^5 and p^6. Layer 10 the remaining groups of squarefree order and of cubefree order at most 50 000. Layer 11 the orders p^7 with p = 3,5,7,11.

This setup has been chosen for two reasons. First, the organisation in layers permits adding new layers (and thus new group orders) without changes to the existing catalogue. Secondly, it is possible to install a part of the layers only. This might be interesting for computer systems with little hard disk space available.

For each layer i there are two internal functions SMALL_AVAILABLE_FUNCS[i]( order ) and ID_AVAILABLE_FUNCS[i]( order ). Both functions return fail if the given order is not contained in this layer. Otherwise the functions return an information record on the layer and the order. This record contains the information needed to handle the groups of the specified order internally. If the layer i is not installed, then the entries SMALL_AVAILABLE_FUNCS[i] and ID_AVAILABLE_FUNCS[i] are unbound.

Moreover, there are two internal header functions SMALL_AVAILABLE( order ) and ID_AVAILABLE( order ). These two functions are used to loop over the layer functions and find the first layer containing the given order. If this layer is found, then an information record on the order is returned. If there is no installed layer containing the groups of this order, then the functions return fail.
Files and Directories For each layer x there are either one or two directories in the package directory: smallx, or smallx and idx. The directory smallx contains the groups of layer x and the directory idx contains the corresponding identification routine, if available. An exception to this rule is the first layer; here all the code for the groups is contained in the files gap/smlgp1.g and gap/idgrp1.g. For the layers 7, 8 and 11 there is no identification routine available at the moment and thus there are no directories id7, id8 and id11.

The other relevant files of the package are: read.g for loading the small groups catalogue into &GAP;. gap/small.gd, gap/small.gi these are the files where the main functions for the small groups catalogue are defined. gap/gap3cat.g identification of a small group in the &GAP; 3 SolvableGroup catalogue. gap/smlinfo.gi the code for the function .
Layer 1 The groups whose order factorises into at most 3 primes have been classified by Hölder, see . An algorithm to find the isomorphism type of such groups was first implemented in &GAP; 3 by Frank Celler and Hans-Georg Esser.

Hölder's classification distinguishes the groups of the following order types. Let p, q and r be different primes with p < q < r. p 1 cyclic group. p^2 1 cyclic and 1 elementary abelian group. p^3 3 abelian groups and 2 extraspecial groups. pq 1 cyclic group; 1 group of type q:p if q \equiv 1 \bmod p. p^2q 2 abelian groups; 1 group of type q:p \times p if q \equiv 1 \bmod p; 1 group of type A_4 if p = 2 and q = 3; 1 group of type (p \times q).p if q \equiv 1 \bmod p; 1 group of type q:C_{p^2} if q \equiv 1 \bmod p^2. pq^2 2 abelian groups; 1 group of type q:p \times q if q \equiv 1 \bmod p; 1 group of type C_{q^2}:p if q \equiv 1 \bmod p; 1 group of type (q \times q):p if p > 2 and q + 1 \equiv 0 \bmod p; groups of type q:p + q:p (1 group for p = 2 and (p+1)/2 groups otherwise). pqr 1 cyclic group; 1 group of type q:p \times r if q \equiv 1 \bmod p; 1 group of type r:p \times q if r \equiv 1 \bmod p; 1 group of type r:q \times p if r \equiv 1 \bmod q; 1 group of type r:(p \times q) if r \equiv 1 \bmod pq; p-1 groups of type q:p + r:p.

The groups in this layer are stored generically; that is, using functions instead of explicit presentations. The identification routine follows the classification as well.
Layer 2 Determination The nilpotent groups of this layer have been constructed using p-group generation, see and the ANUPQ package. The 2- and 3-groups of this layer have also been available in the TwoGroup and ThreeGroup libraries of &GAP; 3, see and . The non-nilpotent groups in this layer have been determined using the Frattini extension method, the cyclic split extension method and cyclic extension for the non-solvable groups, see , and . Storing The solvable groups in this layer are stored by a single long integer describing a pc presentation of the group (see or ). For p-groups the encoded pc presentations are standard presentations and they are equal to the presentations known in the 2- and 3-groups libraries of &GAP; 3. The non-solvable groups are stored by generating permutations. Selection arguments For the selection functions the values of the following attributes are precomputed and stored: for p-groups , , , FrattinifactorSize and FrattinifactorId. for non-p-groups , , , , LGLength, FrattinifactorSize and FrattinifactorId. Identification The identification routine uses invariants of groups to identify a given group. The invariants are organised as a tree and stored in ID_GROUP_TREE, which contains a leaf for every group. In some cases this invariants tree is not sufficient to distinguish all groups. In this case the tree is used to reduce to a small number of possible groups and then a special random isomorphism test finds the correct group among the possible ones.
Layer 3 Determination These groups have been determined using the generic variation of the cyclic split extension method and the Frattini extension method, see . Storing: nilpotent groups For each order in this layer we first list the nilpotent groups P \times Q where P is the cyclic group of order p and Q is a group of order 2^n. These groups are sorted by the catalogue number of Q and the group Q is stored in layer 2. Storing: groups with normal Sylow p-subgroup Then we consider the remaining groups with normal Sylow p-subgroup. These are split extensions of type P : Q. The isomorphism type of these groups depends on the isomorphism type of Q and the action homomorphism Q \to \mathrm{Aut}(P). Let K be the kernel of this homomorphism with [Q : K] = 2^i for some i. We encode the homomorphisms for fixed Q and fixed i as follows.

Consider a generator g of P and let g_1, \ldots, g_r be a generating set of Q (we choose the first r pc generators of Q for r the rank of Q). Consider a homomorphism a : Q \to \mathrm{Aut}(P) and let a_i be the image of g_i. Clearly, a_i is of the form a_i : g \to g^{l_i} with 0 \leq l_i \leq 2^i-1. Thus we describe the homomorphism a by the sequence l_1, \ldots, l_r. We encode this sequence as a 2^i-adic number of length r; that is, we encode this sequence as l_1 q^{r-1} + l_2 q^{r-2} + \ldots + l_r q^0 where q = 2^i. Hence each homomorphism a is encoded as an integer and for fixed Q and i we write all these integers into a list.

A special case occurs if there are two homomorphisms a and b such that a_i = b_i^j for some integer j; that is, the images of a are powers of the images of b. In this case a is stored directly after b and we only store the integer -j instead of the encoding for the sequence l_1, \ldots, l_r for a.

Now we consider the case that for certain Q and i we have a list L consisting of non-negative integers only; that is, the special power-case did not occur for this group Q and index i. In some cases we encode such a list L further. Recall that the integers in L correspond to different homomorphisms. Thus the integers are all different and we may sort them in increasing order. Then we can store this sorted list as a binary number; that is, the list c_1, \ldots, c_s is encoded as 2^{c_1-1} + \ldots + 2^{c_s-1}. This is only useful if the list L does not contain large numbers.

Thus for Q and i we have either a list or an integer stored to encode the corresponding homomorphisms. For each index i we write all lists or integers into a list such that at position \mathrm{Id}(Q) the corresponding list or integer to Q and i is found.

For i = 1 this list has no holes, since there exists at least one group P : Q with K of index 2 for each group Q. Often the entry at position j in this list is equal to the entry at position j-1. In this case we delete the entry at position j and thus create a hole in the list.

For i > 1 there might be holes in the list if there is no group P : Q with K of index 2^i. Hence the previous idea cannot be applied for i > 1. However, in all the lists we have equal entries in various places. If this is the case, then we substitute an entry in a list by a negative integer -j meaning that the entry at this position is equal to the entry at position j.

Finally, if the list for some index i has only few entries left, then we substitute the list by a record containing two lists: first the non-empty positions and secondly their entries. Storing: groups with normal Sylow 2-subgroup Next we consider the remaining groups of type Q : P. These exist for a few primes p only. These groups are described by operation homomorphisms of the type P \to \mathrm{Aut}(Q). Here we store the catalogue number of Q and a long integer for each such homomorphism a : P \to \mathrm{Aut}(Q). Consider a generator g of P. Clearly, we just need to store the image of g for each homomorphism a. The image g^a is an automorphism of Q and thus we store the images under g^a of a fixed generating set of Q. For this purpose we consider a canonical numbering of the elements of Q and store the generator images by storing their number in the element list. The sequence of these numbers is encoded as a (|Q|+1)-adic number; that is, the sequence e_1, \ldots, e_r is stored as e_1 q^{r-1} + e_2 q^{r-2} + \ldots + e_r where q = |Q| + 1. Storing: groups without normal Sylow subgroup Now there are the groups of order 2^n \cdot p without any normal Sylow subgroup left. These exist for very few primes p only. They have been computed as in layer 2 using the Frattini extension method. They are stored as described in layer 2 using one long integer for each group. Identification The identification of nilpotent groups P \times Q relies on the identification of Q only and this is done as in layer 2. Similarly, the groups without normal Sylow subgroup are identified as described in layer 2.

For the groups Q : P with normal Sylow 2-subgroup we first determine the prime p and the isomorphism type of Q. If this is not sufficient, then we use the methods as described in layer 2. (Recall that there are only finitely many primes p possible here.)

For the groups P : Q with normal Sylow p-subgroup we first determine the prime p and the isomorphism type of Q as well. Moreover, we compute the centralizer K of P in Q and consider its index [Q : K] = 2^i.

In a large number of cases the catalogue numbers \mathrm{Id}(Q) and \mathrm{Id}(K) are sufficient to determine the group P:Q uniquely. Moreover, this feature is independent of p. We can recognise these cases using the tree of invariants: if we can determine P:Q uniquely with \mathrm{Id}(Q) and \mathrm{Id}(K) only, then there is no node corresponding to this case in the tree.

If there is a node in the tree, then \mathrm{Id}(Q) and \mathrm{Id}(K) are not sufficient to determine the group uniquely. In this case we apply methods similar to layer 2; that is, we use invariants. Since there are infinitely many primes p which might turn up here, we need an additional idea to use the invariant tree for this purpose. In the invariant tree we have invariants stored for groups of type R : Q of order 2^n \cdot r for the primes r = 3, 5, 17 and 97. Now we first choose the smallest prime r such that r is congruent to 1 modulo [Q : K]. The cases that [Q : K] = 2^i for i = 6, 7 or 8 cannot occur here, because in these cases \mathrm{Id}(Q) and \mathrm{Id}(K) are always sufficient to determine the group.

Then we use that the groups P : Q behave essentially similarly to the groups R : Q for the chosen r. Thus we can construct a group R : Q corresponding to P : Q and identify R : Q instead of P : Q.
Layer 4 Determination The groups of order 2401 = 7^4 and 3125 = 5^5 have been computed using p-group generation as described in layer 2 and they are stored and handled similarly. The other groups in this layer have been determined as outlined in layer 3 and, again, they are organised similarly. Storing There is a difference from layer 3 in the compression of the group data; that is, the groups in this layer are not compressed as efficiently as the groups in layer 3. We consider the groups of type P : Q for P the cyclic group of order p and Q a group of order q^n. These groups are determined by Q and an operation homomorphism Q \to \mathrm{Aut}(P). As in layer 3 we first compute one integer for each operation homomorphism and then proceed to compress the lists of integers for each Q and each i where q^i is the index of the centralizer of P in Q. Unlike in layer 3, we do not compress the lists of non-negative integers to a single integer and we do not reduce the list for i = 1 further, since in both cases the obtained compression is very small. Identification The group identification is essentially similar to the procedure described in layer 3. However, for the groups with normal Sylow p-subgroup we additionally use a special random isomorphism test to identify groups. It is a special version of the general algorithm: in the general algorithm we use certain sets to choose generating sets from. In this special version we restrict these sets suitably. (We use non-trivial elements of P and certain elements of Q.) This is necessary, since the groups which appear here can be too large to compute explicitly the elements sorted in conjugacy classes.
Layer 5 This layer is similar to layer 2.
Layer 6 Determination The nilpotent groups in this layer are determined as direct products of p-groups. Most of the groups of orders 1152 and 1920 have a normal Sylow 3-subgroup or Hall \{3,5\}-subgroup, respectively. They are constructed using the coprime split extension method, see . The remaining groups of these orders (without normal 2-complement) have been determined using the Frattini extension method and the cyclic split extension method, see also . Storing Only the non-nilpotent groups with normal Sylow 3-subgroup or Hall \{3,5\}-subgroup are stored in a special way. The remaining groups are organised similarly to layer 2.

For the groups of type P : Q where P is the normal Sylow 3-subgroup or the Hall \{3,5\}-subgroup and Q is a group of order 2^7 we split a pc presentation in two parts. The first part are the relators of Q. These are stored within the groups of order 2^7 and thus can be considered as known. The second part of the relators determine P and the operation of Q on P. These second parts are stored as long integers. If we consider these second parts for all the groups, then we find that they are highly redundant. There are only 921 (1722) different long integers for the groups of order 1152 and 1920, respectively. Thus we store these two sets of long integers only.

Then for each group Q of order 2^7 we store for each extension P : Q the position of the corresponding long integer. Hence we obtain a list of integers for Q. Again we note that the lists of integers obtained for all the groups Q of order 2^7 are highly redundant. There are only 1298 (722) different lists for the groups of order 1152 (1920). Thus we can compress the lists using the same idea again. Identification The identification of groups of these orders is similar to layer 2.
Layer 7 Determination The groups of order 512 have first been enumerated, see and . Later they have been determined explicitly using p-group generation, see and . There are 10 494 213 groups of this order. Storing We introduce a special method to store such groups. Consider a pc presentation of a given group of order 512. If we concatenate the exponent vectors of the right hand sides of such a presentation, then we obtain a bit vector of length 120. This can be used as encoding of a group of order 512. We compress the list of such encodings further using the following method.

First we split the list of all groups of order 512 in sublists containing 1000 groups each. Let L be such a sublist. We suppose that L consists of 1000 bitlists of length 120 and we describe a method to encode this list of bitlists in a single vector V using an alphabet of 83 symbols (81 symbols for the encoding and 2 special symbols as signs).

First we find those indices i in the range 1 to 120 such that for all lists in L the entries at position i are fixed. Thus we note for each of the 120 entries whether the entries are all 0, all 1 or variable in L. There are 3^{120} combinations possible, which we store in a vector of length 30 using 81 symbols. This vector of length 30 is the initial segment of V. Now we can reduce to consider the variable positions only and thus work with bitlists of shorter length.

We split the shorter bitlists in blocks such that the bitlists in each block have at most 6 variable entries. For this purpose we start at the beginning of L and add bitlists to the current block until the next bitlist would lead to more than 6 variable entries for this block. Each block can contain from 1 up to 64 bitlists. Now we determine a headcode and a tailcode to describe the block. The head and tail codes are then concatenated to V. The head contains an encoding of the variable bits for this block as above. The tail then describes the vectors in the variable bits. Since there are at most 64 possible combinations arising here, we need only one letter in the 81-alphabet to describe a single vector of variable bits.

Finally, we note that some of the tailcodes arise for many blocks. These tailcodes are stored in a special list. If such a tailcode is occurring in V, then only the reference to the special list is stored instead of the full tailcode. This special list contains 1890 different tailcodes, so 2 letters in the alphabet are sufficient to refer to a tailcode.

To distinguish references and actual tailcodes we use a special sign. Each new head begins with a special symbol in V. There are two special symbols available for this purpose. One is used if the corresponding tailcode is referenced, and the other if it is an ordinary tailcode.

This compression allows the groups of order 512 to be stored in 4.4 MB.
Layer 8 Determination The 408 641 062 groups of order 1536 have been determined using the cyclic split extension method and the improved version of the Frattini extension method, see and . Storing We store these groups in a special way as well and we obtain a compression to 6.1 MB. The groups are stored using references to the groups of order 512 of layer 7.

The nilpotent groups C_3 \times Q for Q a group of order 512 are based on the groups of order 512.

The major part of the groups are the 398 032 384 non-nilpotent groups with normal Sylow 3-subgroup. The compression of these groups follows the one outlined in layer 3. As described there, for each group Q of order 512 we compute an integer which determines all groups of type C_3 : Q. There are only 15 249 different integers of this type. In a first compression we create a list of references to the different integers.

Now we note that a group often has the same reference as the previous one. Thus we split the groups of order 512 in sets of 100 000 groups and create a record for each set. The record contains two lists which are used to recall the blocks in a set having the same reference. Thus the first list contains the length of the blocks and the second list contains the corresponding references.

Again, in both lists certain patterns occur often. In the first list the length 1 appears often. Thus the 1 is not stored explicitly, but a blank in this list has to be considered as 1. In the second list often an entry is the same as the one 2 places before. Thus these entries are deleted as well and blanks can be reconstructed by this rule.

Additionally we store the number of resulting groups for each integer to speed up the method to find the group with a given catalogue number.

The 18 028 groups with normal Sylow 2-subgroup are stored similarly to layer 3. As described for layer 3 we determine a long integer for each homomorphism Q \to \mathrm{Aut}(P) for Q of order 512 and P the cyclic group of order 3. The occurring integers are highly redundant – there are 6774 different ones. Thus we use a list of references again.

The 96 437 groups without normal Sylow subgroup are determined using the Frattini extension method. As described in layer 2 they can be stored by encoding the relators of a pc presentation as long integer. However, the relators corresponding to the Frattini factors of the groups are known. Thus it is sufficient to store the relators of the Frattini factors once for all groups with this factor and encode the remaining relators only. Note that there are only 16 different Frattini factors and only 12 of them have order less than 1536. The groups are stored in lists corresponding to the 12 Frattini factors.
Layer 9 This layer contains the generic construction of the groups of order p^4, p^5 and p^6 greater than 3125 (those of order up to 3125 are contained in the lower layers of the Small Groups library). Determination The groups of order p^4 were first determined by Hölder (1893), see . The groups of order p^5 were first determined by Bagnera (1898), see . The groups of order p^6 were first (correctly) determined by Newman, O'Brien and Vaughan-Lee (2004), see . Storing The groups of order p^4 (p \geq 11) are given by pc presentations which are produced at run-time by a function SMALL_GROUP_FUNCS[19] provided by Newman.

The groups of order p^5 (p \geq 7) are given by pc presentations which are produced at run-time by a function SMALL_GROUP_FUNCS[20] provided by Girnat, see .

The groups of order p^6 (p \geq 5) are given by pc presentations which are produced at run-time by a function SMALL_GROUP_FUNCS[21] provided by Newman, O'Brien and Vaughan-Lee, see .

The groups of order p^6 are given as a list of partially repetitive structures. These are compressed into the file sml1.z. At run-time, this compressed structure will be expanded, but restricted to those parts of the structure relevant for the given p when needed. It is cached into SMALL_GROUP_LIB[1]. As parts of this structure contain long (O(p^2)) lists of groups which are classified by additional parameters and these lists are not dense, it might be necessary to set up the complete list of indices to find the presentation of a single group. Identification A group of order p^4 can be identified and its catalogue number found by the function ID_GROUP_FUNCS[19] based on a function provided by Newman and modified by Besche.

There is no identification available for the groups of order p^5 and p^6 at present.
Layer 10 Groups of squarefree order These groups have first been determined by Hölder (1895), see . The implemented construction is based on the determination given in . The key invariants are the socle and socle factor.

The groups of squarefree order which are not contained in lower layers are given by pc presentations which are produced at run-time by a function SMALL_GROUP_FUNCS[24] written specially for this library. Such a group can be identified and its catalogue number found by the function ID_GROUP_FUNCS[24]. Groups of cubefree but not squarefree order These groups were first determined in . Every group with cubefree order is either solvable or has a direct decomposition into a solvable group and PSL(2,p) for a suitable prime.

The groups of cubefree order at most 50 000 which are not contained in lower layers and are not squarefree are given by pc presentations which are produced at run-time by a function SMALL_GROUP_FUNCS[25] written specially for this library. Such a group can be identified and its catalogue number found by the function ID_GROUP_FUNCS[25].
Layer 11 Determination The groups of order p^7 have been determined by O'Brien and Vaughan-Lee, see . This layer contains these groups for some small primes: p = 3,5,7,11. Storing The groups are encoded by and then the various codes are stored in compressed form. See the groups of order 512 for details. Pc presentations for the groups are produced at run-time for these groups. Identification There is no identification function available for the groups in this layer.
Concluding remarks The compression of group data for layer 2 has been developed long ago. A first version has been used in the p-group generation algorithm and thus to encode the 2- and 3-groups library of &GAP; 3. This compression is described in .

The remaining compression methods for layers 3 – 8 have been introduced by Hans Ulrich Besche specially for this library. They are designed to store the groups in as little space as possible. For different layers there are different methods, since certain approaches had been successful for certain groups but not for others. Moreover, already published layers have not been changed again and thus useful concepts sometimes have not been applied to earlier layers.