An extension of a result of Gauss to finite groups: a linear algebraic approach