Is 1.7 x 10^10 Unknowns the Largest Finite Element System that Can Be Solved Today?

The hierarchical hybrid Grids (HHG) framework attempts to remove limitations on the size of problem that can be solved using a finite element discretization of a partial differential equation (PDE) by using a process of regular refinement, of an unstructured input grid, to generate a nested hierarchy of patch-wise structured grids that is suitable for use with geometric multigrid. The regularity of the resulting grids may be exploited in such a way that it is no longer necessary to explicitly assemble the global discretization matrix. In particular, given an appropriate input grid, the discretization matrix may be defined implicitly using stencils that are constant for each structured patch. This drastically reduces the amonnt of memory required for the discretization, thus allowing for a much larger problem to be solved. Here we present a brief description of the HHG framework: detailing the principles that led to solving a finite element system with 1.7 x 10^10 unknowns, on an SGI Altix supercomputer, using 1024 nodes, with an overall performance of 0.96 TFLOP/s, on a logically unstructured grid, using geometric mmiltigrid as a solver.