Subgrid-scale physics (or just subgrid physics) are simulations of physical processes that are applied to each point in a simulation that involves physics. In astrophysics, such a simulation could be of clouds, stars, atmospheres, galaxies, supernovae, black hole-instigated phenomena, etc., that use three dimensional models or two dimensional models. Such models are typically organized as a 3D (or 2D) grid and step through time simulating the interaction between points on the grid or volumes outlined by the grid. This nicely handles interactions that operate on a scale larger than the grid, but helps less for processes that operate on a smaller scale (a subgrid scale). Anything smaller than the grid size must be handled in an "averaged" manner: i.e., "here's what's likely to be the result of all these small-scale interactions".
An example would be an atmospheric model (e.g., a numerical weather prediction (NWP) model) that lays a grid over the surface of an extra-solar planet at roughly 100-km intervals (the distances must vary because the surface of a sphere can't be covered by uniform grids of just any size). The effect of eddies and small gusts, updrafts, downdrafts that are much less than 100 km in scale wouldn't be simulated by the interactions among regions discretely handled by this grid. Thus, an averaged result (generally a function of the current conditions at that point) of the expected small-scale processes must be modeled, i.e., subgrid-scale physics.
This is analogous to the averaging of processes happening in shells (i.e., altitudes, or distances from the center of a sphere in one dimensional models (e.g., one dimensional climate models or simple stellar structure models). The processes in a whole shell must be "averaged", and, for example, convection is only handled by an averaging of what convection is likely to be produced. Convective parameterization is the development of general parameters usable to account for the sum of the effects of convection on a subgrid scale.