The redshift-angular size relation (or angular size-redshift relation or angular diameter distance) is a function of redshift (or distance) that yields a ratio associated with that redshift. The ratio is that of an object's (e.g., galaxy's) width versus the angle that it traverses over the celestial sphere at that redshift.
In Euclidean space, such a ratio is straight-forward, but given the universe's overall expansion and curvature, and that long-distance observations look backward in time, this function is not so straight-forward and is of interest. Functions can be derived for various Lambda-CDM model parameters and Friedmann models, and observations and analysis revealing the actual relation can be used to select and tweak such models.
One strategy is to compare the brightness with the angular size of distant galaxies, e.g., by measuring the angular size of the brightest galaxies (radio galaxies are suitable) at various redshifts. Another is simply to observe brightness, size, and redshift.
The models under consideration as well as observation indicate that beyond a certain redshift/distance, the angle no longer decreases with smaller with distance, but actually increases. While this is non-intuitive if your intuition takes Euclidean space as its basis, the universe was much smaller then, yet is spread over the entire celestial sphere, so objects must appear "larger". And despite appearing "larger" to us, they still appear dimmer with distance because their light broadcast in all directions must eventually span our current larger universe.