yes you raise a valid point regarding Fitness, The Mean gives you a starting point, you make slight adjustments to the Mean based on past analysis of your population(database)or if you have enough specific data on each horse you would use its own data.
While your looking at z scores the other relevant area for horses, is to describe its distribution pattern by that you look at the Kurtosis of its data set.