Validator sets can only get so large before the messaging complexity gets in the way, causing a direct reduction in throughput (TPS).
So what is the messaging complexity that the official docs are referring to?
The validator nodes communicate with each other using a mesh topology - that is, each validator node in the validator set connects to every other validator node in the validator set.
For a validator set of 100, this means that there are 4950 channels of communication (100 * (100 -1) / 2) between all of the validator nodes assuming a full mesh network.
If the validator set was increased to, say, 250 nodes, then there would be 31,125 channels of communication between all of the nodes.
The more validators that need to communicate with each other to achieve consensus, the longer it takes - and therefore the throughput decreases.
A value of 100 was chosen for the size of the validator set as it strikes a balance between safety and speed.