Only three stats are technically physical, though charisma is often tossed around as beauty as well.
In truth I only really think strength, dexterity and *potentially* consitution represent actual physical appearance. In particular strength, dexterity and agility have a relationship in real life, whereas in most roleplaying systems they don't.
Strength could range from being small or simply 'not fit' / flabby to very well toned or hulked out.
Dexterity is harder, because its more about movement and grace than it is physical appearance. A 'mid/high strength high dex, but average con' character could look something like bruce lee. Whereas a relatively high strength, high dex character could look like an acrobat or ballet performer.
Constitution is a mixed bag on its relation to appearance, because it's more about 'healthiness' and stamina than anything else. A higher con character is more likely to look 'filled out' than a low one, and in general doesn't get sick very often. They might be 'fatter', but they don't need to be. A very high str / high con character could look like a strongman.
Int and Wis have absolutely no relation to physical appearance, they are mental stats.
Charisma is technically a mental stat, but people like to relate it to beauty, instead of represent it as what it is: force of personality. a low charisma character is a shrinking violet, imagine a meek potentially agoraphobiac librarian that no one really takes any notice of - She could be beautiful still, but no one ever notices her and she lacks the confidence to try and be noticed. Where a high charisma character could be small, half blind, scarred and ugly, yet still command the attention of entire rooms, Tyrion lannister for example is an exceedingly high charisma character, despite not having a nose, mismatched eyes, being a dwarf and in general being regarded as ugly.