Do all wallet addresses start with rdx1qsp? If so, why not just rdx?

Radix wallet addresses are bech32 encoded. rdx is the Human Readable Part (hrp), 1 is a separator between the hrp and the rest of the address, and the qsp is because every Radix wallet address is a 33 byte compressed public key starting with 0x02 or 0x03 that is prefixed with a 0x04 byte.

Ref: Telegram


To add to Stuart’s explanation…
The first 15 bits of 0x0402 or 0x0403 are (grouped into 5 bit pieces):
00000 10000 00001
Which maps to ‘qsp’ under the bech32 character set.
The mapping for those interested is: