Because I
run browsers as different users I need all three different users to be able to play sound and by default pulse will only allow the currently logged in user to play sound.
To let all three users play sound (usually when I click on a flash video or stream play music from
Pandora) it's sufficient to edit /etc/default/pulseaudio, set PULSEAUDIO_SYSTEM_START=1, and then restart pulse (with a logout/login or an X restart).
On Mint that port isn't accessible to the network by default. if it is though, then add appropriate iptables rules so that network access is disabled.
Any local users will now be able to access pulse, so only do this if you trust your local users.
UPDATE:
In 2016, the way to do this is now via paprefs. Install the paprefs package and run paprefs as the main user (i.e., I'd run it as tiger and allow localhost only access to sound).
Do this only if you trust all users on the machine.