I agree on public by default. The forums of a site are great for learning more about the site and for asking questions, and only certain sections for specific purposes should be private. (Like a forum for staff to discuss things in private on-site, a section just for devs/admins, etc.)
In my opinion, I find it frustrating when I join a site and the forums are blocked off until you get staff approval; especially when you can't even view it.