I once was in charge of running a summer youth camp, and one of my mentors gave me a great tip of advice in being an effective leader. He said that my job as leader was to oversee running the entire thing. Does that mean I would interact with kids? Of course. But does it also mean I would have to spend many many hours behind the scenes planning and organizing? Absolutely! There were just not enough hours in the day for me to fully interact with kids all day. That is where the necessity of a good team comes into play. The responsibility of community outreach should not lie solely on you. Your job as the leader/owner is to equip your staff in the best possible way to take some of that load off. You should make sure you have a healthy staff with the same vision as you. If you ensure that, you will have a healthy community. This also will set your site up for growth. If you begin by having a good team to back you and allow you to work on other areas, you'll be able to more quickly expand.
To sum it up: Your priority as leader should be to first, continuously be working on growing your site. Second, communicate with your staff. They are there to help...use that. Get rid of bad ones and keep ones who are teachable and willing. And third, listen to your community. You should have enough interaction with your community that you at least know their concerns, wants, etc.
Hope that helps some!