Pilgrim: Ven. Bhikkhu Bodhi has spoken on this issue - "engaged buddhism". He has founded Buddhist Global Relief to address relief and justice issues. If you do an I-Net search - you can find much material (from his point of view) on this subject. You might also look at the site - Ecological Buddhism.
I think there should be outreach. The temple near my home does prison outreach, for example, but I know that they are seeking to be more in touch with the local community. (Although they do join the Franciscans for the blessing of the animals each year - awww! - I'm not snarking by the way).
One doesn't have to "do" outreach or be involved in the community to 'gather converts' - one does it to connect, to help people - to alleviate suffering. Yeh, I know (the Asura said to the devas) the the thrust of Buddhism is to cool down - to nibbana'ise' - by knowing and understanding suffering, it's causes, it's origination, etc. but personally, I don't think that the personal practise has to negate - shall we say - practise towards others? There are plenty of suttas connected with the blessings of giving - no - sorry - can't site verse and chaptre - but I'm sure someone else here can
Even if there's "no one there" - there's still quite a bit of dukkha (with all its variegated meanings) in this world - and anything one can do to help alleviate it is a good thing -
So, (after all of that) a ringing, affirmative "yes" - I think Buddhists should develop community support!