I think we draw the line in terms of wise conduct that tends to be conducive for liberation. Of course we're not always going to know what that conduct is, so we just have to do our best.
But for example: If I witness a person in the grocery store physically beating his 3-year-old son, then it's my business. Out of compassion for both of them, I have to step in and intervene so that the child is not seriously hurt. If I decide it's none of my business, that is not conducive for liberation, because through inaction in that moment I forego an opportunity to develop parami, and I become a participant in the abuse by enabling it through unwise tolerance of violence. In a sense, by not "acting" I am still acting out of fear by suppressing my impulse to want to help and finding excuses not to do so.
On the other hand, if I witness a person in the grocery store scolding his 3-year-old son in a manner that I personally deem inappropriate, or not scolding a 3-year-old whom I believe should be scolded, it's none of my business. Out of compassion for both of them, I have to refrain from stepping in to intervene, because the result is likely to be an argument with the father and confusion for the young boy. Basically, I would just be trying to impose "my" standards of behavior on someone else. And even if it's true that I would handle the situation better than that person, it's none of my business.