I have had wasps crawling in and out of a crack in my siding for a few years. Nothing ever happened to me nor to my children. I also find nests in the trees around my house. Some I knock down. Some I leave alone. The wasps never bother me unless I bother them, which mostly happens if I'm mowing the lawn and bump into their tree branch as I pass by. Those I try to remove. But I just found one huge nest way up high in a tree. That doesn't bother me as I can't bump into them.
I have thought about killing the wasps many times; the arising of such thoughts is perhaps unavoidable for the time being. It's just a one time thing. Then they will be gone. But then it occurs to me that even if I killed the wasps today, soon enough there will be more wasps. They live in my neighborhood. They will always find their way back to my yard. Do I really want to commit to killing wasps over and over again? It's not worth it to me, killing wasps over and over for only temporary reprieve plus the karmic burden. Plus I would be teaching my kids bad behaviors.
I have these fake wasp nests which I use to dissuade them from moving in. Apparently wasps are territorial and won't move in if they think another colony is already living there. Between knocking down nests which are built and using the fake nests to discourage the building of new nests, I hope I can keep the stinging down to a minimum. Wasps stings hurt!
I wish I could relocate their nests to the edge of my property. Then they could have their nest and I could have a buffer zone of no stinging. I regularly relocate crawling bugs and mice but I haven't figured a way to move the wasp nests yet.