I do sometimes say, "I am a Buddhist,' but rarely and reluctantly. The main obstacle is that such a simple, unqualified statement aligns me, in the listeners' minds, with religious structures and beliefs I'm ambivalent about as well as those I wholeheartedly support. So I am more likely to say something like, 'I am more Buddhist than atheist or Christian,' or, 'I try to meditate regularly,' (being honest, here, with the 'try'!) or, 'There is a lot in Buddhism that I like.'
The other obstacle to the simple, mostly-truthful statement is that I'm a habitual non-joiner - not quite commitophobic, but close. I usually support organisations from the outside, not the inside.
A separate reason for avoiding the simple statement is that a less absolute statement helps the other person feel more comfortable about talking to me about the dharma, if they want to.