I agree with you that the
Christmas association is really strong and I think it would bug me a little to see this name used on a child who wasn't born around Christmastime. So I don't have any suggestions for how to get this theory out of your head haha.
Noel doesn't
look manly to me, but I would assume that the name rhymed with the word "mole" and if I didn't know the child's gender, I'd assume it was a boy. I think if you want the no-ELL pronunciation, it'd be best to spell it
Noelle.
What about
Nola or
Nuala?