Fay is lovely. I don't think it's too fragile sounding. I like both
Fay,
Fae, and
Faye. I think
Fay is probably my favorite spelling.
I live in the U.S. Despite being rare as a first name for little girls these days,
Fay /
Faye is still used regularly as a middle name here.