I think one could wear a tie, or at least a bow, with a peter pan collar, which is generally a women's style.

Going back in time, blouses stop being their own thing, and plenty of upper-class shirts had lace or poofy sleeves. That's not so helpful for this distinction, though.

And now I want to figure out a corduroy blouse, though I'm not sure that'd work ...