Well, he was raised by 5 sisters... makes sense to me!
Here's the book that convinced John it would be okay for Yoko and him to explore other sexual partners--the notion that eventually became The Lost Weekend.
Portrait of a Marriage: Vita Sackville-West and Harold Nicolson
by their son Nigel Nicolson:
Here's the Amazon blurb: Vita Sackville-West, novelist, poet, and biographer, is best known as the friend of Virginia Woolf, who transformed her into an androgynous time-traveler in Orlando. The story of Sackville-West's marriage to Harold Nicolson is one of intrigue and bewilderment. In Portrait of a Marriage, their son Nigel combines his mother's memoir with his own explanations and what he learned from their many letters. Even during her various love affairs with women, Vita maintained a loving marriage with Harold. Portrait of a Marriage presents an often misunderstood but always fascinating couple.