Dilworth maintained a sophisticated and polished lifestyle in Philadelphia.  His home at the intersection of Walnut and 22nd Streets was stylishly decorated with 18th-century English furniture and Venetian paintings.  Dilworth himself exuded an air of elegance as well: his hair was always perfectly styled, his suits matched the latest fashions, he spoke fluent French, and he always enjoyed a cocktail party.  He often mingled with the social elite, including author John O’Hara, literary critic Dorothy Parker, the Mellon family, and the New York businessmen and bankers who founded Southampton as a summer retreat for the rich.  With this crowd, Dilworth was often tagged as the most handsome and was admired for his movie star appearance.

In addition to his political and social careers, Dilworth was a family man.  He raised his children in the world in which he worked.  They handed out flyers and brochures at political rallies endorsing their father’s campaign and heated constituents would often throw them back.  In 1957, Dilworth and his family moved to 223-225 South Sixth Street on Washington Square East in Society Hill to encourage a revival of the neighborhood as well as historic preservation initiatives across the city.  The house was always quite full, for along with the six Dilworth children living there, the family owned twelve miniature toy poodles.