After he was rejected from the Marines, the Air Force, and the Navy due to his "hammerhead toes," Pat Cooper returned to his hometown of New York City and married Dolores Nola, per The New York Times. The couple would go on to have two children: Michael and Louise Caputo. This was when Pat was trying to build a career in comedy. In 1961, Pat and Dolores divorced. Pat rarely saw his children.

After Michael wrote a book about their lack of relationship, he was invited on Geraldo to chat about it. Pat called into the show, but it wasn't to explain himself. "Let me tell you something, I don’t have to be your father, you’re not that thrilling,” he said. "And I don’t want to be your father." Later, Pat would accuse his ex-wife and biological children of being jealous of his success and only wanting money from him.

The New York Times profiled the late funnyman in September 2010. He told the outlet, "To them, I became successful to hurt them," and said "it would be hypocrisy of the highest order" to reconnect. That didn't stop his son Michael from trying, who said on his website, "It never failed; no matter how many times I tried to connect with my father Pasquale Caputo, all I ever got from him was Pat Cooper the angry comedian."

Pat married his second wife, singer Patti Del Prince, in 1964. He met Patti at the Silver Edge, a club in the Bronx. "I was her opening act," he wrote in his memoir "How Dare You Say How Dare Me!: An Autobiography of a Life in Comedy." He described her as "petite, pretty," and someone who "sang her a-- off." They stayed married for 41 years until her death in 2005. Together they adopted a daughter who Pat maintained a relationship with.