Col. Oliver Hazard Payne was born in 1839 and named for Oliver Hazard Perry, the U.S. naval officer who became a national hero when he defeated a British squadron in the Battle of Lake Erie in the War of 1812. Growing up in Cleveland, Ohio, Payne was a primary school classmate of John D. Rockefeller. He attended Yale University, where he was a classmate of William C. Whitney, and entered the 124th Ohio Infantry as a first lieutenant in 1862, serving through the Civil War. After being promoted to colonel, he was brevetted brigadier general of volunteers in recognition of his meritorious service during the war.

After the war, Payne became interested in iron manufacturing and oil refining in Cleveland, founding Clark, Payne & Co. In 1872, his company was purchased by Rockefeller, and Payne went on to serve as treasurer of Rockefeller’s Standard Oil Co. and to become one of the richest men in America. In 1998, American Heritage Magazine placed him at number 26 on a list of the “40 wealthiest Americans of all time.”

In 1905, he purchased the estate of John Jacob Astor in Esopus, N.Y., and supervised the construction of a mansion designed by the firm Carrère and Hastings. He became a philanthropist, donating to many educational and medical causes. Cured of a serious illness by physician Alfred Loomis, Payne became interested in assisting the medical profession. In 1887 he endowed the Loomis Laboratory in New York City for teaching and research in chemistry, biology, and pathology. In 1889 he donated $500,000 to found Cornell Medical School, and his subsequent donations to the school totaled more than $8 million. He gave New York University $150,000 for its medical school and $100,000 each to New York City's Post-Graduate Hospital, the University of Virginia, and Western Reserve University to establish laboratories of experimental medicine. He also donated $1 million to Lakeside Hospital in Cleveland, $200,000 to St. Vincent's Charity Hospital in Cleveland, and $200,000 to the Cleveland Jewish Orphan Asylum.

In his will he left $500,000 to Phillips Academy in Andover, $200,000 to Hamilton College, $200,000 to the University of Virginia, and $1 million to Yale University. He also left $1 million to the New York Public Library.

 
 
Payne was also considered one of the best yachtsmen in America. He spent most of his later summers aboard his yacht, the Aphrodite, which he had built by Bath Iron Works of Maine. Delivered in 1898, it was the longest steam-powered yacht in the world at 330 feet. He traveled to Europe and the Mediterranean every year from 1898 until 1914, after which he confined his sailing to U.S. waters.

Payne died in 1917, leaving the Esopus property to his nephew Harry Payne Bingham. In 1933, Bingham donated the Esopus estate to the Episcopal Diocese of New York. From 1937 to 1966, the site served as the Wiltwyck School for Boys, a noted home for troubled children in which First Lady Eleanor Roosevelt took great interest.

In 1942, the Wiltwyck School became nonsectarian and the property was divided, with a portion sold to the Marist Brothers, the founders of Marist College. Until 1986, the brothers’ part of the estate, including the Payne Mansion, was used as a school and retreat house. Raymond A. Rich purchased the mansion, boathouse, and 60 acres from the brothers in 1986 and restored the estate and its boathouse to their former glory.