Young Wharton, on his return, however, showed by his conduct that his visit to Avignon had been little more than a prank, for while he had accepted a dukedom from the Pretender, he, in 1718, being still a minor, accepted a dukedom from the British Sovereign--the single instance of such a dignity being conferred upon a minor.
Wharton, who did everything in haste, had in his seventeenth year eloped with Martha, daughter of Major-General Richard Holmes, and married her in the Fleet on March 2, 1715.

As was only to be expected from a person so volatile he from the beginning neglected his wife; but, as is put quaintly in that unreliable work, _Memoirs of a Certain Island adjacent to the Kingdom of Utopia_, which was concocted by Mrs.Eliza Haywood, "after some years of continu'd extravagance, the Duke, either through the natural Inconsistency of his Temper, or the Reflection how much he had been drawn in by his unworthy Companions to embezel his Estate ...
began to think there were Comforts in Retirement; and falling into the Conversation of the sober part of Mankind, more than he had done, was persuaded by them to take home his Dutchess....

He brought her to his House; but Love had no part in his Resolution.

He lived with her indeed but she is with him as a Housekeeper, as a Nurse." The relations were, however, more intimate than Mrs.Haywood believed, for in March, 1719, a son was born to them.
"The Duke of Wharton has brought his Duchess to town, and is fond of her to distraction; in order to break the hearts of all other women that have any claim on him," Lady Mary wrote to Lady Mar.

"He has public devotions twice a day, and assists at them in person with exemplary devotion; and there is nothing pleasanter than the remarks of some pious ladies on the conversion of such a sinner." The letter from which the above passage is an extract must have been written not later than the early spring of 1720, for after that date the Duke and Duchess of Wharton did not again live together.