Isaac's two eldest sons were clothiers likewise, but soon after their father's death they retired from business.

He apparently allowed his third son, John, to occupy the house as his tenant, and John was still living there in 1740.
But Isaac had left the house by will in 1732 to his youngest son, Samuel, and Samuel, dying in 1737, left it to his brother Charles, the fourth son of Isaac.

Charles never lived in it, because he spent most of his life in the pursuit of his business as an oil merchant in London, though he is buried among his ancestors in Coggeshall Church.

In 1746 he sold the house to Robert Ludgater and it passed completely out of the Paycocke-Buxton connexion, and in the course of time fell upon evil days and was turned into two cottages, the beautiful ceilings being plastered over.

It was on the verge of being destroyed some years ago when it was bought and restored to its present fine condition by Mr Noel Buxton, a direct lineal descendant of the Charles Buxton who sold it.