The Governor's resignation soon followed. His life was filled with much anxiety for the financial welfare of his family as during his eleven years of residence in America, his private fortune had not been increased. He received a pension, but many troubles arose which greatly taxed his physical and mental strength. Mrs. Bernard and the remaining members of her family, moved from their country home at Jamaica Pond, which was afterwards occupied by Sir William Pepperell, to a new residence called the Cherry House, which the Governor caused to be built on a lot of land containing about 30 acres on the "Road leading to Castle William" at Dorchester Neck, now South Boston. The Governor probably selected this location on which to build his house on account of its nearness to Castle Island, to which he and his family could take refuge in case of mob violence.[166] John Bernard's name continued for some time to head the list of proscribed traders and his position, entailing loss, insult, and even danger, must have been a constant source of apprehension to his relatives. After learning that her husband had definitely resigned, Lady Bernard prepared to join him in England. Many of their household possessions were sold at the Province house on September 11. Just before the vessel sailed, young Francis Bernard died November 20, 1770, at the age of twenty-seven, and is probably buried beside his brother Shute in the burial ground of the King's Chapel at Boston. Mrs. Bernard was accompanied by four of her children, Amelia, William, Scrope and Julia. [166] One lot of 26½ acres was purchased of John Baker et al. in 1762. Lib. 98, Fol. 113. Another lot adjoining same, of 3 acres of James Baker in 1764. Lib. 102, Fol. 39. During a raid made by the "Ministerial Troops" from the Castle on Feb. 13th, 1776, nearly all the houses on the Neck were burnt; among them was "An House and Stable and Barn belonging to Francis Bernard burnt; valued at £100.00," also damage done "by our Soldiers," £40.00. (See New Eng. Gen. Reg. Jan. 1897.) This tract of land extended from Fourth street (Way leading to Castle William) to Dorchester Bay, M street running through the center of it. The writer's father in 1858 purchased a portion of this land, and it was here he spent his boyhood days. After the war another house was erected on the site of the one burnt; its location was on Fourth street between M and N streets. The writer remembers that a boyhood companion that lived there picked up in the garden an English guinea.