Early in 1778 Smith removed to Edinburgh. He was now in the enjoyment of a certain income of £900 a year apart from the considerable sums which he derived from the sale of his books. He took Panmure House in the Canongate, not far from the deserted palace of Holyrood—a fashionable quarter where some of the Scottish nobility, forsaken by King and Court, still kept their town houses. Panmure House is now a dismantled store; and it needs some imagination to realise how Windham, accustomed to London palaces, should have called it “magnificent,” as he looked from its newly painted windows and plastered walls “over the long strip of terraced garden on to the soft green slopes of the Calton.”[41] The rent was probably very nearly £20 a year. But Smith was one of the richest men in Edinburgh, and felt, no doubt, that he could well afford to take one of the best houses in the city. To share and crown his happiness he brought his mother, his cousin Miss Douglas, and her nephew, a schoolboy David Douglas (afterwards Lord Strathendry), whom he made his heir. From Panmure House “Mr. Commissioner Smith” walked every day to his official duties in Exchange Square, attired in a light-coloured coat, white silk stockings, and a broad-brimmed hat, holding a cane at his shoulder as a soldier carries a musket. He used to turn his head gently from side to side as he walked, and swayed his body “vermicularly,” as if at every other step he meant to alter his direction or even to turn back.[42] His lips often moved, and he would smile like one conversing with an invisible companion. He was not always unaware of his surroundings, and was fond of relating how a market woman in the High Street took him for a well-to-do lunatic. “Hech, sirs!” she cried, “to let the like of him be about! And yet he’s weel enough put on!”