During the summer months he zealously promotes the holiday amusements of his children. Visitors, calling to see him on public affairs, would find him, not in a library, but in a drawing-room surrounded by his family. In the autumn he visits his birthplace, Richmond in Yorkshire. Thence he goes to Inverary to be the guest of the Duke and Duchess of Argyll, with both of whom he forms a lasting friendship. Then he receives the Freedom of the City of Glasgow and returns to London. Early in the following year, 1861, he leaves London and takes a roomy old-fashioned house at Southgate, a few miles to the north of London, where he remains for the remainder of his sojourn in England. To the house is attached some land where he may indulge his taste for farming and his fondness for animals. In the week days he attends the Council of India in London, but his summer evenings he spends at home with his family, and mainly lives a country life. His position in the Indian Council, where Sir Charles Wood (afterwards Lord Halifax) had succeeded Lord Stanley as Secretary of State for India, was not such as to call his individuality into play. Though he had a voice in the affairs of India, he was no longer a man of action. Even then, however, he impressed his colleagues favourably, and especially the Secretary of State. He felt and expressed great regret at the abolition of the local army of India, and its amalgamation with the army of the Crown. He was not what is termed in England a party man, but he certainly was a moderate Liberal in politics. As a churchman of the Church of England, he was content with his Bible and the Book of Common Prayer. In 1862 he met Lord Canning, who had resigned his high office as Governor-General, returning home very shortly to die. Then he saw Lord Elgin appointed to fill the important place.