Never again did this genial gentleman sun himself in the rays of Executive power, or recover the smallest share of influence. He returned to Baltimore, where he lived thirty years longer without distinguishing himself; but about three months after his retirement from office, in the month of June, 1811, he published an Address to the People, charging President Madison with offences more or less grave, and surprising every one by representing himself as having persistently but vainly opposed Madison’s fixed purpose of making a virtual alliance with France. The evidence of the late Secretary of State, who might reasonably be thought the best informed and most competent judge, confirmed the Federalist and British theory that Madison was under secret pledges to Napoleon. So gravely did it compromise Madison that he caused Joel Barlow to write a semi-official reply in the “National Intelligencer;” and although Barlow wrote in a bad temper, Madison himself wrote privately in a worse. “You will have noticed in the ‘National Intelligencer,’” he told Jefferson July 8,[293] “that the wicked publication of Mr. Smith is not to escape with impunity. It is impossible, however, that the whole turpitude of his conduct can be understood without disclosures to be made by myself alone, and of course, as he knows, not to be made at all. Without these his infamy is daily fastening upon him, leaving no other consolation than the malignant hope of revenging his own ingratitude and guilt on others.” Robert Smith hardly deserved such invective. If the taunts of Madison, Barlow, and the Republican press and party at his incompetence were well-founded, the party had only itself to blame for putting such a man in so high a position. If triumphant in nothing else, Smith overthrew both Madison and Barlow by the retort with which he met their sneers, and retaliated the charge of incompetence.