From his small retired parish, among the companions of his childhood, he was called to Torrington, Connecticut, where he continued preaching two years to large audiences.[10] It is said that at Torrington a leading citizen was much displeased that the church should have "a nigger minister," and, to show his disrespect, this man went to church and sat with his hat on his head. "He hadn't preached far," said he, "when I thought I saw the whitest man I ever knew in that pulpit, and I tossed my hat under the pew." The number of communicants increased during the term of his residence in Torrington. Some of the most respectable families from adjoining towns, particularly from Goshen, became his warmest friends, who constantly attended on his ministry. His biographer says: "The aged refer to his ministry with many delightful recollections. He was held in high estimation, especially by the church, and was esteemed by all classes as "an apt and very ready man in the pulpit." The mere mention of his name even now, after the lapse of half a century, seems to renew in their minds interesting associations. The church and society were strengthened by his labors, and many wished to retain him as their permanent pastor. The sensibility of a few individuals prevented, it is said, the accomplishment of their desires. His eloquence and Christian nobility won him much attention and led to his being called to the pastorate of the Congregational Church in West Rutland, Vermont. The town was a country seat, and the church was one of importance. Then in the meridian of life, rich with the spirit and devoted to his calling, he was singularly successful; and while there were those who saw in him "that colored minister," all knew his pure white soul. The first year of his pastorate he received forty-two members by profession. In 1803 there came a great revival, and there were one hundred and three conversions, together with one hundred and fifty in the adjoining town of Pittsfield. Five years later there was another revival and Haynes received one hundred and nine. Naturally he was in demand by other churches as a revival preacher.