The earliest act of this motley crew, when they tired of killing the cattle which the Spaniards had been at great pains to breed on the sea-washed savannahs on the south side, was to erect a fort at Cagway under Sedgwick in March 1656. At the Restoration it not unnaturally received the name of Fort Charles, and the collection of houses that grew up around it was called Port Royal, at the dictates of loyalty or sycophancy, according to the political creed of the members of the new colony. But the governor and council had often great difficulty in persuading the Assembly to vote the funds necessary for its upkeep and improvement. A writer during Sir Charles Lyttelton’s governorship, 1662–1664, says it was called Port Royal from the excellency of the harbour; but it was apparently not till February 1674 that that became its official name, when the Assembly voted “Point Conway (an obvious misprint for Cagway) to be called Port Royal.” In addition to Fort Charles there were three other principal forts called James, Carlisle and Rupert. For a time Port Royal was the residence of the Lieutenant-Governor, while the Governor’s official residence was at Spanish Town. In November 1661, it was ordered “that no person remain on Point Cagua without giving security to a Justice of the Peace not to be chargeable to the inhabitants for more than one month. Any waterman bringing a person likely to be chargeable to pay a piece of eight and carry him back again.” In 1664 the Assembly desired and advised the Council that the Court of Common Pleas should be held constantly “in St. Jago de la Vego and no more at Port Royal,” and the Council agreed. In March 1674–5 Peter Beckford wrote home: Lord Vaughan arrived on the 13th inst. at night and landed at Port Royal; next day his commission was read, and he was entertained as well as the island could afford; 15th, he remained on Port Royal, viewing the fortifications; came next day to St. Jago, being received at the seaside [at Passage Fort, probably] by 150 horse and a company of foot, besides the gentlemen of the country and seven coaches, all which attended him to the town, where he was received with two companies of foot, and dined with Sir Thos. Modyford. By 1675, residence by the Governor at Port Royal had gone out of favour. In that year was passed a resolution to the effect that—the “Captain-General’s salary to be £2000 per annum, residing usually at St. Jago, his residence at Port Royal to be omitted.” In 1680, the custos of Port Royal was Sir Henry Morgan, J.P. The judges of court of common pleas were William Beeston, Reginald Wilson and Anthony Swimmer. The justices of the peace were, besides the judges, John White, Theodore Cary, Prichard Herne and Harbottle Wingfield.