And now misfortunes fell thick and fast upon the patriots. They still held Fort Washington on Manhattan Island, and Fort Lee on the opposite side of the Hudson, the garrisons of which were under the command of General Greene. Washington now advised him to abandon the forts, but did not give him absolute orders to do so. It is probably that he would have taken his commander's advice had not Congress interfered and sent orders that Fort Washington was not to be given up, except as a last necessity. Greene, believing that it was possible to hold it, tried to obey Congress. But on the 16th of November, after a fierce fight against tremendous odds, the fort was surrounded, and all the defenders to the number of about three thousand were taken prisoner. The loss was a bitter blow to Washington, for the men taken prisoners were some of his best soldiers. Four days later Fort Lee was also taken, and although the garrison escaped they left behind them large stores of food, ammunition, baggage of all sorts, as well as cannon, which they could ill spare. Washington now resolved on a retreat towards Philadelphia, and gloom settled on the ragged little army of patriots. They were weary of retreats and defeats, and felt that their cause was already lost. Winter was fast coming on and many shouldered their arms and marched homeward. And so the once buoyant enthusiastic army melted away to a hungry and dispirited troop of little more than four thousand. General Lee had at this time but lately returned from his triumphs in South Carolina, and he was more boastful and arrogant than ever. After Washington he was second in command, but he had no doubt in his own mind that he ought to be first. Now he was not slow to let others know what he thought. And while Washington, noble and upright gentleman as he was, trusted Lee as a friend, and believed in him as a soldier, Lee schemed to supplant him. Washington had left Lee at North Castle with seven thousand men. Now he sent him orders to join him at once, so that if he should have to fight a battle he could have at least some sort of army to fight with. But Lee pretended to misunderstand. He made excuses for delay, he argued, and lied, and stayed where he was. Perhaps he thought that it would be no bad thing if Washington should be defeated and captured. Then he would be commander-in-chief. But it was Lee who was captured, not Washington. He had in a leisurely fashion at last begun to move, and on the march he spent a night at a wayside inn. The British, hearing of his whereabouts, surrounded the inn and took him prisoner. For more than a year he remained in their hands, a very comfortable captive, and his army, under General John Sullivan, marched to join Washington, who was still retreating southward through New Jersey before the overwhelming force of the British.