As the bombardment continued during the evening of September 13th, the night became thickly dark as a line of thunderstorms passed over the region, bring unseasonably cold temperatures. Soldiers, sailors, and marines on both sides huddled in the open under pouring rain.
Colonel Brooke received a communication from Admiral Cochrane that stated "in consequence of the entrance of the harbour being closed up by vessels sunk for that purpose by the enemy, a naval co-operation against the town and camp was found impracticable." After deliberating with his officers, Brooke decided that "the capture of the town would not have been a sufficient equivalent to the loss which might probably be sustained in storming the heights." He informed the admiral that he had ordered a "retreat to take place” the following morning."
September 14: Admiral Cochrane received word of Colonel Brookes decision not to attack, too late to abort a diversionary raid up the Middle Branch of the Patapsco River. Shortly after 12:30 a.m. on the Wednesday morning, September 14th, Lieutenant Napier's force of Royal Marines and several hundred seaman in 20 vessels, accompanied by the rocket ship HMS Erebus, made their way, as silently as possible, up the river.
Sailing-Master John Webster at Battery Babcock, and Lieutenant Newcomb at Fort Covington, both claimed the honor of having first spotted the British force. Both positions opened fire, and were soon joined by the guns at Fort Lookout Hill commanded by Lieutenant George Budd. The British returned fire and the witnesses were "presented the whole awful spectacle of shot and shells, and rockets, shooting and bursting through the air... as the darkness increased the awful grandeur of the scene..." At Fort McHenry, Major Armistead reported: "we once more had an opportunity of opening our batteries, and kept up a continued blaze for nearly two hours, which had the effect again to drive them off."
"Fort Covington, the Lazaretto, and the American barges in the river now simultaneously poured a galling fire upon the unprotected enemy... as they endeavored to regain their ships, which came closer to the fortifications in an endeavor to protect the retreat. A fierce battle ensued. Fort McHenry opened the full force of all her batteries upon them as they re-passed... the waters of the harbor, lashed into an angry sea by the vibrations… as though in a tempest. It is recorded that the houses in the city of Baltimore, two miles distant, were shaken to their foundations."
In the darkness and rain, several British boats approached the Lazaretto. Lieutenant Frazier, commanding the battery there, sent word to Commodore Rodgers that enemy boats were approaching his position. Rodgers dispatched his aide, 19 year old Lieutenant Robert Field Stockton, to accompany Major Beal Randall's Light Corps, which had been placed under Rodger's command, "to dislodge a party of men in the enemy's boats, which it was supposed intended landing near the Lazaretto to take possession of our little three gun battery." The British withdrew to their ships. Upon his return, Mr. Stockton reported to Commodore Rodgers "in very high terms of zeal and gallantry displayed by the major and his corps on the occasion."
There was a brief lull in the firing around 4:00 a.m., then the British ships continued their bombardment until 7:00 a.m., when they ceased altogether. About 9:00 a.m. the ships began to get underway and move down the river.