In 1847, a group of Alexandria business owners William Fowle, Anthony Cazenove, Hugh Smith, Henry Daingerfield, William Gregory, John Withers, Pobert Jamieson, John C. Vowell, William Stabler, and Robert Miller proposed the founding of a cotton manufacturing industry for Alexandria. The enterprise began with the building of a four-story manufacturing plant, The Mt. Vernon Cotton Factory, on the corner of Oronoco and Washington, at what is now 515 N. Washington St. The business did not prosper, and was offered for sale in 1855 and was bought by Lewis McKenzie, John Withers, Robert Jamieson, James Green, Henry Daingerfield, and William Gregory. The business was again unsuccessful and was bought by John Rosencrantz of Philadelphia in 1858. The building was seized by the U.S. Federal government during the Civil War, 1963, and was used as a prison and later a barracks. After the war, Abijah Thomas tried to restart the plant, and entered into a deed of trust with C. Turnbull Baxter and Company, and borrowed money from the First National Bank of Alexandria to finance the restoration. Thomas failed, and the resulting legal issues surrounding the building reached the Supreme Court in 1875. In 1877, Robert H. Garrett of Baltimore, bought the company and shut it down so as to prevent competition with his flour company. The building has since exchanged many hands, serving as a bottling plant for the Robert Portner brewing company, a factory for the Express Spark Plug Factory, and an apartment complex.