James Waters Allison was born in Richmond, Virginia on December 15, 1833 to William Allison (1783-1850) and Nancy Ann Waters Allison (1799-1860). Allison, a life-long resident of Richmond, began in business with the firm of Ludlam & Watson, forwarding merchants and agents for the Powhatan Steamship Company. He later worked with the Old Dominion Steamship Company. After his father's death in 1850, Allison began a tobacco manufacturing company with his partner William Palmer. The business closed at the outbreak of the Civil War. In 1858 Allison married his first wife Mary Robbins Zimmerman of Alexandria, Virginia. They had a daughter, Dora, born in 1859.

After the war ended in 1865, Allison and his partner Edmund B. Addison established the seed and fertilizer firm of Allison and Addison. By the 1890s the company had greatly expanded and become one of the most successful fertilizer producers in the South with branches in Charleston, South Carolina, Columbus and Savannah, Georgia, as well as other southern cities. After the death of his first wife, Allison married Bettie L. Thomas (1848-1876) of Richmond, but she died not long after their marriage. Minnie Clemens Jones (1870-1927) became his third wife in December 1890 and Allison began searching for a new home in 1891.

At some point Allison decided to build rather than buy and acquired a lot on West Franklin Street, Richmond's most fashionable residential neighborhood during that period. The house was designed by New York architectural firm of Griffin & Randall and built between 1894 and 1896. Both partners boasted fine architectural pedigrees. Percy Griffin (1866-1921) graduated from the architectural school at the Massachusetts Institute of Technology (MIT) in 1884 and then worked in the office of H. H. Richardson. T. Henry Randall (1862-1905), a native of Annapolis, Maryland had also worked for Richardson after attending Johns Hopkins University, MIT, and the Ecole de Beaux-Arts in Paris. Randall, chose the Colonial Revival design of the house, making it one of the first of that style built in Richmond. It pre-dates other early Colonial Revival houses on Monument Avenue by some 10 years. In 1894, while engaged on this project Griffin and Randall decided to dissolve their partnership. Allison was forced to choose which architect he wanted to oversee the completion of his house. Randall, who was the senior partner and had the most influence over the building's design, was selected.

Allison had little time to enjoy his new home as he died suddenly at the age of 65 in 1898. The funeral services were conducted at the home on West Franklin and he was buried in Hollywood Cemetery. At the time of his death Allison left an estate of nearly one million dollars. Allison had been active in other endeavors beyond his firm. He was on the board of directors of the Nation Bank of Virginia and President of the Old Dominion Fruit Growing Company. He as was a member of the James River Improvement Committee and was a founding member of the Broad Street Methodist Episcopal Church, South.

Allison's son James Waters Allison, Jr. (1894-1979) eventually inherited the house on West Franklin Street. In 1938 the family sold the home to the Richmond Professional Institute (RPI), which was used as the office and residence of Henry Hibbs, dean of RPI. The house continued to be used as the office of the president when RPI and the Medical College of Virginia merged to form VCU in 1968.