William & Mary received its first gift of art in 1732. Throughout the centuries, numerous gifts of art were received and dispersed throughout the campus. The College art collection held American and English Colonial painters as well as modern works such as White Flower by Georgia O'Keeffe, given by Mrs. John D. Rockefeller, Jr. in 1934. Early advocates for preservation and cataloguing the College's art include Dr. Earl Gregg Swem and Thomas Thorne. During the 1970s, Dr. Miles Chappell along with art history students conducted a survey to determine what artworks The College owned. The survey revealed that over 300 years, William & Mary amassed a sizeable collection of art and established the need for a museum to preserve and protect the artwork.

The Muscarelle Museum of Art was made possible by generous funds from alumni and friends. The Museum opened in 1983 with Dr. Glenn Lowry (director at the Museum of Modern Art, MoMA, in New York), serving as the first Director, who oversaw the building construction. The major benefactor was Joseph L. Muscarelle (W&M '27) and his wife, Margaret who generously supported the formation of a museum and whose family has continued their support throughout the years.

Subsequent gifts extended the collection beyond its roots in American portraiture; the collection spans centuries, including works by Hans Hofmann, Pablo Picasso, Henri Matisse, and John Singleton Copley. Integrated into the design of the building was the "world's first solar painting," designed by Gene Davis, the noted Washington Color School painter. This design transforms the south façade of the Museum into a work of art when multi-colored tubes are illuminated from behind.

In 1987, the Museum underwent an expansion to nearly double its original size. At this time, Mark Johnson succeeded Glenn Lowry and oversaw the expansion. Mr. Johnson is now the Director at the Montgomery Museum of Fine Arts in Alabama. The third Director was Dr. Bonnie G. Kelm, who served from 1996 – 2002, and who recently retired from the University of California-Santa Barbara Art Museum.

The Muscarelle Museum of Art was accredited by the American Association of Museums (AAM) (now American Alliance of Museums) in 1988 and received subsequent accreditations in 2000 and in 2012. The Museum was the first university or college museum of art in the Commonwealth of Virginia to be accredited by the AAM. This distinction is held by fewer than five percent of museums in the U.S.

The collection has continued to grow and now numbers over 5,000 works of art from many cultures and historical eras. The strength of the collection is the holdings in English and American portraits of the seventeenth and eighteenth centuries that have national importance. Also included are a survey collection of European and American prints and drawings from the sixteenth through the twentieth centuries, Japanese prints, African art, Asian ceramics, and a remarkable collection of German Expressionist works on paper.

The Museum collection is supplemented and enhanced by numerous special exhibitions that bring works of art from public and private collections worldwide. These exhibitions provide opportunities for the viewing and study of material not otherwise available in this area. The Museum collaborates on special thematic exhibitions with academic departments at The College as well as with other cultural institutions and organizations. Numerous educational opportunities are offered throughout the year in conjunction with the Museum collection and loan exhibitions, including lectures, gallery talks, demonstrations, seminars, and symposia.