In 1765 German settlers near Lovettsville, Virginia established the New Jerusalem Lutheran Congregation. A church building was soon constructed, and before 1770 burials began in the adjacent cemetery. The earliest legible grave marker is of "Isack Vckens" [sic], although there are several graves that are unmarked or with unreadable inscriptions that may predate Vickens burial. Pastors of the church began recording burials in 1785.

In 1949, the cemetery was overgrown and in disrepair. Aurelia M. Jewell (1900 -1992) inventoried headstones that were visible. In 1959, Pastor Michael Kretsinger (1929 -1993) organized a cleanup of the cemetery, and at that time, several additional stones were located and added to Jewell's list. In 1995 Marty Hiatt and Craig Roberts Scott compiled a new list of burials in the cemetery between 1770 and 1943, publishing it as New Jerusalem Lutheran Church Cemetery: Lovettsville, Virginia, 1770-1943. Proceeds from the sale of this book were donated to the New Jerusalem Lutheran Church Cemetery Restoration Fund, a project initiated to stabilize the 700 identifiable stones and gravesites Hiatt and Scotts were able to locate during their survey. An additional 400 burials are known to exist in the cemetery, however, their locations are unknown.