Originally located at 855 West Main Road, now the site of fast food restaurants and a hotel, the Sweet-Anthony house was moved to Prescott Farm in 1970. This broad-gable roofed 1½ story farmhouse came with much original woodwork intact. It is a good example of simple rural architecture, complete with additions which were made in the 18th and 19th centuries. It is also valuable as an example of a middle-class farm dwelling, a nice counterpoint to the extravagance of the Nichols-Overing House.
Not much is known about the house prior to its purchase by NRF. The house dates to c. 1730 although the first definitive record of the building is not until 1786. The Anthony family is recorded as having lived in the house from 1786 until 1918 when Joseph S. Anthony left the property to his son-in-law Thomas J. Sweet. It is this transition in ownership that inspired NRF to refer to the house as the Sweet-Anthony House.

The property has been used for a number of different functions by NRF. Initially it was used as living quarters for a farm caretaker and later was used to house NRF staff, interns, an artist-in-residence and is now part of NRF's unique tenant steward program and rented as a private property.