Foster was born in Massachusetts in 1752. He engaged in classical studies at the Rhode Island College (now known as Brown University), graduating in 1770. He then studied law and was admitted to the bar association. He remained in Rhode Island to practice law. Until 1790 Foster held various positions in the government of Rhode Island. He was then appointed to the United States Senate, beginning his service on 7 June 1790. Rhode Island's state legislature reappointed him in 1791 and 1797, and he served until 3 March 1803 when he retired from public life to engage in writing and historical research. During this period, he also served as a trustee of Brown University. Foster returned to public life to serve in the Rhode Island state legislature from 1812 to 1816. He died in 1828.