In 1784, a treaty with the Mississauga Indians ceded a large tract of land along the north shore of Lake Ontario to the British Crown. A number of villages sprang up along the shoreline, one of which was to become the village of Grafton. Originally called Haldimand Corners, the village was located at the junction of Danforth Road, (1800), the road leading southward towards Grafton Harbour, and the surveyed baseline road from Trenton (1790).

One of the earliest inns in the area was Glover's Inn. This property was later sold to Edward Pepper, and renamed Pepper's Tavern. It was at a meeting of citizens at the Inn in the early 1800's that the decision was made to rename the village Grafton.