1 June 1761 he was appointed to succeed the Rev. Joseph Brett in the mastership of Scarning school. When he went to take possession of the premises the inhabitants barred his entrance by force, as they desired the appointment of a master called Coe, who had been working the school for some time, and Potter was unable to enter until Sir Armine Wodehouse, a magistrate, had read the riot act. He kept, like Brett, a good boarding-school, and had many pupils, whom he educated himself, while he taught the village children by deputy. With this position he combined the duties of curate of Scarning, and here he remained for twenty-eight years until 1789, occupying his spare hours with translating the works of the Greek tragedians. These he regularly sent, as they passed through the press, to Lord Thurlow, then lord chancellor, who had been educated at Scarning school. On the receipt in 1788 of a copy of the translation of Sophocles, a letter was sent by the lord chancellor to Potter intimating his pleasure at receiving these versions, and offering him the second canonical stall in Norwich Cathedral, which he held until his death. According to the anecdote given by Lord Campbell (Lives of the Lord Chancellors, v. 642), Thurlow, in giving the stall, observed, ‘I did not like to promote him earlier for fear of making him indolent.’ In the next year (26 June 1789) he was appointed by the bishop of Norwich, without any application on his part, to the important vicarage of Lowestoft, with the rectory of Kessingland, and the house occupied by his predecessor was at the same time acquired as a parsonage and vested in Potter and his successors (Gillingwater, Hist. of Lowestoft, pp. 313, 354). He thereupon resigned his charge at Scarning, and devoted himself to his new duties. He was found dead in his bed at Lowestoft on 9 Aug. 1804 (Pratt, Harvest Home, p. 503). A mural monument to his memory was erected by the parishioners in Lowestoft churchyard. Romney painted his picture in 1779 as a gift to him, and also painted his son's portrait (John Romney, Life of Romney, pp. 159–61, 220–2, where are several letters from Potter to Romney). His wife was Elizabeth, daughter of J. Colman of Hardingham, by Elizabeth, daughter of John Howes of Morningthorpe. She was buried at Scarning on 6 July 1786. Potter was described as ‘a tall man, about six feet high, very handsome, with an aquiline nose,’ and as ‘of great merit, small preferment, and large family’ (Forbes, Life of Beattie, ii. 220–1). His daughter Elizabeth was buried at Scarning on 12 June 1782.
Potter's chief work was his translation of the tragedies of Æschylus. The first edition appeared in 1777, and in the following year he printed and presented to the subscribers his ‘Notes on the Tragedies of Æschylus,’ which were drawn up at the request of Mrs. Montagu and addressed to her. His correspondence with Dr. Parr on these ‘Notes’ is in Parr's ‘Works,’ viii. 225–30. Subsequent editions of the translation came out in 1779, 1808, 1809, 1819, and 1833; it formed in 1886 vol. xli. of Morley's ‘Universal Library,’ and it was issued in 1892 as No. 30 of Sir John Lubbock's ‘Hundred Books.’ Beattie called it ‘the best translation that ever appeared in English of any Greek poet,’ and Sir James Mackintosh read it ‘with very great admiration.’
The first volume of Potter's translation of the tragedies of Euripides came out in 1781, with a dedication to the Duchess-dowager of Beaufort, and the second in 1783. The assignment by him to James Dodsley of the copyright is in the Egerton MS. Brit. Mus. 2334, f. 19. It was reprinted in 1808, 1814, and 1832, and some of his versions of the plays were also published separately. In 1887 there appeared, as vol. liv. of Morley's ‘Universal Library,’ Potter's rendering of ‘Alcestis and other Plays by Euripides.’ His translation of the tragedies of Sophocles was given to the world in 1788, with a dedication to Georgiana, countess-dowager Spencer, and a new edition was published at Oxford in 1808. The verdict of Parr was that Potter lost the fame established by his Æschylus by his translation of Euripides. Dr. Johnson characterised all Potter's efforts as ‘verbiage.’
Potter's other productions in poetry were: 1. ‘Retirement: an Epistle,’ 1748. 2. ‘A Farewell Hymne to the Country in the manner of Spenser's Epithalamion,’ 1749; 2nd ed. 1750; it is also inserted in Bell's ‘Collection of Fugitive Poetry,’ xi. 105. 3. ‘Holkham: a Poem,’ to the Earl of Leicester, 1757; also included in Pearch's ‘Collection of Poems,’ ii. 259–67. 4. ‘Kymber: a Monody to Sir Armine Wodehouse,’ 1759; a poem in praise of that family, also in Pearch's ‘Collection,’ iii. 184–99. 5. ‘Poems by Mr. Potter,’ 1774 (containing the poems to that date). 6. ‘The Oracle concerning Babylon’ and ‘The Song of Exultation’ [two odes] from Isaiah, chap. xiii. and xiv., 1785. Some verses by Dr. Johnson in derision of Potter's attempts at poetry were read at Mrs. Thrale's house at Streatham in July 1779 (Early Diary of Frances Burney, ii. 256–8). An account of Johnson's rough treatment of him when introduced by Mrs. Montagu is given in E. H. Barker's ‘Anecdotes,’ i. 1–2. The victim did not suffer in silence. He published in 1783