James Olmsted (April 17, 1645 – April 28, 1731) was an early settler of Norwalk, Connecticut. He was a deputy of the General Assembly of the Colony of Connecticut from Norwalk in the sessions of October 1691, October 1692, October 1693, and May 1699. He was the son of founding settler of Norwalk, Richard Olmsted and Elizabeth Haugh Olmsted. He served as town clerk of Norwalk for 29 years, from 1678 to 1707 and again in 1721. He was town judge. In 1671, and from 1682 to 1685, he was chosen as a selectman.