John Allen was the only son of a joiner (furniture maker) from Great Barrington, Massachusetts. His only other sibling, a sister, married Elizur Goodrich of New Haven, Connecticut. John Allen only had a common education before attending the Litchfield Law School, but still worked as a teacher for several years. As a minor, he taught younger children in a noted establishment in Germantown, Pennsylvania. Allen then taught at New Milford, Connecticut for six months. He was admitted to the bar after attending the Litchfield Law School from 1784 until 1786. From 1793 to 1796, Allen represented Litchfield in the Connecticut General Assembly as a State Representative. In 1797, he was elected to the fifth U.S. Congress at the age of thirty-five. During his time in the House, he was an avid proponent of the Alien and Sedition Acts.

After serving his first term however, Allen declined any further election to Congress. In 1800, he was elected Assistant Judge to the Connecticut Supreme Court of Errors and was subsequently re-elected for the next five years. His wife, Ursula McCurdy, was a graduate of the Litchfield Female Academy and the daughter of Lyne and Ursula McCurdy of Norwich, Connecticut. During their marriage, John and Ursula had one son and one daughter. Their son, John Allen, became a U.S. Congressman from Cleveland, Ohio and married Litchfield Female Academy student Anna M. Perkins.