Abijah Gilbert first attended the Gilbertsville Academy, a private school in his hometown before he enrolled at Hamilton College in Clinton. There he made his degree in 1822. By 1850, he then worked as a merchant in New York City; in 1865 he finally moved to St. Augustine in northeast Florida.
As a supporter of abolitionism Gilbert supported in recent years, the Whigs; later, he joined then in the Republican Party. For the Republicans, he was elected during the period of Reconstruction in the Senate in Washington. He remained there from March 4, 1869 to March 3, 1875, and until 1989 was the last Republican who held the Class 1 Senate seat.
After his retirement from the Senate Abijah Gilbert also withdrew from the labor market. He died in 1881 in his birthplace of Gilbertsville.