John McDuffie was taught by private tutors and attended Southern University in Greensboro. He graduated in 1904 at the Alabama Polytechnic Institute in Auburn, and in 1908 at the Law School of the University of Alabama in Tuscaloosa. McDuffie also pursued a political career: Between 1907 and 1911 he sat in the House of Representatives from Alabama. He was admitted in 1908 and then practiced in Monroeville. He then worked 1911-1919 as a prosecutor in the first judicial district of Alabama.
McDuffie was elected to the 66th U.S. Congress and reelected to the eight succeeding Congresses. He served in the U.S. House of Representatives from March 4, 1919 until his resignation on March 2, 1935. During this time he held the post of Minority Whip ( 71 Congress ) and Majority Whip ( 72 Congress ). He also chaired the Committee on Insular Affairs ( 73rd and 74th Congress ).
He took 1935 appointment as a judge at the Federal District Court for the Southern District of Alabama and held this post until his death. John McDuffie was buried in the Pine Crest Cemetery.