From 1903 to 1907 he studied art history in Strasbourg, Bonn and Vienna. He received his doctorate in 1906 under the early Vienna school art historians Franz Wickhoff and Max Dvořák in Vienna. From 1906 to 1909 he was assistant to Franz Wickhoff at the University of Vienna. From 1909 to 1914 he worked for the project Monuments of German Art of the German Association for Art History and began collecting material on Carolingian book illumination.

In 1918 he became director of the newly founded State Art Collections in Weimar, where he made contact with the Bauhaus School. From 1920 he also taught art history at the University of Jena (1920 private lecturer, 1924 associate professor). In 1920 he married the Bauhaus student Margarete Bittkow. In 1932 Koehler first went to Harvard University as a visiting professor, then finally emigrated in 1934 and was appointed Professor of Medieval Art History as the successor to Arthur Kingsley Porter. He taught at Harvard until his retirement in 1953. From 1941 to 1944 he was the first senior research fellow at Dumbarton Oaks. From 1946, he was a corresponding member of the British Academy.

The focus of his academic work was research on Carolingian book illumination, which he presented in the corpus volumes Die Carolingischen Miniatures. He began working on it immediately after completing his dissertation, the first volume being published in 1930 and the second in 1955.