First appearance: Crack Comics #27 (January 1943) from Quality Comics:


For his first appearances in a DC comic book, a story from Crack Comics #42 (May 1946) was reprinted in 100-Page Super Spectacular #DC-18 (July 1973).
Also, if you looked closely, you'd see that Captain Triumph was included on the cover to All-Star Squadron #1 (September 1981):


From his Who's Who entry in Who's Who: The Definitive Directory of the DC Universe Vol. XXVI (April 1987),