Beginning his career in Britain in 1947, he played bit parts in various films, with his first role coming in Whom the Gods Love, Die Young. In 1958, he appeared in A Night to Remember, an adaptation of the RMS Titantic disaster. In 1963, he played Lt. Commander Eric Ashley-Pitt in John Sturges's critically acclaimed World War II film The Great Escape. Also in 1963 he appeared in an episode of the original SciFi series "The Outer Limits" titled "The Sixth Finger".
After The Great Escape, McCallum was cast as Russian secret agent Illya Kuryakin on NBC's TV series The Man from U.N.C.L.E.. The series ran for three and a half seasons, and his role earned him two Emmy Award nominations and a Golden Globe nomination.
Although he never earned much popular success after U.N.C.L.E., he did have a steady stream of work. He appeared on British series such as Colditz, Sapphire & Steel, and a short-lived U.S. remake of The Invisible Man. In 1983, he played Illya Kuryakin again in the TV movie The Return of the Man from U.N.C.L.E.: The Fifteen Years Later Affair.
In 2003, McCallum appeared in two episodes of "JAG", playing Donald Mallard, NCIS's medical examiner. He continued the role as a regular on NCIS, and Legend, Part Two (episode), the backdoor pilot to NCIS's spin-off series, NCIS: Los Angeles has been the only episode McCallum hasn't appeared in so far.
In the 1960s, he recorded four albums for Capitol Records. Opting not to sing, he instead arranged instrumental interpretations of hits of the day.
McCallum is the son of acclaimed violinist David McCallum, Sr., and his brother Iain is a noted author.
From 1957 to 1967, McCallum was married to actress Jill Ireland, who made a few guest appearances on The Man from U.N.C.L.E. They had three sons: Paul, Jason, and Val. Jason died in 1989 of an accidental drug overdose. McCallum and Ireland divorced after McCallum introduced Ireland to Charles Bronson on the set of The Great Escape.
Since 1967, McCallum has been married to Katherine Carpenter. They have a son and a daughter, and live in New York. While filming NCIS, McCallum also lives part-time in California.