The health secretary, who supports Aberdeen, says he should have gone to a "wider range" of matches.