oh, my stars and garters

Attested (in the figurative sense) since before 1763 (see James Otis' 1763 pamphlet, The Rights of the British Colonies Asserted and Proved: "These representations of us, many of the good people of England swallow with as much ease, as they would a bottle-bubble, or any other story of a cock and a bull; and the worst of it is, among some of the most credulous, have been found Stars and Garters."). For the stars, see oh my stars. The garters refer to a heraldic order (in a stellar shape) that British monarchs could bestow. Together they were used as a name for the trappings of high office or the individuals who occupied such.

来源：wiktionary