Our story begins in the town of Titipu in Japan where, having introduced themselves, Japanese nobles meet a wandering minstrel named Nanki-Poo. He tells them of his love for Yum-Yum, whom he first saw as he was playing in the Titipu town band. Yum-Yum was engaged to Ko-Ko, a cut price tailor, but he has been sentenced to death for flirting, so Nanki-Poo returns to find his beloved Yum-Yum and marry her.

The death sentence for flirting is the idea of the bloodthirsty ruler of Japan, the Mikado. The gentlemen of Titipu decide to get round this law by creating Ko-Ko Lord High Executioner on the grounds that he is then unlikely to execute himself. All the senior Titipu civil servants resign in protest at these shenanigans, only to find that the ambitious Pooh-Bah snaps up all their jobs, salaries, and perks to become Lord High Everything Else.

Ko-Ko is set to marry Yum-Yum, who arrives with her girl friends, Pitti Sing, Peep-bo, and Yo-Ko— three little maids from school. When Yum-Yum sees the handsome Nanki-Poo, she is overjoyed since she does not love Ko-Ko, but duty makes her reluctantly realise that her wedding with the recently ennobled cheap tailor must take place.

Then a crisis hits the town. The Mikado is coming to visit Titipu to check that his savage law has been carried out. Since Nanki-Poo has decided to kill himself because he cannot marry Yum-Yum, Ko-Ko convinces him to become the necessary victim the Mikado wants to have executed. Nanki-Poo agrees on the condition: that he weds Yum-Yum and they enjoy a month's married bliss before he gets the chop— after which, Ko-Ko can marry her.

This seems to fit the bill, and everyone rejoices until another crisis descends in the frightening person of Katisha, an aristocratic older lady who was engaged to Nanki-Poo before he fled the court, unable to face marrying her. She tries to reveal Nanki-Poo's identity as the son of the Mikado and heir to the Japanese throne, but the local people refuse to listen to her, and she storms off vowing revenge.

On Yum-Yum's wedding day, Ko-Ko brings news that due to another cruel law, the wife of a beheaded man must be buried alive, a fate Yum-Yum does not relish on account of its stuffiness. Pooh-Bah and Ko-Ko decide that the only solution is to do a snow job on the Mikado. They will let Yum-Yum and Nanki-Poo get married, pack them off abroad on a honeymoon, then draw up false documents and lie to the Mikado about the execution.

Accompanied by Katisha, the Mikado arrives and is informed by Ko-Ko, Pooh-Bah, and Pitti-Sing of the alleged execution of Nanki-Poo. The deceit boomerangs, however, since yet another crazy Japanese law insists that no matter how ignorant of their actions, everyone remotely associated with the death of the heir apparent must be boiled alive. Since the bloodthirsty and husband-hungry Katisha is keen to see the plotters killed, the only solution seems to be for someone to marry Katisha as quickly as possible, then get her to beg the Mikado for mercy. Ko-Ko draws the short straw and marries the dragon lady.

When Nanki-Poo and his wife appear, there is a lot of complicated explaining to be done. Fortunately, by a justification even crazier than the fictitious laws of the stage Japan, Ko-Ko manages to placate the Mikado, and with laughing song and merry dance, everyone lives happily ever after.