Literally: leisure – talk – rest – topic
Alternately: “But setting that aside….” A phrase used to put an end to a digression, aside, or idle chatter, and return the listener’s attention to the main topic or thread of conversation, or the main plotline in storytelling. A more casual way to express the same thought would be それはさておき (sore wa sate oki).
Notes: This phrase comes from a passage in the famous 14th century Chinese historical novel Water Margin (Japanese 水滸伝, Suiko den).
閑 may be replaced with homophone 間, “interval,” without any change in meaning – note that both 閑話 and 間話 can refer to “idle talk,” although the latter compound seems to be less commonly used in current Japanese.