NLP and Machine Translation for Kpop Concerts
For the benefit of all international fans
2 weeks ago, I finally caught Mamamoo live!! After watching countless youtube videos of their performances, following all their social media accounts, even finding comprehensive lists of their appearances on variety shows so that I could ̶s̶t̶a̶l̶k support them, I finally got the chance to see them in person and absorb all their positive vibes in real time!
There was just 1 problem though, I couldn’t understand what they were saying in Korean.
In the earlier years when NLP and machine translation technologies did not exist, many Youtube videos were only available in Korean. Only a few had translations but they were largely dependent on fans’ effort and usually delayed by a few months. If there were any captions or text in social media posts, they were almost a 100% in Korean. As a result, international fans had access to very limited content.
However, as the technologies progressed in the recent years and were adopted onto the various platforms, we are now able to understand and connect at a much faster rate without having to learn an entire language. In a single tap, we can translate the captions in Instagram from Korean to English. In a click, we can turn on captions on Youtube, and in 2 clicks, select the language we want the captions to be in.
I was so pampered by these technologies that despite having a translator translating during the concert, as a very excited fan, I wished there was something more instant and seamless.
One problem with the arrangement of a translator was that the Mamamoo members had to speak and wait for the translator to translate. This broke the rhythm and tempo of natural conversations and produced a delay in the reaction and response of the fans, and subsequently another delay in the reaction and response of the Mamamoo members. As a result, the entire interaction was stiff.
Another problem with this arrangement was when the Mamamoo members went back to their usual self, speaking and commenting without skipping a beat, the translator was not able to find a good time to interject and translate. When she managed to at times, it was clear there was a delay because the body language of the members suggested that they were already on another conversation point. It was like watching a video with a 3 seconds audio lag.
Now, imagine NLP and machine translation technologies were incorporated into the concert experience.
When the Mamamoo members start greeting the fans, we’d see the English translations live, appearing on the huge LED screens on the centre and side stages. To make the translations more accessible, audio channels could be set up for fans to tune into during songs breaks, so that we could listen to the translations in our preferred language through our ear pieces (a little like those museum audio guides, but live, in real time).
Depending on the technologies, it would be possible to train it specifically for the Korean language. It might even be possible to achieve more accurate translations by fine tuning it to the kpop artists when they rehearse or do sound checks.
Forget hologram, metaverse and AR/VR for now. Make NLP and machine translations work so that all the Moomoos can understand what Moon Byul, Hwee In and Hwasa are colluding against Solar 🤭
