Continuing our series of evaluating Meta's new SeamlessM4T multimodal translation model, let's expand our social media experiments with a full-length news article. While the factual elements of the story are correctly translated and the overall gist of the article preserved, transliteration is very different from that of Google Translate, Bing Translator and GPT-3.5. For some names, transliteration varies across sentences, with the same name being transliterated differently. Names are often translated literally into English, rather than preserved as the actual listed toponym. Overall, the SeamlessM4T provides quite strong translation, but the transliteration issue is highly problematic.
Let's take a look at this full-length news article:
在跟母亲断联12个小时后，8月2日中午，身在北京的张萌终于收到了位于涿州的母亲的“报平安”短信。母亲住在河北保定涿州市三步桥村附近，是借用别人手机发来的消息，她长舒了一口气，“悬着的心终于放下了”。 张萌母亲所在地附近的救援冲锋舟。受访者供图 7月29日以来，受台风“杜苏芮”影响，京津冀地区持续强降雨。据公开信息，7月29日8时至8月1日11时，涿州市出现明显降水天气过程。全市平均降水量355.1毫米，最大降水量为两河村435.7毫米，多个乡镇、街道降水量均超300毫米。截至8月1日上午10时，涿州市受灾人数133913人。 24岁的李志辉已经多次安慰女朋友，但女朋友的一句“家没了，那个房子是我爸妈一辈子的心血”，也瞬间让他陷入无助。目前，涿州境内北拒马河、小清河、白沟河等多条河流流量较大，小清河分洪区、兰沟洼蓄滞洪区已相继启动。涿州境内防汛形势严峻，多地遭受洪水灾害，多个村庄被洪水围困。 目前，李志辉女朋友的家人所在的涿州市刁窝镇东辛庄村仍在等待救援，包括附近的白塔村、小营村等同样或在等待救援或正在被救援。在社交平台上，也依然能看到大量求助救援的信息正在发出。告急的村庄。 “一共五位家人，其中一位老人已经80多岁，一个孩子只有五六岁，目前都在白塔村一处自建二层小楼上，水已经漫到腰部，有一米多深了，家人都在二楼等候救援。”肖俊介绍，因为到处被淹，也为了陪伴老人，他的家人们8月1日就搬去了刁窝镇白塔村，“结果今天水就涨起来了。”36岁的肖俊平时在北京工作，赶不回去的他觉得现在非常揪心，断断续续的信号也无法获知现场的情况，同时他也认为村子救援难度很大，人员分散也比较难找。“刁窝镇东辛庄村的水已经漫过了一楼。”李志辉告诉新黄河记者，他目前在沧州，女朋友是刁窝镇东辛庄村人，8月1日几位家人有的已经搬到附近的白塔村住，有的还在东辛庄村，跟张俊一样，李志辉女朋友一家也没想到，当地的水涨得如此快。
It was broken into sentences for translation:
time python scripts/m4t/predict/predict.py "在跟母亲断联12个小时后，8月2日中午，身在北京的张萌终于收到了位于涿州的母亲的“报平安”短信。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "母亲住在河北保定涿州市三步桥村附近，是借用别人手机发来的消息，她长舒了一口气，“悬着的心终于放下了”。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "张萌母亲所在地附近的救援冲锋舟。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "受访者供图 7月29日以来，受台风“杜苏芮”影响，京津冀地区持续强降雨。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "据公开信息，7月29日8时至8月1日11时，涿州市出现明显降水天气过程。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "全市平均降水量355.1毫米，最大降水量为两河村435.7毫米，多个乡镇、街道降水量均超300毫米。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "截至8月1日上午10时，涿州市受灾人数133913人。 " t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "24岁的李志辉已经多次安慰女朋友，但女朋友的一句“家没了，那个房子是我爸妈一辈子的心血”，也瞬间让他陷入无助。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "目前，涿州境内北拒马河、小清河、白沟河等多条河流流量较大，小清河分洪区、兰沟洼蓄滞洪区已相继启动。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "涿州境内防汛形势严峻，多地遭受洪水灾害，多个村庄被洪水围困。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "目前，李志辉女朋友的家人所在的涿州市刁窝镇东辛庄村仍在等待救援，包括附近的白塔村、小营村等同样或在等待救援或正在被救援。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "在社交平台上，也依然能看到大量求助救援的信息正在发出。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "告急的村庄。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "“一共五位家人，其中一位老人已经80多岁，一个孩子只有五六岁，目前都在白塔村一处自建二层小楼上，水已经漫到腰部，有一米多深了，家人都在二楼等候救援。”" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "肖俊介绍，因为到处被淹，也为了陪伴老人，他的家人们8月1日就搬去了刁窝镇白塔村，“结果今天水就涨起来了。”" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "36岁的肖俊平时在北京工作，赶不回去的他觉得现在非常揪心，断断续续的信号也无法获知现场的情况，同时他也认为村子救援难度很大，人员分散也比较难找。" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "“刁窝镇东辛庄村的水已经漫过了一楼。”" t2tt eng --src_lang cmn time python scripts/m4t/predict/predict.py "李志辉告诉新黄河记者，他目前在沧州，女朋友是刁窝镇东辛庄村人，8月1日几位家人有的已经搬到附近的白塔村住，有的还在东辛庄村，跟张俊一样，李志辉女朋友一家也没想到，当地的水涨得如此快。" t2tt eng --src_lang cm
To ease comparison with Google Translate, the text has been chunked into sentences or groups of sentences (though it was translated sentence-by-sentence in Seamless and as a passage in Google Translate). Overall the factual elements of the story are translated fairly well, but transliteration is very different from what Google Translate, Bing Translator and GPT-3.5 yield. In some cases, transliteration varies sentence by sentence, from "Dongxingzhou" to "Dong Xingzhou" and from "Suzhou" to "Yangzhou". Note the failure mode of "such as the North River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River" as well.
- SeamlessM4T: After 12 hours of being cut off from her mother, Zhang Ming, who was in Beijing, finally received a text message from her mother in Yangzhou on August 2 at noon.
- Google Trans: After losing contact with her mother for 12 hours, at noon on August 2, Zhang Meng, who was in Beijing, finally received a text message from her mother in Zhuozhou, "reporting safety".
- SeamlessM4T: The mother, who lives near the village of Three Steps Bridge in the city of Baoding, Hebei, borrowed a message from someone else's cell phone, and she breathed a sigh of relief, and her hanging heart finally let go.
- Google Trans: My mother lives near Sanbuqiao Village, Zhuozhou City, Baoding, Hebei. She borrowed the news from someone else's mobile phone. She breathed a sigh of relief, "The hanging heart is finally let go."
- SeamlessM4T: A rescue raft near the location of Zhang's mother. Since July 29, heavy rains have continued in the area of Beijing and Zhejiang, affected by typhoon Zhejiang.
- Google Trans: A rescue assault boat near where Zhang Meng's mother lives. Photo provided by the interviewee Since July 29, affected by Typhoon "Dusuri", heavy rainfall has continued in the Beijing-Tianjin-Hebei region.
- SeamlessM4T: According to public information, between 8 a.m. on July 29 and 11 a.m. on August 1, there was a noticeable rainfall in the city of Suzhou.
- Google Trans: According to public information, from 8:00 on July 29 to 11:00 on August 1, Zhuozhou experienced significant precipitation.
- SeamlessM4T: The city-wide average rainfall was 355.1 mm, with a maximum of 435.7 mm in Two Rivers Village, and more than 300 mm in several townships and streets. As of 10 a.m. on August 1, 133,913 people were affected in the city of Suzhou.
- Google Trans: The city's average precipitation is 355.1 millimeters, and the maximum precipitation is 435.7 millimeters in Lianghe Village. The precipitation in many towns and streets exceeds 300 millimeters. As of 10 am on August 1, the number of people affected by the disaster in Zhuozhou City was 133,913.
- SeamlessM4T: Twenty-four-year-old Li Zhihui has comforted his girlfriend many times, but the loss of his girlfriend's house, the house that my parents had loved all their lives, instantly left him helpless.
- Google Trans: 24-year-old Li Zhihui has comforted his girlfriend many times, but his girlfriend's sentence "the home is gone, that house is the painstaking effort of my parents all their lives", instantly made him feel helpless.
- SeamlessM4T: At present, the flow of several rivers in the state, such as the North River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River, the Little River.
- Google Trans: At present, the North Juma River, Xiaoqing River, Baigou River and other rivers in Zhuozhou have relatively large flows, and the flood diversion area of Xiaoqing River and the flood storage and detention area of Langou depression have been started one after another.
- SeamlessM4T: The flood prevention situation in Yangzhou is critical, many places have been affected by flooding, and several villages have been besieged by flooding. Currently, Li Zhihui's girlfriend's family is still waiting for rescue in the village of Dongxingzhou, Zhejiang City, including the nearby village of Baita, the small camp village, etc. are also waiting for rescue or are being rescued.
- Google Trans: The flood control situation in Zhuozhou is severe. Many places have suffered from floods, and many villages have been besieged by floods. At present, Dongxinzhuang Village, Diaowo Town, Zhuozhou City, where Li Zhihui's girlfriend's family lives, is still waiting for rescue, including the nearby Baita Village and Xiaoying Village, which are also waiting for rescue or being rescued.
- SeamlessM4T: On social platforms, you can still see a lot of messages asking for help. Emergency village. A total of five families, including an elderly man in his 80s and a child who is only five or six years old, are currently living in a self-built two-story building in the village of White Tower, where the water is up to one meter deep and the family is waiting for help on the second floor.
- Google Trans: On social platforms, you can still see a large number of messages for help and rescue being sent out. A desperate village. "A total of five family members, one of whom is over 80 years old, and one child who is only five or six years old, are currently living in a self-built two-story building in Baita Village. The water has reached their waists and is more than one meter deep. The family members are all waiting for rescue on the second floor.
- SeamlessM4T: Shao Jun explained that because of the flooding everywhere, and in order to accompany the elderly, his family moved to the village of Baita in the town of Taiyuan on August 1, and as a result, the water rose today. Shao Junping, 36, who worked in Beijing and couldn't hurry back, feels very distressed now, and the intermittent signals don't tell him what's going on, and he also believes that it's very difficult to rescue the village, and it's more difficult to disperse the people.
- Google Trans: Xiao Jun introduced that his family moved to Baita Village, Diaowo Township on August 1 because they were flooded everywhere and to accompany the elderly. As a result, the water rose today. 36-year-old Xiao Jun usually works in Beijing, and he feels very worried now that he can’t make it back, because the intermittent signal can’t know the situation on the scene.
- SeamlessM4T: Water has flooded a floor of the village of Dong Xingzhou in the town of Zhejiang. Li Zhihui told the New Yellow River reporter that he is currently in Yangzhou, and his girlfriend is from the village of Dongxingzhou in the town of Zhejiang. On August 1, several families had moved to the nearby village of White Tower, and some were still living in Dongxingzhou.
- Google Trans: "The water in Dongxinzhuang Village, Diaowo Town has already overflowed the first floor." Li Zhihui told the reporter of New Yellow River that he is currently in Cangzhou, and his girlfriend is from Dongxinzhuang Village, Diaowo Town. On August 1, several family members had moved. They went to live in the nearby Baita Village, and some were still in Dongxinzhuang Village. Like Zhang Jun, Li Zhihui's girlfriend's family never expected that the local water would rise so fast