More Experiments With LLM Translation: GCP's PaLM Vs ChatGPT Vs Google Translate On News Translation

With the General Availability of Chinese language support in Google's PaLM LLM, let's repeat our full-length news media translation experiments, comparing PaLM and ChatGPT (GPT-3.5) LLM translations against Google Translate's NMT translations.

Let's take a look at this full-length news article:

在跟母亲断联12个小时后,8月2日中午,身在北京的张萌终于收到了位于涿州的母亲的“报平安”短信。母亲住在河北保定涿州市三步桥村附近,是借用别人手机发来的消息,她长舒了一口气,“悬着的心终于放下了”。 张萌母亲所在地附近的救援冲锋舟。受访者供图 7月29日以来,受台风“杜苏芮”影响,京津冀地区持续强降雨。据公开信息,7月29日8时至8月1日11时,涿州市出现明显降水天气过程。全市平均降水量355.1毫米,最大降水量为两河村435.7毫米,多个乡镇、街道降水量均超300毫米。截至8月1日上午10时,涿州市受灾人数133913人。 24岁的李志辉已经多次安慰女朋友,但女朋友的一句“家没了,那个房子是我爸妈一辈子的心血”,也瞬间让他陷入无助。目前,涿州境内北拒马河、小清河、白沟河等多条河流流量较大,小清河分洪区、兰沟洼蓄滞洪区已相继启动。涿州境内防汛形势严峻,多地遭受洪水灾害,多个村庄被洪水围困。 目前,李志辉女朋友的家人所在的涿州市刁窝镇东辛庄村仍在等待救援,包括附近的白塔村、小营村等同样或在等待救援或正在被救援。在社交平台上,也依然能看到大量求助救援的信息正在发出。告急的村庄。 “一共五位家人,其中一位老人已经80多岁,一个孩子只有五六岁,目前都在白塔村一处自建二层小楼上,水已经漫到腰部,有一米多深了,家人都在二楼等候救援。”肖俊介绍,因为到处被淹,也为了陪伴老人,他的家人们8月1日就搬去了刁窝镇白塔村,“结果今天水就涨起来了。”36岁的肖俊平时在北京工作,赶不回去的他觉得现在非常揪心,断断续续的信号也无法获知现场的情况,同时他也认为村子救援难度很大,人员分散也比较难找。“刁窝镇东辛庄村的水已经漫过了一楼。”李志辉告诉新黄河记者,他目前在沧州,女朋友是刁窝镇东辛庄村人,8月1日几位家人有的已经搬到附近的白塔村住,有的还在东辛庄村,跟张俊一样,李志辉女朋友一家也没想到,当地的水涨得如此快。

PaLM (note the "rescue assault boat"):

At noon on August 2, after 12 hours of losing contact with her mother, Zhang Meng, who was in Beijing, finally received a "safe and sound" text message from her mother in Zhuozhou. Her mother lives near Sanbuqiao Village in Zhuozhou City, Baoding, Hebei Province. She sent the message using someone else's mobile phone. She took a deep breath and "finally let go of her hanging heart." Rescue assault boat near Zhang Meng's mother's location. Provided by the interviewee Since July 29, affected by Typhoon Dujuan, the Beijing-Tianjin-Hebei region has experienced continuous heavy rainfall. According to public information, from 8:00 on July 29 to 11:00 on August 1, there was significant rainfall in Zhuozhou City. The average precipitation in the city was 355.1 mm, the maximum precipitation was 435.7 mm in Lianghe Village, and the precipitation in many towns and streets exceeded 300 mm. As of 10:00 am on August 1, 133,913 people were affected by the disaster in Zhuozhou City. Li Zhihui, 24, has repeatedly comforted his girlfriend, but her words "home is gone, that house is my parents' lifelong effort", also made him feel helpless in an instant. At present, the flow of many rivers in Zhuozhou, such as Beijuma River, Xiaoqing River, and Baigou River, is relatively large. The Xiaoqing River Flood Diversion Area and the Langouwa Flood Detention Area have been put into operation one after another. The flood control situation in Zhuozhou is severe, many areas have suffered flood disasters, and many villages have been besieged by floods. At present, the Dongxinzhuang Village in Diaowo Town, Zhuozhou City, where Li Zhihui's girlfriend's family is located, is still waiting for rescue, including nearby villages such as Baitawu Village and Xiaoying Village, which are also waiting for rescue or are being rescued. On social platforms, a large amount of information about seeking help and rescue can still be seen. The village is in urgent need of help. "There are five family members in total, one of whom is over 80 years old and one child is only five or six years old. They are currently on the second floor of a self-built small building in Baitawu Village. The water has reached their waists, more than one meter deep. The family members are all waiting for rescue on the second floor." Xiao Jun introduced that because of the flooding everywhere and in order to accompany the elderly, his family moved to Baitawu Village, Diaowo Town on August 1, "but today the water rose." Xiao Jun, 36, usually works in Beijing. He couldn't go back in time and felt very worried. The intermittent signal could not get the situation on the scene. At the same time, he also believed that the rescue of the village was very difficult, and it was difficult to find scattered people. "The water in Dongxinzhuang Village, Diaowo Town has flooded the first floor." Li Zhihui told the New Yellow River reporter that he is currently in Cangzhou. His girlfriend is from Dongxinzhuang Village, Diaowo Town. On August 1, some family members had already moved to the nearby Baitawu Village, and some were still in Dongxinzhuang Village. Like Zhang Jun, Li Zhihui's girlfriend and her family did not expect that the local water would rise so quickly.

ChatGPT (note the absence of "A rescue boat near Zhang Meng’s mother’s location. Photo provided by interviewee"):

After being out of contact with her mother for 12 hours, Zhang Meng, who was in Beijing, finally received a "safe and sound" message from her mother in Zhuozhou, Hebei. Her mother lives near Sanbuqiao Village in Zhuozhou City and had borrowed someone else's phone to send the message. Zhang Meng breathed a sigh of relief, saying, "I can finally put my anxious heart at ease."

Since July 29th, due to the influence of Typhoon "Dusu Ray," there has been continuous heavy rainfall in the Beijing-Tianjin-Hebei region. According to public information, from 8:00 AM on July 29th to 11:00 AM on August 1st, Zhuozhou City experienced significant precipitation. The average precipitation in the city was 355.1 millimeters, with the highest recorded precipitation of 435.7 millimeters in Lianghe Village. Many townships and streets received over 300 millimeters of rainfall. As of 10:00 AM on August 1st, the number of people affected by the disaster in Zhuozhou City was 133,913.

24-year-old Li Zhihui has comforted his girlfriend many times, but when his girlfriend said, "Our home is gone, that house was my parents' lifelong effort," he felt helpless instantly. Currently, there are several rivers with large water volumes within Zhuozhou, including Beijuma River, Xiaoqing River, and Baigou River. Flood diversion areas for Xiaoqing River and flood storage areas for Langouwa have been activated successively. The flood control situation in Zhuozhou is severe, with many areas suffering from flooding, and several villages are surrounded by floodwaters.

Currently, the family of Li Zhihui's girlfriend in Dongxinzhuang Village, Diaowo Town, Zhuozhou City, is still waiting for rescue. Nearby villages such as Baita Village and Xiaoying Village are either waiting for rescue or currently being rescued as well. On social media platforms, numerous calls for help and rescue information are still being posted.

"One family consists of five members, including an elderly person in their 80s and a child who is only five or six years old. They are currently on the second floor of a self-built two-story building in Baita Village, where the water has risen to waist-deep, over a meter deep. The family is waiting for rescue on the second floor," said Xiao Jun. Because the entire area is flooded and to accompany the elderly, his family moved to Baita Village in Diaowo Town on August 1st. "As a result, the water started rising today." Xiao Jun, 36, normally works in Beijing and cannot return, so he is very worried. The intermittent signal also prevents him from knowing the situation on-site. At the same time, he believes that rescuing the village is difficult due to the challenging conditions, and finding scattered people is also difficult. "The water in Dongxinzhuang Village, Diaowo Town, has already flooded the first floor," Li Zhihui told the New Yellow River reporter. He is currently in Cangzhou, and his girlfriend is from Dongxinzhuang Village in Diaowo Town. On August 1st, some of her family members had already moved to nearby Baita Village, while others were still in Dongxinzhuang Village. Like Zhang Jun, Li Zhihui's girlfriend's family also didn't expect the local water to rise so quickly.

Google Translate:

After being disconnected from her mother for 12 hours, at noon on August 2, Zhang Meng, who was in Beijing, finally received a "safe" text message from her mother in Zhuozhou. The mother lives near Sanbuqiao Village in Zhuozhou City, Baoding, Hebei Province. The message was sent by borrowing someone else's mobile phone. She breathed a sigh of relief, "I finally let go of my hanging heart." A rescue boat near Zhang Meng’s mother’s location. Photo provided by interviewee: Since July 29, due to the impact of Typhoon Dusuri, heavy rainfall has continued in the Beijing-Tianjin-Hebei region. According to public information, from 8:00 on July 29 to 11:00 on August 1, Zhuozhou City experienced significant precipitation. The city's average precipitation is 355.1 mm, with the maximum precipitation being 435.7 mm in Lianghe Village. The precipitation in many towns and streets exceeds 300 mm. As of 10 a.m. on August 1, the number of people affected by the disaster in Zhuozhou City was 133,913. 24-year-old Li Zhihui has comforted his girlfriend many times, but his girlfriend's words, "My home is gone. That house is the life-long effort of my parents" instantly made him helpless. At present, many rivers in Zhuozhou, including the Beijuma River, Xiaoqing River, and Baigou River, have large flows, and the Xiaoqing River flood diversion area and Langouwa flood storage and detention area have been activated one after another. The flood control situation in Zhuozhou is severe. Many places have suffered flood disasters and many villages have been besieged by floods. Currently, Dongxinzhuang Village, Diaowo Town, Zhuozhou City, where the family of Li Zhihui’s girlfriend lives, is still waiting for rescue, and nearby Baita Village, Xiaoying Village, etc. are also either waiting for rescue or being rescued. On social platforms, you can still see a large number of messages asking for help and rescue being sent out. A desperate village. "There are five family members in total, including an old man who is over 80 years old and a child who is only five or six years old. They are currently living in a self-built two-story building in Baita Village. The water has reached their waists and is more than one meter deep. The family is waiting for rescue on the second floor." Xiao Jun said that because everywhere was flooded and to accompany the elderly, his family moved to Baita Village, Diaowo Town on August 1. "As a result, the water rose today. " 36-year-old Xiao Jun usually works in Beijing. He feels very worried now that he can't go back in time. The intermittent signal makes it impossible to know the situation at the scene. At the same time, he also thinks that the rescue in the village is very difficult and the personnel are scattered and difficult to find. "The water in Dongxinzhuang Village, Diaowo Town has already reached the first floor." Li Zhihui told the New Yellow River reporter that he is currently in Cangzhou and his girlfriend is from Dongxinzhuang Village, Diaowo Town. Some of his family members have moved on August 1. They went to live in nearby Baita Village, and some were still living in Dongxinzhuang Village. Like Zhang Jun, Li Zhihui’s girlfriend’s family never expected that the local water would rise so fast.

What about a second example, of a summer camp?

昆明一夏令营被举报:160多名孩子住工棚!有人10多天没洗澡。春城晚报-开屏新闻消息,正值暑假,不少家长都选择将孩子送到各类夏令营体验生活、学习技能。连日来,不断有读者向春城晚报-开屏新闻反映:在昆明万科城市运动公园里,有一家夏令营机构,从云南各地组织了160多名孩子来昆明参加夏令营。可孩子们来了后,所谓的“营地”在一块荒地里,孩子们住的是临时搭建的工棚,每晚被蚊虫叮咬。部分孩子参加夏令营10多天,还没洗过一次澡。而营地里所谓的“教官”,有的是暑假期间做兼职的大学生。 集装箱搭建成夏令营宿舍。 7月31日下午,记者来到昆明市万科城市运动公园,周围都是小区楼房,一点都看不出像夏令营集中训练的“营地”,通往“营地”的大门口上挂着一个广告牌写着:“2023城市猎人夏令营”,另外还挂着两块红字黄底的招牌,分别是怒江滇西明珠旅游产业投资开发有限公司、云南一四九校企教育服务有限公司。看来,这里举办的夏令营是这两家公司联合举办的。 按照爆料人指引,记者通过二层楼的过道时看到,工人正在施工搭建楼梯,再往里走,一股从厕所飘出来的臭味扑鼻而来。而这里,就是参加夏令营的孩子们就餐的地方。 据一名知情人士介绍:孩子们的“营地”就在对面的荒地上,平常训练、睡觉都在那里。 记者来到所谓“营地”上看到,主办方用集装箱搭建了多个工棚,四周用军绿色彩布蒙着,两排工棚中间的一块空地上铺着人工草皮。

PaLM:

A summer camp in Kunming was reported: More than 160 children live in sheds! Some have not taken a bath for more than 10 days. According to the news from Spring City Evening News-Kai Screen News, it is summer vacation, and many parents choose to send their children to various summer camps to experience life and learn skills. In recent days, readers have been reflecting to Spring City Evening News-Kai Screen News: In Kunming Vanke City Sports Park, there is a summer camp organization that has organized more than 160 children from all over Yunnan to Kunming to participate in the summer camp. However, after the children arrived, the so-called "camp" was in a wasteland, and the children lived in temporary sheds, being bitten by mosquitoes every night. Some children have not taken a bath after participating in the summer camp for more than 10 days. The so-called "instructors" in the camp are college students who work part-time during the summer vacation. The container is built into a summer camp dormitory. On the afternoon of July 31, the reporter came to Kunming Vanke City Sports Park. There are residential buildings all around, and it does not look like a "camp" for summer camp intensive training. There is an advertisement牌 at the gate of the "camp" that reads: "2023 City Hunter Summer Camp". There are also two red-lettered and yellow-bottomed signs, which are Nujiang Dianxi Mingzhu Tourism Industry Investment and Development Co., Ltd. and Yunnan 149 School-Enterprise Education Service Co., Ltd. It seems that the summer camp held here is co-organized by these two companies. According to the whistleblower's instructions, when the reporter passed through the corridor on the second floor, he saw that the workers were building stairs. Going further inside, a stench from the toilet came to his nostrils. And this is where the children who participated in the summer camp dine. According to an informed source: The children's "camp" is on the wasteland opposite, where they usually train and sleep. When the reporter came to the so-called "camp", he saw that the organizer had built several sheds with containers, covered with army green tarpaulin around them, and a piece of artificial turf was laid on the open space between the two rows of sheds.

ChatGPT:

Summer camp in Kunming reported: Over 160 children living in makeshift sheds! Some haven't bathed for over 10 days. In the midst of summer vacation, many parents choose to send their children to various summer camps to experience life and learn new skills. In recent days, readers have continuously reported to the Chun Cheng Evening News – Kai Ping News: in the Kunming Vanke City Sports Park, there is a summer camp organization that has brought over 160 children from various places in Yunnan to participate in the summer camp in Kunming. However, after the children arrived, the so-called "campsite" was in a barren area, and the children were staying in temporary makeshift sheds, being bitten by mosquitoes every night. Some children have been in the summer camp for over 10 days without taking a bath. The so-called "instructors" at the camp are some part-time college students during the summer vacation.

On the afternoon of July 31st, reporters arrived at the Kunming Vanke City Sports Park. Surrounded by residential buildings, there was no sign of a "campsite" for concentrated summer camp training. Above the entrance gate to the "campsite," there was a sign that read "2023 City Hunter Summer Camp." In addition, there were two red-letter yellow-background signs, which belonged to Nujiang Dianxi Mingzhu Tourism Industry Investment and Development Co., Ltd. and Yunnan 149 School-Enterprise Education Service Co., Ltd. It appears that this summer camp is jointly organized by these two companies.

Following the tip-off, the reporter saw that workers were constructing stairs in the corridor on the second floor. Further inside, a strong odor wafted from the restroom. This was where the children participating in the summer camp had their meals.

According to an informant, the children's "campsite" is in the barren area opposite, where they usually train and sleep.

When the reporter arrived at the so-called "campsite," they saw that the organizers had used shipping containers to construct multiple makeshift sheds. Military green canvas surrounded the area, and artificial grass was laid on the open space between the two rows of sheds.

Google Translate:

A summer camp in Kunming was reported: more than 160 children lived in work shacks! Some people didn't take a shower for more than 10 days. Spring City Evening News – Open screen news, it is summer vacation, and many parents choose to send their children to various summer camps to experience life and learn skills. In the past few days, readers have been reporting to Spring City Evening News – Open Screen News: There is a summer camp organization in Kunming Vanke City Sports Park, which organized more than 160 children from all over Yunnan to come to Kunming to participate in the summer camp. But when the children arrived, the so-called "camp" was in a wasteland. The children lived in temporary work sheds and were bitten by mosquitoes every night. Some children attended the summer camp for more than 10 days and had not even had a bath. Many of the so-called "instructors" in the camp are college students who work part-time during the summer vacation. Containers built into summer camp dormitories. On the afternoon of July 31, the reporter came to Kunming Vanke City Sports Park. It was surrounded by residential buildings. It did not look like a "camp" for concentrated summer camp training at all. There was a billboard hanging on the door leading to the "camp". It reads: "2023 City Hunter Summer Camp", and there are also two signboards with red letters and yellow backgrounds, which are Nujiang West Yunnan Pearl Tourism Industry Investment and Development Co., Ltd. and Yunnan 149 School Enterprise Education Service Co., Ltd. It seems that the summer camp held here is jointly organized by these two companies. Following the whistleblower's instructions, the reporter saw workers building stairs when passing through the corridor on the second floor. As he walked further in, a smell coming from the toilet hit his nostrils. And here is where the children attending the summer camp eat. According to an insider, the children’s “camp” is on the wasteland opposite, where they usually train and sleep. The reporter came to the so-called "camp" and saw that the organizers had built multiple work sheds out of containers, surrounded by military green cloth, and an open space in the middle of the two rows of work sheds was covered with artificial turf.

Let's take two paragraphs from this news article from this past May and ask ChatGPT to translate them into English:

工作人員當時正在調配染髮劑,發現後上前阻止已經來不及,女客則是覺得有異樣,轉頭一看發現自己被剪掉一大磋頭髮,氣得當場報警處理,最終三方達成和解,母親賠償女客人11500人民幣,美容院則是未盡管理責任,必須免費提供8次共12000人民幣的美髮服務。 不少大陸網友也搖頭:「這種熊孩子不管以後還得了」、「熊孩子的罪,家長必須承擔」、「媽媽顧著滑手機完全不理女兒也是奇葩」、「一刀1萬,好貴的學費」、「其實美容院應該才是主責方」、「是我不會要錢,我會直接把熊孩子剃光頭」。

PaLM:

The staff were preparing hair dye at the time. When they found out, it was too late to stop her. The female customer felt something was wrong and turned her head to find that a large section of her hair had been cut off. She was so angry that she immediately called the police. In the end, the three parties reached a settlement. The mother compensated the female customer with 11,500 yuan, and the beauty salon, which had failed to fulfill its management responsibilities, had to provide 8 free hairdressing services worth a total of 12,000 yuan. Many mainland netizens also shook their heads: "If this kind of naughty child is not disciplined, what will happen in the future?" "The parents must bear the blame for the naughty child's sins." "It's strange that the mother was so engrossed in her mobile phone that she completely ignored her daughter." "10,000 yuan for a haircut, what an expensive lesson." "In fact, the beauty salon should be the main party responsible." "If it were me, I wouldn't ask for money. I would just shave the naughty child's head."

ChatGPT:

The staff were in the process of preparing hair dye at the time. When they noticed it, it was already too late to stop it. The female customer felt something unusual and turned her head to find that a significant portion of her hair had been cut off. She was so angry that she immediately called the police. Eventually, all three parties reached a settlement. The mother compensated the female customer with 11,500 RMB, while the beauty salon was found to have not fulfilled its management responsibilities and was required to provide eight free hairdressing services worth a total of 12,000 RMB.

Many mainland Chinese netizens also shook their heads, saying, "These mischievous kids need to be disciplined in the future," "The parents must bear the responsibility for the actions of these mischievous kids," "The mom was too engrossed in her phone and completely ignored her daughter; that's bizarre," "10,000 RMB for a single cut, what an expensive lesson," "Actually, the beauty salon should be the primary party responsible," "If it were me, I wouldn't ask for money; I would just shave the mischievous kid's head bald."

Google Translate:

The staff was mixing hair dye at the time. When they found out, it was too late to stop them. The female guest felt something was strange. She turned around and found that a large amount of her hair had been cut off. She was so angry that she called the police on the spot. In the end, the three parties reached a settlement and the mother The female customer was compensated 11,500 RMB. The beauty salon failed to fulfill its management responsibilities and must provide 8 free hairdressing services totaling 12,000 RMB. Many mainland netizens also shook their heads: "I don't care if this kind of naughty kid will get it in the future", "Parents must bear the blame for the naughty kid", "It's weird for a mother to completely ignore her daughter while scrolling on her phone", "10,000 for one knife, so expensive. "The tuition fee", "In fact, the beauty salon should be the main responsible party", "I won't ask for money, I will just shave the naughty child's head."