With the General Availability of Chinese language support in Google's PaLM LLM, let's repeat our Weibo social media post translation experiments, comparing PaLM and ChatGPT (GPT-3.5) LLM translations against Google Translate's NMT translations. In all three examples, the LLMs provide superior results to Google Translate's NMT translations. Comparing PaLM and ChatGPT, each performs better in some cases and worse in others, with no clear winner. However, in one of the examples below and in additional testing, PaLM has a more limited Chinese vocabulary, frequently passing through characters untranslated, while ChatGPT appears to have much better coverage.
Let's start with one of the Weibo posts we looked at last time:
【#他撑伞蹲了很久让女孩知道世界很美好#】近日，湖南长沙，一女子因感情问题一时想不开，独自坐在楼顶哭泣。消防员赶赴，温柔蹲在女孩身旁安抚情绪，夜色沉沉，天空下起小雨，救援人员为其撑伞暖心守护，经过近一小时的劝导，将女孩平安带下15楼天台。ps：答应我们，好好爱自己，世间还有很多美好~#女孩感情受挫消防员撑伞守护# 中国消防的微博视频
PaLM yields:
【#He held an umbrella and crouched for a long time to let the girl know that the world is beautiful#】Recently, in Changsha, Hunan Province, a woman was so upset about her relationship that she sat alone on the roof and cried. The firefighters rushed to the scene and crouched gently beside the girl to comfort her. The night was dark and the sky was drizzling. The rescuers held an umbrella for her and guarded her warmly. After nearly an hour of persuasion, the girl was safely brought down from the 15th floor rooftop. PS: Promise us, love yourself well, there are still many beautiful things in the world~ #The girl was frustrated by her relationship and the firefighter held an umbrella to protect her# Weibo video of China Fire
ChatGPT:
【#He Held an Umbrella and Squatted for a Long Time to Show the Girl the Beauty of the World#】Recently, in Changsha, Hunan, a young woman who was going through a tough time emotionally sat alone on the rooftop, crying due to relationship problems. Firefighters rushed to the scene, gently squatting beside the girl to console her. As the night grew darker, a light rain began to fall from the sky. The rescue team held an umbrella over her, providing warm protection. After nearly an hour of counseling, they safely brought the girl down from the 15th-floor rooftop. PS: Promise us to love yourself and remember that there is still much beauty in the world. #Girl Emotionally Troubled, Firefighters Holding Umbrella to Protect# Video shared on the official Weibo account of China's Firefighters.
Both are more fluent than Google Translate:
[#他 hold an umbrella and squat for a long time to let girls know that the world is beautiful#] Recently, in Changsha, Hunan, a woman couldn't think about it for a while because of emotional problems, so she sat alone on the roof of a building and cried. The firefighters rushed over and squatted beside the girl to comfort her. It was dark at night and it started to rain lightly. The rescuers held an umbrella to warmly protect her. After nearly an hour of persuasion, the girl was safely brought down to the rooftop of the 15th floor. ps: Promise us, love yourself well, there is still a lot of beauty in the world~ #girls’ feelings are frustrated, firefighters hold umbrellas to protect # Chinese firefighting Weibo video
What about the second Weibo post we tested last time?
一堂童画课，用色彩传递温暖！送困境儿童“美育盒子”，送自闭症儿童艺术疗愈课程。让我们与@延参法师 一堂童画课公益发起人，在这个充满爱与温暖的日子里，一起感受艺术的魅力，用童心绘制美好，为公益事业贡献一份力量！@一堂童画课 北京市行远公益基金会的微博视频
PaLM yields:
A children's painting class, using colors to convey warmth! Send "art education boxes" to children in difficult situations, and send art therapy courses to children with autism. Let us, together with @延参法师, the initiator of the public welfare of a children's painting class, feel the charm of art on this day full of love and warmth, and use innocence to draw a better future and contribute to the public welfare cause! @一堂童画课 Beijing Xingyuan Public Welfare Foundation's Weibo video
ChatGPT:
"A children's painting class, conveying warmth through colors! We are sending 'Art Education Boxes' to underprivileged children and providing art therapy classes for autistic children. Let's join hands with @Master Yancan, the initiator of the children's painting class, on this day filled with love and warmth. Let's experience the charm of art together, create beauty with childlike hearts, and contribute to the cause of public welfare! @Children's Painting Class, Beijing Xingyuan Public Welfare Foundation's Weibo video."
Google Translate:
A children's painting class, using colors to convey warmth! Provide "aesthetic education boxes" to children in need and art therapy courses to children with autism. Let us join @yancan法师, the charity sponsor of a children's painting class, to feel the charm of art together on this day full of love and warmth, draw beauty with childlike innocence, and contribute to public welfare! @ a children's painting class Weibo video of Beijing Xingyuan Charity Foundation
What about a more complex example from this Quora post about Chinese text that is difficult to translate?
阿呆给领导送红包时，两人的对话颇有意思。 领导：你这是什么意思？ 阿呆：没什么意思，意思意思。 领导：你这就不够意思了。 阿呆：小意思，小意思。 领导：你这人真有意思。 阿呆：其实也没有别的意思。 领导：那我就不好意思了。 阿呆：是我不好意思。 领导：你肯定有什么意思。 阿呆：真的没有什么意思。 领导：既然没有什么意思，那你是什么意思？ 阿呆：其实，我的意思就是想意思意思。 领导：你既然是想意思意思，那就是有什么意思。 阿呆哭了：我就是想意思意思。但是，真的没有什么别的意思。这么个小红包能有什么意思？也就是意思意思而已。 领导笑了：呵呵。我对你有点意思了。 阿呆心想：嘻嘻。我就是这个意思。
Original translation provided by author:
One day, Adele wanted to bribed her boss with money. Their conversation was pretty interesting. Boss: What do you mean? Adele: No bad purposes. It's just my tiny present to you. Boss: Then you are so mean. It could do harm to my career. Adele: Just a tiny present, just a tiny present. Boss: Oh, you are very funny. Adele: Actually, I have no bad purposes. Boss: Well, I am a little embarrassed….I shouldn’t get you wrong… Adele: Oh,no, I am the one who should feel shamed. Boss: But to be honest,what do you want from me? Adele: I want nothing from you, my boss. Boss: Since you want nothing, then why do you bribe me? Adele: Actually I just want to show you my heart. Boss: Since you just want to show me your heart, then you must want something! Adele: Please don't get me wrong! I just want to show my heart, no other bad purposes! Money can represent nothing at all! Boss( smiled): Then I have feelings for you. Adele(thought): Haha, that’s what I want from you.
PaLM yields:
When Ah-Dai gave the red envelope to the leader, the conversation between the two was quite interesting.
Leader: What do you mean by this?
Ah-Dai: It doesn't mean anything, just a little something.
Leader: That's not good enough.
Ah-Dai: It's just a small thing, a small thing.
Leader: You're really interesting.
Ah-Dai: Actually, it doesn't mean anything else.
Leader: Then I'm embarrassed.
Ah-Dai: I'm the one who's embarrassed.
Leader: You must mean something.
Ah-Dai: I really don't mean anything.
Leader: Since it doesn't mean anything, then what do you mean?
Ah-Dai: Actually, I just want to mean something.
Leader: Since you want to mean something, then that means something.
Ah-Dai cried: I just want to mean something. But, I really don't mean anything else. What can such a small red envelope mean? It's just a little something.
The leader smiled: Hehe. I'm a little interested in you.
Ah-Dai thought: Hehe. That's what I mean.
ChatGPT:
When A'dai gave a red envelope to the leader, their conversation was quite amusing.
Leader: What does this mean?
A'dai: It doesn't mean much, just a gesture.
Leader: Your gesture is a bit lacking.
A'dai: It's just a small gesture, a small one.
Leader: You're an interesting person.
A'dai: Actually, there's nothing else to it.
Leader: Well, then, I feel a bit embarrassed.
A'dai: It's my fault, I'm sorry.
Leader: You must have some intention.
A'dai: I really don't have any intention.
Leader: If you have no intention, then what's your intention?
A'dai: Actually, my intention is just to make a gesture.
Leader: If you intend to make a gesture, then it means something.
A'dai started crying: I just wanted to make a small gesture. There really isn't any other meaning. What could a small red envelope mean? It's just a gesture.
The leader laughed: Haha, I'm starting to understand your meaning.
A'dai thought to himself: Hehe, that's exactly my intention.
Google Translate:
When Ah-Dai gave the leader a red envelope, the conversation between the two was quite interesting. Leader: What do you mean? Dumb: It doesn’t mean anything, it means nothing. Leader: You are not interesting enough. Dumb: It’s trivial, it’s trivial. Leader: You are really interesting. Dumb: Actually, it doesn’t mean anything else. Leader: Then I’m sorry. Dumb: I'm sorry. Leader: You must have something on your mind. Dumb: It’s really meaningless. Leader: Since it doesn’t mean anything, what do you mean? Dumb: Actually, I just mean what I mean. Leader: Since you want to mean something, then that’s what it means. Dumb cried: I just want to mean something. But, it really doesn’t mean anything else. What's the meaning of such a small red envelope? That's just what it means. The leader laughed: Haha. I'm interested in you. Dumb thought to himself: Hee hee. That's what I mean.