The GDELT Project

Experiments In Meme Tracking: Dividing Transcripts Into Stories – Part 4 & The Impact Of Capitalization, Punctuation & Hallucination

Kalev Leetaru

Continuing our meme tracking series, let's take a closer look at the general task of summarization/distillation. In particular, let's take a closer look at capitalization, punctuation and hallucination in our summaries.

Let's start with a summarization request for one of the passages that we struggled with in our previous episode:

time curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/[YOURPROJECTID]/locations/us-central1/publishers/google/models/text-bison:predict -d \
$'{
  "instances": [
    { "prompt": "Summarize in one paragraph the following text:\n\n   commercial pilots radioing in. >> we got that balloon in sight also. looks like it\'s way up there, maybe 50,000 feet or so. >> reporter: the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana. >> this thing is up in the sky. >> what the heck is that? >> that thing is not the moon. >> any help would be appreciated. >> reporter: montana republican congressman ryan zinke was blunt, tweeting \\"shoot. it. down.\\" the pentagon today sayig that option was considered and rejected, for now. >> we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options. >> reporter: the white house today saying president biden agreed with the pentagon\'s strong recommendation. the risks involved with shooting down the balloon just too great. >> any potential debris field would be significant and potentially cause civilian injuries or deaths or significant property damage. >> reporter: for its part, china insisting today the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. the pentagon flatly rejecting that claim. >> we are aware of the prc\'s statement. however, the fact is, we know that it\'s a surveillance pballo. we have clearly communicated that this balloon is violating u.s. air space and international law and that this is unacceptable. >> reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it\'s likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing. windchill ever recorded in the u.s., including alaska. snow squalls along with that arctic chill. a time lapse from vermont. a cold emergency in place in boston. this is through sunday now. rob marciano standing by to time this out, and abc\'s victor oquendo in boston where the wind chill is 13 below zero right now. >> reporter: tonight, millions of families are in the midst of what the national weather service is calling a \\"once in a generation\\" arctic blast. >> we\'re seeing wind chills right now which are already below zero and before the night\'s over with, these are going to drop even more. >> reporter: parts of new england expecting wind chills at or near 60 below zero. outside springfield, massachusetts, as winds gusted above 50 miles per hour, a tree crashing onto a car, killing a 6-month-old girl inside. the woman driving rushed to the hospital. in boston, pilots battling high winds as they landed at logan airport. >> we\'re going to load food and stuff in here. >> reporter: and outreach teams hitting the streets to care for the needy. >> if anyone is out and sees someone else who might be in distress, please call 911. >> reporter: doctors say hypothermia can set in in just minutes. >> this weather can be deadly. it\'s life threatening. this is something that people have to really take seriously. >> reporter: the arctic front visible in the form of powerful snow squalls pushing from upstate new york through new england. here in boston, a brutal night ahead. a cold emergency is in place. winds will gust to 40 miles per hour, so there\'s a real concern about power outages, and it will feel like at least 30 below. david? >> victor oquendo in boston. victor, thank you. let\'s get to rob marciano who\'s timing this out for us tonight. hey, rob. >> reporter: hi, david. if you live in the northeast, you\'ve felt it all day long. temperatures have been dropping. now these winds are really ripping, measured over 40 miles per hour already. and the wind chill advisory is pretty much everywhere north of new york city. here you see it on the map. it\'s no picnic south of here either. minus 23 for a wind chill right now in albany. minus 13 in boston. but numbers in the morning will be even more dangerous, and that red spot blizzard warning, that\'s for blowing snow on the ground. minus 57 in caribou. we get to that figure or less, that\'s an all-time record. minus 11 in new york. this is hypothermic stuff, frostbite in minutes. but a rapid rebound in temperatures coming during the day sunday. we\'ll start in single digits in many spots all the way into the 40s for sunday, monday, and tuesday. so it\'s just a matter of survival for the next 36 hours. david? >> whiplash for sure, but take saturday seriously. rob, we thank you. get back inside. we\'re going to turn next to the doctor struck by a car. police say he was then attacked. he did not survive. that emergency room doctor riding his bike on the pacific coast highway, struck by a driver. police say the driver then got out of the car and stabbed the doctor. bystanders taking the attacker down. here\'s our chief correspondent matt gutman from california tonight. >> reporter: tonight, an e.r. doctor is dead after police in california say a motorist slammed into him, then got out of his car and stabbed him."}
  ],
  "parameters": {
    "temperature": 0.0,
    "maxOutputTokens": 1024,
    "topK": 40,
    "topP": 0.95,
  }
}' > O; cat O | jq -r .predictions[0].content

Let's try this with a temperature of 0.0. Strangely the LLM does not correct the casing of the text. Importantly, it fixates on only the last story and hallucinates details of the story including the names:

the attack happened on the pacific coast highway in the city of monterey. police say the victim, 54-year-old gary riese, was riding his bike when he was hit by a car. the driver, 23-year-old vincent trent johnson, then got out of his car and stabbed riese multiple times. good Samaritans stopped to help and held johnson down until police arrived. johnson is now in custody and facing charges of murder and hit-and-run. riese was an e.r. doctor at monterey bay hospital. he was also a volunteer firefighter and a member of the local cycling club. his death has sent shockwaves through the community.

With a temperature of 0.2 it still fixates and hallucinates:

Temperature 0.4 continues this trend of fixation and hallucination. However, in one of the stories it did include some detail about the balloon:

Max temperature of 0.99:

What if we remove the ">>" speaker change notations from the transcript and use a temperature of 0.0?

the attack happened on the pacific coast highway in the city of monterey. police say the driver, 32-year-old vincent trent pugh, was arrested and booked into jail on suspicion of murder. the victim has been identified as 54-year-old gary ries. he was a doctor at monterey memorial hospital. police say pugh was driving erratically when he hit ries, who was riding his bike. pugh then got out of his car and stabbed ries multiple times. good Samaritans stopped pugh and held him until police arrived. ries was pronounced dead at the scene. pugh is scheduled to appear in court on monday.

What if we get rid of the speaker notation of "reporter:"? This yields the following prompt:

Summarize in one paragraph the following text:\n\n commercial pilots radioing in. we got that balloon in sight also. looks like it\'s way up there, maybe 50,000 feet or so. the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana. this thing is up in the sky. what the heck is that? that thing is not the moon. any help would be appreciated. montana republican congressman ryan zinke was blunt, tweeting \\"shoot. it. down.\\" the pentagon today sayig that option was considered and rejected, for now. we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options. the white house today saying president biden agreed with the pentagon\'s strong recommendation. the risks involved with shooting down the balloon just too great. any potential debris field would be significant and potentially cause civilian injuries or deaths or significant property damage. for its part, china insisting today the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. the pentagon flatly rejecting that claim. we are aware of the prc\'s statement. however, the fact is, we know that it\'s a surveillance pballo. we have clearly communicated that this balloon is violating u.s. air space and international law and that this is unacceptable. china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it\'s likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing. windchill ever recorded in the u.s., including alaska. snow squalls along with that arctic chill. a time lapse from vermont. a cold emergency in place in boston. this is through sunday now. rob marciano standing by to time this out, and abc\'s victor oquendo in boston where the wind chill is 13 below zero right now. tonight, millions of families are in the midst of what the national weather service is calling a \\"once in a generation\\" arctic blast. we\'re seeing wind chills right now which are already below zero and before the night\'s over with, these are going to drop even more. parts of new england expecting wind chills at or near 60 below zero. outside springfield, massachusetts, as winds gusted above 50 miles per hour, a tree crashing onto a car, killing a 6-month-old girl inside. the woman driving rushed to the hospital. in boston, pilots battling high winds as they landed at logan airport. we\'re going to load food and stuff in here. and outreach teams hitting the streets to care for the needy. if anyone is out and sees someone else who might be in distress, please call 911. doctors say hypothermia can set in in just minutes. this weather can be deadly. it\'s life threatening. this is something that people have to really take seriously. the arctic front visible in the form of powerful snow squalls pushing from upstate new york through new england. here in boston, a brutal night ahead. a cold emergency is in place. winds will gust to 40 miles per hour, so there\'s a real concern about power outages, and it will feel like at least 30 below. david? victor oquendo in boston. victor, thank you. let\'s get to rob marciano who\'s timing this out for us tonight. hey, rob. hi, david. if you live in the northeast, you\'ve felt it all day long. temperatures have been dropping. now these winds are really ripping, measured over 40 miles per hour already. and the wind chill advisory is pretty much everywhere north of new york city. here you see it on the map. it\'s no picnic south of here either. minus 23 for a wind chill right now in albany. minus 13 in boston. but numbers in the morning will be even more dangerous, and that red spot blizzard warning, that\'s for blowing snow on the ground. minus 57 in caribou. we get to that figure or less, that\'s an all-time record. minus 11 in new york. this is hypothermic stuff, frostbite in minutes. but a rapid rebound in temperatures coming during the day sunday. we\'ll start in single digits in many spots all the way into the 40s for sunday, monday, and tuesday. so it\'s just a matter of survival for the next 36 hours. david? whiplash for sure, but take saturday seriously. rob, we thank you. get back inside. we\'re going to turn next to the doctor struck by a car. police say he was then attacked. he did not survive. that emergency room doctor riding his bike on the pacific coast highway, struck by a driver. police say the driver then got out of the car and stabbed the doctor. bystanders taking the attacker down. here\'s our chief correspondent matt gutman from california tonight. tonight, an e.r. doctor is dead after police in california say a motorist slammed into him, then got out of his car and stabbed him.

Unfortunately, we still have hallucination:

the attack happened on the pacific coast highway in the city of monterey. the doctor was riding his bike when he was hit by a car. police say the driver got out of his car and stabbed the doctor multiple times. good evening, everyone. this is a developing story. we're learning new details about a deadly road rage attack in california. a 32-year-old emergency room doctor is dead after police say he was hit by a car and then stabbed multiple times. it happened on the pacific coast highway in monterey. the doctor was riding his bike when he was hit by a car. police say the driver got out of his car and stabbed the doctor multiple times. the doctor was taken to a hospital where he died. police have arrested a suspect in the case. they say the suspect is a 32-year-old man from monterey. he's being held on suspicion of murder. we'll continue to follow this story and bring you updates as they become available.

Same prompt with temperature of 0.2. Notably we still see some hallucination (Laguna Beach), but no longer the hallucination of person names:

Temperature of 0.4. Note the alternation of what details it emphasizes:

Let's try correcting its punctuation and capitalization:

Correct the capitalization and punctuation of the following text: commercial pilots radioing in. we got that balloon in sight also. looks like it\'s way up there, maybe 50,000 feet or so. the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana. this thing is up in the sky. what the heck is that? that thing is not the moon. any help would be appreciated. montana republican congressman ryan zinke was blunt, tweeting \\"shoot. it. down.\\" the pentagon today sayig that option was considered and rejected, for now. we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options. the white house today saying president biden agreed with the pentagon\'s strong recommendation. the risks involved with shooting down the balloon just too great. any potential debris field would be significant and potentially cause civilian injuries or deaths or significant property damage. for its part, china insisting today the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. the pentagon flatly rejecting that claim. we are aware of the prc\'s statement. however, the fact is, we know that it\'s a surveillance pballo. we have clearly communicated that this balloon is violating u.s. air space and international law and that this is unacceptable. china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it\'s likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing. windchill ever recorded in the u.s., including alaska. snow squalls along with that arctic chill. a time lapse from vermont. a cold emergency in place in boston. this is through sunday now. rob marciano standing by to time this out, and abc\'s victor oquendo in boston where the wind chill is 13 below zero right now. tonight, millions of families are in the midst of what the national weather service is calling a \\"once in a generation\\" arctic blast. we\'re seeing wind chills right now which are already below zero and before the night\'s over with, these are going to drop even more. parts of new england expecting wind chills at or near 60 below zero. outside springfield, massachusetts, as winds gusted above 50 miles per hour, a tree crashing onto a car, killing a 6-month-old girl inside. the woman driving rushed to the hospital. in boston, pilots battling high winds as they landed at logan airport. we\'re going to load food and stuff in here. and outreach teams hitting the streets to care for the needy. if anyone is out and sees someone else who might be in distress, please call 911. doctors say hypothermia can set in in just minutes. this weather can be deadly. it\'s life threatening. this is something that people have to really take seriously. the arctic front visible in the form of powerful snow squalls pushing from upstate new york through new england. here in boston, a brutal night ahead. a cold emergency is in place. winds will gust to 40 miles per hour, so there\'s a real concern about power outages, and it will feel like at least 30 below. david? victor oquendo in boston. victor, thank you. let\'s get to rob marciano who\'s timing this out for us tonight. hey, rob. hi, david. if you live in the northeast, you\'ve felt it all day long. temperatures have been dropping. now these winds are really ripping, measured over 40 miles per hour already. and the wind chill advisory is pretty much everywhere north of new york city. here you see it on the map. it\'s no picnic south of here either. minus 23 for a wind chill right now in albany. minus 13 in boston. but numbers in the morning will be even more dangerous, and that red spot blizzard warning, that\'s for blowing snow on the ground. minus 57 in caribou. we get to that figure or less, that\'s an all-time record. minus 11 in new york. this is hypothermic stuff, frostbite in minutes. but a rapid rebound in temperatures coming during the day sunday. we\'ll start in single digits in many spots all the way into the 40s for sunday, monday, and tuesday. so it\'s just a matter of survival for the next 36 hours. david? whiplash for sure, but take saturday seriously. rob, we thank you. get back inside. we\'re going to turn next to the doctor struck by a car. police say he was then attacked. he did not survive. that emergency room doctor riding his bike on the pacific coast highway, struck by a driver. police say the driver then got out of the car and stabbed the doctor. bystanders taking the attacker down. here\'s our chief correspondent matt gutman from california tonight. tonight, an e.r. doctor is dead after police in california say a motorist slammed into him, then got out of his car and stabbed him.

Unfortunately, this triggers a guardrail false positive and fails. This is one of the more frustrating aspects of current LLM guardrails: their extreme unpredictability and false positive rate, where one prompt triggers a guardrail and another doesn't:

{
  "predictions": [
    {
      "safetyAttributes": {
        "blocked": true
      },
      "content": ""
    }
  ],
  "metadata": {
    "tokenMetadata": {
      "inputTokenCount": {
        "totalTokens": 1223,
        "totalBillableCharacters": 4319
      },
      "outputTokenCount": {
        "totalBillableCharacters": 0,
        "totalTokens": 1024
      }
    }
  }
}

Here we've gone ahead and used an external tool to correct the text, so let's try our summarization again.

The corrected text:

Summarize in one paragraph the following text: Commercial pilots radioing in. We got that balloon in sight also. Looks like it\'s way up there, maybe 50,000 feet or so. The balloon is huge, with a technology bay attached below that is itself the size of three buses, loaded with high-resolution cameras, according to a senior U.S. official. Equipped with what appear to be solar panels on the side that could power its technology. It first entered American airspace over Alaska, then flew into southwest Canada before dipping down over Billings, Montana. This thing is up in the sky. What the heck is that? That thing is not the moon. Any help would be appreciated. Montana Republican Congressman Ryan Zinke was blunt, tweeting, \\"Shoot it down.\\" The Pentagon today saying that option was considered and rejected, for now. We assessed that currently it does not pose a physical or military risk to people on the ground. For now, we are continuing to monitor and review options. The White House today said President Biden agreed with the Pentagon\'s strong recommendation. The risks involved with shooting down the balloon are just too great. Any potential debris field would be significant and could potentially cause civilian injuries, deaths, or significant property damage. For its part, China is insisting today that the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. The Pentagon is flatly rejecting that claim. We are aware of the PRC\'s statement; however, the fact is, we know that it\'s a surveillance balloon. We have clearly communicated that this balloon is violating U.S. airspace and international law, and that this is unacceptable. China says the balloon deviated far from its planned course, but the Pentagon is questioning that as well, saying the balloon can be maneuvered. A senior U.S. official told ABC News that it\'s likely being navigated with spy satellites. This is all happening as Secretary of State Antony Blinken prepares for a weekend trip to Beijing. The coldest wind chill ever recorded in the U.S., including Alaska, with snow squalls along with that Arctic chill. A time-lapse from Vermont, with a cold emergency in place in Boston. This is through Sunday now. Rob Marciano is standing by to time this out, and ABC\'s Victor Oquendo is in Boston where the wind chill is 13 below zero right now. Tonight, millions of families are in the midst of what the National Weather Service is calling a \\"once-in-a-generation\\" Arctic blast. We\'re seeing wind chills right now that are already below zero, and before the night is over, these will drop even more. Parts of New England are expecting wind chills at or near 60 below zero. Outside Springfield, Massachusetts, as winds gust above 50 miles per hour, a tree crashed onto a car, killing a 6-month-old girl inside. The woman driving was rushed to the hospital. In Boston, pilots are battling high winds as they land at Logan Airport. We\'re going to load food and supplies in here. Outreach teams are hitting the streets to care for the needy. If anyone is out and sees someone else who might be in distress, please call 911. Doctors say hypothermia can set in in just minutes. This weather can be deadly. It\'s life-threatening. This is something that people have to really take seriously. The Arctic front is visible in the form of powerful snow squalls pushing from upstate New York through New England. Here in Boston, a brutal night is ahead. A cold emergency is in place. Winds will gust to 40 miles per hour, so there\'s a real concern about power outages, and it will feel like at least 30 below. David? Victor Oquendo in Boston. Victor, thank you. Let\'s get to Rob Marciano, who\'s timing this out for us tonight. Hey, Rob. Hi, David. If you live in the Northeast, you\'ve felt it all day long. Temperatures have been dropping. Now these winds are really ripping, measured over 40 miles per hour already. The wind chill advisory is pretty much everywhere north of New York City. Here you see it on the map. It\'s no picnic south of here either. Minus 23 for a wind chill right now in Albany. Minus 13 in Boston. But numbers in the morning will be even more dangerous, and that red spot is a blizzard warning for blowing snow on the ground. Minus 57 in Caribou. If we get to that figure or less, that\'s an all-time record. Minus 11 in New York. This is hypothermic stuff, frostbite in minutes. But a rapid rebound in temperatures is coming during the day Sunday. We\'ll start in single digits in many spots, all the way into the 40s for Sunday, Monday, and Tuesday. So it\'s just a matter of survival for the next 36 hours. David? Whiplash for sure, but take Saturday seriously. Rob, we thank you. Get back inside. Let\'s turn next to the doctor struck by a car. Police say he was then attacked and did not survive. That emergency room doctor was riding his bike on the Pacific Coast Highway when he was struck by a driver. Police say the driver then got out of the car and stabbed the doctor. Bystanders took the attacker down. Here\'s our chief correspondent, Matt Gutman, from California tonight. Tonight, an ER doctor is dead after police in California say a motorist slammed into him, then got out of his car and stabbed him.

And the result with a temperature of 0.0. Less hallucination than before, but still some (Torrance) and it still fixates:

The attack happened on the Pacific Coast Highway in Torrance. The doctor was riding his bike when he was hit by a car. Police say the driver got out of his car and stabbed the doctor multiple times. Bystanders intervened and held the driver down until police arrived. The doctor was taken to a hospital, where he died. The driver is in custody. Police have not released a motive for the attack.

What about a temperature of 0.2? Hallucination of the city and hallucinated details like the repeated stabbing. Again, fixated on the doctor case:

Temperature of 0.4? Interestingly, correcting the capitalization and punctuation of the text appears to have stopped the hallucination of the victim and suspect names and the city hallucination is consistent now:

Clearly, the LLM is extremely sensitive to the capitalization and punctuation: an issue we've observed with other LLMs we've tested.

Now the question is whether it was just the formatting correction that reduced the name hallucination or whether removing the speaker annotations in the transcript also played a role. Let's go back to the original version and correct it in situ without removing the "Reporter:" entries (though we'll remove the ">>" entries).

Thus our prompt becomes:

Summarize in one paragraph the following text: Commercial pilots radioing in: We got that balloon in sight also. Looks like it\'s way up there, maybe 50,000 feet or so. Reporter: The balloon is huge, with a technology bay attached below that is itself the size of three buses, loaded with high-resolution cameras, according to a senior U.S. official. Equipped with what appear to be solar panels on the side that could power its technology. It first entered American airspace over Alaska, then flew into southwest Canada before dipping down over Billings, Montana. This thing is up in the sky. What the heck is that? That thing is not the moon. Any help would be appreciated. Reporter: Montana Republican Congressman Ryan Zinke was blunt, tweeting, \\"Shoot it down.\\" The Pentagon today saying that option was considered and rejected, for now. We assessed that currently it does not pose a physical or military risk to people on the ground. For now, we are continuing to monitor and review options. Reporter: The White House today said President Biden agreed with the Pentagon\'s strong recommendation. The risks involved with shooting down the balloon are just too great. Any potential debris field would be significant and could potentially cause civilian injuries, deaths, or significant property damage. Reporter: For its part, China is insisting today that the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. The Pentagon is flatly rejecting that claim. We are aware of the PRC\'s statement. However, the fact is, we know that it\'s a surveillance balloon. We have clearly communicated that this balloon is violating U.S. airspace and international law, and that this is unacceptable. Reporter: China says the balloon deviated far from its planned course, but the Pentagon is questioning that as well, saying the balloon can be maneuvered. A senior U.S. official told ABC News that it\'s likely being navigated with spy satellites. This is all happening as Secretary of State Antony Blinken prepares for a weekend trip to Beijing. The coldest wind chill ever recorded in the U.S., including Alaska, with snow squalls along with that Arctic chill. A time-lapse from Vermont, with a cold emergency in place in Boston. This is through Sunday now. Rob Marciano is standing by to time this out, and ABC\'s Victor Oquendo is in Boston where the wind chill is 13 below zero right now. Reporter: Tonight, millions of families are in the midst of what the National Weather Service is calling a \\"once-in-a-generation\\" Arctic blast. \'We\'re seeing wind chills right now which are already below zero and before the night\'s over with, these are going to drop even more. Reporter: Parts of New England expecting wind chills at or near 60 below zero. Outside Springfield, Massachusetts, as winds gusted above 50 miles per hour, a tree crashed onto a car, killing a 6-month-old girl inside. The woman driving was rushed to the hospital. In Boston, pilots are battling high winds as they land at Logan Airport. We\'re going to load food and supplies in here. Reporter: Outreach teams are hitting the streets to care for the needy. If anyone is out and sees someone else who might be in distress, please call 911. Reporter: Doctors say hypothermia can set in in just minutes. This weather can be deadly. It\'s life-threatening. This is something that people have to really take seriously. Reporter: The Arctic front is visible in the form of powerful snow squalls pushing from upstate New York through New England. Here in Boston, a brutal night is ahead. A cold emergency is in place. Winds will gust to 40 miles per hour, so there\'s a real concern about power outages, and it will feel like at least 30 below. David? Reporter: Victor Oquendo in Boston. Victor, thank you. Let\'s get to Rob Marciano, who\'s timing this out for us tonight. Hey, Rob. Reporter: Hi, David. If you live in the Northeast, you\'ve felt it all day long. Temperatures have been dropping. Now these winds are really ripping, measured over 40 miles per hour already. The wind chill advisory is pretty much everywhere north of New York City. Here you see it on the map. It\'s no picnic south of here either. Minus 23 for a wind chill right now in Albany. Minus 13 in Boston. But numbers in the morning will be even more dangerous, and that red spot is a blizzard warning for blowing snow on the ground. Minus 57 in Caribou. If we get to that figure or less, that\'s an all-time record. Minus 11 in New York. This is hypothermic stuff, frostbite in minutes. But a rapid rebound in temperatures is coming during the day Sunday. We\'ll start in single digits in many spots, all the way into the 40s for Sunday, Monday, and Tuesday. So it\'s just a matter of survival for the next 36 hours. David? Reporter: Whiplash for sure, but take Saturday seriously. Rob, we thank you. Get back inside. Reporter: We\'re going to turn next to the doctor struck by a car. Police say he was then attacked. He did not survive. That emergency room doctor was riding his bike on the Pacific Coast Highway when he was struck by a driver. Police say the driver then got out of the car and stabbed the doctor. Bystanders took the attacker down. Here\'s our chief correspondent, Matt Gutman, from California tonight. Reporter: Tonight, an ER doctor is dead after police in California say a motorist slammed into him, then got out of his car and stabbed him.

Let's try a temperature of 0.0. This hits a guardrail false positive again. It looks like the presence of "Reporter:" is what's triggering the guardrail.

{
  "predictions": [
    {
      "safetyAttributes": {
        "blocked": true
      },
      "content": ""
    }
  ],
  "metadata": {
    "tokenMetadata": {
      "inputTokenCount": {
        "totalTokens": 1218,
        "totalBillableCharacters": 4503
      },
      "outputTokenCount": {
        "totalTokens": 78,
        "totalBillableCharacters": 0
      }
    }
  }
}

Let's go back to our original five-minute transcript that we used in our prior episodes of this series. We'll keep the punctuation and capitalization as-is:

Summarize in one paragraph the following text: tonight, several developing stories as we come on the air. news coming in, the suspected chinese spy balloon hovering over the u.s. where it is now, and will there be a window to shoot it down? also tonight, the dangerous life-threatening cold moving into the northeast. the live readings already. first, the pentagon tracking that chinese balloon, the intelligence bay hanging beneath it, the size of three busses. where it was spotted today over the u.s., where it\'s believed to be headed now, 60,000 feet in the air. will the u.s. shoot this down? secretary of state antony blinken postponing his high stakes trip to china. mola lenghi, mary bruce standing by at the white house. the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out. the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers. the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down. the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh\'s housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper\'s fall. but will the jury hear about it? eva pilgrim reporting. robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you\'ll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear. good evening. it is great to have you with us on a friday night. almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. the readings coming in right now, and where the wind chill is already more than 100 degrees below zero. that\'s in a moment here. but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites. today the balloon was spotted over missouri. so where is it tonight, and where u.s. authorities believe it\'s headed. among the first sightings over billings, montana. that intelligence bay hanging beneath the balloon, that\'s the size of three busses. the pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? what we\'re learning tonight. secretary of state antony blinken postponing his high-stakes trip to beijing, calling china\'s actions unacceptable. so, will this be shot down? if so, where? and what china is saying tonight. mary bruce at the white house, and abc\'s mola lenghi leading us off tonight in montana. reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east. the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that? reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri. commercial pilots radioing in. we got that balloon in sight also. looks like it\'s way up there, maybe 50,000 feet or so. reporter: the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana. this thing is up in the sky. what the heck is that? that thing is not the moon. any help would be appreciated. reporter: montana republican congressman ryan zinke was blunt, tweeting \\"shoot. it. down.\\" the pentagon today sayig that option was considered and rejected, for now. we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options. reporter: the white house today saying president biden agreed with the pentagon\'s strong recommendation. the risks involved with shooting down the balloon just too great. any potential debris field would be significant and potentially cause civilian injuries or deaths or significant property damage. reporter: for its part, china insisting today the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. the pentagon flatly rejecting that claim. we are aware of the prc\'s statement. however, the fact is, we know that it\'s a surveillance pballo. we have clearly communicated that this balloon is violating u.s. air space and international law and that this is unacceptable. reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it\'s likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing.

Temperature of 0.0. Note the hallucination that it is Ukraine that caused Blinken's trip to China to be canceled:

but that trip is now on hold. the white house saying secretary blinken will postpone his trip to china and will instead travel to japan and south korea to discuss the situation in ukraine. A large Chinese spy balloon is hovering over the United States. It was first spotted in Alaska and then flew into Canada before entering Montana. The balloon is equipped with high-resolution cameras and solar panels. The Pentagon is considering shooting it down, but the White House has said that the risks involved are too great. China insists that the balloon is a civilian airship used for research, but the Pentagon says that it is a surveillance balloon. Secretary of State Antony Blinken has postponed his trip to Beijing to discuss the situation in Ukraine.

Temperature of 0.2. Note the hallucination about Ukraine again and the addition of Japan and South Korea: