Experiments In Meme Tracking: Dividing Transcripts Into Stories – Part 1

Kalev Leetaru

1 year ago

Meme and story tracking involves identifying the core details of a specific story and tracking how it is covered over time and space, as it is rereported, contextualized and updated over time. For small discrete memes such as a memorable and unique quote, tracking can be fairly straightforward. But what happens when the source input is a lengthy transcript or article that contains many different unrelated stories? The first step is to annotate that text into the underlying discrete stories that comprise it. Here we will explore "dynamic semantic windowing" as an annotation strategy.

Despite the popular conception of LLMs as human-like intelligent agents that "allow anyone to program using plain English", the results below show that these claims could not be further from the truth. While LLMs continue to improve, current SOTA models are extremely brittle, with the addition of a single space or a period becoming a colon meaning the difference between reasonable output and a null response. Worse, behaviors are extremely unpredictable, with two different passages below triggering hallucination, while a third incorrectly triggers a safety guardrail that blocks it from returning output. Others return just the literal number "2" or "3" rather than the selected passage. Current SOTA models are found to be extraordinarily sensitive to prompt/input combinations, meaning modern LLM usage is not a simple "fire and forget" approach of crafting a singular prompt and leaving the LLM to its work.

Television news offers an ideal testbed through which to test story tracking in that television closed captioning represents essentially a 24/7 perpetual stream-of-consciousness flow of words that frequently switches back-and-forth amongst multiple stories at a time without explicit boundaries delineating them. Last month we showed how large-token LLMs (in this case Anthropic's 100K token Claude 2 model) can be used to segment television news transcripts into their underlying stories and summarize them. For a short 30-minute evening news broadcast that was just over the limit of traditional LLMs, the results were extremely good and matched expert human annotation. In the case of a longer news broadcast that was well below the model's limits, but also much longer than that processable through ordinary LLMs, the results were far more mixed. This approach also requires the transcript to be distilled down to a summary of each extracted story that necessarily removes much of its underlying detail, making evolution tracking (how a story's portrayal and contextualization changes over time) all but impossible.

How might an alternative approach of asking an LLM to annotate the transcript in situ help? Let's explore this same ABC Evening News broadcast that we've used in our past experiments.

We'll start by examining the first five minutes of the broadcast:

tonight, several developing stories as we come on the air. news coming in, the suspected chinese spy balloon hovering over the u.s. where it is now, and will there be a window to shoot it down? also tonight, the dangerous life-threatening cold moving into the northeast. the live readings already. first, the pentagon tracking that chinese balloon, the intelligence bay hanging beneath it, the size of three busses. where it was spotted today over the u.s., where it's believed to be headed now, 60,000 feet in the air. will the u.s. shoot this down? secretary of state antony blinken postponing his high stakes trip to china. mola lenghi, mary bruce standing by at the white house. the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out. the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers. the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down. the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh's housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper's fall. but will the jury hear about it? eva pilgrim reporting. robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you'll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear. good evening. it is great to have you with us on a friday night. almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. the readings coming in right now, and where the wind chill is already more than 100 degrees below zero. that's in a moment here. but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites. today the balloon was spotted over missouri. so where is it tonight, and where u.s. authorities believe it's headed. among the first sightings over billings, montana. that intelligence bay hanging beneath the balloon, that's the size of three busses. the pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? what we're learning tonight. secretary of state antony blinken postponing his high-stakes trip to beijing, calling china's actions unacceptable. so, will this be shot down? if so, where? and what china is saying tonight. mary bruce at the white house, and abc's mola lenghi leading us off tonight in montana. reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east. the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that? reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri. commercial pilots radioing in. we got that balloon in sight also. looks like it's way up there, maybe 50,000 feet or so. reporter: the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana. this thing is up in the sky. what the heck is that? that thing is not the moon. any help would be appreciated. reporter: montana republican congressman ryan zinke was blunt, tweeting "shoot. it. down." the pentagon today sayig that option was considered and rejected, for now. we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options. reporter: the white house today saying president biden agreed with the pentagon's strong recommendation. the risks involved with shooting down the balloon just too great. any potential debris field would be significant and potentially cause civilian injuries or deaths or significant property damage. reporter: for its part, china insisting today the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. the pentagon flatly rejecting that claim. we are aware of the prc's statement. however, the fact is, we know that it's a surveillance pballo. we have clearly communicated that this balloon is violating u.s. air space and international law and that this is unacceptable. reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it's likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing.

Let's first ask for a summarization of that passage with a temperature of 0.0 to ensure deterministic results:

time curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/[YOURPROJECTID]/locations/us-central1/publishers/google/models/text-bison:predict -d \
$'{
"instances": [
{ "prompt": "Provide a summary of one paragraph for the following article: tonight, several developing stories as we come on the air. news coming in, the suspected chinese spy balloon hovering over the u.s. where it is now, and will there be a window to shoot it down? also tonight, the dangerous life-threatening cold moving into the northeast. the live readings already. first, the pentagon tracking that chinese balloon, the intelligence bay hanging beneath it, the size of three busses. where it was spotted today over the u.s., where it\'s believed to be headed now, 60,000 feet in the air. will the u.s. shoot this down? secretary of state antony blinken postponing his high stakes trip to china. mola lenghi, mary bruce standing by at the white house. the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out. the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers. the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down. the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh\'s housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper\'s fall. but will the jury hear about it? eva pilgrim reporting. robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you\'ll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear. good evening. it is great to have you with us on a friday night. almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. the readings coming in right now, and where the wind chill is already more than 100 degrees below zero. that\'s in a moment here. but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites. today the balloon was spotted over missouri. so where is it tonight, and where u.s. authorities believe it\'s headed. among the first sightings over billings, montana. that intelligence bay hanging beneath the balloon, that\'s the size of three busses. the pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? what we\'re learning tonight. secretary of state antony blinken postponing his high-stakes trip to beijing, calling china\'s actions unacceptable. so, will this be shot down? if so, where? and what china is saying tonight. mary bruce at the white house, and abc\'s mola lenghi leading us off tonight in montana. reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east. the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that? reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri. commercial pilots radioing in. we got that balloon in sight also. looks like it\'s way up there, maybe 50,000 feet or so. reporter: the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana. this thing is up in the sky. what the heck is that? that thing is not the moon. any help would be appreciated. reporter: montana republican congressman ryan zinke was blunt, tweeting \\"shoot. it. down.\\" the pentagon today sayig that option was considered and rejected, for now. we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options. reporter: the white house today saying president biden agreed with the pentagon\'s strong recommendation. the risks involved with shooting down the balloon just too great. any potential debris field would be significant and potentially cause civilian injuries or deaths or significant property damage. reporter: for its part, china insisting today the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. the pentagon flatly rejecting that claim. we are aware of the prc\'s statement. however, the fact is, we know that it\'s a surveillance pballo. we have clearly communicated that this balloon is violating u.s. air space and international law and that this is unacceptable. reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it\'s likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing. "}
],
"parameters": {
"temperature": 0.0,
"maxOutputTokens": 1024,
"topK": 40,
"topP": 0.95,
}
}' > O; cat O | jq -r .predictions[0].content

The output fixates exclusively on the spy balloon, providing considerable detail about it, but excluding all of the other stories in the passage:

A large Chinese spy balloon was spotted in the US. It is 60,000 feet above the ground and heading east. It was first spotted in Alaska, then flew into southwest Canada before dipping down over Billings, Montana. The balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras. The Pentagon considered shooting it down but decided not to because of the risks involved. China insists that the balloon is not a spy vessel but a civilian airship used for research, mainly meteorological purposes. The Pentagon flatly rejected that claim.

What if we increase the temperature to 0.2 and run three times? All three runs again fixate only on the balloon:

The Chinese government has released a statement claiming that the balloon is a weather balloon that has drifted off course. However, the US government believes that the balloon is a surveillance balloon being maneuvered by China. The balloon is currently over the United States and is heading east. The US government is considering shooting down the balloon, but they are concerned about the potential debris field that could be created.
The US is tracking a Chinese spy balloon that entered US airspace over Alaska and is now heading east. The balloon is 60,000 feet above the ground and is equipped with high-resolution cameras and solar panels. The Pentagon has considered shooting down the balloon, but has decided against it for now because of the risk of debris. China insists that the balloon is a civilian airship used for research, but the US says it is a surveillance balloon.
A large Chinese spy balloon was spotted in the US airspace. It is 60,000 feet above the ground and heading east. It was first spotted in Alaska, then flew into southwest Canada before dipping down over Billings, Montana. The US government is monitoring the balloon and has considered shooting it down, but decided not to do so because of the risk of debris. China is claiming that the balloon is a civilian airship used for research, but the US government is saying that it is a surveillance balloon.

Let's increase to 0.4. Again, all three runs exclusively focus on the balloon:

A large Chinese spy balloon was spotted in the United States. It was first seen in Alaska and then flew over Canada before entering Montana. The balloon is 60,000 feet above the ground and is heading east. The Pentagon is monitoring the balloon and has considered shooting it down, but has decided not to do so for now because of the risk of debris. China claims that the balloon is a civilian airship used for research, but the Pentagon says that it is a surveillance balloon. Secretary of State Antony Blinken has postponed his trip to Beijing because of the balloon.
A large Chinese spy balloon is hovering over the United States. The balloon was first spotted over Alaska and then flew into southwest Canada before dipping down over Billings, Montana. The balloon is equipped with high-resolution cameras and solar panels, and it is believed to be being maneuvered by China using spy satellites. The Pentagon has considered shooting down the balloon, but has decided against it for now because of the risks involved. The White House has also said that President Biden agrees with the Pentagon's decision. China, for its part, insists that the balloon is not a spy vessel but a civilian airship used for research purposes. The Pentagon flatly rejects this claim. This incident is happening as Secretary of State Antony Blinken prepares for a weekend trip to Beijing.
A large Chinese spy balloon was spotted in the US airspace. It is believed to be a surveillance balloon being maneuvered by China, likely using spy satellites. The balloon was first spotted over Montana and is now heading towards North Carolina. The Pentagon has considered shooting it down but decided against it due to the risks involved. China has denied that the balloon is a spy vessel and insists that it is a civilian airship used for research.

What about dialing up to the maximum 0.99 temperature? Remarkably, even with the maximum possible temperature, all three summaries fixate exclusively on the balloon:

A large Chinese spy balloon is hovering over the United States. The balloon was first spotted in Alaska and then flew into Canada before entering American airspace. The balloon is equipped with high-resolution cameras and solar panels. The Pentagon is monitoring the balloon and has considered shooting it down, but has decided not to do so for now because of the risks involved. China is claiming that the balloon is a civilian airship used for research, but the Pentagon says that it is a surveillance balloon. Secretary of State Antony Blinken has postponed his trip to Beijing in response to the balloon incident.
The US is tracking a Chinese spy balloon that is hovering over the US. The balloon is 60,000 feet above the ground and is heading east. It is believed to be headed towards North Carolina. The balloon is equipped with high-resolution cameras and solar panels. The Pentagon has considered shooting down the balloon, but has decided not to do so for now because of the risks involved. China has denied that the balloon is a spy vessel and has said that it is a civilian airship used for research.
The US is monitoring a Chinese spy balloon that is hovering over the US. The balloon is 60,000 feet above the ground and is heading east. It is believed to be headed towards North Carolina. The balloon was first spotted in Alaska and then flew into southwest Canada before dipping down over Billings, Montana. The Pentagon has considered shooting down the balloon, but has decided against it for now because of the risks involved. China insists that the balloon is not a spy vessel, but a civilian airship used for research. The Pentagon disagrees, and says that the balloon is violating US airspace and international law.

What if we change our instruction to just "Provide a summary for the following article:" and use a temperature of 0.0? Strangely we get only the text below:

the united states is aware of the prc's statement. however, the fact is, we know that it's a surveillance balloon. we have clearly communicated that this balloon is violating u.s. airspace and international law and that this is unacceptable.

What about "Provide a summary for the following article:" and use a temperature of 0.2? Again the summary fixates on the balloon. Oddly, lower-case text appears in the summaries below, which is extremely uncharacteristic of LLMs:

secretary blinken has postponed his high stakes trip to china, citing the chinese government's continuing military aggression in the region. reporter: blinken was set to meet with his counterpart, foreign minister wang yi, in shanghai on saturday. the meeting was to be the highest level u.s.-china contact since president biden took office in january. There is a Chinese spy balloon hovering over the US. It is 60,000 feet above the ground and heading east. It was first spotted over Alaska, then flew into southwest Canada before dipping down over Billings, Montana. The balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, equipped with what appear to be solar panels on the side that could power its technology. The Pentagon is monitoring the balloon and has considered shooting it down, but the risks involved with shooting it down are too great. China is insisting that the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. The Pentagon flatly rejects that claim.
we have raised this issue with the chinese. we have made very clear that this is unacceptable. reporter: but today, blinken postponed that trip, citing china's actions as concerning. we have a range of concerns that we've been raising with the chinese, including the behavior that we've seen with regard to this particular situation. reporter: the pentagon says the balloon is now over the central part of the continental united states and is heading east. it is expected to remain in the air for several more days.
we have raised this issue with the chinese. we will continue to do so at senior levels. reporter: but today, blinken postponed that trip, calling china's actions unacceptable. we have made clear to the chinese that we consider this to be a serious matter and that we will continue to pursue this with them. reporter: the pentagon says it will continue to track the balloon and will shoot it down if it poses a threat to the u.s.

Even a temperature of 0.99 yields lowercase text and a balloon fixation:

we have raised this directly with the chinese. we have made very clear that this is unacceptable. reporter: but today, blinken postponed that trip, citing china's actions. we will not proceed with the meeting at this time. reporter: blinken says he will reschedule the trip once the chinese government makes clear that it's committed to a productive and substantive dialogue. The article is about a Chinese spy balloon that is hovering over the U.S. The balloon was first spotted in Alaska and then flew into southwest Canada before dipping down over Billings, Montana. The balloon is huge, with a technology bay attached below that is itself the size of three busses. It is loaded with high resolution cameras and equipped with what appear to be solar panels on the side that could power its technology. The U.S. government believes that the balloon is a surveillance balloon being maneuvered by China, likely using spy satellites. The Pentagon is considering shooting down the balloon, but the White House has said that the risks involved with shooting down the balloon are too great. China insists that the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. The Pentagon flatly rejects that claim. Secretary of State Antony Blinken has postponed his trip to Beijing in light of China's actions.

Let's change our prompt to "Provide a summary for the following article that breaks it into distinct stories" and a temperature of 0.0:

we have raised this issue with the chinese. we will continue to do so at senior levels. reporter: but today, blinken postponed that trip, calling china's actions unacceptable. we have made clear to the chinese that we consider this to be a serious matter. reporter: the pentagon says the balloon could remain in u.s. airspace for days, even weeks.

What about "Provide a summary for the following article that breaks it into distinct stories" and a temperature of 0.2? Suddenly we're getting somewhere! Though the opening line is a strange addition:

we have raised this issue with the chinese. we will continue to do so at all levels. reporter: but today, blinken postponed that trip, calling china's actions unacceptable. we have made clear to the chinese that we consider this to be a serious matter. reporter: the pentagon says the balloon could remain in the air for weeks, and it's monitoring its movements closely. 1. The suspected Chinese spy balloon hovering over the U.S.
- The balloon is 60,000 feet above the ground and heading east.
- It is currently over the center of the continental United States.
- The balloon is believed to be headed towards North Carolina.
- The balloon was first spotted over Alaska, then flew into southwest Canada before dipping down over Billings, Montana.
- The balloon is huge with a technology bay attached below that is itself the size of three busses.
- The technology bay is loaded with high resolution cameras and equipped with what appear to be solar panels on the side that could power its technology.
- The Pentagon says that the balloon does not pose a physical or military risk to people on the ground.
- The White House says that the risks involved with shooting down the balloon are too great.
- China insists that the balloon is not a spy vessel but a civilian airship used for research, mainly meteorological purposes.
- The Pentagon flatly rejects that claim.
- The balloon could remain in the air for weeks, and the Pentagon is monitoring its movements closely.
2. The deadly cold already tonight
- The National Weather Service is calling the cold snap a once in a generation arctic blast.
- 25 million Americans in multiple states are affected.
- New York City, Philadelphia, and Boston are among the cities that will be hit hard by the cold.
- Wind-chill readings in Maine are expected to reach 60 below zero.
- In Massachusetts, the wind chill is already 106 degrees below zero.
3. The deadly assault on an ER doctor riding his mountain bike
- Police say the suspect in his car hit the victim, then allegedly got out and attacked him.
- The victim is in critical condition.
4. Another scare at the airport
- Two passenger jets clipped wings at Newark Airport.
- One plane was full of passengers.
5. The stunning and unexpected jobs report tonight
- President Biden is celebrating the numbers.
- The unemployment rate is not this low since 1969.
6. The Alex Murdaugh double murder trial
- The defense team acknowledged that a ballistics expert made a mistake.
- The story of Alex Murdaugh's housekeeper dying in a fall at the home.
- Prosecutors say Murdaugh got a $5 million payout from a policy that covered the housekeeper's fall.
7. Robin Roberts tonight and her epic jump
- Robin Roberts will jump from the tallest building in the southern hemisphere.
- She will share the lesson on conquering fear.

Running a second time includes that same line, but again does a decent job of segmenting the story:

we have made clear to the chinese that this is unacceptable. reporter: but that trip is now on hold. secretary blinken postponing his high-stakes visit to china, citing the balloon incident as a reason. we have made clear to the chinese that this is unacceptable. reporter: the pentagon says it will continue to monitor the balloon and is prepared to shoot it down if it poses a threat. * **Suspected Chinese Spy Balloon Hovering Over the U.S.**

The Pentagon is tracking a Chinese balloon that is hovering over the U.S. The balloon is equipped with a technology bay that is the size of three busses and is believed to be carrying high-resolution cameras. The balloon first entered American airspace over Alaska and then flew into southwest Canada before dipping down over Billings, Montana.

The Pentagon says that the balloon does not pose a physical or military risk to people on the ground, but it is continuing to monitor the situation and review options. Secretary of State Antony Blinken has postponed his high-stakes trip to Beijing, citing the balloon incident as a reason.

* **Dangerous, Life-Threatening Cold Moving Into the Northeast**

The National Weather Service is calling the cold weather moving into the Northeast a "once in a generation arctic blast." The cold front is expected to bring wind-chill readings of 60 below zero to Maine and 106 degrees below zero to Massachusetts.

The cold weather is expected to cause power outages, transportation disruptions, and hazardous driving conditions. People are advised to stay indoors and avoid unnecessary travel.

* **Deadly Assault on an ER Doctor Riding His Mountain Bike**

A 31-year-old ER doctor was killed in a hit-and-run while riding his mountain bike in Philadelphia. Police say the suspect in his car hit the victim, then allegedly got out and attacked him. The victim was pronounced dead at the scene.

* **Another Scare at the Airport**

Two passenger jets clipped wings at Newark Liberty International Airport on Friday. One of the planes was full of passengers. No injuries were reported.

* **Stunning and Unexpected Jobs Report**

President Biden is celebrating the news that the unemployment rate fell to 3.8% in January, the lowest level since 1969. The report also showed that the economy added 467,000 jobs last month.

* **Alex Murdaugh Double Murder Trial**

The defense team for Alex Murdaugh acknowledged in court on Friday that a ballistics expert found the same type of ammunition at the scene of the murders of Murdaugh's wife and son as was found at the scene of a fatal hit-and-run that Murdaugh is accused of.

* **Robin Roberts' Epic Jump**

Robin Roberts will jump from the tallest building in the Southern Hemisphere on Saturday. The jump is part of a fundraiser for the Robin Roberts Foundation.

And how about the same prompt with a temperature of 0.99? Strangely these results fixate on the balloon again:

 we have made clear to the prc that we consider this to be a serious matter and that we will continue to engage with them on this issue.   reporter: but that trip is now on hold, as blinken says he's postponing the visit until next month. * **China Spy Balloon:** The Pentagon is tracking a Chinese spy balloon that is hovering over the U.S. The balloon is 60,000 feet above the ground and is heading east. It was first spotted over Alaska, then flew into southwest Canada before dipping down over Billings, Montana. The balloon is huge, with a technology bay attached below that is itself the size of three busses. It is loaded with high resolution cameras and equipped with what appear to be solar panels on the side that could power its technology.
* **The Pentagon:** The Pentagon says that the balloon does not pose a physical or military risk to people on the ground. However, they are continuing to monitor and review options. The White House says that President Biden agreed with the Pentagon's strong recommendation. The risks involved with shooting down the balloon are just too great.
* **China:** China insists that the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. The Pentagon flatly rejects that claim.
* **Secretary of State Antony Blinken:** Secretary of State Antony Blinken has postponed his trip to Beijing. He says that he will continue to engage with China on this issue.

The encouraging results above suggest the LLM is able to sense the natural divisions of the text.

What if we ask it to literally annotate the text by inserting an XML tag in the text wherever the topical focus shifts? Thus our prompt becomes:

Break the text below into distinct stories by inserting the XML tag '<TOPICBREAK>' in between any pair of sentences that represent a shift from one story to another

And our actual command is:

time curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/[YOURPROJECTID]/locations/us-central1/publishers/google/models/text-bison:predict -d \
$'{
"instances": [
{ "prompt": "In the following text, insert the tag \'<TOPICBREAK>\' anywhere there is a shift from one story to another. Your goal is to divide the text into distinct stories, grouping sentences together that discuss a single story: tonight, several developing stories as we come on the air. news coming in, the suspected chinese spy balloon hovering over the u.s. where it is now, and will there be a window to shoot it down? also tonight, the dangerous life-threatening cold moving into the northeast. the live readings already. first, the pentagon tracking that chinese balloon, the intelligence bay hanging beneath it, the size of three busses. where it was spotted today over the u.s., where it\'s believed to be headed now, 60,000 feet in the air. will the u.s. shoot this down? secretary of state antony blinken postponing his high stakes trip to china. mola lenghi, mary bruce standing by at the white house. the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out. the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers. the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down. the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh\'s housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper\'s fall. but will the jury hear about it? eva pilgrim reporting. robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you\'ll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear. good evening. it is great to have you with us on a friday night. almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. the readings coming in right now, and where the wind chill is already more than 100 degrees below zero. that\'s in a moment here. but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites. today the balloon was spotted over missouri. so where is it tonight, and where u.s. authorities believe it\'s headed. among the first sightings over billings, montana. that intelligence bay hanging beneath the balloon, that\'s the size of three busses. the pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? what we\'re learning tonight. secretary of state antony blinken postponing his high-stakes trip to beijing, calling china\'s actions unacceptable. so, will this be shot down? if so, where? and what china is saying tonight. mary bruce at the white house, and abc\'s mola lenghi leading us off tonight in montana. reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east. the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that? reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri. commercial pilots radioing in. we got that balloon in sight also. looks like it\'s way up there, maybe 50,000 feet or so. reporter: the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana. this thing is up in the sky. what the heck is that? that thing is not the moon. any help would be appreciated. reporter: montana republican congressman ryan zinke was blunt, tweeting \\"shoot. it. down.\\" the pentagon today sayig that option was considered and rejected, for now. we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options. reporter: the white house today saying president biden agreed with the pentagon\'s strong recommendation. the risks involved with shooting down the balloon just too great. any potential debris field would be significant and potentially cause civilian injuries or deaths or significant property damage. reporter: for its part, china insisting today the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. the pentagon flatly rejecting that claim. we are aware of the prc\'s statement. however, the fact is, we know that it\'s a surveillance pballo. we have clearly communicated that this balloon is violating u.s. air space and international law and that this is unacceptable. reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it\'s likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing. "}
],
"parameters": {
"temperature": 0.0,
"maxOutputTokens": 1024,
"topK": 40,
"topP": 0.95,
}
}' > O; cat O | jq -r .predictions[0].content

This yields the following output (we added a carriage return before each tag to make it easier to read):

<TOPICBREAK>the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers.

<TOPICBREAK>the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down.

<TOPICBREAK>the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh's housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper's fall. but will the jury hear about it? eva pilgrim reporting.

<TOPICBREAK>robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you'll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear.

At first glance, the results look quite reasonable, with the initial fast-paced roundup of stories grouped together and each of the other major stories grouped on its own, though it does incorrectly group an ER doctor being attacked and two passenger jets clipping wings as the same story (likely due to both involving impact). However, notice how the transcript ends less than halfway through the original text.

Looking at our token metadata, we output only 531 tokens, so we are nowhere near the output cap, but we can see that the transcript counts as 1363 input tokens, meaning the output will exceed the maximum possible 1024 tokens:

  "metadata": {
    "tokenMetadata": {
      "outputTokenCount": {
        "totalBillableCharacters": 1904,
        "totalTokens": 531
      },
      "inputTokenCount": {
        "totalBillableCharacters": 5112,
        "totalTokens": 1363
      }
    }
  }

How does increasing temperature to 0.2 help? This time it does a terrible job of grouping the text:

<TOPICBREAK>tonight, several developing stories as we come on the air. news coming in, the suspected chinese spy balloon hovering over the u.s. where it is now, and will there be a window to shoot it down? also tonight, the dangerous life-threatening cold moving into the northeast. the live readings already. first, the pentagon tracking that chinese balloon, the intelligence bay hanging beneath it, the size of three busses. where it was spotted today over the u.s., where it's believed to be headed now, 60,000 feet in the air. will the u.s. shoot this down? secretary of state antony blinken postponing his high stakes trip to china. mola lenghi, mary bruce standing by at the white house. the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out. the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers. the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down. the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh's housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper's fall. but will the jury hear about it? eva pilgrim reporting. robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you'll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear.

<TOPICBREAK>good evening. it is great to have you with us on a friday night. almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. the readings coming in right now, and where the wind chill is already more than 100 degrees below zero. that's in a moment here. but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites. today the balloon was spotted over missouri. so where is it tonight, and where u.s. authorities believe it's headed. among the first sightings over billings, montana. that intelligence bay hanging beneath the balloon, that's the size of three busses. the pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? what we're learning tonight. secretary of state antony blinken postponing his high-stakes trip to beijing, calling china's actions unacceptable. so, will this be shot down? if so, where? and what china is saying tonight. mary bruce at the white house, and abc's mola lenghi leading us off tonight in montana.

What about a second time with a 0.2 temperature? Basically the same results we got before:

And at 0.99 temperature? Again, a terrible segmentation:

What about a different but related prompt of "In the following text, insert the tag '<STORY>' anywhere there is a shift from one story to another. Your goal is to divide the text into distinct stories, grouping sentences together that discuss a single story" and a temperature of 0.0. This time it correctly distinguishes the ER doctor and passenger jet wing clipping:

<STORY> tonight, several developing stories as we come on the air. news coming in, the suspected chinese spy balloon hovering over the u.s. where it is now, and will there be a window to shoot it down? also tonight, the dangerous life-threatening cold moving into the northeast. the live readings already. first, the pentagon tracking that chinese balloon, the intelligence bay hanging beneath it, the size of three busses. where it was spotted today over the u.s., where it's believed to be headed now, 60,000 feet in the air. will the u.s. shoot this down? secretary of state antony blinken postponing his high stakes trip to china. mola lenghi, mary bruce standing by at the white house. the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out.

<STORY> the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting.

<STORY> another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers.

<STORY> the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down.

<STORY> the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh's housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper's fall. but will the jury hear about it? eva pilgrim reporting.

<STORY> robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you'll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear.

However, in every single case it truncates the majority of the text. Perhaps this is simply because the input text is just too long for the model. Let's truncate the text a bit:

time curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/[YOURPROJECTID]/locations/us-central1/publishers/google/models/text-bison:predict -d \
$'{
"instances": [
{ "prompt": "Your job is to divide a transcript into individual stories. In the following text, insert the tag \'<STORY>\' anywhere there is a shift from one topic to another between sentences: tonight, several developing stories as we come on the air. news coming in, the suspected chinese spy balloon hovering over the u.s. where it is now, and will there be a window to shoot it down? also tonight, the dangerous life-threatening cold moving into the northeast. the live readings already. first, the pentagon tracking that chinese balloon, the intelligence bay hanging beneath it, the size of three busses. where it was spotted today over the u.s., where it\'s believed to be headed now, 60,000 feet in the air. will the u.s. shoot this down? secretary of state antony blinken postponing his high stakes trip to china. mola lenghi, mary bruce standing by at the white house. the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out. the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers. the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down. the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh\'s housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper\'s fall. but will the jury hear about it? eva pilgrim reporting. robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you\'ll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear. good evening. it is great to have you with us on a friday night. almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. the readings coming in right now, and where the wind chill is already more than 100 degrees below zero. that\'s in a moment here. but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites. today the balloon was spotted over missouri. so where is it tonight, and where u.s. authorities believe it\'s headed. among the first sightings over billings, montana. that intelligence bay hanging beneath the balloon, that\'s the size of three busses. the pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? what we\'re learning tonight. secretary of state antony blinken postponing his high-stakes trip to beijing, calling china\'s actions unacceptable. so, will this be shot down? if so, where? and what china is saying tonight. mary bruce at the white house, and abc\'s mola lenghi leading us off tonight in montana. reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east. the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that? reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri. commercial pilots radioing in. we got that balloon in sight also. looks like it\'s way up there, maybe 50,000 feet or so. reporter: the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. "}
],
"parameters": {
"temperature": 0.0,
"maxOutputTokens": 1024,
"topK": 40,
"topP": 0.95,
}
}' > O; cat O | jq -r .predictions[0].content

Despite being the exact same text, just with a few sentences removed from the bottom, this time we trigger a safety guardrail and the API refuses to produce output. This is a reminder that even with commercial APIs designed for enterprise use, guardrails can malfunction readily, wreaking havoc for downstream applications:

{
  "predictions": [
    {
      "safetyAttributes": {
        "blocked": true
      },
      "content": ""
    }
  ],
  "metadata": {
    "tokenMetadata": {
      "outputTokenCount": {
        "totalBillableCharacters": 0,
        "totalTokens": 1024
      },
      "inputTokenCount": {
        "totalBillableCharacters": 3758,
        "totalTokens": 1017
      }
    }
  }
}

What if we just increase our temperature from 0.0 to 0.2? All we get is the <STORY> tag repeated, minus any of the text:

<STORY> <STORY> <STORY> <STORY> <STORY>
<STORY> <STORY> <STORY>
<STORY> <STORY> <STORY> <STORY> <STORY> <STORY> <STORY>

What about increasing temperature to 0.4? This time we get the exact same output all three times:

<STORY> <STORY> <STORY>
<STORY> <STORY> <STORY>
<STORY> <STORY> <STORY>

What about a temperature of 0.99? This time we get results! Though, at such an extreme temperature setting there is an extremely high risk of hallucination:

<STORY> the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out.

<STORY> the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. <STORY> another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers.

What if we instead first ask it to correct the capitalization and punctuation of the text:

time curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/[YOURPROJECTID]/locations/us-central1/publishers/google/models/text-bison:predict -d \
$'{
"instances": [
{ "prompt": "Correct the punctuation and capitalization of this transcript: tonight, several developing stories as we come on the air. news coming in, the suspected chinese spy balloon hovering over the u.s. where it is now, and will there be a window to shoot it down? also tonight, the dangerous life-threatening cold moving into the northeast. the live readings already. first, the pentagon tracking that chinese balloon, the intelligence bay hanging beneath it, the size of three busses. where it was spotted today over the u.s., where it\'s believed to be headed now, 60,000 feet in the air. will the u.s. shoot this down? secretary of state antony blinken postponing his high stakes trip to china. mola lenghi, mary bruce standing by at the white house. the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out. the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers. the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down. the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh\'s housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper\'s fall. but will the jury hear about it? eva pilgrim reporting. robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you\'ll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear. good evening. it is great to have you with us on a friday night. almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. the readings coming in right now, and where the wind chill is already more than 100 degrees below zero. that\'s in a moment here. but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites. today the balloon was spotted over missouri. so where is it tonight, and where u.s. authorities believe it\'s headed. among the first sightings over billings, montana. that intelligence bay hanging beneath the balloon, that\'s the size of three busses. the pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? what we\'re learning tonight. secretary of state antony blinken postponing his high-stakes trip to beijing, calling china\'s actions unacceptable. so, will this be shot down? if so, where? and what china is saying tonight. mary bruce at the white house, and abc\'s mola lenghi leading us off tonight in montana. reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east. the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that? reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri. commercial pilots radioing in. we got that balloon in sight also. looks like it\'s way up there, maybe 50,000 feet or so. reporter: the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. "}
],
"parameters": {
"temperature": 0.0,
"maxOutputTokens": 1024,
"topK": 40,
"topP": 0.95,
}
}' > O; cat O | jq -r .predictions[0].content

This yields:

Tonight, several developing stories as we come on the air. News coming in, the suspected Chinese spy balloon hovering over the U.S. Where it is now, and will there be a window to shoot it down? Also tonight, the dangerous life-threatening cold moving into the Northeast. The live readings already. First, the Pentagon tracking that Chinese balloon, the intelligence bay hanging beneath it, the size of three busses. Where it was spotted today over the U.S., where it's believed to be headed now, 60,000 feet in the air. Will the U.S. shoot this down? Secretary of State Antony Blinken postponing his high stakes trip to China. Mola Lenghi, Mary Bruce standing by at the White House. The deadly cold already tonight, the National Weather Service is calling it a once in a generation Arctic blast. 25 million Americans, multiple states. New York City, Philadelphia, Boston, wind-chill readings in Maine expected to reach 60 below zero. And where the wind chill is already 106 degrees below zero tonight. Victor Oquendo in Massachusetts. Rob Marciano timing this out. The deadly assault on an ER doctor riding his mountain bike. Police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. Matt Gutman reporting. Another scare at the airport. This time, Newark. Two passenger jets clipping wings. One plane full of passengers. The stunning and unexpected jobs report tonight. President Biden celebrating the numbers. The unemployment rate not this low since 1969. So, how does this complicate steps by the Fed moving forward? Rebecca Jarvis, as always, breaking it down. The Alex Murdaugh double murder trial. The defense team, and what a ballistics expert acknowledged today. And tonight, the story of Alex Murdaugh's housekeeper dying in a fall at the home. What the family says they never knew. Prosecutors say Murdaugh got a $5 million payout from a policy that covered the housekeeper's fall. But will the jury hear about it? Eva Pilgrim reporting. Robin Roberts tonight and her epic jump. Robin from New Zealand, the birthplace of the bungee jump. You'll see her jump from the tallest building in the Southern Hemisphere. Tonight, the lesson on conquering fear. Good evening. It is great to have you with us on a Friday night. Almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. The readings coming in right now, and where the wind chill is already more than 100 degrees below zero. That's in a moment here. But we begin with the Chinese spy balloon hovering over the U.S. tonight. China claiming it is a weather balloon that blew off course. U.S. authorities believe this is a surveillance balloon being maneuvered by China, likely using spy satellites. Today the balloon was spotted over Missouri. So where is it tonight, and where U.S. authorities believe it's headed. Among the first sightings over Billings, Montana. That intelligence bay hanging beneath the balloon, that's the size of three busses. The Pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? What we're learning tonight. Secretary of State Antony Blinken postponing his high-stakes trip to Beijing, calling China's actions unacceptable. So, will this be shot down? If so, where? And what China is saying tonight. Mary Bruce at the White House, and ABC's Mola Lenghi leading us off tonight in Montana. Reporter: Tonight, the Pentagon confirming the massive Chinese spy balloon is on the move 60,000 feet above the ground and heading east. The balloon continues t move eastward and is currently over the center of the continental United States. What the heck is that? Reporter: A senior U.S. official tells ABC News the balloon now appears headed towards North Carolina. Across the country today, Americans with their eyes on the skies, posting images like this one, the balloon floating over Missouri. Commercial pilots radioing in. We got that balloon in sight also. Looks like it's way up there, maybe 50,000 feet or so. Reporter: The balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior U.S. official, equipped with what appear to be solar panels on the side that could power its technology.

Now let's repeat our story tokenization request. Guardrails still block us, suggesting that merely correcting the appearance of the text does not improve the guardrail's performance.

What if we increase our temperature to 0.2? This time we get results, suggesting that the LLM is sensitive to capitalization:

<STORY> Tonight, several developing stories as we come on the air. News coming in, the suspected Chinese spy balloon hovering over the U.S. Where it is now, and will there be a window to shoot it down? Also tonight, the dangerous life-threatening cold moving into the Northeast. The live readings already. First, the Pentagon tracking that Chinese balloon, the intelligence bay hanging beneath it, the size of three busses. Where it was spotted today over the U.S., where it's believed to be headed now, 60,000 feet in the air. Will the U.S. shoot this down? Secretary of State Antony Blinken postponing his high stakes trip to China. Mola Lenghi, Mary Bruce standing by at the White House. The deadly cold already tonight, the National Weather Service is calling it a once in a generation Arctic blast. 25 million Americans, multiple states. New York City, Philadelphia, Boston, wind-chill readings in Maine expected to reach 60 below zero. And where the wind chill is already 106 degrees below zero tonight. Victor Oquendo in Massachusetts. Rob Marciano timing this out.

<STORY> The deadly assault on an ER doctor riding his mountain bike. Police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. Matt Gutman reporting. Another scare at the airport. This time, Newark. Two passenger jets clipping wings. One plane full of passengers.

<STORY> The stunning and unexpected jobs report tonight. President Biden celebrating the numbers. The unemployment rate not this low since 1969. So, how does this complicate steps by the Fed moving forward? Rebecca Jarvis, as always, breaking it down.

<STORY> The Alex Murdaugh double murder trial. The defense team, and what a ballistics expert acknowledged today. And tonight, the story of Alex Murdaugh's housekeeper dying in a fall at the home. What the family says they never knew. Prosecutors say Murdaugh got a $5 million payout from a policy that covered the housekeeper's fall. But will the jury hear about it? Eva Pilgrim reporting.

<STORY> Robin Roberts tonight and her epic jump. Robin from New Zealand, the birthplace of the bungee jump. You'll see her jump from the tallest building in the Southern Hemisphere. Tonight, the lesson on conquering fear.

Let's start over and try a different approach: dynamic semantic windowing. We'll take the first minute of the broadcast and ask the LLM to annotate it by inserting a <TOPICBREAK> XML tag in between any story transitions in the text. Then we'll repeat for the next minute. Since our hard one minute cutoff means the text will likely be truncated in the middle of the last story of the first minute, we'll rewind slightly and include the text of the last story of the first minute with the full contents of the next minute. We'll repeat this process minute-by-minute over the transcript.

Let's truncate to just the first minute of the transcript:

Now let's repeat our prompt with a temperature of 0.0:

time curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/[YOURPROJECTID]/locations/us-central1/publishers/google/models/text-bison:predict -d \
$'{
  "instances": [
    { "prompt": "Break the text below into distinct stories by inserting the XML tag \'<TOPICBREAK>\' in between any pair of sentences that represent a shift from one story to another:  tonight, several developing stories as we come on the air. news coming in, the suspected chinese spy balloon hovering over the u.s. where it is now, and will there be a window to shoot it down? also tonight, the dangerous life-threatening cold moving into the northeast. the live readings already. first, the pentagon tracking that chinese balloon, the intelligence bay hanging beneath it, the size of three busses. where it was spotted today over the u.s., where it\'s believed to be headed now, 60,000 feet in the air. will the u.s. shoot this down? secretary of state antony blinken postponing his high stakes trip to china. mola lenghi, mary bruce standing by at the white house. the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast.  "}
  ],
  "parameters": {
    "temperature": 0.0,
    "maxOutputTokens": 1024,
    "topK": 40,
    "topP": 0.95,
  }
}' > O; cat O | jq -r .predictions[0].content

This produces the following output:

<TOPICBREAK> the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast.

The second story begins right at the end of the first minute. So, let's take the next minute of transcript and back up slightly to start with the start of that story to yield:

the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out. the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting. another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers. the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down. the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today. and tonight, the story of alex murdaugh\'s housekeeper dying in a fall at the home. what the family says they never knew.

Running this yields:

<TOPICBREAK>the deadly cold already tonight, the national weather service is calling it a once in a generation arctic blast. 25 million americans, multiple states. new york city, philadelphia, boston, wind-chill readings in maine expected to reach 60 below zero. and where the wind chill is already 106 degrees below zero tonight. victor oquendo in massachusetts. rob marciano timing this out.
<TOPICBREAK>the deadly assault on an e.r. doctor riding his mountain bike. police say the suspect in his car hitting the victim, then allegedly getting out and attacking him. matt gutman reporting.
<TOPICBREAK>another scare at the airport. this time, newark. two passenger jets clipping wings. one plane full of passengers.
<TOPICBREAK>the stunning and unexpected jobs report tonight. president biden celebrating the numbers. the unemployment rate not this low since 1969. so, how does this complicate steps by the fed moving forward? rebecca jarvis, as always, breaking it down.
<TOPICBREAK>the alex murdaugh double murder trial. the defense team, and what a ballistics expert acknowledged today.
<TOPICBREAK>and tonight, the story of alex murdaugh's housekeeper dying in a fall at the home. what the family says they never knew.

And we'll repeat the process with the next minute – again backing up to the start of the last story:

and tonight, the story of alex murdaugh\'s housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper\'s fall. but will the jury hear about it? eva pilgrim reporting. robin roberts tonight and her epic jump. robin from new zealand, the birthplace of the bungee jump. you\'ll see her jump from the tallest building in the southern hemisphere. tonight, the lesson on conquering fear. good evening. it is great to have you with us on a friday night. almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. the readings coming in right now, and where the wind chill is already more than 100 degrees below zero. that\'s in a moment here. but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites.

Yielding:

<TOPICBREAK>and tonight, the story of alex murdaugh's housekeeper dying in a fall at the home. what the family says they never knew. prosecutors say murdaugh got a $5 million payout from a policy that covered the housekeeper's fall. but will the jury hear about it? eva pilgrim reporting.

And again:

good evening. it is great to have you with us on a friday night. almost to the weekend, but it will be brutally cold, dangerous, life-threatening cold. the readings coming in right now, and where the wind chill is already more than 100 degrees below zero. that\'s in a moment here. but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites. today the balloon was spotted over missouri. so where is it tonight, and where u.s. authorities believe it\'s headed. among the first sightings over billings, montana. that intelligence bay hanging beneath the balloon, that\'s the size of three busses. the pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? what we\'re learning tonight. secretary of state antony blinken postponing his high-stakes trip to beijing, calling china\'s actions unacceptable. so, will this be shot down? if so, where? and what china is saying tonight. mary bruce at the white house, and abc\'s mola lenghi leading us off tonight in montana. reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east. the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that? reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri.

Yielding the following. Note how above it grouped record cold with the balloon, lumping two different stories together, while here in our next run it correctly segments them, demonstrating the importance of the dynamic windowing:

<TOPICBREAK>but we begin with the chinese spy balloon hovering over the u.s. tonight. china claiming it is a weather balloon that blew off course. u.s. authorities believe this is a surveillance balloon being maneuvered by china, likely using spy satellites. today the balloon was spotted over missouri. so where is it tonight, and where u.s. authorities believe it's headed. among the first sightings over billings, montana. that intelligence bay hanging beneath the balloon, that's the size of three busses. the pentagon tonight on discussions over whether or not to shoot it down, and will there be a small window of opportunity to do just that? what we're learning tonight. secretary of state antony blinken postponing his high-stakes trip to beijing, calling china's actions unacceptable. so, will this be shot down? if so, where? and what china is saying tonight. mary bruce at the white house, and abc's mola lenghi leading us off tonight in montana.

<TOPICBREAK>reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east. the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that? reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri.

And again:

reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east. the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that? reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri. commercial pilots radioing in. we got that balloon in sight also. looks like it\'s way up there, maybe 50,000 feet or so. reporter: the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana. this thing is up in the sky. what the heck is that? that thing is not the moon. any help would be appreciated. reporter: montana republican congressman ryan zinke was blunt, tweeting \\"shoot. it. down.\\" the pentagon today sayig that option was considered and rejected, for now. we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options. reporter: the white house today saying president biden agreed with the pentagon\'s strong recommendation.

Yielding the hallucinated results of:

reporter: <TOPICBREAK>
reporter: the fbi is investigating a series of bomb threats that have been made to jewish community centers and day schools across the country. the threats have been made over the phone and in emails. the fbi says it is taking the threats seriously and is working with local law enforcement to investigate. reporter: <TOPICBREAK>
reporter: the washington post is reporting that the u.s. is planning to withdraw all of its troops from afghanistan by the end of the year. the report says that the decision was made by president biden and that it will be announced in the coming days. the u.s. has about 25,000 troops in afghanistan. reporter: <TOPICBREAK>
reporter: the u.s. economy grew at a faster pace than expected in the fourth quarter of last year. the gross domestic product grew at an annual rate of 6.9 percent, according to the commerce department. that's up from a revised 6.5 percent growth rate in the third quarter. the growth was driven by strong consumer spending and business investment.

What just happened? Recall that this is the command we used:

Despite a temperature setting of 0.0, the API hallucinated an entirely different passage.

Given how prominently the FBI is mentioned above, what if we ask it whether the text mentions the FBI?

Does the following text mention the FBI?

The answer?

No. The text does not mention the FBI.

Thus, hallucination appears to be fully prompt/passage-dependent, with both the prompt and passage combining to yield hallucination.

What if we add the sentence "Use only the text below" to our prompt:

Break the text below into distinct stories by inserting the XML tag \'<TOPICBREAK>\' in between any pair of sentences that represent a shift from one story to another. Use only the text below:

Thus, our final command is:

This removes the hallucination, but enters a failure state where it annotates every sentence as being a topical divider:

reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east.

<TOPICBREAK>the balloon continues t move eastward and is currently over the center of the continental united states.

<TOPICBREAK>what the heck is that?

<TOPICBREAK>reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina.

<TOPICBREAK>across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri.

<TOPICBREAK>commercial pilots radioing in. we got that balloon in sight also. looks like it's way up there, maybe 50,000 feet or so.

<TOPICBREAK>it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana.

<TOPICBREAK>this thing is up in the sky. what the heck is that?

<TOPICBREAK>that thing is not the moon.

<TOPICBREAK>any help would be appreciated.

<TOPICBREAK>reporter: montana republican congressman ryan zinke was blunt, tweeting "shoot. it. down."

<TOPICBREAK>the pentagon today sayig that option was considered and rejected, for now.

<TOPICBREAK>we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options.

<TOPICBREAK>reporter: the white house today saying president biden agreed with the pentagon's strong recommendation.

What if we boost the temperature to 0.2?

reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east.

<TOPICBREAK>the balloon continues t move eastward and is currently over the center of the continental united states. what the heck is that?

<TOPICBREAK>reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina. across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri. commercial pilots radioing in. we got that balloon in sight also. looks like it's way up there, maybe 50,000 feet or so.

<TOPICBREAK>reporter: the balloon is huge with a technology bay attached below that is itself the size of three busses, loaded with high resolution cameras, according to a senior u.s. official, equipped with what appear to be solar panels on the side that could power its technology. it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana. this thing is up in the sky. what the heck is that? that thing is not the moon. any help would be appreciated.

<TOPICBREAK>reporter: montana republican congressman ryan zinke was blunt, tweeting "shoot. it. down." the pentagon today sayig that option was considered and rejected, for now. we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options.

<TOPICBREAK>reporter: the white house today saying president biden agreed with the pentagon's strong recommendation.

What if we run it a second time? Despite the same exact parameters as above, we're back to sentence-level division. This means that even with a temperature of 0.2, the output is highly unstable.

reporter: tonight, the pentagon confirming the massive chinese spy balloon is on the move 60,000 feet above the ground and heading east.

<TOPICBREAK>the balloon continues t move eastward and is currently over the center of the continental united states.

<TOPICBREAK>what the heck is that?

<TOPICBREAK>reporter: a senior u.s. official tells abc news the balloon now appears headed towards north carolina.

<TOPICBREAK>across the country today, americans with their eyes on the skies, posting images like this one, the balloon floating over missouri.

<TOPICBREAK>commercial pilots radioing in. we got that balloon in sight also. looks like it's way up there, maybe 50,000 feet or so.

<TOPICBREAK>it first entered american airspace over alaska, then flying into southwest canada before dipping down over billings, montana.

<TOPICBREAK>this thing is up in the sky. what the heck is that?

<TOPICBREAK>that thing is not the moon.

<TOPICBREAK>any help would be appreciated.

<TOPICBREAK>reporter: montana republican congressman ryan zinke was blunt, tweeting "shoot. it. down."

<TOPICBREAK>the pentagon today sayig that option was considered and rejected, for now.

<TOPICBREAK>we assessed that currently it does not pose a physical or military risk to people on the ground. for now we are continuing to monitor and review options.

<TOPICBREAK>reporter: the white house today saying president biden agreed with the pentagon's strong recommendation.

Let's pretend that we have a postfilter that rejects annotations that exceed a certain density of sentences-to-divisions and thus we go with the first output. Let's repeat our process and revert back to a 0.0 temperature:

Break the text below into distinct stories by inserting the XML tag \'<TOPICBREAK>\' in between any pair of sentences that represent a shift from one story to another. Use only the text below: reporter: the white house today saying president biden agreed with the pentagon\'s strong recommendation. the risks involved with shooting down the balloon just too great. any potential debris field would be significant and potentially cause civilian injuries or deaths or significant property damage. reporter: for its part, china insisting today the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. the pentagon flatly rejecting that claim. we are aware of the prc\'s statement. however, the fact is, we know that it\'s a surveillance pballo. we have clearly communicated that this balloon is violating u.s. air space and international law and that this is unacceptable. reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it\'s likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing.

Yielding the following. While it doesn't split by sentence, it is still not an ideal grouping of the text, though it does roughly break it by subtopic:

reporter: the white house today saying president biden agreed with the pentagon's strong recommendation. the risks involved with shooting down the balloon just too great.

<TOPICBREAK>any potential debris field would be significant and potentially cause civilian injuries or deaths or significant property damage.

<TOPICBREAK>reporter: for its part, china insisting today the balloon is not a spy vessel at all but a civilian airship used for research, mainly meteorological purposes. the pentagon flatly rejecting that claim.

<TOPICBREAK>we are aware of the prc's statement. however, the fact is, we know that it's a surveillance pballo. we have clearly communicated that this balloon is violating u.s. air space and international law and that this is unacceptable.

<TOPICBREAK>reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it's likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing.

Let's continue the process:

Break the text below into distinct stories by inserting the XML tag \'<TOPICBREAK>\' in between any pair of sentences that represent a shift from one story to another. Use only the text below: reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it\'s likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing. that visit now canceled. in light of china\'s unacceptable action, i am postponing my planned travel this weekend to china. we concluded that conditions were not conducive for a constructive visit at this time. reporter: there have been other foreign surveillance balloons above the u.s., but none like this. what makes this different is the duration and the length of which it has been over u.s. territory. reporter: the pentagon says the balloon is expected to linger above america for a few days. this is really something. let\'s bring in mola lenghi. he\'s in billings, montana, for us again tonight. that state with several sensitive nuclear sites. as you reported, u.s. authorities say the balloon is likely navigated by chinese satellites. the pentagon says they\'ve decided not to shoot it down at this point, but that option isn\'t completely off the table? reporter: that\'s right, david, a u.s. official familiar with the situation says the current thinking is to wait for the balloon to drift over the atlantic ocean and then shoot it down once it\'s over u.s. territoryial waters.

Yielding the following, which once again breaks the text every few sentences:

<TOPICBREAK>in light of china's unacceptable action, i am postponing my planned travel this weekend to china. we concluded that conditions were not conducive for a constructive visit at this time.

<TOPICBREAK>reporter: there have been other foreign surveillance balloons above the u.s., but none like this. what makes this different is the duration and the length of which it has been over u.s. territory.

<TOPICBREAK>reporter: the pentagon says the balloon is expected to linger above america for a few days.

<TOPICBREAK>this is really something. let's bring in mola lenghi. he's in billings, montana, for us again tonight. that state with several sensitive nuclear sites. as you reported, u.s. authorities say the balloon is likely navigated by chinese satellites. the pentagon says they've decided not to shoot it down at this point, but that option isn't completely off the table?

<TOPICBREAK>reporter: that's right, david, a u.s. official familiar with the situation says the current thinking is to wait for the balloon to drift over the atlantic ocean and then shoot it down once it's over u.s. territoryial waters.

Obviously this is not ideal. What if we remove the "Use only the text below:" part of our prompt and revert to "Break the text below into distinct stories by inserting the XML tag \'<TOPICBREAK>\' in between any pair of sentences that represent a shift from one story to another.":

This yields the following. Once again, the API has hallucinated an entirely unrelated passage:

<TOPICBREAK> reporter: the u.s. is warning china that it will take action if it does not stop sending military aircraft into taiwan's air defense identification zone. the warning comes after china sent 29 military aircraft into the zone on monday. the u.s. says the flights are a threat to peace and stability in the region. reporter: china says the flights are routine and are not a threat. it says it has the right to fly in the zone as it claims sovereignty over taiwan. reporter: the u.s. has long supported taiwan and has pledged to help it defend itself. the u.s. has also warned china against using force to take control of taiwan. reporter: the latest flights come as tensions between the u.s. and china are rising over a number of issues, including trade, human rights, and the coronavirus pandemic.

<TOPICBREAK> reporter: the u.s. is sending a new aircraft carrier to the middle east in a show of force against iran. the carrier, the uss john c. stennis, will be deployed to the region in the coming weeks. the move comes after iran recently launched a series of ballistic missiles into the sea. the u.s. says the missiles were a threat to international peace and security. reporter: iran says the missiles were a warning to the u.s. and its allies not to interfere in the region. the u.s. has said it will continue to work with its allies to prevent iran from developing nuclear weapons. reporter: the deployment of the uss john c. stennis is the latest in a series of moves by the u.s. to increase its military presence in the middle east. the u.s. has also sent additional troops to the region and has conducted airstrikes against iran-backed militias. the u.s. says the moves are necessary to deter iran from further aggression.

In a sign of how brittle LLM outputs can be, what if we replace the colon with a period and a single space before the text?

Break the text below into distinct stories by inserting the XML tag \'<TOPICBREAK>\' in between any pair of sentences that represent a shift from one story to another. reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it\'s likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing. that visit now canceled. in light of china\'s unacceptable action, i am postponing my planned travel this weekend to china. we concluded that conditions were not conducive for a constructive visit at this time. reporter: there have been other foreign surveillance balloons above the u.s., but none like this. what makes this different is the duration and the length of which it has been over u.s. territory. reporter: the pentagon says the balloon is expected to linger above america for a few days. this is really something. let\'s bring in mola lenghi. he\'s in billings, montana, for us again tonight. that state with several sensitive nuclear sites. as you reported, u.s. authorities say the balloon is likely navigated by chinese satellites. the pentagon says they\'ve decided not to shoot it down at this point, but that option isn\'t completely off the table? reporter: that\'s right, david, a u.s. official familiar with the situation says the current thinking is to wait for the balloon to drift over the atlantic ocean and then shoot it down once it\'s over u.s. territoryial waters.

This yields:

reporter: china says the balloon deviated far from its planned course, but the pentagon questioning that as well, saying the balloon can be maneuvered. a senior u.s. official telling abc news, it's likely being navigated with spy satellites. this all happening as secretary of state antony blinken prepared for a weekend trip to beijing. that visit now canceled. <TOPICBREAK>in light of china's unacceptable action, i am postponing my planned travel this weekend to china. we concluded that conditions were not conducive for a constructive visit at this time. <TOPICBREAK>there have been other foreign surveillance balloons above the u.s., but none like this. what makes this different is the duration and the length of which it has been over u.s. territory. <TOPICBREAK>the pentagon says the balloon is expected to linger above america for a few days. <TOPICBREAK>this is really something. let's bring in mola lenghi. he's in billings, montana, for us again tonight. that state with several sensitive nuclear sites. as you reported, u.s. authorities say the balloon is likely navigated by chinese satellites. the pentagon says they've decided not to shoot it down at this point, but that option isn't completely off the table? <TOPICBREAK>that's right, david, a u.s. official familiar with the situation says the current thinking is to wait for the balloon to drift over the atlantic ocean and then shoot it down once it's over u.s. territoryial waters.

But what if we have a second space between the period and the start of the text?

Our output is now null. It produces no output at all:

 "content": ""

Clearly we have a problem. What if we modify our prompt slightly to the following:

Thus, our command becomes:

This enters an infinite loop and additionally hallucinates, including "the u.s. is demanding that china return the balloon":

<TOPICBREAK> reporter: the u.s. is accusing china of sending a high-tech surveillance balloon into u.s. airspace. the balloon was spotted over montana last week. reporter: china says the balloon is part of a weather research project and that it has drifted off course. reporter: the pentagon says the balloon is equipped with cameras and other sensors and that it is being maneuvered by chinese satellites. reporter: the u.s. is demanding that china return the balloon. reporter: china has not responded to the u.s. demand. reporter: the u.s. is also considering shooting down the balloon. reporter: the pentagon says that shooting down the balloon would be a last resort. reporter: the u.s. is accusing china of sending a high-tech surveillance balloon into u.s. airspace. the balloon was spotted over montana last week. reporter: china says the balloon is part of a weather research project and that it has drifted off course. reporter: the pentagon says the balloon is equipped with cameras and other sensors and that it is being maneuvered by chinese satellites. reporter: the u.s. is demanding that china return the balloon. reporter: china has not responded to the u.s. demand. reporter: the u.s. is also considering shooting down the balloon. reporter: the pentagon says that shooting down the balloon would be a last resort. reporter: the u.s. is accusing china of sending a high-tech surveillance balloon into u.s. airspace. the balloon was spotted over montana last week. reporter: china says the balloon is part of a weather research project and that it has drifted off course. reporter: the pentagon says the balloon is equipped with cameras and other sensors and that it is being maneuvered by chinese satellites. reporter: the u.s. is demanding that china return the balloon. reporter: china has not responded to the u.s. demand. reporter: the u.s. is also considering shooting down the balloon. reporter: the pentagon says that shooting down the balloon would be a last resort. reporter: the u.s. is accusing china of sending a high-tech surveillance balloon into u.s. airspace. the balloon was spotted over montana last week. reporter: china says the balloon is part of a weather research project and that it has drifted off course. reporter: the pentagon says the balloon is equipped with cameras and other sensors and that it is being maneuvered by chinese satellites. reporter: the u.s. is demanding that china return the balloon. reporter: china has not responded to the u.s. demand. reporter: the u.s. is also considering shooting down the balloon. reporter: the pentagon says that shooting down the balloon would be a last resort. reporter: the u.s. is accusing china of sending a high-tech surveillance balloon into u.s. airspace. the balloon was spotted over montana last week. reporter: china says the balloon is part of a weather research project and that it has drifted off course. reporter: the pentagon says the balloon is equipped with cameras and other sensors and that it is being maneuvered by chinese satellites. reporter: the u.s. is demanding that china return the balloon. reporter: china has not responded to the u.s. demand. reporter: the u.s. is also considering shooting down the balloon. reporter: the pentagon says that shooting down the balloon would be a last resort. reporter: the u.s. is accusing china of sending a high-tech surveillance balloon into u.s. airspace. the balloon was spotted over montana last week. reporter: china says the balloon is part of a weather research project and that it has drifted off course. reporter: the pentagon says the balloon is equipped with cameras and other sensors and that it is being maneuvered by chinese satellites. reporter: the u.s. is demanding that china return the balloon. reporter: china has not responded to the u.s. demand. reporter: the u.s. is also considering shooting down the balloon. reporter: the pentagon says that shooting down the balloon would be a last resort. reporter: the u.s. is accusing china of sending a high-tech surveillance balloon into u.s. airspace. the balloon was spotted over montana last week. reporter: china says the balloon is part of a weather research project and that it has drifted off course. reporter: the pentagon says the balloon is equipped with cameras and other sensors and that it is being maneuvered by chinese satellites. reporter: the u.s. is demanding that china return the balloon. reporter: china has not responded to the u.s. demand. reporter: the

Let's try a different prompt:

You are an intelligent agent that segments news transcripts into stories. Take the following news transcript and identify how many different stories it contains. Insert the phrase \' NEWSTORY \' anywhere the story being described changes in a major way.

And the final command:

This yields just the number "2" and that's it:

That's it. What if we increase temperature to 0.2? All we get are single numbers still:

Increasing to 0.4 yields results, but not what we asked for. Remember we asked for a marked-up text, not a bulleted list of summaries:

3 stories:

1. China sends balloon into US airspace
2. US cancels Blinken's visit to China
3. US may shoot down balloon

Let's run it again. This time we got just the number 3 again.

Let's run again with temperature of 0.99. We get results, but not the annotated version of the transcript that we asked for:

1. NEWSTORY China says its balloon that drifted into American airspace was a weather balloon that malfunctioned. But the Pentagon says it is a sophisticated spy balloon that was maneuvered by Chinese satellites.
2. NEWSTORY Secretary of State Antony Blinken has canceled a planned visit to Beijing in response to China's "unacceptable action."
3. NEWSTORY The Pentagon says the balloon is expected to linger over the United States for a few days.
4. NEWSTORY The U.S. is not planning to shoot the balloon down at this time, but the option is not completely off the table.

Running a second time we get even worse results:

2 stories:

STORY 1: Chinese balloon over USA for several days
STORY 2: US officials decide not to shoot down Chinese balloon

Clearly, a different approach is needed, which we will explore in Part 2.