As we continue to explore Google's Gemini 1.5 Pro model's native video capabilities, what if we scale our analysis from a brief 6 minute clip of Russian television news to a full 30-minute evening news broadcast in the US? Using the same ABC World News Tonight broadcast whose spoken word transcript we've repeatedly examined in the past, this time we're going to examine only the visual content itself. Using Google's AI Studio, we're going to analyze the complete 30-minute broadcast using a single prompt to see how it performs in real life. Remember that Gemini works by sharding the video into thousands of individual still image frames and does not consider the audio channel in any way. You can compare the results below with the Vanderbilt Television News Archive's human annotation of the broadcast into discrete stories to see how well Gemini matches the human story delineation. For the technically-minded, the 30-minute broadcast consumed 488,917 tokens out of Gemini 1.5 Pro's 1 million available.
The end result is that Gemini does an extraordinary job, given only an MP4 video file as input, of segmenting an entire 30-minute broadcast into discrete stories using just a single zero-shot textual prompt. In other words, there is no RAG, prompt chaining, pyramidal stacking or other mess – you simply give Gemini an MP4 up to one hour in length and an ordinary textual prompt and the model applies that prompt to that video file!
At the same time, the output is highly unstable and exhibits lower than desired prompt coherence, with some runs yielding incredibly strong results and others yielding either truncated results or results that were so substantially different from what was asked for as to be problematic in interpreting them. Despite these limitations, the ability of Gemini to analyze a complete evening news broadcast and segment it into a story-based chronology without any examples or assistance is extraordinary.
Let's start with a simple prompt:
Describe this video in detail.
The results start off reasonably, but then truncate:
The video begins with a commercial for National University, a WSCUC-accredited university that offers online and on-campus programs. The commercial features a montage of students and faculty interacting in various settings, including classrooms, libraries, and online. The commercial ends with the tagline "Supporting the Whole You."
The next segment is a news report about a Chinese spy balloon that is being tracked over the United States. The report features footage of the balloon and interviews with experts who discuss the potential implications of the balloon's presence.
The next segment is a weather report about a "once-in-a-generation" arctic blast that is affecting the Northeast. The report features footage of the snow and interviews with people who are affected by the cold weather.
The next segment is a news report about a deadly assault arrest in California. The report features footage of the crime scene and interviews with police and witnesses.
The next segment is a news report about two United Airlines jets that clipped wings at Newark Liberty International Airport. The report features footage of the incident and interviews with passengers and officials.
The next segment is a commercial for Metamucil, a fiber supplement that can help with digestive
Since the results were truncated, let's try a second time:
Describe this video in detail.
Unfortunately, this time Gemini fixates on the advertisements in the video and ignores the news segments:
The video is a compilation of several different commercials and news segments. The first commercial is for National University, a WSCUC-accredited online university. The commercial shows a montage of people of different ages and backgrounds studying, working, and spending time with their families. The commercial ends with the tagline "Supporting the Whole You."
The second commercial is for MetaMucil, a fiber supplement. The commercial features a woman who is feeling sluggish and weighed down. She mixes a glass of MetaMucil and drinks it. The commercial then shows an animation of how MetaMucil works to trap and remove waste from the digestive system. The commercial ends with the woman feeling lighter and more energetic.
The third commercial is for Angie, a home services marketplace. The commercial shows a father and son playing basketball in their driveway. The father then goes inside and uses Angie to find a contractor to install a new garage door. The commercial ends with the tagline "Your home for everything home."
The fourth commercial is for Ashley Furniture HomeStore. The commercial features a dachshund dog who is tired and can't sleep. The dog's owners buy a new Tempur-Pedic mattress from Ashley Furniture HomeStore. The commercial then shows the dog sleeping soundly on the new mattress. The commercial ends with the tagline "Shop in-store."
The fifth commercial is for Vicks VapoRub, a cough suppressant. The commercial features a man who is coughing and can't sleep. He rubs VapoRub on his chest and throat. The commercial then shows an animation of how VapoRub works to relieve cough symptoms. The commercial ends with the man sleeping soundly.
The sixth commercial is for Red Lobster. The commercial features a family enjoying a seafood feast at Red Lobster. The commercial ends with the tagline "Welcome to Fun Dining."
The seventh commercial is for LifeLock, an identity theft protection service. The commercial shows a man working on his computer late at night. He receives a credit inquiry alert from LifeLock. The commercial then shows a LifeLock agent calling the man to help him resolve the issue. The commercial ends with the tagline "LifeLock helps detect identity threats. If there's a problem, we work to fix it."
The eighth commercial is for Nature's Bounty, a vitamin and supplement brand. The commercial features a montage of people of different ages and backgrounds taking Nature's Bounty vitamins and supplements. The commercial ends with the
Let's try a third time:
Describe this video in detail.
This time we get longer output that mixes advertisements and news, but it still truncates:
The video begins with a commercial for National University, a WSCUC-accredited university that offers online and on-campus programs. The commercial features a montage of students and faculty interacting in various settings, including classrooms, libraries, and online. The commercial ends with the tagline "Supporting the Whole You."
The next segment is a news report about a Chinese spy balloon that is being tracked over the United States. The report features footage of the balloon and interviews with experts who discuss the potential implications of the balloon's presence.
The next segment is a weather report about a "once-in-a-generation" arctic blast that is affecting the Northeast. The report features footage of the snow and interviews with people who are affected by the cold weather.
The next segment is a news report about a deadly assault arrest in California. The report features footage of the crime scene and interviews with police and witnesses.
The next segment is a news report about two United Airlines jets that clipped wings at Newark Liberty International Airport. The report features footage of the incident and interviews with passengers and officials.
The next segment is a commercial for Metamucil, a fiber supplement that can help with digestive health. The commercial features a woman who is feeling sluggish and weighed down. She takes Metamucil and feels better after just 14 days.
The next segment is a commercial for Angie, a home services marketplace. The commercial features a father and son playing basketball in their driveway. The father goes inside to use Angie to find a handyman to fix their garage door.
The next segment is a commercial for Ashley Furniture HomeStore. The commercial features a dachshund dog who is tired and can't sleep. The dog's owners buy a new Tempur-Pedic mattress from Ashley Furniture HomeStore, and the dog is finally able to get a good night's sleep.
The next segment is a commercial for Vicks VapoRub, a cough suppressant. The commercial features a man who is coughing and can't sleep. He rubs VapoRub on his chest and is able to breathe easier and fall asleep.
The next segment is a commercial for Red Lobster, a seafood restaurant. The commercial features a family enjoying a meal of lobster and shrimp tacos.
The next segment is a commercial for Ancestry, a genealogy website. The commercial features a woman and her daughter visiting a museum and learning about their family history.
The next segment
Let's try a fourth time, explicitly asking it to ignore the ads:
Describe this video in detail. Ignore the advertisements.
This time the model does not truncate, but excludes many of the stories:
The video contains several news segments. The first segment is about a Chinese spy balloon that was tracked over the United States. The segment includes footage of the balloon and interviews with government officials and residents of Montana, where the balloon was first spotted.
The second segment is about a deadly assault in California. The segment includes footage of the crime scene and interviews with police and the victim's colleagues.
The third segment is about a "once-in-a-generation" arctic blast that is affecting the Northeast. The segment includes footage of the snow and interviews with residents and meteorologists.
Let's try again:
Describe this video in detail. Ignore the advertisements.
It extends its summary, but still misses a lot:
The video starts with a breaking news report about a Chinese spy balloon being tracked over the U.S. The balloon is described as being the size of three buses and having high-resolution cameras and solar panels. The Pentagon says it is a surveillance balloon and can be maneuvered. China claims it is a civilian airship used for research, mainly meteorological purposes. Secretary of State Antony Blinken has postponed his trip to China in response to the balloon.
The video then cuts to a report about a deadly assault arrest in California. A bicyclist was hit by a car and then fatally stabbed by the driver. The driver, Vanroy Evan Smith, was detained by bystanders and is now facing murder charges.
The video then cuts to a report about a "once-in-a-generation" arctic blast that is hitting the Northeast. The wind chill is expected to be below zero in many areas, and there is a blizzard warning in effect for parts of Maine.
The video then cuts to a report about the Oakland Zoo reopening after being closed for several weeks due to flooding. The zoo is now open to the public, and visitors can see all of the animals, including the giraffes
Let's try for a more detailed summary:
Describe this video second-by-second in a table, where each row is one second of airtime. The first column should be the offset from the start of the video. The second column should be a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that second of airtime. The third column should be a detailed textual description of that second of airtime that describes it in sufficient detail to someone who can't see the video.
This yields the following, but it truncates after just 2:47:
Second-by-Second Description of Video
| Offset | Keywords | Description |
| 00:00 | National University, education, students, faculty | A commercial for National University begins with a montage of students and faculty interacting in various settings, including classrooms, libraries, and online. |
| 00:01 | National University, education, online learning | The commercial continues with a focus on the university's online learning platform. |
| 00:02 | National University, graduation, celebration | The commercial shows students graduating and celebrating their achievements. |
| 00:03 | National University, support, community | The commercial emphasizes the university's supportive community and commitment to helping students succeed. |
| 00:04 | National University, diversity, inclusion | The commercial highlights the university's diverse student body and inclusive environment. |
| 00:05 | National University, career success, opportunities | The commercial shows students pursuing their careers and achieving their goals. |
| 00:06 | National University, logo, tagline | The commercial ends with the National University logo and the tagline "Supporting the Whole You." |
| 00:07 | News report, China, spy balloon, United States | A news report begins with footage of a large, white balloon in the sky. The report states that the balloon is a suspected Chinese spy balloon that is being tracked over the United States. |
| 00:08 | News report, China, spy balloon, Montana | The news report shows a map of the United States with the balloon's current location over Montana highlighted. |
| 00:09 | News report, China, spy balloon, surveillance | The news report discusses the potential implications of the balloon's presence, including the possibility that it is being used for surveillance. |
| 00:10 | News report, China, spy balloon, response | The news report discusses the U.S. government's response to the balloon, including the possibility of shooting it down. |
| 00:11 | News report, China, spy balloon, tensions | The news report discusses the rising tensions between the United States and China over the balloon incident. |
| 00:12 | News report, weather, arctic blast, Northeast | A weather report begins with footage of snow falling and strong winds blowing. The report states that a "once-in-a-generation" arctic blast is affecting the Northeast. |
| 00:13 | Weather report, arctic blast, temperatures, wind chills | The weather report shows a map of the Northeast with the current temperatures and wind chills. |
| 00:14 | Weather report, arctic blast, safety, precautions | The weather report discusses the dangers of the cold weather and provides safety precautions for viewers. |
| 00:15 | Weather report, arctic blast, travel, disruptions | The weather report discusses the travel disruptions caused by the arctic blast, including flight cancellations and road closures. |
| 00:16 | News report, California, assault, arrest | A news report begins with footage of a crime scene. The report states that a man has been arrested for fatally stabbing a bicyclist in California. |
| 00:17 | News report, California, assault, victim | The news report shows a photo of the victim, a physician named Michael Mammone. |
| 00:18 | News report, California, assault, suspect | The news report shows a photo of the suspect, Vanroy Evan Smith. |
| 00:19 | News report, California, assault, investigation | The news report discusses the ongoing investigation into the assault. |
| 00:20 | News report, California, assault, community | The news report discusses the impact of the assault on the local community. |
| 00:21 | News report, United Airlines, Newark Airport, incident | A news report begins with footage of two United Airlines jets with their wings clipped. The report states that the incident occurred at Newark Liberty International Airport. |
| 00:22 | News report, United Airlines, Newark Airport, investigation | The news report discusses the ongoing investigation into the incident. |
| 00:23 | News report, United Airlines, Newark Airport, passengers | The news report interviews passengers who were on board the planes when the incident occurred. |
| 00:24 | News report, United Airlines, Newark Airport, safety | The news report discusses the safety implications of the incident. |
| 00:25 | News report, jobs, economy | A news report begins with footage of a "Help Wanted" sign. The report states that the U.S. economy added 517,000 new jobs in January. |
| 00:26 | News report, jobs, unemployment | The news report states that the unemployment rate dipped to 3.4%, the lowest level since 1969. |
| 00:27 | News report, jobs, sectors | The news report discusses the sectors that saw the most job growth, including leisure and hospitality, professional and business services, and government. |
| 00:28 | News report, jobs, wages | The news report discusses the increase in wages that has accompanied the job growth. |
| 00:29 | News report, jobs, outlook | The news report discusses the outlook for the job market in the coming months. |
| 00:30 | News report, Murdaugh trial, South Carolina | A news report begins with footage of Alex Murdaugh, a former lawyer who is on trial for the murders of his wife and son. |
| 00:31 | News report, Murdaugh trial, evidence | The news report discusses the evidence that has been presented in the trial, including testimony from a firearms examiner. |
| 00:32 | News report, Murdaugh trial, motive | The news report discusses the possible motive for the murders, including financial problems and a potential lawsuit. |
| 00:33 | News report, Murdaugh trial, family | The news report interviews members of the Murdaugh family. |
| 00:34 | News report, Murdaugh trial, verdict | The news report states that the jury is still deliberating and a verdict has not yet been reached. |
| 00:35 | News report, Robin Roberts, New Zealand | A news report begins with footage of Robin Roberts, co-anchor of "Good Morning America," standing on a platform high above the ground in New Zealand. |
| 00:36 | News report, Robin Roberts, New Zealand, bungee jump | The news report shows Robin Roberts preparing to bungee jump off the platform. |
| 00:37 | News report, Robin Roberts, New Zealand, fear | The news report discusses Robin Roberts' fear of heights and how she is overcoming it to do the bungee jump. |
| 00:38 | News report, Robin Roberts, New Zealand, jump | Robin Roberts jumps off the platform and falls towards the ground. |
| 00:39 | News report, Robin Roberts, New Zealand, reaction | Robin Roberts screams and laughs as she falls through the air. |
| 00:40 | News report, Robin Roberts, New Zealand, experience | Robin Roberts describes her experience bungee jumping as "exhilarating." |
| 00:41 | News report, weather, arctic blast, Northeast | The weather report continues with footage of snow falling and strong winds blowing. |
| 00:42 | Weather report, arctic blast, temperatures, wind chills | The weather report shows a map of the Northeast with the current temperatures and wind chills. |
| 00:43 | Weather report, arctic blast, safety, precautions | The weather report discusses the dangers of the cold weather and provides safety precautions for viewers. |
| 00:44 | Weather report, arctic blast, travel, disruptions | The weather report discusses the travel disruptions caused by the arctic blast, including flight cancellations and road closures. |
| 00:45 | Weather report, arctic blast, warming up | The weather report states that the arctic blast is expected to end soon and temperatures will begin to warm up. |
| 00:46 | News report, jobs, economy | The news report continues with footage of people working in various jobs. |
| 00:47 | News report, jobs, unemployment | The news report states that the unemployment rate is at a 50-year low. |
| 00:48 | News report, jobs, sectors | The news report discusses the sectors that saw the most job growth, including leisure and hospitality, professional and business services, and government. |
| 00:49 | News report, jobs, wages | The news report discusses the increase in wages that has accompanied the job growth. |
| 00:50 | News report, jobs, outlook | The news report discusses the outlook for the job market in the coming months. |
| 00:51 | News report, Murdaugh trial, South Carolina | The news report continues with footage of Alex Murdaugh in court. |
| 00:52 | News report, Murdaugh trial, evidence | The news report discusses the evidence that has been presented in the trial, including testimony from a firearms examiner. |
| 00:53 | News report, Murdaugh trial, motive | The news report discusses the possible motive for the murders, including financial problems and a potential lawsuit. |
| 00:54 | News report, Murdaugh trial, family | The news report interviews members of the Murdaugh family. |
| 00:55 | News report, Murdaugh trial, verdict | The news report states that the jury is still deliberating and a verdict has not yet been reached. |
| 00:56 | News report, Robin Roberts, New Zealand | The news report continues with footage of Robin Roberts standing on the platform high above the ground in New Zealand. |
| 00:57 | News report, Robin Roberts, New Zealand, bungee jump | The news report shows Robin Roberts preparing to bungee jump off the platform. |
| 00:58 | News report, Robin Roberts, New Zealand, fear | The news report discusses Robin Roberts' fear of heights and how she is overcoming it to do the bungee jump. |
| 00:59 | News report, Robin Roberts, New Zealand, jump | Robin Roberts jumps off the platform and falls towards the ground. |
| 01:00 | News report, Robin Roberts, New Zealand, reaction | Robin Roberts screams and laughs as she falls through the air. |
| 01:01 | News report, Robin Roberts, New Zealand, experience | Robin Roberts describes her experience bungee jumping as "exhilarating." |
| 01:02 | News report, weather, arctic blast, Northeast | The weather report continues with footage of snow falling and strong winds blowing. |
| 01:03 | Weather report, arctic blast, temperatures, wind chills | The weather report shows a map of the Northeast with the current temperatures and wind chills. |
| 01:04 | Weather report, arctic blast, safety, precautions | The weather report discusses the dangers of the cold weather and provides safety precautions for viewers. |
| 01:05 | Weather report, arctic blast, travel, disruptions | The weather report discusses the travel disruptions caused by the arctic blast, including flight cancellations and road closures. |
| 01:06 | Weather report, arctic blast, warming up | The weather report states that the arctic blast is expected to end soon and temperatures will begin to warm up. |
| 01:07 | News report, California, assault, arrest | The news report continues with footage of the crime scene. |
| 01:08 | News report, California, assault, victim | The news report shows a photo of the victim, Michael Mammone. |
| 01:09 | News report, California, assault, suspect | The news report shows a photo of the suspect, Vanroy Evan Smith. |
| 01:10 | News report, California, assault, investigation | The news report discusses the ongoing investigation into the assault. |
| 01:11 | News report, California, assault, community | The news report discusses the impact of the assault on the local community. |
| 01:12 | News report, United Airlines, Newark Airport, incident | The news report continues with footage of the two United Airlines jets with their wings clipped. |
| 01:13 | News report, United Airlines, Newark Airport, investigation | The news report discusses the ongoing investigation into the incident. |
| 01:14 | News report, United Airlines, Newark Airport, passengers | The news report interviews passengers who were on board the planes when the incident occurred. |
| 01:15 | News report, United Airlines, Newark Airport, safety | The news report discusses the safety implications of the incident. |
| 01:16 | News report, jobs, economy | The news report continues with footage of people working in various jobs. |
| 01:17 | News report, jobs, unemployment | The news report states that the unemployment rate is at a 50-year low. |
| 01:18 | News report, jobs, sectors | The news report discusses the sectors that saw the most job growth, including leisure and hospitality, professional and business services, and government. |
| 01:19 | News report, jobs, wages | The news report discusses the increase in wages that has accompanied the job growth. |
| 01:20 | News report, jobs, outlook | The news report discusses the outlook for the job market in the coming months. |
| 01:21 | News report, Murdaugh trial, South Carolina | The news report continues with footage of Alex Murdaugh in court. |
| 01:22 | News report, Murdaugh trial, evidence | The news report discusses the evidence that has been presented in the trial, including testimony from a firearms examiner. |
| 01:23 | News report, Murdaugh trial, motive | The news report discusses the possible motive for the murders, including financial problems and a potential lawsuit. |
| 01:24 | News report, Murdaugh trial, family | The news report interviews members of the Murdaugh family. |
| 01:25 | News report, Murdaugh trial, verdict | The news report states that the jury is still deliberating and a verdict has not yet been reached. |
| 01:26 | News report, Robin Roberts, New Zealand | The news report continues with footage of Robin Roberts standing on the platform high above the ground in New Zealand. |
| 01:27 | News report, Robin Roberts, New Zealand, bungee jump | The news report shows Robin Roberts preparing to bungee jump off the platform. |
| 01:28 | News report, Robin Roberts, New Zealand, fear | The news report discusses Robin Roberts' fear of heights and how she is overcoming it to do the bungee jump. |
| 01:29 | News report, Robin Roberts, New Zealand, jump | Robin Roberts jumps off the platform and falls towards the ground. |
| 01:30 | News report, Robin Roberts, New Zealand, reaction | Robin Roberts screams and laughs as she falls through the air. |
| 01:31 | News report, Robin Roberts, New Zealand, experience | Robin Roberts describes her experience bungee jumping as "exhilarating." |
| 01:32 | Commercial, Metamucil, digestive health | A commercial for Metamucil begins with a woman sitting at her kitchen counter reading a newspaper. She looks tired and sluggish. |
| 01:33 | Commercial, Metamucil, benefits | The commercial states that Metamucil can help you feel less sluggish and weighed down. |
| 01:34 | Commercial, Metamucil, benefits | The commercial states that Metamucil can help promote healthy blood sugar levels. |
| 01:35 | Commercial, Metamucil, benefits | The commercial states that Metamucil can help lower cholesterol. |
| 01:36 | Commercial, Metamucil, 2-week challenge | The commercial encourages viewers to sign up for the Metamucil 2-week challenge. |
| 01:37 | Commercial, Angie, home services | A commercial for Angie begins with a father and son playing basketball in their driveway. |
| 01:38 | Commercial, Angie, website | The commercial shows the father using the Angie website to find a handyman to fix their garage door. |
| 01:39 | Commercial, Angie, app | The commercial shows the Angie app and how it can be used to find and book home services. |
| 01:40 | Commercial, Angie, happiness guarantee | The commercial states that Angie offers a happiness guarantee. |
| 01:41 | Commercial, Ashley Furniture HomeStore, Tempur-Pedic | A commercial for Ashley Furniture HomeStore begins with a dachshund dog in a robe lying in bed looking tired. |
| 01:42 | Commercial, Ashley Furniture HomeStore, Tempur-Pedic | The commercial shows the dog's owners buying a new Tempur-Pedic mattress from Ashley Furniture HomeStore. |
| 01:43 | Commercial, Ashley Furniture HomeStore, Tempur-Pedic | The commercial shows the dog sleeping soundly on the new mattress. |
| 01:44 | Commercial, Vicks VapoRub, cough relief | A commercial for Vicks VapoRub begins with a man coughing in bed. |
| 01:45 | Commercial, Vicks VapoRub, benefits | The commercial states that VapoRub can provide fast-acting cough relief. |
| 01:46 | Commercial, Red Lobster, seafood | A commercial for Red Lobster begins with a waiter serving a plate of lobster and shrimp tacos. |
| 01:47 | Commercial, Red Lobster, lobster | The commercial shows a close-up of the lobster and shrimp tacos. |
| 01:48 | Commercial, Red Lobster, new menu items | The commercial states that Red Lobster has new lobster and shrimp tacos on the menu. |
| 01:49 | Commercial, Red Lobster, fun dining | The commercial shows people enjoying their meals at Red Lobster. |
| 01:50 | Commercial, Ancestry, family history | A commercial for Ancestry begins with a woman and her daughter sitting in a museum. |
| 01:51 | Commercial, Ancestry, historical records | The commercial shows the woman and her daughter looking at historical records on a tablet. |
| 01:52 | Commercial, Ancestry, family story | The commercial states that Ancestry can help you bring your family story to life. |
| 01:53 | News report, weather, arctic blast, Northeast | The weather report continues with footage of snow falling and strong winds blowing. |
| 01:54 | Weather report, arctic blast, temperatures, wind chills | The weather report shows a map of the Northeast with the current temperatures and wind chills. |
| 01:55 | Weather report, arctic blast, safety, precautions | The weather report discusses the dangers of the cold weather and provides safety precautions for viewers. |
| 01:56 | Weather report, arctic blast, travel, disruptions | The weather report discusses the travel disruptions caused by the arctic blast, including flight cancellations and road closures. |
| 01:57 | Weather report, arctic blast, warming up | The weather report states that the arctic blast is expected to end soon and temperatures will begin to warm up. |
| 01:58 | News report, California, assault, arrest | The news report continues with footage of the crime scene. |
| 01:59 | News report, California, assault, victim | The news report shows a photo of the victim, Michael Mammone. |
| 02:00 | News report, California, assault, suspect | The news report shows a photo of the suspect, Vanroy Evan Smith. |
| 02:01 | News report, California, assault, investigation | The news report discusses the ongoing investigation into the assault. |
| 02:02 | News report, California, assault, community | The news report discusses the impact of the assault on the local community. |
| 02:03 | News report, United Airlines, Newark Airport, incident | The news report continues with footage of the two United Airlines jets with their wings clipped. |
| 02:04 | News report, United Airlines, Newark Airport, investigation | The news report discusses the ongoing investigation into the incident. |
| 02:05 | News report, United Airlines, Newark Airport, passengers | The news report interviews passengers who were on board the planes when the incident occurred. |
| 02:06 | News report, United Airlines, Newark Airport, safety | The news report discusses the safety implications of the incident. |
| 02:07 | News report, jobs, economy | The news report continues with footage of people working in various jobs. |
| 02:08 | News report, jobs, unemployment | The news report states that the unemployment rate is at a 50-year low. |
| 02:09 | News report, jobs, sectors | The news report discusses the sectors that saw the most job growth, including leisure and hospitality, professional and business services, and government. |
| 02:10 | News report, jobs, wages | The news report discusses the increase in wages that has accompanied the job growth. |
| 02:11 | News report, jobs, outlook | The news report discusses the outlook for the job market in the coming months. |
| 02:12 | News report, Robin Roberts, New Zealand | The news report continues with footage of Robin Roberts standing on the platform high above the ground in New Zealand. |
| 02:13 | News report, Robin Roberts, New Zealand, bungee jump | The news report shows Robin Roberts preparing to bungee jump off the platform. |
| 02:14 | News report, Robin Roberts, New Zealand, fear | The news report discusses Robin Roberts' fear of heights and how she is overcoming it to do the bungee jump. |
| 02:15 | News report, Robin Roberts, New Zealand, jump | Robin Roberts jumps off the platform and falls towards the ground. |
| 02:16 | News report, Robin Roberts, New Zealand, reaction | Robin Roberts screams and laughs as she falls through the air. |
| 02:17 | News report, Robin Roberts, New Zealand, experience | Robin Roberts describes her experience bungee jumping as "exhilarating." |
| 02:18 | Commercial, Metamucil, digestive health | The commercial for Metamucil continues with the woman stirring the supplement into a glass of water. |
| 02:19 | Commercial, Metamucil, benefits | The commercial shows an animation of Metamucil working in the digestive system. |
| 02:20 | Commercial, Metamucil, benefits | The commercial states that Metamucil can help promote healthy blood sugar levels. |
| 02:21 | Commercial, Metamucil, benefits | The commercial states that Metamucil can help lower cholesterol. |
| 02:22 | Commercial, Metamucil, 2-week challenge | The commercial encourages viewers to sign up for the Metamucil 2-week challenge. |
| 02:23 | Commercial, Angie, home services | The commercial for Angie continues with the father and son playing basketball in their driveway. |
| 02:24 | Commercial, Angie, website | The commercial shows the father using the Angie website to find a handyman to fix their garage door. |
| 02:25 | Commercial, Angie, app | The commercial shows the Angie app and how it can be used to find and book home services. |
| 02:26 | Commercial, Angie, happiness guarantee | The commercial states that Angie offers a happiness guarantee. |
| 02:27 | Commercial, Ashley Furniture HomeStore, Tempur-Pedic | The commercial for Ashley Furniture HomeStore continues with the dachshund dog sleeping soundly on the new mattress. |
| 02:28 | Commercial, Ashley Furniture HomeStore, Tempur-Pedic | The commercial shows the dog's owners sleeping soundly on their new Tempur-Pedic mattress. |
| 02:29 | Commercial, Vicks VapoRub, cough relief | The commercial for Vicks VapoRub continues with the man sleeping soundly after using the product. |
| 02:30 | Commercial, Vicks VapoRub, benefits | The commercial states that VapoRub can help you breathe easier and fall asleep. |
| 02:31 | Commercial, Red Lobster, seafood | The commercial for Red Lobster continues with the family enjoying their meal. |
| 02:32 | Commercial, Red Lobster, lobster | The commercial shows a close-up of the lobster and shrimp tacos. |
| 02:33 | Commercial, Red Lobster, new menu items | The commercial states that Red Lobster has new lobster and shrimp tacos on the menu. |
| 02:34 | Commercial, Red Lobster, fun dining | The commercial shows people enjoying their meals at Red Lobster. |
| 02:35 | Commercial, Ancestry, family history | The commercial for Ancestry continues with the woman and her daughter looking at historical records on a tablet. |
| 02:36 | Commercial, Ancestry, family story | The commercial states that Ancestry can help you bring your family story to life. |
| 02:37 | News report, weather, arctic blast, Northeast | The weather report continues with footage of snow falling and strong winds blowing. |
| 02:38 | Weather report, arctic blast, temperatures, wind chills | The weather report shows a map of the Northeast with the current temperatures and wind chills. |
| 02:39 | Weather report, arctic blast, safety, precautions | The weather report discusses the dangers of the cold weather and provides safety precautions for viewers. |
| 02:40 | Weather report, arctic blast, travel, disruptions | The weather report discusses the travel disruptions caused by the arctic blast, including flight cancellations and road closures. |
| 02:41 | Weather report, arctic blast, warming up | The weather report states that the arctic blast is expected to end soon and temperatures will begin to warm up. |
| 02:42 | News report, California, assault, arrest | The news report continues with footage of the crime scene. |
| 02:43 | News report, California, assault, victim | The news report shows a photo of the victim, Michael Mammone. |
| 02:44 | News report, California, assault, suspect | The news report shows a photo of the suspect, Vanroy Evan Smith. |
| 02:45 | News report, California, assault, investigation | The news report discusses the ongoing investigation into the assault. |
| 02:46 | News report, California, assault, community | The news report discusses the impact of the assault on the local community. |
| 02:47 | News report, United |
Let's try again, but use a one minute, rather than one second, time interval given the length of the video:
Describe this video minute-by-minute in a table, where each row is one minute of airtime. The first column should be the offset from the start of the video. The second column should be a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that minute of airtime. The third column should be a detailed textual description of that minute of airtime that describes it in sufficient detail to someone who can't see the video.
This yields a complete summary:
Minute-by-Minute Description of Video
| Offset | Keywords | Description |
| 00:00 | National University, education, online learning, scholarships | An advertisement for National University, an online university that offers scholarships and supports the "whole you." |
| 01:00 | Breaking news, China, spy balloon, Pentagon, surveillance | Breaking news report about a Chinese spy balloon being tracked over the U.S. The balloon is described as being the size of three buses and having high-resolution cameras and solar panels. The Pentagon says it is a surveillance balloon and can be maneuvered. |
| 02:00 | China, spy balloon, weather balloon, research | China claims the balloon is a civilian airship used for research, mainly meteorological purposes. |
| 03:00 | Montana, Billings, residents react to spy balloon | Residents of Billings, Montana react to the news of the Chinese spy balloon. |
| 04:00 | China, spy balloon, technology bay, high-resolution cameras, solar panels | More details about the technology bay of the Chinese spy balloon are revealed, including high-resolution cameras and solar panels. |
| 05:00 | China, spy balloon, surveillance, Secretary of State Blinken, trip postponed | Secretary of State Antony Blinken postpones his trip to China in response to the balloon. |
| 06:00 | Montana, Billings, residents react to spy balloon, concerns | Residents of Billings, Montana express their concerns about the Chinese spy balloon. |
| 07:00 | White House, President Biden, China, spy balloon, briefings | The White House says that President Biden is receiving regular briefings on the Chinese spy balloon. |
| 08:00 | China, spy balloon, surveillance, maneuverability | More details about the Chinese spy balloon are revealed, including its maneuverability. |
| 09:00 | Northeast, arctic blast, wind chill, blizzard warning | A report about a "once-in-a-generation" arctic blast that is hitting the Northeast. The wind chill is expected to be below zero in many areas, and there is a blizzard warning in effect for parts of Maine. |
| 10:00 | Boston, Massachusetts, cold weather, shelters | People in Boston, Massachusetts are trying to stay warm during the arctic blast. Shelters are open for those who need them. |
| 11:00 | Northeast, arctic blast, wind chill, weather map | A weather map shows the wind chill forecast for the Northeast. |
| 12:00 | California, Los Angeles, deadly assault, arrest, bicyclist, stabbing | A report about a deadly assault arrest in California. A bicyclist was hit by a car and then fatally stabbed by the driver. The driver, Vanroy Evan Smith, was detained by bystanders and is now facing murder charges. |
| 13:00 | United Airlines, Newark Airport, planes clip wings, FAA investigation | Two United Airlines planes clip wings at Newark Airport. The FAA is investigating the incident. |
| 14:00 | Economy, jobs report, unemployment rate | A report about the January jobs report. The U.S. economy added 517,000 new jobs in January, and the unemployment rate dipped to 3.4%, the lowest since 1969. |
| 15:00 | New York City, job market, economy | People in New York City react to the news of the strong jobs report. |
| 16:00 | South Carolina, Murdaugh double murder trial, new developments | New developments in the Murdaugh double murder trial in South Carolina. Prosecutors say the defendant got millions in a past settlement. |
| 17:00 | South Carolina, Murdaugh double murder trial, testimony, firearms examiner | A firearms examiner testifies in the Murdaugh double murder trial. |
| 18:00 | New York City, cold weather, homeless population | A report about the impact of the arctic blast on the homeless population in New York City. |
| 19:00 | Robin Roberts, New Zealand, adventure, bungee jump, skywalk | Robin Roberts goes on an adventure in New Zealand, including a bungee jump and a skywalk. |
| 20:00 | Eliquis, blood thinner, stroke prevention, atrial fibrillation | An advertisement for Eliquis, a blood thinner that can help prevent strokes in people with atrial fibrillation. |
| 21:00 | WeatherTech, car mats, floor liners | An advertisement for WeatherTech car mats and floor liners. |
| 22:00 | Carfax, vehicle history reports | An advertisement for Carfax vehicle history reports. |
| 23:00 | Consumer Cellular, cell phone plans | An advertisement for Consumer Cellular cell phone plans. |
| 24:00 | Dupixent, eczema treatment | An advertisement for Dupixent, a prescription medication used to treat eczema. |
| 25:00 | Nature's Bounty, vitamins and supplements | An advertisement for Nature's Bounty vitamins and supplements. |
| 26:00 | Fidelity Investments, wealth management | An advertisement for Fidelity Investments wealth management services. |
| 27:00 | Swiffer WetJet, floor cleaner | An advertisement for Swiffer WetJet floor cleaner. |
| 28:00 | Coricidin HBP, cold and flu medicine | An advertisement for Coricidin HBP cold and flu medicine. |
| 29:00 | Oakland, California, school safety, prostitution | A report about concerns over prostitution happening outside a school in Oakland, California. |
| 30:00 | Weather forecast, another storm |
And a second time to test consistency:
Describe this video minute-by-minute in a table, where each row is one minute of airtime. The first column should be the offset from the start of the video. The second column should be a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that minute of airtime. The third column should be a detailed textual description of that minute of airtime that describes it in sufficient detail to someone who can't see the video.
Unfortunately, despite asking it to break the video down into one-minute increments, here is much choppier:
| Offset | Keywords | Description |
| 00:00 | National University, WSCUC-Accredited | A commercial for National University, highlighting its WSCUC accreditation and its commitment to supporting the whole student. |
| 01:10 | Breaking News, China Spy Balloon, David Muir | ABC World News Tonight with David Muir begins with breaking news about a suspected Chinese spy balloon over the U.S. |
| 02:23 | Breaking News, China Spy Balloon, World News Tonight | Continued coverage of the Chinese spy balloon, with footage and analysis. |
| 03:40 | Breaking News, Chinese Spy Balloon, Shootdown, Atlantic Ocean | Breaking news update: the U.S. has shot down the Chinese spy balloon over the Atlantic Ocean. |
| 05:00 | Breaking News, Chinese Spy Balloon, Pentagon, Surveillance, Maneuver | Pentagon officials confirm the balloon was a surveillance device and that it was capable of maneuvering. |
| 06:27 | Breaking News, Once-in-a-Generation Arctic Blast, Northeast, Wind Chills | Coverage of the "once-in-a-generation" Arctic blast affecting the Northeast, with wind chills as low as -50 degrees Fahrenheit. |
| 08:32 | Once-in-a-Generation Arctic Blast, Northeast, Wind Chills, Boston, Warming Up | Continued coverage of the Arctic blast, with reports from Boston and a forecast for warmer weather this weekend. |
| 10:05 | Deadly Assault Arrest, Bicyclist, Stabbing, California | Developing news about a deadly assault in California, where a bicyclist was hit by a car and then fatally stabbed by the driver. |
| 12:22 | Deadly Assault Arrest, Vanroy Evan Smith, Mammone, Knife, Bystanders | Details about the deadly assault, including the suspect's name (Vanroy Evan Smith) and the victim's name (Dr. Michael Mammone). Bystanders detained Smith until police arrived. |
| 14:15 | New Developments, Jobs Added, Unemployment, January | New economic data shows that 517,000 new jobs were added in January, and the unemployment rate dipped to 3.4%, the lowest since 1969. |
| 15:36 | New Developments, Jobs Added, Unemployment, Leisure & Hospitality, Professional & Business Services | Continued coverage of the strong jobs report, highlighting job gains in various sectors. |
| 17:37 | MetaMucil, Digestive System, 2 Week Challenge | A commercial for MetaMucil, a fiber supplement that promotes digestive health. |
| 18:53 | Red Lobster, Fun Dining, Lobster & Shrimp Tacos, Lobsessed | A commercial for Red Lobster, featuring their new lobster and shrimp tacos. |
| 20:40 | WeatherTech, Floor Mats, Seat Protectors | A commercial for WeatherTech car floor mats and seat protectors. |
| 22:41 | The Index, Monkey, Dallas Zoo | A segment called "The Index" features a story about a missing monkey from the Dallas Zoo. |
| 24:45 | Dupixent, Eczema, Asthma | A commercial for Dupixent, a prescription medication for eczema and asthma. |
| 26:30 | Eliquis, Blood Thinner, Stroke Risk, Atrial Fibrillation | A commercial for Eliquis, a blood thinner that reduces the risk of stroke in people with atrial fibrillation. |
| 28:49 | World News Tonight with David Muir | The end of the program, with David Muir thanking viewers for watching. |
And again:
Describe this video minute-by-minute in a table, where each row is one minute of airtime. The first column should be the offset from the start of the video. The second column should be a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that minute of airtime. The third column should be a detailed textual description of that minute of airtime that describes it in sufficient detail to someone who can't see the video.
Once again it chops the video into uneven boundaries:
| Offset | Keywords | Description |
| 00:00 | National University, WSCUC-Accredited | A commercial for National University, highlighting its WSCUC accreditation and its commitment to supporting the whole student. |
| 01:10 | Breaking News, China Spy Balloon, Montana | Breaking news coverage of a suspected Chinese spy balloon spotted over Montana. David Muir reports on the developing story, with footage of the balloon and interviews with officials and residents. |
| 02:23 | Breaking News, China Spy Balloon, Pentagon, Secretary Blinken | Continued coverage of the Chinese spy balloon, with updates from the Pentagon and Secretary of State Antony Blinken. |
| 03:40 | Breaking News, China Spy Balloon, Shootdown | Breaking news that the U.S. is considering shooting down the Chinese spy balloon over the Atlantic Ocean. |
| 04:43 | Breaking News, China Spy Balloon, Technology Bay | A detailed look at the technology bay of the Chinese spy balloon, including its high-resolution cameras and solar panels. |
| 05:45 | Breaking News, China Spy Balloon, Pentagon, China Statement | The Pentagon confirms that the balloon is a surveillance device, while China maintains it is a civilian weather balloon. |
| 06:47 | Once-in-a-Generation Arctic Blast, Northeast | Coverage of a dangerous arctic blast affecting the Northeast, with wind chills dropping to dangerously low levels. |
| 08:32 | Once-in-a-Generation Arctic Blast, Boston, New York | Reports from Boston and New York City, where residents are bracing for the extreme cold. |
| 09:24 | Once-in-a-Generation Arctic Blast, Wind Chills, Northeast | Meteorologist Mike Wankum provides a forecast for the Northeast, highlighting the brutal wind chills. |
| 10:05 | Once-in-a-Generation Arctic Blast, Boston, Massachusetts General Hospital | A report from Boston's Massachusetts General Hospital, where doctors are treating patients for cold-related injuries. |
| 11:00 | Dangerous Cold, Wind Chill Warning, Blizzard Warning | A map shows the areas under wind chill and blizzard warnings due to the arctic blast. |
| 12:06 | Deadly Assault Arrest, California, Bicyclist, Stabbing | Breaking news coverage of a deadly assault in California, where a bicyclist was hit by a car and then fatally stabbed by the driver. |
| 13:05 | United Jets Clip Wings, Newark Airport | Breaking news of two United Airlines jets clipping wings at Newark Liberty International Airport. |
| 14:15 | 517,000 New Jobs Added, January, Unemployment | A report on the strong January jobs report, with 517,000 new jobs added and the unemployment rate dipping to 3.4%. |
| 15:35 | New Developments, Murdaugh Double Murder Trial, South Carolina | Updates from the Murdaugh double murder trial in South Carolina, where prosecutors allege the defendant received millions in a past settlement. |
| 17:23 | Robin's Epic New Zealand Journey, Skywalk, Bungee Jump | Robin Roberts takes on a skywalk and bungee jump in New Zealand. |
| 19:09 | The Index, Dallas Zoo, Monkeys | A segment called "The Index" features a story about two monkeys that were stolen from the Dallas Zoo and later found. |
| 20:40 | WeatherTech, Floor Mats, Car Accessories | A commercial for WeatherTech car accessories, highlighting their floor mats and other products. |
| 21:25 | Consumer Cellular, Target, Cell Phone Plans | A commercial for Consumer Cellular, a cell phone service provider available at Target. |
| 22:30 | Ancestry, Family History, Black History Month | A commercial for Ancestry, encouraging viewers to explore their family history during Black History Month. |
| 23:30 | Coricidin HBP, Cold and Flu Relief, High Blood Pressure | A commercial for Coricidin HBP, a cold and flu relief medication for people with high blood pressure. |
| 24:30 | Dupixent, Eczema, Asthma | A commercial for Dupixent, a prescription medication for eczema and asthma. |
| 25:30 | GoLo, Weight Loss, Diet and Exercise | A commercial for GoLo, a weight loss program. |
| 26:46 | Tonight 6PM, ABC 7 News | A promo for the upcoming ABC 7 News at 6PM. |
| 28:56 | Tonight 6PM, ABC 7 News, I-Team Exclusive | A promo for an I-Team exclusive report airing on ABC 7 News at 6PM. |
| 29:43 | Concerns Outside School, Oakland, St. Anthony Catholic School |
Let's change tactics and instead ask for Gemini to break the video into stories, rather than by airtime:
Break this video down into individual discrete stories. Output a table in which each row is one story. The first column is the start time of the story. The second column is the end time of the story. The third column is a descriptive title of the story. The fourth column is a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that story. The fifth column should be a detailed textual description of that story that describes it in sufficient detail to someone who can't see the video.
This time we get perhaps the best results yet that closely track Vanderbilt's human summarization:
| Start Time | End Time | Title | Keywords | Description |
| 00:00 | 01:10 | National University Commercial | National University, WSCUC-Accredited | A commercial for National University, highlighting its WSCUC accreditation and its commitment to supporting the whole student. |
| 01:10 | 03:40 | U.S. Tracks Suspected Chinese Spy Balloon Over Montana | Breaking News, China Spy Balloon, Montana, Pentagon, Secretary Blinken | Breaking news coverage of a suspected Chinese spy balloon spotted over Montana. David Muir reports on the developing story, with footage of the balloon and interviews with officials and residents. The Pentagon confirms that the balloon is a surveillance device, while China maintains it is a civilian weather balloon. Secretary of State Antony Blinken postpones his trip to China in response to the incident. |
| 03:40 | 06:47 | Once-in-a-Generation Arctic Blast Hits Northeast | Once-in-a-Generation Arctic Blast, Northeast, Boston, New York, Wind Chills, Blizzard Warning | Coverage of a dangerous arctic blast affecting the Northeast, with wind chills dropping to dangerously low levels. Reports from Boston and New York City show residents bracing for the extreme cold. Meteorologist Mike Wankum provides a forecast, highlighting the brutal wind chills and areas under wind chill and blizzard warnings. |
| 06:47 | 07:23 | Deadly Assault Arrest in California | Deadly Assault Arrest, California, Bicyclist, Stabbing | Breaking news coverage of a deadly assault in California, where a bicyclist was hit by a car and then fatally stabbed by the driver. Police have arrested Vanroy Evan Smith, who is accused of attacking the victim, Michael John Mammone, with a knife. Mammone died at the hospital. |
| 07:23 | 08:32 | United Jets Clip Wings at Newark Airport | United Jets Clip Wings, Newark Airport, Boeing 757-200, Delta DL1943, FAA Investigation | Breaking news of two United Airlines jets clipping wings at Newark Liberty International Airport. One of the planes was empty and being towed when it struck the other plane, which was waiting at the gate and preparing to leave for Orlando. The FAA is investigating the incident. |
| 08:32 | 09:24 | Strong January Jobs Report | 517,000 New Jobs Added, January, Unemployment | A report on the strong January jobs report, with 517,000 new jobs added and the unemployment rate dipping to 3.4%, the lowest level since 1969. President Biden touts the job gains as evidence of a strong economy. |
| 09:24 | 10:05 | Murdaugh Double Murder Trial Continues | Murdaugh Double Murder Trial, South Carolina, Prosecutors, Defense Attorney, Past Settlement | Updates from the Murdaugh double murder trial in South Carolina, where prosecutors allege the defendant received millions in a past settlement. The defense attorney argues that the evidence is circumstantial. |
| 10:05 | 11:32 | Robin Roberts' Epic New Zealand Journey | Robin's Epic New Zealand Journey, Skywalk, Bungee Jump | Robin Roberts takes on a skywalk and bungee jump in New Zealand. |
| 11:32 | 12:30 | The Index: Monkeys Found After Being Stolen from Dallas Zoo | The Index, Dallas Zoo, Monkeys | A segment called "The Index" features a story about two monkeys that were stolen from the Dallas Zoo and later found. |
| 12:30 | 14:14 | Concerns About Safety Outside Oakland School | I-Team Exclusive, Oakland, St. Anthony Catholic School, Prostitution, Drug Use | An I-Team exclusive report on concerns about safety outside St. Anthony Catholic School in Oakland, where parents and staff have witnessed prostitution and drug use. |
| 14:14 | 15:34 | Another Storm Headed for California | Another Storm, California, Weather Forecast | Meteorologist Sandhya Patel provides a forecast for California, warning of another storm system headed for the state. |
| 15:34 | 17:23 | Murdaugh Double Murder Trial: New Developments | Murdaugh Double Murder Trial, South Carolina, Prosecutors, Defense Attorney, Past Settlement, Satterfield Family | Continued coverage of the Murdaugh double murder trial, with prosecutors presenting evidence about a past settlement the defendant received and the defense attorney cross-examining witnesses. |
| 17:23 | 19:09 | Robin Roberts Skywalks and Bungee Jumps in New Zealand | Robin's Epic New Zealand Journey, Skywalk, Bungee Jump | Robin Roberts completes her skywalk and bungee jump in New Zealand. |
| 19:09 | 20:40 | The Index: Owl Rescued from New York City Building | The Index, Owl, New York City | A segment called "The Index" features a story about an owl that was rescued from a building in New York City. |
| 20:40 | 22:30 | Coricidin HBP Commercial | Coricidin HBP, Cold and Flu Relief, High Blood Pressure | A commercial for Coricidin HBP, a cold and flu relief medication for people with high blood pressure. |
| 22:30 | 24:24 | Dupixent Commercial | Dupixent, Eczema, Asthma | A commercial for Dupixent, a prescription medication for eczema and asthma. |
| 24:24 | 25:30 | Progressive Insurance Commercial | Progressive, Home Insurance, TV Dad | A humorous commercial for Progressive home insurance, featuring the "TV Dad" character. |
| 25:30 | 26:46 | GoLo Commercial | GoLo, Weight Loss, Diet and Exercise | A commercial for GoLo, a weight loss program. |
| 26:46 | 27:23 | Eliquis Commercial | Eliquis, Atrial Fibrillation, Stroke Risk, Blood Clots | A commercial for Eliquis, a blood thinner medication used to reduce the risk of stroke in people with atrial fibrillation. |
| 27:23 | 28:55 | Robin Roberts Bungee Jumps in New Zealand | Robin's Epic New Zealand Journey, Skywalk, Bungee Jump | Robin Roberts bungee jumps off the Sky Tower in Auckland, New Zealand. |
| 28:55 | 30:08 | ABC 7 News I-Team Exclusive: Concerns Outside Oakland School |
And again:
Break this video down into individual discrete stories. Output a table in which each row is one story. The first column is the start time of the story. The second column is the end time of the story. The third column is a descriptive title of the story. The fourth column is a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that story. The fifth column should be a detailed textual description of that story that describes it in sufficient detail to someone who can't see the video.
Unfortunately, this time it truncates:
World News Tonight with David Muir – February 3, 2023
| Start Time | End Time | Title | Keywords | Description |
| 00:00 | 00:08 | National University: Supporting the Whole You | National University, online education, flexible learning, 4-week courses, scholarships | This ad for National University highlights the school's commitment to supporting the whole student, offering online and flexible learning options, 4-week courses, and scholarships. The ad features a montage of students and graduates in various settings, including at home, at work, and in the classroom. |
| 00:10 | 00:27 | Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, U.S., national security, Pentagon, Montana | This news segment reports on a suspected Chinese spy balloon that is being tracked by the U.S. military as it travels across the continental United States. The balloon was first spotted over Montana, and officials are concerned that it could be used to gather intelligence on sensitive military sites. |
| 00: |
And again:
Break this video down into individual discrete stories. Output a table in which each row is one story. The first column is the start time of the story. The second column is the end time of the story. The third column is a descriptive title of the story. The fourth column is a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that story. The fifth column should be a detailed textual description of that story that describes it in sufficient detail to someone who can't see the video.
Again it truncates:
World News Tonight with David Muir – February 3, 2023
| Start Time | End Time | Title | Keywords | Description |
| 00:00 | 00:08 | National University Commercial | National University, education, online learning, 4-week courses, scholarships | A commercial for National University, an online university that offers 4-week courses and scholarships. The commercial features a montage of students and graduates of National University, highlighting the flexibility and affordability of their programs. |
| 00:10 | 00:26 | Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, surveillance, Pentagon, U.S. | David Muir reports on a suspected Chinese spy balloon that is being tracked by the Pentagon over the continental United States. The balloon is believed to be collecting intelligence, and China has denied that it is a spy balloon, claiming it is a civilian weather balloon that drifted off course. |
| 00:27 | 01:13 | "Once-in-a-Generation" Arctic Blast | Arctic blast, winter storm, Northeast, cold weather, wind chill | David Muir reports on a "once-in-a-generation" arctic blast that is bringing dangerously cold temperatures and wind chills to the Northeast. The storm is expected to cause widespread power outages and travel disruptions. |
| 01:14 | 01:24 | Deadly Assault Arrest | California, road rage, stabbing, murder, arrest | David Muir reports on the arrest of a man in California who is accused of fatally stabbing a bicyclist after a road rage incident. The victim, Dr. Michael Mammone, was a well-respected physician at Providence Mission Hospital. |
| 01:25 | 01:44 | United Jets Clip Wings | Newark Airport, United Airlines, Delta Air Lines, plane collision, FAA investigation | David Muir reports on an incident at Newark Airport in which two United Airlines jets clipped wings while taxiing. No injuries were reported, and the FAA is investigating the incident. |
| 01:45 | 02:10 | 517,000 New Jobs Added | U.S. economy, jobs report, unemployment, January 2023 | Rebecca Jarvis reports on the January 2023 jobs report, which showed that the U.S. economy added 517,000 new jobs, and the unemployment rate dipped to 3.4%, the lowest level since 1969. |
| 02:11 | 02:30 | Robin's Epic New Zealand Journey | Robin Roberts, New Zealand, travel, adventure, bungee jumping | Robin Roberts takes viewers on her epic journey to New Zealand, where she goes bungee jumping and experiences the country's stunning scenery. |
| 02:31 | 03:40 | Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, surveillance, Pentagon, U.S., Montana, Billings | David Muir continues his report on the suspected Chinese spy balloon, with new details about the balloon's technology and its path across the United States. The balloon is believed to be carrying high-resolution cameras and solar panels, and it has been spotted over Montana and Billings. |
| 03:41 | 04:40 | "Once-in-a-Generation" Arctic Blast | Arctic blast, winter storm, Northeast, cold weather, wind chill, Boston, Massachusetts, New York | David Muir continues his report on the arctic blast, with updates on the storm's impact on the Northeast. The storm is causing widespread power outages and travel disruptions, and temperatures are expected to remain dangerously cold through the weekend. |
| 04:41 | 05:50 | Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, surveillance, Pentagon, U.S., Montana, Billings, Secretary of State Blinken | David Muir continues his report on the suspected Chinese spy balloon, with new information from the Pentagon and Secretary of State Antony Blinken. The Pentagon says the balloon is "maneuverable," and Blinken has postponed his trip to China in response to the incident. |
| 05:51 | 07:17 | "Once-in-a-Generation" Arctic Blast | Arctic blast, winter storm, Northeast, cold weather, wind chill, Boston, Massachusetts, New York, Mount Washington Observatory | David Muir and Rob Marciano report on the arctic blast, with updates on the storm's impact on the Northeast. The storm is causing widespread power outages and travel disruptions, and temperatures are expected to remain dangerously cold through the weekend. |
| 07:18 | 08:30 | Deadly Assault Arrest | California, road rage, stabbing, murder, arrest, Dr. Michael Mammone, Providence Mission Hospital | Matt Gutman reports on the arrest of a man in California who is accused of fatally stabbing a bicyclist after a road rage incident. The victim, Dr. Michael Mammone, was a well-respected physician at Providence Mission Hospital. |
| 08:31 | 09:40 | "Once-in-a-Generation" Arctic Blast | Arctic blast, winter storm, Northeast, cold weather, wind chill, Boston, Massachusetts, New York, Mount Washington Observatory, Boston | David Muir and Victor Oquendo report on the arctic blast, with updates on the storm's impact on the Northeast. The storm is causing widespread power outages and travel disruptions, and temperatures are expected to remain dangerously cold through the weekend. |
| 09:41 | 10:07 | "Once-in-a-Generation" Arctic Blast | Arctic blast, winter storm, Northeast, cold weather, wind chill, Boston, Massachusetts | David Muir and Mike Wankum report on the arctic blast, with updates on the storm's impact on the Northeast. The storm is causing widespread power outages and travel disruptions, and temperatures are expected to remain dangerously cold through the weekend. |
| 10:08 | 10:40 | "Once-in-a-Generation" Arctic Blast | Arctic blast, winter storm, Northeast, cold weather, wind chill, Boston, Massachusetts, Dr. Derek Monette, Massachusetts General Hospital | David Muir and Dr. Derek Monette of Massachusetts General Hospital report on the arctic blast, with updates on the storm's impact on the Northeast. The storm is causing widespread power outages and travel disruptions, and temperatures are expected to remain dangerously cold through the weekend. |
| 10:41 | 11:30 | "Once-in-a-Generation" Arctic Blast | Arctic blast, winter storm, Northeast, cold weather, wind chill, Boston, Massachusetts, New York, Syracuse, Albany, Burlington, Caribou | David Muir and Rob Marciano report on the arctic blast, with updates on the storm's impact on the Northeast. The storm is causing widespread power outages and travel disruptions, and temperatures are expected to remain dangerously cold through the weekend. |
| 11:31 | 12:30 | Deadly Assault Arrest | California, road rage, stabbing, murder, arrest, Dr. Michael Mammone, Providence Mission Hospital, Vanroy Evan Smith, Orange County Sheriff's Department | Matt Gutman reports on the arrest of a man in California who is accused of fatally stabbing a bicyclist after a road rage incident. The victim, Dr. Michael Mammone, was a well-respected physician at Providence Mission Hospital. The suspect, Vanroy Evan Smith, was detained by bystanders and arrested by the Orange County Sheriff's Department. |
| 12:31 | 13:40 | United Jets Clip Wings | Newark Airport, United Airlines, Delta Air Lines, plane collision, FAA investigation, Boeing 757-200, Rebecca Blum | Gio Benitez reports on an incident at Newark Airport in which two United Airlines jets clipped wings while taxiing. No injuries were reported, and the FAA is investigating the incident. Passenger Rebecca Blum describes feeling a "jolt" when the planes collided. |
| 13:41 | 15:30 | 517,000 New Jobs Added | U.S. economy, jobs report, unemployment, January 2023, President Biden, leisure and hospitality, professional and business services, government, health care | Rebecca Jarvis reports on the January 2023 jobs report, which showed that the U.S. economy added 517,000 new jobs, and the unemployment rate dipped to 3.4%, the lowest level since 1969. President Biden touted the strong jobs report as evidence that his economic policies are working. |
| 15:31 | 17:17 | Murdaugh Double Murder Trial | Alex Murdaugh, South Carolina, murder trial, Maggie Murdaugh, Paul Murdaugh, Satterfield family, Gloria Satterfield, Mallory Beach | David Muir reports on the latest developments in the Alex Murdaugh double murder trial. Prosecutors allege that Murdaugh killed his wife and son to cover up his financial crimes. The defense is arguing that Murdaugh is innocent and that the prosecution's case is based on circumstantial evidence. |
| 17:18 | 18:30 | Robin's Epic New Zealand Journey | Robin Roberts, New Zealand, travel, adventure, bungee ju |
Let's tweak our prompt ever so slightly to ask it to count ads and stories separately, just to ensure that ads are counted:
Break this video down into individual discrete stories and advertisements. Output a table in which each row is one story or advertisement. The first column is the start time of the story. The second column is the end time of the story. The third column is a descriptive title of the story. The fourth column is a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that story. The fifth column should be a detailed textual description of that story that describes it in sufficient detail to someone who can't see the video.
Unfortunately, this time it truncates almost the entire video:
Breakdown of Video into Stories and Advertisements
| Start Time | End Time | Title | Keywords | Description |
| 00:00 | 00:08 | National University Commercial | Education, online learning, flexible schedule, scholarships | A commercial for National University, an online university that offers flexible schedules and scholarships. The commercial features a montage of students studying, working, and spending time with their families. |
| 00:10 | 00:27 | Breaking News: Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, national security, Montana, Pentagon | Breaking news report about a suspected Chinese spy balloon being tracked over the continental United States. The report features footage of the balloon and interviews with government officials and residents of Montana, where the balloon was first spotted. |
| 00:27 | 01:14 | Breaking News: "Once-in-a-Generation" Arctic Blast | Winter storm, Northeast, cold weather, snow, ice, wind chill | Breaking news report about a major winter storm impacting the North |
Let's try again:
Break this video down into individual discrete stories and advertisements. Output a table in which each row is one story or advertisement. The first column is the start time of the story. The second column is the end time of the story. The third column is a descriptive title of the story. The fourth column is a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that story. The fifth column should be a detailed textual description of that story that describes it in sufficient detail to someone who can't see the video.
Unfortunately, it truncates again:
World News Tonight with David Muir – February 3, 2023
| Start Time | End Time | Title | Keywords | Description |
| 00:00 | 00:08 | National University Commercial | National University, education, online learning, 4-week courses, scholarships | A commercial for National University, an online university that offers 4-week courses and scholarships. The commercial features a montage of students and graduates of National University, highlighting the flexibility and affordability of their programs. |
| 00:10 | 00:26 | Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, surveillance, U.S., Pentagon, Montana, Billings | David Muir reports on a suspected Chinese spy balloon that is being tracked by the U.S. military. The balloon was first spotted over Montana and is believed to be collecting intelligence. The Pentagon is considering shooting down the balloon, but is concerned about the potential for debris to fall on populated areas. |
| 00:27 | 01:13 | "Once-in-a- |
And again:
Break this video down into individual discrete stories and advertisements. Output a table in which each row is one story or advertisement. The first column is the start time of the story. The second column is the end time of the story. The third column is a descriptive title of the story. The fourth column is a list of descriptive keywords indicating the topics, themes, people, organizations, locations and other key descriptive terms of that story. The fifth column should be a detailed textual description of that story that describes it in sufficient detail to someone who can't see the video.
This time we get results again, though most advertisements are missing, contrary to instruction:
World News Tonight with David Muir – February 3, 2023
| Start Time | End Time | Title | Keywords | Description |
| 00:00 | 00:08 | National University | education, online learning, flexible schedule, scholarships | This advertisement promotes National University, an online university that offers flexible schedules and scholarships. The ad features a montage of students studying, working, and spending time with their families. |
| 00:10 | 00:26 | Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, national security, Pentagon, Montana | David Muir reports on a suspected Chinese spy balloon that is being tracked by the Pentagon over the continental United States. The balloon was first spotted over Montana and has since moved eastward. Officials are concerned that the balloon is being used to gather intelligence. |
| 00:27 | 01:13 | "Once-in-a-Generation" Arctic Blast | weather, cold, snow, wind chill, Northeast, travel | David Muir reports on a dangerous arctic blast that is bringing record-breaking cold temperatures and wind chills to the Northeast. The cold weather is causing travel disruptions and power outages. |
| 01:14 | 01:24 | Deadly Assault Arrest | California, road rage, stabbing, arrest | Matt Gutman reports on the arrest of a man in California who is accused of fatally stabbing a bicyclist after a road rage incident. The victim, Dr. Michael Mammone, was a well-respected physician. |
| 01:25 | 01:44 | United Jets Clip Wings | Newark Airport, United Airlines, Delta Airlines, plane collision, FAA investigation | Gio Benitez reports on an incident at Newark Airport in which two United Airlines jets clipped wings while taxiing. No injuries were reported, and the FAA is investigating the incident. |
| 01:45 | 02:10 | 517,000 New Jobs Added | jobs, economy, unemployment, January | Rebecca Jarvis reports on the latest jobs report, which shows that the U.S. economy added 517,000 jobs in January. The unemployment rate fell to 3.4%, the lowest level since 1969. |
| 02:11 | 02:30 | Robin's Epic New Zealand Journey | Robin Roberts, New Zealand, travel, adventure, bungee jumping, skywalk | Robin Roberts takes viewers on an epic journey to New Zealand, where she goes bungee jumping and skywalking. |
| 02:31 | 03:14 | Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, national security, Pentagon, Montana, Blinken | David Muir reports on the latest developments in the story of the suspected Chinese spy balloon over the U.S. Secretary of State Antony Blinken has postponed his trip to China in response to the incident. |
| 03:15 | 04:44 | Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, national security, Pentagon, Montana, Blinken, shootdown | David Muir reports on the latest developments in the story of the suspected Chinese spy balloon over the U.S. The balloon has been shot down by the U.S. military off the coast of South Carolina. |
| 04:45 | 05:50 | Tracking Chinese Spy Balloon Over U.S. | China, spy balloon, national security, Pentagon, Montana, Blinken, shootdown, technology bay | David Muir reports on the latest developments in the story of the suspected Chinese spy balloon over the U.S. The balloon was carrying a technology bay that is believed to have contained surveillance equipment. |
| 05:51 | 07:17 | "Once-in-a-Generation" Arctic Blast | weather, cold, snow, wind chill, Northeast, travel, Boston, Mount Washington | David Muir reports on the latest developments in the story of the arctic blast that is affecting the Northeast. The cold weather is causing travel disruptions and power outages. |
| 07:18 | 08:37 | "Once-in-a-Generation" Arctic Blast | weather, cold, snow, wind chill, Northeast, travel, Boston, Mount Washington, shelters | David Muir reports on the latest developments in the story of the arctic blast that is affecting the Northeast. The cold weather is causing travel disruptions and power outages. Shelters are opening up to help people stay warm. |
| 08:38 | 09:40 | "Once-in-a-Generation" Arctic Blast | weather, cold, snow, wind chill, Northeast, travel, Boston, Mount Washington, shelters, accidents | David Muir reports on the latest developments in the story of the arctic blast that is affecting the Northeast. The cold weather is causing travel disruptions, power outages, and accidents. |
| 09:41 | 10:44 | "Once-in-a-Generation" Arctic Blast | weather, cold, snow, wind chill, Northeast, travel, Boston, Mount Washington, shelters, accidents, warming up | David Muir reports on the latest developments in the story of the arctic blast that is affecting the Northeast. The cold weather is causing travel disruptions, power outages, and accidents. However, temperatures are expected to warm up this weekend. |
| 10:45 | 12:40 | Deadly Assault Arrest | California, road rage, stabbing, arrest, Dr. Michael Mammone, Vanroy Evan Smith | Matt Gutman reports on the latest developments in the story of the arrest of a man in California who is accused of fatally stabbing a bicyclist after a road rage incident. The victim, Dr. Michael Mammone, was a well-respected physician. The suspect, Vanroy Evan Smith, is being held without bail. |
| 12:41 | 13:44 | United Jets Clip Wings | Newark Airport, United Airlines, Delta Airlines, plane collision, FAA investigation, passenger | Gio Benitez reports on the latest developments in the story of the incident at Newark Airport in which two United Airlines jets clipped wings while taxiing. A passenger describes feeling a "jolt" when the planes collided. |
| 13:45 | 15:35 | 517,000 New Jobs Added | jobs, economy, unemployment, January, wages, inflation, President Biden | Rebecca Jarvis reports on the latest developments in the story of the January jobs report. President Biden is touting the strong job growth as a sign that the economy is doing well. |
| 15:36 | 17:22 | Murdaugh Double Murder Trial | South Carolina, Alex Murdaugh, murder trial, Mallory Beach, Satterfield family | Eva Pilgrim reports on the latest developments in the Murdaugh double murder trial. Prosecutors are alleging that Murdaugh killed his wife and son to cover up his financial crimes. |
| 17:23 | 19:30 | Robin's Epic New Zealand Journey | Robin Roberts, New Zealand, travel, adventure, bungee jumping, skywalk | Robin Roberts takes viewers on an epic journey to New Zealand, where she goes bungee jumping and skywalking. |
| 19:31 | 20:08 | Memphis Officer Hemphill Fired | Memphis, police, Tyre Nichols, body camera footage | David Muir reports on the firing of Memphis police officer Preston Hemphill, who was involved in the traffic stop that led to the death of Tyre Nichols. Body camera footage of the incident is expected to be released soon. |
| 20:09 | 21:44 | Consumer Cellular | cell phone service, Target, affordable, seniors | This advertisement promotes Consumer Cellular, an affordable cell phone service provider that is available at Target. The ad features testimonials from satisfied customers. |
| 21:45 | 26:24 | Eliquis | atrial fibrillation, stroke, blood thinner, warfarin | This advertisement promotes Eliquis, a blood thinner that is used to reduce the risk of stroke in people with atrial fibrillation. The ad features testimonials from satisfied customers and information about the risks and benefits of Eliquis. |
| 26:25 | 28:44 | Robin's Epic New Zealand Journey | Robin Roberts, New Zealand, travel, adventure, bungee jumping, skywalk | Robin Roberts takes viewers on an epic journey to New Zealand, where she goes bungee jumping and skywalking. |
| 28:45 | 30:55 | Concerns Outside School | Oakland, California, school, prostitution, police, FBI | Stephanie Sierra reports on concerns about prostitution happening outside of an elementary school in Oakland, California. The I-Team's report has prompted action from the Oakland Police Department and the FBI. |