Imagn Images Tests AI in Captioning Super Bowl LIX Photos in Proof of Concept With Eluvio

Using the workflow, the photo wire service delivered 3,000+ photos during Super Bowl Week

As the sports-media industry experiments with how artificial intelligence (AI) can augment production and operations, an interesting proof of concept (POC) occurred on the photography side at Super Bowl LIX. Imagn Images, the nation’s largest sports-image wire service, used AI to caption, tag, and process more than 3,000 photos shot and distributed during last month’s Super Bowl Week in New Orleans.

Partnering with Eluvio, Imagn deployed a multimodal AI, multiple machine-learning (ML) models, and the Eluvio Content Fabric to revolutionize the workflow for correctly captioning photos in near real time and making them available to media clients nationwide. It’s a potential game-changer for a wire service like Imagn, which is built on scale, accuracy, and speed.

Photo wire service Imagn Images used AI to caption photos at Super Bowl LIX with the help of Eluvio.

“[This is] a very competitive space,” says Imagn Images President/CEO Bruce Odle. “Not only do you have to compete on price and your customer service, you also have to scale what you cover and [increase] the speed of delivery from the event to the customer. You have to produce the photos from the event, have them captioned, tagged, and produced to the customer almost in real time. It’s a challenging environment. We looked at [Eluvio] and their tech to help solve that.”

Imagn Images, which was previously a part of media company Gannett but is now on its own, employs more than 300 photographers and shoots more than 10,000 events each year. It’s still the official photo partner of Gannett and is also the photo wire service of choice for Reuters in the U.S.

For this Super Bowl POC, Eluvio delivered an end-to-end hosting, data, and rendering environment. Photos shot by Imagn photographers at Super Bowl Week events and Super Bowl LIX itself were fed into the Fabric, where captions, tags, and other metadata were generated and applied to photo listings made available in the wire service.

The challenge was not just to create accurate captions quickly but to do so in the authentic style of an Imagn caption.

“It was true state of the art,” says Eluvio CEO Michelle Munson. “I don’t think anybody has done this. I also don’t think anybody has attempted to do it with the level of accuracy that would be suitable for an online caption without any editing — or with only the slightest editing — by a person.”

This kind of work in a multimodal AI is nothing new for Eluvio, which works on large models in video using such metadata as audio, frame detection, and computer vision along with text-based semantic descriptions. The rub in photography, though, is that few of those context clues are available, making it more challenging to build a model that can accurately scan and identify critical information quickly.

“This is an intellectually intensive process for a human,” says Munson. “There are three learning activities: one is to say what it is, two is to put it in the right style, and three is make sure it’s accurate. All of that has to happen, and the goal is two seconds per photo. That sets up a perfect environment for the things the [Eluvio’s] Fabric does.”

In today’s instant-gratification media environment, photographers face demands for increasingly immediate delivery, and it is difficult for a wire service like Imagn to provide such service at scale. Odle notes that key roadblocks in venue connectivity and camera hardware have largely been alleviated. It’s everything that comes after that remains the obstacle.

“Those two limiters have seen massive advancements,” says Odle. “Most stadiums have dedicated WI-FI, so connectivity is not the issue. The cameras have a wireless transmitter in the body, so that’s no longer the issue. As you look at the workflow, it’s the editing process where you can gain efficiency and speed. That’s why this project is so important now.”

Left: Imagn Images President/CEO Bruce Odle. Right: Eluvio CEO Michelle Munson

The partnership between Eluvio and Imagn Images originated at SEICon (Sports, Entertainment & Innovation Conference) last summer. There, Eluvio won a technology innovation competition that was, in part, judged by a USA Today journalist. Its winning entry showcased the potential of AI in photography workflows, particularly for captioning and metadata extraction.

A collaborative effort was born and targeted the POC at the Super Bowl. In an intense, three-week process, Eluvio trained the model by uploading to it tens of thousands of captions from Imagn’s NFL archives. That helped teach the Fabric about football gameplay and the terminology common in Imagn’s captions. On game day, pairing that AI knowledge with the team rosters helped establish accuracy, speed, and style.

Both sides are pleased with the project but note that there’s still plenty of work to be done.

“The efficiency and speed of the workflow is first and foremost,” says Odle. “We recognize there’s more investment in time required to be able to be at the accuracy and the speed required across every sport. Every sport has its own unique challenges. There’s a lot of work to do there, and we appreciate that. The accuracy issue is what everyone in this space has to worry about and is a part of the solution. We have hopes that that’s an outcome of this process.

“Also,” he continues, “we think, in and of itself, this data – these millions of images with this highly refined metadata associated with it — has its own unique IP and its own unique value. The extent to which we can help others who have similar challenges on workflow efficiencies — or if we can help with discovery of images or accuracy of images – those are things that we’d like to see if we can’t use in a commercial sense to help others who may have the same challenges.”

Additionally, Eluvio is able to provide authenticity to Imagn Images and its clients. Eluvio’s Fabric can encrypt, watermark, authorize access, and provide verifiable proof to a photo through blockchain technology.

“The Fabric uniquely hosts the content IP: the images, the metadata, even the ground truth with the security context of the tenant,” Munson points out. “The tenant is the owner, and Imagn, as the tenant, can literally secure their IP and secure the data that goes into their models. That is not true in other AI pipelines.”