Our first ChatGPT ad tests. What did Realist’s client campaigns reveal?

ChatGPT ads have only been available in Estonia for a few weeks, but we already have some early experiences to share from clients who were ready to try the new channel. Results can vary considerably across sectors. Here, we look at campaigns for two service providers, one B2B and one B2C. In both cases, ChatGPT seemed like a good fit and a useful addition to Google Search campaigns. We used context hints from the start in the B2C campaign, while the B2B campaign ran in two stages, first without hints and then with them. The click-through rate in the second stage was approximately 38% higher.

Ads appear below responses in ChatGPT conversations. When selecting a relevant ad, the system considers factors such as the topic of the conversation, the ad content and the landing page. This gives businesses an opportunity to be visible while someone is asking for advice, clarifying their needs and comparing solutions.

Our two campaigns promoted services to different audiences: businesses and consumers. In the B2B campaign, we tested the same ads first without context hints and then with them. Context hints are a feature of ChatGPT advertising that helps the system understand which conversations might benefit from the advertised product or service. In our test, ads in the stage with hints had a 38% higher click-through rate. Put simply, we received more clicks through to the advertised website for the same number of ad impressions.

What can we learn from these first campaigns?

In the consumer campaign, we used context hints from the outset. The campaign’s click-through rate, or CTR, was 0.75%. We ran several creative variations, but one stood out, accounting for 62% of all campaign clicks and 46% of impressions. This shows why it is worth looking beyond campaign averages and monitoring individual ads. Doing so helps identify which messages and creative approaches resonate most. As with Meta, it makes sense to test several ad variations in ChatGPT and use the results to improve the next round of creative.

During the same period, the same client’s Meta campaign had an outbound click-through rate of 1.74%, so ChatGPT’s CTR was lower. That does not mean the channel is ineffective. What also interests us is the opportunity to reach someone while they are clarifying their needs and comparing solutions. For a business, that may be where the value lies: being visible before someone even knows which provider to look for.

Do context hints affect campaign performance?

We tested the B2B service provider’s ads in two consecutive stages. In the first, we added no context hints. In the second, we did. The ads, landing page and daily budget remained the same.

Context hints describe needs and situations in which a service might be useful. They are neither exact-match keywords nor strict targeting rules. Our hints described conversations about challenges related to the client’s services. Even without hints, the system could draw information from the ad copy and landing page. We wanted to see whether providing additional context would improve performance. Our hypothesis was that it would help the system match ads to more relevant conversations.

Without hints, the CTR was 1.22%. With hints, it was 1.68%, an increase of approximately 38%. Although the cost per thousand impressions was around 30% higher in the stage with hints, those impressions generated proportionally more clicks. As a result, the average cost per click was approximately 6% lower.

It would be tempting to say that context hints caused the improvement. The data does not yet support such a firm conclusion. The stages ran on different days, the stage with hints generated relatively few clicks, and impressions were distributed differently across the ads. We can see a difference in the results, but the effect of context hints needs further testing.

A higher click-through rate, but where did the impressions go?

The stage with context hints generated approximately 78% fewer impressions, despite the higher CTR. Ad delivery almost stopped before the planned end of the campaign, even though the ads were approved and the ad group was active. We received no notification explaining why. Was it an early platform issue, something related to ad matching, or another factor? The available data does not tell us. Unexpected delivery patterns are therefore something to keep in mind when testing ChatGPT ads.

According to OpenAI, updates to its systems during the beta period can affect both delivery and spend. This offers a possible explanation, although it does not establish what happened in our case. Some unpredictability is still to be expected when working with a new channel.

A lower cost per click matters, but so does a campaign’s ability to generate traffic consistently. As with Meta campaigns, it is worth looking at the cost per click alongside the total number of clicks. Cheaper clicks alone do not mean more visitors to the website.

ChatGPT Ads is quickly becoming more than ad space

Our first tests focused primarily on how ads were delivered and the click-through rates they achieved. The platform itself, however, is developing quickly. Campaigns can now be optimized for conversions as well as reach and clicks. Advertisers can use the OpenAI Pixel for measurement, alongside UTM parameters and existing analytics tools, much as they do with Meta and Google.

Even more interesting is the direction OpenAI is taking its ad formats. On September 16, the company announced that it was testing Sponsored Agents. This format allows someone to click an ad and start a conversation with a business-sponsored agent inside ChatGPT. They can ask follow-up questions about a product or service and then continue to the business’s website. Sponsored Agents are initially being tested with selected advertisers in the US.

Is it worth trying ChatGPT ads already?

Our first tests were mainly about understanding how ad delivery works and whether people click. In our next tests, we will focus more on what happens after the click: how many inquiries, bookings or other meaningful business actions the campaigns generate, and at what cost. The addition of the OpenAI Pixel and other measurement options gives us a stronger foundation for doing that.

For now, we would not approach ChatGPT Ads by asking whether it works better than Google or Meta. The more interesting question is which products, services and customer needs lend themselves to reaching someone while they are still defining a problem or considering possible solutions. This is where the channel differs most from traditional search advertising, and where we see the strongest reason to keep exploring.

The question is therefore not only what results the channel delivers today. It is also whether businesses should start learning how it works while the ad format is still evolving rapidly.

Getting started early has another benefit. Businesses can learn which conversational contexts, offers and ad messages work for them before the channel matures and competition grows. For us, the next step is less about asking “Does ChatGPT Ads work?” and more about understanding where and how it can create the most value in our clients’ marketing funnels.

Want to try ChatGPT ads for your business? Get in touch. We’ll help you decide what to test and how to measure the results.