Product
NPS 100, and Six Promises My Landing Page Couldn't Keep
My first cohort scored an NPS of 100 and seven of eight people left with something working. A later audit still found six claims on my landing page that the course had not delivered.
landing page promisesnps 100landing page audit
My first cohort scored an NPS of 100. Then I audited my own landing page against what we had actually delivered and found six claims the weekend had contradicted.
I still don’t know which of those two facts matters more.
What happened in the room
I ran a two-day hands-on AI course with a co-facilitator, with 8 people giving feedback. The numbers were strong: seven 10s and one 9, so an NPS of 100. Seven of the eight said they left with a working thing they would use on Monday.
The room also got more talkative. Participants’ share of talking time went from 20% on day one to 39% on day two. The best hour of the weekend was the block where everyone built one thing for their own desk. Eight of eight ticked it on the form.
Now the caveats, which I wrote down at the time. It was eight people. The scores were collected in the room, with both facilitators present, which is about the friendliest way to ask someone for a number. Take the NPS as a pleasant fact and not as proof.
The six promises
A few days later I checked the landing page against the delivery data. In my own shorthand, the page said:
- You build all four capstones and keep them. Nobody built four, and one of them was built by nobody.
- The page headlined that unbuilt capstone. It had been voted the favorite beforehand and then attracted no builders at all.
- A footnote said testimonials were published with each person’s written consent. One had gone up with a name attached before that person had answered the attribution question. Consent to publish was not consent to be named.
- The button said to reserve a seat. What actually happened was a waitlist. The page and the action used different words for the same step.
- The page said a certain number of seats. The cohort ran with fewer, and I now think the format works because it is small.
- The page said that if your office laptop is locked down, a cloud fallback has you covered. That fallback was planned in July and never made it into the room. Several attendees came from locked-down companies, one of them asked about it more than once, and nobody gave a prepared answer.
I’m leaving out what else was off (a clinic sold unchanged, a capstone described more grandly than its real output, follow-up sessions that never ran). Six was enough to make my point to myself.
I’m told the page has been changed. I didn’t re-audit it before writing this, so I won’t claim it is clean.
Why the happy room made it harder to see
The strangest part of the audit was how little the scores helped. I had a clean number and a pile of kind comments, and neither pointed at the page. The feedback form asked about the weekend. Nobody was asked whether the page had matched it, so the only way to find the gaps was to sit down and compare the page line by line with the transcripts and form answers.
Some of the gaps were small. A sentence that overstates what a capstone produces costs nobody much. Others were different in kind, because they promised something to a person who had a specific problem, and the promise was the reason that person said yes.
So what is landing page copy for?
This is the part I haven’t resolved.
One answer is that copy is a promise, and the audit is just a list of broken ones. Under that reading, the NPS is almost beside the point. Eight people were happy, and the ninth person, the one reading the page, was told things that weren’t true. Some of them were things I could check.
The other answer is that the people in the room scored the weekend and not the page. They never saw the six gaps because the experience they got was good enough that the claims didn’t matter to them. If the product delights people, who is harmed by a headline that oversells? I can see that argument and I don’t fully reject it. I don’t like how comfortable it is, though.
Where I land today, with hedging: the sixth promise is the one I can’t argue my way out of. A person with a locked-down laptop reads that line, decides they are safe, and signs up. Their NPS would have been a different number. The eight I surveyed were the ones who stayed and filled in the form.
That is a survivorship problem I can name and can’t measure.
What I took from it
None of these settles the question, but they are the habits I’m trying:
- Audit the page against delivery data after every cohort, as a standing task.
- Ask the awkward screening question before money changes hands, such as whether you can install software on the laptop you will bring.
- Publish my own worst number when it explains a change. A post about the room-share jump and the setup complaints did better as an angle than any testimonial did.
- Read the page as a skeptical stranger. Every sentence either describes something I did or it needs to go.
- Check consent separately for publishing and for naming.
I’m not sure the audit would have happened if the feedback had been worse. High scores make it very easy to leave a page alone. That may be the real cost of a good NPS: it takes away the pressure to look.
If you’ve run a cohort, a launch or a product page, I’d like to know whether you’ve ever audited your own copy against delivery, and what you found.