Session analysis

What's the right amount of information?

A completed AI session generates a lot of data: a scenario checklist of specific behaviors, a skill score across multiple dimensions, existing strengths and opportunities for growth, and a plain-language summary. All of it has value on its own but none of it is useful if a learner closes the screen without reading it. A core challenge was figuring out how much to show by default and what to prioritize in the interface.

How do you make a soft-skill score feel credible?

Telling a learner they scored 3 out of 5 on "Coaching" means nothing without grounding it in something they already understand. The solution was to anchor every piece of feedback in the specific behaviors that were or weren't demonstrated during the session. Not "your coaching score was low" but "for a 4 or 5, ask more targeted questions that lead to concrete next steps and insights" along with a verbatim example from their actual conversation where possible. That specificity is what makes the feedback feel factual rather than subjective.

Who is this screen for?

Learners finishing a session are often mid-workday, context-switching to something else almost immediately. The screen has to digestible in 30 seconds or less.

Collaborating across teams

The Learning Experience team at Hone — our the in-house team responsible for content and measurement strategy — was a close collaborator throughout this project. They sit within the product org and were involved in QA, copy review, and pushing back on what to include and how to frame it.

This internal tension was productive. The LX team's instinct was toward completeness, with the perspective that every data point the session generates has value for a learner who wants to improve. The product team's instinct was toward digestibility. Information a learner doesn't read isn't useful at all. That back-and-forth shaped the design in every iteration. My consistent position was that a learner who closes the screen without reading it has learned nothing, regardless of how thorough the content is and that perspective has ultimately led to the current, streamlined interface.

Launching

The first version of the screen launched with a pretty full information set containing a scenario checklist, skill scores across multiple dimensions, and a strengths and opportunities breakdown. We also added a simple thumbs up/down prompt at the bottom asking learners whether the screen was helpful, which turned into a useful feedback loop with more interactions than we anticipated.

Feedback arc

Late February: 80% helpful on a small sample. A reasonable start, but not enough data to draw strong conclusions.

March: 53% helpful, with a cluster of not-helpful votes in a single day. At this point, the screen was showing a significant amount of information by default, and the signal from learners was that it was too much.

In response, we scaled back the information density. Some content was removed entirely, giving learners a less overwhelming entry point while keeping information breadth accessible.

April: 72% helpful, a meaningful improvement. But the more significant change came later in the month, when the team aligned on a philosophical shift in how to frame session feedback: the scenario checklist (the specific behaviors a learner did or didn't demonstrate) should be the primary focus of post-session analysis. Skill scores, which reflect longer-term patterns across many interactions rather than a single session, were repositioned accordingly. The screen was redesigned to center the checklist by default and move the skill rubric and suggested next steps to their own tabs, where they could be accessible to motivated learners without distracting those who needed the highest-priority information quickly.

May: 97% helpful, with 32 out of 33 learners rating the screen positively.

What drove the improvement

The copy grounding scores in specific behaviors was present from the beginning so that wasn't the variable. The hypothesis, supported by the metric trajectory, is that the improvement came from giving learners a single, focused surface to engage with rather than asking them to process everything at once. The checklist is also the most time-efficient feedback format on the screen. It's concrete, scannable, and directly tied to what happened in the session.