CONCEPT TESTING · PRODUCT DIRECTION · B2B SAAS MARKETPLACE · CRAFT

Two rounds of concept testing that redirected a product roadmap

The client had already started building designs for a calendar integration. Users had other ideas. Our job was to generate the evidence that made it safe to follow them.

MY ROLE

Co-researcher;

owned recruitment, moderation, and synthesis

TIMELINE

16–20 across two rounds

METHODS

Speed-dating concept testing, wireframe evaluation

TIME TO SHIP

~10 months after initial testing

THE CHALLENGE

Our client had an idea they were already excited about. They'd started building designs for a calendar integration and wanted to test it — but no research had been done yet to evaluate whether a calendar was actually what users needed. We'd seen this pattern before: a team moves fast on a promising direction and tests execution before testing the premise.

THE PLAN

Working alongside my director, who led the overall research strategy, I designed and ran two rounds of bare-bones concept testing. Rather than evaluate the polished designs the client had already made, we proposed slowing down just enough to test the idea itself: nine wireframe-level concepts at matched fidelity, showing features of a calendar integration. Equal fidelity kept the comparison honest. Low fidelity kept us moving fast.

ezCater _ Research _ Testing Stimuli.png

ROUND ONE

Round one showed muted enthusiasm for the calendar concepts — not rejection, but not excitement. Participants described what they actually wanted: to complete tasks without leaving the context they were already in. A pattern emerged pointing toward messaging and Slack-style integration. I recruited and moderated all sessions, synthesized the findings, and brought the direction back to the team.

ROUND TWO

Round two tested revised concepts inspired by the users' needs and interests. The difference in response was immediate and visible. Participants who had been politely neutral became genuinely engaged.

If we'd tested only the calendar concepts, we would have found out how to make a calendar integration marginally better — not whether it was what users needed.

THE OUTCOME

That contrast — not just what users said, but how differently they responded across two rounds — gave the team something concrete to act on. The client shifted direction, set the calendar integration aside, and took our truly validated conceptual designs to pursue a messaging integration. Within six months, they were executing on those concepts and building customer-driven features.

"We needed that broader perspective — it was perfect for Craft to come in and really evaluate whether this was worth pursuing or not."

Key product stakeholder

THE REFLECTION

I would have included more concepts outside the calendar space in round one from the start. That first round was, in part, still a solution looking for a problem. Including a broader range of directions earlier might have surfaced the messaging insight one round sooner, though the two-round structure ultimately produced clear, defensible direction that the client could act on with confidence.