AI Investigation

The Issue

With the recent boom in AI applications throughout the tech world, Flodesk wanted to investigate user opinions of AI as well as potential user pain points that could be solved by AI.

We had previously formulated several hypothesis surrounding AI applications, including:

• Members are already using other tools for drafting and refining written content and creating image content, so there is an opportunity and interest to have these integrated into Flodesk (they want us to have AI)
• Members would find more value in Flodesk if we’re able to help them save time and create workflow efficiencies by leveraging AI (they simply want to operate faster and are open to AI to enable this)
• And, members don’t have concrete marketing strategies in mind to fully utilize all of Flodesk’s services, so they would appreciate AI guiding them to maximize the potential of our platform to grow their business (they want AI to give them guidance)

Flodesk had also recently hired a summer intern to work under me, and we determined that this was the perfect project for her to work on with my mentorship and oversight.

The Process

From the above hypotheses, I worked with our VP of Product and our new summer intern to turn them into the following main research questions:

• What are the most common pain points with the current Flodesk or competitor workflow/features?
• How do our members perceive the recent developments related to AI? Do they see it as an opportunity or a threat?
• And, do our members currently use AI in their day-to-day work apart from Flodesk?

I then held several sessions directly with our summer intern to brainstorm potential research plans and methodologies. I was careful not to lead with too firm a hand, as it's important to leave room for people to fail and learn from their mistakes.

Ultimately, she decided on an interview study and chose to target several user segments based on years of business experience and email marketing experience. I then held several more sessions with her to coach her on interview best practices and run an internal pilot with my supervision, as she had never conducted formal user interviews before. I also supervised and gave advice as she came up with a recruitment plan using our internal database as well as external recruitment methods.

During the interviews themselves, of which there were eleven, I attended most of them to supervise and give feedback on what she was doing well as well as ways she could be improving for the next interview.

By the end of the study, she was easily handling the interviews by herself. I then gave her some tips for analysis and any resources she might need to do cross-referencing with prior research and let her handle the rest, as I was confident in her ability.

The Results

One major outcome of this study was increased confidence in my own mentoring ability. I'm no stranger to coaching PMs and designers on running their own research, as I've done it many times at Flodesk and previously at Tableau. But it was a whole different experience mentoring an intern through the process while being sure to allow them space to fail and learn for themselves.

As for the results of the intern's research, a major finding was that user trust in AI wasn't very high, although she did find several use cases for AI for our customers which have been further reinforced by other research studies since then.

The Impact

The initial impact of the intern's research was a slight divestment in AI discussions, as we found user trust was not quite as high as we had hoped. However, since then other studies that I've run have continued to see more and more potential use cases for AI. For this reason, Flodesk recently made an additional AI Engineer hire and we are going to start brainstorming ways to incorporate AI in a way that is more trustworthy to our users, because we really do see many ways it could help them increase their efficiency and increase their profits.

So much more AI research is pending, which is an exciting new forefront for me.