Research 1 Approach

Round 1

There was a round of research earlier in 2022 that focused on the internal audience, internal maintainers, authors, and the barriers to keeping the guides up to date, summarized in this presentation (GSA only)

• It generally concluded that...

Round 2

Approach

What we were looking for

We wanted to explore...

• Who was using the guides, especially at other government agencies? Were they having success? What were they trying to do with what they learned?
• Which guides were still adding value?
• More detail in the research plan

Who we talked to

• We wanted to speak to real readers from across the US government and maybe some state, local, or even foreign governments
• We ended up talking with people who had experience using the guides at the US Forest Service, California Office of Digital Innovation, Centers for Medicare and Medicaid, GSA, Senate Appropriations, and Indian Health Service
• We recruited through word of mouth as well as the Open Opportunities Board (more on that below)

How we approached recruitment, ethics, and consent

• We looked into compensation but ultimately decided we wouldn't be able to do it here...
• We sent a short summary of what we'd do in advance, and asked if they wanted just the whole interview guide guide
• We made sure to ask if they wanted to record before the session, and set a deletion date
• We did a schpeal before each interview, and took frequent breaks
• Related to all tracks of the project, but we held an equity pause early on
• Bias acknowledgement in the research plan
• Read through the [Ethical Researchers Checklist] (link)

"Research as a team sport"

• The beginning of the project was going to be research heavy and that's a lot for one researcher to do, so we held an exercise to hash out what everyone's role was...
• More...

Recruiting with the open opportunities board

• Link back to guide and describe what worked and didn't for us
• We wanted to do a bit better than "I know a guy" for finding research participants, and wanted to make sure we spoke to real readers from multiple agencies...
• So we tried out the OOB (what is it, etc.)...
• Got some good hits...
• A few turned out not to be good matches so there was some screening effort in the beginning...
• After initial round it would have required some external marketing to go further

Activities

Semi-structured interviews

• What they were

Round 3

Card sort

• We did an open one and closed one sort of
• The audience was...

Other conversations and impromptu feedback

• Content...

A light look at metrics

• We poked at metrics...
• Turns out Guides and Methods get more traffic than the main 18F site