Research 2 Findings

A brief tour of the AirTable base

• Nuggets
  • These are the "atomic units" from our research sessions. Each represents an observation, note, or quote either directly from or adapted from our notes.
• Tags (Themes)
  • Somewhat like an affinity map, each of the nuggets can be tagged with related themes (though they can have many tags). Using the Themes tab can help show which themes are most prominent. We used this to help us determine what issues we were hearing about most often or which were most severe.
• Guide / Method
  • This refers to which of the 10 guides a nugget was referring to, if it was specified. The tab for this includes some metadata about each of the guides.
• Interviews
  • This tab lists each of our research activities, though without PII. We can track exactly who said what as well as some supporting metadata for each activity. There are a few "over time" entries like "organic Slack feedback" and "general observation" that don't tie to a single activity.
• Hypotheses
  • This is where we tried to make some decisions and pursue some hypotheses and tie them directly back to our themes and nuggets. Some have multiple variants like "this thing is good" and "this thing is bad" for when we had evidence supporting both.
• Hypotheses backlog
  • Our hypotheses table became overwhelming so we moved many in here to focus on what we though where the most important at the time. There are many unexplored directions in here.
• Design / feature ideas
  • This is a bit of an undeveloped tab that we had hoped to use as a sort of "we should build this into the website" decision tracker, sort of like hypotheses, but with actual website features. The idea was we could say "we included this feature directly because of x, y, and z feedback."
• Research questions
  • We moved some of our research questions from our research plan here basically to track if we had answered any of them. It ended up not really developing in a super concrete way but future folks could go back and cross reference the nuggets with this to see if we're any closer on answers.

Findings and data

Who was the audience after all?

• We found it was dependent on the guides and the authors
• Some were explicitly geared toward outside readers, others to TTS and 18F staff
• But the audience definitely isn't just internal
• And they are a valuable internal resource

Are the guides valuable? Which ones?

• They are referenced and replicated broadly
• They have a wide reach
• They bring people to government
• They improve awareness of 18F

Success stories

• Senate Appropriations + the De-Risking Guide
• Indian Health Services + the UX Guide
• US Forest Service + UX Methods

But also

• Shared infrastructure fails
• Poor site maintenance
• Ownership challenges

Select other hypotheses (split out as some of these were central to what we did on the project)

• We should combine the guides (reasons x, y, and z)
• Guides should be editable by folks from a wide variety of knowledge levels (CMS)
• Guild leaders should not be the owners

More

• ...

TBD

Most of our more recent findings reside in this presentation or this Airtable (GSA- or 18F-only, respectively). The initial round of research is summarized in this presentation, but the data also resides in the same Airtable, with similar permission restrictions.