2022 Case Study Report

Current phase:
Documentation development. See timeline.

Season of Docs is a sustainability program managed by the Google Open Source Programs Office. The goals of Season of Docs are to:

  • Provide support for open source projects to solve project problems with documentation
  • Give technical writers opportunities to gain experience in open source
  • Raise awareness of open source, documentation, and technical writing
  • Collect and share information about effective metrics in open source documentation

More information about Season of Docs is available on the program’s website.

2022 program overview

How Season of Docs works

In Season of Docs, organizations apply by submitting a project proposal. The project proposal includes:

  • Information about the organization
  • A description of the problem the project is facing
  • How the project will use documentation to help solve their problem
  • How the project will measure the effectiveness of their documentation (metrics)
  • A timeline for the work
  • A project budget
  • Any additional information, such as the organization’s experience in similar programs, or anything else that would help the Season of Docs administrators understand their project and problem

Once accepted to the program, organizations recruit and hire their own technical writers directly. Season of Docs uses Open Collective to fund the organizations, and organizations pay the technical writers via Open Collective. Project budgets and payments are transparent; budgets are included in the organization project proposals available on the Season of Docs site, and payments are visible in the Season of Docs Open Collective account.

Organizations are considered to have successfully completed the program when they submit their case study report. Organizations are also asked to complete monthly evaluations during the program and three quarterly follow up surveys during the year following the program completion.

2022 Highlights

“After the new document was released, daily visits to Casbin and Casdoor almost doubled, and bounce rates dropped by about 30%.”—Casbin

“A delightful outcome of this project has been watching [our technical writers] grow into leadership roles in our community. Both contributors are now leading working group and community meetings, as well as contributing to the design and maintenance of our projects.” —moja-global

“[GSoD] helped us recruit two talented technical writers which is highly difficult in a regular setup, who continue to be active OS contributors to OpenMined, and with whom we have had a great work experience.” —OpenMined

“Also, the new manual is much easier for newbies to computational mass spectrometry. To illustrate this point: the CZI grant also provides stipends for historically underprivileged individuals and some awardees have used the new OpenMS manual to jump-start their six-week internship period and have given positive reviews of the new manual.” —OpenMS

2022 Summary data

In 2022, the Season of Docs program accepted 31 projects from 67 applications, and 30 projects successfully completed the program. Of the 31 accepted organizations, 17 organizations were repeat applicants.

The 31 accepted projects hired 58 technical writers. More than 190 technical writers indicated their interest in participating in the program by adding their contact information and links to their portfolios in the Season of Docs GitHub repo.

For the 2022 program:

  • 100% of organizations had a positive experience with the application process
  • 100% of organizations had a positive experience with the program website documentation/content
  • 93% of organizations had a positive experience with the program
  • 90% of organizations felt their documentation project was successful

About the organizations

The organizations participating in Season of Docs 2022 represented a diverse range of open source projects. The 2022 cohort included:

A bar graph showing the domains represented by the accepted projects: Data: 5 projects; Development tools: 4 projects; End-user applications: 7 projects; Hardware and robotics: 2 projects; Infrastructure and cloud: 4 projects; Programming languages and tools: 3 projects; Science and medicine: 3 projects; Security: 1 project; Social and communications: 1 project; Web tools and frameworks: 1 project

We did not collect any metadata about the projects (such as date founded, geographic distribution of contributors, number of contributors, or size of user base).

We did ask projects to indicate which open source license they used.

A bar graph showing the number of projects using each OSS license: AGPL-3.0: 2 project; Apache-2.0: 9 project; BSD-3-Clause: 4 project; GPL-3.0: 3 projects; LGPL 3.0: 3 projects; MIT: 5 projects; Mozilla Public license 2.0: 2 project; BSL-1.0, GPL-2.0, LGPL-2.1: one project each

About the documentation projects

Documentation problems

The top problems that organizations were hoping to solve with documentation in the 2022 program included:

A bar graph showing the problems reported by organizations: Documentation is lacking for specific use cases of aspects of a project: 16 projects; Documentation is disorganized: 11 projects; Documentation is outdated: 7 projects; Documentation is not consistent: 1 project; Documentation needs to be converted to a different tool, platform, or format: 8 projects

Note that organizations could report multiple documentation problems. For more specifics, see the Season of Docs 2022 results page, which links to the original project proposals and full case studies for each organization.

Types of documentation created

How-to documentation was the most commonly mentioned documentation type in the 2022 case studies.

A chart showing the documentation types created:  How-tos: 12 projects; Tutorials: 9 projects; Reference: 8 projects; Landing page: 5 projects; API Docs: 4 projects; Diagrams, screenshots, illustrations: 4 projects; Getting Started, Style Guide, Handbook: 3 projects each; Examples, Concept documentation, user research: 2 projects each

Other documentation types mentioned in case studies included:

  • Quickstart
  • Glossary
  • FAQ
  • Knowledgebase
  • Components
  • Blog/social media content
  • Maintainer guide

Some of these categories are fuzzy and a single documentation project could contain multiple documentation types or features.

For more specifics, see the Season of Docs 2022 results page, which links to the original project proposals and full case studies for each organization.

Budgets

The average budget request was $11,679 and the median was $12,150. Five organizations requested and received the highest available grant ($15k) and three requested the lowest (between $5k-$7K).

The metrics

Projects outlined in their case studies the metrics they were using to gauge the success of their documentation projects.

The top proposed metrics were:

A bar graph showing documentation success metrics: More contributors/pull requests: 12 projects; Total percentage of target info covered by docs: 8 projects; Fewer project issues/questions: 7 projects; More visitors to documentation/docs usage: 6 projects;  Better SEO: 5 projects; Increased documentation satisfaction (via survey), Increased project use, More GitHub stars/forks: 3 projects each; Total number of docs created and Qualitative user testing: two projects each

Other proposed metrics included:

  • More documentation pull requests/contributions
  • More direct feedback on documentation pages
  • Time spent on page
  • Issues raised (as a proxy for use)
  • Participants in forums
  • Number of partners/volunteers/integrations
  • Reduced bounce rate
  • Increased awareness in the community.

Because of the short window between finishing the technical writing projects and submitting the case studies, most of the 2022 cohort had not been able to collect enough data at the time of submitting their case studies to determine whether or not their initial metrics had been met.

As we receive answers to the followup surveys in 2023, we’ll update this report to add information about which projects have achieved their metrics or revised their metrics.

For more specifics, see the Season of Docs 2022 results page, which links to the original project proposals and full case studies for each organization.

Working with technical writers

In the Season of Docs program, projects are expected to recruit, interview, hire, and pay technical writers directly. Technical writers can add themselves to the directory maintained by Season of Docs in our GitHub repository, but Season of Docs staff do not vet or recommend technical writers.

Best practices for hiring technical writers for open source projects

Projects were asked to share their best practices for recruiting, hiring, and working with technical writers. The top recommendations were:

Recruiting

  • Interview fewer candidates, and use a live practice session, rather than just reviewing CVs
  • Value written and oral communications skills over proficiency in your project’s language or tooling
  • Ask direct questions about how the technical writer will acquire any domain knowledge necessary to work with your project
  • Someone with enthusiasm for your project’s mission and who shares core open source values is more likely to stay motivated through the entire project
  • Be open to applicants from all over the world, because a diversity of viewpoints and backgrounds will help your project—but be mindful that having writers and mentors in too many conflicting time zones may require a lot of extra effort to maintain good communication

Hiring

  • Use a contract that clearly spells out deliverables, payment schedules, and specific time commitments
  • If your project has a lot of unknowns, include a milestone for discovery or research separate from documentation creation

Coordination and communication

  • Keep a meeting log recording decisions to make it easier for everyone working on the project to understand context and next steps
  • Be clear about what communication is expected and how often, whether that’s weekly calls, daily emails, or status updates in a chat channel
  • Be responsive and give clear feedback that includes 'why' and not just 'what’
  • Connect your technical writers with the wider community to give them context and to socialize their work

Processes and tooling

  • Create a documentation process that will last beyond the Season of Docs program, and that the whole community can contribute to
  • Documentation review will take at least as long and is just as intensive as code review; make sure you allow enough time for it

Some recommendations have been edited and condensed for clarity.

As in the 2021 program, most technical writers in Season of Docs 2022 applied directly to the organizations they worked with.

A bar graph showing the source of technical writer candidates: Applied directly to program: 18; SoD GitHub or previous SoD participant: 6; Community member: 5; Not specified: 3; Applied via jobs site: 1

Common issues in working with technical writers

A bar graph showing technical writer issues: TW dropping out: 4 projects; Communications issues, TW onboarding, TW skills, Lack of domain knowledge, Hardware confiscated, Conflict with other ongoing work: 1 project each

Fewer projects reported issues working with their technical writers in the 2022 program. Technical writers being unable to complete the program was the largest issue, due to illness, taking a full-time job, or being unable to meet the time commitment.

One project reported that their documentation project was dependent on work being done as part of Google Summer of Code, and that those dependencies were difficult to manage. Another project ran into difficulties when the hardware their technical writer needed to document was confiscated by the Ministry of Defense in the writer’s country and could not be imported.

Followup surveys

Three followup surveys will be sent to the 2022 participants in May, August, and November of 2023. We will update this section with results as they are received.

Future questions

As always, the more we learn about documentation in open source, the more we want to learn!

In future seasons, we hope to:

  • Collect more project metadata to look for correlations between project age, community size, or language and documentation needs
  • Analyze documentation projects to see if they can be generalized into shareable templates
  • Develop a rubric for interviewing technical writers in open source projects

Although there are many questions we'd like to investigate, we also want to respect the time of the open source project admins and maintainers who participate in Season of Docs. The top priority of the program is supporting projects in solving their problems with documentation.