Product Analytics Documentation. What, Why, and Where to start?
Every IT product is a complex system that needs Internal Product Documentation (IPD) — a single source of truth that every member of the Product team could refer to. People responsible for IPD (Technical Writers, Business Analysts, Product Managers) often focus on its Technical part — the software architecture, APIs, and business logic of product components. And while this documentation is vital for the team, it benefits mainly Developers, QA engineers, and Product Managers, overlooking Product Analysts’ interests.
Despite Product Analytics Documentation is not as discussed as, for example, API Documentation, it can drastically improve team collaboration and speed up product development.
In this article, I want to share some gained-from-personal-experience insights on what Product Analytics Documentation is, why it should be a part of Internal Product Documentation, and where to start building it.
What is Product Analytics Documentation?
To understand whether users like a product or not, analyze the effectiveness of particular features, and adjust the product development strategy in the right direction, we need quantitative data. Therefore, all user interactions with the product are tracked, stored in the database, and processed by Product Analysts to derive valuable qualitative insights on improving user experience.
While all products track analytics, approaches to its processing vary from one company to another. That’s why there is no universal definition for Product Analytics Documentation (PAD).
Depending on the product specifics, PAD may include the following aspects:
- business metrics (what they are and how they are calculated);
- collecting and storing data (which data is collected and where it is stored);
- A/B testing (how A/B tests are conducted for the product, where test logs are, etc.);
- reporting and data visualization (how to interpret analytical reports and graphs).
Usually, Product Analysts ask questions that help identify what Product Analytics Documentation should reveal.
Why document Product Analytics?
Despite Product Analytics Documentation sounds like it serves only Product Analysts, it brings much more to the table:
- Deep understanding of the product
Having detailed and coherent PAD enables the Product team to interpret quantitative data in close connection to user experience and thus develop more effective strategies for its enhancement. It also helps all teams working on the product see the big picture — what is the product’s primary purpose, and what is the team’s role in fulfilling the purpose. - Сonscious development
Changes in the product functionality can break tracking analytics. While developers fix issues that caused the break, the product loses valuable data and time (eventually converted to business profit). PAD helps restore the initial process of tracking analytics and minimize the time spent identifying and eliminating weak points in code. - Fast onboarding
It is also about saving time. With relevant PAD, the Product team can reduce the number of onboarding meetings and spend the time more wisely. - Better communication
PAD is the universal language that prevents the Product team from building The Tower of Babel. When everyone in the team is on the same page, the risk of delivering something opposite to what was expected in the first place is minimized.
Where to start?
In our team, we started working on Product Analytics Documentation with data. To be more specific, we started with documenting database tables that store analytical data. We assumed that every article of PAD would refer to these tables, so it seemed logical to start on the data layer.
Tables in the database
Analytical tables store valuable information about user interactions with the product. They need detailed explanations for Product Analysts to reach a perfect exactitude in data interpretation.
From multiple requests from our Product Analysts, we figured out the following questions that must be answered when explaining analytical tables:
- What are the triggers to add, modify, and delete records in a table?
Whether a new record is created by the system or caused by a user action, under which conditions an existing record can be modified, and where records removed from the table are stored (backed up) — all these inquiries will eventually rise. Even if those explanations seem obvious and intuitive, it’s better to write them down.
Example
Assume we have a table session_profile that records whenever users open the My profile page on the website. The section dedicated to data lifecycle in the table may look like this:
Notes
1. External links
Readers should not have gaps in understanding the data lifecycle. External links to related articles will answer potential readers’ questions like ‘Where the backup is stored?’ and ‘How do I know that a session expires?’
2. Examples
Examples that cover tricky cases help readers comprehend the process of tracking analytics. You can hide them inside expanding content elements not to overload the article.
- What are possible values for columns that represent types of something?
When it comes to storing large amounts of data, using a number format for particular parameters saves memory space occupied by the data and speeds up the procedures of searching and sorting. Therefore Developers often use digits to represent a defined set of possible values for a table column. And explaining each option will definitely save Product Analysts’ time and speed up the procedures of analyzing and interpreting data.
Example
Among the fields in the session_profile table, there is a column source that indicates where the user came from when they opened the My profile page. Possible values in the column are integers from 1 to 4, and each integer corresponds to a particular source:
Note
References to related tables
To build analytical reports, Product Analysts create complicated SQL queries with multiple JOIN clauses to combine rows from several tables. While an experienced Analyst familiar with the product database can quickly figure out what tables are related to each other, a newcomer may spend time exploring those connections. Therefore, it is helpful to refer to related tables
(like in id_session and id_user columns).
- What are the functional meanings of bit flags in a table?
Bit flags allow storing multiple states or characteristics of a record in one column in numerical format. Bitwise operators are used to figure out whether a record has a particular characteristic or not.
I recommend creating a child article dedicated to each flag column in a table to give a profound explanation for every possible bit.
Example
In the session_profile table, there is a column flags that gives additional details on how exactly a user opened the My profile page — whether they used a mobile phone or a desktop computer; if they logged in with a phone number or email:
Note
References to theory
When Product Documentation is built upon a known technical concept, it is better to refer readers to an article where the concept is explained understandably. Such a reference saves time for team members who are not familiar with the subject in searching for information sufficient to grasp the idea of the Product Documentation.
When documenting tables and views in a database, don’t forget to give your readers the hosting address, where they can find it, and instructions on how to access the data.
User scenarios in the context of tracking analytics
Documentation on analytical tables reveals what data is collected and where it is stored but not how it is recorded. And while explaining user scenarios to the Product team, who designed those scenarios in the first place, seems pointless, describing user scenarios emphasizing recording analytics sounds very reasonable.
But what to do if literally everything in the product is covered with analytics?
Sometimes, even when a user waits until a page is loaded, analytics on the loading is being recorded. Unless you have a Technical Writer assigned to each product’s feature, it is impossible to document all user scenarios where they track analytics.
A tip that helps understand which user scenarios to cover in Product Analytics Documentation is to know the product’s main objectives.
Let’s consider a job board platform. For some vacancies, the platform redirects job seekers to original vacancy sources; for others, it enables them to apply for vacancies directly on the platform.
The ultimate goals are the following:
1. to help people find a job in the shortest time;
2. to provide employers with the most relevant candidates.
Based on these objectives, we understand that Product Analysts will be focused on analyzing two product funnels — how a user applies for a vacancy and how they create a profile. So, documentation for these two processes has the highest priority.
How to document a user scenario?
I wish I had a best practice or universal template for documenting user scenarios in the most concise and informative way. Since I don’t have one, I will share what works for our Product team and inspire you to elaborate on the approach.
So, our procedure implies two steps:
- Dividing a user scenario into steps
The steps are those milestones that users pass to achieve the ultimate goal.
They can be as atomic as filling a single input field on a form or more general, like submitting the whole form and getting to the next one. - Creating a Product analytics section for each step
For each step (milestone), we create a discernible panel called Product analytics, where we describe what data is tracked within the milestone and the tables where this data is recorded.
Note
References to analytical tables
As we mentioned before, there should be references to the documentation on analytical tables in every article of PAD and user scenarios are no exception. Links to the corresponding tables allow us to omit excessive details about each flag and type in articles dedicated to user scenarios. If readers want to learn more about a particular flag, they follow the link.
So, I hope you can take away some valuable thoughts from the article.
It takes time to realize that Product Analytics deserves a special place in Internal Product Documentation. It takes even more time to develop a working approach to maintaining Product Analytics Documentation. But the earlier you start, the sooner you will pick the harvest in the form of better communication within the team and more profound understanding of the product.
