Comprehensive report: Handing raw data

[janagombitova]

Context

Why Comprehensive reports

The value of Comprehensive reports is to give Salim and Petra a quick summary of the data. Without them having to calculate frequencies or sums, we do the work for them and show standard simple summaries of the data for a form. This helps them to get a quick understanding of the data or of the data collection progress.

Once the want to analyse the data further they can use the Data analysis export or Akvo Lumen.

Current structure

Currently a Comprehensive report consists of two sheets: one holding the raw data and a Summary sheet with calculated frequencies, sums, and charts.

The raw data sheet however has a structure of the old raw data report structure repetitions are in rows in one sheet.

Opportunity

What should the layout of the raw data sheet be? Should is follow the structure of one of the data export options we now have (Data cleaning or Data analysis)?

Why do we put that raw data in this report in the first place, if the user can export it differently and chooses this one for the summaries?

Next steps

We agreed to check with a few of our users why they use these reports, how they work with them and for what reasons, before we make a decision on how to go about the changes

[janagombitova]

From user research

"I know partners like seeing the summary data as a way to glance at things before they do proper analysis (i.e. helvetas, sustainable harvest). As I think about it, I think I wondered why they also have the raw data report…"

"We use the charts and summary primarily, but do sometimes cross check in the raw data. But that we can also download separately. Sometimes we copy and paste the visuals directly from the comprehensive report into powerpoints or reports."

"Ultimately I could just download the raw data separately. Only thing is having them together makes version control easier. Like if the raw data is taken out completely, and I am using the comprehensive report and raw data sheets separately they could end up out of sync. But that is possibly quite a small worry. I do find the different structure of the raw data confusing"

I have not done an extensive research as this is feature is used by a few but they agreed that the value of this export type/report is in getting a quick overview of the data. Some partners said that they then actually use the raw data sheet in the report for further analysis, but also noted the can download that as a separate sheet. They also concluded that they actually would use the new data exports to get the raw data as the structure is better for them in the Data analysis report as options are split.

Opportunities

Option 1 - Remove raw data sheet

We can consider removing the raw data file from this report and only have it holding a summary of the data set and overviews for the questions. This will then create a more clear split on the value of this report and will not have overlapping functionality confusing users. If they want to work with the data they select Data cleaning, if they want to analyse they select the Data analysis one and to get a quick view of the data, they select this one. (Eventually in the future, we can consider how to show data summaries in the UI, but this also depends on the future of Flow and Lumen.)

Currently the report holds a summary of the data set, some calculations, overview and charts for number questions, option and cascade questions.

@stellanl pointed out, if we remove the raw data sheet, we then need to add all the questions to the Summary sheet. Karolina noted that we could for the other question types simply show the total number of submissions made for the question and the number of empty cells.

@muloem then pointed out that we should rename the report as the word 'comprehensive' means it holds everything. Mulo suggested Summary report as a more descriptive, simple and clear alternative.

Option 2 - Keep the raw data, unify its structure

As one partner in the Pacific mentioned, they like having the report holding both stats and data as they know the stats are based on this version of the data set. By removing the raw data, version control will become harder.

If we choose for this option, we then need to tackle the fact that the structure of the raw data sheet now is not reflecting the recent changes we made. Here I would suggest to choose we either show the data using the Data cleaning export format or the Data analysis format. My choice would be Data analysis, as it keeps the data in one sheet and is meant for users to use to analyse the data, something the Comprehensive report is giving them a little helping hand.

Next steps

Decide.
Mulo and Stellan do you have any thoughts, comments?

[stellanl]

We get very little simplification from removing the raw data sheet, since the code is all there for the other reports. Maybe as little as removing one line of code. And we have to fetch all the data to create the collection statistics. So no performance gain either. I'm happy with option 2+analysis format.

[janagombitova]

All right. Then if we do not gain on simplification, let's go for the less risky one from the users' point of view and keep the concept of comprehensiveness for now. Option 2 it is 👍

[janagombitova]

Looks good 👍