Chek the ATLAS Z0 8 TeV low mass data set

[achiefa]

This PR addresses #2267.

[achiefa]

Dear @enocera,

As mentioned in issue #2267, I've implemented the "light" variant of the dataset. In this variant, statistical and systematic uncertainties are taken from Table 6 in 1710.05167, on top of which I included the luminosity uncertainty (1.9%). However, I took the liberty to play with the luminosity uncertainty and I combined it with different variants, which are listed below:

1. HepData v3 (ATLASLUMI12): Here I took version 3 of the HepData table and I correlated the luminosity uncertainty using the key ATLASLUMI12. This corresponds to the new default implementation.
2. HepData v3 (CORR LUMI): As (1), but now the luminosity is correlated within the same experiment (key CORR).
3. HepData v3 (UNCORR LUMI): As (1), but now the luminosity is not correlated at all (key UNCORR).
4. HepData v1 (ATLASLUMI12): As (1), but with version 1 in HepData
5. HepData v1 (CORR LUMI): As (2), but with version 1 in HepData
6. HepData v1 (UNCORR LUMI): As (3), but with version 1 in HepData
7. Syst. light (ATLASLUMI12): As (1), but using the light version of systematic uncertainties.
8. Syst. light (CORR LUMI): As (2), but using the light version of systematic uncertainties.
9. Syst. light (UNCORR LUMI): As (3), but using the light version of systematic uncertainties.

For each of these variants, I computed the $\chi^2$ to ATLAS_Z0_8TEV_LOWMASS using the same pdf set (250122-jth-01-data-check, the one used by RS in his report). You can find the report here.

I think the message is rather clear -- the $\chi^2$ improves when the luminosity uncertainty is not correlated. This is true for version 3 on HepData and the light version (3 and 9 in the list above). On the other hand, the $\chi^2$ for version 1 is always high (~50) regardless of the type of correlation assigned to the luminosity.

The list above should provide a comprehensive set of combinations useful for this investigation. Please, let me know if something is missing. Also, I've pushed the uncertainty data files that I used to produce the variants, so that you can have a look if something does not convince you.

P.S. In the original re-implementation, the luminosity uncertainty was set to 1.8%, although the paper claims 1.9%. Apart from version 1, which already includes (God knows what) luminosity uncertainty in the HepData table, all the other variants use 1.9%.

[scarlehoff]

Some results from today's code meeting, using directly the NNLO grids (which are slightly different) to compute the chi2 and NNPDF40_nnlo_as_01180 as PDF.

legacy: 0.6911
default: 8.743
default with CT18Z: 5.649

[achiefa]

Dear @enocera $ @scarlehoff,

As agreed in the last code meeting, I computed the values of the $\chi^2$ for each bin in the invariant mass of the lepton pair. The dataset comes with seven bins, although the last two are excluded by the cuts in the NNPDF4.0. For each bin, I computed the $\chi^2$ using all the dataset variants that I discussed in the previous comment, and you can find the reports below (bins are labelled as $[m_{\ell\ell, min}, m_{\ell\ell, max}]$ GeV):

1. $[46, 66]$: link
2. $[66, 80]$: link
3. $[80, 91]$: link
4. $[91, 102]$: link
5. $[102, 116]$: link
6. $[116, 150]$: link
7. $[150, 200]$: link

First of all, version 1 of HepData always results in an odd $\chi^2$ regardless of the bin and of the correlation of the luminosity uncertainty. Therefore, I think we can rule it out once and for all.

Second, the $\chi^2$ for the new implementation is good for bins (1), (2), and (5), which incidentally are those that do not include the $Z$-peak. Furthermore, for bins (2) and (5) the $\chi^2$ of the new implementation is better than the legacy value upon inclusion of the correlation of the luminosity.

Finally, the origin of the crazily high $\chi^2$ that we observed might origin from the bins that include the peak at $m_Z$, namely (3) and (4). Indeed, there we can see the problematic values of the $\chi^2$ when we account for the correlation of the luminosity. Note that, for these bins, the "light" version does not help reduce the $\chi^2$ either.

[scarlehoff]

Thank you very much for this @achiefa
Is this the same dataset that was problematic in the pheno paper? I thought we had checked that the old version was ok even for the peak or is this just another dataset which happens to also have problems in the peak?

edit: silly me, the old version was always legacy so luminosity was always uncorrelated

[achiefa]

edit: silly me, the old version was always legacy so luminosity was always uncorrelated

Indeed, I was just about to say that.

edit:
But maybe that can explain why we could not describe well the new ATLAS analysis at $Z$-peak. That's just me thinking out loudly, but maybe the new ATLAS analysis, which is at the $Z$-peak, fixes the problematic correlation of the luminosity. If that is true, then the old dataset (ATLAS_Z0_8_TEV_LOWMASS @ $Z$-peak) and the new one are simply incompatible.

[enocera]

edit: But maybe that can explain why we could not describe well the new ATLAS analysis at Z -peak. That's just me thinking out loudly, but maybe the new ATLAS analysis, which is at the Z -peak, fixes the problematic correlation of the luminosity. If that is true, then the old dataset (ATLAS_Z0_8_TEV_LOWMASS @ Z -peak) and the new one are simply incompatible.

Very well thought - this is the explanation I'm leaning towards.

[scarlehoff]

The new one is incompatible also with other datasets though, since we were not able to fit it by itself without mhou...

[enocera]

@achiefa Question: all these chi2 are w/o MHOUs, right? So that we can compare with the old NNPDF4.0 fit?

[achiefa]

Yes, all those chi2's are w/o MHOUs.

[enocera]

Dear @achiefa I looked into the MSHT paper, see in particular Fig. 7 of https://arxiv.org/pdf/2012.04684. They have cuts on the ATLAS 8 TeV 2D low mass data, in particular they exclude, for each invariant mass bin, the two most forward rapidity bins. They also remove the lowest invariant mass bin altogether. Could you please try to recompute the chi2 with these cuts? Thanks.

[achiefa]

Hi @enocera,

I will compute the chi2. On top of the cuts that you mentioned, should I relax the cuts that remove the two last bins in the invariant mass?

[enocera]

In our framework, we want to always remove the last two invariant mass bins, because these are incorporated in the high-mass data set. So I would NOT relax the cuts and continue to exclude the last two invariant mass bins.

[achiefa]

Follow-up

I computed the $\chi^2 as requested by @enocera, namely applying the same cuts as MSHT. You can find the report here. As far as I can understand, I don't see any improvement in the $\chi^2$ - actually, the value worsens in any dataset variant.

I also computed the $\chi^2$ for two other configurations:

• computing the $\chi^2$ for all the bins but for those at the Z peak using the same set as in the previous comments (250122-jth-01-data-check). The last two invariant mass bins are still excluded. The report is here.
• computing the $\chi^2$ using 241003-01-rs-mhou which includes MHOUs. The report is here.

I'll compute the $\chi^2$ for other configurations, for instance as done in the pheno paper where the calculation of the $\chi^2$ includes the covmat for MHOUs.

[enocera]

Thanks, Amedeo. So should we conclude that, as soon as we put all the invariant mass bins together (but those on the Z peak), we get a horrible chi2?

[achiefa]

I think it depends on what we are looking at. The report in the first bullet point should be compared with the report I generated for the first comment (here for brevity). Then what I see is that the legacy implementation - which does not correlate the luminosity - worsens, but the $\chi^2$ improves when we correlate the luminosity, that is for the variant implemented by Mark (version 3).