CMS dijet dataset (blinded)¶
CMS search for high-mass dijet resonances at sqrt(s) = 13 TeV
Differential dijet spectrum (Figure 5), public data taken from HEPDATA
See >>notebook<< for the complete procedure.
This fit generates 21 candidate functions in total! The output files can be found here (feel free to download them and look at what a typical fit will produce).
Lets look at the output file candidates_reduced.csv, which is a csv table storing all candidate functions and their evaluations:
| Unnamed: 0 | Parameterized equation, unscaled | Parameters: (best-fit, +1, -1) | Covariance | Correlation | RMSE | R2 | NDF | Chi2 | Chi2/NDF | p-value |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1.0*(a1) | {'a1': (-0.0797, 0, 0)} | {} | {} | 37.11 | -0.218 | 35 | 8.035e+07 | 2.296e+06 | 0 |
| 1 | 1.0*(a1) | {'a1': (0.000278, 0, 0)} | {} | {} | 37.07 | -0.2158 | 35 | 6.359e+06 | 181700 | 0 |
| 2 | 1.0(a1*((x0 - 1568.5) * 0.000145275)) | {'a1': (0.000744, 0, 0)} | {} | {} | 36.73 | -0.1931 | 35 | 6.161e+06 | 176000 | 0 |
| 3 | 1.0(a1((x0 - 1568.5) * 0.000145275)a2) | {'a1': (1.26e-06, 0, 0), 'a2': (75.568, 6.25, -6.25)} | {} | {} | 15.49 | 0.7878 | 34 | 1.2e+06 | 35300 | 0 |
| 4 | 1.0(a2*(a1 + ((x0 - 1568.5) * 0.000145275))) | {'a1': (-0.346159, 0.0227, -0.0227), 'a2': (6.89e-05, 0, 0)} | {} | {} | 28.25 | 0.2939 | 34 | 3.923e+06 | 115400 | 0 |
| 5 | 1.0(a2(a1 + a3((x0 - 1568.5) * 0.000145275))) | {'a1': (-0.689, 0, 0), 'a2': (0.00075246, 1.81e-05, -1.81e-05), 'a3': (3.2893, 0.0409, -0.0409)} | {'a2, a3': -4.1023686239306564e-07, 'a2, a2': 3.2629197974065944e-10, 'a3, a3': 0.0016756906326589956} | {'a2, a3': -0.5542} | 1.466 | 0.9981 | 33 | 28850 | 874.2 | 0 |
| 6 | 1.0(a2(a1 + a3tanh(((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.689, 0, 0), 'a2': (0.000751809, 1.67e-05, -1.67e-05), 'a3': (3.29696, 0.0378, -0.0378)} | {'a2, a3': -3.494976698931696e-07, 'a2, a2': 2.8030361965513605e-10, 'a3, a3': 0.0014298238686357306} | {'a2, a3': -0.5537} | 1.442 | 0.9982 | 33 | 24920 | 755.3 | 0 |
| 7 | 1.0(a2(a1 + ((x0 - 1568.5) * 0.000145275) + tanh(a3((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.774854, 0.02, -0.02), 'a2': (0.00163843, 0.000269, -0.000269), 'a3': (2.79769, 0.113, -0.113)} | {'a1, a2': -5.367970330708592e-06, 'a1, a3': -0.0021926039143888676, 'a2, a3': 2.8942686130441554e-05, 'a1, a1': 0.00040189124766762895, 'a2, a2': 7.214059239985313e-08, 'a3, a3': 0.012825212348295487} | {'a1, a2': -0.9978, 'a1, a3': -0.9702, 'a2, a3': 0.9522} | 1.068 | 0.999 | 32 | 10090 | 315.4 | 0 |
| 8 | 1.0(a2*(a1 + ((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/(a3 + ((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.571056, 0.00296, -0.00296), 'a2': (0.00015696, 6.95e-06, -6.95e-06), 'a3': (0.471326, 0.00495, -0.00495)} | {'a1, a2': -2.0509525530262746e-08, 'a1, a3': 1.4370853433639262e-05, 'a2, a3': -3.34452223607015e-08, 'a1, a1': 8.751022732270554e-06, 'a2, a2': 4.825362026153916e-11, 'a3, a3': 2.45133472729267e-05} | {'a1, a2': -0.997, 'a1, a3': 0.9808, 'a2, a3': -0.9722} | 0.1218 | 1 | 32 | 243.3 | 7.602 | 2.464e-34 |
| 9 | 1.0(a2*(a1 + ((x0 - 1568.5) * 0.000145275) + (a3 + ((x0 - 1568.5) * 0.000145275))/(a4 + ((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.572265, 0.00296, -0.00296), 'a2': (0.000156067, 6.89e-06, -6.89e-06), 'a3': (0.000744, 0, 0), 'a4': (0.471007, 0.00494, -0.00494)} | {'a1, a2': -2.0363519749487308e-08, 'a1, a4': 1.4369880094769795e-05, 'a2, a4': -3.310371226849207e-08, 'a1, a1': 8.775992686140511e-06, 'a2, a2': 4.743811645342973e-11, 'a4, a4': 2.4433297646394353e-05} | {'a1, a2': -0.9985, 'a1, a4': 0.9827, 'a2, a4': -0.9726} | 0.1215 | 1 | 32 | 242.5 | 7.577 | 3.489e-34 |
| 10 | 1.0(a2*(a1 + ((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + ((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.669584, 0.00103, -0.00103), 'a2': (0.000565992, 6.31e-06, -6.31e-06), 'a3': (0.384834, 0.00134, -0.00134)} | {'a1, a2': -6.463395121994439e-09, 'a1, a3': 1.341030640216447e-06, 'a2, a3': -8.101339978407406e-09, 'a1, a1': 1.0572283627667835e-06, 'a2, a2': 3.976767752781891e-11, 'a3, a3': 1.8075023131422993e-06} | {'a1, a2': -0.9945, 'a1, a3': 0.9716, 'a2, a3': -0.9581} | 0.02931 | 1 | 32 | 33.78 | 1.056 | 0.3814 |
| 11 | 1.0(a2*(a1 + ((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + ((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.669584, 0.00103, -0.00103), 'a2': (0.000565992, 6.31e-06, -6.31e-06), 'a3': (0.384834, 0.00134, -0.00134)} | {'a1, a2': -6.463395121994439e-09, 'a1, a3': 1.341030640216447e-06, 'a2, a3': -8.101339978407406e-09, 'a1, a1': 1.0572283627667835e-06, 'a2, a2': 3.976767752781891e-11, 'a3, a3': 1.8075023131422993e-06} | {'a1, a2': -0.9945, 'a1, a3': 0.9716, 'a2, a3': -0.9581} | 0.02931 | 1 | 32 | 33.78 | 1.056 | 0.3814 |
| 12 | 1.0(a2(a1 + ((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + a4((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.680024, 0.00621, -0.00621), 'a2': (0.000635294, 4.29e-05, -4.29e-05), 'a3': (0.377228, 0.00458, -0.00458), 'a4': (0.950788, 0.0274, -0.0274)} | {'a1, a2': -2.6660946193355025e-07, 'a1, a3': 2.8216241810658335e-05, 'a1, a4': 0.00016801306414184462, 'a2, a3': -1.9478264643595885e-07, 'a2, a4': -1.162402085830468e-06, 'a3, a4': 0.00012062842616929576, 'a1, a1': 3.857613369715255e-05, 'a2, a2': 1.8429643143208677e-09, 'a3, a3': 2.0973917547886757e-05, 'a4, a4': 0.000752768131501252} | {'a1, a2': -1, 'a1, a3': 0.9921, 'a1, a4': 0.9874, 'a2, a3': -0.9914, 'a2, a4': -0.9889, 'a3, a4': 0.9612} | 0.0237 | 1 | 31 | 30.54 | 0.985 | 0.4897 |
| 13 | 1.0(a2(a1 + ((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + a4((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.680024, 0.00621, -0.00621), 'a2': (0.000635294, 4.29e-05, -4.29e-05), 'a3': (0.377228, 0.00458, -0.00458), 'a4': (0.950788, 0.0274, -0.0274)} | {'a1, a2': -2.6660946193355025e-07, 'a1, a3': 2.8216241810658335e-05, 'a1, a4': 0.00016801306414184462, 'a2, a3': -1.9478264643595885e-07, 'a2, a4': -1.162402085830468e-06, 'a3, a4': 0.00012062842616929576, 'a1, a1': 3.857613369715255e-05, 'a2, a2': 1.8429643143208677e-09, 'a3, a3': 2.0973917547886757e-05, 'a4, a4': 0.000752768131501252} | {'a1, a2': -1, 'a1, a3': 0.9921, 'a1, a4': 0.9874, 'a2, a3': -0.9914, 'a2, a4': -0.9889, 'a3, a4': 0.9612} | 0.0237 | 1 | 31 | 30.54 | 0.985 | 0.4897 |
| 14 | 1.0(a2(a1 + ((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + a4((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.680024, 0.00621, -0.00621), 'a2': (0.000635294, 4.29e-05, -4.29e-05), 'a3': (0.377228, 0.00458, -0.00458), 'a4': (0.950788, 0.0274, -0.0274)} | {'a1, a2': -2.6660819037114085e-07, 'a1, a3': 2.821617543895786e-05, 'a1, a4': 0.00016801285852781304, 'a2, a3': -1.94781858760041e-07, 'a2, a4': -1.1623987038947393e-06, 'a3, a4': 0.00012062836038378047, 'a1, a1': 3.857601482911213e-05, 'a2, a2': 1.8429524145356392e-09, 'a3, a3': 2.0973884696884504e-05, 'a4, a4': 0.0007527686452301347} | {'a1, a2': -1, 'a1, a3': 0.9921, 'a1, a4': 0.9874, 'a2, a3': -0.9913, 'a2, a4': -0.9889, 'a3, a4': 0.9612} | 0.0237 | 1 | 31 | 30.54 | 0.985 | 0.4897 |
| 15 | 1.0(a2(a1 + ((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + a4((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.680024, 0.00621, -0.00621), 'a2': (0.000635294, 4.29e-05, -4.29e-05), 'a3': (0.377228, 0.00458, -0.00458), 'a4': (0.950788, 0.0274, -0.0274)} | {'a1, a2': -2.6660793941944285e-07, 'a1, a3': 2.8216162337635693e-05, 'a1, a4': 0.00016801281789017417, 'a2, a3': -1.9478170322693104e-07, 'a2, a4': -1.1623980356636931e-06, 'a3, a4': 0.00012062834731059505, 'a1, a1': 3.8575991382226084e-05, 'a2, a2': 1.8429500654313467e-09, 'a3, a3': 2.0973878202711968e-05, 'a4, a4': 0.0007527687458505787} | {'a1, a2': -1, 'a1, a3': 0.9921, 'a1, a4': 0.9874, 'a2, a3': -0.9913, 'a2, a4': -0.9889, 'a3, a4': 0.9612} | 0.0237 | 1 | 31 | 30.54 | 0.985 | 0.4897 |
| 16 | 1.0(a2(a1 + a4((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + ((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.679, 0, 0), 'a2': (0.000627721, 6.06e-07, -6.06e-07), 'a3': (0.381219, 0.000983, -0.000983), 'a4': (1.03087, 0.0035, -0.0035)} | {'a2, a3': 3.2023612057736235e-10, 'a2, a4': 7.929003975748651e-10, 'a3, a4': 3.2365147886287552e-06, 'a2, a2': 3.676961654714449e-13, 'a3, a3': 9.663605191471658e-07, 'a4, a4': 1.2238492430854328e-05} | {'a2, a3': 0.5376, 'a2, a4': 0.3738, 'a3, a4': 0.9407} | 0.03167 | 1 | 32 | 35.2 | 1.1 | 0.3191 |
| 17 | 1.0(a2(a1 + a4((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + ((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.679, 0, 0), 'a2': (0.000627721, 6.06e-07, -6.06e-07), 'a3': (0.381219, 0.000983, -0.000983), 'a4': (1.03087, 0.0035, -0.0035)} | {'a2, a3': 3.2023612057736235e-10, 'a2, a4': 7.929003975748651e-10, 'a3, a4': 3.2365147886287552e-06, 'a2, a2': 3.676961654714449e-13, 'a3, a3': 9.663605191471658e-07, 'a4, a4': 1.2238492430854328e-05} | {'a2, a3': 0.5376, 'a2, a4': 0.3738, 'a3, a4': 0.9407} | 0.03167 | 1 | 32 | 35.2 | 1.1 | 0.3191 |
| 18 | 1.0(a2(a1 + a4((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + ((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.679, 0, 0), 'a2': (0.000627721, 6.06e-07, -6.06e-07), 'a3': (0.381219, 0.000983, -0.000983), 'a4': (1.03087, 0.0035, -0.0035)} | {'a2, a3': 3.2023641703661977e-10, 'a2, a4': 7.929010670464016e-10, 'a3, a4': 3.2365202947876796e-06, 'a2, a2': 3.6769619145435207e-13, 'a3, a3': 9.663623445588988e-07, 'a4, a4': 1.2238509387667279e-05} | {'a2, a3': 0.5376, 'a2, a4': 0.3738, 'a3, a4': 0.9407} | 0.03167 | 1 | 32 | 35.2 | 1.1 | 0.3191 |
| 19 | 1.0(a2(a1 + a4((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + ((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.679, 0, 0), 'a2': (0.000627721, 6.06e-07, -6.06e-07), 'a3': (0.381219, 0.000983, -0.000983), 'a4': (1.03087, 0.0035, -0.0035)} | {'a2, a3': 3.2023641703661977e-10, 'a2, a4': 7.929010670464016e-10, 'a3, a4': 3.2365202947876796e-06, 'a2, a2': 3.6769619145435207e-13, 'a3, a3': 9.663623445588988e-07, 'a4, a4': 1.2238509387667279e-05} | {'a2, a3': 0.5376, 'a2, a4': 0.3738, 'a3, a4': 0.9407} | 0.03167 | 1 | 32 | 35.2 | 1.1 | 0.3191 |
| 20 | 1.0(a2(a1 + a3tanh(a5*((x0 - 1568.5) * 0.000145275)) + ((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a4 + ((x0 - 1568.5) * 0.000145275)))) | {'a1': (-0.679, 0, 0), 'a2': (0.00063, 0, 0), 'a3': (0.0180787, 0.00161, -0.00161), 'a4': (0.392088, 0.00343, -0.00343), 'a5': (4.27315, 0.784, -0.784)} | {'a3, a4': 2.8784456094790742e-06, 'a3, a5': 7.047405305671733e-05, 'a4, a5': 0.0023635036819616576, 'a3, a3': 2.6031449636020696e-06, 'a4, a4': 1.178130982461471e-05, 'a5, a5': 0.6153049292230256} | {'a3, a4': 0.5212, 'a3, a5': 0.05583, 'a4, a5': 0.8789} | 0.05388 | 1 | 32 | 36.43 | 1.138 | 0.2701 |
The goodness-of-fit scores are plotted in candidates_gof.pdf, such as the chi2/ndf:
For other goodness-of-fit scores:
Click to expand
^ p-value
^ Root-mean-square error
^ Coefficient of determination R2
Now, lets take a look at one of the candidate functions, say candidate #19. The functional form can be found in the corresponding plots from the PDF files and in the csv table above, which is:
1.0*(a2**(a1 + a4*((x0 - 1568.5) * 0.000145275) + ((x0 - 1568.5) * 0.000145275)/tanh(a3 + ((x0 - 1568.5) * 0.000145275)))).
Here we have set input_rescale = True and scale_y_by = None when configuring the fits.
Therefore the functions here have, i.e., no overall normaliztion (y -> 1.0*y) and an un-standardization (rescaling x -> (x-1568.5)*0.000145275)).
This candidate function has 4 parameters, originally: a1, a2, a3, a4.
However, there are only 3 final varying parameters: a2, a3, a4, as can be seen from the Parameters: (best-fit, +1, -1) column in the csv tables or directly from the pdf files:
{'a1': (-0.679, 0, 0), 'a2': (0.000627721, 6.06e-07, -6.06e-07), 'a3': (0.381219, 0.000983, -0.000983), 'a4': (1.03087, 0.0035, -0.0035)}
where a1 has zeros at both +1 and -1 unc entries, meaning this parameter was held fixed during the re-optimization.
This is because during the re-optimization loop, the objective function was too complex to minimize, therefore some parameters are held fixed to lower the number of degrees of freedom in order to achieve a better fit.
This is common when the functions or the distribution shapes are not very simple.
To see how this candidate function behaves when each of these 3 parameters is varied to its +/-1 sigma value:
Click to expand
^ +/-1 sigma variations of parameter a2
^ +/-1 sigma variations of parameter a3
^ +/-1 sigma variations of parameter a4
^ Correlation matrix
As shown in the correlation matrix, these parameters are not all independent to each other, so it will be nice to see the actual uncertainty coverage considering uncertainties from all parameters in a candidate function.
These are plotted in candidates_sampling.pdf.
Here, what it does is to generate an ensemble of functions for a candidate function by sampling its parameters, where the sampling is done by sampling from a multidimensional normal distribution for the parameters, with the best-fit parameter values being the mean location and the covariance matrix for the parameters being the covarience.
In this way, the total uncertainty is obtained by considering uncertainties from all parameters simultaneously.
Then the 68% quantile range of this function ensemble as green bands in the plots and compared with the input data.
Note the 95% quantile range can also be added by sampling_95quantile = True.