HWW(mumu) NN/BDT Studies

Andy Haas

DØ HWW Meeting

July 10, 2008

Infrastructure

nn_training.C

COLLIE

Data / MC
Rootuples

CalcNN /
PChannel

Plots

Plots

NN_165.cxx

NN Studies

COLLIE : CLfast (no systematics)

mH=165 GeV

NN output from -.2 to 1.2 (300 iters, 8 hidden neurons)

80 bins: 4.439

40 bins: 4.515

20 bins: 4.614

Training iterations (80 bins, 8 hidden neurons)

(1200 – overtrained, limit was worse)

600: 4.399

300: 4.439

150: 4.690

50: 5.142

10: 5.191

NN Studies

Number of hidden neurons (300 iters, 80 bins):

24: 4.423

16: 4.428

12: 4.438

8: 4.415

4: 4.705

Min dR(mu,jet)

For mh=120 (600 iters, 80 bins, 8 hidden)

no cut: 24.420

minmujetdr>0.1: 21.269

An asside: dR<0.5

Plenty of signal, and little background in dR(mu,mu)<0.5 !

Only reason we currently cut it out is due to poor choice of muon track isolation definition

Simply subtract other muon's pT from track isolation if dR<0.5

Cal iso is fine in dR<0.5

Track iso has trouble!

Infrastructure

nn_training.C
(train NN,
define train/test samples)

COLLIE

Data / MC
Rootuples

CalcNN /
PChannel

Plots / Rootuple (w/ NN)

Plots

NN_165.cxx

TMVAnalysis.C

(training)

TMVApplication.C

(reading / testing)

Plots / Rootuple (w/ NN, BDT, etc...)

BDT_165.C

TMVA Studies

Compare a Boosted Decision Tree (BDT) to old NN

mH=165 GeV, 8 hidden, 80 bins, 600 iters

Added log10musumiso into NN and required minmudrjet>0.1

Old NN: 4.03, 3.99, 4.00

BDT (prune strength 4): 3.99

BDT (prune strength 4.5): 3.69

BDT (prune strength 5): 3.72

Old NN (run through TMVA code): 4.05, 4.03, 4.06

MLP (TMVA NN): 3.95, 3.99, 3.91

Conclusion:

I don't have a bug transporting around the old NN values :)

BDT is slightly better, at PS=4.5 (5%)

MLP is about the same as old NN

TMVA Studies

mH=120 GeV

Old NN: 21.6, 21.7, 21.8

BDT (prune strength 4): 21.2

BDT (prune strength 4.5): 21.0

BDT (prune strength 5): 21.6

Old NN (run through TMVA code): 21.6, 21.9, 21.7

MLP (TMVA NN): 25.3, 25.9, 26.0

Conclusion:

BDT is just a little better, at PS=4.5 (2-3%)

MLP has gotten over-trained (less background stat here)

TMVA Studies

Compare a Boosted Decision Tree (BDT) to old NN

mH=165 GeV, 8 hidden, 80 bins, 600 iters

Added log10musumiso into NN and required minmudrjet>0.1

Allow BDT and MLP to use old NN output as an input

Old NN: 4.00, 4.02, 4.04, ...

BDT (prune strength 3): 3.78

BDT (prune strength 4): 3.59, 3.64

BDT (prune strength 4.5): 3.66, 3.60

BDT (prune strength 5): 3.82, 3.90

BDT (prune strength 6): 3.93

Old NN (run through TMVA code): 4.08, 4.02, 4.04

MLP (TMVA NN): 4.33, 4.35, 4.30, ...

Conclusion:

BDT+NN is significantly better, at PS=4.5 (10%)

MLP has gotten over-trained (less background stat here)

TMVA Studies

mH=120

Allow BDT and MLP to use old NN output as an input

Old NN: 21.5, 22.0, 21.5, 21.8, 21.7, ...

BDT (prune strength 2): 21.6

BDT (prune strength 3): 20.0

BDT (prune strength 4): 20.3, 19.9

BDT (prune strength 4.5): 19.5, 19.6

BDT (prune strength 5): 19.0, 19.2

BDT (prune strength 6): 20.0

BDT (prune strength 7): 20.8

Old NN (run through TMVA code): 21.8, 21.8, 22.0...

MLP (TMVA NN): 23.0, 22.8, 23.1, 23.1, ...

Conclusion:

BDT+NN is significantly better, at PS=4.5-5 (10-12%)

MLP has gotten over-trained (less background stat here)

TMVA Studies

mH=200

Old NN: 14.0, 14.0, 14.1

BDT (prune strength 4): 12.5 (12.1) (w/NN input)

BDT (prune strength 4.5): 12.7 (12.1) (w/NN input)

BDT (prune strength 5): 12.9 (12.1) (w/NN input)

Old NN (run through TMVA code): 14.0, 13.9, 14.0

MLP (TMVA NN): 13.6, 13.8, 13.9

Conclusion:

BDT is quite a bit better (9%)

BDT+NN is significantly better (14%)

MLP+NN is just slightly better

Andy Haas

2004-07-07T15:59:13