1. num_boost_round – n_estimators
Afterwards, you must decide the variety of resolution timber (usually referred to as base learners in XGBoost) to plant throughout coaching utilizing num_boost_round. The default is 100 however that is hardly sufficient for at this time’s massive datasets.
Growing the parameter will plant extra timber however considerably will increase the possibilities of overfitting because the mannequin turns into extra complicated.
One trick I discovered from Kaggle is to set a excessive quantity like 100,000 for num_boost_round and make use of early stopping rounds.
In every boosting spherical, XGBoost vegetation another resolution tree to enhance the collective rating of the earlier ones. That’s why it’s referred to as boosting. This course of continues till num_boost_round rounds, regardless whether or not every new spherical is an enchancment on the final or not.
However through the use of early stopping, we are able to cease the coaching and thus planting of pointless timber when the rating hasn’t been enhancing for the final 5, 10, 50 or any arbitrary variety of rounds.
With this trick, we are able to discover the right variety of resolution timber with out even tuning num_boost_round and we are going to save time and computation sources. Right here is how it might appear like in code:
# Outline the remainder of the params
params = {...}# Construct the prepare/validation units
dtrain_final = xgb.DMatrix(X_train, label=y_train)
dvalid_final = xgb.DMatrix(X_valid, label=y_valid)
bst_final = xgb.prepare(
params,
dtrain_final,
num_boost_round=100000 # Set a excessive quantity
evals=[(dvalid_final, "validation")],
early_stopping_rounds=50, # Allow early stopping
verbose_eval=False,
)
The above code would’ve made XGBoost use 100k resolution timber however due to early stopping, it’s going to cease when the validation rating hasn’t been enhancing for the final 50 rounds. Normally, the variety of required timber shall be lower than 5000–10000.
Controlling num_boost_round can be one of many greatest components in how lengthy the coaching course of runs as extra timber require extra sources.
