mirror of
https://github.com/microsoft/autogen.git
synced 2025-07-16 13:30:55 +00:00
2ff1035733
* precommit: end-of-file-fixer * exclude .gitignore * apply --------- Co-authored-by: Shaokun <shaokunzhang529@gmail.com>
97 lines
8.0 KiB
Markdown
97 lines
8.0 KiB
Markdown
# AutoML - Rank
|
|
|
|
### A simple learning-to-rank example
|
|
|
|
```python
|
|
from sklearn.datasets import fetch_openml
|
|
from flaml import AutoML
|
|
|
|
X_train, y_train = fetch_openml(name="credit-g", return_X_y=True, as_frame=False)
|
|
y_train = y_train.cat.codes
|
|
# not a real learning to rank dataaset
|
|
groups = [200] * 4 + [100] * 2 # group counts
|
|
automl = AutoML()
|
|
automl.fit(
|
|
X_train, y_train, groups=groups,
|
|
task='rank', time_budget=10, # in seconds
|
|
)
|
|
```
|
|
|
|
#### Sample output
|
|
|
|
```
|
|
[flaml.automl: 11-15 07:14:30] {1485} INFO - Data split method: group
|
|
[flaml.automl: 11-15 07:14:30] {1489} INFO - Evaluation method: holdout
|
|
[flaml.automl: 11-15 07:14:30] {1540} INFO - Minimizing error metric: 1-ndcg
|
|
[flaml.automl: 11-15 07:14:30] {1577} INFO - List of ML learners in AutoML Run: ['lgbm', 'xgboost']
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 0, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:30] {1944} INFO - Estimated sufficient time budget=679s. Estimated necessary time budget=1s.
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.1s, estimator lgbm's best error=0.0248, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 1, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.1s, estimator lgbm's best error=0.0248, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 2, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.2s, estimator lgbm's best error=0.0248, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 3, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.2s, estimator lgbm's best error=0.0248, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 4, current learner xgboost
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.2s, estimator xgboost's best error=0.0315, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 5, current learner xgboost
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.2s, estimator xgboost's best error=0.0315, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 6, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.3s, estimator lgbm's best error=0.0248, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 7, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.3s, estimator lgbm's best error=0.0248, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 8, current learner xgboost
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.4s, estimator xgboost's best error=0.0315, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 9, current learner xgboost
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.4s, estimator xgboost's best error=0.0315, best estimator lgbm's best error=0.0248
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 10, current learner xgboost
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.4s, estimator xgboost's best error=0.0233, best estimator xgboost's best error=0.0233
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 11, current learner xgboost
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.4s, estimator xgboost's best error=0.0233, best estimator xgboost's best error=0.0233
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 12, current learner xgboost
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.4s, estimator xgboost's best error=0.0233, best estimator xgboost's best error=0.0233
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 13, current learner xgboost
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.4s, estimator xgboost's best error=0.0233, best estimator xgboost's best error=0.0233
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 14, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.5s, estimator lgbm's best error=0.0225, best estimator lgbm's best error=0.0225
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 15, current learner xgboost
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.5s, estimator xgboost's best error=0.0233, best estimator lgbm's best error=0.0225
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 16, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:30] {2029} INFO - at 0.5s, estimator lgbm's best error=0.0225, best estimator lgbm's best error=0.0225
|
|
[flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 17, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.5s, estimator lgbm's best error=0.0225, best estimator lgbm's best error=0.0225
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 18, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.6s, estimator lgbm's best error=0.0225, best estimator lgbm's best error=0.0225
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 19, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.6s, estimator lgbm's best error=0.0201, best estimator lgbm's best error=0.0201
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 20, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.6s, estimator lgbm's best error=0.0201, best estimator lgbm's best error=0.0201
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 21, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.7s, estimator lgbm's best error=0.0201, best estimator lgbm's best error=0.0201
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 22, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.7s, estimator lgbm's best error=0.0201, best estimator lgbm's best error=0.0201
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 23, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.8s, estimator lgbm's best error=0.0201, best estimator lgbm's best error=0.0201
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 24, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.8s, estimator lgbm's best error=0.0201, best estimator lgbm's best error=0.0201
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 25, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.8s, estimator lgbm's best error=0.0201, best estimator lgbm's best error=0.0201
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 26, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.9s, estimator lgbm's best error=0.0197, best estimator lgbm's best error=0.0197
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 27, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 0.9s, estimator lgbm's best error=0.0197, best estimator lgbm's best error=0.0197
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 28, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 1.0s, estimator lgbm's best error=0.0197, best estimator lgbm's best error=0.0197
|
|
[flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 29, current learner lgbm
|
|
[flaml.automl: 11-15 07:14:31] {2029} INFO - at 1.0s, estimator lgbm's best error=0.0197, best estimator lgbm's best error=0.0197
|
|
[flaml.automl: 11-15 07:14:31] {2242} INFO - retrain lgbm for 0.0s
|
|
[flaml.automl: 11-15 07:14:31] {2247} INFO - retrained model: LGBMRanker(colsample_bytree=0.9852774042640857,
|
|
learning_rate=0.034918421933217675, max_bin=1023,
|
|
min_child_samples=22, n_estimators=6, num_leaves=23,
|
|
reg_alpha=0.0009765625, reg_lambda=21.505295697527654, verbose=-1)
|
|
[flaml.automl: 11-15 07:14:31] {1608} INFO - fit succeeded
|
|
[flaml.automl: 11-15 07:14:31] {1610} INFO - Time taken to find the best model: 0.8846545219421387
|
|
[flaml.automl: 11-15 07:14:31] {1624} WARNING - Time taken to find the best model is 88% of the provided time budget and not all estimators' hyperparameter search converged. Consider increasing the time budget.
|
|
```
|