42 Commits

Author SHA1 Message Date
Anonymous-submission-repo
2d18c49cdd update 2022-10-12 04:31:51 +00:00
Anonymous-submission-repo
9bc32acafb first 2022-10-09 11:39:29 -04:00
Chi Wang
860cbc233e
move searcher and scheduler into tune (#746)
* move into tune

* correct path

* correct path

* import path
2022-10-04 16:03:22 -07:00
Chi Wang
b7a010e657
Move import location for Ray 2 (#721)
* ray version check when importing

* display learner_class when starting_points removed

* test ray 2
2022-09-13 19:13:06 -07:00
Chi Wang
d60d38b3e9
log_file_name in tune.run() (#681)
* log_file_name in tune.run()

* use_ray validates log_file_name

* assert no ray_args when not use_ray

* import os and use os.path
2022-08-15 06:15:31 -07:00
Chi Wang
9bf13d66f1
use relative url in doc (#620)
* use relative url in doc

* update link
2022-07-01 13:28:16 -07:00
Chi Wang
1b40b4b3a6
set_search_properties (#595)
* update the signature of set_search_properties
2022-06-16 16:30:50 -07:00
Chi Wang
1111d6d43a
backup & recover global vars for nested tune.run (#584)
* backup & recover global vars for nested tune.run

* ensure recovering global vars before return
2022-06-14 11:03:54 -07:00
Chi Wang
1af682b7f5
update doc about scheduler exception (#564)
* update doc about scheduler exception

* remove assert
2022-05-31 17:21:57 -07:00
Qingyun Wu
dcc640c1a3
Documentation, test and bugfix (#556)
* add bs restore test

* use default metric when not provided

* update documentation

* remove print

* period

* remove bs restore test

* Update website/docs/Use-Cases/Task-Oriented-AutoML.md
2022-05-26 07:12:30 -07:00
Qingyun Wu
bcdfdc8735
handle non-flaml scheduler in flaml.tune (#532)
* handle non-flaml scheduler in flaml.tune

* revise time budget

* Update website/docs/Use-Cases/Tune-User-Defined-Function.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Use-Cases/Tune-User-Defined-Function.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/tune/tune.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* add docstr

* remove random seed

* StopIteration

* StopIteration format

* format

* Update flaml/tune/tune.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* revise docstr

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-05-06 14:09:35 -04:00
Chi Wang
a1c49ca27b
allow evaluated_rewards shorter than points_to_evaluate (#522)
* allow evaluated_rewards shorter than points_to_evaluate

* docstr update
2022-04-23 16:22:34 -07:00
Chi Wang
9128c8811a
handle failing trials (#505)
* handle failing trials

* clarify when to return {}

* skip ensemble in accuracy check
2022-03-28 16:57:52 -07:00
Qingyun Wu
6c16e47e42
Bug fix and add documentation for metric_constraints (#498)
* metric constraint documentation

* update link

* update notebook

* fix a bug in adding 'time_total_s' to result

* use the default multiple factor from config file

* update notebook

* format

* improve test

* revise test budget for macos

* bug fix in adding time_total_s

* increase performance check budget

* revise test

* update notebook

* uncomment test

* remove redundancy

* clear output

* remove n_jobs

* remove constraint in notebook

* increase budget

* revise test

* add python version

* use getattr

* improve code robustness

Co-authored-by: Qingyun Wu <qxw5138@psu.edu>
2022-03-26 21:11:45 -04:00
Chi Wang
7eb7b46ea9
version number and doc (#497)
* version number

* add missing tasks in documentation

* update node-forge version
2022-03-25 17:32:37 -07:00
Qingyun Wu
05f9065ade
Docstr update (#460)
* parallel tuning docstr

* update n_concurrent_trials docstr

* n_jobs default

* parallel tuning in tune docstr
2022-02-15 09:41:53 -08:00
Chi Wang
9e88f22167
fix a bug when using ray & update ray on aml (#455)
* fix a bug when using ray & update ray on aml
When using with_parameters(), the config argument must be the first argument in the trainable function.
* make training function runnable standalone
2022-02-11 20:14:10 -08:00
Chi Wang
569908fbe6
fix issues in logging, bug in space.py, constraint sign, and improve code coverage (#388)
* console log handler

* version update

* doc

* skippable steps

* notebook update

* constraint sign

* doc for constraints

* bug fix: define-by-run and unflatten_hierarchical

* const

* handle nested space in indexof()

* test grid search

* test suggestion

* model test

* >1 ckpts

* always increase iter count

* log total # iterations

* security patch

* make iter_per_learner consistent
2022-01-14 13:39:09 -08:00
Chi Wang
efd85b4c86
Deploy a new doc website (#338)
A new documentation website. And:

* add actions for doc

* update docstr

* installation instructions for doc dev

* unify README and Getting Started

* rename notebook

* doc about best_model_for_estimator #340

* docstr for keep_search_state #340

* DNN

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
Co-authored-by: Z.sk <shaokunzhang@psu.edu>
2021-12-16 17:11:33 -08:00
Z.sk
7b24662dca
Makes the evaluation_function could receive the incumbent best result as input in Tune (#339)
* update tune function

* pass incumbent result to the training function

* Update test/tune/test_record_incumbent.py

* Update flaml/searcher/search_thread.py

* Update flaml/searcher/blendsearch.py

* Update flaml/tune/tune.py

* add constant variable

Co-authored-by: 张少坤 <zhangshaokun@fuzhi.ai>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-12-15 21:12:47 -08:00
Chi Wang
228b00cbee
use cfo in tune.run if bs is not installed (#334)
Use cfo in tune.run if bs is not installed and issue a warning.
2021-12-12 21:10:06 -08:00
Qingyun Wu
dd60dbc5eb
rename training_function (#327)
* rename training_function

* add docstr

* update docstr

* update docstr and comments

Co-authored-by: Qingyun Wu <qxw5138@psu.edu>
2021-12-06 17:03:43 -05:00
Qingyun Wu
17b17d084f
tune api for schedulers (#322)
* revise api and tests

* rename prune_attr

* update finetune notebook

* add scheduler test and notebook

* update tune api for scheduler

* remove scheduler notebook

* Update flaml/tune/tune.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* docstr

* fix imports

* clear notebook output

* fix ray import

* Update flaml/tune/tune.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* improve docstr

* Update flaml/searcher/blendsearch.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* remove redundant import

Co-authored-by: Qingyun Wu <qxw5138@psu.edu>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-12-04 21:52:20 -05:00
Chi Wang
0d9439212f update docstr 2021-11-06 09:37:33 -07:00
Qingyun Wu
94a81a95ad
Add documentation for warm-start (#255)
* add documentation for warm-start

* fix typo

* fix typo

* Update flaml/tune/tune.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update automl.py

Co-authored-by: Qingyun Wu <qxw5138@psu.edu>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-10-19 16:39:28 -04:00
Chi Wang
f48ca2618f
warning -> info for low cost partial config (#231)
* warning -> info for low cost partial config
#195, #110

* when n_estimators < 0, use trained_estimator's

* log debug info

* test random seed

* remove "objective"; avoid ZeroDivisionError

* hp config to estimator params

* check type of searcher

* default n_jobs

* try import

* Update searchalgo_auto.py

* CLASSIFICATION

* auto_augment flag

* min_sample_size

* make catboost optional
2021-10-08 16:09:43 -07:00
Chi Wang
a9d39b71da
consider num_samples in bs thread priority (#207)
* consider num_samples in bs thread priority

* continue search for bs
2021-09-14 18:36:10 -07:00
Chi Wang
71219df6c6
notebook example (#189)
* config in result

* value can be float

* pytorch notebook example

* docker, pre-commit

* max_failure (#192); early_stop

* extend starting_points (#196)

Co-authored-by: Chi Wang (MSR) <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qw2ky@virginia.edu>
2021-09-10 16:39:16 -07:00
Chi Wang
e46573a01d
warmstart blendsearch (#186)
* increase test coverage

* use define by run only when needed

* warmstart bs

* classification -> binary, multi

* warm start with evaluated rewards

* data transformer; resource attr for gs

* BlendSearchTuner bug fix and unittest

* bug fix

* docstr and import

* task type
2021-09-04 01:42:21 -07:00
Qingyun Wu
10082b9262
v0.5.12 (#150)
* remove extra comma

* exclusive bound

* log file name

* add cost to space

* dataset_format

* add load_openml_dataset test

* docstr

* revise test format

* simplify restore

* order categories

* openml server exception in test

* process space

* add warning

* log format

* reduce n_cpu

* nested space

* hierarchical search space for CFO

* non hierarchical for bs

* unflatten hierarchical config

* connection error

* random sample

* config signature

* check ray version

* preprocess numpy array

* catboost preprocess

* time budget

* seed, verbose, hpo_method

* test cfocat

* shallow copy in flatten_dict
prevent lgbm model duplication

* match estimator name

* quantize and log

* test qloguniform and qrandint

* test qlograndint

* thread.running

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyunwu@Qingyuns-MacBook-Pro-2.local>
2021-08-11 23:02:22 -07:00
Eduardo Büll
46752083a2
fix UnboundLocalError in tune.run (#142) (#145)
Fix UnboundLocalError exception in tune.run when training_function returns a value.

Resolves #142
2021-08-01 17:55:38 -07:00
Qingyun Wu
e24265ee5d
automl fit with starting points (#141)
* add starting point in fit

* add estimator best config

* add test

* add doc string

* when there are multiple points_to_evaluate in CFO, use the best one to start local search; after that use low cost partial config as the start point; then, remove the points whose performance is worse than the converged, and start local search from the remaining ones ordered by their performance.

Co-authored-by: Qingyun Wu <qingyunwu@Qingyuns-MacBook-Pro-2.local>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-07-31 13:39:31 -07:00
Chi Wang
b3bb00966d
coverage (#135)
* coverage

* readme

* timeout
2021-07-20 17:00:44 -07:00
Chi Wang
2dbf38da0a
discount running thread (#121)
* discount running thread

* version

* limit dir

* report result

* catch

* remove handler
2021-06-25 14:24:46 -07:00
Xueqing Liu
d40993d920
apidoc (#116)
* fixing apidoc errors

Co-authored-by: Chi Wang (MSR) <wang.chi@microsoft.com>
Co-authored-by: liususan091219 <Xqq630517>
2021-06-19 19:09:49 -07:00
Qingyun Wu
e031c2eb7d
Test restore (#103)
* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* prevent divide by zero

* test roberta

* BlendSearchTuner

* sync

* version number

* update gitignore

* delta time

* reindex columns when dropping int-indexed columns

* add seed

* add seed in Args

* merge

* stabilize SearchThread speed

* add seed

* fix import

* use except

* add restore test for CFO

* remove test_restore

* remove inspect

* remove print

* change to SearchThread._esp

* add _eps lower bound

* _eps in SearchThread

* add test_restore

* 1<<32

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
2021-06-07 19:49:45 -04:00
Chi Wang
0925e2b308
constraints (#88)
* pre-training constraints

* metric constraints after training
2021-05-18 15:57:42 -07:00
Chi Wang
97a7c114ee
Issue58 (#59)
* iter per learner

* code cleanup
2021-04-08 09:29:55 -07:00
Chi Wang
b7a91e0385
V0.3.0 (#55)
* flaml v0.3

* low cost partial config
2021-04-06 11:37:52 -07:00
Chi Wang
ae5f8e5426
data validation (#45)
* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* prevent divide by zero

* test roberta

* BlendSearchTuner

* delta time

* reindex columns when dropping int-indexed columns

* test drop columns and small training data

* param set for ensemble builder

* fillna on copy

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
2021-03-19 09:50:47 -07:00
Chi Wang
7bd231e497
v0.2.6 (#32)
* xgboost notebook

* finetuning notebook

* finetuning test

* experimental nni support

* support nested search space

* log file name

* record training_iteration

* eps

* reset times

* std set to default step size if 0
2021-02-28 12:43:43 -08:00
Chi Wang
776aa55189
V0.2.2 (#19)
* v0.2.2

separate the HPO part into the module flaml.tune
enhanced implementation of FLOW^2, CFO and BlendSearch
support parallel tuning using ray tune
add support for sample_weight and generic fit arguments
enable mlflow logging

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: qingyun-wu <qw2ky@virginia.edu>
2021-02-05 21:41:14 -08:00