614 Commits

Author SHA1 Message Date
Chi Wang
f4529dfe89
package name in setup (#198)
* package name

* learning to rank example: close #200

* try import prophet #201
2021-09-11 21:19:18 -07:00
Chi Wang
8f9f08cebc
try import catboost (#197) 2021-09-10 20:09:08 -07:00
Chi Wang
71219df6c6
notebook example (#189)
* config in result

* value can be float

* pytorch notebook example

* docker, pre-commit

* max_failure (#192); early_stop

* extend starting_points (#196)

Co-authored-by: Chi Wang (MSR) <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qw2ky@virginia.edu>
2021-09-10 16:39:16 -07:00
Chi Wang
339eb80f44
variable name (#187) 2021-09-04 20:28:37 -07:00
Chi Wang
e46573a01d
warmstart blendsearch (#186)
* increase test coverage

* use define by run only when needed

* warmstart bs

* classification -> binary, multi

* warm start with evaluated rewards

* data transformer; resource attr for gs

* BlendSearchTuner bug fix and unittest

* bug fix

* docstr and import

* task type
2021-09-04 01:42:21 -07:00
Qingyun Wu
5fdfa2559b
Cleanml (#185)
* reorg ml

* return y_pred in eval_estimator

* add train loss into metric_for_logging dict
2021-09-02 13:07:30 -07:00
Gian Pio Domiziani
63bba92fd0
Fix decide_split_type bug. (#184)
* Fix decide_split_type bug.
2021-09-02 08:50:22 -07:00
Kevin Chen
ec34427ca8
Forecast v2 (#182)
* update flaml_forecast.ipynb

* visualize predictions for comparison

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>
2021-09-01 18:06:37 -07:00
Chi Wang
6ab0730793
remove catboost training dir; ensemble api; blendsearch for hierarchical space; ranking task; forecast improvement (#178)
* remove catboost training dir

* close #48

* bs for hierarchical space. close #85

* retrain for hierarchical space

* clean ml (#180)

Co-authored-by: Qingyun Wu <qxw5138@psu.edu>

* support ranking task

* examples

* cv shuffle

* forecast api and implementation cleaner

* period constraints

* delete groups after fit
2021-09-01 16:25:04 -07:00
Chi Wang
1bc8786dcb
remove big objects after fit (#176)
* remove big objects after fit

* xgboost>1.3.3 has a weird auc socre on:
kr-vs-kp, fold 5, 1h1c

* keep_search_state
2021-08-26 13:45:13 -07:00
Qingyun Wu
a229a6112a
Support parallel and add random search (#167)
* non hashable value out of signature

* parallel trials

* add random in _search_parallel

* fix bug in retraining

* check memory constraint before training

* retrain_full

* log custom metric

* retraining budget check

* sample size check before retrain

* remove 'time2eval' from result

* report 'total_search_time' in result

* rename total_search_time to wall_clock_time

* rename train_loss boolean to log_training_metric

* set default train_loss to None

* exclude oom result

* log retrained model

* no subsample

* doc str

* notebook

* predicted value is NaN for sarimax

* version

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qxw5138@psu.edu>
2021-08-23 16:36:51 -07:00
Kevin Chen
3d0a3d26a2
Forecast (#162)
* added 'forecast' task with estimators ['fbprophet', 'arima', 'sarimax']

* update setup.py

* add TimeSeriesSplit to 'regression' and 'classification' task

* add 'time' split_type for 'classification' and 'regression' task

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* feature importance

* variable name

* Update test/test_split.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update test/test_forecast.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* prophet installation fail in windows

* upload flaml_forecast.ipynb

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>
2021-08-23 13:26:46 -07:00
すずまる
6270353458
support ROC and AUC for multi-class classification (#170)
* support ROC and AUC for multi-class classification

* add a test case to cover ROC and AUC for multi-class classification
2021-08-22 15:16:10 -07:00
Qingyun Wu
10082b9262
v0.5.12 (#150)
* remove extra comma

* exclusive bound

* log file name

* add cost to space

* dataset_format

* add load_openml_dataset test

* docstr

* revise test format

* simplify restore

* order categories

* openml server exception in test

* process space

* add warning

* log format

* reduce n_cpu

* nested space

* hierarchical search space for CFO

* non hierarchical for bs

* unflatten hierarchical config

* connection error

* random sample

* config signature

* check ray version

* preprocess numpy array

* catboost preprocess

* time budget

* seed, verbose, hpo_method

* test cfocat

* shallow copy in flatten_dict
prevent lgbm model duplication

* match estimator name

* quantize and log

* test qloguniform and qrandint

* test qlograndint

* thread.running

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyunwu@Qingyuns-MacBook-Pro-2.local>
2021-08-11 23:02:22 -07:00
Naga Budigam
2fb888e64e
Tutorial on using AutoML in sklearn pipeline (#157)
* tutorial on using AutoML in sklearn pipeline
2021-08-11 17:46:46 -07:00
Jeff
68733314f3
None type object cannot call close (#143)
* Fix AttributeError in file.close
2021-08-03 11:28:03 -07:00
Xueqing Liu
eeaf5b5963
space -> main (#148)
* subspace in flow2

* search space and trainable from AutoML

* experimental features: multivariate TPE, grouping, add_evaluated_points

* test experimental features

* readme

* define by run

* set time_budget_s for bs

Co-authored-by: liususan091219 <Xqq630517>

* version

* acl

* test define_by_run_func

* size

* constraints

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-08-02 16:10:26 -07:00
Eduardo Büll
46752083a2
fix UnboundLocalError in tune.run (#142) (#145)
Fix UnboundLocalError exception in tune.run when training_function returns a value.

Resolves #142
2021-08-01 17:55:38 -07:00
Qingyun Wu
e24265ee5d
automl fit with starting points (#141)
* add starting point in fit

* add estimator best config

* add test

* add doc string

* when there are multiple points_to_evaluate in CFO, use the best one to start local search; after that use low cost partial config as the start point; then, remove the points whose performance is worse than the converged, and start local search from the remaining ones ordered by their performance.

Co-authored-by: Qingyun Wu <qingyunwu@Qingyuns-MacBook-Pro-2.local>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-07-31 13:39:31 -07:00
Chi Wang
15fd8adac4
max_leaves (#138)
* max_leaf_nodes in rf and extra_tree

* preprocess numpy str

* free up mem after training
2021-07-27 18:02:49 -07:00
Qingyun Wu
58c0ec959d
Update readme for flaml.tune (#137)
* add time_budget_s for bs in readme

* version update

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-07-24 17:10:43 -07:00
Chi Wang
95aa719b01
version (#136) 2021-07-20 17:45:02 -07:00
Chi Wang
b3bb00966d
coverage (#135)
* coverage

* readme

* timeout
2021-07-20 17:00:44 -07:00
Chi Wang
072e9e4588
constraint (#132)
* constraint

* ensemble
2021-07-10 09:02:17 -07:00
Qingyun Wu
b04b00dc9d
V0.5.6 (#128)
* recover ConcurrencyLimiter

* cost attribute

* update notebooks

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
2021-07-06 08:32:20 -07:00
Qingyun Wu
a291abfab9
Cha cha (#127)
* unordered categorical

* allow cost attribute to be None

* tensorboardX version

* quote

* cfo cat

* trunc

* Update version.py

* incumbent is normalized

* python 3.9

* remove ConcurrencyLimiter

* seed

* estimator

* update autovw notebook

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
2021-07-05 18:17:26 -07:00
Xueqing Liu
e41b42842a
fixing "discount running thread " (#122)
* add tf to test dependency

Co-authored-by: liususan091219 <Xqq630517>
2021-06-25 22:26:47 -07:00
Chi Wang
2dbf38da0a
discount running thread (#121)
* discount running thread

* version

* limit dir

* report result

* catch

* remove handler
2021-06-25 14:24:46 -07:00
Xueqing Liu
da9fc51e62
update notebook 2021-06-22 22:20:16 -07:00
Chi Wang
83080d94bb
ML.NET (#119) 2021-06-22 21:57:36 -07:00
Xueqing Liu
54fbf09322
fixing jupyter notebook (#118) 2021-06-21 19:08:24 -07:00
Xueqing Liu
d40993d920
apidoc (#116)
* fixing apidoc errors

Co-authored-by: Chi Wang (MSR) <wang.chi@microsoft.com>
Co-authored-by: liususan091219 <Xqq630517>
2021-06-19 19:09:49 -07:00
Xueqing Liu
6133db84e8
remove learning_rate and weight_decay (#113)
* remove varying_arg1, varying_args
2021-06-19 09:27:51 -07:00
Chi Wang
e039861ab0
multiple logged metrics in cv (#114) 2021-06-18 21:19:59 -07:00
Chi Wang
3a2b6cdddc
Update version.py (#111) 2021-06-18 10:59:00 -07:00
Xueqing Liu
cd4be9c0e5
add notebook (#109)
* added support for transformers==3.4.0

* updating error message

* adding arxiv
2021-06-17 21:42:26 -07:00
Chi Wang
183b867856
groups (#107)
* groups

* version

* developer's guide
2021-06-15 18:52:57 -07:00
Xueqing Liu
a5a5a4bc20
fixed API doc and import (#108)
* removed run_analysis.py, run_autohf.py, test_jupyter.py
2021-06-15 09:55:23 -07:00
Xueqing Liu
926589bdda
exception, coverage for autohf (#106)
* increase coverage

* fixing exception messages

* fixing import
2021-06-14 14:11:40 -07:00
Chi Wang
c26720c299
api doc for chacha (#105)
* api doc for chacha

* update params

* link to paper

* update dataset id

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
2021-06-11 10:25:45 -07:00
Xueqing Liu
a4049ad9b6
autohf (#43)
automate huggingface transformer
2021-06-09 08:37:03 -07:00
Qingyun Wu
e031c2eb7d
Test restore (#103)
* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* prevent divide by zero

* test roberta

* BlendSearchTuner

* sync

* version number

* update gitignore

* delta time

* reindex columns when dropping int-indexed columns

* add seed

* add seed in Args

* merge

* stabilize SearchThread speed

* add seed

* fix import

* use except

* add restore test for CFO

* remove test_restore

* remove inspect

* remove print

* change to SearchThread._esp

* add _eps lower bound

* _eps in SearchThread

* add test_restore

* 1<<32

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
2021-06-07 19:49:45 -04:00
Chi Wang
682cb49654
move import position (#102)
* move import position
2021-06-05 11:36:26 -07:00
Qingyun Wu
c0be2eefe8
typo (#101) 2021-06-04 11:13:28 -07:00
Qingyun Wu
c8da829e4d
ChaCha documentation (#100)
* update readme

* naming
2021-06-04 10:52:52 -07:00
Chi Wang
f7cf2ea45a
Multiclass (#99)
* utility functions

* stepsize lower bound
2021-06-04 10:31:33 -07:00
Qingyun Wu
0d3a0bfab6
Add ChaCha (#92)
* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* prevent divide by zero

* test roberta

* BlendSearchTuner

* sync

* version number

* update gitignore

* delta time

* reindex columns when dropping int-indexed columns

* add seed

* add seed in Args

* merge

* init upload of ChaCha

* remove redundancy

* add back catboost

* improve AutoVW API

* set min_resource_lease in VWOnlineTrial

* docstr

* rename

* docstr

* add docstr

* improve API and documentation

* fix name

* docstr

* naming

* remove max_resource in scheduler

* add TODO in flow2

* remove redundancy in rearcher

* add input type

* adapt code from ray.tune

* move files

* naming

* documentation

* fix import error

* fix format issues

* remove cb in worse than test

* improve _generate_all_comb

* remove ray tune

* naming

* VowpalWabbitTrial

* import error

* import error

* merge test code

* scheduler import

* fix import

* remove

* import, minor bug and version

* Float or Categorical

* fix default

* add test_autovw.py

* add vowpalwabbit and openml

* lint

* reorg

* lint

* indent

* add autovw notebook

* update notebook

* update log msg and autovw notebook

* update autovw notebook

* update autovw notebook

* add available strings for model_select_policy

* string for metric

* Update vw format in flaml/onlineml/trial.py

Co-authored-by: olgavrou <olgavrou@gmail.com>

* make init_config optional

* add _setup_trial_runner and update notebook

* space

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
Co-authored-by: olgavrou <olgavrou@gmail.com>
2021-06-02 22:08:24 -04:00
Chi Wang
61d1263dfd
log best model (#96)
* log best model
2021-06-02 13:11:41 -07:00
Qingyun Wu
a2a37cb60f
add tutorial (#93) 2021-05-27 16:04:13 -04:00
Gian Pio Domiziani
c4c15f533f
datetime feature engineering added. (#89)
* datetime feature engineering added.

* check if datetime in columns moved after drop check. Check if the new columns do not already exist.

* check the drop condition before to add new_column. In transform, check directly if new columns are present in num_column.

* check if new_column is in X.columns.

* fixed lint issue. update version to 0.4.1.
2021-05-25 08:30:08 -07:00