288 Commits

Author SHA1 Message Date
Chi Wang
70d86942f4
skip test in py 3.6 (#832) 2022-11-29 13:10:35 -08:00
Chi Wang
595af7a04f
install editable package in codespace (#826)
* install editable package in codespace

* fix test error in test_forecast

* fix test error in test_space

* openml version

* break tests; pre-commit

* skip on py10+win32

* install mlflow in test

* install mlflow in [test]

* skip test in windows

* import

* handle PermissionError

* skip test in windows

* skip test in windows

* skip test in windows

* skip test in windows

* remove ts_forecast_panel from doc
2022-11-27 14:22:54 -05:00
Anonymous-submission-repo
5eb9927642
Add performance test for LexiFlow (#812)
* add test

* fix

* change test name
2022-11-15 10:44:53 -05:00
Chi Wang
30e200985c
Fix issues related to zero-shot automl (#783)
* skip in-search-space check for small max iter

* resolve Pickle Transformer #730

* resolve default config unrecognized #784

* Change definition of init_config

* copy points_to_evaluate

* make test pass

* check learner selector
2022-11-13 12:47:59 -08:00
Anonymous-submission-repo
2daaa4c637 clean up 2022-10-15 03:53:08 +00:00
Anonymous-submission-repo
6df7782c5e
Update test/tune/test_lexiflow.py
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-10-14 22:52:07 -04:00
Anonymous-submission-repo
a1d9e333fe update 2022-10-14 23:48:05 +00:00
Anonymous-submission-repo
c3baf2d4ee delete automl 2022-10-14 23:30:24 +00:00
Anonymous-submission-repo
585bde1ce6 Merge branch 'LexiFlow' of https://github.com/Anonymous-submission-repo/FLAML into LexiFlow 2022-10-14 20:43:50 +00:00
Anonymous-submission-repo
bf81912f09 update 2022-10-14 20:40:49 +00:00
Chi Wang
cafb67123a
Merge branch 'main' into LexiFlow 2022-10-14 11:04:18 -07:00
Susan Xueqing Liu
2ebddd67ae
Remove NLP classification head (#756)
* rm classification head in nlp

* rm classification head in nlp

* rm classification head in nlp

* adding test cases for switch classification head

* adding test cases for switch classification head

* Update test/nlp/test_autohf_classificationhead.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* adding test cases for switch classification head

* run each test separately

* skip classification head test on windows

* disabling wandb reporting

* fix test nlp custom metric

* fix test nlp custom metric

* fix test nlp custom metric

* fix test nlp custom metric

* fix test nlp custom metric

* fix test nlp custom metric

* fix test nlp custom metric

* fix test nlp custom metric

* fix test nlp custom metric

* fix test nlp custom metric

* fix test nlp custom metric

* Update website/docs/Examples/AutoML-NLP.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/AutoML-NLP.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* fix test nlp custom metric

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-10-12 17:04:42 -07:00
Anonymous-submission-repo
2d18c49cdd update 2022-10-12 04:31:51 +00:00
Anonymous-submission-repo
4e37826417 update 2022-10-10 01:24:22 +00:00
Anonymous-submission-repo
f7a9d42dc7 update 2022-10-10 01:15:17 +00:00
Anonymous-submission-repo
9bc32acafb first 2022-10-09 11:39:29 -04:00
Chi Wang
860cbc233e
move searcher and scheduler into tune (#746)
* move into tune

* correct path

* correct path

* import path
2022-10-04 16:03:22 -07:00
Xueqing Liu
ceb3e300cd
Issue724 (#745)
* fixing issue724

* fixing issue724
2022-10-04 10:51:12 -04:00
Chi Wang
b7a010e657
Move import location for Ray 2 (#721)
* ray version check when importing

* display learner_class when starting_points removed

* test ray 2
2022-09-13 19:13:06 -07:00
Xueqing Liu
2314cc5a7e
"intermediate_results" TypeError: argument of type 'NoneType' is not iterable (#695)
* fix mlflow bug

* bump version
2022-08-22 13:36:50 -04:00
Chi Wang
dffa802b3e
use_best_model for catboost (#679)
* use_best_model for catboost

* bump version to 1.0.11
2022-08-20 18:38:56 -07:00
Xueqing Liu
3d1a28bfc0
Add preserve_checkpoint to preserve the checkpoint after del (#692)
* fix del bug
2022-08-20 18:17:10 -04:00
Qingyun Wu
8b3c6e4d7b
VW version requirement and documentation on config_constraints vs metric_constraints (#686)
* add vw version requirement

* vw version

* version range

* add documentation

* vw version range

* skip test on py3.10

* vw version

* rephrase

* don't install vw on py 3.10

* move import location

* remove inherit

* 3.10 in version

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-08-15 20:16:11 -07:00
Chi Wang
d60d38b3e9
log_file_name in tune.run() (#681)
* log_file_name in tune.run()

* use_ray validates log_file_name

* assert no ray_args when not use_ray

* import os and use os.path
2022-08-15 06:15:31 -07:00
Chi Wang
5e1059ab82
check config constraints for the initial config (#685)
* check config constraints for the initial config

* default config value
2022-08-15 05:30:23 -07:00
jmrichardson
e43485607a
Disable shuffle for custom CV (#659)
* Disable shuffle for custom CV

* Add custom fold shuffle test

* Update test_split.py

* Update test_split.py
2022-08-12 17:05:32 -07:00
Chi Wang
ca9f9054e7
categorical choice can be ordered or unordered (#677)
* categorical choice can be ordered or unordered

* ordered -> order

* move choice into utils

* version comparison

* packaging -> setuptools

* import version

* version_parse

* test order for choice
2022-08-12 13:55:17 -07:00
Kevin Chen
f718d18b5e
time series forecasting with panel datasets (#541)
* time series forecasting with panel datasets
- integrate Temporal Fusion Transformer as a learner based on pytorchforecasting

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update setup.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update setup.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update setup.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py and test_forecast.py
- remove blank lines

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py to prevent errors

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl.py and data.py
- change forecast task name
- update documentation for fit() method

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py
- add performance test
- use 'fit_kwargs_by_estimator'

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* add time index function

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py performance test

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update data.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update data.py to prevent type error

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update setup.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update for pytorch forecasting tft on panel datasets

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl.py documentations

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* - rename estimator
- add 'gpu_per_trial' for tft estimator

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* include ts panel forecasting as an example

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update documentations

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl_time_series_forecast.ipynb

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update documentations

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* "weights_summary" argument deprecated and removed for pl.Trainer()

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py tft estimator prediction method

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update `fit_kwargs` documentation

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-08-12 08:39:22 -07:00
jmrichardson
25ad397d55
Skip transform (#665)
* Skip transform

* Fix logic and docstring, add test

* Add period ending to skip_transform doc

* Add skip_transform to retrain_from_log method

* Update test/automl/test_classification.py

Co-authored-by: Xueqing Liu <liususan091219@users.noreply.github.com>

Co-authored-by: Xueqing Liu <liususan091219@users.noreply.github.com>
2022-08-11 19:41:23 -04:00
Rui Zhuang
b6e8b9ccca
Add pipeline tuner component and dependencies. (#671)
* add pipeline tuner component and dependencies.

* clean code.

* do not need force rerun.

* replace the resources.

* update metrics retrieving.

* Update test/pipeline_tuning_example/requirements.txt

* Update test/pipeline_tuning_example/train/env.yaml

* Update test/pipeline_tuning_example/tuner/env.yaml

* Update test/pipeline_tuning_example/tuner/tuner_func.py

* Update test/pipeline_tuning_example/data_prep/env.yaml

* fix issues found by lint with flake8.

* add documentation

* add data.

* do not need AML resource for local run.

* AML -> AzureML

* clean code.

* Update website/docs/Examples/Tune-AzureML pipeline.md

* rename and add pip install.

* update figure name.

* align docs with code.

* remove extra line.
2022-08-10 20:20:21 -07:00
Chi Wang
816a82a115
make test result more stable (#646) 2022-08-05 10:17:41 -07:00
Xueqing Liu
21fa6c10ec
Fixing the issue that FLAML trial number is significantly smaller than Transformers.hyperparameter_search (#657)
* fix 636

* adding low cost config

* update padding; update tokenization output y type (series -> DF); update low cost init config

* updating todf; updating metric_loss_score
2022-08-03 00:11:29 -04:00
Xueqing Liu
5eb5d43d7f
Fix HPO evaluation bug (#645)
* fix eval automl metric bug on val_loss inconsistency

* updating starting point search space to continuous

* shortening notebok
2022-07-28 23:08:42 -04:00
Xueqing Liu
731afec9eb
This PR fixes the frequent NLP bugs in the other PRs (#647)
* fix nlp bug

* resetting model to electra small

* removing model_path from fit_kwargs_by_estimator
2022-07-25 17:46:33 -04:00
Chi Wang
e14e909af9
Feature names and importances (#621)
* feature names and importances

* None check

* StackingClassifier has no feature_importances_

* StackingClassifier has no feature_names_in_
2022-07-10 12:25:59 -07:00
Qingyun Wu
b7846048dc
Allow FLAML_sample_size in starting_points (#619)
* FLAML_sample_size

* clean up

* starting_points as a list

* catch AssertionError

* per estimator sample size

* import

* per estimator min_sample_size

* Update flaml/automl.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update test/automl/test_warmstart.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* add warnings

* adding more tests

* fix a bug in validating starting points

* improve test

* revise test

* revise test

* documentation about custom_hp

* doc and efficiency

* update test

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-07-09 16:04:46 -04:00
Xueqing Liu
6108493e0b
fix ner bug; refactor post processing of TransformersEstimator prediction (#615)
* fix ner bug; refactor post processing

* fix too many values to unpack

* supporting id/token label for NER
2022-07-05 13:38:21 -04:00
Chi Wang
74cca60606
Allow custom GroupKFold object as split_type (#616)
* Allow custom GroupKFold object

* handle unpickle error for prophet 1.1

* eval_method in test_object()
2022-06-29 21:04:25 -07:00
Chi Wang
cbb85e2aab
Py36 (#614)
* allow installation in py 3.6

* test py 3.6
2022-06-26 08:32:28 -07:00
Chi Wang
7d6822aa40 cath URLError 2022-06-24 08:07:26 -07:00
Chi Wang
4377d53a73
update got version (#607)
* update got version

* None check
2022-06-23 08:02:46 -07:00
Chi Wang
c45741a67b
support latest xgboost version (#599)
* support latest xgboost version

* Update test_classification.py

* Update 

Exists problems when installing xgb1.6.1 in py3.6

* cleanup

* xgboost version

* remove time_budget_s in test

* remove redundancy

* stop support of python 3.6

Co-authored-by: zsk <shaokunzhang529@gmail.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2022-06-21 18:59:07 -07:00
Chi Wang
1b40b4b3a6
set_search_properties (#595)
* update the signature of set_search_properties
2022-06-16 16:30:50 -07:00
Chi Wang
1111d6d43a
backup & recover global vars for nested tune.run (#584)
* backup & recover global vars for nested tune.run

* ensure recovering global vars before return
2022-06-14 11:03:54 -07:00
Qingyun Wu
65fa72d583
Class variables, cost_attr, and reproducibility (#587)
* test reproducibility

* cost_attr, _eps and penalty as instance variables
2022-06-13 21:52:42 -07:00
Chi Wang
f8cc38bc16
enable ensemble when using ray (#583)
* enable ensemble when using ray

* sanitize config
2022-06-10 21:28:47 -07:00
Chi Wang
0642b6e7bb
init value type match (#575)
* init value type match

* bump version to 1.0.6

* add a note about flaml version in notebook

* add note about mismatched ITER_HP

* catch SSLError when accessing OpenML data

* catch errors in autovw test

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2022-06-09 08:11:15 -07:00
Xueqing Liu
e0e317bfb1
fixing trainable and update function, completing NOTE (#566)
* fix checkpoint naming + trial id for non-ray mode, fix the bug in running test mode, delete all the checkpoints in non-ray mode

* finished testing for checkpoint naming, delete checkpoint, ray, max iter = 1
2022-06-03 15:19:22 -04:00
Chi Wang
1af682b7f5
update doc about scheduler exception (#564)
* update doc about scheduler exception

* remove assert
2022-05-31 17:21:57 -07:00
Chi Wang
49e8f7f028
use zeroshot when no budget is given; custom_hp (#563)
* use zeroshot when no budget is given; custom_hp

* update Getting-Started

* protobuf version

* X_val
2022-05-28 17:22:09 -07:00