127 Commits

Author SHA1 Message Date
Chi Wang
fa5ccea862
extract code from text; solve_problem; request_timeout in config; improve code (#999)
* extract code from text

* solve_problem; request_timeout in config

* improve

* move import statement

* improve code

* generate assertions

* constant

* configs for implement; voting

* doc

* execute code in docker

* success indicator of code executation in docker

* success indicator

* execute code

* strip n

* add cost in generate_code

* add docstr

* filename

* bytes

* check docker version

* print log

* python test

* remove api key address

* rename exit code

* success exit code

* datasets

* exit code

* recover openai tests

* cache and pattern match

* wait

* wait

* cache and test

* timeout test

* python image name and skip macos

* windows image

* docker images

* volume path and yaml

* win path -> posix

* extensions

* path

* path

* path

* path

* path

* path

* path

* path

* path

* path

* path

* skip windows

* path

* timeout in windows

* use_docker

* use_docker

* hot fix from #1000

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-04-23 11:50:29 +00:00
Chi Wang
da0d8c05e1
Blog post for LLM tuning (#986)
* outline

* revision

* eval function signature

* first draft

* link

* format

* example

* cleanup

* average

* move figure

* tldr

* bold

* bold

* tag
2023-04-22 04:41:16 +00:00
Chi Wang
d4070e24c1
make context optional; improve error handling and doc (#997)
* make context optional

* better error handling and doc

* skip instantiation if no context

* skip test
2023-04-16 21:18:32 +00:00
Chi Wang
c780d79004
Post release update (#985)
* news update

* doc update

* avoid KeyError

* bump version to 1.2.1

* handle empty responses

* typo

* eval function
2023-04-10 20:46:28 +00:00
Chi Wang
82f0a4309d
autogen subpackage (#968)
* math utils in autogen

* cleanup

* code utils

* remove check function from code response

* comment out test

* GPT-4

* increase request timeout

* name

* logging and error handling

* better doc

* doc

* codegen optimized

* GPT series

* text

* no demo example

* math

* import openai

* import openai

* azure model name

* azure model name

* openai version

* generate assertion if necessary

* condition to generate assertions

* init region key

* rename

* comments about budget

* prompt

---------

Co-authored-by: Susan Xueqing Liu <liususan091219@users.noreply.github.com>
2023-04-08 03:04:01 +00:00
Andrea Ruggerini
7f9402b8fd
Add Holt-Winters exponential smoothing (#962)
* tentatively implement holt-winters-no covariates

* fix forecast method, clean class

* checking external regressors too

* update test forecast

* remove duplicated test file, re-add sarimax, search space cleanup

* Update flaml/automl/model.py

removed links. Most important one probably was: https://robjhyndman.com/hyndsight/ets-regressors/

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* prevent short series

* add docs

---------

Co-authored-by: Andrea W <a.ruggerini@ammagamma.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2023-04-04 17:29:54 +00:00
Ayesh Vininda
5616ce0a54
Noctus contribute (#961)
* indentation space #884 changing indentation of python code blocks

* indentation space #884 changing indentation of python code blocks 2
2023-04-03 01:50:44 +00:00
Qingyun Wu
45641000c0
Adding a test function for OpenAI completion in flaml (#951)
* improve max_valid_n and doc

* Update README.md

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* add support for chatgpt

* notebook

* newline at end of file

* chatgpt notebook

* ChatGPT in Azure

* doc

* math

* warning, timeout, log file name

* handle import error

* doc update; default value

* paper

* doc

* docstr

* eval_func

* add a test func in completion

* update notebook

* update math notebook

* improve notebok

* lint and handle exception

* flake8

* exception in test

* add agg_method

* NameError

* refactor

* Update flaml/integrations/oai/completion.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/integrations/oai/completion.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* add example

* merge files from oai_eval_test

* Revert "merge files from oai_eval_test"
This reverts commit 1e6a550f913bb94df6e9680934ccb7175d00702e.

* merge

* save results to notebook_output

* update version and cache

* update doc

* save nb cell results to file

* fix typo in model name

* code improvements

* improve docstr

* docstr

* docstr on the Returns of test

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Susan Xueqing Liu <liususan091219@users.noreply.github.com>
2023-04-02 16:14:11 +00:00
Chi Wang
595f5a8025
gpt-4 support; openai workflow fix; model str; timeout; voting (#958)
* workflow; model str; timeout

* voting

* notebook

* pull request

* recover workflow

* voted answer

* aoai

* ignore None answer

* default config

* note

* gpt-4

* n=5

* cleanup

* config name

* introduction

* readme

* avoid None

* add output/ to gitignore

* openai version

* invalid var

* comment long running cells
2023-03-26 17:13:06 +00:00
Chi Wang
169012f3e7
ChatGPT support (#942)
* improve max_valid_n and doc

* Update README.md

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* add support for chatgpt

* notebook

* newline at end of file

* chatgpt notebook

* ChatGPT in Azure

* doc

* math

* warning, timeout, log file name

* handle import error

* doc update; default value

* paper

* doc

* docstr

* eval_func

* prompt and messages

* remove confusing words

* notebook name

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Susan Xueqing Liu <liususan091219@users.noreply.github.com>
2023-03-10 19:35:36 +00:00
Shaokun
3a606930d1
Update doc and notebook (#946)
* update

* update doc

---------

Co-authored-by: “skzhang1” <“shaokunzhang529@gmail.com”>
2023-03-10 13:54:48 +00:00
Chi Wang
1ec77b58b4
improve max_valid_n and doc (#933)
* improve max_valid_n and doc

* Update README.md

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* newline at end of file

* doc

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Susan Xueqing Liu <liususan091219@users.noreply.github.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-03-05 16:40:57 +00:00
Susan Xueqing Liu
97928609ba
adding openai documentation (#938)
* adding openai documentation

* adding openai documentation

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* addressing comments

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* adding Azure

* rerun

* addressing comments about Azure

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-03-05 01:15:49 +00:00
Jirka Borovec
2ff1035733
precommit: end-of-file-fixer (#929)
* precommit: end-of-file-fixer

* exclude .gitignore

* apply

---------

Co-authored-by: Shaokun <shaokunzhang529@gmail.com>
2023-02-28 16:27:14 +00:00
Animaholic
0eaac7d018
added a documentation about small time budget (#914)
* added documentation about small time budget

* small change for better clarity

* Update website/docs/Use-Cases/Task-Oriented-AutoML.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2023-02-18 23:56:58 +00:00
Li Jiang
138eb78dbc
Added extras for synapse (#916)
* Added extras for synapse

* Update Installation doc
2023-02-17 16:38:55 +00:00
Animaholic
bcc6a197fd
Add bibtex entries for research publications (#904)
* Update Research.md

* Update website/docs/Research.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Research.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Research.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Research.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Research.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Research.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Research.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-02-05 20:48:07 -05:00
Qingyun Wu
d0f7b30b61 update doc for research papers 2023-02-03 16:57:16 -08:00
skzhang1
c6f8e004e7 fix link 2023-01-30 06:01:20 -08:00
Andrea W
58ac6b61c4 Merge branch 'first_contribution' of https://github.com/coffepowered/FLAML into first_contribution 2023-01-30 11:21:43 +00:00
Andrea W
3e6247ae59 add link to notebook showing group cv 2023-01-30 11:17:27 +00:00
Shaokun
df5efa5c2d
Merge branch 'main' into first_contribution 2023-01-29 22:56:45 -05:00
Shaokun
175a2eaaf6
Merge branch 'main' into support_percentages 2023-01-28 10:00:57 -05:00
Andrea W
bdbba6e3bc modify doc 2023-01-26 14:31:54 +00:00
Shaokun
cdc4201188
update (#883) 2023-01-18 14:35:57 -05:00
skzhang1
b7c0c24269 support percentage tolerance for lexicographic 2023-01-07 11:41:24 -08:00
Li Jiang
da2cd7ca89
Add supporting using Spark as the backend of parallel training (#846)
* Added spark support for parallel training.

* Added tests and fixed a bug

* Added more tests and updated docs

* Updated setup.py and docs

* Added customize_learner and tests

* Update spark tests and setup.py

* Update docs and verbose

* Update logging, fix issue in cloud notebook

* Update github workflow for spark tests

* Update github workflow

* Remove hack of handling _choice_

* Allow for failures

* Fix tests, update docs

* Update setup.py

* Update Dockerfile for Spark

* Update tests, remove some warnings

* Add test for notebooks, update utils

* Add performance test for Spark

* Fix lru_cache maxsize

* Fix test failures on some platforms

* Fix coverage report failure

* resovle PR comments

* resovle PR comments 2nd round

* resovle PR comments 3rd round

* fix lint and rename test class

* resovle PR comments 4th round

* refactor customize_learner to broadcast_code
2022-12-23 08:18:49 -08:00
Shaokun
4140fc9022
Format errors on the web. (#855)
* fix_doc

* update

* fix lint

* fix lint

* reformat

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2022-12-22 22:36:34 -05:00
Jing Dong
5778227a71
Fix checkpoint path issue
checkpoint path may be named dir_or_data instead of value
2022-12-16 14:41:33 +08:00
Chi Wang
2338431990
FAQ about OOM (#849)
* FAQ about OOM

* reduce -> resolve
2022-12-13 18:09:50 -05:00
Shaokun
10303854e2
Update website/docs/Examples/Tune-Lexicographic-objectives.md
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-12-07 13:04:55 -05:00
Shaokun
28c7626e99
Update Tune-Lexicographic-objectives.md 2022-12-07 10:57:33 -05:00
skzhang1
5adce62857 update 2022-12-06 20:42:06 -05:00
Mark Harley
44ddf9e104
Refactor into automl subpackage (#809)
* Refactor into automl subpackage

Moved some of the packages into an automl subpackage to tidy before the
task-based refactor. This is in response to discussions with the group
and a comment on the first task-based PR.

Only changes here are moving subpackages and modules into the new
automl, fixing imports to work with this structure and fixing some
dependencies in setup.py.

* Fix doc building post automl subpackage refactor

* Fix broken links in website post automl subpackage refactor

* Fix broken links in website post automl subpackage refactor

* Remove vw from test deps as this is breaking the build

* Move default back to the top-level

I'd moved this to automl as that's where it's used internally, but had
missed that this is actually part of the public interface so makes sense
to live where it was.

* Re-add top level modules with deprecation warnings

flaml.data, flaml.ml and flaml.model are re-added to the top level,
being re-exported from flaml.automl for backwards compatability. Adding
a deprecation warning so that we can have a planned removal later.

* Fix model.py line-endings

* Pin pytorch-lightning to less than 1.8.0

We're seeing strange lightning related bugs from pytorch-forecasting
since the release of lightning 1.8.0. Going to try constraining this to
see if we have a fix.

* Fix the lightning version pin

Was optimistic with setting it in the 1.7.x range, but that isn't
compatible with python 3.6

* Remove lightning version pin

* Revert dependency version changes

* Minor change to retrigger the build

* Fix line endings in ml.py and model.py

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
Co-authored-by: EgorKraevTransferwise <egor.kraev@transferwise.com>
2022-12-06 15:46:08 -05:00
Chi Wang
92b79221b6
make performance test reproducible (#837)
* make performance test reproducible

* fix test error

* Doc update and disable logging

* document random_state and version

* remove hardcoded budget

* fix test error and dependency; close #777

* iloc
2022-12-06 10:13:39 -08:00
Shreyas
3b3b0bfa8e
roc_auc_weighted metric addition (#827)
* Pending changes exported from your codespace

* Update flaml/automl.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/automl.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/ml.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/ml.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - Scikit-learn Pipeline.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* added documentation for new metric

* Update flaml/ml.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* minor notebook changes

* Update Integrate - Scikit-learn Pipeline.md

* Update notebook/automl_classification.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update integrate_azureml.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-12-02 19:27:32 -08:00
Chi Wang
595af7a04f
install editable package in codespace (#826)
* install editable package in codespace

* fix test error in test_forecast

* fix test error in test_space

* openml version

* break tests; pre-commit

* skip on py10+win32

* install mlflow in test

* install mlflow in [test]

* skip test in windows

* import

* handle PermissionError

* skip test in windows

* skip test in windows

* skip test in windows

* skip test in windows

* remove ts_forecast_panel from doc
2022-11-27 14:22:54 -05:00
Shaokun Zhang
586afe0d6b
fix typo (#823) 2022-11-22 11:58:02 -05:00
Sayan Roy
289f4d61d3
Merge branch 'microsoft:main' into main 2022-11-07 01:34:15 +05:30
Sayan Roy
f1d4adf4ff
limiting the scope to tabular data.
limiting the scope to tabular data.
2022-11-07 01:29:12 +05:30
zsk
958432190f
Merge branch 'main' into main 2022-11-04 23:07:13 -04:00
ElinaAndreeva
723662c502 Fixed typo 2022-11-04 12:06:07 +04:00
satya-vinay
ddebfaea1c Removed extra | in documentation 2022-11-03 13:30:42 +05:30
Nicolas Beltran-Velez
65ad6508ce
Docs (#765)
* Small docstring change for clarity

* Added tentative changes to docs

* Update website/docs/Use-Cases/Task-Oriented-AutoML.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/model.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Updated model.py to reflect `n_jobs = None` suggestion

* Updated tutorial to reflect `n_jobs=None` suggestion

* Update model.py

Improved string

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2022-11-01 15:14:08 -04:00
evensure
4d4e6ba499
Added hint to escape brackets for pip setup
As described in #777, I had trouble executing the setup using zsh.
Eventually, I noticed that I had to escape the brackets.

My proposed change includes both brackets to be escaped, however for me it was enough to escape the opening one only, since as far as I know a trailing non-escaped closing bracket will then be recognized, accordingly.
2022-10-31 17:51:11 +01:00
Kirito
b63f1bc8b8
Indentation corrected (#778) 2022-10-29 21:15:46 -07:00
Anonymous-submission-repo
79c479ae46 update 2022-10-18 23:18:09 +00:00
Anonymous-submission-repo
2f8f8ba9d8 update 2022-10-18 14:48:04 +00:00
Anonymous-submission-repo
e42cfabf21 fix 2022-10-18 13:45:23 +00:00
Anonymous-submission-repo
0869e9e99a clean up 2022-10-18 13:33:37 +00:00