Skip to content

Commit a9c09d4

Browse files
author
Github Actions
committed
ArlindKadra: Update unit test for debugging purposes
1 parent ff6e878 commit a9c09d4

File tree

17 files changed

+369
-711
lines changed

17 files changed

+369
-711
lines changed

‎refactor_development_regularization_cocktails/_downloads/2f0f54a9831653ac5c12ee0e06324a62/basics_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/8ef6602cf8ed40221edd89244fb031af/basics_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/b38295cf56f02b1c4547385a8f389f90/advanced_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/df038041811db9bd567de83692a8f994/advanced_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_custom_configuration_space.rst.txt‎

Lines changed: 110 additions & 66 deletions
Large diffs are not rendered by default.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_resampling_strategy.rst.txt‎

Lines changed: 40 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ a 67% train size split.
3838

3939
.. code-block:: none
4040
41-
<smac.runhistory.runhistory.RunHistory object at 0x7f4360a11b20> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
41+
<smac.runhistory.runhistory.RunHistory object at 0x7f06ca993760> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
4242
data_loader:batch_size, Value: 64
4343
encoder:__choice__, Value: 'OneHotEncoder'
4444
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -77,7 +77,7 @@ a 67% train size split.
7777
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
7878
trainer:StandardTrainer:weighted_loss, Value: True
7979
trainer:__choice__, Value: 'StandardTrainer'
80-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0027060508728027344, budget=0), TrajEntry(train_perf=0.1871345029239766, incumbent_id=1, incumbent=Configuration:
80+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0020017623901367188, budget=0), TrajEntry(train_perf=0.17543859649122806, incumbent_id=1, incumbent=Configuration:
8181
data_loader:batch_size, Value: 64
8282
encoder:__choice__, Value: 'OneHotEncoder'
8383
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -116,7 +116,7 @@ a 67% train size split.
116116
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
117117
trainer:StandardTrainer:weighted_loss, Value: True
118118
trainer:__choice__, Value: 'StandardTrainer'
119-
, ta_runs=1, ta_time_used=5.8215172290802, wallclock_time=7.364225149154663, budget=5.555555555555555), TrajEntry(train_perf=0.17543859649122806, incumbent_id=2, incumbent=Configuration:
119+
, ta_runs=1, ta_time_used=4.823267698287964, wallclock_time=6.263789653778076, budget=5.555555555555555), TrajEntry(train_perf=0.14619883040935677, incumbent_id=2, incumbent=Configuration:
120120
data_loader:batch_size, Value: 33
121121
encoder:__choice__, Value: 'OneHotEncoder'
122122
feature_preprocessor:Nystroem:kernel, Value: 'cosine'
@@ -169,20 +169,21 @@ a 67% train size split.
169169
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: False
170170
trainer:StandardTrainer:weighted_loss, Value: False
171171
trainer:__choice__, Value: 'StandardTrainer'
172-
, ta_runs=7, ta_time_used=49.7866108417511, wallclock_time=66.29531669616699, budget=5.555555555555555)]
173-
{'accuracy': 0.8786127167630058}
174-
| | Preprocessing | Estimator | Weight |
175-
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
176-
| 0 | None | CatBoostClassifier | 0.36 |
177-
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.18 |
178-
| 2 | None | ExtraTreesClassifier | 0.12 |
179-
| 3 | None | SVC | 0.1 |
180-
| 4 | None | RFClassifier | 0.06 |
181-
| 5 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
182-
| 6 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
183-
| 7 | None | LGBMClassifier | 0.04 |
184-
| 8 | None | KNNClassifier | 0.04 |
185-
<smac.runhistory.runhistory.RunHistory object at 0x7f4352899f10> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
172+
, ta_runs=7, ta_time_used=38.83146834373474, wallclock_time=53.90706920623779, budget=5.555555555555555)]
173+
{'accuracy': 0.8728323699421965}
174+
| | Preprocessing | Estimator | Weight |
175+
|---:|:------------------------------------------------------------------|:-------------------------------------------------------------------|---------:|
176+
| 0 | None | CatBoostClassifier | 0.24 |
177+
| 1 | None | ExtraTreesClassifier | 0.22 |
178+
| 2 | None | RFClassifier | 0.16 |
179+
| 3 | None | KNNClassifier | 0.1 |
180+
| 4 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
181+
| 5 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
182+
| 6 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
183+
| 7 | SimpleImputer,NoEncoder,Normalizer,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
184+
| 8 | None | SVC | 0.04 |
185+
| 9 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
186+
<smac.runhistory.runhistory.RunHistory object at 0x7f06e16da610> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
186187
data_loader:batch_size, Value: 64
187188
encoder:__choice__, Value: 'OneHotEncoder'
188189
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -221,7 +222,7 @@ a 67% train size split.
221222
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
222223
trainer:StandardTrainer:weighted_loss, Value: True
223224
trainer:__choice__, Value: 'StandardTrainer'
224-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0022881031036376953, budget=0), TrajEntry(train_perf=0.14317987458537698, incumbent_id=1, incumbent=Configuration:
225+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002043485641479492, budget=0), TrajEntry(train_perf=0.13354779414154527, incumbent_id=1, incumbent=Configuration:
225226
data_loader:batch_size, Value: 64
226227
encoder:__choice__, Value: 'OneHotEncoder'
227228
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -260,19 +261,19 @@ a 67% train size split.
260261
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
261262
trainer:StandardTrainer:weighted_loss, Value: True
262263
trainer:__choice__, Value: 'StandardTrainer'
263-
, ta_runs=1, ta_time_used=11.273290157318115, wallclock_time=12.906458139419556, budget=5.555555555555555)]
264+
, ta_runs=1, ta_time_used=9.001122236251831, wallclock_time=10.450978994369507, budget=5.555555555555555)]
264265
{'accuracy': 0.8728323699421965}
265266
| | Preprocessing | Estimator | Weight |
266267
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
267-
| 0 | None | TabularClassifier | 0.5 |
268-
| 1 | None | TabularClassifier | 0.14 |
269-
| 2 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.1 |
268+
| 0 | None | TabularClassifier | 0.44 |
269+
| 1 | None | TabularClassifier | 0.16 |
270+
| 2 | None | TabularClassifier | 0.12 |
270271
| 3 | None | TabularClassifier | 0.1 |
271-
| 4 | None | TabularClassifier | 0.08 |
272-
| 5 | None | TabularClassifier | 0.04 |
272+
| 4 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.08 |
273+
| 5 | None | TabularClassifier | 0.06 |
273274
| 6 | None | TabularClassifier | 0.02 |
274275
| 7 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
275-
<smac.runhistory.runhistory.RunHistory object at 0x7f434054f880> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
276+
<smac.runhistory.runhistory.RunHistory object at 0x7f06ba2ad190> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
276277
data_loader:batch_size, Value: 64
277278
encoder:__choice__, Value: 'OneHotEncoder'
278279
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -311,7 +312,7 @@ a 67% train size split.
311312
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
312313
trainer:StandardTrainer:weighted_loss, Value: True
313314
trainer:__choice__, Value: 'StandardTrainer'
314-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0024034976959228516, budget=0), TrajEntry(train_perf=0.1578947368421053, incumbent_id=1, incumbent=Configuration:
315+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002048492431640625, budget=0), TrajEntry(train_perf=0.1286549707602339, incumbent_id=1, incumbent=Configuration:
315316
data_loader:batch_size, Value: 64
316317
encoder:__choice__, Value: 'OneHotEncoder'
317318
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -350,7 +351,7 @@ a 67% train size split.
350351
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
351352
trainer:StandardTrainer:weighted_loss, Value: True
352353
trainer:__choice__, Value: 'StandardTrainer'
353-
, ta_runs=1, ta_time_used=5.775749206542969, wallclock_time=7.325538396835327, budget=5.555555555555555), TrajEntry(train_perf=0.14619883040935677, incumbent_id=2, incumbent=Configuration:
354+
, ta_runs=1, ta_time_used=4.722014427185059, wallclock_time=6.179686546325684, budget=5.555555555555555), TrajEntry(train_perf=0.13450292397660824, incumbent_id=2, incumbent=Configuration:
354355
data_loader:batch_size, Value: 33
355356
encoder:__choice__, Value: 'OneHotEncoder'
356357
feature_preprocessor:Nystroem:kernel, Value: 'cosine'
@@ -403,17 +404,19 @@ a 67% train size split.
403404
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: False
404405
trainer:StandardTrainer:weighted_loss, Value: False
405406
trainer:__choice__, Value: 'StandardTrainer'
406-
, ta_runs=7, ta_time_used=50.16112446784973, wallclock_time=67.17039346694946, budget=5.555555555555555)]
407-
{'accuracy': 0.8497109826589595}
407+
, ta_runs=11, ta_time_used=68.92335557937622, wallclock_time=93.00793981552124, budget=16.666666666666664)]
408+
{'accuracy': 0.8670520231213873}
408409
| | Preprocessing | Estimator | Weight |
409410
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
410-
| 0 | None | CatBoostClassifier | 0.34 |
411-
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.18 |
412-
| 2 | None | LGBMClassifier | 0.12 |
413-
| 3 | None | ExtraTreesClassifier | 0.12 |
414-
| 4 | None | KNNClassifier | 0.12 |
415-
| 5 | None | RFClassifier | 0.1 |
416-
| 6 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
411+
| 0 | None | ExtraTreesClassifier | 0.18 |
412+
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.16 |
413+
| 2 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.12 |
414+
| 3 | None | CatBoostClassifier | 0.12 |
415+
| 4 | None | SVC | 0.12 |
416+
| 5 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.1 |
417+
| 6 | None | RFClassifier | 0.08 |
418+
| 7 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
419+
| 8 | None | KNNClassifier | 0.06 |
417420
418421
419422
@@ -577,7 +580,7 @@ a 67% train size split.
577580
578581
.. rst-class:: sphx-glr-timing
579582

580-
**Total running time of the script:** ( 10 minutes 37.477 seconds)
583+
**Total running time of the script:** ( 10 minutes 10.131 seconds)
581584

582585

583586
.. _sphx_glr_download_advanced_tabular_example_resampling_strategy.py:

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/sg_execution_times.rst.txt‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@
55

66
Computation times
77
=================
8-
**22:30.583** total execution time for **advanced_tabular** files:
8+
**21:47.269** total execution time for **advanced_tabular** files:
99

1010
+--------------------------------------------------------------------------------------------------------------------+-----------+--------+
11-
| :ref:`sphx_glr_advanced_tabular_example_custom_configuration_space.py` (``example_custom_configuration_space.py``) | 11:53.107 | 0.0 MB |
11+
| :ref:`sphx_glr_advanced_tabular_example_custom_configuration_space.py` (``example_custom_configuration_space.py``) | 11:37.138 | 0.0 MB |
1212
+--------------------------------------------------------------------------------------------------------------------+-----------+--------+
13-
| :ref:`sphx_glr_advanced_tabular_example_resampling_strategy.py` (``example_resampling_strategy.py``) | 10:37.477 | 0.0 MB |
13+
| :ref:`sphx_glr_advanced_tabular_example_resampling_strategy.py` (``example_resampling_strategy.py``) | 10:10.131 | 0.0 MB |
1414
+--------------------------------------------------------------------------------------------------------------------+-----------+--------+

0 commit comments

Comments
 (0)