Skip to content

Commit a803edd

Browse files
author
Github Actions
committed
ArlindKadra: Fixing bug in unit tests
1 parent a9c09d4 commit a803edd

File tree

17 files changed

+549
-613
lines changed

17 files changed

+549
-613
lines changed

‎refactor_development_regularization_cocktails/_downloads/2f0f54a9831653ac5c12ee0e06324a62/basics_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/8ef6602cf8ed40221edd89244fb031af/basics_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/b38295cf56f02b1c4547385a8f389f90/advanced_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/df038041811db9bd567de83692a8f994/advanced_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_custom_configuration_space.rst.txt‎

Lines changed: 163 additions & 108 deletions
Large diffs are not rendered by default.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_resampling_strategy.rst.txt‎

Lines changed: 78 additions & 43 deletions
Large diffs are not rendered by default.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/sg_execution_times.rst.txt‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@
55

66
Computation times
77
=================
8-
**21:47.269** total execution time for **advanced_tabular** files:
8+
**21:50.044** total execution time for **advanced_tabular** files:
99

1010
+--------------------------------------------------------------------------------------------------------------------+-----------+--------+
11-
| :ref:`sphx_glr_advanced_tabular_example_custom_configuration_space.py` (``example_custom_configuration_space.py``) | 11:37.138 | 0.0 MB |
11+
| :ref:`sphx_glr_advanced_tabular_example_custom_configuration_space.py` (``example_custom_configuration_space.py``) | 11:39.015 | 0.0 MB |
1212
+--------------------------------------------------------------------------------------------------------------------+-----------+--------+
13-
| :ref:`sphx_glr_advanced_tabular_example_resampling_strategy.py` (``example_resampling_strategy.py``) | 10:10.131 | 0.0 MB |
13+
| :ref:`sphx_glr_advanced_tabular_example_resampling_strategy.py` (``example_resampling_strategy.py``) | 10:11.029 | 0.0 MB |
1414
+--------------------------------------------------------------------------------------------------------------------+-----------+--------+

‎refactor_development_regularization_cocktails/_sources/basics_tabular/example_tabular_classification.rst.txt‎

Lines changed: 18 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ with AutoPyTorch
3636

3737
.. code-block:: none
3838
39-
<smac.runhistory.runhistory.RunHistory object at 0x7f06e3589100> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
39+
<smac.runhistory.runhistory.RunHistory object at 0x7f56e5c26100> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
4040
data_loader:batch_size, Value: 64
4141
encoder:__choice__, Value: 'OneHotEncoder'
4242
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -75,7 +75,7 @@ with AutoPyTorch
7575
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
7676
trainer:StandardTrainer:weighted_loss, Value: True
7777
trainer:__choice__, Value: 'StandardTrainer'
78-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.001990079879760742, budget=0), TrajEntry(train_perf=0.18128654970760238, incumbent_id=1, incumbent=Configuration:
78+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0020422935485839844, budget=0), TrajEntry(train_perf=0.1286549707602339, incumbent_id=1, incumbent=Configuration:
7979
data_loader:batch_size, Value: 64
8080
encoder:__choice__, Value: 'OneHotEncoder'
8181
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -114,73 +114,21 @@ with AutoPyTorch
114114
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
115115
trainer:StandardTrainer:weighted_loss, Value: True
116116
trainer:__choice__, Value: 'StandardTrainer'
117-
, ta_runs=1, ta_time_used=4.762185096740723, wallclock_time=6.311216831207275, budget=5.555555555555555), TrajEntry(train_perf=0.14035087719298245, incumbent_id=2, incumbent=Configuration:
118-
data_loader:batch_size, Value: 33
119-
encoder:__choice__, Value: 'OneHotEncoder'
120-
feature_preprocessor:Nystroem:kernel, Value: 'cosine'
121-
feature_preprocessor:Nystroem:n_components, Value: 5
122-
feature_preprocessor:__choice__, Value: 'Nystroem'
123-
imputer:categorical_strategy, Value: 'most_frequent'
124-
imputer:numerical_strategy, Value: 'most_frequent'
125-
lr_scheduler:__choice__, Value: 'NoScheduler'
126-
network_backbone:ResNetBackbone:activation, Value: 'tanh'
127-
network_backbone:ResNetBackbone:blocks_per_group_0, Value: 4
128-
network_backbone:ResNetBackbone:blocks_per_group_1, Value: 1
129-
network_backbone:ResNetBackbone:blocks_per_group_10, Value: 3
130-
network_backbone:ResNetBackbone:blocks_per_group_2, Value: 1
131-
network_backbone:ResNetBackbone:blocks_per_group_3, Value: 4
132-
network_backbone:ResNetBackbone:blocks_per_group_4, Value: 1
133-
network_backbone:ResNetBackbone:blocks_per_group_5, Value: 2
134-
network_backbone:ResNetBackbone:blocks_per_group_6, Value: 1
135-
network_backbone:ResNetBackbone:blocks_per_group_7, Value: 1
136-
network_backbone:ResNetBackbone:blocks_per_group_8, Value: 1
137-
network_backbone:ResNetBackbone:blocks_per_group_9, Value: 3
138-
network_backbone:ResNetBackbone:num_groups, Value: 10
139-
network_backbone:ResNetBackbone:num_units_0, Value: 645
140-
network_backbone:ResNetBackbone:num_units_1, Value: 368
141-
network_backbone:ResNetBackbone:num_units_10, Value: 32
142-
network_backbone:ResNetBackbone:num_units_2, Value: 733
143-
network_backbone:ResNetBackbone:num_units_3, Value: 16
144-
network_backbone:ResNetBackbone:num_units_4, Value: 45
145-
network_backbone:ResNetBackbone:num_units_5, Value: 573
146-
network_backbone:ResNetBackbone:num_units_6, Value: 961
147-
network_backbone:ResNetBackbone:num_units_7, Value: 714
148-
network_backbone:ResNetBackbone:num_units_8, Value: 16
149-
network_backbone:ResNetBackbone:num_units_9, Value: 12
150-
network_backbone:ResNetBackbone:use_batch_norm, Value: False
151-
network_backbone:ResNetBackbone:use_dropout, Value: False
152-
network_backbone:ResNetBackbone:use_skip_connection, Value: False
153-
network_backbone:__choice__, Value: 'ResNetBackbone'
154-
network_embedding:__choice__, Value: 'NoEmbedding'
155-
network_head:__choice__, Value: 'fully_connected'
156-
network_head:fully_connected:num_layers, Value: 1
157-
network_init:XavierInit:bias_strategy, Value: 'Normal'
158-
network_init:__choice__, Value: 'XavierInit'
159-
optimizer:AdamOptimizer:beta1, Value: 0.9861841015403867
160-
optimizer:AdamOptimizer:beta2, Value: 0.9705096117397842
161-
optimizer:AdamOptimizer:lr, Value: 0.00011070133341954491
162-
optimizer:AdamOptimizer:use_weight_decay, Value: False
163-
optimizer:__choice__, Value: 'AdamOptimizer'
164-
scaler:__choice__, Value: 'StandardScaler'
165-
trainer:StandardTrainer:use_lookahead_optimizer, Value: False
166-
trainer:StandardTrainer:use_snapshot_ensemble, Value: False
167-
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: False
168-
trainer:StandardTrainer:weighted_loss, Value: False
169-
trainer:__choice__, Value: 'StandardTrainer'
170-
, ta_runs=7, ta_time_used=39.218220472335815, wallclock_time=54.161311864852905, budget=5.555555555555555)]
171-
{'accuracy': 0.861271676300578}
172-
| | Preprocessing | Estimator | Weight |
173-
|---:|:----------------------------------------------------------|:-------------------------------------------------------------------|---------:|
174-
| 0 | SimpleImputer,NoEncoder,Normalizer,PolynomialFeatures | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.34 |
175-
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.12 |
176-
| 2 | None | RFClassifier | 0.12 |
177-
| 3 | None | ExtraTreesClassifier | 0.1 |
178-
| 4 | None | CatBoostClassifier | 0.08 |
179-
| 5 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
180-
| 6 | SimpleImputer,NoEncoder,Normalizer,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
181-
| 7 | None | KNNClassifier | 0.06 |
182-
| 8 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
183-
| 9 | None | LGBMClassifier | 0.02 |
117+
, ta_runs=1, ta_time_used=4.937290668487549, wallclock_time=6.409359693527222, budget=5.555555555555555)]
118+
{'accuracy': 0.8901734104046243}
119+
| | Preprocessing | Estimator | Weight |
120+
|---:|:------------------------------------------------------------------|:-------------------------------------------------------------------|---------:|
121+
| 0 | SimpleImputer,NoEncoder,Normalizer,PolynomialFeatures | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.22 |
122+
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.18 |
123+
| 2 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.16 |
124+
| 3 | None | ExtraTreesClassifier | 0.14 |
125+
| 4 | None | CatBoostClassifier | 0.12 |
126+
| 5 | None | RFClassifier | 0.08 |
127+
| 6 | None | LGBMClassifier | 0.02 |
128+
| 7 | SimpleImputer,NoEncoder,StandardScaler,PolynomialFeatures | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
129+
| 8 | None | SVC | 0.02 |
130+
| 9 | None | KNNClassifier | 0.02 |
131+
| 10 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
184132
185133
186134
@@ -258,7 +206,7 @@ with AutoPyTorch
258206
259207
.. rst-class:: sphx-glr-timing
260208

261-
**Total running time of the script:** ( 5 minutes 57.013 seconds)
209+
**Total running time of the script:** ( 5 minutes 54.838 seconds)
262210

263211

264212
.. _sphx_glr_download_basics_tabular_example_tabular_classification.py:

‎refactor_development_regularization_cocktails/_sources/basics_tabular/example_tabular_regression.rst.txt‎

Lines changed: 9 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ with AutoPyTorch
3636

3737
.. code-block:: none
3838
39-
<smac.runhistory.runhistory.RunHistory object at 0x7f06cabf72b0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
39+
<smac.runhistory.runhistory.RunHistory object at 0x7f56d6094850> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
4040
data_loader:batch_size, Value: 64
4141
encoder:__choice__, Value: 'OneHotEncoder'
4242
imputer:categorical_strategy, Value: 'most_frequent'
@@ -73,7 +73,7 @@ with AutoPyTorch
7373
trainer:StandardTrainer:use_snapshot_ensemble, Value: True
7474
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
7575
trainer:__choice__, Value: 'StandardTrainer'
76-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0018274784088134766, budget=0), TrajEntry(train_perf=0.6657934001689969, incumbent_id=1, incumbent=Configuration:
76+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0021834373474121094, budget=0), TrajEntry(train_perf=0.407800650115584, incumbent_id=1, incumbent=Configuration:
7777
data_loader:batch_size, Value: 64
7878
encoder:__choice__, Value: 'OneHotEncoder'
7979
imputer:categorical_strategy, Value: 'most_frequent'
@@ -110,84 +110,14 @@ with AutoPyTorch
110110
trainer:StandardTrainer:use_snapshot_ensemble, Value: True
111111
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
112112
trainer:__choice__, Value: 'StandardTrainer'
113-
, ta_runs=1, ta_time_used=4.159854412078857, wallclock_time=7.72027325630188, budget=5.555555555555555), TrajEntry(train_perf=0.40097212177595754, incumbent_id=2, incumbent=Configuration:
114-
data_loader:batch_size, Value: 89
115-
encoder:__choice__, Value: 'NoEncoder'
116-
imputer:categorical_strategy, Value: 'most_frequent'
117-
imputer:numerical_strategy, Value: 'most_frequent'
118-
lr_scheduler:__choice__, Value: 'NoScheduler'
119-
network_backbone:ShapedMLPBackbone:activation, Value: 'relu'
120-
network_backbone:ShapedMLPBackbone:max_units, Value: 165
121-
network_backbone:ShapedMLPBackbone:mlp_shape, Value: 'hexagon'
122-
network_backbone:ShapedMLPBackbone:num_groups, Value: 5
123-
network_backbone:ShapedMLPBackbone:output_dim, Value: 56
124-
network_backbone:ShapedMLPBackbone:use_dropout, Value: False
125-
network_backbone:__choice__, Value: 'ShapedMLPBackbone'
126-
network_embedding:__choice__, Value: 'NoEmbedding'
127-
network_head:__choice__, Value: 'fully_connected'
128-
network_head:fully_connected:num_layers, Value: 1
129-
network_init:NoInit:bias_strategy, Value: 'Zero'
130-
network_init:__choice__, Value: 'NoInit'
131-
optimizer:AdamOptimizer:beta1, Value: 0.8807659366281321
132-
optimizer:AdamOptimizer:beta2, Value: 0.9798886944835528
133-
optimizer:AdamOptimizer:lr, Value: 0.002130276824426696
134-
optimizer:AdamOptimizer:use_weight_decay, Value: True
135-
optimizer:AdamOptimizer:weight_decay, Value: 0.0015643941083428694
136-
optimizer:__choice__, Value: 'AdamOptimizer'
137-
scaler:__choice__, Value: 'StandardScaler'
138-
trainer:RowCutOutTrainer:Lookahead:la_alpha, Value: 0.7581060744734438
139-
trainer:RowCutOutTrainer:Lookahead:la_steps, Value: 9
140-
trainer:RowCutOutTrainer:cutout_prob, Value: 0.9603666031316771
141-
trainer:RowCutOutTrainer:patch_ratio, Value: 0.43329354649550056
142-
trainer:RowCutOutTrainer:se_lastk, Constant: 3
143-
trainer:RowCutOutTrainer:use_lookahead_optimizer, Value: True
144-
trainer:RowCutOutTrainer:use_snapshot_ensemble, Value: True
145-
trainer:RowCutOutTrainer:use_stochastic_weight_averaging, Value: True
146-
trainer:__choice__, Value: 'RowCutOutTrainer'
147-
, ta_runs=3, ta_time_used=15.746753454208374, wallclock_time=24.34652805328369, budget=5.555555555555555), TrajEntry(train_perf=0.21880654288470802, incumbent_id=3, incumbent=Configuration:
148-
data_loader:batch_size, Value: 64
149-
encoder:__choice__, Value: 'OneHotEncoder'
150-
imputer:categorical_strategy, Value: 'most_frequent'
151-
imputer:numerical_strategy, Value: 'mean'
152-
lr_scheduler:ReduceLROnPlateau:factor, Value: 0.1
153-
lr_scheduler:ReduceLROnPlateau:mode, Value: 'min'
154-
lr_scheduler:ReduceLROnPlateau:patience, Value: 10
155-
lr_scheduler:__choice__, Value: 'ReduceLROnPlateau'
156-
network_backbone:ShapedMLPBackbone:activation, Value: 'relu'
157-
network_backbone:ShapedMLPBackbone:max_units, Value: 200
158-
network_backbone:ShapedMLPBackbone:mlp_shape, Value: 'funnel'
159-
network_backbone:ShapedMLPBackbone:num_groups, Value: 5
160-
network_backbone:ShapedMLPBackbone:output_dim, Value: 200
161-
network_backbone:ShapedMLPBackbone:use_dropout, Value: False
162-
network_backbone:__choice__, Value: 'ShapedMLPBackbone'
163-
network_embedding:__choice__, Value: 'NoEmbedding'
164-
network_head:__choice__, Value: 'fully_connected'
165-
network_head:fully_connected:activation, Value: 'relu'
166-
network_head:fully_connected:num_layers, Value: 2
167-
network_head:fully_connected:units_layer_1, Value: 128
168-
network_init:XavierInit:bias_strategy, Value: 'Normal'
169-
network_init:__choice__, Value: 'XavierInit'
170-
optimizer:AdamOptimizer:beta1, Value: 0.9
171-
optimizer:AdamOptimizer:beta2, Value: 0.9
172-
optimizer:AdamOptimizer:lr, Value: 0.01
173-
optimizer:AdamOptimizer:use_weight_decay, Value: True
174-
optimizer:AdamOptimizer:weight_decay, Value: 0.0001
175-
optimizer:__choice__, Value: 'AdamOptimizer'
176-
scaler:__choice__, Value: 'StandardScaler'
177-
trainer:StandardTrainer:Lookahead:la_alpha, Value: 0.6
178-
trainer:StandardTrainer:Lookahead:la_steps, Value: 6
179-
trainer:StandardTrainer:se_lastk, Constant: 3
180-
trainer:StandardTrainer:use_lookahead_optimizer, Value: True
181-
trainer:StandardTrainer:use_snapshot_ensemble, Value: True
182-
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
183-
trainer:__choice__, Value: 'StandardTrainer'
184-
, ta_runs=11, ta_time_used=51.92302918434143, wallclock_time=78.90511298179626, budget=16.666666666666664)]
185-
{'r2': 0.9115917714787758}
113+
, ta_runs=1, ta_time_used=4.138164281845093, wallclock_time=7.867510557174683, budget=5.555555555555555)]
114+
{'r2': 0.8990494132483559}
186115
| | Preprocessing | Estimator | Weight |
187116
|---:|:-------------------------------------------|:----------------------------------------------------------------|---------:|
188-
| 0 | SimpleImputer,OneHotEncoder,StandardScaler | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.8 |
189-
| 1 | SimpleImputer,NoEncoder,StandardScaler | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.16 |
190-
| 2 | SimpleImputer,OneHotEncoder,NoScaler | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
117+
| 0 | SimpleImputer,OneHotEncoder,StandardScaler | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.66 |
118+
| 1 | SimpleImputer,OneHotEncoder,StandardScaler | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.16 |
119+
| 2 | SimpleImputer,OneHotEncoder,StandardScaler | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.12 |
120+
| 3 | SimpleImputer,OneHotEncoder,NoScaler | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
191121
192122
193123
@@ -279,7 +209,7 @@ with AutoPyTorch
279209
280210
.. rst-class:: sphx-glr-timing
281211

282-
**Total running time of the script:** ( 5 minutes 14.211 seconds)
212+
**Total running time of the script:** ( 5 minutes 15.257 seconds)
283213

284214

285215
.. _sphx_glr_download_basics_tabular_example_tabular_regression.py:

‎refactor_development_regularization_cocktails/_sources/basics_tabular/sg_execution_times.rst.txt‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@
55

66
Computation times
77
=================
8-
**11:11.224** total execution time for **basics_tabular** files:
8+
**11:10.095** total execution time for **basics_tabular** files:
99

1010
+----------------------------------------------------------------------------------------------------------+-----------+--------+
11-
| :ref:`sphx_glr_basics_tabular_example_tabular_classification.py` (``example_tabular_classification.py``) | 05:57.013 | 0.0 MB |
11+
| :ref:`sphx_glr_basics_tabular_example_tabular_classification.py` (``example_tabular_classification.py``) | 05:54.838 | 0.0 MB |
1212
+----------------------------------------------------------------------------------------------------------+-----------+--------+
13-
| :ref:`sphx_glr_basics_tabular_example_tabular_regression.py` (``example_tabular_regression.py``) | 05:14.211 | 0.0 MB |
13+
| :ref:`sphx_glr_basics_tabular_example_tabular_regression.py` (``example_tabular_regression.py``) | 05:15.257 | 0.0 MB |
1414
+----------------------------------------------------------------------------------------------------------+-----------+--------+

0 commit comments

Comments
 (0)