Some questions on the reproduction of CVRPTW

[jieyibi]

Dear authors,

👍Thanks for your insightful work. I have some questions during my reproduction of your work on CVRPTW.

When I run the command of CVRPTW, I found that the inference time of my reproduction is relatively much faster than yours and the performance is worse than LKH-3 (30k), which can not exactly match the results in your paper (i.e. Figure 14).

So I'm wondering if there is any wrong settings of the parameters during my reproduction. Could you kindly give us the exact command of CVRPTW testing so that I can better reproduce your results? In addtion, what's your pyg version?

P.S. Some details of my reproduction are attached below.

First, I downloaded your given zip file.

As you described in Issue 2, I ran the preprocess.py first to generate the preprocessed validation file $DATASET_DIR/val_routeneighbors10.npz.

Then, I followed your instructions in README,

For the CVRPTW distribution, the main difference from uniform CVRP is in problem generation.
For other steps of the framework, add --ptype CVRPTW as an argument to every uniform CVRP command above.

We used the following command to test your given pretrained model (in exps/cvrptw_uniform_N2000_routeneighbors5_beam1_depth160) on your given test dataset (i.e. generations/cvrptw_uniform_N2000/problems_test.npz), whose settings of parameters was the same as your uniform CVRP command.

export TRAIN_DIR=exps/cvrptw_uniform_N2000_routeneighbors5_beam1_depth160/rotate_flip_augnode0.005_augroute0.0005_xfc_ln_lr0.001_batch512
export GENERATE_CHECKPOINT_STEP=40000
export GENERATE_SAVE_DIR=generations/cvrptw_uniform_N2000
export GENERATE_PARTITION=test
export GENERATE_SUFFIX=_test
export DEPTH=3000 # respectively for N = [500,1000,2000,3000], use [400,600,1200,2000] for K = 10 or [1000,2000,3000,4500] for K = 5
export N_INSTANCES=40
export N_RUNS=5 # use 1 for experimentation to save time

python supervised.py $DATASET_DIR $TRAIN_DIR --generate --step $GENERATE_CHECKPOINT_STEP --generate_partition $GENERATE_PARTITION --save_dir $GENERATE_SAVE_DIR --save_suffix $GENERATE_SUFFIX --generate_depth $DEPTH --generate_index_start 0 --generate_index_end $N_INSTANCES --n_lkh_trials 500 --n_trajectories $N_RUNS --n_cpus $N_CPUS --device cpu --ptype CVRPTW

Your help would be greatly appreciated.