About the code implementation on the QoR task

[Jedges]

I'm sorry to bother you again. I implemented the code and tested it according to the previous discussion, but the results do not seem to be consistent with those in the paper, so I have a few questions.

1. HOGA needs to calculate multi-hop features in advance. In the QoR task, its initial feature is a 3D vector, which is then mapped to a high-order vector through an embedding layer. At which step does Hoga calculate multi-hops? If it is the latter, it must be recalculated in each epoch?
2. I used the evalute function in OpenABC-D to calculate the result statistics. Did you also use this function? It used mean_absolute_percentage_error to calculate MAPE, and the result I got was two orders of magnitude larger than the one reported in your paper. I wonder if you have encountered this problem. This is the test result on some data
   
3. If it is convenient, could you provide the model weights or training code? I'm not sure if my implementation is correct.

I sincerely hope to get your reply, thanks very much.

[Chenhui1016]

Hi,

Here are the answers to your questions:

1. We calculate the hop-wise features based on the initial 3D vector.
2. Your results are actually consistent with ours. Please refer to our note for details.
3. We have provided all the necessary scripts and sources for reproducing our results (which seems you have already done). Additionally, we provided a simple script to quickly reproduce all MAPE scores on test set and plots of HOGA predictions vs. ground truth. Since I’m fully occupied with other work, we do not plan to add additional scripts in the near future.

[Jedges]

Thanks very much for your reply, which answered my question about MAPE calculation. It seems that I have successfully reproduced the results of HOGA on the QoR task.
I wish you all the best in your future work.