Skip to content

Conversation

@huangxiaopingRD
Copy link
Contributor

What changes were proposed in this pull request?

Remove getRecoveryPath method from YarnShuffleService.java, it is no longer used.

Does this PR introduce any user-facing change?

No

How was this patch tested?

No need test

Was this patch authored or co-authored using generative AI tooling?

No

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I ask why you think like that, @huangxiaopingRD ?

Remove getRecoveryPath method from YarnShuffleService.java, it is no longer used.

@huangxiaopingRD
Copy link
Contributor Author

May I ask why you think like that, @huangxiaopingRD ?

Remove getRecoveryPath method from YarnShuffleService.java, it is no longer used.

@dongjoon-hyun Thank you for your reply. After this PR (https://github.com/apache/spark/pull/14999/files), getRecoveryPath is not used anywhere else.

@dongjoon-hyun
Copy link
Member

However, we maintained both setter and getter methods like Apache Hadoop AuxiliaryService abstract class. I'm not sure it's a good idea for us to remove the only way to get the recovery path at Apache Spark 4.2.0.

public void setRecoveryPath(Path recoveryPath) {
protected Path getRecoveryPath(String fileName) {

IIUC, this is public class, isn't it?

public class YarnShuffleService extends AuxiliaryService {

@huangxiaopingRD
Copy link
Contributor Author

However, we maintained both setter and getter methods like Apache Hadoop AuxiliaryService abstract class. I'm not sure it's a good idea for us to remove the only way to get the recovery path at Apache Spark 4.2.0.

public void setRecoveryPath(Path recoveryPath) {
protected Path getRecoveryPath(String fileName) {

IIUC, this is public class, isn't it?

public class YarnShuffleService extends AuxiliaryService {

I understand what you mean. Thank you @dongjoon-hyun

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants