Skip to content

Question - File is not getting moved /delete after processing #93

@iamrc90

Description

@iamrc90

I am trying to move the csv file to another folder after processing is done. I can see it being processed to kafka topic but the file is not moved to the given path. Also delete is not working.

Please find my config below

{
    "name": "csv-source-second-conn8",
    "connector.class": "com.github.mmolimar.kafka.connect.fs.FsSourceConnector",
    "tasks.max": "1",
    "fs.uris": "file:///data/unprocessed",
    "topic": "csv4",
    "policy.class": "com.github.mmolimar.kafka.connect.fs.policy.SimplePolicy",
    "policy.recursive": "true",
    "policy.regexp": ".*",
    "policy.batch_size": "0",
    "policy.cleanup": "delete",
    "file_reader.class": "com.github.mmolimar.kafka.connect.fs.file.reader.CsvFileReader",
    "file_reader.batch_size": "0",
    "file_reader.delimited.settings.header": "true",
    "file_reader.delimited.settings.line_separator_detection": "true",
    "file_reader.delimited.settings.delimiter_detection": "true"
}
{
    "name": "csv-source-second-conn7",
    "connector.class": "com.github.mmolimar.kafka.connect.fs.FsSourceConnector",
    "tasks.max": "1",
    "fs.uris": "file:///data/unprocessed",
    "topic": "csv4",
    "policy.class": "com.github.mmolimar.kafka.connect.fs.policy.SimplePolicy",
    "policy.recursive": "true",
    "policy.regexp": ".*",
    "policy.batch_size": "0",
    "policy.cleanup": "move",
    "policy.cleanup.move": "file:///data/processed",
    "file_reader.class": "com.github.mmolimar.kafka.connect.fs.file.reader.CsvFileReader",
    "file_reader.batch_size": "0",
    "file_reader.delimited.settings.header": "true",
    "file_reader.delimited.settings.line_separator_detection": "true",
    "file_reader.delimited.settings.delimiter_detection": "true"
}

Please suggest if anything needs to be modified.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions