Skip to content

feat: Parquet DataSource should provide ability to read multiple GCS buckets for creating multiple streams #105

@Meghajit

Description

@Meghajit

As part of this issue, want to add support for handling multiple streams for Parquet Data Source.
That is, users should be able to specify multiple GCS URLs. Dagger should create a parquet data source, and hence a data stream for each of these GCS URLs.

This issue is needed so that the user can do joins and other operations with multiple streams on Parquet DataSource similar to KafkaSource.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions