SEA: add support for Hybrid disposition (#631)

varun-edachali-dbx · web-flow · commit 640cc82eb339 · 2025-07-17T09:38:42.000+05:30
* Revert "Merge branch 'sea-migration' into exec-models-sea" This reverts commit 8bd12d8, reversing changes made to 030edf8. * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e, reversing changes made to 37813ba. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove fetch phase methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code repetititon + introduce gaps after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description extraction to helper func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add more unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * streamline unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test getting the list of allowed configurations Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * house constants in enums for readability and immutability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add note on hybrid disposition Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from cloudfetch-sea] introduce external links + arrow functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce responsibility of Queue Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce repetition in arrow tablee creation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce redundant code in CloudFetchQueue Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move chunk link progression to separate func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improve logging Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove reliance on schema_bytes in SEA Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant note on arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use more fetch methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant schema_bytes from parent constructor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * only call get_chunk_link with non null chunk index Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align SeaResultSet structure with ThriftResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe _fill_result_buffer from SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code repetition Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align SeaResultSet with ext-links-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidental venv changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pre-fetch next chunk link on processing current Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce nesting Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line break after multi line pydoc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce schema_bytes for better abstraction (likely temporary) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add fetchmany_arrow and fetchall_arrow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidental changes in sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary test changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes in thrift backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unimplemented methods test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unimplemented method tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * modify example scripts to include fetch calls Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes to sea test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce accidentally removed description extraction method Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type errors (ssl_options, CHUNK_PATH_WITH_ID..., etc.) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * access ssl_options through connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * DEBUG level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove explicit multi chunk test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move cloud fetch queues back into utils.py Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move ThriftCloudFetchQueue above SeaCloudFetchQueue Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix sea connector tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct patch module path in cloud fetch queue tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unimplemented methods test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct add_link docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better align queries with JDBC impl Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line breaks after multi-line PRs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: introduce ExecuteResponse import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unimplemented metadata methods test, un-necessary imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce unit tests for metadata methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove verbosity in ResultSetFilter docstring Co-authored-by: jayant <167047871+jayantsing-db@users.noreply.github.com> * remove un-necessary info in ResultSetFilter docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove explicit type checking, string literals around forward annotations Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * house SQL commands in constants Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * convert complex types to string if not _use_arrow_native_complex_types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce unit tests for altered functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'fetch-json-inline' into ext-links-sea" This reverts commit dabba55, reversing changes made to dd7dc6a. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce verbosity of ResultSetFilter docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'fetch-json-inline' into ext-links-sea" This reverts commit 3a999c0, reversing changes made to a1f9b9c. * Revert "reduce verbosity of ResultSetFilter docstring" This reverts commit a1f9b9c. * Reapply "Merge branch 'fetch-json-inline' into ext-links-sea" This reverts commit 48ad7b3. * Revert "Merge branch 'fetch-json-inline' into ext-links-sea" This reverts commit dabba55, reversing changes made to dd7dc6a. * remove un-necessary filters changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove constants changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in filters tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unit test backend and JSON queue changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in sea result set testing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "remove changes in sea result set testing" This reverts commit d210ccd. * Revert "remove unit test backend and JSON queue changes" This reverts commit f6c5950. * Revert "remove changes in filters tests" This reverts commit f3f795a. * Revert "remove constants changes" This reverts commit 802d045. * Revert "remove un-necessary backend changes" This reverts commit 20822e4. * Revert "remove un-necessary filters changes" This reverts commit 5e75fb5. * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * working version Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * adopy _wait_until_command_done Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce metadata commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use new backend structure Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * constrain backend diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes to filters Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make _parse methods in models internal Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce changes in unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * run small queries with SEA during integration tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * run some tests for sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * allow empty schema bytes for alignment with SEA Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass is_vl_op to Sea backend ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove catalog requirement in get_tables Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move filters.py to SEA utils Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * prevent circular imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove cast, throw error if not SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass param as TSparkParameterValue Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove failing test (temp) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet type assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change errors to align with spec, instead of arbitrary ValueError Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make SEA backend methods return SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use spec-aligned Exceptions in SEA backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove defensive row type check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise ProgrammingError for invalid id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make is_volume_operation strict bool Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove complex types code Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "remove complex types code" This reverts commit 138359d. * introduce type conversion for primitive types for JSON + INLINE Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SEA running on metadata queries (known failures Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align expected types with databricks sdk Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * link rest api reference to validate types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove test_catalogs_returns_arrow_table test metadata commands not expected to pass Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetchall_arrow and fetchmany_arrow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove thrift aligned test_cancel_during_execute from SEA tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes in example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary chagnes in example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * _convert_json_table -> _create_json_table Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove new unit tests (to be re-added based on new arch) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in sea_result_set functionality (to be re-added) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce more integration tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SEA tests in parameterized queries Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove partial parameter fix changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary timestamp tests (pass with minor disparity) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * slightly stronger typing of _convert_json_types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of json utility func s Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of fetch*_json Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused helper methods in SqlType Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line breaks after multi line pydocs, remove excess logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line breaks after multi line pydocs, reduce diff of redundant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff of redundant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * mandate ResultData in SeaResultSet constructor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove complex type conversion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct fetch*_arrow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * recover old sea tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move queue and result set into SEA specific dir Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass ssl_options into CloudFetchQueue Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant conversion.py Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ValueError not ProgrammingError Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce SEA cloudfetch e2e tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * allow empty cloudfetch result Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add unit tests for CloudFetchQueue and SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip pyarrow dependent tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify download process: no pre-fetching Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct class name in logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align with old impl Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align next_n_rows with prev imple Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align remaining_rows with prev impl Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary Optional params Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes in thrift field if tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * init hybrid * run large queries Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * hybrid disposition Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-ncessary log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * multi frame decompression of lz4 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove custom multi-frame decompressor for lz4 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move link fetching immediately before table creation so link expiry is not an issue Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix param type in unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct param extraction Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove common constructor for databricks client abc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make SEA Http Client instance a private member Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make GetChunksResponse model more robust Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add link to doc of GetChunk response model Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass result_data instead of "initial links" into SeaCloudFetchQueue Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move download_manager init into parent CloudFetchQueue Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise ServerOperationError for no 0th chunk Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * unused iports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return None in case of empty respose Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure table is empty on no initial link s Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for total chunk count Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * iterate over chunk indexes instead of link Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove string literals around type defs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce DownloadManager import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return None for immediate out of bounds Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * iterate by chunk index instead of link Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improve docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary (?) changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_chunk_link -> get_chunk_links in unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align tests with old message Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify attachment handling Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add unit tests for hybrid disposition Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate total_chunk_count assignment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
diff --git a/examples/experimental/tests/test_sea_async_query.py b/examples/experimental/tests/test_sea_async_query.py
@@ -45,6 +45,7 @@ def test_sea_async_query_with_cloud_fetch():
             use_sea=True,
             user_agent_entry="SEA-Test-Client",
             use_cloud_fetch=True,
+            enable_query_result_lz4_compression=False,
         )
 
         logger.info(
diff --git a/examples/experimental/tests/test_sea_sync_query.py b/examples/experimental/tests/test_sea_sync_query.py
@@ -43,6 +43,7 @@ def test_sea_sync_query_with_cloud_fetch():
             use_sea=True,
             user_agent_entry="SEA-Test-Client",
             use_cloud_fetch=True,
+            enable_query_result_lz4_compression=False,
         )
 
         logger.info(
diff --git a/src/databricks/sql/backend/sea/backend.py b/src/databricks/sql/backend/sea/backend.py
@@ -130,6 +130,8 @@ def __init__(
             "_use_arrow_native_complex_types", True
         )
 
+        self.use_hybrid_disposition = kwargs.get("use_hybrid_disposition", True)
+
         # Extract warehouse ID from http_path
         self.warehouse_id = self._extract_warehouse_id(http_path)
 
@@ -456,7 +458,11 @@ def execute_command(
             ResultFormat.ARROW_STREAM if use_cloud_fetch else ResultFormat.JSON_ARRAY
         ).value
         disposition = (
-            ResultDisposition.EXTERNAL_LINKS
+            (
+                ResultDisposition.HYBRID
+                if self.use_hybrid_disposition
+                else ResultDisposition.EXTERNAL_LINKS
+            )
             if use_cloud_fetch
             else ResultDisposition.INLINE
         ).value
@@ -637,7 +643,9 @@ def get_execution_result(
             arraysize=cursor.arraysize,
         )
 
-    def get_chunk_link(self, statement_id: str, chunk_index: int) -> ExternalLink:
+    def get_chunk_links(
+        self, statement_id: str, chunk_index: int
+    ) -> List[ExternalLink]:
         """
         Get links for chunks starting from the specified index.
         Args:
@@ -654,17 +662,7 @@ def get_chunk_link(self, statement_id: str, chunk_index: int) -> ExternalLink:
         response = GetChunksResponse.from_dict(response_data)
 
         links = response.external_links or []
-        link = next((l for l in links if l.chunk_index == chunk_index), None)
-        if not link:
-            raise ServerOperationError(
-                f"No link found for chunk index {chunk_index}",
-                {
-                    "operation-id": statement_id,
-                    "diagnostic-info": None,
-                },
-            )
-
-        return link
+        return links
 
     # == Metadata Operations ==
 
diff --git a/src/databricks/sql/backend/sea/models/responses.py b/src/databricks/sql/backend/sea/models/responses.py
@@ -4,6 +4,7 @@
 These models define the structures used in SEA API responses.
 """
 
+import base64
 from typing import Dict, Any, List, Optional
 from dataclasses import dataclass
 
@@ -91,6 +92,11 @@ def _parse_result(data: Dict[str, Any]) -> ResultData:
                 )
             )
 
+    # Handle attachment field - decode from base64 if present
+    attachment = result_data.get("attachment")
+    if attachment is not None:
+        attachment = base64.b64decode(attachment)
+
     return ResultData(
         data=result_data.get("data_array"),
         external_links=external_links,
@@ -100,7 +106,7 @@ def _parse_result(data: Dict[str, Any]) -> ResultData:
         next_chunk_internal_link=result_data.get("next_chunk_internal_link"),
         row_count=result_data.get("row_count"),
         row_offset=result_data.get("row_offset"),
-        attachment=result_data.get("attachment"),
+        attachment=attachment,
     )
 
 
diff --git a/src/databricks/sql/backend/sea/queue.py b/src/databricks/sql/backend/sea/queue.py
@@ -5,6 +5,8 @@
 
 from databricks.sql.cloudfetch.download_manager import ResultFileDownloadManager
 
+from databricks.sql.cloudfetch.downloader import ResultSetDownloadHandler
+
 try:
     import pyarrow
 except ImportError:
@@ -23,7 +25,12 @@
 from databricks.sql.exc import ProgrammingError, ServerOperationError
 from databricks.sql.thrift_api.TCLIService.ttypes import TSparkArrowResultLink
 from databricks.sql.types import SSLOptions
-from databricks.sql.utils import CloudFetchQueue, ResultSetQueue
+from databricks.sql.utils import (
+    ArrowQueue,
+    CloudFetchQueue,
+    ResultSetQueue,
+    create_arrow_table_from_arrow_file,
+)
 
 import logging
 
@@ -62,6 +69,18 @@ def build_queue(
             # INLINE disposition with JSON_ARRAY format
             return JsonQueue(result_data.data)
         elif manifest.format == ResultFormat.ARROW_STREAM.value:
+            if result_data.attachment is not None:
+                arrow_file = (
+                    ResultSetDownloadHandler._decompress_data(result_data.attachment)
+                    if lz4_compressed
+                    else result_data.attachment
+                )
+                arrow_table = create_arrow_table_from_arrow_file(
+                    arrow_file, description
+                )
+                logger.debug(f"Created arrow table with {arrow_table.num_rows} rows")
+                return ArrowQueue(arrow_table, manifest.total_row_count)
+
             # EXTERNAL_LINKS disposition
             return SeaCloudFetchQueue(
                 result_data=result_data,
@@ -150,7 +169,11 @@ def __init__(
         )
 
         initial_links = result_data.external_links or []
-        first_link = next((l for l in initial_links if l.chunk_index == 0), None)
+        self._chunk_index_to_link = {link.chunk_index: link for link in initial_links}
+
+        # Track the current chunk we're processing
+        self._current_chunk_index = 0
+        first_link = self._chunk_index_to_link.get(self._current_chunk_index, None)
         if not first_link:
             # possibly an empty response
             return None
@@ -173,21 +196,24 @@ def _convert_to_thrift_link(self, link: ExternalLink) -> TSparkArrowResultLink:
             httpHeaders=link.http_headers or {},
         )
 
-    def _get_chunk_link(self, chunk_index: int) -> Optional[ExternalLink]:
-        """Progress to the next chunk link."""
+    def _get_chunk_link(self, chunk_index: int) -> Optional["ExternalLink"]:
         if chunk_index >= self._total_chunk_count:
             return None
 
-        try:
-            return self._sea_client.get_chunk_link(self._statement_id, chunk_index)
-        except Exception as e:
+        if chunk_index not in self._chunk_index_to_link:
+            links = self._sea_client.get_chunk_links(self._statement_id, chunk_index)
+            self._chunk_index_to_link.update({l.chunk_index: l for l in links})
+
+        link = self._chunk_index_to_link.get(chunk_index, None)
+        if not link:
             raise ServerOperationError(
-                f"Error fetching link for chunk {chunk_index}: {e}",
+                f"Error fetching link for chunk {chunk_index}",
                 {
                     "operation-id": self._statement_id,
                     "diagnostic-info": None,
                 },
             )
+        return link
 
     def _create_table_from_link(
         self, link: ExternalLink
diff --git a/src/databricks/sql/backend/sea/utils/constants.py b/src/databricks/sql/backend/sea/utils/constants.py
@@ -28,7 +28,7 @@ class ResultFormat(Enum):
 class ResultDisposition(Enum):
     """Enum for result disposition values."""
 
-    # TODO: add support for hybrid disposition
+    HYBRID = "INLINE_OR_EXTERNAL_LINKS"
     EXTERNAL_LINKS = "EXTERNAL_LINKS"
     INLINE = "INLINE"
 
diff --git a/src/databricks/sql/client.py b/src/databricks/sql/client.py
@@ -99,6 +99,10 @@ def __init__(
         Connect to a Databricks SQL endpoint or a Databricks cluster.
 
         Parameters:
+            :param use_sea: `bool`, optional (default is False)
+                Use the SEA backend instead of the Thrift backend.
+            :param use_hybrid_disposition: `bool`, optional (default is False)
+                Use the hybrid disposition instead of the inline disposition.
             :param server_hostname: Databricks instance host name.
             :param http_path: Http path either to a DBSQL endpoint (e.g. /sql/1.0/endpoints/1234567890abcdef)
                 or to a DBR interactive cluster (e.g. /sql/protocolv1/o/1234567890123456/1234-123456-slid123)
diff --git a/tests/unit/test_sea_backend.py b/tests/unit/test_sea_backend.py
@@ -959,8 +959,8 @@ def test_get_columns(self, sea_client, sea_session_id, mock_cursor):
                 )
             assert "Catalog name is required for get_columns" in str(excinfo.value)
 
-    def test_get_chunk_link(self, sea_client, mock_http_client, sea_command_id):
-        """Test get_chunk_link method."""
+    def test_get_chunk_links(self, sea_client, mock_http_client, sea_command_id):
+        """Test get_chunk_links method when links are available."""
         # Setup mock response
         mock_response = {
             "external_links": [
@@ -979,7 +979,7 @@ def test_get_chunk_link(self, sea_client, mock_http_client, sea_command_id):
         mock_http_client._make_request.return_value = mock_response
 
         # Call the method
-        result = sea_client.get_chunk_link("test-statement-123", 0)
+        results = sea_client.get_chunk_links("test-statement-123", 0)
 
         # Verify the HTTP client was called correctly
         mock_http_client._make_request.assert_called_once_with(
@@ -989,7 +989,10 @@ def test_get_chunk_link(self, sea_client, mock_http_client, sea_command_id):
             ),
         )
 
-        # Verify the result
+        # Verify the results
+        assert isinstance(results, list)
+        assert len(results) == 1
+        result = results[0]
         assert result.external_link == "https://example.com/data/chunk0"
         assert result.expiration == "2025-07-03T05:51:18.118009"
         assert result.row_count == 100
@@ -999,30 +1002,14 @@ def test_get_chunk_link(self, sea_client, mock_http_client, sea_command_id):
         assert result.next_chunk_index == 1
         assert result.http_headers == {"Authorization": "Bearer token123"}
 
-    def test_get_chunk_link_not_found(self, sea_client, mock_http_client):
-        """Test get_chunk_link when the requested chunk is not found."""
+    def test_get_chunk_links_empty(self, sea_client, mock_http_client):
+        """Test get_chunk_links when no links are returned (empty list)."""
         # Setup mock response with no matching chunk
-        mock_response = {
-            "external_links": [
-                {
-                    "external_link": "https://example.com/data/chunk1",
-                    "expiration": "2025-07-03T05:51:18.118009",
-                    "row_count": 100,
-                    "byte_count": 1024,
-                    "row_offset": 100,
-                    "chunk_index": 1,  # Different chunk index
-                    "next_chunk_index": 2,
-                    "http_headers": {"Authorization": "Bearer token123"},
-                }
-            ]
-        }
+        mock_response = {"external_links": []}
         mock_http_client._make_request.return_value = mock_response
 
-        # Call the method and expect an exception
-        with pytest.raises(
-            ServerOperationError, match="No link found for chunk index 0"
-        ):
-            sea_client.get_chunk_link("test-statement-123", 0)
+        # Call the method
+        results = sea_client.get_chunk_links("test-statement-123", 0)
 
         # Verify the HTTP client was called correctly
         mock_http_client._make_request.assert_called_once_with(
@@ -1031,3 +1018,7 @@ def test_get_chunk_link_not_found(self, sea_client, mock_http_client):
                 "test-statement-123", 0
             ),
         )
+
+        # Verify the results are empty
+        assert isinstance(results, list)
+        assert results == []
diff --git a/tests/unit/test_sea_queue.py b/tests/unit/test_sea_queue.py

Original file line number	Diff line number	Diff line change
`@@ -45,6 +45,7 @@ def test_sea_async_query_with_cloud_fetch():`
`45`	`45`	`use_sea=True,`
`46`	`46`	`user_agent_entry="SEA-Test-Client",`
`47`	`47`	`use_cloud_fetch=True,`
	`48`	`+ enable_query_result_lz4_compression=False,`
`48`	`49`	`)`
`49`	`50`
`50`	`51`	`logger.info(`
Original file line number	Diff line number	Diff line change
`@@ -43,6 +43,7 @@ def test_sea_sync_query_with_cloud_fetch():`
`43`	`43`	`use_sea=True,`
`44`	`44`	`user_agent_entry="SEA-Test-Client",`
`45`	`45`	`use_cloud_fetch=True,`
	`46`	`+ enable_query_result_lz4_compression=False,`
`46`	`47`	`)`
`47`	`48`
`48`	`49`	`logger.info(`