Commit b9dd02b
[SPARK-54524][BUILD] Fix Connect JDBC driver dependencies
### What changes were proposed in this pull request?
The Connect JDBC driver is built on top of the Connect JVM client, so it should declare the Connect JVM client as the dependency, and Maven will resolve transitive deps correctly.
Note, many RDBMS products provide a single no-deps JDBC driver jar to simplify downstream projects' integration, while this is not achieved at the current stage, and may be considered in the future.
### Why are the changes needed?
Fix Connect JDBC driver dependency management.
For example, it should not pull protobuf libs which are already shaded in connect-jdbc-client.
https://mvnrepository.com/artifact/org.apache.spark/spark-connect-client-jdbc_2.13/4.1.0-preview4
### Does this PR introduce _any_ user-facing change?
No, the Connect JDBC driver is not released yet.
### How was this patch tested?
Pass GHA, plus manual verifications.
- Checked the effective `pom.xml` which is located at `sql/connect/client/jdbc/dependency-reduced-pom.xml` after compiling, only one compile scope dep.
```xml
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-connect-client-jvm_2.13</artifactId>
<version>4.2.0-SNAPSHOT</version>
<scope>compile</scope>
</dependency>
```
- Verified locally by running tests using Maven/SBT
Maven
```
build/mvn -Phive,hive-thriftserver clean install -DskipTests
build/mvn -Phive,hive-thriftserver -pl sql/connect/client/jdbc test
```
SBT
```
build/sbt -Phive,hive-thriftserver "connect-client-jdbc/test"
```
- Verified CLI cases with BeeLine by creating a binary release artifact using Maven/SBT
Launch a Connect Server first, then
```
dev/make-distribution.sh [--sbt-enabled] -Phive,hive-thriftserver
cd dist
SPARK_CONNECT_BEELINE=1 bin/beeline -u jdbc:sc://localhost:15002 -e "select 'Hello, Spark Connect!', version() as server_version;"
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #53230 from pan3793/SPARK-53484.
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: yangjie01 <yangjie01@baidu.com>1 parent d36bd62 commit b9dd02b
1 file changed
+3
-79
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
| 40 | + | |
108 | 41 | | |
109 | | - | |
110 | | - | |
111 | 42 | | |
112 | 43 | | |
113 | 44 | | |
| |||
116 | 47 | | |
117 | 48 | | |
118 | 49 | | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | 50 | | |
127 | 51 | | |
128 | 52 | | |
| |||
142 | 66 | | |
143 | 67 | | |
144 | 68 | | |
145 | | - | |
| 69 | + | |
146 | 70 | | |
147 | 71 | | |
148 | | - | |
| 72 | + | |
149 | 73 | | |
150 | 74 | | |
151 | 75 | | |
| |||
0 commit comments