-
Notifications
You must be signed in to change notification settings - Fork 85
docs: Add sample Cloud Run example application #1335
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
60bc4fa
docs: Add sample cloudrun application
kgala2 fdbccc8
chore: update license and exception
kgala2 14c1c7c
chore: update documentation for lazy instantiation and global variables
kgala2 7c9de11
chore: update db password setup
kgala2 b4f8863
chore: remove add-cloudsql-instances flag
kgala2 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| # Connecting Cloud Run to Cloud SQL with the Python Connector | ||
|
|
||
| This guide provides a comprehensive walkthrough of how to connect a Cloud Run service to a Cloud SQL instance using the Cloud SQL Python Connector. It covers connecting to instances with both public and private IP addresses and demonstrates how to handle database credentials securely. | ||
|
|
||
| ## Develop a Python Application | ||
|
|
||
| The following Python applications demonstrate how to connect to a Cloud SQL instance using the Cloud SQL Python Connector. | ||
|
|
||
| ### `mysql/main.py` and `postgres/main.py` | ||
|
|
||
| These files contain the core application logic for connecting to a Cloud SQL for MySQL or PostgreSQL instance. They provide two separate authentication methods, each exposed at a different route: | ||
| - `/`: Password-based authentication | ||
| - `/iam`: IAM-based authentication | ||
|
|
||
|
|
||
| ### `sqlserver/main.py` | ||
|
|
||
| This file contains the core application logic for connecting to a Cloud SQL for SQL Server instance. It uses the `cloud-sql-python-connector` to create a SQLAlchemy connection pool with password-based authentication at the `/` route. | ||
|
|
||
| > [!NOTE] | ||
| > | ||
| > Cloud SQL for SQL Server does not support IAM database authentication. | ||
|
|
||
|
|
||
| > [!NOTE] | ||
| > **Lazy Refresh** | ||
| > | ||
| > The sample code in all three `main.py` files initializes the `Connector` with `refresh_strategy=lazy`. This is a recommended approach to avoid connection errors and optimize cost by preventing background processes from running when the CPU is throttled. | ||
|
|
||
| ## IAM Authentication Prerequisites | ||
|
|
||
|
|
||
| For IAM authentication to work, you must ensure two things: | ||
|
|
||
| 1. **The Cloud Run service's service account has the `Cloud SQL Client` role.** You can grant this role with the following command: | ||
| ```bash | ||
| gcloud projects add-iam-policy-binding PROJECT_ID \ | ||
| --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \ | ||
| --role="roles/cloudsql.client" | ||
| ``` | ||
| Replace `PROJECT_ID` with your Google Cloud project ID and `SERVICE_ACCOUNT_EMAIL` with the email of the service account your Cloud Run service is using. | ||
|
|
||
| 2. **The service account is added as a database user to your Cloud SQL instance.** You can do this with the following command: | ||
| ```bash | ||
| gcloud sql users create SERVICE_ACCOUNT_EMAIL \ | ||
| --instance=INSTANCE_NAME \ | ||
| --type=cloud_iam_user | ||
| ``` | ||
| Replace `SERVICE_ACCOUNT_EMAIL` with the same service account email and `INSTANCE_NAME` with your Cloud SQL instance name. | ||
|
|
||
| ## Deploy the Application to Cloud Run | ||
|
|
||
| Follow these steps to deploy the application to Cloud Run. | ||
|
|
||
| ### Build and Push the Docker Image | ||
|
|
||
| 1. **Enable the Artifact Registry API:** | ||
|
|
||
| ```bash | ||
| gcloud services enable artifactregistry.googleapis.com | ||
| ``` | ||
|
|
||
| 2. **Create an Artifact Registry repository:** | ||
|
|
||
| ```bash | ||
| gcloud artifacts repositories create REPO_NAME \ | ||
| --repository-format=docker \ | ||
| --location=REGION | ||
| ``` | ||
|
|
||
| 3. **Configure Docker to authenticate with Artifact Registry:** | ||
|
|
||
| ```bash | ||
| gcloud auth configure-docker REGION-docker.pkg.dev | ||
| ``` | ||
|
|
||
| 4. **Build the Docker image (replace `mysql` with `postgres` or `sqlserver` as needed):** | ||
|
|
||
| ```bash | ||
| docker build -t REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME mysql | ||
| ``` | ||
|
|
||
| 5. **Push the Docker image to Artifact Registry:** | ||
|
|
||
| ```bash | ||
| docker push REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME | ||
| ``` | ||
|
|
||
| ### Deploy to Cloud Run | ||
|
|
||
| Deploy the container image to Cloud Run using the `gcloud run deploy` command. | ||
|
|
||
|
|
||
| **Sample Values:** | ||
| * `SERVICE_NAME`: `my-cloud-run-service` | ||
| * `REGION`: `us-central1` | ||
| * `PROJECT_ID`: `my-gcp-project-id` | ||
| * `REPO_NAME`: `my-artifact-repo` | ||
| * `IMAGE_NAME`: `my-app-image` | ||
| * `INSTANCE_CONNECTION_NAME`: `my-gcp-project-id:us-central1:my-instance-name` | ||
| * `DB_USER`: `my-db-user` (for password-based authentication) | ||
| * `DB_IAM_USER`: `my-service-account@my-gcp-project-id.iam.gserviceaccount.com` (for IAM-based authentication) | ||
| * `DB_NAME`: `my-db-name` | ||
| * `DB_SECRET_NAME`: `projects/my-gcp-project-id/secrets/my-db-secret/versions/latest` | ||
| * `VPC_NETWORK`: `my-vpc-network` | ||
| * `SUBNET_NAME`: `my-vpc-subnet` | ||
|
|
||
|
|
||
| **For MySQL and PostgreSQL (Public IP):** | ||
|
|
||
| ```bash | ||
| gcloud run deploy SERVICE_NAME \ | ||
| --image=REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME \ | ||
| --add-cloudsql-instances=INSTANCE_CONNECTION_NAME \ | ||
| --set-env-vars=DB_USER=DB_USER,DB_IAM_USER=DB_IAM_USER,DB_NAME=DB_NAME,DB_SECRET_NAME=DB_SECRET_NAME,INSTANCE_CONNECTION_NAME=INSTANCE_CONNECTION_NAME \ | ||
| --region=REGION | ||
| ``` | ||
|
|
||
| **For MySQL and PostgreSQL (Private IP):** | ||
|
|
||
| ```bash | ||
| gcloud run deploy SERVICE_NAME \ | ||
| --image=REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME \ | ||
| --add-cloudsql-instances=INSTANCE_CONNECTION_NAME \ | ||
| --set-env-vars=DB_USER=DB_USER,DB_IAM_USER=DB_IAM_USER,DB_NAME=DB_NAME,DB_SECRET_NAME=DB_SECRET_NAME,INSTANCE_CONNECTION_NAME=INSTANCE_CONNECTION_NAME,IP_TYPE=PRIVATE \ | ||
| --network=VPC_NETWORK \ | ||
| --subnet=SUBNET_NAME \ | ||
| --vpc-egress=private-ranges-only \ | ||
| --region=REGION | ||
| ``` | ||
|
|
||
| **For SQL Server (Public IP):** | ||
|
|
||
| ```bash | ||
| gcloud run deploy SERVICE_NAME \ | ||
| --image=REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME \ | ||
| --add-cloudsql-instances=INSTANCE_CONNECTION_NAME \ | ||
| --set-env-vars=DB_USER=DB_USER,DB_NAME=DB_NAME,DB_SECRET_NAME=DB_SECRET_NAME,INSTANCE_CONNECTION_NAME=INSTANCE_CONNECTION_NAME \ | ||
| --region=REGION | ||
| ``` | ||
|
|
||
| **For SQL Server (Private IP):** | ||
|
|
||
| ```bash | ||
| gcloud run deploy SERVICE_NAME \ | ||
| --image=REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_name \ | ||
| --add-cloudsql-instances=INSTANCE_CONNECTION_NAME \ | ||
| --set-env-vars=DB_USER=DB_USER,DB_NAME=DB_NAME,DB_SECRET_NAME=DB_SECRET_NAME,INSTANCE_CONNECTION_NAME=INSTANCE_CONNECTION_NAME,IP_TYPE=PRIVATE \ | ||
| --network=VPC_NETWORK \ | ||
| --subnet=SUBNET_NAME \ | ||
| --vpc-egress=private-ranges-only \ | ||
| --region=REGION | ||
| ``` | ||
|
|
||
| > [!NOTE] | ||
| > **`For PSC connections`** | ||
| > | ||
| > To connect to the Cloud SQL instance with PSC connection type, create a PSC endpoint, a DNS zone and DNS record for the instance in the same VPC network as the Cloud Run service and replace the `IP_TYPE` in the deploy command with `PSC`. To configure DNS records, refer to [Connect to an instance using Private Service Connect](https://docs.cloud.google.com/sql/docs/mysql/configure-private-service-connect) guide |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Use the official lightweight Python image. | ||
| # https://hub.docker.com/_/python | ||
| FROM python:3.12-slim | ||
|
|
||
| # Allow statements and log messages to immediately appear in the Knative logs | ||
| ENV PYTHONUNBUFFERED True | ||
|
|
||
| # Copy local code to the container image. | ||
| ENV APP_HOME /app | ||
| WORKDIR $APP_HOME | ||
| COPY . . | ||
|
|
||
| # Install production dependencies. | ||
| RUN pip install --no-cache-dir -r requirements.txt | ||
|
|
||
| # Run the web service on container startup. | ||
| # Use gunicorn for production deployments. | ||
| CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| import os | ||
| import sqlalchemy | ||
| from flask import Flask | ||
| from google.cloud.sql.connector import Connector, IPTypes | ||
| from google.cloud import secretmanager | ||
|
|
||
| # Initialize Flask app | ||
| app = Flask(__name__) | ||
|
|
||
| # Initialize the Connector object with lazy refresh | ||
| connector = Connector(refresh_strategy="lazy") | ||
| secret_client = secretmanager.SecretManagerServiceClient() | ||
|
|
||
| # Function to create a database connection using IAM authentication | ||
| def get_iam_connection() -> sqlalchemy.engine.base.Connection: | ||
| instance_connection_name = os.environ["INSTANCE_CONNECTION_NAME"] | ||
| db_user = os.environ["DB_IAM_USER"] # IAM service account email | ||
| db_name = os.environ["DB_NAME"] | ||
| ip_type_str = os.environ.get("IP_TYPE", "PUBLIC") | ||
| ip_type = IPTypes[ip_type_str] | ||
|
|
||
| conn = connector.connect( | ||
| instance_connection_name, | ||
| "pymysql", | ||
| user=db_user, | ||
| db=db_name, | ||
| ip_type=ip_type, | ||
| enable_iam_auth=True | ||
| ) | ||
| return conn | ||
|
|
||
| # Function to create a database connection using password-based authentication | ||
| def get_password_connection() -> sqlalchemy.engine.base.Connection: | ||
| instance_connection_name = os.environ["INSTANCE_CONNECTION_NAME"] | ||
| db_user = os.environ["DB_USER"] # Database username | ||
| db_name = os.environ["DB_NAME"] | ||
| db_secret_name = os.environ["DB_SECRET_NAME"] | ||
| ip_type_str = os.environ.get("IP_TYPE", "PUBLIC") | ||
| ip_type = IPTypes[ip_type_str] | ||
|
|
||
| secret_response = secret_client.access_secret_version(name=db_secret_name) | ||
| db_password = secret_response.payload.data.decode("UTF-8") | ||
|
|
||
| conn = connector.connect( | ||
| instance_connection_name, | ||
| "pymysql", | ||
| user=db_user, | ||
| password=db_password, | ||
| db=db_name, | ||
| ip_type=ip_type, | ||
| ) | ||
| return conn | ||
|
|
||
| # Create the SQLAlchemy engines | ||
| iam_engine = sqlalchemy.create_engine( | ||
kgala2 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| "mysql+pymysql://", | ||
| creator=get_iam_connection, | ||
| ) | ||
| password_engine = sqlalchemy.create_engine( | ||
| "mysql+pymysql://", | ||
| creator=get_password_connection, | ||
| ) | ||
|
|
||
| @app.route("/") | ||
| def password_auth_index(): | ||
| try: | ||
| with password_engine.connect() as conn: | ||
| result = conn.execute(sqlalchemy.text("SELECT 1")).fetchall() | ||
| return f"Database connection successful (password authentication), result: {result}" | ||
| except Exception as e: | ||
| return f"Error connecting to the database (password authentication): {e}", 500 | ||
|
||
|
|
||
| @app.route("/iam") | ||
| def iam_auth_index(): | ||
| try: | ||
| with iam_engine.connect() as conn: | ||
| result = conn.execute(sqlalchemy.text("SELECT 1")).fetchall() | ||
| return f"Database connection successful (IAM authentication), result: {result}" | ||
| except Exception as e: | ||
| return f"Error connecting to the database (IAM authentication): {e}", 500 | ||
|
||
|
|
||
| if __name__ == "__main__": | ||
| app.run(host="0.0.0.0", port=int(os.environ.get("PORT", 8080))) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| cloud-sql-python-connector[pymysql] | ||
| sqlalchemy | ||
| Flask | ||
| gunicorn | ||
| google-cloud-secret-manager |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Use the official lightweight Python image. | ||
| # https://hub.docker.com/_/python | ||
| FROM python:3.12-slim | ||
|
|
||
| # Allow statements and log messages to immediately appear in the Knative logs | ||
| ENV PYTHONUNBUFFERED True | ||
|
|
||
| # Copy local code to the container image. | ||
| ENV APP_HOME /app | ||
| WORKDIR $APP_HOME | ||
| COPY . . | ||
|
|
||
| # Install production dependencies. | ||
| RUN pip install --no-cache-dir -r requirements.txt | ||
|
|
||
| # Run the web service on container startup. | ||
| # Use gunicorn for production deployments. | ||
| CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| import os | ||
| import sqlalchemy | ||
| from flask import Flask | ||
| from google.cloud.sql.connector import Connector, IPTypes | ||
| from google.cloud import secretmanager | ||
|
|
||
| # Initialize Flask app | ||
| app = Flask(__name__) | ||
|
|
||
| # Initialize the Connector object with lazy refresh | ||
| connector = Connector(refresh_strategy="lazy") | ||
| secret_client = secretmanager.SecretManagerServiceClient() | ||
|
|
||
| # Function to create a database connection using IAM authentication | ||
| def get_iam_connection() -> sqlalchemy.engine.base.Connection: | ||
| instance_connection_name = os.environ["INSTANCE_CONNECTION_NAME"] | ||
| db_user = os.environ["DB_IAM_USER"] # IAM service account email | ||
| db_name = os.environ["DB_NAME"] | ||
| ip_type_str = os.environ.get("IP_TYPE", "PUBLIC") | ||
| ip_type = IPTypes[ip_type_str] | ||
|
|
||
| conn = connector.connect( | ||
| instance_connection_name, | ||
| "pg8000", | ||
| user=db_user, | ||
| db=db_name, | ||
| ip_type=ip_type, | ||
| enable_iam_auth=True | ||
| ) | ||
| return conn | ||
|
|
||
| # Function to create a database connection using password-based authentication | ||
| def get_password_connection() -> sqlalchemy.engine.base.Connection: | ||
| instance_connection_name = os.environ["INSTANCE_CONNECTION_NAME"] | ||
| db_user = os.environ["DB_USER"] # Database username | ||
| db_name = os.environ["DB_NAME"] | ||
| db_secret_name = os.environ["DB_SECRET_NAME"] | ||
| ip_type_str = os.environ.get("IP_TYPE", "PUBLIC") | ||
| ip_type = IPTypes[ip_type_str] | ||
|
|
||
| secret_response = secret_client.access_secret_version(name=db_secret_name) | ||
| db_password = secret_response.payload.data.decode("UTF-8") | ||
|
|
||
| conn = connector.connect( | ||
| instance_connection_name, | ||
| "pg8000", | ||
| user=db_user, | ||
| password=db_password, | ||
| db=db_name, | ||
| ip_type=ip_type, | ||
| ) | ||
| return conn | ||
|
|
||
| # Create the SQLAlchemy engines | ||
| iam_engine = sqlalchemy.create_engine( | ||
| "postgresql+pg8000://", | ||
| creator=get_iam_connection, | ||
| ) | ||
| password_engine = sqlalchemy.create_engine( | ||
| "postgresql+pg8000://", | ||
| creator=get_password_connection, | ||
| ) | ||
|
|
||
| @app.route("/") | ||
| def password_auth_index(): | ||
| try: | ||
| with password_engine.connect() as conn: | ||
| result = conn.execute(sqlalchemy.text("SELECT 1")).fetchall() | ||
| return f"Database connection successful (password authentication), result: {result}" | ||
| except Exception as e: | ||
| return f"Error connecting to the database (password authentication): {e}", 500 | ||
|
||
|
|
||
| @app.route("/iam") | ||
| def iam_auth_index(): | ||
| try: | ||
| with iam_engine.connect() as conn: | ||
| result = conn.execute(sqlalchemy.text("SELECT 1")).fetchall() | ||
| return f"Database connection successful (IAM authentication), result: {result}" | ||
| except Exception as e: | ||
| return f"Error connecting to the database (IAM authentication): {e}", 500 | ||
|
||
|
|
||
| if __name__ == "__main__": | ||
| app.run(host="0.0.0.0", port=int(os.environ.get("PORT", 8080))) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| cloud-sql-python-connector[pg8000] | ||
| sqlalchemy | ||
| Flask | ||
| gunicorn | ||
| google-cloud-secret-manager |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Use the official lightweight Python image. | ||
| # https://hub.docker.com/_/python | ||
| FROM python:3.12-slim | ||
|
|
||
| # Allow statements and log messages to immediately appear in the Knative logs | ||
| ENV PYTHONUNBUFFERED True | ||
|
|
||
| # Copy local code to the container image. | ||
| ENV APP_HOME /app | ||
| WORKDIR $APP_HOME | ||
| COPY . . | ||
|
|
||
| # Install production dependencies. | ||
| RUN pip install --no-cache-dir -r requirements.txt | ||
|
|
||
| # Run the web service on container startup. | ||
| # Use gunicorn for production deployments. | ||
| CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.