diff --git a/cloud-console-tutorials/compute_julia_jupyter_notebook/createserver.sh b/cloud-console-tutorials/compute_julia_jupyter_notebook/createserver.sh new file mode 100644 index 0000000..3c05aae --- /dev/null +++ b/cloud-console-tutorials/compute_julia_jupyter_notebook/createserver.sh @@ -0,0 +1,31 @@ +#!/usr/bin/env bash + +PROJECT=${1:-$(gcloud config get-value core/project)} +SERVERNAME=${2:-julia-nb-server} +ZONE=${3:-$(gcloud config get-value compute/zone)} +SOURCECIDR=${4:-"0.0.0.0/0"} + +gcloud beta compute --project=$PROJECT instances create $SERVERNAME \ + --zone=$ZONE \ + --machine-type=n1-standard-2 \ + --subnet=default \ + --network-tier=PREMIUM \ + --maintenance-policy=MIGRATE \ + --scopes=https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/trace.append \ + --tags=jupyter-server \ + --image=debian-9-drawfork-v20180810 \ + --image-project=eip-images \ + --boot-disk-size=256GB \ + --boot-disk-type=pd-standard \ + --boot-disk-device-name=$SERVERNAME \ + --metadata-from-file=startup-script=$PWD/startup.sh + +gcloud compute --project=$PROJECT firewall-rules create default-allow-jupyter \ + --direction=INGRESS \ + --priority=1000 \ + --network=default \ + --action=ALLOW \ + --rules=tcp:8089\ + --source-ranges=$SOURCECIDR \ + --target-tags=jupyter-server + diff --git a/cloud-console-tutorials/compute_julia_jupyter_notebook/startup.sh b/cloud-console-tutorials/compute_julia_jupyter_notebook/startup.sh new file mode 100644 index 0000000..e01686a --- /dev/null +++ b/cloud-console-tutorials/compute_julia_jupyter_notebook/startup.sh @@ -0,0 +1,90 @@ +# +# Install required packages +# +apt update -y +apt install git bzip2 libgl1-mesa-glx -y + +# +# Create a jupyter user and switch to that context +# +adduser jupyter --uid 2001 --disabled-password --gecos "" --quiet + +# +# Install Anaconda +# +cd /tmp +wget https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh +bash Anaconda3-5.2.0-Linux-x86_64.sh -b -p /opt/anaconda3 + +# +# Create a Jupyter configuration or the jupyter user +# +cd /home/jupyter +mkdir .jupyter +git clone https://github.com/JuliaComputing/JuliaBoxTutorials.git notebooks + +openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout nbkey.key -out nbcert.pem --batch + +cat < /home/jupyter/.jupyter/jupyter_notebook_config.py +c.NotebookApp.certfile = u'/home/jupyter/nbcert.pem' +c.NotebookApp.keyfile = u'/home/jupyter/nbkey.key' +c.NotebookApp.allow_origin = '*' +c.NotebookApp.ip = '*' +c.NotebookApp.port = 8089 +c.NotebookApp.password = u'HASHED_PASSWD' +JUPYTER_CONFIG + +chown jupyter -R /home/jupyter + +# +# Install Julia +# +cd /opt +mkdir julia +cd julia +wget https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.0-linux-x86_64.tar.gz +tar zxf julia-1.0.0-linux-x86_64.tar.gz +rm julia-1.0.0-linux-x86_64.tar.gz +ln -s /opt/julia/julia-1.0.0/bin/julia /usr/local/bin + +# +# Add the Julia kernel to the Jupyter setup +# +cat < /tmp/ijulia.jl +using Pkg +Pkg.add("IJulia") +IJULIA + +chown jupyter /tmp/ijulia.jl +su - jupyter -c 'julia /tmp/ijulia.jl' +rm /tmp/ijulia.jl + +# +# Start the notebook server +# +cat < /lib/systemd/system/jupyter.service +[Unit] +Description=Jupyter notebook server +After=network.target + +[Service] +ExecStart=/opt/anaconda3/bin/jupyter notebook \ + --config=/home/jupyter/.jupyter/jupyter_notebook_config.py \ + --no-browser \ + --port=8089 \ + --notebook-dir=/home/jupyter/notebooks + +User=jupyter +Group=jupyter + +Restart=no +PrivateTmp=true +ProtectSystem=true + +[Install] +WantedBy=multi-user.target +JUPYTER_SERVICE + +systemctl enable juypter.service +systemctl daemon-reload +systemctl restart jupyter.service diff --git a/cloud-console-tutorials/compute_julia_jupyter_notebook/tutorial.md b/cloud-console-tutorials/compute_julia_jupyter_notebook/tutorial.md new file mode 100644 index 0000000..6596328 --- /dev/null +++ b/cloud-console-tutorials/compute_julia_jupyter_notebook/tutorial.md @@ -0,0 +1,103 @@ +# Run a private Julia Jupyter Notebook server on Google Cloud Platform + +This tutorial shows you how to set up a Jupyter Notebook server with a +[Julia](https://julialang.org/) kernel installed in a Google Cloud Platform project. + +[![button](http://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/wardharold/julia-notebook&page=editor&tutorial=tutorial.md) + +If you want to create a new project for your notebook server proceed to +[Task 0](#task-0-optional-create-a-project-with-a-billing-account-attached). +Otherwise you can skip that task and use and existing project. + +## Task 0 (OPTIONAL) Create a project with a billing account attached +**(you can also use an existing project and skip to the next step)** +```sh +ORG=[YOUR_ORG] +BILLING_ACCOUNT=[YOUR_BILLING_ACCOUNT_NAME] +ZONE=[COMPUTE ZONE YOU WANT TO USE] +gcloud projects create $PROJECT" --organization=$ORG +gcloud beta billing projects link $PROJECT --billing-account=$(gcloud beta billing accounts list | grep $BILLING_ACCOUNT | awk '{print $1}') +gcloud config configurations create -- activate $PROJECT +gcloud config set compute/zone $ZONE +``` + +## Task 1 Set up your environment variables +Ensure you are working with the project you want to use to run your notebook server. + +Set the environment variables for your project. +```sh +PROJECT=$(gcloud config get-value core/project) +ZONE=$(gcloud config get-value compute/zone) +``` + +Choose a name for your server and set the SERVERNAME envrionment variable. +```sh +SERVERNAME=your-preferred-servername +``` + +Make sure none of these values is empty +```sh +echo $PROJECT $SERVERNAME $ZONE +``` + +Go to the browser you will use to connect to your notebook server and browse to [Your Address As a Service](https://v4.ident.me). Copy the resulting IP address. + +```sh +SOURCECIDR=your-browsers-ip-address +``` + +## Task 2 Set up a password for your notebook server +The ```startup.sh``` script creates a Jupyter Notebook configuration on your notebook server. +Part of that configuration is a hashed password. In this task you will use the Jupyter python +library to create a hashed password for your notebook server. + +Install the jupyter python modules +```sh +pip3 install jupyter +``` + +Generate a hashed password. +``` +PASSWD=your-preferred-password +HASHED_PASSWD=$(python3 -c "from notebook.auth import passwd; print(passwd(\"${PASSWD}\"))") +``` + +Use your password in the Jupyter Notebook configuration set up by the startup.sh script. +```sh +sed -i 's/HASHED_PASSWD/'"${HASHED_PASSWD}"'/' startup.sh +``` + +## Task 3 Create your notebook server +Run the ```createserver.sh``` script to setup your notebook server. +```sh +. ./createserver.sh +``` + +The script creates a Google Cloud Platform compute instance. The instance's name is the value you gave +the ```SERVERNAME``` environment variable in [Task 1](#task-1-set-up-your-environment-variables). The +compute instance executes the ```startup.sh``` script after it boots and does the following: + +* Installs Jupyter via the [Anaconda Distribution](https://www.anaconda.com/download/#linux) +* Creates a jupyter user to run the notebook server +* Installs the [Julia](https://julialang.org/) kernel in the Jupyter notebook server +* Clones some [Julia notebooks](https://github.com/JuliaComputing/JuliaBoxTutorials) +* Starts the Jupyter Notebook server + +Installing the Julia kernel is a time consuming process. Your notebook server will not be ready +for five to ten minutes. + +## Task 4 Connect to your notebook server +Determine the public IP address of your notebook server. +```sh +gcloud compute instances list --format="value(name, networkInterfaces.accessConfigs[0].natIP)" --filter="$SERVERNAME" | awk '{print $}' +``` + +Your notebook server runs on port 8089 and is configured to require TLS (https). The ```startup.sh``` script creates a self-signed certificate to secure communications with your notebook server. + +In your browser navigate to port 8089 of your notebook server's public IP address using https, e.g., +https://1.2.3.4:8089. Because your notebook server uses a self-signed certificate your browser will +complain that it doesn't recognize the certificate. You can safely bypass those warnings and +connect to your notebook server. + +Log into your notebook server using the password you created in [Task 2](#task-2-set-up-a-password-for-your-notebook-server). +Use any of the pre-installed notebooks to learn Julia or create your own. \ No newline at end of file