Merge pull request #2 from Springjunky/KEYCLOAK

Keycloak

Author: Springjunky

.gitignore

@@ -1,5 +1,32 @@
-docker-compose.yml-*
 *.jpi
 *.war
 .env-*
 .env
+.project
+
+.dbeaver*
+
+target/
+!.mvn/wrapper/maven-wrapper.jar
+
+### STS ###
+.apt_generated
+.classpath
+.factorypath
+.project
+.settings
+.springBeans
+
+### IntelliJ IDEA ###
+.idea
+*.iws
+*.iml
+*.ipr
+
+### NetBeans ###
+nbproject/private/
+build/
+nbbuild/
+dist/
+nbdist/
+.nb-gradle/

README.md

@@ -2,23 +2,43 @@
 
 ##### Tired of endless installation and configuration .... ?!
 
-My personal solution is a local, docker-based CI/CD Build Environment ready in a few minutes to offer you a reade-to-use convinience plaground with
-* Jenkins (with open-jdk-8, ansible, docker and maven inside)
-* Gitlab (with a docker based gitlab-runner registrated)
+My personal solution is a local, docker-based CI/CD Build Environment ready in a few minutes to offer you a reade-to-use convinience playground with
+* Jenkins 
+  * open-jdk
+  * maven
+  * ansible
+  * docker
+* Gitlab 
+  * and a docker based gitlab-runner registrated
+  * suecured docker-registry ready (openssh certificate)
+  * push with ssh avialable at port 2222
 * Nexus 3
 * Sonar
+* Keykloak (as Single Sign or JWT playground)
+* Postgres (used by sonar, keycloak and YOUR applications)
 
+so you can be your own local "DevOp"; nearly every category of [periodic table of devoptools][4] has one tool in your local setup. 
 
-#### This is NOT for any cluster (Swarm / Kubernetes)
+## This is NOT for any cluster (Swarm/Kubernetes)
 
 ### System requirements
 * At least 8GB Memory with 3GB Swap and 10GB Disk-Space
 * docker version >= 17.06.0
 * docker-compose version >= 1.15.0
 
+#### Listenports to be claimed
+
+|Port  |  Why  |
+|---|---|
+|80 |NGINX   |
+|5432 |postgres standard |
+|2222 |ssh port of gitlab, used to push via ssh connection |
+|5555 |Gitlab Docker-registry |
+
+If your change the ports in the docker-compose.yml change them also in nginx-reverse/nginx.conf (stream {...} )
 ## Installation
 ### without sonar
-Bring up your own build environment ... just do a
+Bring up your own DevOp Playground  ... just do a
 ```
    git clone https://github.com/Springjunky/docker-local-build-environment.git
    cd docker-local-build-environment
@@ -27,21 +47,25 @@ Bring up your own build environment ... just do a
    docker-compose logs
 ```
 
-### with sonar
+### with sonar and/or Keycloak
 Warning: you need a lot of memory to use the full toolset (more than 10GB)
 ```
    git clone https://github.com/Springjunky/docker-local-build-environment.git
    cd docker-local-build-environment
    sudo ./setupEnvironment.sh
-   docker-compose  -f docker-compose.yml -f docker-compose-sonar.yml up --build
-   docker-compose logs
-```
-
 
+   # Sonar only
+   docker-compose  -f docker-compose.yml -f docker-compose-sonar.yml up --build
 
+   # Sonar AND Keycloak
+   docker-compose  -f docker-compose.yml -f docker-compose-sonar.yml -f docker-compose-keycloak.yml up --build
 
+   # Keycloak only
+   docker-compose  -f docker-compose.yml -f docker-compose-keycloak.yml up --build
+   
+   docker-compose logs
+```
 ### The first startup takes a long time (especially gitlab), so be patient
-
 open your favorite browser (_not_ at localhost, use the $(hostname)/jenkins )
 to prevent jenkins spit out "your reverse proxy is wrong")
 
@@ -53,76 +77,38 @@ Now you are ready to go with a little CI/CD Environment:
  Nexus  http://<your-host-name>/nexus
  Gitlab  http://<your-host-name>/gitlab
  Sonar  http://<your-host-name>/sonar
+ Keycloak http://<your-host-name>/auth
+ Postgres: At standard listenport 5432 for your jdbc-connection-string 
+           stream-passthrough to postgres-container.
 ```
+
 #### Security
 ... not really, its all http .. don't worry about it! It's only local communication
 
 ##### security paranoia
-All the exposed ports are reachable from outer world because docker creates and deletes dynamically iptables FORWARD rules with default policy ACCEPT on startup/shutdown containers wich have exported ports.
-
-To deny acccess from outer world the DOCKER-USER Chain (since docker 17.06) ist the medium of choice for your own rules (this is the first target in the FORWARD-Chain and never touched by docker).
-
-A little Script to deny all access from outer world to your local build environment could be the following (exposed port from nginx are 80,5555,2222)
-```
-#!/bin/bash
-if [ $# -lt 1 ] ; then
-  echo "Need your external interface as one parameter"
-  echo "Common names are eth0, enp...,"
-  echo "List of your names"
-  ifconfig -a | sed 's/[ \t].*//;/^\(lo\|\)$/d'
-  exit
-fi
-
-PORTS_TO_BLOCK="80,5555,2222"
-EXTERNAL_INTERFACE=$1
-
-# Flush and delete custom Chains
-iptables -F DOCKER-USER
-iptables -F EXTERNAL-ACCESS-DENY
-iptables -X EXTERNAL-ACCESS-DENY
-
-# Create a  log-and-drop Chain
-iptables -N EXTERNAL-ACCESS-DENY
-iptables -A EXTERNAL-ACCESS-DENY -j LOG --log-prefix "DCKR-EXT-ACCESS-DENY:" --log-level 6
-iptables -A EXTERNAL-ACCESS-DENY -j DROP
-
-# Block all incomming traffic for docker
-iptables -A DOCKER-USER -i $EXTERNAL_INTERFACE \
-         -p tcp --match multiport \
-         --dports $PORTS_TO_BLOCK \
-         -j EXTERNAL-ACCESS-DENY
-
-# Restore default rule to return all the rest back to the FORWARD-Chain
-iptables -A DOCKER-USER -j RETURN
-
-echo "Rules created "
-iptables -v -L DOCKER-USER
-iptables -v -L EXTERNAL-ACCESS-DENY
-echo "See logs with prefix DCKR-EXT-ACCESS-DENY:"
-```
-
+See Readme in folder security-paranoia if you want to have some hints how to configure your firewall.
 
 ### Logins and Passwords
-
 |Image  |  User  |  Password |
 |---|---|---|
 |Jenkins| admin| admin |
 |Nexus   | admin | admin123 |
 |Gitlab  | root  | gitlab4me |
+|Sonar | admin | admin |
+|Keycloak|admin|admin|
+|Postgres|postgres|admin|
 
 ## The Tools
 ### Jenkins
-
 * MAVEN_HOME is /opt/maven
 * JAVA_HOME is /usr/lib/jvm/java-8-openjdk-amd64
 * Blue Ocean is installed if you choose (M)uch mor plugins and works perfect with a GitHUB Account, not GitLab ... sorry, this is Jenkins.
   You need to be logged as a jenkins-user to use Blue Ocean
 
 ###  Giltab
-
 * the docker-registry from GitLab is at port 5555 (and secured with an openssl certificate ..thats part of
-  prepareEnvironment.sh), just create a project in gitlab and click at the  registry tab to show
-  how to login to the project registry and how to tag your images
+  setupEnvironment.sh), just create a project in gitlab and click at the registry tab to show
+  how to login to the project registry and how to tag your images and upload them.
 * ssh cloning and pushing is at port 2222
 
 #### gitlab-runner
@@ -138,34 +124,54 @@ Gitlab is very very fast with new releases and sometimes the api has breaking ch
 ### Sonar
 You need to install some rules (Administration - System - Update Center - Available - Search: Java)
 
+### Keycloak
+There is a testproject in folder spring-boot-keycloak-sample, it is a standard Spring-Boot which you can start with
+```
+mv spring-boot:run
+``` 
+Use your browser and navigate to the "landing-page" at http://<your host>:8081 the "My products" link will redirect you to Keycloak (must be setup with settings from [this tutorial][3], but use your *REAL* hostname, not _localhost_ as Valid Redirect URI's )
+_tl;dr_
+* login as user:admin, password:admin
+* create realm "springboot"
+* create client "product-app" as openid-connect client with Valid Redirect URI's  http://&lt;your host&gt;:8081/*
+* create role "user"
+* create user "testuser" and map the role "user" to testuser (tab Role Mappings)
+ 
 ### Nexus
 Some ToDo for me described here
 [Unsecure docker-registry in Nexus][1]
 use GitLab as a secured registry
-
 ..
 And _yes_ docker-plugin in jenkins works (docker in docker, usefull but not recommended)
+### Postgres
+You can use any tool to connect to the database at locahost:5432 this is a pass through to the container so any
+JDBC-Connection should work
 
 ## Troubleshooting
-
-In most cases a wrong HOSTNAME:HOSTIP causes trouble, to check this try the follwing.
+##### check Hostname and IP
+In most cases a wrong HOSTNAME:HOSTIP causes trouble, to check this try the following.
 * log into the jenkins-fat container (with id)
 ```
   docker container ls
-  docker container exec -it dockerlocalbuildenvironment_jenkins-fat_1 bash
+  docker container exec -it dockerlocalbuildenvironment_jenkins_1 bash
   apt-get update
   apt-get install -y --allow-unauthenticated iputils-ping
   ping google.de
-  ping jenkins-fat
+  ping jenkins
   ping gitlab
   ping <your local hostname>
 ```
-every ping must work, if not, check extra_hosts in compose-file
+every ping must work, if not, check the .env file, is there the correct DC_HOSTNAME / DC_HOSTIP ?
+
+##### changed networks ?
+  If you change your network (switching between home/office/lan/wifi) your ip-address
+  could be change and the container is not able to resolve your host any more
+  Check the .env file or just run the setup-Script again. 
 
-* consider low memory:
-  with an amount lower than 8GB sonar and eleastic search did not startup
+##### consider low memory:
+  with an amount lower than 8GB sonar and embedded eleastic search did not startup and no message is displayed
 
-* too many plugins to download:
+##### too many plugins to download:
   You can do an "pre download of the plugins", see the readme.md at jenkins-fat direcory
 
 
@@ -178,9 +184,12 @@ every ping must work, if not, check extra_hosts in compose-file
 * ~~apply a gitlab runner~~
 * ~~apply git-lfs~~
 * ~~apply sonar~~
+* ~~apply keycloak~~
 * apply a better registry
 
 
 
 [1]: https://support.sonatype.com/hc/en-us/articles/217542177-Using-Self-Signed-Certificates-with-Nexus-Repository-Manager-and-Docker-Daemon
 [2]: https://hub.docker.com/r/gitlab/gitlab-runner/
+[3]: https://developers.redhat.com/blog/2017/05/25/easily-secure-your-spring-boot-applications-with-keycloak/
+[4]: https://xebialabs.com/periodic-table-of-devops-tools/

docker-compose-keycloak.yml

@@ -0,0 +1,24 @@
+
+# docker-keycloak configuration,
+# add this to your docker-compose up if you want to have SSO Functions
+# docker-compose -f docker-xcompose.yml -f docker-compose-keykloak.yml up
+version: "3"
+services:
+  keycloak:
+    image: jboss/keycloak
+    environment:
+      - KEYCLOAK_USER=admin
+      - KEYCLOAK_PASSWORD=admin
+      - POSTGRES_USER=keycloak 
+      - POSTGRES_PASSWORD=keycloak
+      - PROXY_ADDRESS_FORWARDING=true
+      - KEYCLOAK_LOGLEVEL=DEBUG
+    # Override command to enable docker protokoll  
+    command: ["-b", "0.0.0.0","-Dkeycloak.profile.feature.docker=enabled"]  
+    extra_hosts:
+      - ${DC_HOSTNAME}:${DC_HOSTIP}
+    networks:
+      - devstacknetwork
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+

docker-compose-sonar.yml

@@ -5,28 +5,16 @@
 version: "3"
 
 services:
-  postgres:
-    image: postgres
-    extra_hosts:
-      - ${DC_HOSTNAME}:${DC_HOSTIP}
-    networks:
-      - devstacknetwork
-    environment:
-      - POSTGRES_USER=sonar
-      - POSTGRES_PASSWORD=sonar
-    volumes:
-      - ${DC_BASE_DATA_DIR}/postgres-db/postgresql:/var/lib/postgresql
-      # This needs explicit mapping due to
-      # https://github.com/docker-library/postgres/blob/4e48e3228a30763913ece952c611e5e9b95c8759/Dockerfile.template#L52
-      - ${DC_BASE_DATA_DIR}/postgres-db/postgresql_data:/var/lib/postgresql/data
-
   sonar:
     image: sonarqube
+    depends_on: # start  proxy after all the others
+      - postgres
     extra_hosts:
       - ${DC_HOSTNAME}:${DC_HOSTIP}
     networks:
        - devstacknetwork
     environment:
+      # the Default User and Passwort for Postgress is sonar/sonar in the
       - SONARQUBE_JDBC_URL=jdbc:postgresql://postgres:5432/sonar
     volumes:
       - ${DC_BASE_DATA_DIR}/sonar/sonarqube_conf:/opt/sonarqube/conf

docker-compose.yml

@@ -16,18 +16,36 @@ services:
   ngnix:
      build: nginx-reverse
      ports:
-       - "80:80"       #http://
-       # SSH Bypassing into gitlab, if you want to change this edit nginx.conf also
-       - "2222:2222"   #ssh port of gitlab (ssh://git@myHOST:2222/scott/foo.git)
-       - "5555:5555"   #Gitlab Docker Registry do NOT use 5000, this is an internal PORT of the gitlab-ce Image
+       - "80:80"             #http://
+       - "5432:5432"       # Default-Port of the Postges DB passing jdbc-Connections to the postgres-Container 
+       - "2222:2222"     #ssh port of gitlab (ssh://git@myHOST:2222/scott/foo.git)
+       - "5555:5555"  #Gitlab Docker Registry do NOT use 5000, this is an internal PORT of the gitlab-ce Image
+     command: ["nginx-debug", "-g", "daemon off;"]  # Start nginx in debug to see whats going on    
      depends_on: # start  proxy after all the others
        - gitlab
        - jenkins
        - nexus
      networks:
        - devstacknetwork
-
 # ------------------------------------------------------------------------------
+  postgres:
+    build: postgres
+    extra_hosts:
+      - ${DC_HOSTNAME}:${DC_HOSTIP}
+    environment:
+      - POSTGRES_PASSWORD=admin
+      - POSTGRES_USER=postgres
+    networks:
+      - devstacknetwork
+    volumes:
+      - ${DC_BASE_DATA_DIR}/postgres-db/postgresql:/var/lib/postgresql
+      # This needs explicit mapping due to
+      # https://github.com/docker-library/postgres/blob/4e48e3228a30763913ece952c611e5e9b95c8759/Dockerfile.template#L52
+      # there is ENV PGDATA /var/lib/postgresql/data
+      - ${DC_BASE_DATA_DIR}/postgres-db/postgresql_data:/var/lib/postgresql/data
+    
+# ------------------------------------------------------------------------------
+
   jenkins:
     build: jenkins-fat
     extra_hosts:
@@ -93,3 +111,7 @@ services:
     volumes:
       - ${DC_BASE_DATA_DIR}/gitlab-runner/config:/etc/gitlab-runner
       - /var/run/docker.sock:/var/run/docker.sock
+
+      
+    
+      

nginx-reverse/Dockerfile

@@ -6,4 +6,5 @@ RUN rm /etc/nginx/nginx.conf
 COPY reverse-proxy.conf /etc/nginx/conf.d/reverse-proxy.conf
 COPY nginx.* /etc/nginx/ssl/
 COPY nginx.conf /etc/nginx/nginx.conf
-COPY proxy-settings.conf /etc/nginx/conf.d/proxy-settings.conf
+COPY proxy-settings.conf /etc/nginx/conf.d/proxy-settings.conf
+