Migrate Keycloak H2 database to Postgres on Kubernetes

Table of contents

Reading Time: 4 minutes

In this blog. We will discuss how we can migrate the Keycloak H2 database to Postgres on production or for the new server. Follow stepwise instructions to migrate the Keycloak H2 database to Postgres.

Introduction – Migrate Keycloak

Keycloak: is open source identity & access management software. It provides many services for authentication and security.
H2 database: H2 database in-memory java-based relational database for keycloak.
Postgresql: is open-source relational database management system.
Kubernetes: Orchestration for managing container based microservices.

Scope – Migrate Keycloak

We are migrating from the H2 database of an already deployed Keycloak server to the Postgres database for Keycloak to persist Keycloak data with no downtime & rolling update strategy. Our database is small so we will use migration provider singleFile which imports data into a single JSON file. We will require some basic knowledge of the above tools and corresponding CLIs must be installed.

Prerequisites

Keycloak: Deployed on kubernetes
Postgres knowledge

Let’s start migration from H2 to postgres

1. Export JSON form Keycloak server

1 Exec into master keycloak server

kubectl exec <-keycloak-pod-name> -it -- bash

2 Go to keycloak directory

cd /opt/jboss/keycloak

3 Run exporting script

bin/standalone.sh \
    -Dkeycloak.migration.action=export\
    -Djboss.socket.binding.port-offset=1\
    -Dkeycloak.migration.provider=singleFile -Dkeycloak.migration.file=data-keycloak.json

wait for a minute then the output will look like this

When this output appears wait a couple of seconds and stop the script by aborting signal ctrl+c. A JSON file named data-keycloak.json is created at /opt/jboss/keycloak or at the present working directory.

4 Copy the JSON file to the local system

kubectl cp <namespace>/keycloak-0:opt/bitnami/data-keycloak.json data-keycloak.json

Now, the JSON file is exported successfully & stored in your local system.

2. Create configMap

Now we will create configMap to insert the JSON data into our keycloak pod

1 Make compact of the exported JSON file

For this, you will require jq tool

cat <path-to-json>/data-keycloak.json |jq -c >> keycloak.json

2 Create configMap of JSON

Paste the compact JSON (keycloak.json) in the data section of configMap

configmap.yml

apiVersion: v1
kind: ConfigMap
metadata:
  name: keycloak-data
data:
  keycloak.json: |
    {"id":"c74c7c08-dfed-4028-8307-7d3888bcb7ac","name":"query-groups","description":"${role_query-groups}","composite":false,"clientRole":true,"containerId":"c9a05bc2-3e0f-4e74-971e-73d93e0fcb45","attributes":{}},{"id":"c7040aad-76a6-422d-819e-65eab9803755","name":"manage-clients","description":"${role_manage-clients}","composite":false,"clientRole":true,"containerId":"c9a05bc2-3e0f-4e74-971e-73d93e0fcb45","attributes":{}},{"id":"0d147ea7-a449-4330-9ce5-e1ae67df267e","name":"query-clients","description":"${role_query-clients}","composite":false,"clientRole":true,"containerId":"c9a05bc2-3e0f-4e74-971e-73d93e0fcb45","attributes":{}},{"id":"a8f67dc9-2c16-4523-978c-06b4341fd385","name":"view-events","description":"${role_view-events}","composite":false,"clientRole":true,"containerId":"c9a05bc2-3e0f-4e74-971e-73d93e0fcb45","attribut

3 Deploy configMap

kubectl create -f configmap.yml

3. Deploy Postgres Database

1 Create deployment file for Postgres

Environment Variables

Key	Value
POSTGRES_PASSWORD	secret
PGDATA	/data/pgdata
POSTGRES_USER	keycloak
POSTGRES_DB	keycloak

postrgres-sfs.yml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgresql-db
spec:
  serviceName: postgresql-db-service
  selector:
    matchLabels:
      app: postgresql-db
  replicas: 1
  template:
    metadata:
      labels:
        app: postgresql-db
    spec:
      containers:
      - name: postgresql-db
        image: postgres:latest
        volumeMounts:
        - name: postgresql-db-disk
          mountPath: /data
        env:
        - name: POSTGRES_PASSWORD
          value: password
        - name: PGDATA
          value: /data/pgdata
        - name: POSTGRES_USER
          value: keycloak
        - name: POSTGRES_DB
          value: keycloak
# Volume Claim
  volumeClaimTemplates:
  - metadata:
      name: postgresql-db-disk
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
         storage: 1Gi
---
# PostgreSQL StatefulSet Service
apiVersion: v1
kind: Service
metadata:
  name: postgres-db-lb
spec:
  selector:
    app: postgresql-db
  type: LoadBalancer
  ports:
  - port: 5432
    targetPort: 5432

2 Deploy Postgres on Kubernetes

kubectl apply -f postgres-sfs.yml

4. Create a Job

1 Create a YAML file for the job to import data

importkeycloak.yml

apiVersion: batch/v1
kind: Job
metadata:
  name: migrate
spec:
  ttlSecondsAfterFinished: 100
  template:
    spec:
      containers:
      - name: keycloak-migrate
        image: quay.io/keycloak/keycloak:16.1.1
        volumeMounts:
          - mountPath: "/app/data"
            name: data-kc
            readOnly: true
        env:
        - name: KEYCLOAK_USER
          value: "secret-user"
        - name: KEYCLOAK_PASSWORD
          value: "secret-password"
        - name: DB_VENDOR
          value: "postgres"
        - name: DB_ADDR
          value: "postgres-db-lb"
        - name: DB_DATABASE
          value: "keycloak"
        - name: DB_USER
          value: "keycloak"
        - name: DB_SCHEMA
          value: "public"
        - name: DB_PASSWORD
          value: db-password
        ports:
        - name: http
          containerPort: 8080
        - name: https
          containerPort: 8443
        command:
          - "/opt/jboss/tools/docker-entrypoint.sh"
        args:
          - "-Dkeycloak.migration.action=import"
          - "-Dkeycloak.migration.strategy=OVERWRITE_EXISTING"
          - "-Djboss.socket.binding.port-offset=0"
          - "-Dkeycloak.migration.provider=singleFile"
          - "-Dkeycloak.migration.file=/app/data/keycloak.json"
          - "-b"
          - "0.0.0.0"
      restartPolicy: Never
      volumes:
        - name: data-kc
          configMap:
            name: keycloak-data
            items:
              - key: "keycloak.json"
                path: "keycloak.json"

2 Run the job

kubectl apply -f importkeycloak.yml

3 Check logs of job

kubectl logs job/migrate

This means importing is done. Now you can suspend the job.

4 Suspend the job

kubectl edit job/migrate

Change suspend: false to suspend: true

5. Deploy Keycloak

1 Update existing YAML or create a new deployment for Keycloak

keycloak-deploy.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: keycloak
  namespace: default
  labels:
    app: keycloak
spec:
  replicas: 1
  selector:
    matchLabels:
      app: keycloak
  template:
    metadata:
      labels:
        app: keycloak
    spec:
      containers:
      - name: keycloak
        resources: {}
        image: quay.io/keycloak/keycloak:16.1.1
        env:
        - name: KEYCLOAK_USER
          value: "admin"
        - name: KEYCLOAK_PASSWORD
          value: "admin"
        - name: DB_VENDOR
          value: "postgres"
        - name: DB_ADDR
          value: "postgres-db-lb"
        - name: DB_DATABASE
          value: "keycloak"
        - name: DB_USER
          value: "keycloak"
        - name: DB_SCHEMA
          value: "public"
        - name: DB_PASSWORD
          value: password
        ports:
        - name: http
          containerPort: 8080
        - name: https
          containerPort: 8443
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keycloak
spec:
  tls:
    - hosts:
      - keycloak.192.168.49.2.nip.io
  rules:
  - host: keycloak.192.168.49.2.nip.io
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: keycloak
            port:
              number: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: keycloak
  labels:
    app: keycloak
spec:
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  selector:
    app: keycloak
  type: LoadBalancer

2 Roll out the deployment or create Keycloak deployment