Common deployment to Kubernetes
Set Docker environment variables
First, create an account on Dockerhub.
Then export the following environment variables:
export DOCKER_REGISTRY=docker.ioexport DOCKERHUB_USERNAME=******export DOCKERHUB_PASSWORD=******
Replacing them with their corresponding values
Install Helm
- Install helm v3.3.4
https://github.com/helm/helm/releases/tag/v3.3.4
For linux:
1. Download https://get.helm.sh/helm-v3.3.4-linux-amd64.tar.gz2. Unpack it (tar -zxvf helm-v3.3.4-linux-amd64.tar.gz)3. Find the helm binary in the unpacked directory, and move it to its desired destination (mv linux-amd64/helm /usr/local/bin/helm)
Install Terraform
- Install terraform 0.12.21
https://releases.hashicorp.com/terraform/0.12.21/
For Linux:
wget https://releases.hashicorp.com/terraform/0.12.21/terraform_0.12.21_linux_amd64.zipunzip terraform_0.12.21_linux_amd64.zipsudo mv terraform /usr/local/bin/terraform
Kubernetes version
We have tested our terraform and helm scripts with Kubernetes 1.25. Please use this version. No other versions are supported at this time.
Prerequisites
Vault
We use Vault in order to securely store secrets.
Our applications use certificate based authentication in order to securely talk to Vault.
Our images will generate a self-signed certificate to be used for vault. However, if you would like to use
your own certificate, you can do the following:
- Get a public/private keypair from a valid CA Authority, and put the files in:
- .vault/vault.key - the private key
- .vault/vault.crt - the public key
- Then, run the following command.
. env.shopenssl pkcs12 -export -in .vault/vault.crt -inkey .vault/vault.key -out .vault/vault.jks -password pass:${VAULT_KEYPASS}
If you are using Terraform, make sure that ${VAULT_KEYPASS} above matches
vault.vaultKeyPassword
interraform.tfvars
Also, put all of the resulting files in
openiam-configmap/.vault/
i.e.
cp .vault/vault.key openiam-configmap/.vault/cp .vault/vault.crt openiam-configmap/.vault/cp .vault/vault.jks openiam-configmap/.vault/
SMTP server
You will need to set up an SMTP server. If running in AWS, you use SES, which is very simple to set up. If you do not have a corporate SMTP server, there are numerous SMTP Cloud Servers which you can use. Setting up SMTP is outside the scope of this document.
RabbitMQ TLS
You can optionally run RabbitMQ with TLS enabled.
Adding your own TLS certificates to RabbitMQ
If you would like to use your own Certificates with RabbitMQ, you will need to get a public/private keypair from a valid CA authority, and generate a JKS file. Please follow the instructions in the RabbitMQ TLS file.
Generating a self-signed certificate
You can also generate a self-signed certificate by running
./generate.rabbitmq.certs.sh
HTTPS certificates
When running in kubernetes, we expose port 80, and 443 if https is enabled. Our apache httpd server listens to these ports. To set up https, see our SSL document for a list of required files.
Set required values in env.sh
Set the APP_NAME variable in env.sh to a unique string. This is a unique identifier, and will not be seen by end users.
If you're running in AWS, set the
OSS
variable inenv.sh
to-oss
Configure Extra VHost and Apache Configs
You can optionally add 'extra' vhost and apache configs. To do that, simply modify the following files as needed:
- .apache/extraVHost.conf
- .apache/extraApache.conf
These files shall be put in /usr/local/apache2/conf/add, in the rproxy pod
Initialize and Setup
Run the setup script
./setup.sh
RabbitMQ
Exposing externally
To expose the RabbitMQ Service externally, uncomment the following lines in rabbitmq.values.yaml
:
# Uncomment to expose externally# service:# type: LoadBalancer# clusterIP: None
Note that you will have to re-run setup.sh
after doing this.
Deploying with Terraform
Deploying Kubernetes via Terraform is a recommended approach. The guide to do this is here.
Deploying without Terraform
You can deploy without using Terraform, but this method is much more complex, requires much more configuration, and is not recommended. In case deployment without Terraform is required, you can find the respective guide here.
Confirming successful deployment
Confirm that all pods are up and running with the following command:
kubectl get pods
Ensure that the READY column does not have any failed pods. For example:
Example of running pod
test100-esb-0 1/1 Running 0 2m3s
Example of failed pod
test100-esb-0 0/1 CrashLoopBackOff 4 2m3s
Debugging failed pods
If a certain pod fails, gather it's logs for analysis.
kubectl logs pods/<name_of_failed_pod>
Accessing your deployed instance
To access your deployed instance of OpenIAM, run the following command:
. env.shkubectl get "service/${APP_NAME}-rproxy"
The output of the above command will contain an EXTERNAL IP column, for example:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEtest100-rproxy LoadBalancer 172.20.27.78 a0375c89dd2ec11e98bca0648c64953f-439827441.us-west-2.elb.amazonaws.com 80:32468/TCP 3m57s
Curl the above URL:
curl -L "http://${EXTERNAL_IP_FROM_ABOVE}/webconsole"
You may want to add a CNAME alias for the above URL, to make it more readable.
Switching Between Environments/Clients.
We provide a convenience shell script, which saves off terraform files to a specific location, or copies terraform files from that location to this project.
Example on Saving files
This will save the terraform files from this project into /tmp/client_name/prod
./terraform.client.sh save /tmp client_name prod
Example on Using files
This will use the terraform files from /tmp/client_name/prod
./terraform.client.sh use /tmp client_name prod