Kubernetes
Docker
All the microservices are exposed as Docker images and can be accessed from Nexus repository or Docker Hub.
Nexus Repository
Nexus Repository is an internally hosted private repository and can be accessed using the URL 172.19.32.90:8081.
A credential with read-only access will be created for the implementation teams to access the repository. The user can pull the image from the repository by specifying the user ID, password, image name and revision number.
Example, 172.19.32.90:8123/$ctmetadata:$9890
ctmetadata    -             Image name
9890Â Â Â Â Â Â Â Â Â -Â Â Â Â Â Â Â Revision number
The Nexus URL should be added to the demon.json file in /etc/docker folder to connect to the repository.
The implementation teams can refer to the highlighted section for the commands to download the Docker image from Nexus repository by specifying the image name and revision number.
Docker Hub
Docker Hub is an external repository accessed over internet. If the Nexus URL is not specified, the Docker connects to Docker Hub by default to push and pull images. Therefore, to connect to Docker Hub, the user ID, password, image name and revision number can be specified directly without any URL.
The following screen shot contains the commands to execute the push and pull operation of Docker images to Docker Hub.
In order to externalize the configuration and use the same Docker image across environments, CONFD is used to manage the application configuration files using a standard template and fetch the required configuration values from Redis during runtime. When the images are being loaded, CONFD will pull necessary values from the Redis instance and set them as environment variables within that container.
Update the details of the IPs of the Database, Kafka, Implementation Bundle, Redis, Log stash etc., in setvalues.sh.
Refer the following file for sample configuration where the Ips of the Database, Kafka, Implementation Bundle, Redis, Log stash is specified.
setvalues.shRun the setvalues.sh file to load the configuration to a Redis instance.
redis-cli -h 172.19.32.90 set /CTMETADATA_SERVICE_URL ddm redis-cli -h 172.19.32.90 set /CTMETADATA_NODE_SERVICE_URL sdk redis-cli -h 172.19.32.90 set /CT_DATACACHE_SERVICE_URL idk redis-cli -h 172.19.32.90 set /SERVICE_EXECUTION Y redis-cli -h 172.19.32.90 set /NODEJS_SERVICE_EXECUTION Y redis-cli -h 172.19.32.90 set /DATACACHE_SERVICE_EXECUTION Y redis-cli -h 172.19.32.90 set /DB_VENDOR mysql redis-cli -h 172.19.32.90 set /REDIS_CACHE_URL 172.19.32.91 redis-cli -h 172.19.32.90 set /REDIS_CACHE_PORT 6379 redis-cli -h 172.19.32.90 set /CT_MONGODB_HOST 172.19.32.90 redis-cli -h 172.19.32.90 set /CT_MONGODB_PORT 27017 redis-cli -h 172.19.32.90 set /KAFKA_URL_PORT 172.19.32.90:9092 redis-cli -h 172.19.32.90 set /IMPLEMENTATION_BUNDLE MyImplementation redis-cli -h 172.19.32.90 set /IMPLEMENTATION_DB_BUNDLE databaseconfig redis-cli -h 172.19.32.90 set /IMPL_COMP_PREF_BUNDLE componentspreferences redis-cli -h 172.19.32.90 set /DB_URL_MODEL jdbc:mysql://172.19.32.90:8306/model?autoReconnect=true redis-cli -h 172.19.32.90 set /DB_URL_METADATA jdbc:mysql://172.19.32.90:8306/model?autoReconnect=true redis-cli -h 172.19.32.90 set /DB_URL_SESSION jdbc:mysql://172.19.32.90:8306/model?autoReconnect=true redis-cli -h 172.19.32.90 set /DB_URL_AUTHORIZATION jdbc:mysql://172.19.32.90:8306/model?autoReconnect=true redis-cli -h 172.19.32.90 set /DB_URL_DATACACHE jdbc:mysql://172.19.32.90:8306/model?autoReconnect=true redis-cli -h 172.19.32.90 set /DB_URL_STUDIO jdbc:mysql://172.19.32.90:8306/studio?autoReconnect=true redis-cli -h 172.19.32.90 set /logstash#!/bin/sh CONFIGURATON_SERVER="172.19.32.90:6379" REV="9438" MONGO_DB_HOST="172.19.32.90" MONGO_DB_PORT="27017" EXTERNAL_IP=172.19.32.86 #Create a secret for docker registry details and login into the nexus repository #while deploying the image before that put a entry in daemon.json in /etc/docker kubectl create secret docker-registry registry-details --docker-server=172.19.32.90:8123 --docker-username=canvas --docker-password=canvas123 #This will replace configuration server,revision,mongodb_host,mongodb_port,external_ip in all respective yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctmetadata.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctsessionservice.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctauthorizationservice.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctdatacache.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./expertstudio.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./modelhouse.yaml sed -i "s/mongodb_host/${MONGO_DB_HOST}/g; s/mongodb_port/${MONGO_DB_PORT}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctmetanodegetservice.yaml sed -i "s/mongodb_host/${MONGO_DB_HOST}/g; s/mongodb_port/${MONGO_DB_PORT}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctmetanodeloadservice.yaml #Deployment of all microservices one by onein kubernetes kubectl create -f ./ctmetanodegetservice.yaml kubectl create -f ./ctmetanodeloadservice.yaml kubectl create -f ./ctmetadata.yaml kubectl create -f ./ctsessionservice.yaml kubectl create -f ./ctauthorizationservice.yaml kubectl create -f ./ctdatacache.yaml kubectl create -f ./expertstudio.yaml kubectl create -f ./modelhouse.yaml #After deploying the docker images in kubernetes this will revert to same variable in yaml #which is replaced by an value(configuration server,rev,external_ip,mongodb_host,port) given in the shell script sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctmetadata.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctsessionservice.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctauthorizationservice.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctdatacache.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./expertstudio.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./modelhouse.yaml sed -i "s/${MONGO_DB_HOST}/mongodb_host/g; s/${MONGO_DB_PORT}/mongodb_port/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctmetanodegetservice.yaml sed -i "s/${MONGO_DB_HOST}/mongodb_host/g; s/${MONGO_DB_PORT}/mongodb_port/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctmetanodeloadservice.yamlurl 172.19.32.92:5044
Configure the deployment file to point to the Redis, specify the revision ID of the source build, and provide the host and port details of the Mongo DB in kube-deploy.sh.
#!/bin/sh CONFIGURATON_SERVER="172.19.32.90:6379" REV="9438" MONGO_DB_HOST="172.19.32.90" MONGO_DB_PORT="27017" EXTERNAL_IP=172.19.32.86
Refer the following file for the sample configuration of the Kubernetes deployment.
kube-deploy.shRun the kube-deploy.sh file to pull the images from the Nexus repository and deploy all the microservices one by one in Kubernetes. The following snippet will replace the configuration details during runtime.
#Create a secret for docker registry details and login into the nexus repository #while deploying the image before that put a entry in daemon.json in /etc/docker kubectl create secret docker-registry registry-details --docker-server=172.19.32.90:8123 --docker-username=canvas --docker-password=canvas123 #This will replace configuration server,revision,mongodb_host,mongodb_port,external_ip in all respective yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctmetadata.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctsessionservice.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctauthorizationservice.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctdatacache.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./expertstudio.yaml sed -i "s/config_server/${CONFIGURATON_SERVER}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./modelhouse.yaml sed -i "s/mongodb_host/${MONGO_DB_HOST}/g; s/mongodb_port/${MONGO_DB_PORT}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctmetanodegetservice.yaml sed -i "s/mongodb_host/${MONGO_DB_HOST}/g; s/mongodb_port/${MONGO_DB_PORT}/g; s/rev/${REV}/g; s/userip/${EXTERNAL_IP}/g" ./ctmetanodeloadservice.yaml
The following snippet deploys all microservice modules.
#Deployment of all microservices one by onein kubernetes kubectl create -f ./ctmetanodegetservice.yaml kubectl create -f ./ctmetanodeloadservice.yaml kubectl create -f ./ctmetadata.yaml kubectl create -f ./ctsessionservice.yaml kubectl create -f ./ctauthorizationservice.yaml kubectl create -f ./ctdatacache.yaml kubectl create -f ./expertstudio.yaml kubectl create -f ./modelhouse.yaml
The following snippet is where the values get changed in runtime after deployment.
#After deploying the docker images in kubernetes this will revert to same variable in yaml #which is replaced by an value(configuration server,rev,external_ip,mongodb_host,port) given in the shell script sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctmetadata.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctsessionservice.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctauthorizationservice.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctdatacache.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./expertstudio.yaml sed -i "s/${CONFIGURATON_SERVER}/config_server/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./modelhouse.yaml sed -i "s/${MONGO_DB_HOST}/mongodb_host/g; s/${MONGO_DB_PORT}/mongodb_port/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctmetanodegetservice.yaml sed -i "s/${MONGO_DB_HOST}/mongodb_host/g; s/${MONGO_DB_PORT}/mongodb_port/g; s/${REV}/rev/g; s/${EXTERNAL_IP}/userip/g" ./ctmetanodeloadservice.yaml