Cloud, CMS, DB

AEM CLUSTERING WITH MONGO DB AND SHARED S3 DATA STORE

Overview :

This article talks about the setting up of AEM 6.2 with Mongo DB and AWS S3.

AEM clustering can be achieved by running each of the instances (which are expected to be in sync) on Mongo DB. In this scenario,  the shared data store is on S3 and the document node store is on Mongo DB.

Refer to the below article for setting up a Mongo DB with replica sets :

SETTING UP A MONGO DB REPLICA SET

Installation steps :

  • First create a directory ‘aem’ under /opt and under that create two directories source_files and author or publish(depending on whether you are installing an author or publish with mongo and S3), one to have all the installation files and the other will have an AEM author/publish in it.
  • Install Java – JDK from the “/opt/aem/source_files” directory

#rpm -ivh jdk-8u121-linux-x64.rpm

# java -version

java version “1.8.0_121”

Java(TM) SE Runtime Environment (build 1.8.0_121-b13)

Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

  • Copy the AEM jar and licence.properties to “/opt/aem/author_master” from “/opt/aem/source_files”
  • In our case, we are installing an author hence we create a folder author_master under /opt/aem.

#cd /opt/aem/author_master

#cp /opt/aem/source_files/cq-quickstart-6.2.0.jar .                (assuming the cq jar file has been placed in /opt/aem/source_files)

#cp /opt/aem/source_files/licence.properties .                             (assuming the licence.properties file has been placed in /opt/aem/source_files)

  • Extract the jar file using java -jar cq-quickstart-6.2.0.jar –unpack , where cq-quickstart-6.2.0.jar is AEM binary.
  • On extraction it will create a folder crx-quickstart under /opt/aem/author_master/
  • Change the port number(CQ_PORT) and Run mode(CQ_RUNMODE) in the /opt/aem/author_master/crx-quickstart/bin/start as below

if [ -z “$CQ_PORT” ]; then

        CQ_PORT=4502 {Port No.}

fi 

# runmode(s)

if [ -z “$CQ_RUNMODE” ]; then

        CQ_RUNMODE=’author,dev,nosamplecontent’

Fi 

  • Include the JVM Parameters as below

 

CQ_JVM_OPTS=”-server -Xmx46080m -Xms20480m -XX:MaxPermSize=10240m -XX:PermSize=2048m -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:ParallelGCThreads=8 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/aem/author/heapdump_log/\`date\`.hprof -Djava.io.tmpdir=/opt/aem/author/tmp -Doak.queryLimitInMemory=500000 -Doak.queryLimitReads=100000 -Dupdate.limit=250000 -Doak.fastQuerySize=true -Djava.awt.headless=true”

CQ_JVM_OPTS=”-Djava.rmi.server.hostname=xx.xx.xx.xx -Dcom.sun.management.jmxremote.port=8082 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false ${CQ_JVM_OPTS}”

CQ_JVM_OPTS=”-Xdebug -Xrunjdwp:transport=dt_socket,address=8081,server=y,suspend=n ${CQ_JVM_OPTS}”

 

The above mentioned JVM OPTs also have the parameters that are needed for Jmx monitoring. 

NOTE:   Xmx and MaxPermSize depends on physical memory 

  • In the persistence mode uncomment mongo related line and comment the tar one. 

# ——————————————————————————

# persistence mode

# ——————————————————————————

# the persistence mode can not be switched for an existing repository

#CQ_RUNMODE=”${CQ_RUNMODE},crx3,crx3tar”

CQ_RUNMODE=”${CQ_RUNMODE},crx3,crx3mongo”

# wget https://repo.adobe.com/nexus/content/groups/public/com/adobe/granite/com.adobe.granite.oak.s3connector/1.4.8/com.adobe.granite.oak.s3connector-1.4.8.zip

  • Unzip the downloaded file.
  • Copy the “install” folder from “/com.adobe.granite.oak.s3connector-1.4.8/jcr_root/libs/system/install” to “/opt/aem/author_master/crx-quickstart”
  • Create a file “org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService.cfg” under “/opt/aem/author_master/crx-quickstart/install” and copy the below mentioned fields and save the file.

 

#Mongo server details

mongouri=mongodb://xx.xx.xx.xx:27017,xx.xx.xx.xx:27017,xx.xx.xx.xx:27017/?replicaSet=rs&readPreference=nearest&w=1&j=1

#Name of Mongo database to use

db=sample_name

#Store binaries in custom BlobStore

customBlobStore=true

#This is distributed among various caches used in DocumentNodeStore

cache=512

#capped collection used in Mongo for caching the diff output.

changesSize=512 

Note : The above config file is made for mongo with replica sets. In your case if you do not have a replica set and you have only a single mongo then give the IP of the single mongo in the mongouri field, while all the other fields remain the same. 

Eg : mongouri=mongodb://xx.xx.xx.xx:27017

  • Create a file “org.apache.jackrabbit.oak.plugins.blob.datastore.SharedS3DataStore.config” under “/opt/aem/author_master/crx-quickstart/install” and copy the below mentioned fields and save the file.

# vi org.apache.jackrabbit.oak.plugins.blob.datastore.SharedS3DataStore.config

accessKey=”XXXXXXXXXXXXXXXXXXXX”

connectionTimeout=”120000″

maxConnections=”40″

maxErrorRetry=”10″

s3Bucket=”sample_name”

s3Region=”us-standard”

secretKey=”XXXXXXXXXXXXXXXXXXXX”

socketTimeout=”120000″

writeThreads=”30″

concurrentUploadsThreads=”30″

asyncUploadLimit=”30″

maxCachedBinarySize=”17408″

minRecordLength=”16384″

cachePurgeTrigFactor=”0.95d”

cacheSize=”0″

concurrentUploadsThreads=”10″

path=”/opt/aem/author_master/crx-quickstart/repository/datastore”

uploadRetries=”3″

  • To startthis instance, go to /opt/aem/author_master/crx-quickstart/bin/ and run the below command,

# ./start

Verification :

  • To confirm if the AEM instance has started with S3 or not :

Check the /opt/aem/author/crx-quickstart/repository path after starting the instance, it will have the datastore and s3.init.done files in it.

Also, the S3 bucket in the AWS S3 console will start to have some content in it.

  • To confirm if the AEM instance has started with Mongo or not :

Check the mongo db logs in the primary mongo DB and the replica sets, it will say that the connection has been established with that particular AEM server (shows the IP of the AEM server).

Also, a database for the AEM instance in the data/db folder in mongo with the same name that you have given in the “org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService.cfg” file.

Note : Since our requirement is to have AEM author/publish with both mongo as well as S3, both the above verification steps should be positive.

About The Author

Leave a Reply

*