AEM 6.5 Upgrade
CMS, Digital Marketing

How To Upgrade AEM 6.5 Safely!

Adobe has launched new version of Adobe Experience Manager 6.5 on April 8, 2019. This release went through 23 iterations of quality assurance and bug fixing , which is included with 1345 defect fixes , enhancements & exciting new features.

Adobe Experience Manager 6.5 delivers innovations that enable marketers & IT professionals to collaborate on the rapid delivery of personalized brand experiences—exactly when & where they’re expected.

Is it worth upgrading to AEM 6.5?  Please check out our what’s new in AEM 6.5 blog to understand new features of AEM 6.5 as well key structural changes  introduced in AEM 6.4 before any upgrade process.

Upgrade path to 6.5 (from different versions)

Upgrade process depends on the source version from where you are trying upgrade and the destination version to which you are upgrading to. Below are key insights to be noticed & act upon while upgrading from different source versions.

  • Upgrade from CQ 5.6 to AEM 6.5
    • AEM 6.0 introduced the new Jackrabbit Oak repository. Persistence Managers were replaced by Micro Kernels . Starting from version 6.1, CRX2 is no longer supported.
    • A migration tool called crx2oak needs to be run to migrate CRX2 repositories from 5.6 instances
    • Customers running 5.6.x and below need to upgrade first to version 6.0 or above, with 6.0(SP3) being recommended.
  • Upgrading from AEM 6.x to AEM 6.5
    • Direct in-place upgrade to AEM 6.5 is supported for customers running AEM 6.2, 6.3 and 6.4.
    • AEM 6.3 introduced a new format for the SegmentNodeStore , which is the basis of the TarMK implementation. If you are upgrading from a version older than AEM 6.3, this will require a repository migration as part of the upgrade, involving system downtime.
    • If you are upgrading from AEM 6.2 to 6.3, you should EITHER upgrade from versions (6.2-SP1-CFP1 – -6.2SP1-CFP12.1) or 6.2SP1-CFP15 onwards. Otherwise, if you are upgrading from 6.2SP1-CFP13/6.2SP1CFP14 to AEM 6.3, you must also upgrade to at least version 6.3.2.2. Otherwise, AEM Sites would fail after upgrading.

 

Which is preferred? In-place upgrade or Fresh install ?

Adobe recommends In-place upgrade. You would need to take decision depending up on your application complexity , efforts & duration required in your case. Below are Pros and Cons of each case :

  • In-place Upgrade :
  • Fresh install :

Upgrade Milestones

Below are key milestones involved with the upgrade process. The following outline has been provided as an overview of what needs to be included in an upgrade project:

 

Planning your upgrade

Proper planning is required to upgrade any of the previous AEM versions to AEM 6.5. Planning needs to be included with all the milestones mentioned above for successful migration.

It is  important to ensure that you are running a supported operating system, Java runtime (jdk11), httpd and Dispatcher ( 4.3.2 or higher)version. For more information, see the AEM 6.5 Technical Requirements page.

While Upgrading , below components will need to be accounted for in your project plan and should take place before upgrading AEM. These are key areas which would impact upgrade process. Please see details below :

Testing Aspects / Considerations

A comprehensive test plan should be prepared for testing upgrades , which is one of key factor for successful upgrade.

  • Below are critical areas of any AEM implementation that should be covered by your test plan , once the environment has been upgraded and the upgraded code base has been deployed to dedicated test environment.

Pre-upgrade checklist

Before beginning your upgrade, it is important to follow below maintenance check list to ensure that the system is ready for upgrade:

  • Ensure Sufficient Disk Space  – Upgrade will create a copy of the repository in new Segment Tar format copy. You will need enough disk space to retain a second, potentially larger, version of your repository.
  • Fully Back Up AEM (Repository, Application installation, Datastore, and Mongo instances if applicable)
  • Back Up Changes to /etc (upgrade will make a backup copy of any changes that it cannot merge under /var, we recommend backing up these changes manually before beginning the upgrade)
  • Generate the quickstart.properties file. If AEM has only been started with the start script in the past, this file will not be present and the upgrade will fail. Make sure to check for the existence of this file and restart AEM from the jar file if it is not present.
  • Configure Workflow and Audit Log Purging (These may fail during pre-upgrade task execution ,if configs are missing)
  • Install, Configure, and Run The Pre-Upgrade Tasks (introduced in 6.3)

  • Disable Custom Login Modules (Repository.xml incase of 5.x version. Apache Felix JAAS Configuration Factory service  from 6.x versions)
  • Remove updates (any service packs, feature packs or hot fixes ) from the /install directory
  • Stop any Cold Standby Instances (Rebuild standby from primary after successful upgrade)
  • Disable Custom Scheduled Jobs
  • Execute Offline Revision Cleanup – This will make the repository migration step and subsequent upgrade tasks execute much faster and will help to ensure that Online Revision Cleanup can execute successfully after the upgrade has completed.
  • Execute Datastore Garbage Collection (to remove any unreferenced blobs in the data store)
  • Rotate Log Files

Upgrade Process : Code & Customization’s

Below are major milestones involved in the upgrade implementation process. It’s really important to follow the process for successful release. Please see detailed information about pattern detector in below section.

  1. Pattern Detector :  Assess the Upgrade Complexity with Pattern Detector

    The pattern detector is the easiest way to assess the overall complexity of the upgrade to be expected using reported patterns.

    The Pattern Detector is released separately as a one package working on any source AEM versions from 6.1 to 6.5 targeting AEM 6.5 upgrade. It can be installed using the Package Manager.  Pattern Detector allows to check:

    • OSGi bundles exports and imports mismatch
    • Sling resource types and super types (with search-path content overlays) over usages
    • definitions of Oak indices (compatibility)
    • VLT packages (overuse)
    • rep:User nodes compatibility (in context of OAuth configuration)

    The list of Deprecated and Removed Features should be reviewed not only for the version that you are upgrading to, but also for any versions between your source and target versions. For example, if upgrading from AEM 6.2 to 6.5, it is important to review the AEM 6.3 deprecated and removed features in addition to those for AEM 6.5.

    Pattern Detector Issue categories

    Below are few categories of issues extracted from pattern detector results to demonstrate the type of issues it can detect and possible solutions for the same.

     

  2. Develop Code Base for 6.5
    • Create a dedicated branch or repository for the code base for the Target version. Use info from Pre-Upgrade Compatibility to plan areas of code to update.
  3. Update and Compile with 6.5 Uber jar
    • Update code base POMs to point to 6.5 uber jar and compile code against this.
  4. Update AEM Customizations , connectors
    • Any customizations or extensions to AEM should be updated/validated to work in 6.5 and added to the 6.5 code base. Includes UI Search Forms, Assets Customizations, anything using /mnt/overlay
    • Refer to Key Recommendations / Points to be considered while upgrade section for the tasks to be consider during this upgrade
  5. Deploy to 6.5 Environment
    • A clean instance of AEM 6.5 (Author + Publish) should be stood up in a Dev/QA environment. Updated code base and a representative sample of content (from current production) should be deployed.
  6. QA Validation and Bug fix
    • QA should validate the application on both Author and Publish instances of 6.5. Any bugs found should be fixed and committed to the 6.5 code base. Repeat Dev-Cycle as necessary until all bugs are fixed.

Upgrade Process : TarMK Publish Farm scenario

Below is the high level procedure for upgrading an AEM topology currently running on a version of AEM 6.x. Assuming below is existing AEM infra set up , for sample :

Upgrade execution sequence:

  1. Stop traffic to the Publish 2 instance at the load balancer
  2. Run pre-upgrade maintenance on Publish 2
  3. Run an in-place upgrade on Publish 2
  4. Update the Dispatcher or Web Module if needed
  5. Flush the Dispatcher cache
  6. QA validates Publish 2 through the Dispatcher, behind the firewall
  7. Shutdown Publish 2
  8. Copy the Publish 2 instance
  9. Start Publish 2

In case of successful scenario , please follow same process for the Publish 1 instance after enabling traffic to Publish 2 instance

In case of any failure scenario , follow below process to rollback  to the earlier state.

  • Create a copy of Publish 1
  • Replace the Publish 2 instance with a copy of Publish 1
  • Flush the Dispatcher cache for Publish 2
  • Start Publish 2
  • QA validates Publish 2 through the Dispatcher, behind the firewall
  • Enable traffic to Publish 2

Post upgrade checklist

The following activities need to be executed to finalize the upgrade status:

  • Verify logs for upgrade success (upgrade.log / error.log)
    • If the upgrade is successful , you would see below message in upgrade.log

  • Verify OSGi Bundles
    • If any bundles are in an installed state , check the error.log to determine root issue.
  • Verify Oak Version (Oak version should be 1.10.2  with AEM 6.5 upgrade)
  • Inspect the PreUpgradeBackup folder
    • During the upgrade AEM will attempt to backup customizations and store them beneath /var/upgrade/PreUpgradeBackup/<time-stamp-of-upgrade>
    • The folder with the time stamp should have a property named mergeStatus with a value of COMPLETED. The to-process folder should be empty and the overwritten node indicates which nodes were overwritten during the upgrade. Content beneath the leftovers node indicate content that could not be safely merged during the upgrade. If your implementation is dependent on any of the children nodes (and not already installed by your upgraded code package) they will need to be merged manually.
  • Perform an initial validation of application pages before handing over to QA
  • Apply AEM Service Packs
  • Migrate AEM features
    • Apply changes done as part of Upgrade Process : Code & Customization’s mentioned above
  • Verify Scheduled Maintenance Configurations
  • Enable Replication Agents
  • Enable Custom Scheduled Jobs
  • Execute Test Plan
  • AEM 6.5 will ship with a health check to alert customers if overlaid or referenced content is used in a way inconsistent with the content classification.
    • The Sling/Granite Content Access Check is a new health check (Tools > Operations > Health Reports) that monitors the repository to see if customer code is improperly accessing protected nodes in AEM.
    • This will scan /apps and typically takes several seconds to complete.

 Key Recommendations / Points to be considered while upgrade

  • Phase out use of Administrative Resource Resolver –
    • phase out use of an administrative session through  SlingRepository.loginAdministrative() and ResourceResolverFactory.getAdministrativeResourceResolver()
  • Don’t use deprecated code / functionalities
  • Remove obsolete code
  • Cross check technical compatibilities with new version of AEM
  • Resolve Pattern detector issues
  • Classic UI Authoring – Classic UI authoring is still available in AEM 6.5 but is being deprecated. Plan for Touch UI migration accordingly.
  • Align with 6.5 Repository Structure (/etc)
  • Queries and Oak Indexes
    • Any use of queries in the code base needs to be thoroughly tested as part of upgrading the code base. For customers upgrading from Jackrabbit 2 (versions of AEM older than 6.0) this is especially important as Oak does not index content automatically and custom indexes may need to be created. If upgrading from an AEM 6.x version the out of the box Oak index definitions may have changed and could effect existing queries
  • Upgrading Custom Search Forms – Custom Search Facets require some manual adjustments after the upgrade in order to function properly
  • Assets UI Customizations – (Required only for upgrades from versions older than AEM 6.2.)
  • Adobe  InDesign Script Customizations – Adobe recommends putting custom scripts at /apps/settings/dam/indesign/scripts location.
  • ContextHub configurations are effected by an upgrade. Instructions on how to recover existing ContextHub configurations can be found here. .
  • Editable Templates – required only for Sites upgrades using Editable Templates from AEM 6.2
    • The structure for Editable templates changed between AEM 6.2 and 6.3. If you are upgrading from 6.2 or earlier and If your site content is built using editable templates you will need to use the Responsive Nodes Clean Up Tool. The tool is meant to run after an upgrade in order to clean up content. It will need to be run on both Author and Publish tiers.
  • CUG Implementation Changes
    • The implementation of Closed User Groups has changed significantly to address performance and scalability limitations in previous versions of AEM. The previous version of CUG was deprecated in 6.3 and the new implementation is only supported in the Touch UI. If you are upgrading from 6.2 or earlier then Instructions to migrate to the new CUG implementation can be found here.
  • Review OSGi configurations made from the Felix console
  • Review workflow modifications made from the AEM UI.
  • Add ACLs for tags
  • Determine ACLs and User Groups.
  • Upgrade LDAP authentication integration, if applicable.
  • Upgrade personalization engine from Client Context to ContextHub.
  • Review third-party packages and upgrade if necessary.
  • Upgrade dispatcher version and associated configurations.
  • Before starting an upgrade, please make sure that your application codebase is stable and all the test cases will be executed as per the expected upgrade version.
  • Verify List of Obsolete Bundles Uninstalled After the Upgrade & make sure you contact Adobe Support and ask for a compatibility package for the affected area before upgrade , If your code relies on those bundles.
  • Migrate content from production to lower environments before testing the upgrade.
  • Create Runbook for detailed implementation
  • Refer new AEM Core Components for any new components implementation
  • Test all OSGi services and external integrations.
  • Test all JCR queries
  • Test all data APIs (web services).
  • Test all AEM code overlays and tight integrations.
  • Test all web server (e.g. Apache) redirect and rewrite rules.
  • Plan Security test Performance test

 

Reference Documentation

  • https://helpx.adobe.com/experience-manager/6-5/sites/deploying/user-guide.html
  • https://helpx.adobe.com/experience-manager/6-5/sites/deploying/using/upgrade.html
  • https://helpx.adobe.com/experience-manager/6-5/sites/deploying/using/repository-restructuring.html
  • https://docs.adobe.com/content/help/en/experience-manager-64/release-notes/deprecated-removed-features.html
  • https://helpx.adobe.com/in/experience-manager/6-5/release-notes/deprecated-removed-features.html
  • https://helpx.adobe.com/in/experience-manager/6-5/release-notes/known-issues.html

About The Author