Changelogs

dotCMS 4.2.0 is a minor release which includes a number of significant changes including several new features, considerable performance improvements, and fixes for a number of issues which affected previous releases.

Important Changes

A number of important changes in dotCMS 4.2.0 configuration may impact your performance when upgrading an existing site. Please read the documentation sections on all of these changes before upgrading from a previous release.

Required Post-Upgrade Actions

Important: Due to changes in the database in dotCMS 4.2.0, it is necessary to manually re-index your site after the upgrade is complete.

Configuration Changes

The following important changes have been made to configuration in dotCMS 4.2.0:

• Velocity Cache Region Changes.
• All Velocity cache regions may now be stored to persistent cache (e.g. disk cache, Hazelcast, Redis, etc.).
• However it is important that if the velocitycache region is set to use any persistent cache, the velocitymacrocache region must also be configured to use the same persistent cache.
• For more information, please see the Cache Chaining documentation.
• Inactive Cluster Server Period Removed: The REMOVE_INACTIVE_CLUSTER_SERVER_PERIOD property has been removed (#12573).
• Licenses previously utilized by an inactive clustered node are now immediately freed once the heartbeat time-out on that node has been reached.
• HTTP Only Session Cookie Property Default: The useHttpOnly property is set to false by default.
• This property must be set to true to set all dotCMS cookies to HTTP only, and helps to mitigate the risk of client side scripting when accessing a protected cookie.
• Cluster Configuration Property Changes: The DIST_INDEXATION_ENABLED property has been removed from the portal.properties file.
  • Please see below for more information.

Changes to Default Behavior

The following differences in default behavior in dotCMS 4.2.0 may change your expectations and require a review of your administration practices. Please review all of these changes before upgrading from a previous release.

• WAR File Deployment. When generating a WAR file, the generated file name has been changed from dotcms.war to ROOT.war, to conform better with application server standards and expectations (#11961).
• Struts Actions. For security reasons, ALL front end Struts actions are now disabled by default (#11784).
• This includes all paths found under /dotCMS/*.
• Any Struts actions required for your site operation must be explicitly enabled in the dotmarketing-config.properties file via the properties extension file.
• Vanity URL Conversion. When upgrading an earlier version of dotCMS to release 4.2.0, any existing Vanity URLs (created in an earlier version of dotCMS) will be automatically converted to the new Vanity URL Content Type, and configured as either a 200 forward or a 301 redirect, depending on the URL of the "Enter URL to Redirect To" field in the original Vanity URL:
• URLs beginning with a "/" will be converted to 200/forward.
• URLs containing "//" will be converted to 301/redirect.
• The new Vanity URL Content Type contains additional redirect and forward options and after any upgrade, you should review all your Vanity URLs to ensure they redirect or forward in the way you intend.
• #dotParse Directive Paths. In order to prevent URL errors, the #dotParse directive is less forgiving and no longer supports paths containing invalid slashes ("/"), such as paths with extra slashes (e.g. "/application/vtl///test.vtl") and pages or files with a trailing slash (e.g. "/company/about-us/index/").
• If you have existing Velocity code which contains or generates directives using such invalid paths, it will not longer work after the upgrade, and the code will need to be modified to ensure correct paths are used.

New Features

The following new features have been added in dotCMS 4.2.0:

• Layout as a Service: Page and Content Relationships REST API.
• The new Page REST API enables you to pull all the elements of any page, each encapsulated into a separate JSON obect, either in JSON format or as rendered HTML. This allows single page apps (SPAs) and other iOT applications to retrieve not just raw content, but also fully rendered layouts and modules, which include server side personalization and other context driven information.
• The new Content Relationships REST API enables you to perform a content pull that automatically includes all related items for each item returned in the content pull. This greatly simplifies the retrieval and management of related content without the need for additional content pulls for related content.
• Vanity URLs as Content. Vanity URLs have been converted to a standard content type, and are now accessible as content by all existing dotCMS features and tools, including Push Publishing, importing, exporting, permissions, and REST API management.
• Vanity URLs may now be viewed from both the Vanity URLs tool (Marketing → Vanity URLs) and the standard Content Search tool (Content → Search).
• Existing Vanity URLs will be automatically converted to content when upgrading to release 4.2.0 from an earlier release.
• Language Variables as Content. Language Variables have been converted to a standard content type, and are now accessible as content by all existing dotCMS features and tools, including Push Publishing, importing, exporting, permissions, and REST API management.
• Existing Language Variables will not be converted to content when upgrading to release 4.2.0 from an earlier release. Instead:
• Language Variables created in earlier releases will still show under the Languages tool, while Language Variables created as content will show under the Content Search screen./li>
• Language Variables in both the Languages and Content Search screens (both old and new) will be respected; if the same Language Variable exists in both places, the value from the Languages screen will be used.
• You may convert old Language Variables to content by importing them into the Content Search screen.
• WYSIWYG Field URL Format. You may now specify the URL format that will be used when links are inserted into WYSIWYG fields. For more information, please see the WYSIWYG Field documentation.

Fixes

The 4.2.0 release includes fixes for a number of reported issues, including but not limited to the following:

• Fixed an issue which could cause some assets to lose their contents if multiple site search index jobs were scheduled at the same time (#8552).
• Fixed an issue causing Network status to display as red (error) when more than one server was configured (#11933).
• Fixed a problem pushing content when there is no live version of the Site (#11977).
• Fixed an issue in the dojo library configuration preventing dropdowns and filtering select fields from working properly on Windows 10 (#12007).
• JBoss fix for Boolean Casting error in 4.1.1 - (#12086).
• Fixed an issue which sometimes caused an exception message in the log files when the /edit URL was used to access a page (#12170).
• Timestamp inconsistency fixed for clustered nodes - (#12276).
• Improved push publishing bundle upload to allow upload of a bundle from the same server it was generated on (#12465).
• Fixed an issue preventing the thumbnail from being displayed when a new image is uploaded to a binary field (#12490).
• Fixed an issue preventing the Velocity macro cache from being populated properly (#12523).
• Fixed an issue when saving a Form system field (#12684).
• Fixed an issue which could cause duplicate content on a receiving server if the content type has more than one unique field (#12686).
• Fixed an issue where an extra comma was appended at the end of the options saved to a multi-select field #12715).
• Fixed slow performance issues when push publishing content with many relationships (#12840).
• Fixed an issue preventing a tool from being added to a new Tool Group after it was added to a different Tool Group (#12635).
• Prevented the potential to display the contents of VTL files from the front-end (#12855).
• Fixed incorrect return values and status codes from the content API (#11878)

To view more information on these issues, please visit the dotCMS Github repository.

Additional Changes and Improvements

The following additional changes have been made in dotCMS 4.2.0:

• The generated WAR file name has been changed from dotcms.war to ROOT.war (#11961).
• Push Publishing: A number of significant performance improvements have been made to Push Publishing:
• Categories can now be pushed as a dependency of content (#12125).
• Push Publishing now only retries failed pushes to the specific endpoints which failed (#11500).
• When a push fails, the pushed bundle is no longer regenerated in order to retry the push (#12038).
• Performance Improvements: Significant performance improvements have been made in a number of areas:
• Saving Content: The performance of the contentlet getInode method has been improved (#12488).
• Accessing Content: A number of performance improvements were made when searching, indexing, editing, and saving content from the dotCMS back-end (#12497).
• User Interface: All Angular routes are now loaded lazily, improving performance when loading the application (#12378).
• Library and Tool Upgrades: The following development tools and included libraries have been upgraded or changed:
• The NG user interface library has been upgraded to version 4.2.4.
• The Hazelcast library has been upgraded to version 3.8.4.
• The com.moowork.node library has been upgraded to version 1.2.0.
• Replaced the legacy edu.emory.mathcs.backport.java.util.concurrent library with the java.util.concurrent library to improve portability and maintainability.
• UI build tools have been migrated from webpack to ng-cli.
• Property Changes: The following properties have been changed:
• Removed the DIST_INDEXATION_ENABLED property from the portal.properties file (#12453).
• In most cases, only the following three properties are now required to configure clustering: AUTOWIRE_CLUSTER_TRANSPORT, AUTOWIRE_CLUSTER_ES, and AUTOWIRE_MANAGE_ES_REPLICAS.
• Manually specifying the IP Address and port for cache and Elasticsearch has been deprecated.
• Manual cluster configuration is still supported, but clients using manual cluster configuration should begin planning to change to use cluster auto-wiring or to externalize resources instead, as support for manual cluster configuration may be removed in a future release.
• Added support for running the dotCMS back-end in Internet Explorer 11 (in addition to MS Edge).

Known Issues

• Error Messages Logged when Performing a Static Push (#12187).
• Amazon AWS has a known issue (https://github.com/aws/aws-sdk-java/issues/567), which will generate some noise in the dotcms logs when performing a static push to Amazon AWS.
• Amazon is working to address the issue, but in the meantime, the stack traces returned from AWS in the dotcms.log file can be ignored.

dotCMS 4.1.0 is a considered a minor release but still introduces several new features, enhances a few existing features, introduces performance enhancements, and provides fixes some issues with the 4.0.1 release.

• New Features
• Important Changes
• Configuration Changes
• Default Behavior Changes
• Fixed Issues
• Additional Changes and Improvements
• Known Issues
• Deprecated Features Removed

New Features

In addition to the Important Changes listed above, the following additional new features have been added in dotCMS 4.1.0:

REST API Content Type Management

• New JSON based Content Types
• New REST Endpoints for defining and managing Content Types, Fields and Field Variables
• New API and builders to programmatically create content types.
• Backwards compatible
• Old structure code, factories and caches are all deprecated.

Hazelcast integration

• dotCMS 4.1.0 integrates Hazelcast both as a new cache provider and at the Cluster Transport layer.
• Hazelcast Embedded and Client Cache Implementations
• Hazelcast replaces JGroups for network discovery and topology
• Performance optimizations on distributed caches invalidations (for Hazelcast and Redis) so cache-messages and invalidation-operations are not unnecessarily spread over cluster members

Progressive JPEG Support

• dotCMS now supports progressive JPEG images, both for regular display and as a conversion filter. For more information, please see the Image Resizing and Processing documentation.

Use of Environment Variables in Properties Files

• You may now reference environment and system variables within your properties files, enabling you to maintain common properties files across multiple dotCMS instances, while making changes to to each local server through environment and system variables. For more information, please see the Changing dotCMS Configuration Properties documentation

New Default Cache Provider/Improvements

• caffeineCache has been made the default provider shipping with dotCMS. The caffeineCache provides a more efficient LFU algorithm, vs. Guava�۪s LRU. It will reuse the region settings made for the Guava cache.
• H2CacheProvider has been removed. If you have any regions using the H2CacheProvider, they will need to be updated to use the H22Provider.
• Live and Working caches have been removed and their regions are no longer needed.
• FieldCache has been removed. Calls to it are now made to the ContentType cache
• Velocity only requires 2 cache regions, VelocityCache and VelocityMacroCache. Currently, both of these regions can be serialized and or written to disk or clustered.

Important Changes

A number of important changes in dotCMS 4.1.0 configuration may impact your performance when upgrading an existing site. Please read the documentation sections on all of these changes before upgrading from a previous release.

Configuration Changes

The following important changes have been made to configuration in dotCMS 4.1.0:

• Vanity URLs Upgraded from earlier versions must be manually edited to restore the correct case in the URLs.
• When upgrading to release 4.1.0 from an earlier release of dotCMS, all Vanity URLs will have their case changed to lower case.
• However certain Vanity URLs (such as 404 redirects) require a specific case to work properly.
• Therefore it is recommended that, after upgrading to release 4.1.0, you manually review all upgraded Vanity URLs, and if necessary restore the case.
• If you wish to avoid this issue when upgrading from an earlier version of dotCMS, you may upgrade directly to release 4.2.1 or higher instead of release 4.1.0.
• Properties files are automatically reloaded after each change.
• The dotmarketing-config.properties, dotcms-config-cluster.properties, and portal.properties files are now continually monitored for changes and automatically reloaded as soon as the file timestamp changes.
• In previous versions of dotCMS, changes could take as long as 5 minutes to be reflected, but changes are now reflected almost immediately (typically within a few seconds).
• If you wish to restore the previous behavior, you may add the following property to the dotmarketing-config.properties file: dotcms.usewatchermode=false.
• For more information, please see the Reloading Property Files section of the Changing dotCMS Configuration Properties documentation.
• CSV import now allows only Velocity field variable names as column headers.
• You may no longer use field labels as column names.
• For more information, please see the Importing Content documentation.
• JGroups has been deprecated in favor of Hazelcast for cluster communication.
• If you have jgroups cluster specific config, you will need to remove it and place such configuration in the Hazelcast .xml files.

Changes to Default Behavior

In addition, the following differences in default behavior in dotCMS 4.1.0 may change your expectations and require a review of your administration practices. Please review all of these changes before upgrading from a previous release:

• All Vanity URLs, folder, file and page lookups are now case insensitive - they are forced to lower-case (#11264).
• All new Vanity URLs will be forced to use only lower-case characters when created.
• All existing Vanity URLs will be converted to lower-case (only) on upgrade to dotCMS 4.1.0 from an earlier version of dotCMS.
• OSGI plugins that override dotCMS java classes now use ByteBuddy instead of jamm (#11142).This change will fix some issues with OSGI Plugins, however, all upgrading clients with a plugin that overrides the startup.sh/startup.bat files will need to modify their plugin to incorporate the new changes to those files.
• In addition, a new addClassTodotCMSClassLoader(String className) method was added to the activator, which when called will add your class to the dotCMS Classloader.
• The live cache and working cache have been removed.
• If you have any plugins from earlier versions of dotCMS which make use of the live and working cache, you will need to update them for dotCMS 4.1.0 to use the new pattern.
• For more information on these changes, please see issues #10947 and #11143 in the dotCMS Github repository.
• Properties files now support the inclusion of environment and system variables.
• For more information, please see the Property File Values section of the Changing dotCMS Configuration Properties documentation.
• Properties files are now automatically and immediately reloaded any time they are changed.
• For more information, please see the Reloading Property Files section of the Changing dotCMS Configuration Properties documentation.
• Change in Variable Name used for Binary Uploads in the REST Content API
• RESTfully POSTed content including Binary fields are now expected to use the variable binary + field order.
• For example, if the binary field is the 6th field on the content type, the variable name to use is binary6.
• If you have automated scripts posting binaries using the REST API, please ensure that you are sending the updated variable name in your POST. You can see the proper name on the Content Type definition screen.

Fixes

The 4.1.0 release includes fixes for a number of reported issues, including but not limited to the following:

• Potential read timeout when using JWT authentication behind a proxy (#11348).
• OAuth plugin did not respect the `json.web.token.allowhttp` property (#11412).
• A structure reindex could not be run from the Content Search tab (#11347).
• Hibernate second level cache changed from ehcache to NoCache (#11205).
• Improved messages displayed when a user doesn't have permissions to edit some types of content (#11366).
• Images were not always displayed on Events displayed on the front-end (#11388).

To view more information on these issues, please visit the dotCMS Github repository.

Upgrading to 4.1.0

• Database
  • DB User for dotCMS requires Drop/Add Trigger permissions in order for the database to successfully upgrade to 4.1.
  • There are 2 SQL startup tasks that lowercase vanity urls and identifiers in the database. These are to help DB performance when doing url lookups when no cached object is found.
• -javaagent in startup scripts
  • The jamm.jar is no longer used. Please update any custom bin/startup.sh scripts and remove that as the -javaagent. Instead, set the -javaagent to ByteBuddy as seen here: https://github.com/dotCMS/core/blob/fc239043e782d04ca4a7830cee678f56cc838c92/bin/startup.sh#L60
• H2CacheProvider is removed
  • If you have any custom configed cache regions, you should change these to the newer H22Cache

Additional Changes and Improvements in dotCMS 4.1.0

• Velocity performance improvements: A number of performance and memory usage improvements were made to the Velocity engine. Please click the links below for details on these changes:
• Internal reliance on Velocity macros has been minimized, and dotCMS now relies on pure java Directives vs Velocity macros. This means more compiled code and less interpreted code, resulting in shorter and faster code execution. The #dotParse and #parseContainer macros have been replaced with Velocity **directives**, significantly improving their performance and memory usage (#11362).
• Velocity Memory Improvements (#11236).
• Velocity objects in-memory size reduced 50-75%.
• All velocity objects are now cacheable and Velocity cache regions are much simplified.
• Redundant evaluation of toString disabled in Velocity code (#11441).
• Improved performance of DotTemplateTool and Live mode (#11084).
• Updated the TestInitialContext.java file to read environmental variables (#11319).

Known Issues 4.1.0

• License Notifications for MSSQL and Oracle (#11710)
• Respect Default Page to Default Language Property (#11654)
• Navigation Tool and Page Fall Thru Property (#11640)
• Loading Users Needs Performance Improvement (#11519)

Deprecated Features Removed 4.1.0

• Velocity Macros Removed:
• #pullContent macro code should be replaced with the $dotContent Tool:

Change

#pullContent("+structureName:Client +Client.footer:yes*","6","random")

To

#set($list = $dotcontent.pull("+structureName:Client +Client.footer:yes*","6","random"))
					

• #pullRelated macro code should be replaced with the $dotContent Tool:

Change

#pullRelatedContent("relationshipName" "211e0406-1c2e-4975-9027-3480c2088464" 10 "modDate")

To

#set($list = $dotcontent.pullRelated("relationshipName","$!{parent.identifier}",false,10,"modDate")

• The #dotParse and #parseContainer macros have been replaced with Velocity directives but this requires no code changes, they simply work faster/better.
• The following macros have been removed completely (replace with the $dotContent.find tool):
• #getContent
• #getContentDetail
• #getContentDetailByIdentifier
• #getContentMapDetail
• #getContentMapDetailByIdentifier
• Deprecated API's - see the new Content Type API Documentation
• com.dotmarketing.portlets.structure.business.StructureAPI
• com.dotmarketing.portlets.structure.business.FieldAPI
• com.dotmarketing.portlets.structure.factories.FieldFactory
• com.dotmarketing.cms.*
  This includes all the legacy front end struts actions, including front end login, account creation and management, SubmitContent Macros, Webforms

dotCMS 4.0.1 is a maintenance release to fix an important issue which affected some customers with the dotCMS 4.0.0 release.

Fixes

• Fixed an issue causing an error on startup if the dotCMS server was not connected to the Internet (#11131).

To view more information on this issue, please visit the dotCMS Github repository.

The dotCMS backend UI has moved to an Angular 2 interface!

For more information about the new user interface, please see UI Changes, below.

dotCMS 4.0.0 includes a major update to the dotCMS backend user interface, several important new features, and a number of fixes. We encourage you to check out the new features and improvements in this release on the dotCMS Demo Site, and we welcome your feedback.

• Important Changes
• New Features
• Fixed Issues
• Additional Changes and Improvements
• Deprecated Features Removed
• Known Issues
• Changes to Default Behavior - Permission Inheritance

Important Changes

Important changes in dotCMS 4.0.0 configuration may impact your performance when upgrading an existing site. Please read the documentation sections on all of these changes before upgrading from a previous release.

• Permission Inheritance Changes to default behavior
• Internet Explorer is not currently supported for access to the back-end UI.
• Due to Internet Explorer incompatibilities with the Angular 2 implementation, it is not currently supported to use Internet Explorer to access the dotCMS back-end UI.
• Please see below for more information.
• URL paths to all back-end screens have changed.
• As part of this change, the URL to login to the dotCMS back-end has been changed from {yourdomain}/admin to {yourdomain}/dotAdmin.
• Please see below for more information.
• A number of legacy deprecated features were removed.
• Please see below for a list of the features removed.
• Windows Server is still fully supported, but has been officially deprecated (for termination of support in a future release).
• If you currently use Windows Server for your production dotCMS environment or have questions about support for Windows, please see below for more information.

Changes to Default Behavior

The following difference in default behavior in dotCMS 4.0.0 may change your expectations and require a review of your administration practices. Please review this change before upgrading from a previous release.

• Permission Inheritance
• In dotCMS 3.x and below, when changes were made to a high-level object such as a site or top-level folder, the permissions of all lower level objects which inherit permissions from that object were updated immediately ("eagerly"). In large sites, these updates could cause performance issues and database bottlenecks.
• In dotCMS 4.0.0, the default behavior was changed to perform these permission updates "lazily" instead, so permissions are updated as each of the inheriting objects is accessed (rather than all at once).
• It is recommended that you keep the new behavior even if you are upgrading from an old version; however if for any reason you need to change the default behavior back to the "eager" permission updates performed in earlier versions, you may add the PERMISSIONS_REFERENCES_INSERT_LAZILY property to your dotmarketing-config.properties file (using a properties file extension), and set the value to false.
• For more information, please see the Permission Inheritance documentation.

There are a number of other important changes to be aware of when upgrading to dotCMS 4.0.0, including changes to OSGI plugins and the dotCMS source code base. If you are upgrading from a previous version of dotCMS, and especially if you've done any customization of the dotCMS backend, please read all sections of the Upgrading to dotCMS 4.0 documentation before performing your upgrade.

New Features

The following new features have been added in dotCMS 4.0.0.

New Angular 2 UI

dotCMS 4.0.0 has a newly re-designed, material design based backend UI interface, implemented using Angular 2 and using the PrimeNG set of components. PrimeNG offers, at this time, the richest set of pre-built Angular 2 components and has the additional benefit of being easy to skin. As the rebuilding of the dotCMS admin tool continues, we plan to layer in more prebuilt and custom NG2 components as needed.

As part of this UI change:

• The Roles & Tabs tool has been renamed to Roles & Tools.
• Back-end Tabs have been renamed to Tool Groups.
• The Tool Group Description field no longer displays in the UI.
• The field is now instead used to allow you to specify the name of an icon for the Tool Group (from the FontAwesome font).

JWT Based Authentication

dotCMS 4.0.0 standardizes on JWT or Javascript Web Tokens for authentication. JWT is lightweight and scalable authentication solution that is stateless and does not require server sessions to track a users authentication.

For more information, please see the Authentication Using JWT documentation.

Mobile Preview

A new mobile preview feature has been added which allows you to preview your site as it is displayed on different mobile devices. You may create different device profiles using the new "Device" Content Type, and then "Preview as" any of the devices you've created.

Elastic Search Snapshots

The legacy index backup and restore functionally has been replaced with the Elasticsearch snapshot and restore modules. Prior to dotCMS 4.0.0 the index backup/restoration used a custom dotCMS process. This has been optimized by calling the native snapshot/restore modules included in Elasticsearch.

For more information, please see the Managing Site Indexes with ElasticSearch and RESTful API to Manage Indexes documentation.

New Notification Engine

Many new improvements have been made to the dotCMS notifications system to notify users of process status and changes to the dotCMS system. Most of these changes provide additional notifications (such as when processes complete), but do not otherwise affect how you use dotCMS.

If you would like more information on the notifications implemented, please view the list of Notification Engine improvements on Github.

Fixed Issues

The 4.0.0 release includes fixes for a number of reported issues, including but not limited to the following:

• Some international characters were not recognized in URLs (#8839)
• Multi-valued "X-Forwarded-For" header could cause problems in the Rules Engine (#9797)
• It was sometimes not possible to reorder Key/Value fields (#9935)
• Restore of a downloaded snapshot could fail if the downloaded file was renamed (#9974)
• Binary fields sometimes can not be deleted from a Content Type (#10756)
• It was sometimes not possible to add publishing environments using non-English languages (#10749)
• An error was sometimes generated when downloading a bundle pushed to a static publishing environment (#10740)
• ContentMap viewtool did not always respect the content language when pulling images or files (#10119)
• Icons for Bulgarian and Romanian were missing from the Language tool (#10112)
• Files uploaded through the Site Browser showed the file title instead of the file name (#10011)
• When the ContentAPI delete method was used in a plugin, the content was not deleted (#9956)
• Push publishing a user which already existed on the receiving endpoint could cause the Role id to get out of sync (#9793)
• When upgrading dotCMS on MySQL, upgrade could fail if the database is configured for case-sensitive table names (#9788)
• An error could sometimes be displayed when accessing the Dashboard Analytics portlet (#9775)

To view more information on these issues, please visit the dotCMS Github repository.

Additional Changes and Improvements

• Elasticsearch Results: Elasticsearch results now include Tag and Category fields (#8953).
• Felix version: OSGI (Dynamic Plugins) have been upgraded to Felix version 5.
• Existing plugins built for older versions of dotCMS should work without modification in dotCMS 4.0.0.
• Back-end URL Paths: URL paths to all back-end screens have changed. As part of these changes:
• The URL to login to the back-end has been changed from {yourdomain}/admin to {yourdomain}/dotAdmin
• The old back-end login URL ({yourdomain}/admin) is automatically forwarded to the new URL, so existing links and scripts will work without modification.
• Individual objects no longer have unique back-end URLs.
• To access an individual object, you must select it from the appropriate back-end screen (e.g. Content Search).
• Adding unique back-end URLs for specific objects is planned for a future release.
• Back-end Messaging: Back-end messaging has been thoroughly updated.
• All back-end messaging (control names, help and hints, error and warning messages, etc.) has been reviewed and improved for clarity and correctness.
• Log file messages have not been significantly changed, so administrative tasks or scripts that depend on matching existing messages should work without issues.
• Automated Testing Tools: New automated testing tools have been added to the dotCMS 4.0.0 version to ensure the stability of development modifications.
• FixAsset tasks can now be added via an OSGI plugin (#9822)
• Windows Server Deprecated: Support for Windows Server as a dotCMS production environment is officially deprecated.
• This feature has been deprecated because there are known issues with the feature, there are better alternatives to the feature, and/or support for the feature is planned to be removed in a future release.
• If you are a new to dotCMS we recommend that you plan to use a platform other than Windows for your production environment.
• If you are currently using Windows for your production dotCMS environment, we recommend that you begin planning to migrate away from Windows as a production environment.
• Though deprecated, Windows as a production dotCMS host will continue to be supported by dotCMS for the foreseeable future.
• In addition, even after the EOL for production Windows Server support, dotCMS plans to continue to support Windows as a development environment.
• We do not yet have a timeframe for terminating support for Windows as a production dotCMS host.
• Windows as a production environment will be supported at least through the 4.0 series; the EOL for Windows Server will be in 2020 or later.
• Note that production support for MS SQL Server has not been deprecated and will continue.

Deprecated Features Removed

A number of deprecated legacy features have been removed in dotCMS 4.0.0.

Important: These features are no longer supported, and any content in existing sites which still use these legacy features must be converted to newer, supported versions of these features before the site can be upgraded to dotCMS 4.0.0.

• Inline Help
• Legacy Files
• Legacy HTML Pages
• Deprecated Content Macros
• All macros marked as deprecated in dotCMS 3.6.0 (all macros listed in the 3.6.0 Deprecated Content Macros documentation) have been removed.
• PollsAPI
• RatingAPI
• MP3 Player Button
• MP3 Player Extended
• MP3 Player (Streaming)
• Photo Carrousel
• Photo Gallery
• Video Gallery
• Video Player
• Slide Show: Flash
• Social Bookmarks
• Send to a Friend
• File Repository
• PDF Export a Web Page
• Velocity Viewtools: As part of these changes, the following Velocity Viewtools have been removed:
• $alertMessagesWebAPI (com.dotmarketing.util.AlertMessagesWebAPI)
• $favorites (com.dotmarketing.viewtools.FavoritesWebAPI)
• $googleapi (com.dotmarketing.viewtools.GoogleMiniApi)
• $groovy (com.dotmarketing.scripting.viewtool.GroovyTool)
• $js (com.dotmarketing.scripting.viewtool.JSTool)
• $mailinglist (com.dotmarketing.viewtools.MailingListWebAPI)
• $nav (com.dotmarketing.viewtools.NavigationWebAPI)
• $pageAPI (com.dotmarketing.viewtools.HTMLPageWebAPI)
• $poll (com.dotmarketing.viewtools.PollsWebAPI)
• $python (com.dotmarketing.scripting.viewtool.PythonTool)
• $rating (com.dotmarketing.viewtools.RatingWebAPI)
• $rssapi (com.dotmarketing.viewtools.RSSWebAPI)
• $ruby (com.dotmarketing.scripting.viewtool.RubyTool)
• $sitemapapi (com.dotmarketing.viewtools.SiteMapWebAPI)
• $video (com.dotmarketing.viewtools.VideoGalleryWebApi)

Known Issues

The following known issues exist in dotCMS 4.0.0. For more information on these issues, please contact dotCMS Support.

• Back-end access through Internet Explorer is not supported.
• dotCMS 4.0.0 does not currently support Internet Explorer for access to the dotCMS back-end UI.
• However it is still fully supported for users to access to your front-end site using Internet Explorer (multiple versions), and to access the back-end using Microsoft Edge.
• Support for IE 11 will be considered for a future maintenance release.
• If IE is a requirement for your organization, please continue using the latest 3.7.x version of dotCMS, and contact dotCMS support for information on future IE support.
• The TinyMCE Plugin does not yet work with dotCMS 4.0.
• Support for this plugin will be considered for a future maintenance release.

dotCMS 3.7.1 is a maintenance release which includes several improvements and some fixes for known issues.

Fixes in dotCMS 3.7.1

The 3.7.1 maintenance release includes fixes for the following reported issues:

• Fix to Push Publish dialogue: (#10628).
• SQL Upgrade Task Fix: (#10608).
• Large Metadata Re-indexation Fix: (#9451).

Improvements in dotCMS 3.7.1

The 3.7.1 maintenance release includes the following functionality/feature improvments:

• Licensing Improvement to Request Handling: (#10626).
• Shorty Tool and Associated File Handling: (#10675). - See Documentation
• Static Publish Bucket Region Logic: (#10642).

To view more information on these issues, please visit the dotCMS Github repository.

dotCMS 3.7.0 includes static publishing to AWS S3 for simplified content delivery.  The static publishing works in conjunction with dotCMS’s current push publishing  framework and allows customers to publish to AWS S3 “buckets” to serve static content and sites.  This allows customers to leverage S3 as a point-in-time DR, as a CDN (Cloudfront) or just to serve content and sites statically.  3.7.0  also includes support of shorter “shorty” UUId's for SEO and user-friendly file and image URLs.  3.7.0 also offers enhanced image resizing and thumbnail support of a wider variety of file types. We encourage you to check out the new features and improvements in this release on the dotCMS Demo Site, and we welcome your feedback on these enhancements.

• Static AWS Publishing
• Shorty UUIds
• Image Handling Enhancements

Important Changes

• Licensing - dotCMS 3.7.0 introduces a new license level - “Platform”. The static publishing feature is available at the “Platform” license level and is available to customers with a dotCMS site-license subscription. If you would like to evaluate the new static publishing capabilities, please contact your account representative or the dotCMS Support desk.
• Microsoft SQL Customers - dotCMS 3.7.0 includes a MSSQL only Startup Task. Previously, dotCMS MSSQL implementation did not support UTF-8 characters or the use of SQL indexes when querying SQL Server which can result in poor performance with large datasets. This has been fixed in 3.7.0 which provides a Startup Task to convert the dotCMS database to support UTF-8. This task modifies many tables, constraints and indexes and might take a long time to complete on large MSSQL datasets. Please insure that for this task, the dotCMS DB user supplied in the JDBC connection has drop/create privileges for all dotCMS tables, tmp tables, indexes, triggers and constraints. The other database types supported by dotCMS (Oracle, Postgres and MySQL) are not affected.

Configuration Changes

• AWS Static Publishing is configured directly through the dotCMS Push Publishing UI and does not require configuration changes

Performance Improvements

• The MSSQL startup task described above will result in dotCMS using SQL indexes (instead of full table scans) for content lookups and offer a potential significant performance improvement: #9647

Important Fixes

The 3.7.0 release includes fixes for a number of reported issues, including but not limited to the following:

• Renaming Folders forces a reindex on all content under that folder. Fix: #10547
• WebDav folder renaming fixed on OSX Finder. Fix: #10397
• New tags introduced in 3.5  are now included in REST content response Fix: #9736
• Auto-Cluster does not revoke a server license if not using heartbeat. Fix #10388
• User accessing a non-public file is redirected properly after login.  Fix: #9996
• Content Types can no longer have multiple TAG fields: #9684
• Fix Asset Inconsistencies now includes fixing orphaned assets - #9095

See a complete list of issues in the dotCMS 3.7 release.

New Features

In addition to the Important Fixes listed above, the following additional new features have been added in dotCMS 3.7.0:

• Static Publishing (#9045)
• Shorty UUIds (#9800, #10490)
• ImageIO Enhancements (#9541)

Additional Changes and Improvements in dotCMS 3.7.0

• Improve Push Directory (#10599)
• PDF Thumbnailer (#10499)
• Dynamic Plugin Undeploy Fix (#10103)
• File Resource Link Fix (>#9996)

Known Issues

• SQLserver: IOUtils.toString() can hit OOM by hoversized metadata (#9451)

To view more information on these issues, please visit the dotCMS Github repository.

dotCMS 3.6.2 is a maintenance release which includes a security update for a newly discovered security vulnerability.

Due to the inclusion of a fix for a known security vulnerability, it is recommended that all current customers upgrade to release 3.6.2 as soon as possible.

Important Changes

The following important changes in dotCMS 3.6.2 may impact your site security and performance. Please read the documentation sections on all of these changes before upgrading from a previous release.

• Security Update

Security Update

dotCMS 3.6.2 includes a fix to a newly discovered security vulnerability in older dotCMS releases.

• The details of this vulnerability are being temporarily withheld, to enable customers to upgrade to the new release before the nature of the vulnerability is disclosed.
• It is recommended that all customers upgrade to dotCMS 3.6.2 as soon as possible, to incorporate the fix for this vulnerability.
• Enterprise customers with active support contracts who are not able to upgrade to dotCMS 3.6.2 may request a hotfix for this vulnerability for older versions of dotCMS.
• If you are an Enterprise customer, please contact dotCMS Support (via email or the support portal) to request more information on this security upgrade or to request a hotfix for an older version of dotCMS.

Fixes

The 3.6.2 release includes fixes for a number of reported issues, including but not limited to the following:

• Fixed a license error caused by deleting an inactive server in a cluster when a specific combination of configuration properties are set (#10388).
• Fixed an incorrect redirect to a 404 page after a user accesses a resource link and logs in (#9996).

To view more information on these issues, please visit the dotCMS Github repository.

dotCMS 3.6.1 is a maintenance release to add some performance improvements and to fix a number of reported issues in the 3.6.0 release.

Performance Improvements

• Improved performance of ContainerService Code (#10101)
• Optimized the reindex process to avoid reindexing the same content more than once (#9768)

Fixes

The 3.6.1 release includes fixes for a number of reported issues, including but not limited to the following:

• Empty files are sometimes created when using Finder and WebDAV (#10003)
• File copy fallback does not work when a hardlink is not possible (#9997)
• Server license may be invalidated if dotCMS is started when the database is not running (#9849)
• Errors migrating HTML Pages to content (#9760)
• File Title is incorrectly displayed in the Site Browser instead of File Name (#10011)
• An exception may be thrown when NavTool gets the parent permissionable of a Page Asset (#9893)

To view more information on these issues, please visit the dotCMS Github repository.

Known Issues

None at this time.

dotCMS 3.6.0 includes some important new features and some improvements to our multilingual featureset. We encourage you to check out the new features and improvements in this release on the dotCMS Demo Site, and we welcome your feedback on these enhancements.

• Important Changes
• Database Upgrade Requirements
• Changes to Default Behavior
• Changes to Empty File Handling
• Cluster Autowire Scale Down Improvement
• WebDAV Pathing Has Changed
• Default Cache Behavior Has Improved
• New Default H2 Cache
• New Features
• Google Translate Workflow Sub-Action
• Multilingual Files
• Finnish Back-end Translation
• User accounts may now be deleted
• Performance Improvements
• Important Fixes
• Additional Changes and Improvements

Important Changes

Important changes in dotCMS 3.6.0 configuration may impact your performance when upgrading an existing site. Please read the documentation sections on all of these changes before upgrading from a previous release.

*Database Upgrade Requirements (Applies only to pre-dotCMS 3.5.1 upgrades)

Clients with large datasets or with a large amount of content using tags must make configuration changes to two properties in the context.xml file both prior to and after the upgrade to ensure the upgrade completes successfully.

Before Upgrade:

Before starting up the new 3.6.0 instance, change the following property values in the database configuration in your context.xml file to the values shown below:

maxWaitMillis="-1"
removeAbandonedTimeout="3600"

After Upgrade:

After the upgrade process is completed, the properties for the configuration should be reset to the default settings:

maxWaitMillis="60000"
removeAbandonedTimeout="60"

*  Disregard if you are upgrading from dotCMS version 3.5.1.

Changes to Default Behavior

Changes to Empty File Handling

Default behavior has been changed to reject the upload of empty files.

You may change the default behavior by setting the value of the CONTENT_ALLOW_ZERO_LENGTH_FILES configuration property to true:

CONTENT_ALLOW_ZERO_LENGTH_FILES=true

Note: A warning is always logged any time an empty file is found, regardless of the property setting.

IMPORTANT: Regardless of the property setting, it is highly recommended that empty files are cleaned from the assets folder of your dotCMS installation. This can be done in unix by running:

find $PATH_TO_ASSETS -size 0 -type f -delete

For more information, please see the File Configuration documentation.

Cluster Autowire Scale Down Improvement

Default cluster behavior has changed so that replicas are not considered "timed out" and dropped for 30 minutes. This allows servers to be stopped for short maintenance intervals without being dropped from the cluster.

You can change the timeout by setting the value of the HEARTBEAT_TIMEOUT property in the dotcms-config-cluster.properties file:

HEARTBEAT_TIMEOUT=30

For more information, please see the Cluster Configuration documentation.

WebDAV Pathing Has Changed

The default path when using WebDAV has changed to support multilingual files and pages.

• The default root of the WebDAV path is now /webdav/working/{language_id} or /webdav/live/{language_id}, for example:

  /webdav/working/1/about-us/index

• WebDAV pathing may be changed to Legacy WebDAV pathing (e.g. /webdav/autopub/about-us/index or /webdav/nonpub/about-us/index) by setting the WEBDAV_LEGACY_PATHING property in the dotmarketing-config.properties file to true.

Default Cache Behavior Has Improved

Cache Regions not explicitly configured in the dotmarketing-config.properties file are now divided into separate caches. In dotCMS 3.3, undefined cache regions were all combined into the default cache region. In dotCMS 3.6.0, undefined cache regions are separated into unique cache regions, each one using the cache chain specified by the cache.default.chain property and using the default region size (1000 entries).

For more information, please see dotCMS reported issue #9104.

New Default H2 Cache

A new H22 cache provider has been added to dotCMS as a replacement for the original H2 cache provider, and the H22 cache provider is now the default cache provider. The H22 cache offers an H2DB-based cache with a number of improved features over the original H2 cache provider:

• Performance: The H22 cache provides improved performance and collision avoidance.
• Configurability and Flexibility: The H22 cache provides a number of configuration properties allowing you to tune the cache database size, connection and database parameters, and error handling for your specific needs.
• Scalability: The size of the H22 cache is configurable and the cache is sharded, enabling improved scaling and improved performance with clusters.
• Fault Tolerance and Robustness: The H22 cache can tolerate many common errors and will automatically recover (creating a new database) if the number of errors exceed a configurable threshold.

Note: The original H2 cache provider is still supported for backward compatibility with older versions. For more information, please see the H22 cache provider documentation.

New Features

In addition to the Important Changes listed above, the following additional new features have been added in dotCMS 3.6.0:

• A Workflow Sub-Action has been added to automatically translate content using Google Translate (#9037)
• Multilingual files are now supported via back-end upload, WebDAV, etc. (#9162)
• A Finnish translation has been added, allowing you to use the dotCMS backend in the Finnish language (#9691)
• Users can now be deleted from the system. (#8879)

Performance Improvements

The performance of several Maintenance tasks was improved:

• Reindex has been improved to avoid multiple reindexation of the same content and nested relationships (#8836)
• Execution time when fixing a large number of orphaned identifiers has been greatly reduced (#9234)
• Execution time when fixing a large number of content items with inconsistencies in their inodes has been greatly reduced (#9229)

Important Fixes

The 3.6.0 release includes fixes for a number of reported issues, including but not limited to the following:

• Changed database to maintain compatibility with the new version of Oracle (#9277)
• Delete Push Assets History۪ functionality on Maintenance Portlet is now working when using Oracle and MS SQL Server databases (#9383)
• Push Push Actionlet works in Edit Mode (#9351)
• Push of already pushed bundled assets to multiple environments gets them sent to all selected/expected (#9289)
• Push delete properly deleting content on the receiver when adding the content to a bundle and downloading for deletion (#9053)
• Pushing content with a live version but on working state now properly reflects its working state on the receiver (#8784)
• Users can now be properly push published (#9242)
• Time machine correctly snapshots personalized content (#9097)
• Jobs not leaking database connections (#9159)
• CSV Template now has correct values for key/value fields when downloading the CSV Sample file for a content type with fields of that type (#9137)
• Pagination on the Content Type admin page is now properly working (#9133)
• Fix task for checking inconsistencies in Assets not hanging anymore (#9106)
• Binaries already selected not losing its value when another require field is missing (#9082)
• List of pages referencing a contentlet is now showing the correct ones when those are in different languages (#8996)
• User permissions are now fully cascading from All Sites (#8853)
• Delete content through the Maintenance admin screen properly deletes object (in all languages) (#8820)
• New versions of existing content in different languages are now able to be created via REST (#8775)
• User name and last name now accept unicode characters (#8750)
• Deploy static plugins from git checkout now works correctly on Windows (#8414)
• List of initial hosts for auto clustering is now building properly (#8196)
• Cluster Autowire functionality now verifies that addresses exist on a network interface (#6971)
• Tags are now included when getting content via REST (#8903)
• Tags and Categories of content are now included in the results of the ElasticSearch admin page (#8953)
• In the HTMLPage admin page, the 'Filter By Template' dropdown now properly lists options (#8901)

To view more information on these issues, please visit the dotCMS Github repository.

Additional Changes and Improvements

• Elasticsearch has been upgraded to version 1.7.
• Markdown support has been moved from a plugin into the dotCMS core via the Markdown Viewtool.
• Streamlined Source Code: A number of deprecated SQL tables, Javascript libraries, etc.
• Unmapped several deprecated servlets.
• Record startup times of different modules in the System (Overall, Cache, DB, ElastcSearch, Quartz Jobs, OSGI plugins) (#Performance Tuning - StartupTime)
• Results from Elasticseach via REST can be prettyfied (#REST-prettyfied)
• Deploy-plugins sh, bat and gradle task now runs undeploy-plugins first (#8814)
• Deploy-plugins sh, bat, and gradle task now always compile the code, even though when the build folder already exists (#8703)
• UrlRewriteFilter was moved above the CMSFilter in order to correctly process forwards (#6926)
• JSONTool now is able to handle JSON arrays (#JSONTool, #1991)
• When editing a Site, the All Sites option was removed from the tag storage field to eliminate ambiguity and confusion with System Host

Note:

Clients upgrading from older versions (especially dotCMS 1.x) may need to re-map some of these servlets to enable old migrated content to operate in dotCMS 3.6.0.

Known Issues

• Images added to WYSIWYG fields before the first push don't display on the receiving server (#8140)
• File Asset Resource Link path needs to be the same for all language versions (#9725)

dotCMS version 3.5.1 is a maintenance release to add some performance improvements and fix a number of reported issues in the 3.5 release.

Important Changes

• Postgres Upgrade Requirements
• Upgrading from a pre-3.2.4 Release

Configuration Changes

Important changes in dotCMS 3.5.1 configuration may impact your performance when upgrading an existing site. Please read the documentation sections on all of these changes before upgrading from a previous release.

Postgres Upgrade Requirements

Clients with large datasets that are using a Postgres database must make configuration changes to two properties in the context.xml file both prior to and after the upgrade to ensure the upgrade completes successfully.

Before Upgrade:

Before starting up the new 3.5.1 instance, change the following property values in your context.xml file to the values shown below:

maxWaitMillis="-1"
removeAbandonedTimeout="3600"

After Upgrade:

After the upgrade process is completed, the properties for the Postgress configuration should be reset to the default settings:

maxWaitMillis="60000"
removeAbandonedTimeout="60"

Upgrading from a pre-3.2.4 Release

There is a known issue with the upgrade tasks that will prevent a direct upgrade to version 3.5.1 from any release prior to dotCMS 3.2.4. If you are upgrading from a pre-3.2.4 release, you must first upgrade to dotCMS 3.5, and then after that upgrade is complete, upgrade from dotCMS 3.5 to dotCMS 3.5.1.

Performance Improvements

Several performance improvements have been made in this release:

• Improved cache handling of permissions: The overall system performance has been improved by reducing the number of round trips to the database server when checking permissions (#9312).
• Improved upgrade performance: Upgrade tasks for datasets with a large number of contentlets that utilize tags has been significantly optimized (#9362).
• Improved cache performance under heavy load: The cache implementation has been changed to utilize less CPU resources (#9221).

To view the details on any of these issues, please click the link in parentheses next to each issue.

Fixes

This release includes fixes for a number of reported issues, including but not limited to the following:

• Fixed critical security isssue, "Lack of user verification in the CMSMaintenanceAjax class" (SI-37).
• Changed database to maintain compatibility with the new version of Oracle (#9277).
• Improved the DB locks used in scheduled jobs when running with MS SQLServer (#9270).
• Fixed an issue when deleting a field from a Content Type with large numbers of existing records (#8793).

To view the details on any of these issues, please click the link in parentheses next to each issue.

To view more information on these issues, please visit the dotCMS Github repository.

Known Issues

• There is an issue with the upgrade tasks (#9475) when upgrading to dotCMS 3.5.1 from a version of dotCMS earlier than dotCMS 3.2.4.
• Attempts to upgrade directly from versions of dotCMS earlier than 3.2.4 to dotCMS 3.5.1 will fail.
• To upgrade from a version of dotCMS earlier than 3.2.4, you must first upgrade to dotCMS 3.5, then upgrade to dotCMS 3.5.1.

dotCMS version 3.3.2 is a maintenance release to fix security issues. Note that all fixes in the 3.3.2 release are also included in the dotCMS 3.5 release.

Fixes

This release includes fixes for the following reported security issues:

• XSS vulnerability in the Query Tool (lucene) screen.
• SQL Injection vulnerability when saving content through the REST API.
• Ability to view files on the Log Viewer outside the configured directory.
• SQL Injection vulnerability in the Workflow configuration screen.
• Header injection vulnerability in the send email functionality.
• SQL Injection vulnerability via DWR.
• Lack of user verification in the CMSMaintenanceAjax class.

To view more information on these issues, please see the Known Security Issues documentation.

dotCMS version 3.5 contains several major new features and improvements. We encourage you to evaluate the new features and improvements in this release on the dotCMS Demo Site, and we welcome your feedback.

Important Changes

The following important changes in dotCMS 3.5 may require a change in your configuration, or may impact your performance when upgrading an existing site. Please read the documentation sections on all of these changes before upgrading from a previous release.

dotCMS 3.5 no longer supports Java 7

You must use Java 8 or later versions with dotCMS 3.5.

• The MaxPermSize parameter has been removed from the dotCMS startup script.
• Java 8 no longer supports the MaxPermSize parameter.
• If you have made changes to MaxPermSize in your startup script, you must replace it with MaxMetaspaceSize for the changes to have any effect in Java 8.

The /plugins/common.xml file has Changed

When copying your static plugins to the 3.5 release, you must not copy the entire /plugins/ folder from the previous release.
Instead, copy each individual plugin folder (and the plugins.xml file if you have changed it) from the old release to the 3.5 release folder.

For more information on the proper steps to copy your plugins to 3.5, please see the Upgrading dotCMS documentation.

New Features

dotCMS 3.5 contains two major new features which enable you to customize the display of your site for different users:

1. Rules are a very powerful feature that enable you to set dotCMS properties and change how your pages are displayed based on visitor information such as visitor location and platform.
• Rules may be built from the many built-in Conditions and Actions, or you may create your own custom Conditions and custom Actions using OSGI plugins.
• Rules may be accessed and updated via the REST API.
• Rules may also be push published to other servers.
2. Personalization allows you to provide personalized content to different users that visit your site based on user preferences and browsing behavior.
• The Visitors feature provides you with information about each user who visits to your site, including current information such as user location and platform, and historical information such as the tags of content the user has visited in the past.
• The Personas feature allows you to define different types of users, and then server content to each user based on which type of user you've identified them as.
• The pullPersonalized method enables you to automatically pull and sort personalized content for different users, using the built-in personalization features of both Visitors and Personas.

In addition, dotCMS 3.5 now allows you to sort content pulls by score.

Fixes

The 3.5 release includes fixes for a number of reported issues, including but not limited to the following:

• Fixed critical security issue, "SQL Injection via REST API", SI-35
• Fixed moderate security issue, "SQL Injection from Workflow Screen III", SI-36
• Fixed moderate security issue, "Directory traversal vulnerability by Admin", SI-34
• Fixed low security issue, "XSS in Lucene Search Admin tool", SI-33
• Incorrect Logging Levels in the PluginFileMerger class.
• Image Editor image size slider doesn't display correctly.
• Copy Host -> Copy Content does not link the new content to the new host.
• When using H2 database, the Templates list is empty for non-admins when adding a page.
• A 404 exception generates an unnecessary stack trace in the log file.
• CookieUtil sometimes generates incorrect cookies or cookie syntax.
• Initial Referrer not set correctly in ClickstreamFactory.

In addition, the 3.5 release includes fixes for the following issues (which are also fixed in the 3.3.1 maintenance release):

• After migrating legacy pages, the Redirect URL value is dropped.
• Cannot add folders at root level with limited user.
• File names are not escaped when Downloading.
• Wrong language_id affects the current session.
• Error generating Integrity Checker results using MS SQL Server.
• Log4j2 log rollover stores files in incorrect location.
• FileAssetMap does not have a 'host' field anymore.
• Release aborted file handles.
• Casting exception during Push Publishing when running on MS SQL Server.
• Host Lookup in request does not work in Live Mode.
• The back-end Link to the dotCMS Support Portal is wrong.
• Integrity Checker fails when there are unpublished/archived assets.
• Cluster cache gets corrupted after receiving content.
• OSGI Framework tries to deploy bad plugins indefinitely.
• NavTool returns wrong "active" items.
• Button in Push Publish Workflow Action has misleading label.
• Files under a folder are sorted incorrectly.

To view more information on any of these issues, please visit the dotCMS Github repository.

Additional Changes and Improvements

• The Microsoft JDBC driver has been upgraded from version 4.1 to version 4.2.
• Default Password requirements have changed.
  • Please see the Password Security Configuration documentation for more information.
• The Ratings thread is now turned off (disabled) by default.
  • This deprecated functionality is no longer used except for legacy sites.
• The tagCloud macro has been removed.
  • This macro has been deprecated for some time, and is now fully removed.
  • You may replace the tag cloud macro with a tag cloud implemented using ElasticSearch aggregates.
• Multiple tag fields in a single Content Type are not supported.
  • Multiple Tag fields in a single Content Type are not supported in dotCMS 3.5.
  • Although adding multiple Tag fields in the same Content Type has never been officially supported in dotCMS, in previous versions it was possible to use multiple Tag fields for some use cases.  However Tag changes in dotCMS 3.5 effectively merge all Tag fields in a Content Type into a single Tag field, so that tags entered in one Tag field can not be entered in the other Tag field, and tags entered in all Tag fields will be retrieved when a query is performed on any of the Tag fields.
  • Customers who have implemented multiple Tag fields per Content Type in previous versions should plan accordingly, and consider using Categories instead of Tag fields to separate different taxonomies in the same Content Type, or a custom field to enter and manage keywords independent of the dotCMS tags.

Known Issues

• ContentResource does not include content tags when viewing as JSON or XML.
  • https://github.com/dotCMS/core/issues/8903
• HTMLPage Portlet
  • The HTMLPage portlet 'Filter By Template' dropdown does not list any options.
    • https://github.com/dotCMS/core/issues/8901
  • The HTMLPage portlet text filter only works for legacy HTMLPages.
    • https://github.com/dotCMS/core/issues/8904

dotCMS version 3.3.1 is a maintenance release to add some performance improvements and fix a number of reported issues in the 3.3 release.

Performance Improvements

Several performance improvements have been made in this release, including but not limited to the following:

• Push publishing has been optimized for bundles containing related items, significantly improving speed for large sites.
• Some MySQL queries have been optimized, significantly improving performance for some operations.
• Performance has been improved when viewing pages in Edit or Preview mode on servers with large numbers of Content Types.

Fixes

This release includes fixes for a number of reported issues, including but not limited to the following:

• After migrating legacy pages, the Redirect URL value is dropped.
• Cannot add folders at root level with a limited user, even when permissions should allow it.
• Filenames are not escaped when Downloading.
• In some cases, the wrong language_id affects the current session.
• The Integrity Checker sometimes generates errors when running on MS SQL Server.
• Log4j2 log rollover file locations have changed to support improved log history.
• FileAssetMap no longer has a 'host' field.
• In some specific circumstances, aborted filehandles may not be released properly.
• There is sometimes a casting exception during Push Publishing when running on MS SQL Server.
• Host Lookup in request does not work in Live Mode.
• The back-end Link to the dotCMS Support Portal is incorrect.
• Integrity Checker may sometimes fail if there are unpublished/archived assets.
• The cluster cache may sometimes become corrupted after receiving content.
• NavTool may return incorrect "active" items.
• Button in Push Publish Workflow Action has a misleading label.
• Files under a folder are sometimes sorted incorrectly.
• Pushed Containers and Templates sometimes don't go live on the receiving site.
• getContentletVersionInfo() has inconsistent behavior.
• Log messages for OSGI plugins are displayed in catalina.out instead of dotcms.log.

To view more information on these issues, please visit the dotCMS Github repository.

Important Changes

A number of important changes in dotCMS 3.3 configuration may impact your performance when upgrading an existing site. Please read the documentation sections on all of these changes before upgrading from a previous release.

Configuration Changes

The following important changes have been made to configuration in dotCMS 3.3:

• Velocity Macro Caching
• The Startup Script Has Changed
• Cache Configuration Has Changed
• Quartz Job Scheduler MisfireHandler Log Error
• The log4j.xml File is No Longer Used

Behavior Changes

In addition, the following differences in default behavior in dotCMS 3.3 may change your expectations and require a review of your administration practices. Please review all of these changes before upgrading from a previous release.

• dotCMS Messages are No Longer Written to the Console
• The Log file Rollover Pattern has Changed

Velocity Macro Caching

dotCMS 3.3 uses a new cache chain which allows you to specify multiple cache providers in a chain. Due to the need to support cache providers which handle velocity code in different ways, velocity macros can no longer be cached to disk.

If you have velocity macros in .vtl files, the velocity macros will not render properly on some servers, or after a restart.

If your velocity macros are stored in files with a .vm extension you do not need to make any changes; dotCMS will automatically cache your macros to memory only, and your applications will continue to work normally.

If you have velocity macros stored in .vtl files or files with other extensions, you have two options to prevent problems rendering your macros:

• Move all velocity macros into separate files with an extension of .vm.
• Enterprise Only: Change the Velocity cache region chain (cache.VelocityUserVTLCache.chain) to use cache providers which store data to memory only.

Please see the Cache Configuration documentation for more information.

The Startup Script Has Changed

The dot.jamm jar has been upgraded to a newer version, so the command used to startup the javaagent for the dot.jamm jar has changed. If you have implemented any changes to your startup script, you will need to copy this change to your startup script to make the following replacement:

• 3.2.4:
  

  -javaagent:$DOTCMS_HOME/WEB-INF/lib/dot.jamm-0.2.5_1.jar

• 3.3:
  

  -javaagent:$DOTCMS_HOME/WEB-INF/lib/dot.jamm-0.2.5_2.jar

Cache Configuration has Changed

dotCMS 3.3 includes a new cache chain feature which enables Enterprise customers to configure and customize cache providers. As part of this change, the cache configuration properties have changed.  How you address these changes depends on your dotCMS license version:

Community Customers:

dotCMS 3.3 still uses the cache size parameters (cache.default.size, cache.adminconfigpool.size, etc.), but dotCMS Community Edition no longer uses a disk cache.  So if you have changed your cache .size parameters you must copy them to the dotCMS 3.3 properties file, but no other changes are needed (and you do not need to copy your cache .disk parameters such as cache.default.disk).

Enterprise Customers:

dotCMS 3.3 still uses the cache size parameters (cache.default.size, cache.adminconfigpool.size, etc.), so if you've made changes to these properties you will need to copy them to your dotCMS 3.3 configuration.  However the ".disk" properties (cache.default.disk, cache.contentletcache.disk, etc.) have been replaced with the cache chain properties, so if you've made changes to the cache ".disk" cache properties, you must reconfigure your cache using the new cache chain parameters in dotCMS 3.3.  Please see the Cache Chaining documentation for more information.

Quartz Job Scheduler MisfireHandler Log Error

When upgrading from an older version, you may receive an error in the logs related to the quartz job scheduler:

CST ERROR quartz.DotJobStore: MisfireHandler: Error handling misfires: Couldn't store trigger 'trigger16' for 'CleanBlockCacheScheduledTask'

Although this causes errors to be written to the log file, it does not affect system performance.

Resolution

The CleanBlockCacheScheduledTask is a system job, so it can't be removed from the Job Scheduler tab. To remove the errors from the log file, you must manually run SQL commands against your database to remove the CleanBlockCacheScheduledTask. The following SQL commands will remove the task causing the error (these commands can be used in all databases):

DELETE FROM qrtz_excl_cron_triggers WHERE trigger_name IN (SELECT trigger_name FROM qrtz_excl_triggers WHERE job_name = 'CleanBlockCacheScheduledTask');
DELETE FROM qrtz_excl_triggers WHERE job_name = 'CleanBlockCacheScheduledTask';
DELETE FROM qrtz_excl_job_details WHERE job_name = 'CleanBlockCacheScheduledTask';

Important Logging Changes

dotCMS 3.3 has been upgraded to Log4j version 2. As part of this update, a number of changes have been made to the logging configuration and behavior.

The log4j.xml File is No Longer Used

The log4j configuration file has been renamed to log4j2.xml, and moved to the ROOT/WEB-INF/log4j folder.

If you've made changes to the log4j configuration on your site, you may still use many of your changes, but you will need to merge these changes into the log4j2.xml file provided with dotCMS 3.3.  Please see the Logging Configuration documentation for more information.

dotCMS Messages are No Longer Written to the Console

To improve performance and prevent management issues with application server log files (e.g. catalina.out), dotCMS log messages are no longer written to the console. All dotCMS log messages can still be viewed in the dotCMS log files; please see the Logging Configuration documentation for more information.

The Log file Rollover Pattern has Changed

Rolled log files are now stored in date-stamped directories under the /dotserver/Tomcat-X.xx/webapps/dotsecure/ directory (or whatever directory is specified by your DOTCMS_LOGGING_HOME variable). This means that log files from previous days will not be automatically overwritten as they were in previous versions, giving you access to older log files, but requiring you to periodically remove older log directories (manually or through a scheduled task). In addition, for large sites with significant log activity, this may cause the log files to roll over more frequently than in previous versions.

If you wish to change rolling log file behavior to store all rolled log files in a single folder (e.g. .../dotsecure/logs/archive), edit the log4j2.xml file, replacing the string $${date:yyyy-MM} in all FILENAME_PATTERN properties with the name of the folder to store the files (e.g. archive).

Fixes

• Fixed a number of reported security issues (please contact dotCMS support for more information).
• Fixed several issues with push publishing.
• Fixed a number of issues with the Integrity Checker.
• Fixed some scroll bar issues in Edit Mode in IE11.
• Fixed an issue with incorrect dates set in a .ICS file downloaded from the Calendar.
• Fixed some issues managing files and folders in the Site Browser.
• Fixed an issue preventing newly created cookies from respecting the expire parameter.
• Fixed some ordering issues with Workflow sub-actions.
• Fixed some issues running dotCMS as a WAR in JBoss and WebSphere.
• Fixed an issue with Image Editor URLs.

New Features

• Added a Stop Reindex and Make Active button to allow you to safely stop the reindexing process.
• Added the ability to use external cache providers, including Redis.
• Added the ability to chain the cache, and specify different chains for each cache region.
• Added the ability to directly push languages and add languages to bundles
• Added a User Agent viewtool plugin.

Additional Changes and Improvements

• The Reindex feature has been improved to try all records once before retrying failed records a configurable number of times, ensuring failed records will not hold up the reindex process. In addition logging and notifications of index failures have been improved, and buttons have been added to display failed records and auto switch to the new index if needed.
• Right-click functionality in the WYSIWYG field has been disabled. We now rely entirely on the user's browser for functions such as spell check.
• The JUnit framework is no longer packaged with dotCMS. If you have a plugin that requires the JUnit framework, you must repackage your plugin to include it.
• The class com.dotmarketing.cache.StructureCache was refactored to com.dotmarketing.cache.ContentTypeCache. Any references in plugins or other locations should be updated to the new class name when upgrading.
• Updated the CMIS library to version 1.3.
• The HTML code returned for page redirects has been changed from 302 (moved temporarily) to 301 (moved permanently).
• The navtool Velocity viewtool now accepts a navigation level, allowing you to display the navigation from any level in the navigation tree.
• The jsontool and importtool Velocity viewtools now accept a timeout value to prevent long waiting requests from stalling threads.
• The StructureCache class has been refactored to the ContentTypeCache class.
• Any plugins or code which imports the StructureCache must be refactored to refer to the new class.

Fixes

• Added ability to add files without extensions via webdav
• Improved 1.9 to 3.2.4 upgrade
• Improved logging for the Migrate to Content functionality
• Cut/Paste fixed on Legacy Pages
• Added config to sleep during Reindex logic
• Improved the performance for File Assets when getting their dimensions 

Issue Tracker

• https://github.com/dotCMS/core/issues?q=is%3Aissue+milestone%3A3.2.4+is%3Aclosed

Improvements

• Improvements to Reindex Logic on Failures in ElasticSearch 

Issue Tracker

• https://github.com/dotCMS/core/issues?q=is%3Aissue+milestone%3A3.2.3+is%3Aclosed

Fixes

• Fixed issues with What's Changed functionality in Edit Mode 
• Fixed some issues with Publishing content in multiple languages 
• Fixed some major performance issues with legacy pages meaning pages that are not content

Things to Know

• dotCMS  no longer supports or provides the autoupdater. dotCMS 3.2.1 was the last release with it. For upgrading refer to this document http://dotcms.com/docs/latest/upgrade-dotcms

Issue Tracker

• https://github.com/dotCMS/core/issues?q=is%3Aissue+milestone%3A3.2.2

Improvements

• Updated Spring OSGi example plugin from Spring 3 to Spring 4.
• Made multiple performance improvements.
• Sessions are no longer required and load balancer stickiness is not longer mandatory for front end site delivery in a cluster (the admin functionality and multilingual sites where languages can be toggled still require sessions).  By default, dotCMS still creates sessions for clickstream tracking.  You can turn off automatic session cookie creation by setting ENABLE_CLICKSTREAM_TRACKING to false in your dotmarketing-config.properties.

Fixes

• Fixed the key value field losing data after rendering in the UI.
• Fixed the DB pooling parameters. The context.xml shipped with 3.2 still used Tomcat 7 specific properties which no longer work with Tomcat 8, specifically the maxActive configuration option has been renamed to maxTotal and the maxWait configuration option has been renamed to maxWaitMillis.
• Fixed issues with images in a multi-lingual site
• Fixed static plugin compiling issues with Java 7 and 8
• Fixed multiple issues with the Navtool Viewtool and Multiple Languages on Pages as Content

Things to Know

• dotCMS will no longer be supporting or providing the autoupdater. dotCMS 3.2.1 will be the last release that can be updated to using it.  Moving forward, clients should follow the straightforward upgrade steps found here : http://dotcms.com/docs/latest/upgrade-dotcms
• Everyone running stock dotCMS (meaning running in Tomcat not running as a WAR) and upgrading to 3.2.1 should copy the 3.2.1 context.xml to your Config plugin and replace your connection settings with the new, tomcat 8 specific settings. It has the updated Tomcat 8 datasource properties.
• JbossRulesFilter has been commented out in the web.xml by default.

Issue Tracker

• https://github.com/dotCMS/core/issues?page=1&q=is%3Aissue+milestone%3A3.2.1+is%3Aclosed

Improvements

• Ships with updated Tomcat 8.0.18
• Java 8 now supported. You can still run Java 7 if needed
• Added 3 new out of the box Actionlets for Custom Workflows (Email, Auto Set Value, and an Auto Push)

Fixes

• Fixed the random sort on pull contents
• Fixed some issues with Weblogic
• Fixed 3.x issue where the history code fields in containers was not saved

Things to Know

• If using the Autoupdater you will need to alter any plugins that reference Tomcat as the directory is different with the new Tomcat version. ie.. server.xml, ROOT.xml or context.xml

Issue Tracker

• https://github.com/dotCMS/core/issues?page=2&q=is%3Aissue+milestone%3A3.2+is%3Aclosed

New Features

• HTML Pages are now Content
• As content, HTML Pages enjoys all of the features of content.  i.e. workflow, versioning, multilingual support, pessimistic locking etc.
• To support all the new functionality Edit Mode has been improved on the left side controls when editing an HTML Page as Content including Workflow Controls
• Pages can now exist on the root of a host. This means that if you have a page on the root of your site you no longer need the Virtual Link to map your home page
• Pages no longer require a file extension
• Pages can have UTF-8 URIs.
• Added a more secure Viewtool to query the dotCMS DB directly. The new tool is mapped to com.dotcms.rendering.velocity.viewtools.SQLResultsViewTool@10e7a3fd. The old method in the CMSMacro Viewtool has been deprecated.
• All Enterprise versions have added the ability to pass Elasticsearch queries directly into dotCMS. This allows greater flexibility. The queries can be passed in via REST, directly to our APIs in Java, or from Velocity via a Viewtool
• There is a REST example of this using Geolocalization on the new starter site
• This opens the possibilities to using functionality like suggestion, facets, geolocalization etc…
• There is a new portlet that can be used to test queries

Overview of Fixes/Improvements

• Fixed front end issue where some special characters would use excess CPU cycles
• A number of security improvements have been made regarding how dotCMS manages cookies
• Timemachine will now copy dotAsset files
• The Activity Logger is now logging more events on content modifications
• Fixed an issue where the back end session would time out while logged in
• The CSV Content importer now accepts more date formats
• Updated to the latest ElasticSearch (v1.4.2).  This will necessitate a reindex on upgrade
• Front end forms have much improved multilingual support
• JGroups library was upgraded which should prevent a small potential memory leak when running a cluster
• Many fixes to the Integrity Checker and Push Publishing infrastructure
• Improved the SASS/LESS infrastructure.  There is still an issue with SASS on Windows which is still not working. https://github.com/dotCMS/core/issues/7236
• Improved the REST Portlet infrastructure for those creating portlets in the backend
• Fixed plugins broken from previous TinyMCE upgrade
• LDAP and WebDAV work properly now
• Date ranges are now supported from the Content Search Manager
• Worked with partner to allow them to run with Azure MSSQL database though Azure is not an officially supported dotCMS stack component

Things to Know

• Over 200 fixes have been made between 3.0 and 3.1
• This is the last release that targets Java 7.  3.1 will NOT work in Java 8.
• You can use the Autoupdater to update from 3.x to 3.1 but you cannot use it to update from any pre 3.0 release
• Because of the ElasticSearch upgrade, it is recommended that you run a full index after you upgrade dotCMS
• There is a new property in the dotmarketing-config.properties files named. CMS_INDEX_PAGE.  This setting controls what page is returned when the client requests a folder. (i.e. URL ends with “/”)  Its default setting in this release is “index” without extension.  If you are upgrading from a previous version, you should either change this to be “index.html” or you will need to change the name of the default file in every folder on every host.
• There is now a property named DEFAULT_PAGE_TO_DEFAULT_LANGUAGE which defaults to true. Because pages can now exist in many languages it is possible to hit a Page URI in a language that doesn’t exist. By default the dotCMS will return the page in the default language but render the content in the appropriate language. This mimics the functionality of pages and multilingual support in all dotCMS versions prior to 3.1.  If you set this to false then the dotCMS will return a 404 if the page doesn’t exist in the visitor’s language.
• ESAPI Filter is now turned off by default.

Issue Tracker

• https://github.com/dotCMS/core/issues?q=is%3Aissue+milestone%3A3.1+is%3Aclosed