Changing subdomain in Connections,CCM and Docs

I’ve just completed a process where a customer needed to change their subdomain due to an organisational name change. This site has a pretty large install.  CCM, Docs and also leveraged Windows Desktop SSO via SPNEGO.

The change was connections.OLD.sub.dom.au to connections.NEW.sub.dom.au

There is some documentation out there on how to change host names, but I wanted to compile a list for this specific task. So here you go.

Pre-tasks

SSL CSR created, new key created and imported into new KDB. Make available on IBM HTTP Server.

DNS change implemented, and points to existing IBM HTTP Server. Keep old record in place as well. Also ensure that the new record is resolvable on all Connections hosts.

Change the SSL Certificate over in IHS. (Change the following)

Keyfile D:\IBM\HTTPServer\NEWkey.kdb
SSLStashFile D:\IBM\HTTPServer\NEWkey.sth

CellDefaultTrustStore as it shared a common root with the previous Certificate.

Connections changes

Change the LotusConnectection-config.xml file. Check out with wsadmin, change the references to OLD and replace with NEW. Check-in. Process here

While in wsadmin, change the Notifications file to reflect new mail domain. Change the Administration user that notifications come from. Check-in. Process here

Change and update Blogs

To update the URL’s inside blogs, there is an AdminTask. while still in wsadmin, execute the following.

execfile(blogsAdmin.py)
BlogsAdminService.fixBrokenUrls(https://connections.old.sub.dom.au, https://connections.new.sub.dom.au)

 

Change LTPA to reflect new Domain.

In the ISC, select Global Security > Single sign-on

Update the Domain name field to reflect the new sub domain. i.e from .old.sub.com.au to .new.sub.dom.au

editor_image_cb34e16e-94cb-42a1-8921-1333db1dd095

Re-sync all nodes, stop and restart the entire environment (DMGR and NODES).

Docs Changes

Docs is pretty straight forward.

The property files for Docs all need to be changed. They are found in: <WAS_HOME>/profiles/<DMGR>/config/cells/{cellname}/IBMDocs-config/

Change all instances of OLD url to NEW in the concord-config.json file, and the viewer-config.json file. Just to be safe, verify no instances of OLD url exist in the other .json files in the directory. If there are, change them.

Re-sync all nodes, stop and restart the entire environment (DMGR and NODES).

CCM Changes

For CCM, we only had to update the Activity stream widget. Follow this process

 

Update Scheduled tasks

Jump back into wsadmin, and run the following. Following is from the knowledge centre.

Scheduler.listAllTasks() 
Scheduler.clearAllTasks()

Note: If Scheduler.clearAllTasks() does not clear tasks successfully, run clearScheduler.sql manually for each of the applications. 
For example:

db2 -v -td@ -f activities\db2\clearScheduler.sql
db2 -v -td@ -f homepage\db2\clearScheduler.sql 
The SQL scripts are in the following locations:

AIX or Linux: connections_root/connections.sql directory.

 

Update Search

I planned on updating search, but I didn’t get any errors after the change while using the search. Just to be safe,  I kicked off a new once off index task after the changes were completed.

Process.

SPNEGO SSO

 

…..and finally, a bit of a catch all

IBM HTTP Server rewrite – redirect OLD url to NEW

Add the following to handle any URLS that still come in using the old name. Replace your new server names below.

#Added to support redirect from OLD to NEW
<VirtualHost *:80>
    ServerName connections.OLD.sub.dom.au
    RewriteEngine On
    RewriteRule ^/$    /homepage     [PT] 
    RewriteRule ^/(.*) http://connections.NEW.sub.dom.au/$1 [R,L]
  
</VirtualHost>
RewriteEngine off

<VirtualHost *:443>
    ServerName connections.OLD.sub.dom.au
    SSLEnable
    RewriteEngine on
    RewriteRule ^/$    https://connections.NEW.sub.dom.au/homepage [PT]      RewriteRule ^/(.*) https://connections.NEW.sub.dom.au/$1 [R,L]    
</VirtualHost>
RewriteEngine off
SSLDisable

<VirtualHost *:80>
    ServerName connections.NEW.sub.dom.au
    RewriteEngine On
    RewriteRule ^/$     /homepage     [PT]
</VirtualHost>
RewriteEngine Off

<VirtualHost *:443>
    ServerName connections.NEW.sub.dom.au
    SSLEnable
    RewriteEngine On
    RewriteRule ^/$     /homepage     [PT]
</VirtualHost>
RewriteEngine Off

 

 

 

 

IBM Viewer / Docs error in C5.5 CR2

I’ve been a little quiet on the Blog front, but this year I’ll make an effort to update a bit more frequently. 2016 wasn’t a quiet year for me Technology wise, I got to work on some great Connections/Portal implementations and other environments. Apologies for the lack of updates.

I’ve been investigating an issue with Connections 5.5 CR2. and IBM Docs 2.0. What happens is that user will get an error while trying to view or edit the document. The users will get an error like the following.

"You are not entitled to use Docs or do not have permission to
edit this file. The file must be shared with you and your
access level set to editor. The file might also be locked by
another editor"

or

"The IBM Docs server cannot be reached. Please contact your administrator"

Errors in the Logs for Docs servers are similar to the following:

DocumentDraft E   Failed to generate snapshot for document. docID: 7125b890-4f8e-4a94-8fee-eb6bfc776802 com.ibm.docs.repository.RepositoryAccessException: Exception occurred with error code: CLFAD1000, message: Have no permission on this file in repository, and additional data: {"docUri":"7125b890-4f8e-4a94-8fee-eb6bfc776802","repo_err_code":"AccessDenied","repo_err_msg":"EJPVJ9070E: Unable to logon the user with the J2EE principal connectionsadmin.","repo_http_status":1000}

I’ve seen this sort of thing before after applying patches whereby you need to reapply the security to the J2EE principal and reserve the configuration. I tired that, and had the same issue.

Restarting the Files App seems to resolve the issue temporarily.

Appears that there is a known issue in CR2 and it’s being tracked in this APAR. (You’ll need an IBM Login)

RSA Premaster Secret Error in Connections 5

I’m doing an install for a customer at the moment, and it’s a large install. All on RHEL, integrated with Portal, and hosted in SoftLayer. Very Cool. I love playing with this stuff!

So the Connections install was running along great, until it was time to bring up the Connections Apps and test login. The way we usually configure Connections these days is to setup WAS primary administration user to exist in the LDAP directory, and use the default file based admin as a backup. We also then use this same user as the Connections administrator. We ensure that this user also has a profile. This just seems to make the installation (especially of companion products/extensions) easier..

I go to log into Connections the first time with the Admin user.. I get the “Unable to process your request page”. Damn.

Start troubleshooting. Eventually find the stanza that I think is logging for the error. This was in the Homepage Server System.Out. (Large Connections Install)..

 [14/01/16 19:30:20:532 EST] 00000120 HttpMethodDir I org.apache.commons.httpclient.HttpMethodDirector executeWithRetry Retrying request
 [14/01/16 19:30:20:535 EST] 00000120 HttpMethodDir I org.apache.commons.httpclient.HttpMethodDirector executeWithRetry I/O exception (javax.net.ssl.SSLKeyException) caught when processing request: RSA premaster secret error
 [14/01/16 19:30:20:536 EST] 00000120 HttpMethodDir I org.apache.commons.httpclient.HttpMethodDirector executeWithRetry Retrying request
 [14/01/16 19:30:20:539 EST] 00000120 UserInfoInter E com.ibm.lconn.homepage.web.interceptor.UserInfoInterceptor cacheUserInfo CLFRQ0341E: Could not retrieve details for the user with login ID: ConnectionsAdmin@domain.blah.au due to an exception. The exception occurred when retrieving the details via Profiles Directory Service Extension: [Ljava.lang.Object;@5c0f8907

More digging revealed that this error was due to…….the SSL Key.

What was occurring is this. I’d setup the IBM HTTP Server to use the key provided by the customer. I’d imported this into the CellDefaultTrustStore, as required. It was 4096 bits wide, which Java security in the WAS stack had an issue with. Some secret squirrel stuff about the governments wanting to control Encryption or something. So when the Homepage app was hitting the URL for profiles and verifying I am who I say I am, it caused the SSL error.

How did I fix it?

Copy and replace the Java policy files with the unrestricted policy files, then restart the Connections servers.

cd /opt/IBM/WebSphere/AppServer/java/demo/jce/policy-files/unrestricted/
[root@connwas2 unrestricted]# cp *.jar /opt/IBM/WebSphere/AppServer/java/jre/lib/security/
cp: overwrite `/opt/IBM/WebSphere/AppServer/java/jre/lib/security/local_policy.jar'? yes
cp: overwrite `/opt/IBM/WebSphere/AppServer/java/jre/lib/security/US_export_policy.jar'? yes

That’s it. Restart servers and enjoy.

Thanks to Mikkel Heisterberg,  you pointed me in the right direction with this Blog. Cheers!

 


					

iKeyman: How to receive an online CSR.

I have a love/hate relationship with iKeyman. It never seems to want to work the way I think it should.

Take the instance when a customer manages their SSL keys online, and creates the private certs without actually creating a CSR on the host machine. It just gets tricky. iKeyman won’t allow you to import the private key, and if you try to install the new certificate, you’ll get an error along the lines of “No CSR found for Certificate”

What you can do is use 3rd party tools to create a .p12 certificate, then import this into a new .kdb.

So here is my cheat sheet for how I got this working.

  1. Extract the certificates from email.
  2. Use OpenSSL to create a p12
  3. Create a new KDB
  4. Import the root certificates, intermediate certificates
  5. Import the Personal certificate

Copy the certificates from the email you have been provided. In my case I had the new trusted cert, the Intermediate and I had to get the Private key as well.

Copy the text certificates from the email and create a file with the extensions like the following for each. In my case I had 3, NewCert.key, NewPrvKey.crt and Intermediate.cer.

They will look something like below.

PrvKey

Once you’ve done this, you’ll need access to OpenSSL. I’m on a Mac, so this is installed already.

OpenSSL is accessed via Terminal.

OpenSSL will be used to create a .p12 file so we can import into iKeyman. Use the following command.

openssl pkcs12 -export -name "NewKey" -inkey ~/Desktop/NewPrvKey.crt -in ~/Desktop/NewCert.key -out ~/Desktop/NewCert.p12 -keypbe PBE-SHA1-RC2-40

You’ll be prompted for a password for the .p12 you are creating. Remember this, you’ll need it.

Ok.You’ve now got a .p12 . Transfer this to your IHS server, along with the Intermediate.cer that you created previously.

Fire up iKeyman, I’m running this on my IBM HTTP Server.

Now we are going to –

  1. Extract the existing trusted root and intermediates from the old KDB (optional)
  2. Create a new KDB
  3. Import the Trusted root and intermediates if you need to (optional)
  4. Import your private keys

 

Import the Intermediate Certificate, and also if you required to pull in the other Root Certificates. I’m using TrustiCo so this is the following. I had these already in another keystore, so I just Extracted and then imported from the .kdb I was replacing.

2015-11-17_15-32-29

Create a new .kdb, I’m calling mine 2015Key.kdb.

2015-11-17_15-23-04

Make sure to stash the “Password to a file”, then hit OK.

2015-11-17_15-50-33.png

Now select “Personal Certificates”

Select “Receive”

2015-11-02_13-01-01

Change the Keyfile type to PKCS12 and browse to our .p12 certificate.

2015-11-17_15-55-03

Enter your password for the .p12, then change the label to something that makes sense. Hit Apply and OK

2015-11-17_15-57-21.pngThis will import the new certificate for your server into the KDB file. If it is the only one, it should automatically be set as the default.

Once done you can then Validate trust (Personal Certificates>Validate). If everything is working, then it should give you a nice “Validation successful” If not, you more than likely don’t have the correct Trust Chain. Go to the vendor of your 3rd party cert and get the correct signer certs.

Then all that is left is change the IHS server config to point to the new key and stash (.kdb, .sth). Restart IHS and you should be good to go..

 

Sametime 9 Managed Settings – HTTP port problem

Just a quick update. I’ve recently been doing some work trying to Auto magically configure Sametime using Managed Settings. This has been a bit of a struggle, but we are getting there.

If you’ve not heard of Managed Settings before, they are a way that you can push out and update Sametime Connect (both embedded and standalone) settings. Here is a good overview.

One caveat that we ran into is with the update site that is pushed out via policies must be a http server running on port 80.  Once we enabled tracing on the Sametime Connect clients we were able to get more infiormation on what was going on.

<globalInstanceId="ELc0a8c93700014fac6b9c030000000b" msg="CWPMS0202E: The Managed Settings File Provider was unable to open the URL http://sametime.domain.com:88/sametime/updates/managed-settings.xml due to the following error: Forbidden." severity="50" version="1.0.1">
 <extendedDataElements name="CommonBaseEventLogRecord:level" type="noValue">

We tried everything to make sure that the file was being served correctly. You could access the URL and the managed-settings file was loaded via a browser.

The Workaround

Relocating the managed-setting file on another HTTP server appeared to resolve this issue. We were then able to use the managed settings and push out our changes.

IBM Mail Onboarding Manager – Quick Overview

One of the other technologies I get to play with are the on-boarding tools to IBM SmartCloud. If you didn’t know, the only way that you used to be able to onboard to the cloud with mail file transition from Domino was to engage a BP who has successfully passed the certification.

These tools are known as Onboard Planning Tool (OPT), and Onboard Transition Tool (OTT).

IBM has now release a self service tool called Mail On-boarding Manager. I refuse to call it MOM.

To use the Mail Onboarding Manager, you still need to do all of the steps to setup a Hybrid environment.

More information on the Tool can be found here.

Below is a quick Video overview.

If you like to learn more, my colleague Nick Young and I will be presenting at Inform 2015.

Hope to see you there!

Quick Tip – Name your console..

The guys I work with are full of good tips.

I’d always wondered how to do this, as I thought it was some sort of Developer jiggery-pokery.

Most environments I work in have a Development environment, and a Production environment.

Quite often, I get lost. Am I in Prod or Dev? Especially with cryptic server names.

Quick way to fix this is to Name your Consoles.

  • Logon to your ISC.
  • Expand System Administration > Task Management
  • Select Console Identity.
console1
  • Type in the Console name..ie Prod
console2
  • And hey presto…You know where you are..
Console3

You’re welcome.

CCM CR2 Upgrade/Backup Script errors – Access is denied.

Previous post was how to install CCM. This post will be about an error that I experienced just trying to apply the Fix Pack to get it to CR2 level.

When applying the CR2 update, the Content Engine Server is the first cab off the rank.

Following the instructions, I started with the D:\IBM\Connections\ccm\ccm\ccm\scripts\backup.bat

This script failed to run. Error was ADMN0022E: Access is denied……..because of insufficient or empty credentials.

*sigh*

I checked that the password wasn’t locked out. It wasn’t.

I logged into wsadmin manually, jut to verify all was good. I could.

So google put me onto this awesome post. In this case, wasn’t my issue, although I did try the fix.

Time to dig deeper.

I grabbed the backup.bat file, and scanned though till I found the part that was failing.

I enabled output to the screen by adding echo on..

2015-04-21_14-03-59_01

I then ran the backup script again. This revealed what the issue was.

The customers WAS admin password contains an exclamation point “!” for password quality. This script was passing this password as a variable to the the script which was running the wsadmin command. There appears to be a second time that this script calls the %was.admin.passwd% variable, but as it does it strips the “!” in my password. So I’m supplying the incorrect password, even though it’s correct at the time of input.

I’ve not been able to find a resolution to this apart from changing the password.

I had tried escaping the character out, and used different variations but to no avail. If anyone knows how to do this, please let me know.

CCM Installation fails with CLFRP0038E – Connections 5 CR2

I ran into this issue at a customer site I’m working on at the moment.

Here’s the scenario. Installing the following: Connections, Survey’s Polls (FEB), Social Mail, External Access, CCM, File Viewer. I’ve installed Connections, updated to CR2 and verified all is working. I’ve also installed FEB.

Following this documentation, I ensure I have all of the CR2 required files for install, including the fixpacks extracted to a directory. I’m installing CCM across 2 nodes in a cluster, on separate hosts.

I wanted to make sure that the Clusters/Servers were setup correctly in WAS. I have not had much luck with installers creating WAS clusters or servers,  so as this is pretty straight forward I like to set them up for installations to use.

Below is the summary screen of the CCM_Cluster.

2015-04-15_8-48-02

Time to install.

I’ve checked the prereqs. Note- Make sure you have at least 6GB free in %Temp%. In my case this was located on the C:\

I select the Modify option.

Select Add-on features IBM Connections Content Manager

2015-04-15_8-56-51

You’ll be asked for your existing WebSphere credentials. Put them in in then select Validate.

You’ll then hit this screen

2015-04-15_8-58-47

I selected New Deployment

Left remaining blank, then selected the Directory where I extracted all the installers and FP’s

If you’ve extracted everything out, you should get a Validation Successful. If not, check what files you are missing. The installer was pretty good in identifying what packages I was missing.

I then selected where I wanted to deploy CCM. I select my Servers/Cluster that I previously setup.

2015-04-15_8-59-09

Once this was selected, got to the next screen which I selected Modify.

CCM then installs.

I got the below error.

2015-04-12_8-43-02

Damn…

Looking through log files got me this..

2015-04-17_15-21-53

Further searching through the logs at  <WAS INSTALL>\profiles\Dmgr01\ConfigEngine\log\ConfigTrace.log reveals that a properties file for the nodes/cluster I’d setup did not exist in the D:\IBM\Connections directory.

[echo] Communities server: Cluster1_server1
 [echo] Loading properties from file: D:/IBM/Connections/Cluster1_server1.properties
 [echo] Communities server HTTP Port: 9082
 [echo] CE server: CCM_server1
 [echo] Loading properties from file: D:/IBM/Connections/CCM_server1.properties
--- Exception Thrown ---
D:\IBM\Connections\ccm\ccm\ccm\config\includes\ccm_cust_cfg.xml:1200: Source file does not exist.
at org.apache.tools.ant.taskdefs.LoadProperties.execute(LoadProperties.java:159)

This is where it got weird. I’d thought that is was because I’d setup the nodes/cluster outside of the install and that it was the IBM Installation Manager that created these properties files. Nope. I tried by setting up the nodes/clusters using the installer, same result, CCM not installed, same error.

Fix? Workaround?

To fix this one I created the files manually. This required grabbing the following values from my ISC in WAS.

2015-04-17_15-28-40

I created 2 files, one for each of my servers.

2015-04-17_15-28-56

These files were created before I hit Modify on the final installation screen, i.e after you’ve done all the above. I’m pretty sure that you could do this before you ran the installer.

Ran the installer again and was able to install.

2015-04-17_15-40-41

Hope this helps someone.

Profiles population, helpful search filter (MS AD).

I’m currently installing Connections 5 for a customer. It’s a great place as the onsite guys really know their stuff. Anyway, their LDAP is in a bit of a state. Typical example of an organisation that has grown, gone through many Active Directory upgrades and no hard and fast rules for user management.

For Connections to be useful, you need relevant and current users in the Profiles database.

With populating the profiles DB, you’ve got 2 types of people. You’ve got the Wizards, and the Command lines. I’m a Wizard guy, just because it has come along way since I first used it (2.5) and hey, it automates half of my job. I still like to know and understand what the Wizards are doing, but if they automate it, awesome.

So I’m doing this install, and I run the Wizard with the default search scope.

The default search filter I used was below. I wanted to see how many records…

(&(sAMAccountName=*)(objectClass=user)

As you do, I iterate the log file..

LFRN0027I: After operation, success records is 634, duplicate records 0, failure records is 366.

That’s interesting…these guys only have 400 users?

Opened the PEOPLEDB, and there were heaps..I means heaps… of deactivated users. Doesn’t look like a user account had been deleted. Ever.

So, reset everything, dropped the DB and recreated it.

I had to work out how to populate without syncing accounts that were disabled. I also wanted mail to be populated.

Here is the search string that did it..

(&(sAMAccountName=*)(objectClass=user)(mail=*)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))

The parameter (!(userAccountControl:1.2.840.113556.1.4.803:=2) is a feature with MS AD, and is called a bitwise filter. Essentially, we’ve selected the users with the  useraccountcontrol flag enabled. This link explains it better than I ever could.

After operation, success records is 406, duplicate records 0, failure records is 24, and last successful entry is

This worked, users synced dropped to the correct numbers. Happy days.