Monday, 26 March 2012

What's the deal with the CLM Public URI?

Far too many times I have seen the mistake where a customer has installed one or many of the CLM applications and either failed to set the Public URI or set the Public URI to something that is not appropriate, such as a host name tied to the network infrastructure.

It is important to give thought beforehand about how you would like your clients to access your CLM links and applications. This means, that a scalable URI design needs to be created (to allow more applications to be added) even before the first CLM application is deployed.

Why is it important? Its important because the URI is the de-facto unique identifier that customers and integrations will use when linking to CLM data. Any change to this URI (due to a host name changing for example) can have implications to not only links embedded in emails and bookmarks but other application integrations (including the CLM integrations RM, CCM and QM).

You generally want a URI that reflects the static business meaning of the data that is being linked to. For example: is good, is not because it identifies both the hardware hosting the application and the port.

Some customers have asked why we cannot have the CLM application figure it out itself (like the 2.x products did). Yes, the 2.x era products allowed this and, yes, resulted in unpredictable link behaviour. (It is not allowed any more.)

Here is the problem : There is no reliable way of getting the host name of a machine because the machine itself may not know what it is (if its behind a proxy for example.)

If no public URI is set,  an attempt is made by using something like:


What does this return? Well.. as Java is the CLM implementation technology. it seems that this is determined by the rules specified here

Reverse lookup behaviour is really determined how the network and DNS is configured.  An example sequence to determine the URL could be:

  1. If Public URI is set then no reverse lookup needed the public URI setting is used. end.
  2. If public URI is not set, the host name must be determined through getting the IP address for the primary network device then name resolution will do.
    • look in the host file
    • DNS Reverse lookup
Some problems with this:
  1. If server IP address is not resolvable on the server itself (either in hosts file nor lookup) then an exception is thrown and IP address is used.
  2. If DNS is misconfigured, IP address can return as host name
  3. Some other network misconfiguration/bug is causing getLocalHost call to return
  4. Which host name? Which network device? There could be many combinations.
Even if this worked, it is still not good enough!

Why? Simple example exercise.. I want to host RTC on my work laptop, and i don't know what my public URI should be.

Let me try to find out.

From hostname command:

[kuschel@tpborisk7 conf]$ hostname

Cool.. let's do an IP lookup on the host name:

[kuschel@tpborisk7 conf]$ nslookup

** server can't find NXDOMAIN

No good. This is because i do not have a DNS entry for my laptop's host name.

Let's try my NIC IP Address?

[kuschel@tpborisk7 conf]$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr F0:DE:F1:58:9B:7D 
          inet addr:  Bcast:  Mask:
Do a reverse lookup on the IP:

[kuschel@tpborisk7 conf]$ nslookup

** server can't find NXDOMAIN
Also not good.. DNS has no entry for this IP.  Even if it did, it would be purely local to the subnet! (192.168.x.x).

My Wireless NIC? Resolves locally on my home network but not publically.

Ah! Thats right, I am using VPN:

Now what is my ip address (I am using a VPN tunnel)?

[kuschel@tpborisk7 conf]$ ifconfig tun0<
          inet addr:  P-t-P:  Mask:
What is the hostname that corresponds to this IP?

[kuschel@tpborisk7 conf]$ nslookup

Non-authoritative answer:    name =

Authoritative answers can be found from:    nameserver =    nameserver =    nameserver =    internet address =    internet address =    internet address =
Bingo! So I should use as it seems to be publicly addressable but its simply a bad public URI (machine name plus location).

So how is CLM supposed to figure this out? My laptop has have 10 network devices, some with resolvable names, some not.

What is also key here is that neither my NIC nor my host name is the real public address!

One cannot expect the CLM applications to figure this out as there is simply not enough information. (It could even use something like an IP address if it cannot resolve a name.)

Hence the need for a Public URI!

Put the same thought behind the CLM public URI as you would for a URL you would host pages or application for the general public on the internet. It should be thought of,designed and owned by CLM!

Luckily, For people who are in the unfortunate predicament of having a bad public URI, there is hope in 4.0 with server rename.

Sunday, 4 March 2012

Automated builds using Visual Studio and Rational Team Concert

This is the third part of a blog series on using Rational Team Concert (RTC) with Visual Studio (VS). In part 1, I showed how to set up a basic solution in visual studio integrated with Rational Team Concert. In part 2, I provided a working OSLC sample which we will use in an automated build. We will set up an automated Jazz Build for Visual Studio using using the Jazz Build engine.

In order to do this you will need (in addition to the those required for the previous two parts):

-The Jazz Build Engine
-Rational Team Client for Eclipse

You may wonder why you need the eclipse version of Rational Team Concert. While you can, request builds, examine build status and see build results from visual studio, you can only create build definitions with the eclipse client at this time. So you will need to unzip the client  launch eclipse and connect to the project area to do this. To understand how to do this you can take a look at the CLM information center (Hint: Go to Team Artifact view and create a repository connection)

To keep text editing to a minimum I will provide the visual studio solution which has all the files required to perform an automated build. You can find it here. Unzip this over the corresponding directories in the solution created in the first two parts or if you are starting from here just unzip this and add it to the correct components (as specified in the first part) in RTC.

This is what should be done:

1. To keep things simple we will set up the automated build on the same machine as the visual studio.  Install the Jazz Build Engine (JBE) in the default directory. In my case, this is the C:\Program Files (x86)\IBM\TeamConcertBuild directory. You will see that all property files are configured with the assumption that the build engine is installed here. 

2. To, once again, keep things simple, we will use the current user (the one you used to log into visual studio) as the build engine user. The first thing we need to do is create a workspace which will be used by the build engine to do the build. Go into Visual Studio and create a new repository workspace from the "OLSC Stream". Right click on the OSLC Stream in the project area and click "New Repository Workspace..."

Call the workspace "OSLC Build Workspace" but do not load the workspace when asked as it will solely be used for build 

2. At this point we will need to prepare some files in order to facilitate the build. We will be using the MSBuild process to perform our build and we will need to augment it so that it communicates with the jazz repository once it starts and finishes.  MSBuild uses the .proj files to build and we need to modify this file to add some pre and post-build hooks. To do this we would need to change the "OSLC Consumer.proj" file. To modify this file from visual studio it needs to be unloaded and edited. This can be accomplished by right clicking the project in visual studio, click unload and then subsequently "Edit OSLC Consumer.proj". 
Navigating to the end of the file you will see that the following has been added towards the end of the file:

<Import Project="..\OSLCProperties.xml" />
  <Target Name="BeforeBuild" Condition="'$(fetchDestination)'!= '' ">
    <Exec Command="$(AntCommand)-DactivityLabel=startActivity $(propertyFile) $(jazzbuildxml) startActivity" />
  <Target Name="AfterBuild" Condition="'$(fetchDestination)' !='' ">
    <Exec Command="$(AntCommand) -DactivityLabel=publishActivity $(propertyFile) $(jazzbuildxml) publishActivity" />

This uses an OSLCProperties.xml which contains the properties that are used here: $(AntCommand),  $(propertyFile) and $(jazzbuildxml). The $(fetchDestination) will be passed into MSBuild using a command line argument defined in the build definition. (We will do this later).

You will find the OSLCProperties.xml in the solution directory (the root directory of the work space and contains the following. You may need to adjust the JBE root directory to reflect your environment.

<Project xmlns="">
    <AntCommand>"C:\Program Files (x86)\IBM\TeamConcertBuild\jre\bin\java.exe" -cp "C:\Program Files (x86)\IBM\TeamConcertBuild\buildsystem\buildengine\eclipse\plugins\org.apache.ant_1.7.1.v20090120-1145\lib\ant-launcher.jar" -cp "C:\Program Files (x86)\IBM\TeamConcertBuild\jre\lib" -lib "C:\Program Files (x86)\IBM\TeamConcertBuild\buildsystem\buildtoolkit"</AntCommand>
    <propertyFile>-propertyfile "C:\Program Files (x86)\IBM\TeamConcertBuild\buildsystem\buildengine\eclipse\"</propertyFile>
    <jazzbuildxml>-f $(fetchDestination)\OSLC\jazzbuild.xml</jazzbuildxml>

The last file needed is the actual ant file that contains the tags used to communicate with the jazz repository, the jazzbuild.xml file. It contain two activities, one that is launched after the build, "startActivity" and another that is lauched after the build, "publishActivity" that is used to publish the build result. (They are launched from the msbuild tool using the hooks in the .proj above that it uses to perform the build.)

All of these new files should be checked in and delivered into the "OSLC Stream" so that they may be used by the Build engine.

3. Now lets create a build engine and build definition in Rational Team Concert for eclipse. Launch RTC for eclipse and create a "Jazz Build Engine" build definition for the OSLC project area by right clicking on "Build Engines" and give it the name "OSLC Build Engine".  

You may now see a warning that the build engine is not running:
This is a good indication that the build engine is not running. Lets start it now.

4. To, once again, keep things simple, we will use the current user (the one you used to log into visual studio) as the build engine user. With that in mind execute run the build engine. You should see command prompt output as follows:

jbe -repository https://<rtc host name>:<rtc port>/ccm -userId <rtc build username> -pass <rtc build user password> -engineID "OSLC Build Engine"
2012-03-04 10:29:29 [Jazz build engine] Running build loop...
2012-03-04 10:29:29 [Jazz build engine] Waiting for request...

Now refreshing the build engine page in RTC should eliminate the error.

5. Next we need to create a new build definition called "OSLC Build" for the OSLC project area of type "Jazz Build for Microsoft Visual Studio Solution - Jazz Build Engine"

Ensure that you select "Jazz Source Control" as a pre-build option:

Select MSTest Configuration as a post-build option.

Once you created the build it should be opened and now you must add the build engine we created previously as a supported build engine in the Overview tab.

In the "Jazz Source Control" tab we must specify the workspace we created previously for the purposes of building as the Build Workspace and also specify a directory that will be used to load this workspace during the build process called "OSLC Build Workspace". We should also specify "Delete directory before loading".

Now we must specify some details about the build in the "Microsoft Build" tab. Depending on the install path of your visual studio some of these may need to be adjusted according to your environment:

Some things to note on this page. The file is a file generated by the JBE that contains the properties used during a build. You also see that here we are passing in the $(fetchDirectory) property that we are using above in the .proj hook. We are using MSBuild as it is more suitable for what we want to do but it is also possible to use the Visual studio "devenv.exe" to do the build. This is usually more useful if there are some visual studio integrations that the build depends on (pre compilers, etc.).

The next tab "MSTest Configuration" is also given here but is unused at the present time and is provided for a future lab. It must nevertheless be completed in order for the build to succeed.   You may need to change the paths as appropriate:

Now you should be able to save the build definition and then you are done with RTC for eclipse.

6. Go back to visual studio and refresh the builds area and now you should be able to request a build from here. You may do so now. Leave the default options and submit.

7. After the build request has been submitted you will see it in the build queue. It may take a little while to build and return eventually with a "Completed" status and a green check mark.

8. Whether the build succeeded or not you can double click on the build to see the results, including the logs and, if successful, a link to the resulting .exe in the "Downloads" tab. If something went wrong you can examine the logs to see what the issue is.

Voila! An automated build using the JBE and visual C++!

Saturday, 25 February 2012

Configuring slapd (LDAP) for the CLM "Money that Matters" scenario in Websphere Application Server

Over the last few years i have found myself doing something over and over again and at the same time constantly going back into all sorts of documentation in order to figure it out: How to set up and configure a basic ldap server for a Websphere CLM configuration.

Its not difficult but many things can go wrong and there are some configuration files that are needed to get a basic LDAP server going that has enough information (users and groups) to be able to create a basic functioning system.

In my role, I am constantly bringing up and tearing down topologies and in many cases I have to set up enterprise level systems where there are multiple instances of Websphere Application Server (WAS) servers (or WAS Network Deployment (ND) nodes) each hosting a product (CCM, RM, etc..) linked with each other.  All of these instances should share the authentication realm so that's where LDAP comes in.

Since the "Money that Matters" (MTM) CLM lifecycle sample that is packaged with CLM is a good starting point for test environments, proof of concepts and demos I use it a lot for sample data. 

In this blog, I will show how to set up a basic openldap server for a WAS server that will host the MTM sample. I won't go into the CLM details as those are well covered in the information centre.

As I am using Red Hat Enterprise Linux (RHEL) 64-bit (in this case 6.2)  most of the time to set up servers, these instructions pertain to that particular flavour of Linux. This should work with other Linux, Unix variants and also Windows (with some twists i am sure).

What you need to do

Login in as root (or use sudo on all of the following commands, your choice).
Use yum (our vanilla rpm) to install these rpms:

Make sure that your firewall (iptables) has port 389 open. Add the following line to your /etc/sysconfig/iptables:

-A INPUT -p tcp -m tcp --dport 389 -j ACCEPT

Start the slapd service, you may also want to start slapd on system boot:

service slapd start
chkconfig slapd on

By default, the RHEL installs slapd in such a way that the cn=config slapd configuration realm is only accessible from root. You can verify that slapd is running by issueing the following command:

ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -b cn=config dn

This should list the configuration parameters for the ldap server. We don't need to change the defaults so lets continue to create our new ldap database for the MTM sample. I have created two ldif files for this purpose. The first one, jazzdb.ldif creates the ldap database, and the second one jazz.ldif creates the users and groups for the MTM sample. 

Download jazzdb.ldif and jazz.ldif into a known location. You will need to modify the jazzdb.ldif file with a known password for the administrator. You can generate the password using the following command:


The utility will ask you for a password that you will need to confirm and generate an encrypted value that you will need to replace the current passwd with. So, replace the "olcRootPW: {SSHA}k7U/qvqrBs8T63fJnFa4+4V9S+ecp7Rg" value in the jazzdb.ldif file with the value that was generated by the slappasswd tool. Make sure not to leave any unwanted symbols after or before the line and note that there is a space after "olcRootPW:".

You may also want to customize the olcSuffix ( to something related to your domain) but be careful to search/replace this suffix everywhere in both the jazzdb.ldif and jazz.ldif files.

Once you have replaced this value you can go ahead create the slapd database:

ldapadd -Y EXTERNAL -H ldapi:/// -f jazzdb.ldif

Once this has completed successfully you can test both the new login you created for the RootDN and import the MTM users at the same time using:

ldapadd -x -D uid=servadm,ou=Users,dc=jazz,dc=net -W -f jazz.ldif

The password it asks is the one you used for the slappasswd generation utility. 
After this is complete, you can change the passwords of other users in the database by using the following command (for bob as an example):

ldappasswd -x -D uid=servadm,ou=Users,dc=jazz,dc=net -W -S "uid=bob,ou=Users,dc=jazz,dc=net"

If all of this worked so far then you are basically done setting up the slapd LDAP server and can now configure WAS to use LDAP.

I you have installed was you have either set it up to use a user name and password or with no security at all. In any case, login into the WAS administration console and go to the global security section. (We are using Websphere Application Server v8 but this is almost identical on other versions.)

Once the global security page comes up you should "Enable application security".

Then go to the "User account repository" section and choose "Standalone LDAP registry" as the realm definition and click "Configure..." (between 1 and 2 below):

Now you need to set up the LDAP settings as below. Replace the value of the distinguished names to reflect the value of the suffix, if you changed it, in the ldif files. Use the bind password of the LDAP administrator user for the database we just created.
NOTE: I am using localhost as the server name for LDAP. If you have a Websphere Network Deployment with multiple nodes, this is not a good idea and you should use the actual host name for your final configuration.

 Click Apply and Now click on:

Use these values in the form:

Click OK on this form and the subsequent form. In the next form ensure that the realm is set to LDAP and that the "Set Current" button is click and "Apply" is clicked.

WAS will not ask you to save and reboot which you should do (it may ask for your original WAS admin username and password when shutting down.). Once WAS has restarted you should be able to login with the "servadm" username and LDAP administrator password.
If the authentication fails you can try to change the  ldap password of servadm using the ldappasswd command above (use servadm instead of bob). 
If this still fails and you cannot log into WAS, make sure slapd is running and if all else fails take a look inside the security.xml file of WAS to ensure your settings are correct. This file can typically be found in:

<WAS Installation Dir>/profiles/<profile name>/config/cells/<cell name>/security.xml

If you change it, you will need to reboot WAS.

At this point, WAS is set up and has all the users and groups required for a clean installation of the CLM and creation of the MTM scenario.

For more information on where to go from here, look at the information center on how to install CLM into WAS.

Sunday, 5 February 2012

C# with Visual Studio and Rational Team Concert (OSLC revisted) (Part 2)

In Part 1, I showed how to set up a basic visual studio environment which we will now use to create an OSLC client. The client we will create is quite rudimentary and is based on the code in the OSLC workshop (which is itself, java.) 
The code does not do much more then a form-based authentication with RTC and fetches a list of project areas using XPath.  The .NET framework (I am using v. 4) has everything we need to create a connection with RTC, authenticate and query the content. It's just as easy to do with C# as it is in Java and the code, in fact, is quite similar. I am using Visual Studio 2010 and .NET v4 but the code should either work as is, or with some slight modifications, on other versions.

You will need to get the point we finished in Part 1. Once you are there, let's start fresh and close down Visual studio and unzip the following zip file into your sandbox (overwriting along the way.) 

VS/RTC sandbox: C# OSLC

Open up visual studio and immediately go to the RTC "Team Artifacts" window. Go to the sandbox list Right click and select "Set current" to activate and open the solution in Visual Studio. You can go ahead and go to the "Solution Explorer" and delete the default Program.cs and Class1.cs files that the wizard created. You can the add the new .cs files by your the project context menus and Add->Existing Item. You can then add the missing files (OSLCConsumer.cs and HttpUtils.cs) from there.

You can also go ahead and deliver these changes by going to the "Pending Changes" Window, and check-in and deliver all the new changes.

You can to build and run the project at this point by clicking on the "Play" button as per normal Visual Studio usage but you will probably need to modify a few settings namely:

string server = "https://localhost:9443/ccm"; //<-Your RTC server
string url = "https://localhost:9443/ccm/rootservices"; //<- the rootservices URL
string login = "ADMIN";    //<-Your RTC username
string password = "yourpassword";  //<-Your RTC password

Once you have looked over the code and perhaps run it, it demonstrates the following:
  1. How to do form authentication with C#
  2. How to issue OSLC requests using C# 
  3. Use XPath to fetch Project area names (service providers)

If everything is working file, the output window could look like this:
If you need to have more in depth information about how this code functionality works I suggest to take a look at the OSLC workshop as this code sample is loosely based on it. Enjoy!

Next up (part 3)... I will expand on this example, and create an automated build environment for our new code using RTC SCM and automated builds!

Sunday, 22 January 2012

C# with Visual Studio and Rational Team Concert (OSLC revisted) (Part 1)

I have had some recent customer experiences which have impelled me to write some blogs on the RTC Visual Studio integration. This topic follows the previous series well, where I wrote about RTC and CDT. In fact, I will will basically recreate the OSLC example ( this time using C# ) but using the Visual Studio 2010 development environment instead. Meaning that I will be using visual studio not only for developing but writing unit tests and doing a static analysis all while using the RTC visual studio integration.

If you recall in the previous blog series on OSLC, I created an OSLC client in C/C++ using CDT. I could do the same in Visual Studio (OSLC client using C/C++, that is) but I will instead use C# to create the OSLC client this time to show how this can also be done.

What You Need

This exercise may work with other versions of Visual Studio as the integration supports other versions as well but I will use Visual Studio 2010 .

I am also assuming that these are installed as well as:

Ensure that the CLM servers are setup correctly (at a minimum, CCM and JTS). Please refer to the CLM information center for how to do this. For reference my URLs and directories area as follows, you will need to replace them in this blog as appropriate for your environment:

JTS URL: https://tpborisk7:9443/jts
CCM URL: https://tpborisk7:9443/ccm
CLM install directory: C:\IBM\CLM3012
Sandbox: c:\vsworkspace
Setting up (creating project areas and solutions)

1. If you have installed your visual studio environment correctly, you can launch Visual Studio now. The Visual Studio RTC integration behaves very much like the Eclipse RTC integration. There are some key differences.

2. One of the differences is that it is not possible to create a CCM project area from visual studio. This will need to be done through the CCM web UI. Go to the project area creation page: https://tpborisk7:9443/ccm/

3. Create a new project area called OSLC and use the scrum template. Be sure to assign the user (in my case servadm) you will be connecting from Visual Studio as the “Scrum Master”.

4. Now connect to the project area using the Team Artifacts window:
5. Connect to the server:

6. Click next and select the OSLC project area we created and click "Finish". We are now connected.
(NOTE: You may need to "Load.." your repository workspace into your sandbox at this point in order to see the "Share solution in Jazz" option below.)

7. Lets create a new solution that we will use to develop our c# OSLC client.In Visual Studio select

8. Select a Console application under Visual C#->Windows->Console Application

9. Call the project OSLC Consumer and use a solution called OSLC (as we will created more projects in the future) in a preferred location. Make sure to select the "Share solution in Jazz" option.

10. Once you confirm your solution location you will be asked to select a sandbox. You should choose the sandbox location that we chose as the solution directory above. You can think of a sandbox as a collection of visual studio solutions/components under Jazz source control. So:

11. Lets create a new component for our Consumer called “OSLC”

12. Leave the defaults for the ignore files options. You should end up with the following directory structure (or similar):

13. In your pending changes view you should now see that we could deliver the new solutions with its project into jazz source control which we will do. Right click on the "Share Projects" and "Deliver" them into the OSLC Stream. (Click yes if it asks you if you want to deliver the component changes)

14. (If you understand component solutions relationships in VS skip to 17)
In anticipation of unit test we will eventually create lets create two more project skeletons. One called "OSLC Consumer Common"(containing common code) and a unit test project, "Test OSLC Consumer". Right click on the project and Add a project of type "Class Library"

15. Let's add the test project called "Test OSLC Consumer" of type "Test Project":

16. Unlike the solution there is not default option to add a project to Jazz Source control so we will have to do it now. Select both the new projects and right click and select the “Share Project(s) in Jazz...” option but do not click Finish!

Notice a key difference here between the RTC eclipse and Visual studio integration. The Component alignment is done at the solution level rather then at the project level! 
So in eclipse many projects are associated to a component, where in visual studio, many solutions to a component, where projects in the solution belong to the component of a solution.
In effect, if we wanted to have the unit tests, common code and consumer client in different components you would essentially need to create three solutions (one for each) in the sandbox. As this is exactly what we want this is what we will do (but you don't have to).
Just because we have three solutions does not mean we need to keep three IDE solution instances open because we can easily add the projects from different solutions to the same solution (which we will do.)
As I have gone through this detour to highlight this difference, lets revert and cancel out of the “Share Projects(s) in Jazz...”.. Right click on the two projects and select Right Click->remove. Also delete the corresponding directories on the file system. You will probably also want to delete the following files out of the solution directory: Local.testsettings, OSLC.vsmdi and TraceAndTestImpact,testsettings.

17. In anticipation of unit test we will eventually create lets create two more solution skeletons. One called "OSLC Common" with a project called "OSLC Consumer Common" (containing common code) and one called "Test OSLC" with a unit test project, "Test OSLC Consumer". As we want these in separate components lets create two solutions for these projects. We will create these solutions in the same sandbox as the "OSLC" Solution above.
18. First create the "OSLC Consumer Common" library which is a project of type "Class Library" in the "OSLC Common" solution:

19. Put it in the "OSLC Common" Component:

20. Delver the new solution and project to the OSLC Stream.

21. Let's repeat for "Test OSLC Consumer" library which is a project of type "Test Project" in the "Test OSLC" solution:

22. Now lets add it to the "Test OSLC" component:

23. Deliver the solution to the “OSLC Stream”

24. Go to the Solution Explorer Window (Team Concert->Windows->Sandbox Explorer).. it should looks as follows:

25. As i have named solutions the same as the Components we can easily see the projects in each component.
To avoid having to keep three IDEs open lets add the "OSLC Consumer Common" and "Test OSLC Consumer" projects to the "OSLC" solution by right clicking on the solution and Add->Existing Project...

26. Add the two projects and save the solution. You should have:

27. You can now deliver the OSLC.sln file to keep the changes to the solution. In the future we only need to open the OSLC solution. When you switch sandboxes Visual Studio will automatically ask you for a solution file. We will pick OSLC.sln from now on.

28. I have also gone ahead an deleted the default component and we might as well have the components be owned by the project level. Go to the “Team Artifacts” window and right click on the components and change ownership to the project for each component:

You project areas should look like this now:

This should prepare us for the next blog entry where we will start coding the OSLC client in C#.