Topspin 3.5pl6 on a CentOS 7 64 bit system....more than libraries

Let us start refreshing ourselves with the following notes, before we add the additional points learnt in making TS3.5 work in a CentOS 7 64 bit system :

Post on Topspin and 32 bit libraries


To the above list, I will add one more package that is needed.  This doesn't get pulled in as a dependency of any of the above mentioned packages, but one.

libXtst.i686

pulls in as dependency :

libXi.i686


systemd and Topspin 3.5 installation:

CentOS 7 employs 'systemd'  daemon to control all processes in the system that were individually handled by different binaries such as 'system' or 'chkconfig', etc.  Please refer to the CentOS 7 FAQ and documentation for a through review:

The shell command we use to communicate with systemd is 'systemctl'.

For instance, to query the status of Bruker License Manager daemon that is supposed to start at boot time, we have to say :

# systemctl status bruker_lmgr.service

The suffix 'service' can be omitted as long as there no possibility of another unit file with the same head such as 'bruker_lmgr'.

The above command gives the following :
 _______________________________________________________________________________________
 ● bruker_lmgr.service - BRUKER FlexLM license server
   Loaded: loaded (/etc/systemd/system/bruker_lmgr.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2017-10-16 13:06:00 PDT; 20min ago
 Main PID: 1202 (lmgrd)
   CGroup: /system.slice/bruker_lmgr.service
           ├─1202 /usr/local/flexlm/Bruker/lmgrd -local -2 -p -z -c /usr/local/flexlm/Bruker/licenses/license.dat
           └─1217 bruker_ls -T vgaruda 9.5 3 -c /usr/local/flexlm/Bruker/licenses/license.dat -p --lmgrd_start 59e51129

Oct 16 13:06:01 vgaruda lmgrd[1202]: 13:06:01 (bruker_ls) EXPIRED: SMP_STRUCELU_RANK
Oct 16 13:06:01 vgaruda lmgrd[1202]: 13:06:01 (bruker_ls) EXPIRED: MODGR_IFC
Oct 16 13:06:01 vgaruda lmgrd[1202]: 13:06:01 (bruker_ls) Server started on vgaruda for:        TOPSPIN3
Oct 16 13:06:01 vgaruda lmgrd[1202]: 13:06:01 (bruker_ls) TOPSPIN_1D        TOPSPIN_2D        XWINPLOT
Oct 16 13:06:01 vgaruda lmgrd[1202]: 13:06:01 (bruker_ls) TOPSPIN_ACQU        NMRSIM                NMRGUIDE
Oct 16 13:06:01 vgaruda lmgrd[1202]: 13:06:01 (bruker_ls) TOPSPIN2
Oct 16 13:14:34 vgaruda lmgrd[1202]: 13:14:34 (bruker_ls) OUT: "TOPSPIN3" rajan@vgaruda
Oct 16 13:14:34 vgaruda lmgrd[1202]: 13:14:34 (bruker_ls) OUT: "TOPSPIN2" rajan@vgaruda
Oct 16 13:14:38 vgaruda lmgrd[1202]: 13:14:38 (bruker_ls) IN: "TOPSPIN3" rajan@vgaruda
Oct 16 13:14:38 vgaruda lmgrd[1202]: 13:14:38 (bruker_ls) IN: "TOPSPIN2" rajan@vgaruda
 _______________________________________________________________________________________

The output is more or less what you will see in the flexlm.log file under : /usr/local/flexlm/Bruker/licenses directory.

The above tells us that Bruker license manager has successfully read the license file and Topspin is ready to start.

Perl development libraries for instllation : 

In addition to the packages mentioned above, one crucial rpm we need is the following :

perl-Sys-Syslog.x86_64

Other than this, there are few other perl development components we need in the system for your installation to proceed smoothly. 

To begin with, do :  yum install perl-devel.i686

Since all the relevant Perl modules are going to come from CPAN, I found the following solution very elegant.  

yum install perl-App-cpanminus

This installs the CPAN module downloader and builder that can be used directly to pull any relevant Perl modules that are in CPAN repository.

Two of the relevant ones for us are installed like so :

cpanm Data::Dumper
cpanm Sys::Syslog

If you ran into install errors before, you do the above and then, just remove the /usr/local/flexlm directory completely, before running the Topspin3.5 install script from the medium, again.


For whatever reason, the install script did not create an empty license.dat file under /usr/local/flexlm/Bruker/licenses,  as it used to do with earlier versions.  At least with CentOS 7 distro, I couldn't see this happening. 

For that matter, it doesn't even create the /usr/local/flexlm/Bruker/licenses branch of the tree. I had to create it by hand. 

Topspin3.5 and native Network Interface Card names :

 It came as a great relief for me that Topspin 3.5PL6  install script now recognizes the native Network Interface Card naming scheme. We had to jump through several hoops in the past to somehow force name the first interface card as 'eth0' for the flexlm license daemon to work correctly.   But now, the install script automatically gives a pull down menu of the recognized interface card names and you can choose the appropriate one, whose MAC ID is the one that is found in the license.dat file.

NetworkManager and NIC naming :

When NetworkManager service is running in CentOS 7 (which is the default), it uses the logical naming scheme for the NICs found in the system, based on the bus location and other details.   In my system, for instance, these cards are called enp0s3 and enp0s8.

In the past, to make these NIC-s to be called 'ethX',  we had to disable NetworkManager and create configuration files by hand in /etc/sysconfig/network-scripts. Even then it is not guaranteed that it will work as expected.

From Topspin3.5 onwards, Bruker has provided two major updates to the installation process  :

1. They natively provide the so-called systemd unit file for starting the bruker_lmgr
2. They also provide a unit file that allows the natural NIC scheme by Network Manager to be recognized.  This way, when license manager runs, it can query the MAC ID of the correct interface card and satisfy itself against the license.dat file

The above makes life so much easier.

/etc/hosts file is still important ! :

We still need to tie the IPV4 address to the hostname that is being used in the license.dat file.  This is still a legacy behavior of flexlm license manager, which does not use DNS lookups or such to get the identity of the host, apparently.

IMPORTANT UPDATE ON TOPSPIN 3.5PL5 (free version) - CentOS 7

I am referring below to the generously provided free version of the Topspin processing software by Bruker. For those of you who didn't notice (like me), the shell wrapper for this version includes a "al" in its name :  linux-topspin3.5.5.al.sh

I am mentioning because, there are other shell wrapper versions available for download, that turns out to be for installing a 'full license' scenario.  

Ok, on to the next step : 

I am sure you have taken care of installing all the required 32-bit libraries to make Topspin install script as well as the Topspin gui to function. 

You are rubbing your hands and raring to go....but....when you start Topspin (hopefully from the shell, as I do, so that I can immediately see any messages that are relevant) you see the banner appear and after a while it simply quits. 

This is what I got. 

 TopSpin 3.5 pl 5  -  Copyright (C) 2016 Bruker BioSpin GmbH
Installation directory: "/opt/topspin3.5pl5"
Waiting for FLEXlm license

Cannot checkout TopSpin FLEXlm license
Invalid license file syntax
Feature:       TOPSPIN3
License path:  /usr/local/flexlm/Bruker/licenses/license.dat
FLEXlm error:  -2,413
For further information, refer to the FLEXlm End User Manual,
available at "www.macrovision.com".

The FLEXlm host ID of this machine is 080027fd834a

license for noncommercial use, valid until 2017-03-31
error message cprserver: GetSystemPrinterList: dlopen(libcups.so) failed

premature Java Virtual Machine termination
Program is exiting ...

  Like me, it is possible that you ignore that error message from cprserver on this function call : GetSystemPrinterList: (libcups.so) failed.

What more, I was thrown off guard by the earlier snippet of " Cannot checkout TopSpin FLEXlm license
Invalid license file syntax" 

I was thinking that the former has to do with a printer definition and since I was running this on a virtual machine for testing, I didn't care about this particular library "libcups.so".  I couldn't have been more wrong. 

 When I installed the rpm that provides this library i.e. cups-devel, it pulled in literally a dozen more dependencies and some of them appear rather generic and important, such as libstdc++, zlib and dbus-libs. 

Once I installed these, Topspin launched without any problem and what more, the 'license file' related error turned out to be a dud. 

So, the lesson is that, don't ignore any error message thrown by Topspin when you launch and try to address it then and there. 

It is also clear that the way you build the linux distro, may not include some of the crucial components needed for this software to function correctly.  If there is no straightforward way to figure out these mandatory components, then at least pay attention to the error messages and discover the needed packages using those as hints.

UPDATE ON THIS UPDATE !

 I built another another instance of Topspin 3.5 on another virtual machine but even after this last step, Topspin started crashing.  Then I realized my folly...When I did a 'yum install'  I did not explicitly specify the i686 suffix and so it pulled all the cups-devel and dependencies of x86_64 bit version.  Once I did an explicit

yum install cups-devel.i686 

everything worked perfectly.  This might be helpful for you to remember....

Tidbits 2014

• Topspin and Ubuntu 14.04 LTS

Recently I moved to Ubuntu 14.04 LTS and Topspin 3.2 did not start normally with this platform. Apparently the cprserver was dying and nothing came up after a couple of minutes.  I did install all the 32-bit compatible libraries to take care of all X related issues. But this problem is not remedied by those.

I did find a Bruker knowledge base entry that said that TS3.2 is not playing well with Kernel 3.13.xx yet and understandably Bruker cannot help but wait for the next kernel release. 

Apparently by May '14 Kernel 3.14 has been released, not by the official ubuntu distribution, but from a third party and Bruker encouraged me to update the Kernel and assured that Topspin 3.2 will work well after that.

I am happy to note that, after I updated to Kernel 3.14.3-031403-generic, as I write this, Topspin 3.2 works beautifully, without any issues so far.

Phew !

Topspin and 32 bit libraries in a 64 bit system : Summary

I had mentioned earlier that I will summarize the needed 32 bit libraries for Topspin to play well in a 64 bit platform.  This post is an attempt in that direction.  I am referring to either RedHat Enterprise Linux or its "$free$" re-incarnation that is CentOS.  For this reason, I am referring to rpms in this post.

 But I suspect that this approach should in general be applicable to other popular Linux platforms such as Ubuntu or Mint. You have to pull in the equivalent packages with tools such as apt-get. 

Needed RPMs for TS install to work correctly : 

 glibc.i686        
glibc-devel.i686
libX11.i686 (this gets pulled in as a dependency if you install libX11-devel)
libX11-devel.i686

RPMs needed for proper Topspin functionality such as 'edlock' table display.

 libXext-devel.i686
 libXtst.i686 (gets pulled in as a dependency for the former)

RPMs for gui functionality such as pulse program display and 'nmrsim'

  libXmu.i686
 libXt.i686   

As a data station, the above libraries have resulted in the expected Topspin behavior, thus far. I will update the post if I stumble upon any other problems and the possible solutions reached.

I will also mention the passing, the following tip.  Although there are error log files under the $TOPSPINHOME/prog/curdir/$USER directory,  errors related to the missing libraries above are not recorded in any of them. The Topspin 'hist' command is very handy and it gives the most elaborate capture of events as they occur.  For instance, the failure of 'nmrsim' is recorded in the history along with the missing library names :  libXmu.so.6 and libXt.so.6 

Happy spinning !    

UPDATE - 26 May 2016

Today I am trying to build Topspin 3.5 on a CentOS7 Virtual Machine running on my Fc23 box. I had to follow all the above, of course, but it still fell short of one rpm, which is : 
libXft.i686  
This provides the library module libXft.so.2 for the install to work correctly.

Please note that, this also installs the following dependencies automatically :
-------------------
 expat.i686 0:2.1.0-8.el7          fontconfig.i686 0:2.10.95-7.el7          freetype.i686 0:2.4.11-11.el7          libXrender.i686 0:0.9.8-2.1.el7
-----------------------

UPDATE : 

Installing libXmu.i686 automatically pulls the following as dependencies:

libXt
libXext 

among other packages. Since the above two are essential for normal operation of nmrsim, I am highlighting only these.

TidBits 2013

• When we install Topspin, where does it store the default path information, to start the topspin script ?

Topspin stores this information in two files : 
/etc/profile.d/topspin.sh  
/etc/profile.d/topspin.csh

As can be inferred the former is used for a BASH shell and the latter for a c-shell or its derivatives.  The PATH invocation's syntax is different for the two types of shells and this is the reason for the existence of these two separate files.  

As an example, in my system, inside the topspin.sh file this is what I find : 

 PATH=/opt/topspin3/prog/bin/scripts:$PATH # created by TopSpin Installer

The above defines the global path for Topspin installation.
If you are installing Topspin in two different locations in parallel, for instance, both Topspin 2.1 and Topspin 3.0,  then only one of them will be sourced, when you enter the script name topspin on the shell command line. Which version of Topspin will be started will be decided by the global path defined in the file above.  Normally, you are not expected to kludge this file by hand, as is implied by the comment "created by Topspin Installer".   Having said that, if you change the path here by hand, it does work.

TidBits 2012

Remote connection :  To get the correct behavior when you try to remotely connect to a Topspin session, you should make sure that the firewall allows incoming connections on both the remote machine as well as the client machine.  If the remote host is allowing in-coming connections, but the client does not allow the remote host to connect back, then the tell-tale sign is that you will go as far as the PAM login window and then the connection will fail.   
  
If you have setup openvpn connection between the remote host and client machine, then you don't have to spend time to tweak the ports through which the machines have to allow the connections. You can simply let the incoming connections from the trusted IP-s in the non-routable subnet on all ports. 

popt and  spectrum limits:  

If you are employing a wide SW when running  popt,  make sure that you don't use the entire SW as the difference between F1P and F2P.  The automated ABSF command will fail and the optimization will stop abruptly after the first increment.   Use a limit not greater than 5 ppm to get the desired result.  

Topspin 2.1 and RHEL 6.3 - how to get out of this bind

The bug had bitten me again. When I decided to upgrade my ageing computer box (about 9+ yrs. old now) that runs Topspin 2.1 PL6 with an Avance console,  I wanted to go for the RedHat Enterprise Linux 6.3.  Just for nostalgia, in the days when SGI workstations roamed the earth, XWINNMR used to be the software ruling the Brukerland and it had the special requirement that the graphics card support 8-bit colour depth, while the computer hardware and OS moved on to support 24 bit depth typically.  The graphics card that did double duty, has become so obsolete of late that the 'mga' driver,  seemed to cause problems slowing down my systems. That is why I decided to move on with the newer hardware.  That and I also want to move away from ATAPI hard drives.

Instead of making life easy by going with RHEL 5, I decided to go with 6.3.

Problem:  CCU won't boot.  

Ending : happy ending :-)   

I will summarize quickly what's the matter.

Background : 

Digging a bit with wireshark packet sniffer,  we could clearly see that the initial conversation between 'spect', which is the CCU11 board and 'ASP_ST2', which is the Linux box does take place.  It proceeds to the point where, spect, which is the diskless client, asks for a port number in which bootparamd is listening. It tries to get the info from the ASP_ST2 server.   bootparamd, similar to nfs or rquotad belongs to the RPC family of servers.  Normally, they register themselves to a program called portmap,  which in turn informs a connecting client such as spect, to which port number the client is supposed to send its communications to talk to that particular server, in this case bootparamd.  

I show here a packet, where this request for port number from  spect (IP 149.236.99.99) to ASP_ST2 (IP 149.236.99.1) is rejected.   In the upper half of the window that summarizes the traffic, note the last line "Portmap GETPORT Reply Port:0 PROGRAM_NOT_AVAILABLE".   Also note the 3rd line, Internet Protocol that shows you who is sending this message to whom i.e. 149.236.99.1 --> sends this packet to --> 149.236.99.99.  

 

Problem and resolution : 

With RHEL 6.3  (which is derived from Fedora 14 and higher), the newer program rpcbind is used in place of the conventional portmap. The following wikipedia page underlines the fact that these two programs are different avatars of the same entity.   The portmap seems to be the older version and rpcbind is the newer version.  With RHEL 6.3, for an inexplicable reason, both the portmap  and rpcbind are installed and turned on by default. 

In a system where portmapper function is fine, you can enter the command rpcinfo -p and get a typical output similar to this :

 program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp  34528  status
    100024    1   tcp  52726  status
    100011    1   udp    875  rquotad
    100011    2   udp    875  rquotad
    100011    1   tcp    875  rquotad
    100011    2   tcp    875  rquotad
    100005    1   udp  52514  mountd
    100005    1   tcp  55703  mountd
    100005    2   udp  50364  mountd
    100005    2   tcp  48481  mountd
    100005    3   udp  58813  mountd
    100005    3   tcp  53255  mountd
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs
    100003    4   tcp   2049  nfs
    100227    2   tcp   2049  nfs_acl
    100227    3   tcp   2049  nfs_acl
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    100003    4   udp   2049  nfs
    100227    2   udp   2049  nfs_acl
    100227    3   udp   2049  nfs_acl
    100021    1   udp  46281  nlockmgr
    100021    3   udp  46281  nlockmgr
    100021    4   udp  46281  nlockmgr
    100021    1   tcp  46143  nlockmgr
    100021    3   tcp  46143  nlockmgr
    100021    4   tcp  46143  nlockmgr
    100026    1   udp    721  bootparam

Note that the portmapper (whichever implementation it is i.e. rpcbind or portmap) always listen on port 111. 

 When both rpcbind and portmapper are running, you get an error message saying that : No RPC program registered.   

I tried turning off rpcbind using chkconfig and left the older portmap running. The problem remained as it is.  But, when I turned off portmap and left the newer rpcbind running, I could see that the RPC servers are registering with the portmapper, as shown by the above listing.

With this setup, CCU11 i.e. spect boots correctly.  Now, let us look at the same packet where ASP_ST2 is sending a reply to spect for the latter's GET_PORT request.  As before, in the upper half of the packet image, the last line shows :
 Portmap GETPORT Reply Port : 724 Port: 724.  Now, if you run rpcinfo -p you can confirm that port 724 is where bootparamd is listening.

A sidetrack on bootparamd and dhcpd :

bootparamd is the precursor to the dhcp protocol and this serves the /usr/diskless/client  file tree to the CCU11, which then bootstraps to the code contained therein and a minimal UNIX environment takes shape.

With TS2.1 onwards, the installation of diskless from the Topspin DVD automatically installs a dhcpd.conf also under /etc/ directory.  Couple of points on that :

• As long as your console does not have the newer IPSO, you don't need this dhcp server to be running. The entire diskless boot is happening via bootparamd
• If you happen to upgrade to a newer console AVANCE-II or III that has an IPSO, you need this dhcpd daemon to be running.  
• With RHEL 6.3, you should place the bruker suppled dhcpd.conf in the /etc/dhcp/  directory, since that is where the script expects the conf file. Your dhcpd daemon will not start, with the default /etc/dhcpd.conf  location.