Now, what you don’t find anywhere (only in some scenario explanation), that it is allowed to specifiy the mode (ie /console previously and now /admin) within the full address parameter.

Scenario: In the RDC client UI, you specify Computer_name /console in the Computer box (where Computer_name represents the name of the remote computer to which you want to connect), and then click Connect.

Behaviour: The /console switch is silently ignored. You will be connected to a session to remotely administer the server. (For more information about the Windows Server 2008 behavior, see the “Behavior when you connect to a server that does not have Terminal Server installed” section of this article.)

So my rdp connection file basically looks like this:

screen mode id:i:1
desktopwidth:i:1152
desktopheight:i:864
session bpp:i:24
winposstr:s:0,1,0,0,1219,971
full address:s:ip-address /admin
compression:i:1
keyboardhook:i:2
audiomode:i:0
redirectdrives:i:0
redirectprinters:i:0
redirectcomports:i:0
redirectsmartcards:i:0
displayconnectionbar:i:1
autoreconnection enabled:i:1
alternate shell:s:
shell working directory:s:
disable wallpaper:i:0
disable full window drag:i:0
disable menu anims:i:0
disable themes:i:0
disable cursor setting:i:0
bitmapcachepersistenable:i:1
redirectclipboard:i:1
redirectposdevices:i:0
drivestoredirect:s:
authentication level:i:0
prompt for credentials:i:0
negotiate security layer:i:1
remoteapplicationmode:i:0
allow desktop composition:i:1
allow font smoothing:i:1
gatewayhostname:s:
gatewayusagemethod:i:0
gatewaycredentialssource:i:4
gatewayprofileusagemethod:i:0
prompt for credentials:i:0
EnableCredSSPSupport:i:0

msg.vmxaiomgr.retrycontabort.unknown

The only workaround so far has been to unmount *any* cleanse any CD-Drives attached to the VM. And yes, this is reproduceable, even reinstalling from scratch doesn’t change the fact, that after installing the patch the VM quits working.

I also know, SLES10 SP2 ain’t officially supported yet by VMware, but I’d still suspect it to just work and not produce such weird errors. The only thing I found so far is this VMTN thread ..

Gonna have to try to define the dependency backwards, maybe that’ll help. But that’s a topic for Monday. Guess I’ll finish viewing Ghost in the Shell - Stand Alone Complex first.

As you can imagine, it gets quite confusing with that amount of checks (~150). So the last two days I spent on reorganizing (with Visio), on which object/hostgroup placing a check would make sense. Now, this is my first result of two days planning, reorganizing, reordering and moving hosts into different hostgroups.

Nagios Hostgroup Inheritance - Linux

Nagios Hostgroup Inheritance - Windows

Thanks to Josh (and Chris I think), realizing the above is gonna get quite easy. Gonna talk about the config layout itself about once I have it all wrapped up. Stay tuned!

Now as I went into work today and started my email client, I received 4 nagios warnings about a LOAD service reaching critical state. Looked at the nagios box itself, opened up the VM console, looked into the syslog. Nothing.

Yet over 3/4 of the services were flapping, some ping checks were critical (for whatever reason). So I opened the nagios webinterface again, and noticed it dropping the connection over and over again (had to reauthentificate me again and again).

So I opened up Putty, which established the connection without a single problem, but dropped me like a stone after a short amount of time. I restarted the session and got a security warning from Putty (due to different than the saved sshd public key). That raised my suspicion. So I took a look at the hostname, and lookie there.

Somehow my old nagios box (which is a physical box), got turned online again, thus having the same IP address as my virtualized one. So the virtualized nagios wasn’t really dropping my connection, but I was being directed to the old nagios.

Walked over into the data center, turned of the old box (well, I kept the power button pressed for a short time), and away went my troubles.

So here are some additions to my small repo (that is so far this month):

• iftop 0.17-7 for SLES10 (i586, x86_64, srpm) — Rebuild from Fedora’s SRPM
• nagios-nrpe 2.12-1 for SLES10 (i586, x86_64, srpm) — Rebuild/Bump from my previous SRPM
• nagios 3.0.3-1 for SLES10 — Rebuild/Bump from the original SLES10 SRPM
  • nagios-3.0.3 (i586, x86_64, srpm)
  • nagios-www-3.0.3 (i586, x86_64)
• pnp4nagios 0.4.10 for SLES10 (i586, x86_64, srpm) — Rebuild from CentOS SRPM, modified to fit into the SLES scheme — currently has a small bug, still need to fix it tomorrow (/etc/apache2/conf.d/pnp4nagios.conf has a wrong “Allow” line … ooooops).

The only point you don’t need to do, is to mark the VM as a “template“. You still can clone it and move it around and all that other stuff, but the good part is, that the cloned VM keeps the cpu mask set to the “template” *shrug*

I don’t know, why VMware didn’t include that feature into the templates, since it’s a real freaky way to do.

The bad thing about that is, that stuff becomes dependant on the Active Directory (we do have four domain controllers, so that should be fine).

Now, here’s the ssl-(only) apache2 configuration file for my vhost:

<VirtualHost *:80>
 
  ## mod_core
  DocumentRoot "/usr/share/nagios"
 
  ServerName nagios.barfoo.org
  ServerAlias nagios3.barfoo.org
  ServerAdmin nagiosadmin@barfoo.org
 
  ## mod_rewrite
  RewriteEngine On
  RewriteRule ^/(.*)         https://nagios.barfoo.org/$1 [L,R]
 
</VirtualHost>
 
<VirtualHost *:443>
 
  ## mod_core
  DocumentRoot "/usr/share/nagios"
 
  ServerName nagios.barfoo.org
  ServerAdmin nagiosadmin@barfoo.org
 
  ScriptAlias /nagios/cgi-bin /usr/lib/nagios/cgi
  Alias /nagios /usr/share/nagios
  Alias /pnp /usr/share/nagios/html/pnp4nagios
 
  <DirectoryMatch "/usr/(share/nagios|lib/nagios/cgi)">
    AllowOverride None
    Order deny,allow
    Deny from all
    Allow from 10.0.0.
    Options None
 
    # Authorization
    AuthType Basic
    AuthName "Nagios Barfoo"
 
    # The authentification provider is mod_ldap
    AuthBasicProvider ldap
 
    # mod_ldap is our *only* authentification provider for this!
    AuthzLDAPAuthoritative on
 
    # Redirect the userfile requests to /dev/null
    AuthUserFile /dev/null
 
    # AD requires an authentication DN to access any records
    AuthLDAPBindDN "BARFOO\\ldap_nagios"
    AuthLDAPBindPassword "somethingrandom"
 
    # The URL to search in
    AuthLDAPURL "ldap://dc0.barfoo.org dc1.barfoo.org dc2.barfoo.org dc3.barfoo.org/OU=Users,dc=barfoo,dc=org?sAMAccountName?sub?(objectClass=*)"
 
    # Search the group membership in the specified group, otherwise it's gonna
    # get searched at the binding DN's location
    AuthLDAPGroupAttributeIsDN on
    Require ldap-group CN=gr_nagios,OU=Groups,DC=barfoo,DC=org
  </DirectoryMatch>
 
  ## mod_log
  ErrorLog /var/log/apache2/nagios.barfoo.org.error_log
  TransferLog /var/log/apache2/nagios.barfoo.org.access_log
  CustomLog /var/log/apache2/nagios.barfoo.org.ssl_request_log   ssl_combined
 
  ## mod_ssl
  SSLEngine on
  SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
  SSLCertificateFile /etc/apache2/ssl.crt/nagios.barfoo.org.crt
  SSLCertificateKeyFile /etc/apache2/ssl.key/nagios.barfoo.org.key
 
  <Files ~ "\.(cgi|shtml|phtml|php3|php?)$">
    SSLOptions +StdEnvVars
  </Files>
 
  SetEnvIf User-Agent ".*MSIE.*" \
    nokeepalive ssl-unclean-shutdown \
    downgrade-1.0 force-response-1.0
 
</VirtualHost>

As you can see, AuthLDAPUrl holds the four LDAP servers separated by spaces (that’s what the Apache2 documentation says about that), and that actually works.

The only additional thing I had to change from the nagios part is in /etc/nagios/cgi.cfg to allow everyone to issue system commands. Also, if you ever stumble upon extranous chars in the check_nrpe output, update to a newer NRPE version, that fixed it for me (that is on the receiver side - as in the box running the NRPE agent).

But as we had a cooling outage today (basically ’cause it broke), I needed to turn off some ESX servers. Thus leaving me with the new ones and one of the old ones. *yuck*

So after a bit of googling, I found this VMware KB entry, which luckily lists the registers (on level 1) you need to zero out.

1
2

ecx ---- ---- ---- -0-- ---- --0- ---0 -0--
edx ---- ---- ---- --0- ---- ---- ---- ----

Only problem after that was that it still wasn’t enough. So back to the drawing board. The final solution came rather quick and looks like this:

1
2
3

eax ---- ---- ---- ---- ---- 0--0 ---- ----
ecx ---- ---- ---- -0-- ---- --0- ---0 -0--
edx ---- ---- ---- --0- ---- ---- ---- ----

The only stupid thing about this is, that

1. it ain’t supported by VMware (as in if you’re having trouble with your ESX/VC and you have a VM running with this, you’re shit outta luck!)
2. you have to define this on a *per VM basis*, which really is a pain in the ass for larger installations

True, I just should’ve bought VMotion compatible CPU’s, that would have spared me the hassle … but it’s too late now, I have to live with those ones.

With that I also got a set of “instructions” on how to install freetds from source (remember, I was a Gentoo dev, so I know my way around, when it comes to building from source), as well as a small set of instructions on how to create the connection.

Well, after trying to figure out why the hell the connection ain’t working with unixODBC’s tsql and PHP’s odbc functions, and yet the plain connection using telnet works … *shrug* turns out it was a simple mistake …

The “howto” said something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

[FreeTDS]
Description  = FreeTDS unixODBC Driver
Driver       = /usr/lib64/libtdsodbc.so
Setup        = /usr/lib64/libtdsodbc.so
 
[ODBC Data Sources]
mssql        = Microsoft SQL Server 2005
 
[mssql]
Driver       = /usr/lib64/libtdsodbc.so
Description  = MSSQLServer
Trace        = No
Database     = Database
TraceFile    = /var/log/freetdssql-foobar.log
Servername   = sql.foobar.org
Port         = 2433
TDS_Version  = 8.0
 
[Default]
Driver       = /usr/lib64/libtdsodbc.so

While it should have been this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

[FreeTDS]
Description  = FreeTDS unixODBC Driver
Driver       = /usr/lib64/libtdsodbc.so
Setup        = /usr/lib64/libtdsodbc.so
 
[ODBC Data Sources]
mssql        = Microsoft SQL Server 2005
 
[mssql]
Driver       = /usr/lib64/libtdsodbc.so
Description  = MSSQLServer
Trace        = No
Database     = Database
TraceFile    = /var/log/freetdssql-foobar.log
Server       = sql.foobar.org
Port         = 2433
TDS_Version  = 8.0
 
[Default]
Driver       = /usr/lib64/libtdsodbc.so

See the difference ? If not, I’ll show you a diff:

1
2
3
4
5
6
7
8
9
10

--- odbc.ini.orig
+++ odbc.ini
@@ -12,7 +12,7 @@
 Trace        = No
 Database     = Database
 TraceFile    = /var/log/freetdssql-foobar.log
-Servername   = sql.barfoo.org
+Server       = sql.barfoo.org
 Port         = 2433
 TDS_Version  = 8.0

Something as simple as adding another part of a word (as in “name“) to Server, makes the whole thing go wonko. Well, it ain’t going wonko per se, as Servername is different from the meaning of Server, at least when it comes to freetds.

Servername is the SQL-Server Instance name, while Server is the DNS name .. figures.

For POC or as a hint to myself, here’s where and what I needed to add/change:

Add the following modules to APACHE_MODULES in /etc/sysconfig/apache2:

1. dav_svn (dav_svn needs dav, thus the need to add it too)
2. dav
3. authnz_ldap (authnz_ldap needs ldap, so again we need that too!)
4. ldap

After that, we can add our repository (or our multi-repository folder) to /etc/apache2/conf.d/subversion.conf:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

<IfModule mod_dav_svn.c>
 
<Location /svn>
  DAV svn
  SVNParentPath /srv/svn
 
  # Limit write permission to list of valid users.
  <LimitExcept GET PROPFIND OPTIONS REPORT>
    # Require SSL connection for password protection.
    # SSLRequireSSL
 
    AuthType Basic
    AuthName "Subversion repositories (Domänenzugangsdaten)"
 
    # The authentification provider is mod_ldap
    AuthBasicProvider ldap
 
    # mod_ldap is our *only* authentification provider for this!
    AuthzLDAPAuthoritative on
 
    # AD requires an authentication DN to access any records
    AuthLDAPBindDN "CN=LDAP Subversion,OU=anon_accounts,OU=Users,DC=foobar,DC=org"
    AuthLDAPBindPassword "somethingrandom"
 
    # The URL to search in
    AuthLDAPURL "ldap://dc0.foobar.org/ou=Users,dc=foobar,dc=org?sAMAccountName?sub?(objectClass=*)"
 
    # Search the group membership in the specified group, otherwise it's gonna
    # get searched at the binding DN's location
    AuthLDAPGroupAttributeIsDN on
    Require ldap-group CN=gr_subversion,OU=Groups,DC=foobar,DC=org
 
  </LimitExcept>
</Location>

Now, as you can see, my goal was to not rely on a separate authorization database, but to use our already existing Active Directory at work. Generally this works just fine, but it didn’t. I tried various things, like trying another user, changing the group (as in the “require ldap-group“) as well as changing my own password. Zip.

All I got was this line in the error_log of apache:

[warn] [client 10.0.0.148] [9486] auth_ldap authenticate: user foo authentication failed; URI /svn/admin-scripts/!svn/act/71f2b65f-d050-0410-b33c-3b31fbb94a00 [ldap_search_ext_s() for use
r failed][Operations error]

Now, that itself does tell you what is happening, but not why. So again, I ended up googling till I found this:

The suggested step was to add “REFERRALS off” to /etc/ldap/ldap.conf. Surprise, the file don’t exist. Heck, there’s that one in /etc/ldap.conf. I did that, still zip.

Did I get the wrong file ? Absolutely.

/etc/ldap.conf is used by nsswitch and pam_ldap, but not by openldap2 (which is what apache is using). So reading this comment, adding the line to /etc/openldap2/ldap.conf, and *kaching*! Works.

Now I just need to install redmine (already installed ruby, rubygems and rubygem-rails from the SDK Addon), but I’ll leave that for tommorow, today I’m gonna watch Band of Brothers.

Today it bugged me again, so I used google again. This time with “ppc64 suse rpmbuild“, and guess what I saw within the preview of the second hit ..

rpmbuild -ba --target ppc64 myfile.spec

And here I thought I was missing something, turns out I was really stupid though .. *shrug* Building stuff like nagios works with that just fine ..

Update: or not. It worked only a single time and is broken ever since again. Guess I’m gonna reload the box on Tuesday.

Now, this is just for my own safekeeping, so I don’t end up googling for it again … *shrug*

echo -e "Line containing \nnewlines!" | sed ':a;N;$!ba;s/\n//g'

1. What’s the disadvantage of having a 1016 ported vSwitch ?
2. Any clues on how to exchange the default certificate of the Virtual Center ?
3. Are there any tools to stress test the virtual system ?
4. Are there any performance impacts of having more than 10 users in Virtual Center ?
5. Any clues and/or guides on how to do time synchronization in VMware guests, especially Linux guests ?
6. What’s the preferred NIC type for Linux guests ?
7. Any clues to using Raw Device Mappings with VMotion ?
8. Is there a way of defining CPU masks on a global level ?

Answers:

1. There might be a small overhead, though that’s limited to a really, non-measureable amount
2. Hasn’t done it yet.
3. Yes, there are free stress test tools like cpubusy.vbs, cpubusy.pl, iometer.exe, ..
4. Nope, you should only experienece load problems starting at 25 or so users
5. Select *one* variant, either time synchronization by use of the VMware tools or ntpupdate; if ntpupdate, select a single time source for your whole environment
6. For ESX 3.5.0 that would be “Flexible” (as per VMware Knowledgebase), as the vmxnet type is a leftover from ESX 3.0
7. Raw device mappings are *absolutely* supported by VMware, and also work without any troubles (when mapping/zonig is correctly configured)
8. Currently there’s no known way of doing this

• When adjusting the CPU afinity of a VM, *always* completely stop the virtual machine afterwards
• When trying to figure out CPU bottlenecks, check whether or not hyperthreading is enabled. The hyperthreaded (second) core is only giving you a CPU with 15% of the first.

Also, here are some guidelines on how the trainer extended the defaults:

ESX Server:

• Extend the “/” size to 10GiB
• Extend the “swap” partition to about 1GiB
• Extend the “/var/log” partition to about 4 GiB
• don’t mess around with creating too many vSwitches; just keep it simple
• set the duplex mode manually if the ESX is giving you any trouble
• disable the Traffic Shaping, unless you *really* need it

VirtualCenter:

• There’s two options when installing VirtualCenter: either install it on a physical box or simply put it into a virtual machine itself
• A problem with putting it into a virtual machine is, when the VM is shutting down or powered off due to isolation of the ESX running it, any ESX Server powering up isn’t going to start any virtual machines as that in return requires the License Server
• Only use the SQL Server Express variant if you really have to. It’s limited to 4GB database size, so if your installation grows above say 50 hosts and 2000 VM’s, this is gonna break the limits of SQL Server Express

But I think it’s about time for me to leave. I haven’t done much lately, I’ve been soo damn busy with work these last months like I would never have imagined. I gave it some more thought, and I finally got to the point (again I might add) where all just annoys and/or and frustrates me.

All the damn bickering, the childish behaviour Josh talked about (”noooo, that’s my TOY!“), the constant abuse of power (hey #-chat, #-kde ops). I thought most of us would at least try and behave like the elderish people we are (well besides the few of us, who really are children by law’s definition - hey there welp, omp, keytoaster ). But I guess that’s just been an imaginary thought I had .. *shrug*

Just this last thing, I’m gonna say thank to you to the few of you I’ve been gotten around to call friends.

Thank you

• Christina
• Diego
• Joshua (tsunam)
• Joshua (nightmorph)
• Łukasz
• Jorge
• Markus
• Daniel (dsd)
• Gysbert
• Peter
• Christian (well, not me. But hoffie and opfer)
• Chris
• Mike (well, three of you. ‘taco, mpagano and vapier)
• solar
• Andrew
• Alex
• Raúl
• Tobias (dertobi)
• Tobias (klausman)
• Robert (rbu)
• Robin
• Shyam
• Brent
• Benedikt
• Homer

It’s been a hell of a ride, and I thank all of you, deeply, from the bottom of my heart for the support, advice and resources you all gave me. This is it! This time, there’s no turning back, sorry Christina

So, I’m gonna stick to my day job, pay more attention to my trainees and maybe put some effort into getting something better! So long, farewell people. If people need to reach me; my email addresses still are in ldap, so that shouldn’t be a problem. Hasta luego (or maybe not, we’ll see …)

%ifarch ppc64
export LDFLAGS="$LDFLAGS -m64"
%endif

Otherwise, ld is gonna fail when linking, as it’s gonna try linking the generated 64bit code (-m64 is passed on via RPM_OPT_FLAGS to CFLAGS) as 32bit code, which ain’t gonna work at all …

On top of that, stuff ain’t building due to multiple problems (for example nagios and vim, cause ld is unable to find the fitting -lperl (for nagios) and -lXt (for vim)) as well as source errors …

 gcc -DHAVE_CONFIG_H -I. -I. -I../../include -I../../include -I/usr/include -D_FREETDS_LIBRARY_SOURCE -DUNIXODBC -DHAVE_UNISTD_H -DHAVE_PWD_H -DHAVE_SYS_TYPES_H -DHAVE_LONG_LONG -DSIZEOF_LONG=4 -D_REENTRANT -D_THREAD_SAFE -DDEBUG=1 -Wall -Wstrict-prototypes -Wmissing-prototypes -Wno-long-long -pthread -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -Wdeclaration-after-statement -MT connectparams.lo -MD -MP -MF .deps/connectparams.Tpo -c connectparams.c  -fPIC -DPIC -o .libs/connectparams.o
In file included from connectparams.c:22:
../../include/config.h:375:1: warning: "SIZEOF_LONG" redefined
<command line>:1:1: warning: this is the location of the previous definition
connectparams.c:90: error: static declaration of `SQLGetPrivateProfileString' follows non-static declaration
/usr/include/odbcinst.h:170: error: previous declaration of `SQLGetPrivateProfileString' was here
make[3]: *** [connectparams.lo] Error 1
make[3]: Leaving directory `/srv/BUILD/freetds-0.82/src/odbc'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/srv/BUILD/freetds-0.82/src/odbc'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/srv/BUILD/freetds-0.82/src'
make: *** [all-recursive] Error 1

We tried redoing *everything* (that is, starting with the domain, then terminal server and after that the thin clients) and yet nothing changed, it didn’t work either. That’s what I’ve been doing the last 2 weeks. Up till now, I always thought a user would have access to the ntuser.dat (that is HKEY_CURRENT_USER), if his NTFS permissions would be correct. But nooooooooooooooooooooo, Microsoft had to introduce another layer of permissions.

Old permissions on HKEY_CURRENT_USER

New permissions on HKEY_CURRENT_USER

The inspiration came to me after reading this and taking a look at regedit myself, where I noticed the entry “Permissions” for the first time ever since I’m using regedit. I also noticed, that the regedit permissions seem to be using the same groups, one would assign to NTFS resources.

That said, it really all boils down to the ntuser.dat (which *IS* HKEY_CURRENT_USER). As I created the profile with a different user than I am using it with (basically, I want ~12.000 users to use this one profile), I needed to change the permissions *INSIDE* regedit to include a group containing all these users. After that, any user could again merge the settings from ntuser.pol into HKEY_CURRENT_USER\Software\Policies, which in return gives you the joy of your fucking policies working again.

TADAAAAAA! About two weeks worth of work spent for such a shitty thing, and noticing it when you’re off work — priceless!

• VIA EPIA SN10000EG - Hanobox - 206,60€
• Kingston ValueRAM SO-DIMM 2GB (DDR2-667) - Alkon-PC - 29,90€ each
• Transcend SSD 2.5″, 8GB IDE - K&M Elektronik - 86,35€
• Ably GH-PELX16-C7 PCIe X16 Riser Card - HRT Informationstechnik - 28,69€
• Club 3D GeForce 7300 GT, 256MB DDR2, 2x DVI, PCIe - Xada - 56,50€
• Samsung SyncMaster 204B, 20.1″ - 270,00€ each

So we are currently at 759,24€ for the whole thing.

The node tries bringing it online, then shows a failure and immidiately moves them over to the next node. There the resource is being successfully moved online .. So again, I’m out of ideas ..

Already tried reinstalling the box, after that I could get the third node successfully into the cluster, without the “Advanced (minimum)” trick … *shrug* still ain’t bringing any resources online.

$ wget -q http://tinyurl.com/4atkz7 -O - | grep "(" | cut -d'(' -f2 | \
    cut -d')' -f1 | awk '{ print tolower($1) }' > retirements
$ for developer in $( < retirements )
    do retire.py --metadata $developer /cvs/gentoo-x86/ | \
    diffstat > ~/metadata.$developer
    done

That’ll give you a detailed list of which metadata.xml need to be changed.

On Monday I started installing the “IBM StorageManager RDAC” MultiPath driver (since the box got two single port PCIe FC-HBA’s) and figured I’d be nice if we had this. I asked a IBM Systems Engineer of one of our partners, which told me generally there wouldn’t be a problem with Microsoft Cluster Services (MSCS) and the IBM MPIO driver. Only requirement would be that I’d install the new storport.sys driver (version 5.2.3790.4021) first (as in Microsoft KB932755).

Now, yesterday I finished the zoning, did the mappings on the storage arrays and then figured the box should see the hard disks. So I started adding another node to our existing Microsoft Cluster.

Result: Zip (as in MSCS telling me not all nodes could see the quorum disk)

Reason: a combination of two things. First, said IBM Storage Manager RDAC. The first time I installed it, I forgot about the storage mappings, thus the box seeing zero disks. After uninstalling it, I was seeing 121 (that’s right, one hundred and twenty one) new devices.

Visible volumes previous to installing the RDAC driver

SAN mapping for the box

Visible volumes after installing the RDAC driver

It’s Microsoft and it doesn’t. Now why doesn’t it work ? ‘Cause the Cluster Setup Wizard is getting confused in Typical mode, as it’s creating a “local quorum disk” which naturally isn’t present in the cluster it’s joining. Now, switching the wizard to “Advanced (minimum) configuration” as suggested in Q331801, just works … *shrug*

Also, if you’re looking for a smart way of how to add a certain global group (as in Active Directory group) to a local group, try this:

NET LOCALGROUP Administrators /ADD DOMAIN\GROUPNAME

That simple, doesn’t even need the usual credentials to lookup the object, it apparently bypassed that step *shrug*.

And yet another weird thing is: if I run a certain command from a deployment script, it gives me different result as a manual execution of said script would give me .. *shrug*

NETDOM JOIN %COMPUTERNAME% /domain:barfoo.org \
  /OU:"OU=Thinclients,OU=Computers,DC=barfoo,DC=org" \
  /UserD:%ADMIN% /PasswordD:somepass \
  /User0: Administrator /Password0:Administrator
 
NET LOCALGROUP Administrators /ADD BARFOO\Domain-Users

If I put that into a rsp (that is Wyse Device Manager script), it ain’t working. Would I be executing it myself without the WDM, everything works like a charm … *yuck*

Apparently the mount produced a kernel oops (guess what, the 900GiB is running reiserfs), thus leaving the kernel tainted (or what do they call it ?). So he tried to reboot the box but it didn’t reboot. It started rebooting but then hung (as in not continuing the reboot). He tried to ssh back to the box, and it worked just fine.

This is where sysrq comes in handy.

# This is gonna activate the sysrq
echo 1 > /proc/sys/kernel/sysrq
 
# Now, since we ain't at a console, we can't use the sysrq keys
# ("b" for reboot, "o" for shutdown)
echo b > /proc/sysrq-trigger

That’ll restart the box, and cha-ching ..

So I ended up redoing the whole thing on a 32bit guest and guess what … bada-bing. works … *shrug* I don’t know whether or not that’s a surprising thing .. but what surprises me, is that I do have a working 64bit Integrated Solutions Console and Administration Center running, only difference is that one is running on real hardware.

Anyway, after looking on how the Integrated Solutions Console (that is the Websphere environment - yes *yuck*) did it’s own start up after boot (you know, I’d like to restart an application if it’s hanging without the need to reboot the whole box), I found this particular line of code:

isc6:23:boot:/opt/tivoli/isc/PortalServer/bin/startISC.sh ISC_Portal

And since I was lazy (and it was already Friday afternoon), I ended up writing a small init script which rids you of the need of such a ugly way to start a service.

#! /bin/sh
#
# /etc/init.d/isc
#
### BEGIN INIT INFO
# Provides:       isc
# Required-Start: network
# Should-Start:
# Required-Stop:  network
# Default-Start:  2 3 5
# Default-Stop:
# Description:    Start the Tivoli Integrated Solutions console
### END INIT INFO
 
. /etc/sysconfig/isc
 
BINDIR=/sbin
 
. /etc/rc.status
rc_reset
 
case "$1" in
  start)
    echo -n "Starting Tivoli Integrated Solutions Console server"
    startproc -p /var/run/tivoli-isc.pid \
      $ISC_INSTALL/PortalServer/bin/startISC.sh \
      ISC_Portal 2>&1 >/var/log/tivoli-isc.log
    # I know, this is *real* ugly
    PID=$( tail -n 1 /var/log/tivoli-isc.log | cut -d\  -f10 )
    echo $PID >/var/run/tivoli-isc.pid
    rc_status -v
    ;;
  stop)
    echo -n "Stopping Tivoli Integrated Solutions Console server"
    $ISC_INSTALL/PortalServer/bin/stopISC.sh \
      ISC_Portal $ISC_ADMIN $ISC_PASSWORD \
      2>&1 >/var/log/tivoli-isc.log
    rc_status -v
    ;;
  restart)
    $0 stop
  	$0 start
  	rc_status
  	;;
  status)
    echo -n "Tivoli Integrated Solutions Console server:"
    checkproc -p /var/run/tivoli-isc.pid \
      $ISC_INSTALL/AppServer/java/bin/java
    rc_status -v
    ;;
  *)
  	echo "Usage: $0 {start|stop|status|restart}"
    exit 1
  ;;
esac
rc_exit

And the corresponding sysconfig file:

## Type:     string
## Default:  iscadmin
## Config:   ""
## ServiceRestart:  isc
#
# Adminstration user for the Portal
ISC_ADMIN="iscadmin"
 
## Type:     string
## Default:  iscadmin
## Config:   ""
## ServiceRestart:  isc
#
# Adminstration user password for the Portal
ISC_PASSWORD="iscadmin"
 
## Type:     string
## Default:  /opt/ISC/601
## Config:   ""
## ServiceRestart:  isc
#
# Full path to the ISC Portal installation root directory
ISC_INSTALL="/opt/tivoli/isc"

Et voilá, it’s done. Now just a `chkconfig -a isc‘ and it’s gonna startup nice and easy (when it really should) via the normal service startup and not get spawned from the inittab.

Googling (is that a valid word yet ?!) for it, only resulted in one useful link, which is apparently a guy with the exact same problem … *shrug* I’m completely out of ideas by now, as I don’t even have a place to start (as in where the reason might be located).

Well, I do have a place to start with (that’s the local Events Viewer), which indeed lists some errors, but only such errors which ain’t making any sense. For example I see this:

• Userenv:1086 - “Windows cannot do loopback processing for downlevel or local users. Loopback processing will be disabled.“
• SceCli:1704 - “Security policy in the Group policy objects has been applied successfully.“
• Userenv:1085 - “The Group Policy client-side extension Folder Redirection failed to execute. Please look for any errors reported earlier by that extension.“

Simply doing this, is going to tell the FC HBA “rescan” and discover new devices ..

$ echo "1" > /sys/class/fc_host/host/issue_lip

That should do the trick, and you should be able to get udev to recognize the new devices attached via FibreChannel without the need to reboot the whole box (which might be a bit tricky).

So I ended up writing a small (and yet, ugly) shell script to generate me a mail of my liking ..

#!/bin/bash
 
# Checks the output of `zypper pch` for security/recommended/optional
# updates and prepares a detailed report to be mailed to the
# administrators
 
TO="admin-addr@localhost"
CLASSES="security recommended optional"
 
# Temporary files
ZYPP_LIST="$( mktemp /tmp/zypper-list.XXXXXX )"
ZYPP_DETAILS="$( mktemp /tmp/zypper-details.XXXXXX )"
TMP="$( mktemp /tmp/zypper-report.XXXXXX )"
zypper pch 2>/dev/null > $ZYPP_LIST
 
# Figure out how much updates are still pending
PENDING="$( cat $ZYPP_LIST | grep "| Needed" | wc -l )"
 
if [ $PENDING -eq 0 ] ; then
  exit 0
fi
 
echo > $TMP
echo " Pending updates for $( domainname -f ) on $( date )" >> $TMP
 
for severity in $CLASSES; do
  PACKAGES="$( cat $ZYPP_LIST | egrep "${severity}(.*)\| Needed" | \
    cut -d\| -f2 | sed "s,^ ,," )"
  echo
  echo "  Category: $severity"
  for package in $PACKAGES; do
    zypper patch-info $package 2>/dev/null > $ZYPP_DETAILS
    echo "  * $package ($( cat $ZYPP_DETAILS | grep "Version: " | \
      sed "s,Version,version," ))"
    echo "    $( cat $ZYPP_DETAILS | grep "Summary: ")"
    echo "    $( cat $ZYPP_DETAILS | grep "Reboot Required:" )"
    echo
  done
  echo
done >> $TMP
 
cat $TMP | \
  mail -s "[$( date +%F )] Update report for $( domainname -f )" $TO
trap 'rm -f "$TMP" "$ZYPP_LIST" "$ZYPP_DETAILS" >/dev/null 2>&1' 0
trap "exit 2" 1 2 3 15
 
# vim: set tw=80

Anyway, Saturday morning as I was forced to head downstairs due to my aunt calling (that was at 9am), my brother managed to take a shot at the presents as well as something else:

Cat'se ontop of presents

Barbeque grill

Them benches!

Windows Server 2003:
Standard Edition:        2 * 91,00
CAL:                    50 *  6,00
Terminal Server CAL:    50 * 17,00
                       ___________
                          1.332,00

The above are fixed costs, you need them anyway as both Citrix as well as the 2X solution is only working *on top* of Windows Server 2003 Terminal Services.

Now, here’s the real comparison between 2X Application Server & Loadbalancer and Citrix XenApp Platinum Edition:

2X               2 * 1510,00 =  3.020,00
Citrix          50 *  393,00 = 19.650,00

While 2X is licensed per terminal server, XenApp is licensed per user. As you can see from the above prices, the 2X solution is roughly 1/6 of the Citrix XenApp solution.

Now, as we had a slight problem with the 2X Application Client (first this weird error, “Error Code: [01/0000001E] The published application couldn’t be started.” which I admit was an error on our behalf, by copying the user registry into HKEY_LOCAL_MACHINE) I was already annoyed by their support.

Now, when they released their version 6 of the 2X Application Server/Client software, I tried producing a proof-of-concept for our installation, which improved certain things (like finally having Single-Sign-On or SSO for those of you who like buzzwords), and I tested it out together with one of my trainees (who’s having the subject “thin clients and terminal services” for his final paper of his education).

Now, I stumbled upon this great feature of the Microsoft Remote Desktop Console (mstsc) Version 6.0, called “Drives that I connect to later“, which is basically forwarding drives, which have been added to the computer, *after* the remote session has been established. Once you enable the check mark (before connecting to the remote session obviously), the drive is instantly forwarded to the remote session, and I figured why not try that with the 2X solution. Now, we tried and tried, reinstalled the terminal server and the 2X stuff along the way, and tried again with a plain machine, and it still wasn’t working. So I first tried mailing the German key account manager, which in turn told me, the software already supported it …… uhm, I just tried, and it doesn’t.

So, OK. We tried again. Reinstalled the terminal server, reinstalled the Windows XP workstation, installed 2X and stuff, and tried it again. And still didn’t work. OK, so I went ahead, asked again. The key account manager asked me which OS and version of the RDP we were using, and I told her we’d be using XPe and RDP 6.0.6000. She forwarded those to the technical support, opened a support request in their “Support center“, where one of the technical assistants in return told me *yet* again, it should be working.

Additionally he told me kindly, that I could add drives later, or else I should provide them with a detailed description of what exactly I imagined it doing. So I went ahead, and created a “demo” of some kind, using Adobe Captivate demonstrating them what I was seeing. I also told them, that I wanted the 2X client to simply support that feature. Additonally it works when using plain RDP, but doesn’t when using 2X, which in return should tell them that their software is at fault, and not my terminal server. OK, when I created the first demo, I had the USB drive still attached to my work computer, which in turn got forwarded to the remote session when establishing the connection. I guess that was a ‘lil bit embarrassing, so I created yet another demo.

As you can see from that demo, I launched three separate windows. One showing my local workplace, one the workplace of the terminal server launched in RDP, and one window showing the remote workplace launched via 2X. That apparently was a bit much for the 2X support guys, as they told me I’d be only showing the RDP window, but not the 2X window. *WTF* Didn’t I open just *three* windows, each showing something different ?

After that I’ve been told, they couldn’t reproduce it with their 2X Client. The technical assistant also told me, that due to my dual screen layout (my complete screen resolution is something like 2560×1024) he could only see half of what I’m doing. So I went ahead, and created *yet* another demo (it’s actually a six-part flash video, due to Captivate switching to “Full-Motion” mode .. main, 1, 2, 3, 4, 5).

After two separate tries to upload the 35M file to their support center, which failed due to only allowing 8M uploads, I uploaded it to my private user site. Some hours later, the technical assistant told me, I should press “F5″ for a explorer refresh or View -> Refresh in the 2X Client. I told him, that after trying both the USB drive still isn’t showing up.

After an hour he wrote me back, telling me that after extensive tests, that I should open notepad in the remote (2X) session and see whether or not the drive is appearing there.

That’s it so far. Initially I wanted to only post this if they couldn’t figure it out, and only *after* I talked to the key account manager again, but I’m rather tired. Maybe if they are reading this: I’m fscking TIRED!

And after I picked up Michel, we some when arrived at work (that is one hour later), we had our own adventure park in front of the work place:

Collapsed trees

LOAD 05:00-19:00

As a comparison, here’s the “normal” load for the last week:

LOAD 31.03.08-07.04.08