Mine the Harvest

We provide users with a visualization system with IDL, MatLab, and Visit. X-Forwarding can be painfully slow, and we have found that using NXServer, or Googles open source NeatX equivalent, can provide much faster performance.

However, VisIt would launch when X-Forwarded, but not when using NXClient & NeatX. The only messages were:
…
Running: mdserver2.3.2 -host 127.0.0.1 -port 5601
VisIt's viewer exited abnormally! Aborting the Graphical User Interface. VisIt's developers may be reached via the visit-users mailing list.  Please see:
htttp://visit.llnl.gov/FAQ.html#1

I also found that glxinfo would return:

$ glxinfo
name of display: :826.0
Error: couldn't find RGB GLX visual or fbconfig

Solution:

First, I created an xorg.conf to ensure GLX was actually being loaded. (It turned out this was not required as X was autodetecting everything fine without an explicit xorg.conf, but I include it here as it may be helpful in some cases to have an explicit xorg.conf)

Xorg -configure
(renamed xorg.conf and moved to /etc/X11. Ensured Section “Module” included Load “glx”, etc.)

glxinfo still would not work. I then set the following environmental variable:

export  LIBGL_ALWAYS_INDIRECT=yes

Which allowd glxinfo, glxgears, etc. to work. For more information on this environmental, see:

http://unix.stackexchange.com/questions/1437/what-does-libgl-always-indirect-1-actually-do
https://bugs.launchpad.net/xorg-server/+bug/384001

This could then be made into a Gnome Desktop Launcher:

cat VisIt.desktop
[Desktop Entry]
Version=1.0
Type=Application
Terminal=false
Icon[C]=gnome-panel-launcher
Name[C]=VisIt
Exec=env LIBGL_ALWAYS_INDIRECT=yes /usr/local/visit/bin/visit
Name=VisIt
Icon=/usr/share/icons/visit.jpg
Categories=GNOME;Application;Development;

Perhaps the above may be useful to other VisIt users.

I’ve been waiting 27 years to see this:

joshua@jaguarpf-login2:~> date
Thu Mar 10 16:04:31 EST 2011
joshua@jaguarpf-login2:~>

Translated from Old High Geek (derived from early proto Unix) that output is similar in magnitude to the fabled Enochian Emerald Tablets of Hermes Trismegistus. But to explain how cool this was for me, a bit of a narrative is needed, but there won’t be much computer speek or techno mumbo jumbo in this post.

In 1983, when I was 12 years old, the movie War Games came out. Of course, I was already deeply in love with computers and had spent endless hours already using my 8 Mhz, 8-bit Atari 400 with 24K as well as my Vic 20 with 3K. Naturally, being a geek, I knew about Cray Supercomputers. (The word was pronounced with a slight pause and inflection when uttered, if indeed the ineffable need be put in human words.) The anecdotal relationship in War Game’s between professor Falken (John wood) and the real life Seymour Cray was  also common geek lore – cannon in fact. Cray was THE supercomputing company – the gold standard of computing.

My love for Cray systems continued unabated from that time, and I would always keep an eye on what the company, and there computers, were doing. Of course, the story of the visisituedes of the company and its founder are near legendary. But like a favorite sports team or rock band I was always a loyal fan – even though I had never once used a Cray system. Neither did I own any platinum, or anti-matter, but one need not have something to know its worth.

More years past until in October of 1996, while driving to work, I heard the news of the unexpected death of Seymour Cray. Wow, that sucked. The father of supercomputing was killed in a car crash, apparently by a drunk driver. I thought about it for days, with the surreal feeling you get when a hero figure is made so painfully mortal, and who’s life need not have been ended. It was just one of those days you will always remember.

Over the years computers kept changing. The age of the custom “big iron” vector supercomputers waned, replaced by commodity hardware running thousands of CPUs and powered by a little OS called Linux. Now, if there was one thing I liked equally as much as Cray’s, it would have to be Linux.

One thing lead to another, and I was recently offered a position at Oak Ridge, the home of Jaguar – a Cray XT5  system with over 220,000 cores running at 2.3 PF (2.3 quadrillion calculations a second).  Jaguar, my favorite computer in the World, running my favorite OS, on a Cray platform, was what lead me to the Lab and I’ve been quite happy with my position there.

So, while that output might seem like any other login, or some boring computer output, for me it had a bit more emerald tablet like glow to it.

Today is another one of those days I’ll remember.

Thanks Seymour.

I have officially started my new position at Oak Ridge National Laboratory outside of Oak Ridge, Tennessee. I am working in the Information Technology Services Division, which is part of the Computing and Computational Sciences Directorate. My position is that of Unix / Linux system administrator, working primarily in maintaining systems and infrastructure supporting the Climate Change Science Institute.

The lab has a number of core focus areas including environmental and climate science, materials science  (including new facilities for nanophase research), and is a world leader in neutron science at the Spellation Neutron Source. ORNL is also the home of the National Center for Computational Sciences, and the home of Jaguar and Kraken, currently the first and third most powerful supercomputers in the world. ORNL also supports national security projects, nuclear non-proliferation work,  nuclear science and engineering, fusion power and alternative energy sources as well as being a leader in computational biology.

These diverse research areas all typically heavily utilize Linux and Unix for a number of scientific applications and research tools. Of course, the supercomputing center relies on Linux almost exclusively as the Cray systems, and various support clusters, run various distributions of Linux.

Spallation Neutron Source (SNS) at ORNL

The multi-disciplinary environment of the lab makes for a very interesting work environment. I am currently supporting researchers in their work in climate sciences, for example by maintaining the systems used for the ORNL Distributed Active Archive Center (DAAC), and the Atmospheric Radiation Measurement Program (ARM) systems. Additionally, I will be working more with biological science systems as time goes on.

Jaguar Cray XT5

Eventually, I would like to explore my interests in high performance computing, both in the computational needs for Environmental Sciences, and otherwise. I’m increasingly interested in GPU computing, and would like to work with developing and supporting compute environments which use NVIDIA CUDA and OpenCL which can help realize dramatic performance increases in massively parallel computing.

Fall Colors at ORNL

The lab is a world-class facility and is the showcase of our national laboratory system. The campus grounds are quite nice. The lab is surrounded by trees, and is on a huge reservation. The drive to work is beautiful. Here is a picture of the fall colors behind the Joint Institute for Biological Sciences building.

Everyone  at the lab has been very easy to work with; in fact, I would have to say the people make up one of the best aspects of working at the lab. It has been a pleasure getting to know my new colleagues and customers.

Our move to Tennessee went very well, and Sydney and I are planning to close on a home up here this month. It’s been a big change for us, but we were very much ready to leave Florida. The changing seasons, smaller urban area, and ready access to the Great Smoky Mountains National Park are much more to our liking than the urban sprawl of Tampa bay. So far we have been on three hikes to the Smoky Mountains and each of them was breathtaking. We once again live someplace where we can truly enjoy the outdoors. (We got a bit spoiled living in Colorado and Alaska!)

Moving up here was a big adventure, but I am happy to say that everything is going very well. The relocation assistance provided by the lab has been tremendously helpful and we are starting to feel very much at home here. I’ll post some pictures of our hikes soon, and do some updates from time to time on what projects I am working with at the Lab and other interesting news.