MacPorts for Weather and Science

Not much of an actual blog post as much as a reminder list for myself when it comes to setting up a new mac for developing code for science. A while back I posted an entry about selecting between MacPorts and Fink (link) so I won't discuss that here... bottom line is that they are both great and both work, it mostly depends on which you are more comfortable with and secondly specific individual ports that are not in both, but the number is few.

Anyways so here is my list that I install (and in order of how I installed them the last time):

1. gcc (latest version available unless otherwise needed, at the time of writing this, gcc45 is the latest and includes gfortran)
   
2. gcc_select (with mac ports, I don't think it is available on fink, but there is probably a similar package)
3. gnuplot
4. python (latest 2.x and 3.x versions)
5. python_select
   
6. netcdf, netcdf-bin
7. hdf5
8. grads
9. vis5d
   
10. gdal
11. grass
12. gmt
13. gnupg
14. wget
15. py26-setuptools (or latest/equivalent to the version of python installed)
16. py26-numpy
17. py26-scipy
18. py26-matplotlib

Now this is the list I've installed specifically, I won't even begin to list the number of packages that are prerequisites for packages that are installed as well.

Eclipse, Subversion and Mac OSX

Late last month (June) Eclipse released Ganymede (3.4).  Typically upgrading to the latest release of Eclipse is a no brainer for me, since typically I'm already running the latest beta version, but surprisingly, not this time. I guess I just had too many other things to do that I just never got to it. So when I read the release post on the Eclipse site, I simply downloaded and installed without thought or question... bad idea... well at least on my Mac, I haven't upgraded my Linux system yet.

My typical install process of Eclipse is pretty straight forward: download, replace and install plugins.

1. download: I typically select 'IDE for Java developers' and this time was no different
2. replace: this is really a two step process, 1. archive current installation, 2. unzip latest version
3. install plugins: since I write programs in a number of languages I need IDE feature addons for all of them, plus there are a number of plugins I like that make my life easier.
   the shortlist:
• CDT: IDE for C/C++ coding, yes I could download the CDT version of Eclipse, but last time I did that, I found that it didn't include the Java IDE and I find it easier to add on the CDT then the other way around.
• Photran: IDE for Fortran coding (yes I often write/fix code written in Fortran), this too can be downloaded as a separate IDE but last I checked it had the same issue with Java as the CDT download.
• Pydev: IDE for Python
• Texlipse: an 'IDE' for editing and writing TeX documents.
• Afae: a generic editor enhancement that adds color syntax for many different 'languages', this comes in handy with Unix shell script editing. (also recently the developer/s added an update site to make it easier to install)
• Subclipse: SVN client add-on for Eclipse

Well there you have my basic list. And yes I use them all on a weekly basis. The availability of such plugins/add-ons is why I use Eclipse and endorse it and recommend it, plus the fact that it is available on almost every platform that has Java and works the same on all those platforms, why would you want to use another... but praising Eclipse is not what this post is about...

So as you are probably starting to guess based on the post title, Subversion on MacOSX is where my issues began. Basically without even thinking, I upgraded Eclipse... but at the same time I upgraded subversion via macports... the combined two updates is where things went nuts. The timing was actually completely coincidental, after the last Mac OS X software update, a number of packages on my macports list needed to be updated including subversion.

Well I can't answer for Windows or Linux (since I haven't tried it yet, but I've had a number of other issues with subversion 1.5 on a Centos box I manage, but that will be another post once I figure out how to fix it), but on Mac the latest version of subversion does not work with the latest version of Subclipse on Eclipse 3.4, well maybe it does, but it doesn't for me. At the same time Eclipse has selected the Subversive group to provide the built-in svn team support for Eclipse. Why they selected Subversive over Subclipse, which is developed by the same group that is behind subversion, is beyond me, but in reality the question should be "why did Subclipse retract thier proposal to provide subversion support in Eclipse". So again, here is where my troubles began... when I noticed that Subclipse didn't work I first tried Subversive... that didn't work. That was also when I realized that it may be because of my upgraded of Subversion not Eclipse/Subclipse so then I started to work on it from that angle.

So first I re-installed Subclipse, then I uninstall subversion and install via the binary at Tigris, only to then read reports of issues on mac, plus it didn't fix any of my problems, so I then uninstalled that package and went back to the macports version, of course this meant that I hadn't fixed anything. Then at the same time (as eluded to above) I started noticing issues with svn on one of my linux boxs which lead me to think based on research on that end that it was not an svn issue, but a client access issue. This is the point at which I learn that I needed something call subversion-javahlbinding. Well I quickly installed it via macports... no help.... then I tried installing the package available on the Subclipse update site... that went really bad... Eclipse would crash about 10 seconds after startup. So I had to start over with my 'replace' cycle and install all my other IDE plugins again. This time I made an additional backup of Eclipse with everything minus subversion support.

Anyways, well since Subclipse wasn't working with the JavaHL, I tried the SVNKit which I read about... that too didn't work, so again I decided to go with Subversive. Again I tried to install subversive... and again it didn't work. Then I read that only the UI was included in the Eclipse Ganymede update site and not the connectors. So I installed the connectors from to Polarion site and low and behold, SVN support was working, kind of. I could at least browse my svn repositories it just wasn't working with my projects. Regardless when I noticed svn was working, my mind was made up, I was going to use subversive... what I didn't realize is that it takes some work to migrate to subversive from subclipse, and I only realized this after I hosed the local copy of one of my projects... thankfully I had a backup of the latest version on another machine I was testing on. This is when I read the 'proper' way to move from subclipse to subversive... basically after some digging through the forums site for subversive I found that a number of users had similar issues, this makes sense since for many years now, subclipse was the only option for subversion access in Eclipse.

So basically here are the steps I had to use:

1. install Subclipse, plugin only, not all the extra subversion packages it 'recommends' or chances are Eclipse will crash after startup.
2. restart Eclpise
3. 'Disconnect' all projects using Subclipse
4. restart Eclipse
5. uninstall Subclipse
6. restart Eclipse
7. install Subversive including svn connectors not available in the Eclipse repository that are available on the Polarion update site 
8. restart Eclipse
9. one by one, 'Share Projects' for each project that was previously shared via Subclipse and select 'use existing information'
10. cheer since you should hopefully be done and all should now work using Subversive, well at least as far as I can tell, it works on my end.

So basically that is it... that is my rant and my solution for using Subversion(1.5) on Eclipse (Ganymede) on Mac OSX(10.5.x). Note this is not an 'endorsement' of Subversive over Subclipse, but in my mind since Eclipse has already done that, we might as well join them. As for my 'solution' I'm sure there are other ways, but this one worked for me....

Fink vs MacPorts

Well a few weeks ago I decided that my Fink installation had gotten crazy. There were tones of apps and libraries installed that I needed at one time but no more. I decided it was time to start fresh. Basically I've been using Fink since the Mac OSX 10.0 days. For the most part I've just simply upgraded over the older versions and continued right along. With Apple's ability to migrate between machines, I've even migrated my Fink installation a few times. This all said, it was pretty nasty, so I decided it was time to clean house, also I've been programming in Python lately and realized that I had ended up with multiple versions of Python and add-on modules. So again another reason to flush and start clean.

So first step: 'sudo rm -rf /sw', wises first step, maybe not, but in reality I didn't care. Second step install Fink again... well I didn't do that. A few years ago I gave DarwinPorts a try, liked it but many of the needed libraries and such just weren't available and the few that weren't in Fink but in Ports, I was willing to manually install. But sometime last year (maybe the year before) DarwinPorts became MacPorts and was moved to a host sponsored by Apple. Whether or not this means that Apple endorses or supports MacPorts over Fink is not mentioned anywhere [side note: the name Fink is German for Finch].

Anyways... I decided to give the latest MacPorts a try. Initially (and still am) I was impressed by the available list of ports and its simplicity of use. Fink is based on the Debian package system, while MacPorts is based on the BSD Ports system. I personally am a Debian fan and a Ubuntu desktop user, but Mac OSX is and since the NeXTStep days been based on BSD, so presuminingly, the BSD model should be a better match. Overall both Fink and MacPorts provide mirror functionality, so it should make no difference which route you choose.

So again back to my discussion.....

Well I chose MacPorts this time. Installation - no problem, usage - no problem (besides naming/systax the only main difference is that for MacPorts you need to implicitly call 'sudo', with Fink, it is integrated), package availability - no problem. So where do the problems begin... well they don't... well sort of... well they have nothing to do with MacPorts vs Fink... they have to do with the fact that it is nearly impossible to remember every package you need, want or use. Now I'm remembering why atleast since 10.2 I've always simply upgraded my Fink and not done a full re-install.

Now some packages are easy to remember, like the latest GCC (4.4 with gfortran), and Latex (or Tetex). And since I'm working in the meteorological/science community, NetCDF, HDF5, GDAL, GRADS, qGis, GnuPlot and so forth. But then I added Python to my portfolio, I needed to make sure that I had the right libraries there too. ... This is where I think I went wrong....

When I first started learing Python, I simply found online the packages I needed/wanted, then checked Fink and installed them. Since then (and since I've been using Trac on a project management site for work) I've learned about 'easy_install' for Python. So this time I decided to install most of my Python modules/packages the Python way with easy_install. Well maybe I did something wrong or what not, but I don't think I did something right... so who know... but when I tried to run some of my Python code, it didn't work. So well then I tried the MacPort equivalent to the Fink route, and yes I found that they too had many of the packages available I was looking for, so maybe the easy_install way doesn't work for a Mac, don't know haven't checked... but the bottom line is that in the process of trying to clean things out... I've made things worse so today... I've remove all the MacPorts stuff (sudo rm -rf /opt).

And now.... lets try this again........

Oh yeah... if you didn't guess about the results of MacPorts vs Fink... I personally still think that it doesn't matter.... so you choose.

Skim: a nifty PDF annotating app for MacOSX

Found a real cool freeware app for Mac the other day for annotating PDFs. I was looking for a way to fill in a NCAA bracket for a work pool and came across an app called Skim. The annotations won't show up in other PDF viewers but you can print them on top of the PDF through Skim (also can be printed to PDF via the standard mac print options). It was designed for annotating scientific papers....
Skim | Home
http://skim-app.sourceforge.net/