C++: Tuple my new friend

So I've always been a fan of C/C++. At my previous position I was almost strictly using C only for my part of the project (we also used quite a bit of Java). Towards the end I got really involved with Python (which I wish I was using more of now), but by current job is 99% C++.
One of the things I liked about Python was the ability to return more than 1 value/object from a function. And late last week I ran into a situation where that is exactly what I wanted in CPP. Boost tuple provided just that.


#include <boost/tuple/tuple.hpp> 
using boost::make_tuple;
boost::tuples::tuple<float, float, int, float> getStuff()
{
     return make_tuple<0.1,0.2,3,0.4>;
} 

main()
{ 
     boost::tuples::tuple<float, float, int, float> tup = getStuff(); 
     if(tup<0>.get() == 0.1) /* cheer */ ;
     if(tup<1>.get() == 0.2) /* cheer */ ;
     if(tup<2>.get() == 3) /* cheer */ ;
     if(tup<3>.get() == 0.4) /* cheer */ ;
} 

While not as pretty as Python, but it works.
Nick

boost::assign my new friend

On my current project we do a lot of unit testing sometimes with ‘random’ data. No biggie right… well when you consider the fact that a lot of the attributes being tested are very large arrays and vectors…. coming up with random values can be time consuming and tedious… but luckily I found:
      boost::assign

Being able to set a vector quickly with the ‘+=’ operator is amazing, plus toss in the ‘repeat’ and ‘repeat_fun’ command and I’m golden.

#include <boost/assign/std/vector.hpp>
#include <cstdlib> // for ‘rand()’
// bring ‘operator+=()’ into scope
using namespace boost::assign;
const int LARGE_ARRAY_SIZE = 4200;

std::vector<double> largeV;
largeV += 0.1, repeat_fun(LARGE_ARRAY_SIZE - 1, &rand);

simple as that… not I’ve got a vector containing 4200 items. plus using boost::multi_array and some reshaping and I can massage that into a 2x2100 matrix or any size.

MacPorts for Weather and Science

Not much of an actual blog post as much as a reminder list for myself when it comes to setting up a new mac for developing code for science. A while back I posted an entry about selecting between MacPorts and Fink (link) so I won't discuss that here... bottom line is that they are both great and both work, it mostly depends on which you are more comfortable with and secondly specific individual ports that are not in both, but the number is few.

Anyways so here is my list that I install (and in order of how I installed them the last time):

1. gcc (latest version available unless otherwise needed, at the time of writing this, gcc45 is the latest and includes gfortran)
   
2. gcc_select (with mac ports, I don't think it is available on fink, but there is probably a similar package)
3. gnuplot
4. python (latest 2.x and 3.x versions)
5. python_select
   
6. netcdf, netcdf-bin
7. hdf5
8. grads
9. vis5d
   
10. gdal
11. grass
12. gmt
13. gnupg
14. wget
15. py26-setuptools (or latest/equivalent to the version of python installed)
16. py26-numpy
17. py26-scipy
18. py26-matplotlib

Now this is the list I've installed specifically, I won't even begin to list the number of packages that are prerequisites for packages that are installed as well.

when precision bites

I love math... programming it is even more fun, then mixing integer math with decimal (floating point) math.... even more fun:

In [90]: 26*.25 + 18*.75
Out[90]: 20.0

In [91]: 26/4 + 18*3/4
Out[91]: 19

In [92]: 18*3/4
Out[92]: 13

In [93]: 18.0*3/4
Out[93]: 13.5

In [94]: 26.0/4 + 18.0*3/4
Out[94]: 20.0

pesky '.0'

to Nest or not to Nest, and how many level should you go?

Well I'm working on some pseudo-code.... Personally I really like switch/case statements, but unfortunately they are not available in all languages, plus they aren't always useful for overlapping conditions. So for this instance, I have a series of '<' '>' conditions that overlap. I can easily just do it with a series of 'IF' statements and not use any 'ELSE' statements, but that mean that you have to evaluated every IF every time ... so then there are nested IF statements. Nested statements are great, pretty much bread and butter of programming, but especially IF statements, they tend to become hard to read after awhile.

if condition1:
  ...
  if condition2:
    ...
  else:
    ...
else:
  ...
  

that is pretty straight forward, but in my case I have:

if condition1:
  ...
if condition2:
  ...
if conditio3:
  ...
  if condition4:
    ...
  else if condition5:
    ...
  if condition6:
    ...
  else if condition7:
    ...
    if condition8:
.......
  

you see where this is going.... even with proper indenting, this can be hard to read, and without, no hope. Comments help, but should code ever be that complicated?  So when is nesting to much? what are other options? But then again is the efficiency of the checks more important? Many questions that we all must deal with when writing code.

Python make pretty pictures

Well as those who know me, know that I've been on sort of a Python kick lately. A few months ago I started learning Python. So now that a few months have past I'll share the latest reason I like Python... Graphics....

Early in my time with HRD I started working with graphical products. Initially I was playing with a product called IDL. I thought the product was great, but we had a mathematician in join our group who was way more advanced with IDL than I and I let him take control of the product and was very happy to move on to other projects. This said I've always been interested in computer graphics as far back as I can remember... in grade school it started with Logo, and then Graphics on an Apple IIe.... Back to the some what present... even in grad school I took a course on OpenGL, even made a nifty 3d maze game. Anyways again I digress (pretty typical of all my blog posts...)

So graphics has always been something I've wanted to get into, plus with the always evident idea in my group of making our product portable and deployable, I also new one day that I'd have to find a non-IDL/commercial solution. I state non-commercial as well since I could also have looked at MatLab or such. Anyways the first two products that leapt out at me a while back were GDL a GNU clone of IDL and Octave, another GNU project compatible with MatLab. The only problem was that GDL at the time did not support many of the IDL tools we were now using, and Octave... well I still don't know MatLab so that wasn't going to work.

Anyways.... so I place the idea of a free and distributable version of our product tools on the very back burner... you know the one that was never even attached to the stove...

So we move foward in time to the real present... Like I said in the beginning of the post, a few months ago I started learning Python. Like learning any new programming language, it is mostly syntax and package/module discovery. The syntax part was straight forwards, I'm a C/Fortran/Java/Unix scripter.... so yes, Python syntax, not anything new. As for packages... well they are abuyndant, but easily managable. Unlike Perl which I don't even know where to start on how large it has gotten (and yes in my early days I was even a Perl programmer), trying to find the 'best' package can take as long as trying to write your own... So as for Python, I quickly learned to read and write ASCII files and perform some simple mathematical calculations on data and a few other simpler tasks.

Then a co-worker of mine asked me about writing a program for a project he was looking into. The project seemed straight forward enough... read in a couple years worth of data (all of which were simple 2d matrixs), perform some simple stats on them (mean, std, median, min, max), spit out a simple ASCII summary, and generate some pretty pictures. All of the first steps seemed straight forward, the pretty pictures... well that was a different story. Anyways still being on my Pytohn kick, I quickly started out on this new project in Python. Took a couple of days to get the files read and stats processed, was even easier when I learned about NumPy and SciPy. NumPy is great, it provides great array handling for large matrixes and such. Plus now the reason for this post... an additional package called MatPlotLib is also part of the NumPy/SciPy family. MatPlotLib made plotting the matrixes an almost breeze. Basically once I figured out my dimensions and the geospatial area for my map to plot on, all I had to do was give it my NumPy array and say draw, and poof.... pretty picture.

Don't ask me what the picture is exactly of, but ain't it pretty? I do know that it is some sort of thermal data from the GOES satellite (or atleast I think it is from the GOES satellite). But ain't that cool... all I did was create Basemap and then tell it to plot using the pcolor(...) command. It was that simple. Another reason I think Python and I are going to have a long happy relationship. A few more tweaks and I should have my program fully ready to go... only issue is... currently it is kind of slow.

Fink vs MacPorts

Well a few weeks ago I decided that my Fink installation had gotten crazy. There were tones of apps and libraries installed that I needed at one time but no more. I decided it was time to start fresh. Basically I've been using Fink since the Mac OSX 10.0 days. For the most part I've just simply upgraded over the older versions and continued right along. With Apple's ability to migrate between machines, I've even migrated my Fink installation a few times. This all said, it was pretty nasty, so I decided it was time to clean house, also I've been programming in Python lately and realized that I had ended up with multiple versions of Python and add-on modules. So again another reason to flush and start clean.

So first step: 'sudo rm -rf /sw', wises first step, maybe not, but in reality I didn't care. Second step install Fink again... well I didn't do that. A few years ago I gave DarwinPorts a try, liked it but many of the needed libraries and such just weren't available and the few that weren't in Fink but in Ports, I was willing to manually install. But sometime last year (maybe the year before) DarwinPorts became MacPorts and was moved to a host sponsored by Apple. Whether or not this means that Apple endorses or supports MacPorts over Fink is not mentioned anywhere [side note: the name Fink is German for Finch].

Anyways... I decided to give the latest MacPorts a try. Initially (and still am) I was impressed by the available list of ports and its simplicity of use. Fink is based on the Debian package system, while MacPorts is based on the BSD Ports system. I personally am a Debian fan and a Ubuntu desktop user, but Mac OSX is and since the NeXTStep days been based on BSD, so presuminingly, the BSD model should be a better match. Overall both Fink and MacPorts provide mirror functionality, so it should make no difference which route you choose.

So again back to my discussion.....

Well I chose MacPorts this time. Installation - no problem, usage - no problem (besides naming/systax the only main difference is that for MacPorts you need to implicitly call 'sudo', with Fink, it is integrated), package availability - no problem. So where do the problems begin... well they don't... well sort of... well they have nothing to do with MacPorts vs Fink... they have to do with the fact that it is nearly impossible to remember every package you need, want or use. Now I'm remembering why atleast since 10.2 I've always simply upgraded my Fink and not done a full re-install.

Now some packages are easy to remember, like the latest GCC (4.4 with gfortran), and Latex (or Tetex). And since I'm working in the meteorological/science community, NetCDF, HDF5, GDAL, GRADS, qGis, GnuPlot and so forth. But then I added Python to my portfolio, I needed to make sure that I had the right libraries there too. ... This is where I think I went wrong....

When I first started learing Python, I simply found online the packages I needed/wanted, then checked Fink and installed them. Since then (and since I've been using Trac on a project management site for work) I've learned about 'easy_install' for Python. So this time I decided to install most of my Python modules/packages the Python way with easy_install. Well maybe I did something wrong or what not, but I don't think I did something right... so who know... but when I tried to run some of my Python code, it didn't work. So well then I tried the MacPort equivalent to the Fink route, and yes I found that they too had many of the packages available I was looking for, so maybe the easy_install way doesn't work for a Mac, don't know haven't checked... but the bottom line is that in the process of trying to clean things out... I've made things worse so today... I've remove all the MacPorts stuff (sudo rm -rf /opt).

And now.... lets try this again........

Oh yeah... if you didn't guess about the results of MacPorts vs Fink... I personally still think that it doesn't matter.... so you choose.

New Wishlist

Last month included my birthday, and like usual, no one new what to get me.... that has mostly to do with who I am, I'm picky. Anyways... a few years ago for the holidays, I created a wishlist to help people out. My research actually started when I was trying to build a 'Fair Trade' wishlist for our wedding. Anyways... I knew that many stores had lists, but I wanted something that could handle outside items. I initially tried and used a site call MetaWishlist, it worked for me, but not for our registry... I liked it but is was in way early beta and eventually I got tired of it. At the same time I used GiftHat for our fairtrade registry. So when I heard that the developer of GifHat had an import option for MetaWishlist, I migrated. So for the last few years Gifthat has managed my wishlist, and managed it well. I on the other hand was terrible at updating it and a number of the products, well they were no longer available. This all said, last month my brother remembered that I had a list and managed to find on Google, and bought probably the only item on it I had added in the last year, a Creative Zen media play (other than that, the only thing I've added to my list was books for me to buy). It was probably one of the better gifts I've received in many a year, not because I selected it, but because it was something I really wanted but had no idea some one would or had bought it for me. So it reminded me the value of a wishlist. So today I was looking through my wishlist, and with the exception of a few items, almost everything was from Amazon. Not just that, the items that were directly from Amazon, they were actually there through the partnering shops (most of my items were from ThinkGeek and DealExtreme, both list many if not almost all their items on Amazon). Now while I've heard that Amazon does not treat there employees the greatest, you can't argue with the amazing selection. So basically I'm announcing my new wishlist which is available on Amazon, actually there are 3 wishlist, 1 for generic stuff, 1 for movies and music (and other related stuff), and a 3rd which is basically book about programming and such learning materials that are on my list to eventually buy.


http://www.amazon.com/gp/registry/wishlist/1ETY9C5IV9LDV

When White spaces aren't White

As most know, most of my programming work relates to data processing. One of my responsibilities with the Hurricane Research Division and the H*Wind team to be exact has been to process data and insert it in our database. So over the years (almost 12) I've written a number of programs to help along my way. A few years ago (before I discovered Python), I wrote a rather significant program for my own use. It is a generic parser. I wrote it as part of a project to re-analyze Hurricane Katrina. We had data coming in from all over the place and almost none of it in the same 'format', format in quotes since I refer to column order, field delimiters, units, etc. but almost all the data was ASCII. Anyways... I'm digressing from my point... so this parser was basically designed so that the user (basically me) would simply list the column order and such, and out would pop a file ready for the database. Well to the point of the blog post title, today (and yesterday) I was trying to figure out why my program wasn't working. The default delimiter in my program is WHITESPACE, basically any not alpha-numeric-symbol character, aka space, tab, return.... well incomes this file I processing... it was generated in Fortran, so my first assumption is that it should be space delimited, but it was not, it was TAB delimited. Okay no problem, my program can do that, but it didn't. For some reason, TAB is not being recognized as a whitespace.... argh, however when I distinctly tell my program with command-line arguments to accept TAB as a delimiter, it works.... argh..... Oh well, at least it works... thankfully. I sure the issue has something to do with being prepared on a HPUX machine, transfered to a Linux box, emailed to a Mac box and then processed.... you know somewhere in there something had to get lost in whitespace translation.... anyways... again the program works, but it is always fun when you write a program to do something and it doesn't work the way it should... thank goodness for Unix and the many workarounds that are available.

Of course this whole experience is another reason I'm really starting to like Python programming... maybe I should take the time to re-write this c program into Python... then again it works 99% of the time, so if it is not broke, why fix it?

Comments on: Ask The Readers: Best First Programming Language to Learn?

I was just commenting on the following post at LifeHacker:
Ask The Readers: Best First Programming Language to Learn?

So you can either scroll down and find my comment or just read it here:

---

You name it, and I've probably coded in it. That all said every language has its pluses and minuses, it all depends on what you want to do.

Now if you are just simply looking to learn how to program I recommend Python. Python has lots of power like most languages, but at the same time has a real-time interpreter that allows you to see results as you type them. Check out [python.org] they have some good links and good tutorials to get you started.

Now if you are looking for a more generic programming experience try a language like C, don't worry about C++ unless you really want to learn object-oriented programming and then I'd recommend Java. On the Java topic, Java is also great if you want to get into windowed programing (not Windows the OS, but application with GUI).

Now the three (4 if you include C++) I've listed are the 3 I primarily use since I develop for multiple operating systems at a time, so if you only care about a specific OS then, look at languages designed for that systems: VB or .Net possbily for Windows, or maybe AppleScript for Mac, etc. Also if you are looking to get into web programming start with something like JavaScipt and HTML.

Another thing, just because a language is a 'scripting language' does not mean it is not a programming language.

And again decide what you initial goal is first, there is no reason to learn VB if you never plan on working on a Windows computer, like-wise for AppleScript, and C++ won't help you much for web page programming. So as soon as you decide on your first project, then decide on a language. And after you learn one language, all the rest for the most part is just learning new syntax. That said, all programming is, is learning syntax.

Most of all have FUN! Coding can get tricking, be patient you can learn the basics quick but it will take a long time to be an expert so don't start over your head, there is a reason why most books start with the 'Hello World' program...

Learning Python

I'm a long time C developer. Basically I figure that anything that needed to be coded could be coded in C. I still hold that belief, but 7-8 years ago I came to the realization that while that may be true, it is not always the best answer. For project reasons at work I learned Fortran and began to recognize its value when it comes to math. For other project reasons I learn how powerful simple Unix scripts can be. So basically for the last 7-8 years I've seen Unix scripting as a steering tool, C for text processing, Fortran for math, and Java for object-oriented and gui tasks. But recently based on a number of article from Computing in Science and Engineering (CiSE) I've come to learn about Python.

Python is a scripting language at its core, but overall it is much more. Python has a variety of modules and add-ons. Some basic include modules include NumPy/SciPy for math and science, CSV for delimited text, and XML for xml parsing and generation. My first Python program took me only a day or so to develop and test, it was a CSV translator. Basically it converted one CSV file into another. My next program took me another 2-3 days and it converted a gridded file into a CSV formatted file. And today, I wrote my first XML parser. Using 'minidom' from the xml package I was able to put together a simple parser for converting an XML file into a csv file.

All this I've basically accomplished over a few weeks of learning using only a handful of simple resources both online and in print. Yes I have many years of experience with a variety of programming languages, but I definitely have found Python the easiest to learn yet. Here is a quick list of resources I've found helpful:

• Tutorial at Python.org
• Python.org Library Reference
• Dive into Python by Mark Pilgrim
• Python/XML Howto by A.M. Kuchling
• Python XML Frequently Asked Questions by Dave Kuhlman
• Book: XML Processing with Perl, Python, and PHP by Martin C. Brown
• Magazine: CiSE issue: Python: Batteries Included Volume 9, Number 3
• Python and XML: An Introduction
  
• Creating XML with Python

I've bookmarked a number of other sites for future use but have not gotten to them yet. My next task will be to add a simple XML version of the gridded data file mentioned above, plus add NetCDF writing support and GIS writting support, namely ESRI Shapefiles. I also plan on learning howto wrap my C packages with Python binding. To put it plainly, things that I've spent years developing and testing in C for parsing weather related data, will all slowly be phased out and replaced with python code and modules and all opensource.

update 2008-03-19: added 2 more links

XML Modeling

Well I've been trying to teach myself XML over the last few weeks. Overall I don't see it as that difficult. I've worked with XML data in the past but usually through various interpreters and other applications. This has been my first 'experiment' with developing and XML Schema.

Modeling XML is a pretty straight forward idea, and pretty simple... now doing it right.... So anyways I initially started out by buying a book, after reviewing it from the library I picked up XML By Example by Benoit Marchal for <$5 on Amazon. Then without touching the computer I first read the first 4 chapters and briefly skimmed the rest of the book. I stopped after Chapter 4 since it is on XML Models. So at that point I began my first model... well I think I bit off more than I could chew. I went for designing a model to fit the entire set of formats found in the H*Wind project. Well like I said, much more then I could chew....

So I went back and decided just to model a single 'type' I chose the Storm Center/Forecast type since it also coincides with another project we are starting. Well I first went through the data file and decided which fields we needed for our purposes. I then defined my 'variables' or elements. I then started putting things together. Next I tried to build a sample data file based on the schema... what I found... my model sucked. So back to the modeling stage.... well I can see why this is not so simple now...

Well so I think I'll give it one more modeling try and then I might start looking for an app to reverse model an existing XML file.... I think my main issue may just be in the abstractness of modeling without data.

Draft of HTML 5 Hints at a Brave New Web

This week, the HTML Working Group, which is the arm of World Wide Web Consortium (W3C) charged with updating HTML, published the first official draft of HTML 5, which will eventually supersede HTML 4.

And here I am, trying to learn XHTML and CSS, yes I know I'm way behind schedule but then again for the last 10 years or so I've been developing behind the scene applications and Java applications... my web dev days are long out of date... so yes here I am trying to learn XHTML and CSS, trying to refresh my JavaScript knowledge and pick up Ajax, plus looking into Python/Jython, but again that is more for behind the scenes... I've already worked with J2EE an Servlets, didn't like JSP, but now to have to worry about HTML5.... couldn't they have released this a year ago so that I could just learn it... oh well such is the life of a programmer... always having to learn new technologies and always one day behind....

read more | digg story