ooRexx 4.0 update

The good news is that the Hursley Symposium will take place as planned. The even better news is that it looks like a release of ooRexx 4.0 is going to be ready for the symposium. There is one showstopper still in the Mac version, which is in PULL from the console. This code has been rewritten in its entirety and now sends one of the cpu's sky-high without any further action. I guess it is not recognizing eol - but it is in need of serious debugging. The Symposium, which celebrates 30 years of Rexx, is announced on www.rexxla.org. You should all go!

NetRexx and Open Object Rexx news

Just in time to announce two great developments in the world of Rexx: NetRexx is in the open source process within IBM, and Open Object Rexx 4.0.0 is scheduled to be released not long from now.

For NetRexx this is great news, because the JVM language that was ahead of its time should be able to keep up with new developments, at least annotations, and maybe some more dynamic behaviour than the JVM allowed some 12 years ago.

Open Object Rexx 4.0.0 will solve the long standing shared memory problem (by elimination of shared memory requirements). This is good news because it will make a proper install of ooRexx on the Mac a lot easier. There is a nagging problem with the reading of shell input that needs debugging.

I also have a problem with the 64 bit PPC build: now the 64 bit Rexx interpreter is finally here, my G5 is no more. It fell over after I came back from the US last year, with unclear freezes probably caused by the aging water cooling. Now there is an eight core Xeon MacPro, but that is only good for an Intel 64 bit compile - unless I find how to do a cross compile.

PPC ooRexx

Safari is newer and better and finally supports the Google blogger control, so now I can write in bold and italic without switching to Firefox. The Intel port of ooRexx 3.1.2 is available some time already now; the PowerPC version is late due to the fact that I was not at home for more than two months now (and the PPC resides there).

3.2 has a new problem with BSF - we will be so happy when 4.0 comes out without shared memory for IPC and maybe the new Java 1.6 scripting support built in.

ooRexx 3.1.2

Another year, another Rexx release. Busy with the preparations for 2 (two) presentations at the annual Rexx Language Association Symposium, this year in Tampa, Florida ( http://www.rexxla.org ).

The Mac builds of Open Object Rexx 3.1.2 are going smoothly, as is the testing, and the only lingering problem is that BSF4Rexx fails in an obscure way. Yesterday I added the -e switch to startup the interpreter with a command line script, this was needed in order to integrate ooRexx with the Automator application that is standard on MacOSX Tiger.

I also was experimenting with bridging Applescripts to ooRexx, and had some fun with the Skype API. Finally I can know who are my Friends with a simple API call.

The Automator is a very funny program, allowing you to piece together a workflow (of things to automate) using prefab blocks of functionality with pipes and filters, passing data either through stdin or using command line arguments. Very much like a space age version of VM Pipelines. Polishing in between stages can be done using scripts (be it shell, perl, php, python or ruby - and since yesterday also ooRexx). This is not yet in the release version, but if you really cannot wait, contact me with the react button.

Why I truly, honestly dislike Hibernate

It was such a promise and of course such a letdown. Model driven approach, forward engineer your Java classes, have Hibernate persist them to rdbms behind the scenes. It was the single greatest risk of the project.

First it took ages to incorporate into the generator some of the history and many to many options. Straightforward dog house style classes are easy, the rest requires serious investing in time. So we lost a quarter on that, trying to get it right - and not the smallest minds, but a combination of dbms aces and math majors.

Then - we could not delete objects. Just. Could not delete. Some spooky discord between lazy instantiation and session handling. Some things were not committed. Some things were not really multiuser, and the system balked at having two people working at the same time on an admin screen.

We had it investigated, some of those very highly paid highly focused nerds. It came out our session handling was 'all wrong.' Where you normally, the last 20 years I am in rdbms at least, open a session, have transactions and close it (or have it closed by the OS) when you are done, this thing refused to commit. And had a singleton session object, that forgot about your work when someone else used it. Brrr. So we had a very traumatic rewrite of session handling - ah yes, and the consultant deemed Hibernate2 so bad that we had to fall forward to Hibernate3.

The rewrite literally yielded hundreds of bugs. Simple select from things failed, and the session.iterate calls from their own Hibernate2 manuals were deprecated - it was laughable to use them, they did a select for every row. Presented in a way like we were stupid, only, we were not, because it was all in their own samples and nobody told us about the depth of this stupidity - in their own docs.

Now switched to a default of all lazy instantiation (and yes, the statements from a moderately deep inheritance hierarchy already ran out of DB2's JDBC statement buffer), we had to change SQL Query syntax to very unnatural select from StaffMember left join fetch HisName left join fetch HerSocialSecurityNumber etc. Otherwise you get Null Pointers.

So they managed to change query semantics so that everybody that knows his SQL is a total beginner again and has to adapt to funny quirks. So enough. Out with it. It is the emperors new clothes. Let it sleep.

Now the sad part of it is that in the years before this happened, we had written our own O/R layer, that, erm ... just worked. It was people that did not trust this beautiful small piece of technology that suggested using Hibernate for the big, important project. We thought by giving in, they would help us building the system. They did not. Darn them to Heck. Let them forever Hibernate.

The Workers Movement - Rexx users

Of course I keep my eye on other languages - Java, because its widespread use and excellent VM and Library, Prolog, because it is mighty interesting and lately Ruby, JRuby to be precise. We have seen the enormous inroads Ruby made, and it made me wonder why Object Rexx is not nearly as popular. (Not that I do not know. It is a rethorical wonder).

It is easy to blame IBM, especially because it probably is the guilty party in this. I like some things in Ruby, because they are done in a straightforward, totally consistent way, and I don't mean the Perl-isms that linger. Rexx deserves a clean up with deprecation of old stuff - in the teaching materials at least. Falling through procedures without an exit, 'expose', stem variables. Most of it is done, and we have to take care of a very large installed base. Of which todays vocal programming youth does not know anything.

I failed to see Ruby emerging the way it did, thinking it was a slow, slightly more OO variant of Python, which I sincerely dislike, even coming out of Amsterdam - it must be the indentation levels thing, reminding me of COBOL A-margin and B-Margin, being the last time that I struggled with whitespace, twenty years ago., before Python came around.

Ruby has got a great community and some visible protagonists, like Martin Fowler and Bruce Tate. And Tim Bray. I am afraid the Rexx community does not have this visible array of foremen; we are stronger in the department of system programmers and administrators that were not actually allowed to write applications but did thanks to Rexx. And people of the world, most of your payments go through with some Rexx application, transformation or maintenance program to thank for. We, the Rexx community, are a force of thousands, but thousands that do their work quietly and unnoticed.

The sad thing is that we had most of the things the community opinion leaders of Ruby like in Ruby, in Rexx, NetRexx and ooRexx, years ago.

IBM botched a lot. Moving development, losing some five years in the gestation of Open Object Rexx, charging for it to people. Coupling its name to OS/2 which it hated itself, being bred on very misguided MS cooperative principles - and then letting it drown, ruining its reputation in the process. IBM also saved it by Open Sourcing it in 2004.

We have to publish. Books and Articles. That do not say 'unknown, underrated.' But proud and showing off really advanced working stuff. Unlimited Numeric Precision, Correct handling of decimal numbers, Trace, Parse. Macro support for applications. Metaprogramming, meta classes, builtin Concurrency. At the same time there must be things added to keep up with the times. You would be amazed how many very important systems run on Rexx. Most of which we are not allowed to talk about. Drat.

Integrate Rexx dialects through BSF4Rexx

It's no secret that I would like to see the various dialects of Rexx converge. Object Rexx has done a marvelous job of being backward compatible with Classic Rexx; NetRexx has done wonders for integrating Rexx and Java by making Java classes out of Rexx, and BSF can integrate interpreted Rexx with Java very well. All can integrate with other code by means of external function libraries or JNI.

Unfortunately and unavoidably, there are some differences in syntax between the different dialects. No big problem, but awkward to explain to newcomers and good for the occasional 2 seconds of astonishment. So in NetRexx the method call is by dot (Datum.getDayOfWeek()), and in Object Rexx it is the famous twiddle, as in Datum~getDayOfWeek. As a consequence, NetRexx does not have Classic Rexx's stem notation, a kind of easy multidimensional and possibly associative array.

Now if we would forego the stem, we could have the dot notation for method invocation in Open Object Rexx, like in every other modern language, but it is probably not to be. I suggested this on the last Rexx Language Association Symposium in Austin, but the points of view seem deeply entrenched.

A recent very positive development is that on my suggestion BSF4Rexx now contains integration for calling NetRexx classes from Rexx without having to instantiate an extra NetRexx object - it will consider a NetRexx string a Rexx string and vice versa. Together with another fix for the handling of exceptions from Java in signal on syntax labels, we should be so very happy that our Rexx and NetRexx can at least seamlessly interface using BSF4Rexx.

BSF4Rexx Available on the Apple Macintosh

The Bean Scripting Facility is an IBM tool, since donated to Apache, for embedding scripting into the Java VM. There is a version for Object Rexx, that now also is available under MacOSX.

This is because it needs one native component to take care of the interfacing from Java to the (native) ooRexx interpreter - this is sharp contrast to NetRexx, that lives exclusively in the Java universe.

There are some advantages to using Object Rexx to interact with your Java objects. Firstly, it is easier to script OS commands and get the output back into your program. In Java, you need to do a Runtime.exec(), hook the filehandles for output and spawn a tread to look into those. In Object Rexx, the command is executed by putting it into quotes and sending it into an external queue.

Secondly, in Object Rexx it is possible and even easy to create runtime classes and add fields or methods to them using messages. For NetRexx and Java, we need to do awkward runtime bytecode modifications to accomplish the same goal, and still have class loader issues to run the newly composed classes or their instances.

Thirdly, Object Rexx has mixin classes and metaclasses that are simply not there in Java.

But the most important part about this, is that we now have greater freedom and flexibility to quickly implement our architectures or just run prototypes for feasibility studies.

A quick example using jdbc to a database:

/* ooRexx program to demo jdbc capabilities */
-- specify imports
jclass = .bsf~bsf.import("java.lang.Class")
driverMgr = .bsf~bsf.import("java.sql.DriverManager")

-- instantiate jdbc driver
jclass~forname('org.postgresql.Driver')~newinstance

-- make the dbms connection and open a statement
statement = driverMgr~getConnection('jdbc:postgresql:rvjansen','rvjansen','')~createStatement
signal on syntax
statement~executeUpdate("DROP TABLE test") -- catch exception for first time
syntax:
-- specify query and execute to get result set
statement~executeUpdate("CREATE TABLE test( name char(42), place char(42))")

statement~executeUpdate("INSERT INTO test (name, place) VALUES('Rony Flatscher', 'Vienna')")
statement~executeUpdate("INSERT INTO test (name, place) VALUES('Lee Peedin', 'Wallace')")
statement~executeUpdate("INSERT INTO test (name, place) VALUES('Rene Jansen', 'Amsterdam')")

-- select database content
rs = statement~executeQuery('select name, place from test')
do while rs~next
say rs~getString("name")~strip "from" rs~getString("place")~strip
end

-- calculate total
rs = statement~executeQuery('select count(*) from test')
do while rs~next
say "BSF4Rexx has at least" rs~getString(1) "fans!"
end

::requires bsf.cls

Object Rexx 3.1.1 on the Apple Macintosh

After a few weeks of hard debugging Open Object Rexx on MacOSX is a reality. Tomorrow the CVS repository will be tagged with the release. Some very hard to track down bugs were quashed, while it also became clear that MacOSX's System V Shared Memory call, shmem, leaves something to be desired. More shared memory segments to be exact, because now, just like Postgres, Object Rexx requires you to edit etc/rc to put in some higher values.
While this is ok as a stopgap measure, it is not terribly elegant, and should be avoided by doing the memory management in another way. Another thing to do.

The irony here is that in order to have the programming language I like most running on the platform I like most, I have to do low level work in the language I like least of all, C++. Time we bootstrap the thing.

Language keywords versioning approach

Java 1.5 again made it necessary to review code and prefix classnames that Java appropriated, and we had for years. This is a repeating process and it is not too bad, although it can annoy the hell out of me when I was planning to do other things. The principle here should be, who was first for this particular source file. The import mechanism should account for this problem, and also the set of keywords for the language should not be fixed, so that if I chose to use something that later became a keyword, my choice is what counts. We are not talking about perl, php and vb, that just change the whole language and let the users suffer or go bankrupt through it. No, we are talking serious languages like Java, that should have more support for this than language level switches on the compiler.

JSR292

There is a very interesting development in the Java language which is to put in support for dynamicly typed languages and hotswapping, the runtime addition and subtraction of fields and methods like Object Rexx, Objective C, Ruby and some other languages can do. Gilad Bracha, who blogged about it, doubts whether there can be full hotswapping support for a mandatory typechecked language like Java, but even if that is not feasible, it should be in for the dynamic language support.

My take on this is that is mostly better to have strongly typed constructs, but for speed, shortcuts and prototyping I am sometimes tempted into using those luxurious and rather untyped Rexx strings. Sometimes they stay, and at other times I opt for replacing them with (types) classes, mostly because I find it easier to keep a good oversight when things are called what they are. Rexx strings are their own unique type, because they allow you to do (multiprecision) arithmetic on strings, which is sometimes very welcome, mostly when doing things with strings. As we move to metaprogramming, by which I mean the runtime manipulation of types and classes and simultaneous use of those, we need to go more dynamic as we might be hampered by an extra compile cycle. Of course most cases where dynamic typing seems to be necessary can be solved by generating source and calling the compiler one more time.

Make again

Now what totally baffled me is that James Duncan Davidson's personal webpage states Make among the languages that he knows well. A bit further down the page he states that "I no longer, however, consider Makefiles to be evil. In fact, I’ve written several in the last few years." 

Now this is fine, and only troubling when you know that this is the man who made Ant, the build tool of choice of all Java persons. To build something using Ant, you have to write a Java program, at least if somebody else did not do that yet. Netbeans was ruined for me by Ant, and in its latest versions really deletes your sourcecode when it does not understand a cross language reference or something else in that vain. Actually, he also left Java and went to Ruby, which is a sensible choice, if you are in a position to choose. He also likes Objective-C and Cocoa, which is also sensible. Let's write off Ant to his job at Sun, then. I would like though, that we could undo some of Ant's damage, and have, for example, NetBeans working again with "make" projects.

Scott's Pragmatics has scripting chapter now

The good news is that the latest version of Micael Scott's "Programming Languages Pragmatics" has a whole new chapter dedicated to scripting languages, and mentions Rexx various times. The bad news, however, is that coverage is sketchy, no doubt because it is impossible to know every programming language intimately, but to Rexx this is particularly unfair, because although its age is mentioned and it gets a honorary mention as forerunner of this whole field, lots of the good parts of Rexx are largely neglected.

Rexx is even not mentioned in the chapter that has a short description of every language that is mentioned in the book. Rexx's great assets, like unlimited precision arithmetic out of the box. the unique 'parse' and 'trace' statements, incredible string handling, object orientation including metaclasses, runtime addition of methods and properties, reflection and concurrency, are not mentioned at all. It might be that it is an old language and some languages have caught up with it more or less, but that does not mean that we need not mention the genesis of thing in the right historical order. The problem was, I think, that Rexx has been closed-source IBM proprietary for a long time of its existence; if it had been opened up earlier there would have been no reason for Python and Ruby. Larry Wall, who did Perl, visited one of the first Rexx Language symposia to explain what he thought that Perl had to offer, and indeed the similarity between Object Rexx and Ruby is enormous.

I think it is our duty to be at the same time thankfull to Scott for mentioning the root of scripting and mentioning Rexx in his book, but also to correct him in his fairly superficial treatment of our favourite language.

Shoddy Typing needs more typing

What Bruce's book did make me realize was that although I agree on having some typing eliminated, his arguments against strong typechecking are not that sound. Actually, we get the impression that he sees it as a nuisance causing clerical work; that is not a very good argument against it. If you look at what Rexx does, it gives you a type that can be String and numeric and treats it the same way; now if you go and do impossible things, you'll be flagged at runtime. NetRexx also has almost the same approach, but it also blends in with Java's type system,  so there is actually a wide choice for the programmer to choose from. I have the distinct impression that strong typing is an asset and not a liability; *when your model is sound* strong typing (and we even type our yes/no binary schemes to a more specific type than boolean) only helps you to avoid errors. When dealing with metadata in your application, like model- and meta model data, it is essential in avoiding layering errors that are too easy to commit without strong typing. Where he does hit the hammer more or less on the nail is in the difficulty of having generic algorithms when strongly typing, but this also can hint to a too procedural approach, in which parts of the algorithm are not well hidden in the objects. This is called polymorphic; I am sure that Bruce knows.

Beyond Java

Bruce Tate has written "Beyond Java" and I am mostly in agreement with it, although it is repetitive, hastily written, and another example of something written out of negativity that had very well could have been written in a positive way. But of course, if "Bitter Java" was one of the previous bestsellers, it would be hard to resist.

The first question to pose is of course why it was never necessaty to write a "Beyond COBOL" or even "Beyond PL/1". It might be because it is not yet evident that there is a reason to go beyond Java and Tate knows that very well.

Secondly, as also observed by a colleague of mine, the book freely equates languages to frameworks and sings the praise of Ruby in the form of Ruby on Rails. It is very eclectic in its choosing of alternatives, and, for example does not mention Open Object Rexx or NetRexx, much older languages in the same vein, which slowly but surely are receiving more exposure.   
What has become very tiresome are the kayaking intros to every chapter, like "I was foolishly conquering a very dangerous waterfall when it occurred to me that multiple inheritance can be a lot of trouble". Well, it's no quote but you know what I mean.

The Friendly User

What is particularly dangerous for a system is users that are accepting its flaws because of lack of background in how fast things generally move in a computer. I am parking in a garage that is remotely operated, from Rotterdam to Amsterdam. Once in a while the system thinks my car is in, when actually it's outside; the consequence is that it does not let me enter the carport. Now when I call the service number, the friendly service person at the other end thinks it is normal to have a login sequence of 15 minutes and that it takes half an hour to flip the flag that indicates the in or out position of my car. Yesterday I heard from a more knowledgeable person in the garage company that a software upgrade went sour somewhere last week and the response should be much faster.

Another example of this is when the OS/2 TCP/IP driver in its famous DOS box failed, it had to recover for about 20 minutes, and then it restarted. At the bank where I worked the users duly noted performance problems; which were checked by staff and then closed. When it worked, it worked fine. In reality, the users should have reported a defect (which it obviously was, with outages of more then 15 minutes), so a different crew would have been dispatched.

Logo

I loved Logo to pieces in the eighties. I had the Logo cartridge for my Atari 600XL, and I liked it so much that I bought the 64K ‘outboard’ for this machine even though I could not afford it and it make me walk in old jeans and sweaters for a year; not enough nodes to do anything useful in the standard 16K. Now a common misconception about Logo is that it only does Turtles. I spent numerous hours in 1984 to watch random walks of four turtles, but the real use out of it I got was in its capacity as an easier LISP-without-parenthesis. In fact I first understood recursion during my first endeavours with Logo. In Rexx I still use the “parse line first rest first” recursive parse technique that I learned with Logo’s First and Butfirst (LISP’s CAR and CDR). I was excited to see that LCSI still exists and even have their stuff running on the Mac. The manual from 1983 is the only thing I saved (because it smelled rather nice, and epitomized the eighties with pictures of happy people and happy turtles, in a orange and brown design that just became fashionable again). I look forward to use it again for little playthings and sometimes wish I had children to teach it to. Some things in the Mac version need to be fixed, but when they are, I’ll gladly pay for it.

From the archives: Application on Fire

It must have been around 92 or 93 that our system programmers group was tasked with delivering an application, because the application people themselves were swamped with more important applications. It was a kind of an early information kiosk app. We did this in a few days with two people, me and a good buddy, and everybody was happy. It used CICS, DB2 and Cobol, and a bit of S/370 assembler. It was fast. It contained the phonebook of our company and a lot of information pages that were refreshed by the HRM department staff. It ran fine and we had only one support call for it, ever. I took that call.

Pierre called, who was a fine drawer of cartoons, and a non-technical person. He complained that the application did not react very fast. I duly logged on to Omegamon, and saw that DB2 processed calls, most in about 0.2 s. I looked at CICS, and it ran OK. I looked at MVS and then at VTAM. I could not find the problem. But because lunch hour was approaching, I offered to hop over to the other building; it housed the restaurant and also the HRM department.

I popped my head around the corner, and spotted this particular performance problem: a very small trickle of smoke emanated from his terminal, forming a black-mini tornado going to the ceiling. Pierre looked a bit suprised when I yanked the plug from the socket in the floor, but he was happy again when we had a new terminal installed that afternoon.

RIP OS/2

Rest in Pieces, OS/2. Of course OS/2, though officially disavowed by IBM, is not really dead. The German company eComstation maintains the code and sells you an OS/2 if you want to buy one, though a bit pricey for my taste. I remember buying OS/2 1.1, or trying to. I was grilled by IBM people on the phone, threatening that I would “lose all my programs” if I installed it, asking what I’d do with it, suggesting I could just loan a copy from work until they located a supply in the Netherlands. My answer on their “why” question was that I would like to develop for it; this was met with a sad silence. But did I know it ran DOS programs very badly?
But seriously, I loved OS/2, and still boot it occasionally under Virtual PC, if only to hear the “whoop” sound if a folder is opened - remember sound on a pc was new then. A bug in OS/2 Warp sounded uncannily like an old Amsterdam tram. WPS was far ahead of its time, and of course OS/2 was the first worthwile and stable multitasking pc operating system. It allowed Rexx to enter millions of living rooms and corporate offices, where previously it was mostly confined to VM and MVS. I have the Microsoft OS/2 1.2 API books, which were of course (and deliberately?) incompatible with the IBM development kit. MS also fought to exclude Rexx from OS/2 SE, because it knew that it ate Basic alive. Later on I was involved in a leading edge OS/2 project that needed weekly builds of the OS to stay ahead of the bugs we uncovered; it only grew stronger. Twelve years later, Windows still needs 256M for what OS/2 did in 16M. OS/2 became useful in release 1.3 when IBM rewrote most of it in assembler. Also, in this version it finally could print. The 32 bit 2.0 version really was a better Windows than Windows, forcing Microsoft to release a bogus device driver architecture in W95 to break compatibility. Even that was trapped and emulated by IBM OS Wizards for most applications that “needed” it (all three of them).
When the company I worked for all those years decided to go Windows (undoubtedly inspired by cowardly IBM sales persons), we had to roll out Office in OS/2 Warp, because the then current hardware base did not run NT.
What IBM did to OS/2 is a crime perpetrated by management against their engineers and to the general public that was deprived of any competition until others came around.
It sold its pc’s halfheartedly with the wrong video drivers installed for OS/2, until every small nephew of my wife knew that you had to scratch this “o-esse-dos” thing immediately. And we are not even talking about marketing it with a stop-sign logo and nuns in a convent, or even naming it ‘half an OS’. It did not react at all when MS employees were astroturfing the OS/2 Usenet fora in working hours - this was the time that IBM Legal could have just shut down MS waving unspecified patent or publicizing MS fora mores. Sometimes I have dark images of IBM doing this deliberately to finally put a stop to all these antitrust cases the US Government foisted upon them.

Don't forget to sign the OS/2 Open Source petition! It will never happen though.

Caps Lock to the rescue

After a few months with the Matias Tactilepro I grew restless again when I noticed that ALPS keyswitches are not the same thing as Buckling Springs. They do not become softer as BS keys would, and keep a nasty 'tack' feel and are extremely noisy. Not that the Matias is a bad keyboard, I prefer it over a membrane keyboard anytime. So it was time to hook up my Unicomp again, the rightful heirs to the IBM Model M. As this is a PS/2 keyboard with DIN plug, I run it over a Sitecom USB-Dual PS/2 Adapter. The keyboard is a special one, a 3270 style emulator keyboard with 122 keys. Lamentably, the last system to support it well was OS/2, so it does excellent service on my OS/2 based P/390 mini-mainframe. It has no Apple key, which means that it has no Windows key - one of the more daft developments of the last decade.

So I noticed new stuff in Tiger's keyboard menu- the ability to switch these key functions to other keys - and YES: *caps lock*, that anachronistic holdover from the heavy typewriter era, can be switched to *command*, which is the official moniker of the Apple key. Emacs, of course, does not even blink, as I switch the function keys I am using in .emacs to available, working PF keys.

Ok Apple kernel guys: now make a driver for my 122 key 3270 Unicomp, so I can access all function keys, and do an Attn, SysRq, CrSel and exSel (not to mention ErEOF) when I want to. Who is the Apple OSX keyboard driver champion?

But the Caps Lock switch is pure gold: the most irritating, struck-by-accident key given an innocent, and when intentionally pressed, useful and necessary function.