Integrate Rexx dialects through BSF4Rexx

It's no secret that I would like to see the various dialects of Rexx converge. Object Rexx has done a marvelous job of being backward compatible with Classic Rexx; NetRexx has done wonders for integrating Rexx and Java by making Java classes out of Rexx, and BSF can integrate interpreted Rexx with Java very well. All can integrate with other code by means of external function libraries or JNI.

Unfortunately and unavoidably, there are some differences in syntax between the different dialects. No big problem, but awkward to explain to newcomers and good for the occasional 2 seconds of astonishment. So in NetRexx the method call is by dot (Datum.getDayOfWeek()), and in Object Rexx it is the famous twiddle, as in Datum~getDayOfWeek. As a consequence, NetRexx does not have Classic Rexx's stem notation, a kind of easy multidimensional and possibly associative array.

Now if we would forego the stem, we could have the dot notation for method invocation in Open Object Rexx, like in every other modern language, but it is probably not to be. I suggested this on the last Rexx Language Association Symposium in Austin, but the points of view seem deeply entrenched.

A recent very positive development is that on my suggestion BSF4Rexx now contains integration for calling NetRexx classes from Rexx without having to instantiate an extra NetRexx object - it will consider a NetRexx string a Rexx string and vice versa. Together with another fix for the handling of exceptions from Java in signal on syntax labels, we should be so very happy that our Rexx and NetRexx can at least seamlessly interface using BSF4Rexx.

BSF4Rexx Available on the Apple Macintosh

The Bean Scripting Facility is an IBM tool, since donated to Apache, for embedding scripting into the Java VM. There is a version for Object Rexx, that now also is available under MacOSX.

This is because it needs one native component to take care of the interfacing from Java to the (native) ooRexx interpreter - this is sharp contrast to NetRexx, that lives exclusively in the Java universe.

There are some advantages to using Object Rexx to interact with your Java objects. Firstly, it is easier to script OS commands and get the output back into your program. In Java, you need to do a Runtime.exec(), hook the filehandles for output and spawn a tread to look into those. In Object Rexx, the command is executed by putting it into quotes and sending it into an external queue.

Secondly, in Object Rexx it is possible and even easy to create runtime classes and add fields or methods to them using messages. For NetRexx and Java, we need to do awkward runtime bytecode modifications to accomplish the same goal, and still have class loader issues to run the newly composed classes or their instances.

Thirdly, Object Rexx has mixin classes and metaclasses that are simply not there in Java.

But the most important part about this, is that we now have greater freedom and flexibility to quickly implement our architectures or just run prototypes for feasibility studies.

A quick example using jdbc to a database:

/* ooRexx program to demo jdbc capabilities */
-- specify imports
jclass = .bsf~bsf.import("java.lang.Class")
driverMgr = .bsf~bsf.import("java.sql.DriverManager")

-- instantiate jdbc driver
jclass~forname('org.postgresql.Driver')~newinstance

-- make the dbms connection and open a statement
statement = driverMgr~getConnection('jdbc:postgresql:rvjansen','rvjansen','')~createStatement
signal on syntax
statement~executeUpdate("DROP TABLE test") -- catch exception for first time
syntax:
-- specify query and execute to get result set
statement~executeUpdate("CREATE TABLE test( name char(42), place char(42))")

statement~executeUpdate("INSERT INTO test (name, place) VALUES('Rony Flatscher', 'Vienna')")
statement~executeUpdate("INSERT INTO test (name, place) VALUES('Lee Peedin', 'Wallace')")
statement~executeUpdate("INSERT INTO test (name, place) VALUES('Rene Jansen', 'Amsterdam')")

-- select database content
rs = statement~executeQuery('select name, place from test')
do while rs~next
say rs~getString("name")~strip "from" rs~getString("place")~strip
end

-- calculate total
rs = statement~executeQuery('select count(*) from test')
do while rs~next
say "BSF4Rexx has at least" rs~getString(1) "fans!"
end

::requires bsf.cls

Object Rexx 3.1.1 on the Apple Macintosh

After a few weeks of hard debugging Open Object Rexx on MacOSX is a reality. Tomorrow the CVS repository will be tagged with the release. Some very hard to track down bugs were quashed, while it also became clear that MacOSX's System V Shared Memory call, shmem, leaves something to be desired. More shared memory segments to be exact, because now, just like Postgres, Object Rexx requires you to edit etc/rc to put in some higher values.
While this is ok as a stopgap measure, it is not terribly elegant, and should be avoided by doing the memory management in another way. Another thing to do.

The irony here is that in order to have the programming language I like most running on the platform I like most, I have to do low level work in the language I like least of all, C++. Time we bootstrap the thing.

Language keywords versioning approach

Java 1.5 again made it necessary to review code and prefix classnames that Java appropriated, and we had for years. This is a repeating process and it is not too bad, although it can annoy the hell out of me when I was planning to do other things. The principle here should be, who was first for this particular source file. The import mechanism should account for this problem, and also the set of keywords for the language should not be fixed, so that if I chose to use something that later became a keyword, my choice is what counts. We are not talking about perl, php and vb, that just change the whole language and let the users suffer or go bankrupt through it. No, we are talking serious languages like Java, that should have more support for this than language level switches on the compiler.

JSR292

There is a very interesting development in the Java language which is to put in support for dynamicly typed languages and hotswapping, the runtime addition and subtraction of fields and methods like Object Rexx, Objective C, Ruby and some other languages can do. Gilad Bracha, who blogged about it, doubts whether there can be full hotswapping support for a mandatory typechecked language like Java, but even if that is not feasible, it should be in for the dynamic language support.

My take on this is that is mostly better to have strongly typed constructs, but for speed, shortcuts and prototyping I am sometimes tempted into using those luxurious and rather untyped Rexx strings. Sometimes they stay, and at other times I opt for replacing them with (types) classes, mostly because I find it easier to keep a good oversight when things are called what they are. Rexx strings are their own unique type, because they allow you to do (multiprecision) arithmetic on strings, which is sometimes very welcome, mostly when doing things with strings. As we move to metaprogramming, by which I mean the runtime manipulation of types and classes and simultaneous use of those, we need to go more dynamic as we might be hampered by an extra compile cycle. Of course most cases where dynamic typing seems to be necessary can be solved by generating source and calling the compiler one more time.

Make again

Now what totally baffled me is that James Duncan Davidson's personal webpage states Make among the languages that he knows well. A bit further down the page he states that "I no longer, however, consider Makefiles to be evil. In fact, I’ve written several in the last few years." 

Now this is fine, and only troubling when you know that this is the man who made Ant, the build tool of choice of all Java persons. To build something using Ant, you have to write a Java program, at least if somebody else did not do that yet. Netbeans was ruined for me by Ant, and in its latest versions really deletes your sourcecode when it does not understand a cross language reference or something else in that vain. Actually, he also left Java and went to Ruby, which is a sensible choice, if you are in a position to choose. He also likes Objective-C and Cocoa, which is also sensible. Let's write off Ant to his job at Sun, then. I would like though, that we could undo some of Ant's damage, and have, for example, NetBeans working again with "make" projects.

Scott's Pragmatics has scripting chapter now

The good news is that the latest version of Micael Scott's "Programming Languages Pragmatics" has a whole new chapter dedicated to scripting languages, and mentions Rexx various times. The bad news, however, is that coverage is sketchy, no doubt because it is impossible to know every programming language intimately, but to Rexx this is particularly unfair, because although its age is mentioned and it gets a honorary mention as forerunner of this whole field, lots of the good parts of Rexx are largely neglected.

Rexx is even not mentioned in the chapter that has a short description of every language that is mentioned in the book. Rexx's great assets, like unlimited precision arithmetic out of the box. the unique 'parse' and 'trace' statements, incredible string handling, object orientation including metaclasses, runtime addition of methods and properties, reflection and concurrency, are not mentioned at all. It might be that it is an old language and some languages have caught up with it more or less, but that does not mean that we need not mention the genesis of thing in the right historical order. The problem was, I think, that Rexx has been closed-source IBM proprietary for a long time of its existence; if it had been opened up earlier there would have been no reason for Python and Ruby. Larry Wall, who did Perl, visited one of the first Rexx Language symposia to explain what he thought that Perl had to offer, and indeed the similarity between Object Rexx and Ruby is enormous.

I think it is our duty to be at the same time thankfull to Scott for mentioning the root of scripting and mentioning Rexx in his book, but also to correct him in his fairly superficial treatment of our favourite language.

Shoddy Typing needs more typing

What Bruce's book did make me realize was that although I agree on having some typing eliminated, his arguments against strong typechecking are not that sound. Actually, we get the impression that he sees it as a nuisance causing clerical work; that is not a very good argument against it. If you look at what Rexx does, it gives you a type that can be String and numeric and treats it the same way; now if you go and do impossible things, you'll be flagged at runtime. NetRexx also has almost the same approach, but it also blends in with Java's type system,  so there is actually a wide choice for the programmer to choose from. I have the distinct impression that strong typing is an asset and not a liability; *when your model is sound* strong typing (and we even type our yes/no binary schemes to a more specific type than boolean) only helps you to avoid errors. When dealing with metadata in your application, like model- and meta model data, it is essential in avoiding layering errors that are too easy to commit without strong typing. Where he does hit the hammer more or less on the nail is in the difficulty of having generic algorithms when strongly typing, but this also can hint to a too procedural approach, in which parts of the algorithm are not well hidden in the objects. This is called polymorphic; I am sure that Bruce knows.

Beyond Java

Bruce Tate has written "Beyond Java" and I am mostly in agreement with it, although it is repetitive, hastily written, and another example of something written out of negativity that had very well could have been written in a positive way. But of course, if "Bitter Java" was one of the previous bestsellers, it would be hard to resist.

The first question to pose is of course why it was never necessaty to write a "Beyond COBOL" or even "Beyond PL/1". It might be because it is not yet evident that there is a reason to go beyond Java and Tate knows that very well.

Secondly, as also observed by a colleague of mine, the book freely equates languages to frameworks and sings the praise of Ruby in the form of Ruby on Rails. It is very eclectic in its choosing of alternatives, and, for example does not mention Open Object Rexx or NetRexx, much older languages in the same vein, which slowly but surely are receiving more exposure.   
What has become very tiresome are the kayaking intros to every chapter, like "I was foolishly conquering a very dangerous waterfall when it occurred to me that multiple inheritance can be a lot of trouble". Well, it's no quote but you know what I mean.

The Friendly User

What is particularly dangerous for a system is users that are accepting its flaws because of lack of background in how fast things generally move in a computer. I am parking in a garage that is remotely operated, from Rotterdam to Amsterdam. Once in a while the system thinks my car is in, when actually it's outside; the consequence is that it does not let me enter the carport. Now when I call the service number, the friendly service person at the other end thinks it is normal to have a login sequence of 15 minutes and that it takes half an hour to flip the flag that indicates the in or out position of my car. Yesterday I heard from a more knowledgeable person in the garage company that a software upgrade went sour somewhere last week and the response should be much faster.

Another example of this is when the OS/2 TCP/IP driver in its famous DOS box failed, it had to recover for about 20 minutes, and then it restarted. At the bank where I worked the users duly noted performance problems; which were checked by staff and then closed. When it worked, it worked fine. In reality, the users should have reported a defect (which it obviously was, with outages of more then 15 minutes), so a different crew would have been dispatched.

Logo

I loved Logo to pieces in the eighties. I had the Logo cartridge for my Atari 600XL, and I liked it so much that I bought the 64K ‘outboard’ for this machine even though I could not afford it and it make me walk in old jeans and sweaters for a year; not enough nodes to do anything useful in the standard 16K. Now a common misconception about Logo is that it only does Turtles. I spent numerous hours in 1984 to watch random walks of four turtles, but the real use out of it I got was in its capacity as an easier LISP-without-parenthesis. In fact I first understood recursion during my first endeavours with Logo. In Rexx I still use the “parse line first rest first” recursive parse technique that I learned with Logo’s First and Butfirst (LISP’s CAR and CDR). I was excited to see that LCSI still exists and even have their stuff running on the Mac. The manual from 1983 is the only thing I saved (because it smelled rather nice, and epitomized the eighties with pictures of happy people and happy turtles, in a orange and brown design that just became fashionable again). I look forward to use it again for little playthings and sometimes wish I had children to teach it to. Some things in the Mac version need to be fixed, but when they are, I’ll gladly pay for it.

From the archives: Application on Fire

It must have been around 92 or 93 that our system programmers group was tasked with delivering an application, because the application people themselves were swamped with more important applications. It was a kind of an early information kiosk app. We did this in a few days with two people, me and a good buddy, and everybody was happy. It used CICS, DB2 and Cobol, and a bit of S/370 assembler. It was fast. It contained the phonebook of our company and a lot of information pages that were refreshed by the HRM department staff. It ran fine and we had only one support call for it, ever. I took that call.

Pierre called, who was a fine drawer of cartoons, and a non-technical person. He complained that the application did not react very fast. I duly logged on to Omegamon, and saw that DB2 processed calls, most in about 0.2 s. I looked at CICS, and it ran OK. I looked at MVS and then at VTAM. I could not find the problem. But because lunch hour was approaching, I offered to hop over to the other building; it housed the restaurant and also the HRM department.

I popped my head around the corner, and spotted this particular performance problem: a very small trickle of smoke emanated from his terminal, forming a black-mini tornado going to the ceiling. Pierre looked a bit suprised when I yanked the plug from the socket in the floor, but he was happy again when we had a new terminal installed that afternoon.

RIP OS/2

Rest in Pieces, OS/2. Of course OS/2, though officially disavowed by IBM, is not really dead. The German company eComstation maintains the code and sells you an OS/2 if you want to buy one, though a bit pricey for my taste. I remember buying OS/2 1.1, or trying to. I was grilled by IBM people on the phone, threatening that I would “lose all my programs” if I installed it, asking what I’d do with it, suggesting I could just loan a copy from work until they located a supply in the Netherlands. My answer on their “why” question was that I would like to develop for it; this was met with a sad silence. But did I know it ran DOS programs very badly?
But seriously, I loved OS/2, and still boot it occasionally under Virtual PC, if only to hear the “whoop” sound if a folder is opened - remember sound on a pc was new then. A bug in OS/2 Warp sounded uncannily like an old Amsterdam tram. WPS was far ahead of its time, and of course OS/2 was the first worthwile and stable multitasking pc operating system. It allowed Rexx to enter millions of living rooms and corporate offices, where previously it was mostly confined to VM and MVS. I have the Microsoft OS/2 1.2 API books, which were of course (and deliberately?) incompatible with the IBM development kit. MS also fought to exclude Rexx from OS/2 SE, because it knew that it ate Basic alive. Later on I was involved in a leading edge OS/2 project that needed weekly builds of the OS to stay ahead of the bugs we uncovered; it only grew stronger. Twelve years later, Windows still needs 256M for what OS/2 did in 16M. OS/2 became useful in release 1.3 when IBM rewrote most of it in assembler. Also, in this version it finally could print. The 32 bit 2.0 version really was a better Windows than Windows, forcing Microsoft to release a bogus device driver architecture in W95 to break compatibility. Even that was trapped and emulated by IBM OS Wizards for most applications that “needed” it (all three of them).
When the company I worked for all those years decided to go Windows (undoubtedly inspired by cowardly IBM sales persons), we had to roll out Office in OS/2 Warp, because the then current hardware base did not run NT.
What IBM did to OS/2 is a crime perpetrated by management against their engineers and to the general public that was deprived of any competition until others came around.
It sold its pc’s halfheartedly with the wrong video drivers installed for OS/2, until every small nephew of my wife knew that you had to scratch this “o-esse-dos” thing immediately. And we are not even talking about marketing it with a stop-sign logo and nuns in a convent, or even naming it ‘half an OS’. It did not react at all when MS employees were astroturfing the OS/2 Usenet fora in working hours - this was the time that IBM Legal could have just shut down MS waving unspecified patent or publicizing MS fora mores. Sometimes I have dark images of IBM doing this deliberately to finally put a stop to all these antitrust cases the US Government foisted upon them.

Don't forget to sign the OS/2 Open Source petition! It will never happen though.

Caps Lock to the rescue

After a few months with the Matias Tactilepro I grew restless again when I noticed that ALPS keyswitches are not the same thing as Buckling Springs. They do not become softer as BS keys would, and keep a nasty 'tack' feel and are extremely noisy. Not that the Matias is a bad keyboard, I prefer it over a membrane keyboard anytime. So it was time to hook up my Unicomp again, the rightful heirs to the IBM Model M. As this is a PS/2 keyboard with DIN plug, I run it over a Sitecom USB-Dual PS/2 Adapter. The keyboard is a special one, a 3270 style emulator keyboard with 122 keys. Lamentably, the last system to support it well was OS/2, so it does excellent service on my OS/2 based P/390 mini-mainframe. It has no Apple key, which means that it has no Windows key - one of the more daft developments of the last decade.

So I noticed new stuff in Tiger's keyboard menu- the ability to switch these key functions to other keys - and YES: *caps lock*, that anachronistic holdover from the heavy typewriter era, can be switched to *command*, which is the official moniker of the Apple key. Emacs, of course, does not even blink, as I switch the function keys I am using in .emacs to available, working PF keys.

Ok Apple kernel guys: now make a driver for my 122 key 3270 Unicomp, so I can access all function keys, and do an Attn, SysRq, CrSel and exSel (not to mention ErEOF) when I want to. Who is the Apple OSX keyboard driver champion?

But the Caps Lock switch is pure gold: the most irritating, struck-by-accident key given an innocent, and when intentionally pressed, useful and necessary function.

"Save As" Considered Harmful

It recently occurred to me that I was missing out on many opportunities for reuse of work because editors nowadays allow you to start on a blank page and do a "save as" until which there is no filename attached to that particular piece of work. In the days that I used ISPF/PDF as my sole editor, it repeatedly happened that I set out to do something I apparently did sometime earlier already. In editing a member of a partitioned dataset, you need to specify the membername first. And then, boom, there it was, the exact thing I was planning to enter, just because my naming algorithm seems to be stable and predictable. This saves you from remembering every little bit of maintenance work you ever did. It is a pity it went away, so now, when we do not remember the past, we are literally forced to repeat it.

Incorrectable improvements

The NetBeans story (about how it went from usable to bondage & discipline) goes further. After wasting an inordinate amount of time, we spent some more time at work, because just throwing away the NetBeans-generated GUI programs is not going to help anybody. So we got it to import the project. And the first thing it does is pop up a message box, suggesting to delete all the .class files because they are in the source path. WHY? It is correct they are there, we put them there after careful deliberation. We value a single classpath root. Some tools (even in the SDK) expect them there. It also makes for quick visual inspection of the generated classfile. But WTF should a tool care where I put what? We found the project properties. They were hidden behind a right click at a spot they did not used to be (and we never did need them, being able to mount the jars we needed). So now NetBeans also writes metadata all over the place, in places I need to find it to delete it again. Although we cannot drop it right away, we will be looking around. Or Coyote must be so brilliant, that I’ll be able to use it for ooRexx and NetRexx.

The sorry demise of NetBeans

Some people managed to totally destroy NetBeans usability for our project. They must have been taken over by the Borg. Let me explain. We have a fairly big application that is made more or less in Java. It is actually written in NetRexx, but that is not really the point here. For GUI development, we used NetBeans. Draw a screen, add widgets, doubleclick and add calls to our own methods. Great. This was up until NetBeans 3.6. And it was easy to add the project to NetBeans: our codebase is make and cvs (now subversion) based. Just add the classpath root to a NetBeans virtual file system, let it scan and it works.
In 4.0 and 4.1, not anymore. Not at all. This confirms my worst preconceptions about IDE’s. These people decided to just take out the extremely useful feature of virtual filesystems and make the thing totally ANT based. The new 4.1 release “is even more flexible” and adds “free form projects”. Free form, my ass. It immediately complains: cannot add project that already has a Build directory. I try now to add a “standard project” (where standard also means “ANT”). No dice, because it ‘is already owned by another project.’
I don’t want NetBeans to build my project. I just want to press F9 and compile. It does not let me anymore, and I waited patiently for 4.1 to correct the situation.
There is lots of docs going with this, touting its flexibility, though totally dense and milling on and on about ant. I already spent a lot of time on this, and it did not help. I think I have the option now to add all the packages and subpackages of the hierarchy by hand. They must have totally lost it. I already saw some complaints on the mailing list and was stricken by the sheer arrogance of those people knowing it better than those who suddenly lost their ability to work with the tool. So by moving and renaming a lot, I got NetBeans to digest the project. I edit a file, and IT GREYES OUT the compile option, probably because of some error in an ant file I don’t want in the first place.
There also seems to be a “blueprint” now for enterprise java project layout, without a doubt devised by ‘technical project manager’ people that never design or code but bestow ‘naming conventions’ upon us that do. But taking working functionality out of a tool to make people conform to your ideology is a very, very sick thing to do. So I stay at 3.6 until I find something better. Bye bye NetBeans, it has been fun while it lasted.

Choice

Write a reasonably complicated web application and the testers only complain about missing links and faulty graphics. This is great because these problems are easy to solve, if problems at all. Worse is that it takes the time that was needed to find and fix the real bug you know there must be out there.

This week saw Apple switch to Intel and Jamie Zawinski switch to Apple. Both worry me a little. I agree with Robert Cringely that the leakage of MacOSX for Intel is probably a plot to bait more switchers. For me the worriest thought though is that my main platform will be mainstream one day. The decision where to switch my primary web server to became much more complex this week, but most probably it will be a Mini Mac anyway, but I am seriously doubting MacOSX server in favour of Yellow Dog Linux; read some discouraging statistics on Mach-BSD thread forking in MacOSX Server. Not that it will ever be a high performance server anyway, but give me a break: 5 times more overhead on threading? It certainly calls for hyperthreading ;-).

Lets hope all turns out well. I am pro choice, so it would be better if Apple would just have introduced a parallel line of hardware architecture, giving people the choice to, for example, keep on buying the PPC machines for high end machines, for example with added reliability features like parity memory and a service processor. Or run OSX on IBM’s rock solid RS/6000 and other POWER hardware, so I won't have to complain in every post about how much more trust we can put in a mainframe compared to the dinky machines we trust our data with nowadays.

BSF und die umwertung aller werte

I was quietly working at the Mac port for Open Object Rexx and the related BSF4Rexx, when Apple dropped the bombshell of doing a "switch" for themselves, this time to Intel CPU hardware. They must know something we don't know, because switching your 64 bit OS to a 32 bit Pentium 4 (quad 3.6Ghz) does not really make any sense. Dvorak predicted the Itanium, but it turns out to be ordinary x86; we have to forego The Cell, which is a pity, or switch ourselves to Linux, which probably will run soon on it. I am glad my own stuff is all Java, it only muddles the picture a bit for the C++ ports, which are a headache anyway.

XCode 2.1 came with a new gcc 4.0, that has new compiler errors in previously compiling code; such is the price of progress. Another day, another ABI. I am still looking for a good porting guide from Linux to MacOSX, one that explains that thread_t is a structure in BSD and not a pointer, and why my files do not open when the code is compiled with Tiger, while they work when compiled in Panther. With some bad luck the Intel situation multipies these kind of problems. The advantage is that I *do* know some x86 assembly, while PPC always was a black hole of a myriad of addressing modes and load/store interleaves.

Boy, I am glad my programs are Java. And Rexx.

Keynote Poster

The projectmanager left and I wanted to produce a poster and did not have a lot of time. It had to at least contain the screens of our webapp and some photographs of the team, I also wanted to include most of the graphics we produced for the presentations of the product in the past year. Now I knew that doing this in Photoshop like you are supposed to would have cost me a certain amount of time, that I, also due to the deadline which was connected to the managers departure, did not have. So I tried it in Keynote, a presentation package. I figured that if I could make a slide of a high enough resolution, and then export this to PDF, the printer could plot a sharp enough poster out of it.

The big advantage here was that I could just drag and drop the material, line it up using the automated guides, crop the photographs and send relevant pieces to foreground and background, and be ready in a nick of time, compared to all the layering work that PhotoShop requires for this (combined with my relative inability to use that program well).

So I defined a 4000*4000 slide with a white background, dropped in the screenshots in the pure uncompressed tiff they were made of, dropped and cropped the photographs and dragged and dropped the pdf vector graphics from the other presentations. The titles I did with very large Zapfino and Hoefler text (200 to 300 picas), and put in some backgrounds unsing the standard geometrical figures.

The machine became a bit unresponsive when I finished up the work, and my impression is that when I started adding graphics with alpha channel there was more work to do for the machine. I exported the PDF to a standard X3 format and went to the printer, who luckily is situated just around the corner. After some initial anxiety when the Sony Vaio machine she ran PhotoShop on took several minutes to load the rather large pdf file, we printed a poster of 1 meter by 1 meter and it came out lovely and sharp, and the lady even remarked that she did not yet see a font that came out this sharp on the Epson plotter that was used. So hey presto, I know how to do it next time.