Let’s discuss Conductor 2 strategies for deployment.

One issue is that I can’t just install this on the main automation computer at the museum and just “hope it works the first time”. Even though it’s volunteer work, I take it seriously and I have an unofficial SLA of 7/5 -- my automation needs to work reliably for 7 hours for 5 days a week. That makes my downtime only two days (Sunday and Monday), one being a workday for me, so really I have one day to get stuff done, and it’s on the week-ends when I often have other commitments. Then I wonder why any project takes so long to accomplish… oh well.

So the solution is that I have a clone of the automation computer at Randall. It’s the “backup” computer in case the main one fails. It’s a 99.9% exact setup so it’s nice for testing software upgrades. And because I can’t often easily go on place, I also have another fairly similar machine at home for testing. Thus I deploy on the machine at home, sort of a canary, and if I like it, I deploy on the backup machine at the museum, and finally on the main one. It’s a bit of a process but it works neatly. I can identify issues early, as is the case here.

My early attempt at deployment failed, and I addressed it already.

First let’s look at the facts & specs:

• Conductor 1:
• Built using Java 8 w/ Gradle.
• Targets JMRI linux 4.x with a conductor.py wrapper.
• Runs on Debian Buster 10.13 + dev on Windows 8.
• Conductor 2
• Built using Java 11 or Java 8 + Kotlin w/ Gradle.
• Targets JMRI linux 5.x with a conductor.py wrapper.
• Runs on Debian Buster 10.13 + dev on Windows 8/10/11.

The other important part is how Conductor integrates with JMRI:

• JMRI loads bin/JMRI/jython/Conductor.py when loading the profile.
• Jython loads bin/JMRI/lib/conductor.jar when resolving “import com.alfray.conductor”.
• Both files are symlinks to the git checkout folder + build folder.

Conductor 1 should work just fine with the Java 11 JRE. JMRI now requires Java 11 minimum.

My deployment “concern” is that I want to try the new Conductor 2, while still being able to revert quickly to Conductor 1 if I realize there’s a blocking issue I have to address. They both use different kinds of scripts with different languages, although the feature set is relatively similar. Thus I have designed my Conductor 2 binary to be “backwards compatible” with Conductor 1 -- in the sense that the binary for v2 integrates both engines v1 and v2.

Actually, we must separate the “entry point” versus the “engine”:

• Entry Point 1 only supports Engine v1 with the ANTLR custom grammar text scripts.
• Entry Point 2 supports both Engine v1 and Engine v2. It will load the proper engine based on either the extension (kts, groovy) or the content (txt vs kts).

The “fat JAR” contains both entry points too. I build that fat JAR, and that’s what JMRI / Jython loads.

The Jythong wrapper, “Conductor.py”, now has a variable in the Jython code to force it to call the Entry Point 1 or 2. That can also be set via an environment variable. Thus the deployment scenario is:

• Load JMRI with the new Jython wrapper and the new Fat JAR supporting both engines.
• Start with forcing Conductor 1… Validate this works with v1 scripts only.
• Then restart while forcing the Conductor 2… Validate this works with both v1 or v2 scripts.

When I tried that, things initially went pretty bad, with multiple Java/Jython exceptions stacking up.

The first issue is that I was loading scripts from the JAR resources using the URL resource loader, and converting these to Kotlin Scripting UrlScriptSource. While that’s nice for unit testing, it’s not convenient to load scripts from JMRI as it means I’d have to rebuild the fat JAR every time I change the script, which defeats the purpose of a scripting engine. I quickly figured out Kotlin Scripting as a FileScriptSource and changed the Conductor entry point wrapper to support both.

The second issue I kept getting was multiple crypting exceptions when loading the script. Eventually I figured the root cause was this error: “kotlin.script.experimental.jvm.util.ClasspathExtractionException: Unable to get script compilation classpath from context, please specify explicit classpath via "kotlin.script.classpath" property”. This is described in this Kotlin issue KT-21443 and has to do with the classpath not being properly set up when loading a script from a fat JAR, which is exactly my case. The solution (see commit 2519949) is to get the get the JAR path and add it to kotlin.script.classpath, along with any JAR Manifest Class-Path -- of which there are none in my case, but I still implemented it in case I change that later.

Now with these changes done, I can deploy Conductor 2 as specified above. First with a complete possible fallback to Conductor 1, and then I can easily switch to the new scripting engine. For now I’ve only tested the engine against the simulator, so it’s going to be interesting when I switch to real sensor data.

---