I’m now proceeding with the first tests of the new Conductor 2 “live” at the museum using JMRI and the actual trains, and the first two tests have been… interesting. In the pure line of “no plan survives first contact with the enemy”, both tests were highly “educational” and produced a list of issues to address.

Since I’m doing quotes, let’s also quote Eisenhower’s “plans are worthless, but planning is everything”. The planning I did recently to prepare for the deployment really paid off and I was able to run the tests easily -- that is I can trivially change accounts on the automation computer and switch between the stable current Conductor 1 setup and the experimental Conductor 2 setup. Then when the computer boots every morning, I’m guaranteed it will safely run the stable version automatically. That’s where that bit of planning paid off.

A bunch of unexpected things happened during the experimental Conductor 2 tests, and I collected enough data to know  what to do next. I would have been highly suspicious if the new automation engine had worked perfectly out of the box. It didn’t, and more importantly I got some valuable data to work on it.

One thing I still don’t like in the Conductor 2 DSL is the name of the “ActiveRoute” object.

It’s misleading and confusing.

As a reminder, in the current DSL:

• An “active route” is a list of routes, of which only one is active at a single time.
• A route is a sequence of nodes, each node corresponding to one block in a given direction.

It’s fairly misleading as one may think that an “active route” is one route that has become active, when it is actually a container of routes combined with a switch selector.

Possible name alternatives:

Orion, Allen, and I had a discussion on how to refresh the buildings’ lighting on the layout.

For the record, the current lighting uses the traditional 12V light bulb system, controlled by a central massive power supply under the main yard, with rows of toggle switches mostly unused. Wiring is the usual mess (a.k.a. both neat yet over-complicated and hard to trace). We don’t know which buildings are equipped with lights, nor which light bulbs are either not working or not connected.

I’ve made an attempt identify the power supply and what few buildings are visibly lighted here:
https://www.alfray.com/trains/randall/about.html#h.p043jf65vl34

One thought is that next time we dust the city, we should inventory which buildings are equipped with wires, and map existing through-holes in the support boards.

But going forward, how should we improve the lighting?

Let’s discuss Conductor 2 strategies for deployment.

One issue is that I can’t just install this on the main automation computer at the museum and just “hope it works the first time”. Even though it’s volunteer work, I take it seriously and I have an unofficial SLA of 7/5 -- my automation needs to work reliably for 7 hours for 5 days a week. That makes my downtime only two days (Sunday and Monday), one being a workday for me, so really I have one day to get stuff done, and it’s on the week-ends when I often have other commitments. Then I wonder why any project takes so long to accomplish… oh well.

So the solution is that I have a clone of the automation computer at Randall. It’s the “backup” computer in case the main one fails. It’s a 99.9% exact setup so it’s nice for testing software upgrades. And because I can’t often easily go on place, I also have another fairly similar machine at home for testing. Thus I deploy on the machine at home, sort of a canary, and if I like it, I deploy on the backup machine at the museum, and finally on the main one. It’s a bit of a process but it works neatly. I can identify issues early, as is the case here.

My early attempt at deployment failed, and I addressed it already.

The Software Defined Blocks project uses an ESP32 with sensors to emulate block activations for a train model railroad.

Wifi offers the typical bootstrap dilemma on all these tiny gizmos: we need to select an SSID and enter some wifi credentials to connect to an existing network.

The typical “Arduino way” to deal with this problem is to encode both the SSID and the password in the Arduino code. That forces users to recompile the Arduino sketch before deploying.

On devices with an sdcard, a work-around is to have users create a small config file with the info that we can read at boot.

Here instead I want to have the traditional “wifi-enabled gizmo” behavior.

An ESP32 running SDB shall offer this behavior:

The Software Defined Blocks project uses an ESP32 with sensors to emulate block activations for a train model railroad.

How can the current modules benefit from the FreeRTOS tasks?

An initial prototype was built using MicroPython. There’s good support for ESP32, especially using the Adafruit libraries. Unfortunately, MicroPython on ESP32 still has quite some limitations. For example there’s no OpenCV support, etc. Since the goal is to make the project as accessible as possible, sticking to the regular Arduino / C++ seemed a better option.

One issue with the MicroPython version is that I realized that “MicroPython Threads” are implemented using FreeRTOS tasks all of the same priority. Thus they execute in round robin fashion, with no control from the application. That means a task doing some IO can be pre-empted in the middle of e.g. an I2C-in-software operation.

With better FreeRTOS control, we can use tasks and we can:

• Use priorities to ensure some tasks are run before others.
• Use critical sections or mutexes to avoid same-priority tasks from interrupting each other.

For now I’ve been doing block detection on the Randall layout using NCE BD20 current detectors. They work well. There are others DCC-compatible detectors but they all use through-hole core transformers, which means they cannot be used with existing wiring without desoldering or disconnecting something.

There are other contenders to the BD20, such as the cpOD and the CKT-BD1. I plan to switch to these once I exhaust my current supply of BD20. But the bottom line is that they all use these “through-hole CT”, and sometimes I wished I had the convenience of a split core transformer that I could quickly snap around a track feeder wire.

I had two goals in mind:

• Main goal: Have an alternative to the through-hole CT block detection sensors that do not require disconnecting nor cutting wires to install them.
• Secondary goal: Make a little portable “current detector” that I could quickly snap around a track feed wire to trace wire continuity.

Thus a while ago I got two sensors, with the purpose of figuring out to use them to monitor train track current with an ESP32:

• https://amzn.to/3q7qG4z Hall-sensing ACS712-5A current module.
• https://amzn.to/3AJM5FP Split core SCT013-005 5A 1V current transformer.

Spoiler alert: I discussed that topic on the MRH forum and the Arduini forum and I was clearly told that would not work, and to make a long story short, I never managed to make them work for this application.

Upfront I want to indicate why that was anyway not a surprise and I was likely to fail:

The Software Defined Blocks project uses an ESP32 with sensors to emulate block activations for a train model railroad.

The base idea is to use MQTT to publish block changes, instead of targeting the JMRI JSON protocol. I was originally going to support both, but that seems redundant. We’ll see. JSON is simple and very convenient for debugging, so it’s always a great idea to get started.

Here we’ll detail how that works for this specific project:

• ESP32 monitors sensors, sends messages to an MQTT broker.
• JMRI MQTT subscribes to messages from the MQTT broker.
• Requirement: An MQTT broker running on the network (either the same machine as JMRI, or another local one, or a public one).
• Requirement: Set up the ESP32 with the broker IP, and block/sensor topics.

For background, my RTAC Conductor automation software embeds “KV Server”, a distributed key-value server/client architecture which is fundamentally very similar to MQTT, except mine is nowhere as ubiquitous and it is buried somewhere in my LibUtils project. In the case of the Conductor automation software, I use the KV Server as a way to distribute configuration from the automation script to the accessories. Which means we can do the same here with MQTT, and we can take advantage of that to configure the ESP32 with block triggers defined in the automation script.

For now I’ve been doing block detection on the Randall layout using NCE BD20 current detectors. I have part of the Mountain Panel wired, and I’d like to have more there. I’d like to also have block detection of part of the Valley, and that’s where I have been hitting a wall for the last two years. I just can’t figure out a clean way to do it.

Block detection using BD20-style current detectors can be done at various points. Let’s review the possibilities:

The Software Defined Blocks project uses an ESP32 with sensors to emulate block activations for a train model railroad.

The desire for the MVP is to be designed around “modules”.

Modules

Each module should have:

• An init method called when the main starts.
• A start method called once all other modules have been initialized.
• A loop method called as part of the main loop.
• The loop method will return the max time it wants to sleep before being called again. The loop method may be invoked sooner than requested, but not later.
• Its own thread-safe message queue.

Each module is free to start a thread or do their process in the main loop.