Pathpilot Networking, MQTT, Node Red, NAS, New Tormach Camera

[Adam_White]

Okay here goes a long multiple question topic. I can be more specific if needed on each topic if needed.

I have a rather complicated network and automation setup at my facility and am looking at adding 1-3 robots to the mix. I have a 440, 1100, 15l, and a Haas VF2 that are all networked. I post all of my code to a NAS unit since I want all of my files to be backed up both onsite and offsite. This provides a nice central backup solution to deal with all machines.

My NAS also runs virtual machines that are hosting node red, mqtt and homeassistant which provide me a nice local method of automating things. Again, a central solution for doing snapshots and backups.

For example, many times before leaving for the day I will start long running parts. Cycle times that are greater than 10 hrs. My node red server will send a query to the Haas that asks for a serial number and it will wait for a response. If the machine responds the automations are delayed. Once the haas does not respond after 20 mins the shop will dial the hvac back, shut the air compressor off, shut the auto drain off and arm the interior motion sensors in the shop areas. This is done with a very simple text string.

Does Pathpilot respond to certain commands via ethernet that ask for a status or serial number? How about the robot integration? Is there a specific list of commands and expected responses documented anywhere? I would like to trigger certain things based on statuses on the machines.

Have there been any changes to the file sharing options in Pathpilot? Specifically having the ability to grab a file from the NAS vs pushing the file to the Tormach. I realize you can do an NFS share but I also dont want to change anything in Pathpilot or run into any issues during an update, deleting the share or losing the drive upon reboot etc.

It seems Dropbox isn’t working anymore, and honestly I would rather not add any extra stuff to a machine controller. I would rather do that via my NAS and a mounted drive.

Lastly, I am liking the camera option. I really like the fact it saves video for estops and such. Is there an rtsp/rtmp method of viewing the video stream even if it is running behind? Is there an option for live view or something I can plug into my camera server?

[Davie]

I can answer a few of your questions regarding Pathpilot on the mill.

• we haven’t changed anything related to file sharing
• there is no api or comment built into PathPilot. However, you can, likely, use the USB I/O module to tell when the machine is done for the day. You can even stack 4 up on one machine to have 16 digital inputs and 16 outputs
• there are no provisions for remotely viewing the video feed.

[Rogge]

@Adam_White thanks for posting and welcome to the Tormach forums! I’m impressed by your set up and the extent to which you have automated your shop.

I’ve got limited answers for you but I know a few people who can add detail.

“Does Pathpilot respond to certain commands via ethernet that ask for a status or serial number? How about the robot integration? Is there a specific list of commands and expected responses documented anywhere? I would like to trigger certain things based on statuses on the machines.”

PathPilot Robot and PathPilot CNC run on different codebases. They connect via a service called MachineTalk. Currently that connection allows status queries, MDI commands, cycle start commands to be send from the robot, with the sort of responses you would expect from the CNC. I think we use 0MQ for the communication and Python to glue it all together. I’ll check with the developers and see how hard it would be to come up with an example that works with home assistant.
The PathPilot robot software is more flexible. Because the robot interprets Python programs as its ‘robot programming language’ you could handle status queries and/or regular updates right from the robot’s program itself. Under the hood the robot runs on ROS (Robot Operating System), and it looks like there are a few home assistant projects that bring ROS into a HA component.

File sharing options for PathPilot - https://www.youtube.com/watch?v=FWzs9LE3jZ4
If that’s not what you’re looking for let me know.

The camera was intended to do estop recordings and make self-annotating programs (via M01 image break or video) easier. Unfortunately it’s a bit more overhead to stream via the controller. If you want to pick up your phone and monitor what’s going on inside a machine I recommend those Reolink cameras. They are POE, have IR lighting, and their app is super easy to use. $150 for a pair. We use them in the office for remote workers who want access to hardware to do software development. They are coolant proof too.

Please give me a couple of days to dig up more detail. Thanks!

[Adam_White]

Perfect! I will see if I can find some good info on Machine Talk. I am pretty much looking to just query a status. For example, Haas uses MTconnect for their machine data collection. So I send a ?Q100 and will get a response back of the serial string. From that I can tell if it is alive. Things like parts counter are also available thru the qcodes. I like the idea of using node red for this since I can create an entity to use in home assistant.

Reolink cameras work great, I have about 30 of them and use Blue Iris for the server. Blue Iris works well since you can also create an event based on XYZ and send that to homeassistant or the mqtt broker.

I will have to double check but I am 99% machine data collect runs on port 5051 and even using putty to send the ?qcode works for testing the connection.

For file sharing, I would be looking to do the exact opposite of what the video shows. I would like to map a network drive from Pathpilot vs mapping the drive of Pathpilot. So setting the location of where the gcode could be found in a central NAS.

For example with the Haas stuff you can map a drive on the NAS. All files are stored there so it is a central repository with gcode, setup sheets, pictures, etc. One place to backup as well. Its just easier when you have 4 machines and looking at a few robots.

This new camera Tormach has would be quite useful especially if you could map a NAS. Those files would be synced so if estop was activated or pics, movies etc. All of the traffic especially remote would be on the nas and not the machine control.

I hope that makes sense?

[Reuben_James]

@Adam_White

For file sharing, I would be looking to do the exact opposite of what the video shows. I would like to map a network drive from Pathpilot vs mapping the drive of Pathpilot. So setting the location of where the gcode could be found in a central NAS.

I did this on my Pathpilot. I was never able to get dropbox working correctly so I dug around and found some instructions on how to mount a network share. Now I post all my code to my NAS which then shows up as a new folder inside the Gcode folder on pathpilot. I modified the fstab file in linux to mount the NAS share as a new folder inside Gcode. The attached .PDF explains the process I followed (I did not write these), however I did discover some discrepancy and the final line I had to my fstab file was slightly different than what are in these instructions. I can find that info when I’m back in the shop and add it to this string. To be safe, I’d make sure you back everything up on your pathpilot before attempting.

It’s very nice to be able to post a code from anywhere I have internet access to the NAS which then shows up when I boot pathpilot instead of having to turn the mill on before moving code over.

How to read a network share on pathpilot.pdf (21.6 KB)

[Adam_White]

Thank you! I am familiar with the fstab entries I just get concerned about a reboot or updates etc. Hard to say what some updates modify or what drive an update could potentially seek during a process. The permissions could have some downsides as well.

I think this should be something in Pathpilot. Just a simple remote mount that needs an ip, user and pass. It could be a share from windows, linux, or some combination from a nas.

The other benefit aside from backups is a large storage repository that is not consuming so much space on the control. With the camera option now, making a setup video with audio and not storing it on the local machine is a good thing. I like the fact I can rewind back to a previous setup and see what I did three years ago in form of a sheet, pictures and video. My nas has 100TB and with a raid 6 array onsite and offsite backups are easy to setup and mindless. I can review video before the day of setup and just re jog my memory.

While dropbox is good for some things they do have a tendency to change space allowance and of course payment for it. I don’t think its a great solution for some things.

[Reuben_James]

Sure thing. I’ve done 2 updates to PathPilot since modifying the fstab entries and my network mount has persisted through both. That doesn’t mean it won’t continue forever but so far, so good. I’ll keep my notes handy if a future update wipes them out.

I agree, having things in one central location is a great way to do it. I also have my NAS sync that same folder with my OneDrive account so that I can post files to it anywhere in the world and OneDrive and NAS do their thing so that next time I turn the mill on, the files are readily available.

[Adam_White]

Is Pathpilot using v2.9 of Linuxcnc? I guess I am not certain of how, or what Tormach changes?

I could do what I need to on the automation end of things using MQTT. Looks like v2.9 may have it somewhere?

https://linuxcnc.org/docs/stable/html/man/man1/mqtt-publisher.1.html#AUTHOR

I am not really looking for an ultra fast way or trying to anything real time with motion. More or less trying to get the environment the way I would want it for use with a tending setup.

I suppose I could use the usb IO setup and run that into a plc with mqtt, but that just seems kludgy.

[Paul_Morley]

@Adam_White

My NAS also runs virtual machines that are hosting node red, mqtt and homeassistant which provide me a nice local method of automating things.

What a fantastic post. I have to admit I never would have expected to hear Node Red, MQTT, Homeassistant and a CNC Mill all in the same paragraph but it’s awesome! Looking forward to following this thread more to see if you get it working

[machinekoder]

@Adam_White

Perfect! I will see if I can find some good info on Machine Talk.

If you want to use the robot integration, you could use pymachinetalk to get the data from the running instance and then publish it to MQTT using paho-mqtt for example. The ncurses example should contain the relevant info for connection, note that unicast service discovery (requires the IP address of the machine) is much more reliable than the default multicast discovery used in the example. pymachinetalk/examples/ncurses/ncurses.py at master · machinekit/pymachinetalk · GitHub

Aside from that, we would be curious to learn about your intended use case for the PathPilot remote API. Maybe the MQTT support is something that could be offered by PathPilot out of the box.

I’m also using MQTT for quite a lot of projects in my home and MQTT is becoming more popular for IIoT as well. For example I’ve created a MQTT bridge for my Prusa 3d printer which I use to notify me about completed prints and to automatically turn on the enclosure fan when a print is running. I definitely can see similar scenarios for Tormach machines in a workshop. GitHub - machinekoder/prusalink_mqtt_bridge: This utility facilitates the bridging of PrusaLink data to an MQTT broker, enabling remote monitoring and integration with home automation systems.

That’s how my automation setup looks like anyone is curious. I’m using a Loxone home automation system. As you can see, the data from MQTT is used to control a few shellies and variables in the “Loxone App” like showing the webcam stream as doorbell when the print is finished.

[Charles_McCoy]

Awesome topic. I also have interest in machine status either via MTConnect, MQTT, or other.

[Adam_White]

I think having an MQTT option would be great in Pathpilot. The thing I struggled with as I was building my business was trying to get away from the machine. After proving out programs and knowing there would be a long run time ahead its nice to go do something else, or even an overnight run, maybe starting it before heading out to work for the day. I use shellies as well in some of my integrations. Nice thing is having home assistant integrate with all of these. If using the mql setup “turn on the exhaust fan” or if the machine is on “turn on the air compressor” when a part is complete “flash my phillips hue bulbs green” if estop is hit by someone else “flash the bulbs red”.

I actually have our sleep number bed tied into this as well. Some parts I run are 12 hrs. If the machine errors out from tool break detection my head position will adjust if its between 1am and 230am, otherwise let me sleep…etc.

My thought is to have certain mcodes that will publish a MQTT message and from there you can do whatever setup you want.

I have seen many posts from people asking about notifications, emails and such. If MQTT was in the system that would give someone the ability to do just about anything for messaging etc.

[rsteckman]

PathPilot with MQTT Log Watching and Status

Introduction

This is a sample of how a user could extend their PathPilot Controllers and Tormach machines to integrate with a wider set of automations that can integrate with MQTT.

This sample is provided as-is with no warranty or support. It may cease to function with future releases of PathPilot.

Requirements

1.) PathPilot Controller
I’m using PathPilot 2.10.1 and I have not bothered to test on earlier versions. The requirement is Python3 (and the version of Python3 on the controller I’m using is Python3.4). Additionally, since I’m parsing the logs as they are written, the message formatting/syntax is taken from PathPilot 2.10.1.

2.) A MQTT Broker
This could be something like Home Assistant, or you could run something like HiveMQ on another machine. You could even extend this sample to include a MQTT broker if you really wanted, but all of those are outside the scope of this sample, this sample just assumes there is a broker available.

How it Works

The python program simple watches to changes written to the pathpilotlog.txt file, and if it sees a log entry it feels is important, it publishes to the related topic it has. Specifically, it’s watching for changes to INTERP_STATE, STATUS.TASK_MODE and STATUS.TASK_STATE. A client could then determine if the machine was running or similar by watching for the changes to those statuses.

Install & Setup

All steps can be done from a terminal on your PathPilot controller.

1.) Install pip for Python3.4 (while Python3.4 should be on your controller, pip is usually not) to install the python package we will use for the mqtt client.

curl -o get-pip.py https://bootstrap.pypa.io/pip/3.4/get-pip.py
sudo python3 get-pip.py

2.) Use pip to install the paho-mqtt package this script uses.

pip3 install --user paho-mqtt

3.) Download the mqtt_sample.py file to your controller.

curl -o mqtt_sample.py https://files.torma.ch/code_samples/mqtt/mqtt_sample.py

4.) Edit the mqtt_sample.py file to fit your environment. You can use nano or gedit both of which should already be installed on your controller.

nano mqtt_sample.py
gedit mqtt_sample.py

It’s REQUIRED you must set the MQTT_BROKER and MQTT_BROKER_PORT variables at the top of the script, since those are needed to know where it’s publishing to. Feel free to further modify the script as desired for other log messages you are interested in bubbling up to your broker.

Running

Run the script from the terminal (leave terminal open, Ctrl-C to stop). Further enhancements could move this to a systemd like service on startup, but that is well outside the scope of this sample.

python3 mqtt_sample.py

Viewing

While brokers and integrations are outside the scope of this sample, you can see everything running by opening a sample client at https://files.torma.ch/code_samples/mqtt/mqtt_client.html (ideally on a different device than your PathPilot controller, but on same network)

Entering in your broker’s address and port information and click the Connect button, then on the PathPilot Controller start running a program and watch the changes in the client browser.

IMPORTANT NOTE MQTT over WebSockets is a different port than the normal MQTT protocol, and not every broker supports both TCP and WebSockets, so the port you enter in this script should not be the same as you edited in the python.

Video

Again, this was a quick and dirty proof of concept, not designed as a fully featured and functional notification solution, but something that, if useful, could be expanded by the community to use as you see fit.

[Adam_White]

Thank you Ryan! I have this working great in Home assistant. I will work on changing some security settings for Auth and such, but since I am running this locally and over vpn it wasnt a huge deal for my use.

Is there a specific area in that log that would show what machine it is? Some sort of identifier like 1100 or 440 or a serial number of the machine? I can play around with that script, just thought I would ask.

Once I make a few changes I will make a video showing how to setup a broker and Home assistant. I used a docker container with Hive running locally.

[Davie]

I did a quick scan through a few logs and found the line below but I’m not sure how often it is repeated.

“model”: “770M”

[rsteckman]

Yeah, it is written in the logs, but not too frequently (like at startup) so you’d end up having to ensure the python script was started before PathPilot (example, using systemd to start it).

Honestly, if it were me, I’d just define it in the config section, and make it slightly different per machine in my shop I was running this on. Set a machine constant, or customize the topic strings for each machine, eg:
change: MQTT_TOPIC_INTERP = 'pathpilot/interp_state'
to: MQTT_TOPIC_INTERP = '770M/pathpilot/interp_state'
and repeat for the topics you are publishing.