At a high level, what I need to do is have a python script that does a few things based on the commands it receives from various applications. At this stage, it's not clear what the application may be. It could be another python program, a MATLAB application, or a LAMP configuration. The commands will be sent rarely, something like a few times every hour.
The problem is - What is the best way for my python script to receive these commands, and indicate to these applications that it has received them?
Right now, what I'm trying to do is have a simple .txt file. The application(s) will write commands to the file. The python script will read it, do its thing, and remove the command from the file.
I didn't like this approach for 2 reasons-
1) What happens if the file is being written/read by python and a new command is sent by an application?
2) This is a complicated approach which does not lead to anything robust and significant.
Python has since early stages a very comfortable PyZMQ binding for ZeroMQ.
MATLAB can have the same, a direct ZeroMQ at work for your many-to-many communications.
Let me move in a bit broader view, from a few, but KEY PRINCIPAL POINTS, that are not so common in other software-engineering "products" & "stacks" we meet today around us:
[1] ZeroMQ is first of all a very powerful concept rather than a code or a DIY kit
[2] ZeroMQ's biggest plus for any professional grade project sits in rather using the genuine Scaleable Formal Communication Patterns end-to-end, not in the ability to code pieces or to "trick/mod" the published internals
[3] ZeroMQ team has done a terrific job and saves users from re-inventing wheels ("inside") and allows to rather stay on the most productive side by a re-use of the heroic knowledge ( elaborated, polished & tested by the ZeroMQ gurus, supporters & team-members ) from behind the ZMQ-abstraction-horizon.
Having said these few principles, my recommendation would be to spend some time on the concepts in a published book from Peter Hintjens on ZeroMQ ( also available in PDF). This a worthwhile place to start from, to get the bigger picture.
Then, there it would be a question of literally a few SLOC-s to make these world most powerful ( and believe me, that this sounds bold only on first sight, as there are not many real alternatives to compare ZeroMQ with ... well, ZeroMQ co-architect Martin Sustrik's [nanomsg] is that case, to mention at least one, if you need to go even higher in speed / lower in latency, but the above key principal points hold & remain the same even there ... )
Having used a ZeroMQ orchestrated Python & MQL4 & AI/ML system in FOREX high speed trading infrastructure environment is just a small example, where microseconds matter and nanosecond make a difference in the queue ...
Presented in a hope that your interest in ZeroMQ library will only grow & that you will benefit as much as many other uses of this brilliant piece of art have gained giant leap & benefited from whatever the PUB/SUB, PAIR/PAIR, REQ/REP formal patterns does best match the very communication need of your MATLAB / Python / * heterogeneous multi-party / multi-host Project.
How do I communicate and share data between python and other applications?