Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Dr. Slack, Fri Jul 06 2018, 02:30PM... with some vague interest in hardware.
I play around with software, reasonably competent with python and microcontroller C, but have never really reached critical mass on things like linux, git, sockets, I2C. While I can search around for how to do any one thing, it's all a bit of a struggle. I'm aware that trying to invent too many things at once is likely to founder. So I regard my idea for a wifi enabled meter a bit too big for me to do on my own.
The general idea is this. Control and wifi done by something like Onion Omega 2+ (other micros are available). I've got options for the front end more or less sketched out in my mind. Two channels of voltage from 100mV to 1000V, three current channels (electronic switched pA to 1A or 10A, socket selected 1A to 10s or 100s of A, and a low accuracy log reading nA to A for wide instantaneous dynamic range, 2T/4T resistance, temperature, and maybe a few relay outputs for controlling battery test loads. What I need is a web interface that can be operated from a browser, and an API that a program can talk to.
This is just for fun. I'm not anticipating trying to sell it. That would involve timescales, specifications, funding, responding to competition. I'm retired, I don't need all that.
Anybody want to play?
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Sulaiman, Fri Jul 06 2018, 07:02PM
Assuming that you are more into the software than the hardware;
You are aware of cheap usb or RS232 interface digital multimeters via eBay ?
I lost my £9.99 Maplin dmm w/RS232 but it was really useful for stuff like battery charge/discharge etc.
USB oscilloscopes with software are also available for faster stuff,
USB VNA and SDR cover the upper frequencies.
That would leave you just with the WiFi:USB to make/buy.
Sulaiman, Fri Jul 06 2018, 07:02PM
Assuming that you are more into the software than the hardware;
You are aware of cheap usb or RS232 interface digital multimeters via eBay ?
I lost my £9.99 Maplin dmm w/RS232 but it was really useful for stuff like battery charge/discharge etc.
USB oscilloscopes with software are also available for faster stuff,
USB VNA and SDR cover the upper frequencies.
That would leave you just with the WiFi:USB to make/buy.
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Dr. Slack, Sat Jul 07 2018, 04:59AM
Perhaps I'm mistaken, but I thought I'd spelled out that my software was smattering, and I was more au fait with the hardware end?
Yeah, got one of those. I also use an Arduino as a crude USB connected multichannel voltmeter for battery charge/discharge monitoring, it's how I got the 'AA shoot-out' results
of a while ago. My 4 channel Rigol will go fairly slow for monitoring tasks as well.
Am I correct that you're saying that you're not the man for the job?
Dr. Slack, Sat Jul 07 2018, 04:59AM
Sulaiman wrote ...
Assuming that you are more into the software than the hardware;
Assuming that you are more into the software than the hardware;
Perhaps I'm mistaken, but I thought I'd spelled out that my software was smattering, and I was more au fait with the hardware end?
wrote ...
You are aware of cheap usb or RS232 interface digital multimeters via eBay ?
You are aware of cheap usb or RS232 interface digital multimeters via eBay ?
Yeah, got one of those. I also use an Arduino as a crude USB connected multichannel voltmeter for battery charge/discharge monitoring, it's how I got the 'AA shoot-out' results
of a while ago. My 4 channel Rigol will go fairly slow for monitoring tasks as well.Am I correct that you're saying that you're not the man for the job?
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Shrad, Mon Jul 09 2018, 06:48AM
I'd use a ESP8266 or ESP32 as it covers the complete range of your requirements, plus it can host the web interface without any hassle... and it is covered by arduino IDE and libraries for what I know
Shrad, Mon Jul 09 2018, 06:48AM
I'd use a ESP8266 or ESP32 as it covers the complete range of your requirements, plus it can host the web interface without any hassle... and it is covered by arduino IDE and libraries for what I know
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Sulaiman, Mon Jul 09 2018, 08:34AM
Based on my ability to appreciate the question,
I gues the answer has to be ... YES
You have the wrong guy :(
Sulaiman, Mon Jul 09 2018, 08:34AM
Dr. Slack wrote ...
Am I correct that you're saying that you're not the man for the job ?
Am I correct that you're saying that you're not the man for the job ?
Based on my ability to appreciate the question,
I gues the answer has to be ... YES
You have the wrong guy :(
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Plasma, Sun Jul 15 2018, 11:28PM
I might be able to help, couple of questions
do you want a Web server thing to accept Get and Post requests or just socket sending hex across the wire, do you need dns and tcp/ip stack made or is Linux taken care of that.
Got a couple of projects on but one computer so don't want to set up Linux, using Windows, so any code want be tested.
Can you post the code you're done, would be easier going from there
Plasma, Sun Jul 15 2018, 11:28PM
I might be able to help, couple of questions
do you want a Web server thing to accept Get and Post requests or just socket sending hex across the wire, do you need dns and tcp/ip stack made or is Linux taken care of that.
Got a couple of projects on but one computer so don't want to set up Linux, using Windows, so any code want be tested.
Can you post the code you're done, would be easier going from there
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Dr. Slack, Mon Jul 16 2018, 06:18AM
Get and Post. I've often seen those referred to as I google around. I don't know the difference, and what using either would do for me. What should I want?
As far as I can tell, you ask for UDP or TCP at socket setup time. Though my minimalist nature leans me towards UDP, TCP would probably be better, and closer in spirit to what serial I/O does. I do want something that doesn't hang, or is easy to detect and reset hanging behaviour, should ends get unsynced due to a disturbance on the LAN, program bug, or whatever.
The device should sit on my local wifi network, so having an address of 192.168.1.x, accessible to everybody on my LAN, just like my printer or ADSL, and not visible to the world the WAN side of my ADSL. That suggests I don't need dns.
I want two modes of behaviour. Whether that needs two interfaces writing for the device I don't know, it may be easy enough to handle both behaviours through just one. It's all low speed, so there's a lot of scope for doing things inefficiently.
Mode 1
When I set up my ADSL, it has a web page I visit, that I can fill in and read values. I'd like to be able to do something like that, using a browser as a very manual method of driving, for initial setup, debugging etc, zero programming at the client end. I tell it to change the range of a channel, read a voltage, no more. The web page would essentially be a soft front panel into the device hardware. Once the web magic gets as far as running a program, preferably python but C would do, that presents one button and one entry box on a web page, I know where I am and can do the rest.
Mode 2
When I plug an Arduino into my USB, I can program it to work over serial I/O, and write a python program on my PC to talk to it, that works fine. I understand essentially the same thing can be done over sockets to a device on the LAN. That's the mode I'd want to use for running it automatically from a custom python program. I'd write my battery charger/tester or whatever else in python, and use the device as a fairly dumb peripheral, just as I've written a battery tester to use hardware hanging off my Arduino board.
If it's possible to fake this mode from the first, and I suspect it is, by using the host PC python program to construct web requests, then maybe I only need mode 1 on the device. However, I suspect mode 2 is an order of magnitude easier to implement, at least on the device.
I've looked at python socket tutorials and been left confused. I've read around peoples' experiences putting such things together, and found things like 'it will work on loopback but won't work over my LAN', which have inhibited me from mustering enough effort to put something together and try. I have not yet reached sufficient a critical mass of knowledge and warm fuzzy feeling to start along the path of trying things out. I do not yet know what a good tutorial looks like, a pointer at a good one might be sufficient. I always have this difficulty starting with new things, don't understand until I've got my hands dirty, can't bring myself to get my hands dirty until I've got some basic level of understanding of what I should canonically be doing.
The path would probably be something like this.
1) Write a simple server that can be set, read, and generate new information, like a clock, running on the same machine as a client to talk to it over loopback, both ends in python. Implement both modes.
2) Move the server to a different machine on the LAN, initially a laptop and then an RPi over RJ45 for instance.
3) Get an Onion Omega or similar, and repeat over wifi.
Software side 'hello world' achieved, I should be able to take it from there.
I only have linux Mint on my machines, but I can ask my son, who has python on his win10 laptop, to try odd things out from time to time if necessary. I do use wine for things like LTSpice, but don't want to introduce yet another potentially buggy interface into a process that I'm already struggling with. I had rather hoped that working over LAN would be fairly platform independent, with python to take care of any differences.
Dr. Slack, Mon Jul 16 2018, 06:18AM
Get and Post. I've often seen those referred to as I google around. I don't know the difference, and what using either would do for me. What should I want?
As far as I can tell, you ask for UDP or TCP at socket setup time. Though my minimalist nature leans me towards UDP, TCP would probably be better, and closer in spirit to what serial I/O does. I do want something that doesn't hang, or is easy to detect and reset hanging behaviour, should ends get unsynced due to a disturbance on the LAN, program bug, or whatever.
The device should sit on my local wifi network, so having an address of 192.168.1.x, accessible to everybody on my LAN, just like my printer or ADSL, and not visible to the world the WAN side of my ADSL. That suggests I don't need dns.
I want two modes of behaviour. Whether that needs two interfaces writing for the device I don't know, it may be easy enough to handle both behaviours through just one. It's all low speed, so there's a lot of scope for doing things inefficiently.
Mode 1
When I set up my ADSL, it has a web page I visit, that I can fill in and read values. I'd like to be able to do something like that, using a browser as a very manual method of driving, for initial setup, debugging etc, zero programming at the client end. I tell it to change the range of a channel, read a voltage, no more. The web page would essentially be a soft front panel into the device hardware. Once the web magic gets as far as running a program, preferably python but C would do, that presents one button and one entry box on a web page, I know where I am and can do the rest.
Mode 2
When I plug an Arduino into my USB, I can program it to work over serial I/O, and write a python program on my PC to talk to it, that works fine. I understand essentially the same thing can be done over sockets to a device on the LAN. That's the mode I'd want to use for running it automatically from a custom python program. I'd write my battery charger/tester or whatever else in python, and use the device as a fairly dumb peripheral, just as I've written a battery tester to use hardware hanging off my Arduino board.
If it's possible to fake this mode from the first, and I suspect it is, by using the host PC python program to construct web requests, then maybe I only need mode 1 on the device. However, I suspect mode 2 is an order of magnitude easier to implement, at least on the device.
I've looked at python socket tutorials and been left confused. I've read around peoples' experiences putting such things together, and found things like 'it will work on loopback but won't work over my LAN', which have inhibited me from mustering enough effort to put something together and try. I have not yet reached sufficient a critical mass of knowledge and warm fuzzy feeling to start along the path of trying things out. I do not yet know what a good tutorial looks like, a pointer at a good one might be sufficient. I always have this difficulty starting with new things, don't understand until I've got my hands dirty, can't bring myself to get my hands dirty until I've got some basic level of understanding of what I should canonically be doing.
The path would probably be something like this.
1) Write a simple server that can be set, read, and generate new information, like a clock, running on the same machine as a client to talk to it over loopback, both ends in python. Implement both modes.
2) Move the server to a different machine on the LAN, initially a laptop and then an RPi over RJ45 for instance.
3) Get an Onion Omega or similar, and repeat over wifi.
Software side 'hello world' achieved, I should be able to take it from there.
I only have linux Mint on my machines, but I can ask my son, who has python on his win10 laptop, to try odd things out from time to time if necessary. I do use wine for things like LTSpice, but don't want to introduce yet another potentially buggy interface into a process that I'm already struggling with. I had rather hoped that working over LAN would be fairly platform independent, with python to take care of any differences.
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Shrad, Mon Jul 16 2018, 10:34AM
try this kind of implementation :
easy, fast, repeatable
Shrad, Mon Jul 16 2018, 10:34AM
try this kind of implementation :
easy, fast, repeatable
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Plasma, Mon Jul 16 2018, 11:25AM
If a client sends a Get request, the values get added to the Url, if it is post it is values in the page.
I would probably get a client to send a request to the server, something like 192.168.2.2,you server on Linux sends a packet /string array back to the client using write socket with \ r\n \r \n as the last four bytes, you can then use a Web browser on the client, the string array is just copy paste of a Web page.
The Web pages has box's to enter data, HTML and JavaScript with a sibmit button that sends a Get request to 192.168.2.2/index.html/volts=200mV,display=1sec the sever code receives data of GET index. Html/volts, you use string functions to parse the read socket looking for volts= you then send that value to the hardware.
On Mint download or install "wireshark" it's a network packet anylser it will show you what a request from a Web browser looks like.
Sorry hope it helps
Plasma, Mon Jul 16 2018, 11:25AM
If a client sends a Get request, the values get added to the Url, if it is post it is values in the page.
I would probably get a client to send a request to the server, something like 192.168.2.2,you server on Linux sends a packet /string array back to the client using write socket with \ r\n \r \n as the last four bytes, you can then use a Web browser on the client, the string array is just copy paste of a Web page.
The Web pages has box's to enter data, HTML and JavaScript with a sibmit button that sends a Get request to 192.168.2.2/index.html/volts=200mV,display=1sec the sever code receives data of GET index. Html/volts, you use string functions to parse the read socket looking for volts= you then send that value to the hardware.
On Mint download or install "wireshark" it's a network packet anylser it will show you what a request from a Web browser looks like.
Sorry hope it helps
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Dr. Slack, Fri Jul 20 2018, 10:12AM
@PLasma Let's see if I can start hauling myself out of the swamp.
I've located the python http library, which will serve files like the default index.html when you access the base site. I'm trying to understand when the choice between making a get or post request is made? Is it the case that the html in that page has javascript in it which constructs either a get or post request when the client at his browser hits a button or makes some other triggering action? The server library has functions like do_get() and do_post() which I would imagine get called at the obvious times?
Dr. Slack, Fri Jul 20 2018, 10:12AM
@PLasma Let's see if I can start hauling myself out of the swamp.
I've located the python http library, which will serve files like the default index.html when you access the base site. I'm trying to understand when the choice between making a get or post request is made? Is it the case that the html in that page has javascript in it which constructs either a get or post request when the client at his browser hits a button or makes some other triggering action? The server library has functions like do_get() and do_post() which I would imagine get called at the obvious times?
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Plasma, Fri Jul 20 2018, 01:02PM
I'm not 100% sure, if you a form and click submit, it should appear in the urls, saying that both will work the data will be in the packet, but get will be near the top and you can manually enter it in the urls, anything that pass data to another page will work.
I think you will need a auto reload code and if in the index. Html you have hard code editing 123.4 when the dimm gets a value based on the settings the client submitted, you modify the 123.4 as the html page is in a string array, and then send that to the client with the auto reload code will after X time requests the page again.
I was never any good at html and JavaScript, you might want some else to answer that question.
Plasma, Fri Jul 20 2018, 01:02PM
I'm not 100% sure, if you a form and click submit, it should appear in the urls, saying that both will work the data will be in the packet, but get will be near the top and you can manually enter it in the urls, anything that pass data to another page will work.
I think you will need a auto reload code and if in the index. Html you have hard code editing 123.4 when the dimm gets a value based on the settings the client submitted, you modify the 123.4 as the html page is in a string array, and then send that to the client with the auto reload code will after X time requests the page again.
I was never any good at html and JavaScript, you might want some else to answer that question.
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
the_anomaly, Fri Jul 20 2018, 04:54PM
Get and Post are client requests to a server so I believe do_get() and do_post() are server side functions that fill those requests.
Generally, Get is used to retrieve information from a server and Post is used to send information to the server W3Schools Get/Post. My experience has been with writing trading bots for crypto exchanges and they manage to implement their API by using only Get requests. I am not sure what is the reason for this since one must send data to their server to execute a trade. You make the Get request in such a way the server can parse the request for the data it needs to take the action you want.
I think the best way to approach this project is to view the server as a slave device and the client(s) as both the webpage and the DMM. The DMM will send its data to the server periodically. Then the webpage using Javascript or some other client side language will request updates from the server to get the latest values. I should clarify it's the browser (Chrome/Firefox) that is the client not the webpage; ie. Javascript runs on chrome and makes Get requests to updates the webpage display. Making Get request with Javascript
As you indicated, it does not need to be a web page as a client, it could just be another device sitting on the network. This device will make the get requests to the server.
Without knowing more about Post requests, I think you should have the DMM make Post requests to the server to send over its data.
I could be out in left field on a bunch of this too...
the_anomaly, Fri Jul 20 2018, 04:54PM
Get and Post are client requests to a server so I believe do_get() and do_post() are server side functions that fill those requests.
Generally, Get is used to retrieve information from a server and Post is used to send information to the server W3Schools Get/Post. My experience has been with writing trading bots for crypto exchanges and they manage to implement their API by using only Get requests. I am not sure what is the reason for this since one must send data to their server to execute a trade. You make the Get request in such a way the server can parse the request for the data it needs to take the action you want.
I think the best way to approach this project is to view the server as a slave device and the client(s) as both the webpage and the DMM. The DMM will send its data to the server periodically. Then the webpage using Javascript or some other client side language will request updates from the server to get the latest values. I should clarify it's the browser (Chrome/Firefox) that is the client not the webpage; ie. Javascript runs on chrome and makes Get requests to updates the webpage display. Making Get request with Javascript
As you indicated, it does not need to be a web page as a client, it could just be another device sitting on the network. This device will make the get requests to the server.
Without knowing more about Post requests, I think you should have the DMM make Post requests to the server to send over its data.
I could be out in left field on a bunch of this too...
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Dr. Slack, Sat Aug 04 2018, 05:10AM
I think I've now got a handle on this, so probably no further help needed until I have some much more specific questions. Thanks to anyone who's tried to think down to my level and failed. I was approaching it at too low a level, trying to ride this horse in the direction it wasn't going.
After a month of having the odd poke around at google and youtube, I've realised there are lots of server tutorials around for the ESP82(66) hardware, acrobotic seems particularly thorough and at my level, that could get me going. As I can already drive Arduino, using the arduino core approach on the ESP82 looks like a good fit.
While a web server on the server is not necessarily efficient, a DMM is really slow, and all computers wait at the same speed. Getting python to make web requests to such a server would get me up and running. If I wanted I could then develop that into a human-usable web interface, or socket level server, but probably wouldn't need to do either.
Dr. Slack, Sat Aug 04 2018, 05:10AM
I think I've now got a handle on this, so probably no further help needed until I have some much more specific questions. Thanks to anyone who's tried to think down to my level and failed. I was approaching it at too low a level, trying to ride this horse in the direction it wasn't going.
After a month of having the odd poke around at google and youtube, I've realised there are lots of server tutorials around for the ESP82(66) hardware, acrobotic seems particularly thorough and at my level, that could get me going. As I can already drive Arduino, using the arduino core approach on the ESP82 looks like a good fit.
While a web server on the server is not necessarily efficient, a DMM is really slow, and all computers wait at the same speed. Getting python to make web requests to such a server would get me up and running. If I wanted I could then develop that into a human-usable web interface, or socket level server, but probably wouldn't need to do either.
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Conundrum, Wed Sept 26 2018, 05:48AM
Hi, have you considered using yellow OLED panels for the display?
The ones here are basically the same but share a common controller and electrical interface.
Conundrum, Wed Sept 26 2018, 05:48AM
Hi, have you considered using yellow OLED panels for the display?
The ones here are basically the same but share a common controller and electrical interface.
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Dr. Slack, Wed Sept 26 2018, 09:08AM
You know conundrum, I could have sworn that my question was about getting help with the software, rather than suggestions for what sort of display to use.
I've now bought a NodeMCU 8266, and established that the ArduinoCore tools do work to flash its LEDs. However, the funny thing is, I haven't been motivated to program it as a server and talk to it, preferring other jobs around the house and garden, and other programming projects. Perhaps that's part of wanting somebody to collaborate on software, to give me some motivation to do it.
Dr. Slack, Wed Sept 26 2018, 09:08AM
You know conundrum, I could have sworn that my question was about getting help with the software, rather than suggestions for what sort of display to use.
I've now bought a NodeMCU 8266, and established that the ArduinoCore tools do work to flash its LEDs. However, the funny thing is, I haven't been motivated to program it as a server and talk to it, preferring other jobs around the house and garden, and other programming projects. Perhaps that's part of wanting somebody to collaborate on software, to give me some motivation to do it.
Re: Build an IoT DMM, hardware engineer with smattering of software seeks software engineer ...
Dr. Slack, Tue Oct 09 2018, 06:26AM
Actual progress! I think I ran out of other jobs to do, and perhaps the embarrassment of seeing this NodeMCU plugged into a breadboard, leering at me every morning from across the desk, propelled me into doing something.
I've elected to start playing with UDP rather than TCP, as it suits my dinosaur-like assembler brain better. I've now got a PC talking to itself, and to another on the network in python, and handling it when they're started up in the wrong order or stop responding for some reason. UDP is more than enough for 'take a reading' - 'here's the reading' exchanges. Tomorrow, the 8266!
Dr. Slack, Tue Oct 09 2018, 06:26AM
Actual progress! I think I ran out of other jobs to do, and perhaps the embarrassment of seeing this NodeMCU plugged into a breadboard, leering at me every morning from across the desk, propelled me into doing something.
I've elected to start playing with UDP rather than TCP, as it suits my dinosaur-like assembler brain better. I've now got a PC talking to itself, and to another on the network in python, and handling it when they're started up in the wrong order or stop responding for some reason. UDP is more than enough for 'take a reading' - 'here's the reading' exchanges. Tomorrow, the 8266!
Print this page