Friday, 5 December 2025

Absolute Overkill for a Bedside Clock


Our ancient bedside clock displayed very comforting dim red digits until it died. An iPhone in standby mode would have been a great replacement, except mine seems to cut out in the middle of the night on its magnetic stand, and other times I leave it in my pants pocket by mistake. The new bedside clock we bought turned out to be too bright and too blue, making us think dawn was coming at 3 AM, so I decided it should be easy to build a clock that meets our very simple needs:

  • It should be dim but readable in the dark without glasses.
  • The digits should be a nice friendly red.
  • The time should be accurate and not need resetting.

And ideally I should be able to build it from parts I have around the studio, and not need any new infrastructure.


The Overkill

If the clock could only just pull the correct time off the internet, it wouldn't need any hardware or code or human effort to always be right. If it checked the time often enough, it wouldn't even need to remember what time it was from one second to the next, or keep track of how much time went by. It turns out Adafruit IO has a service that will notify you every second just to let you know what time it is. 

I find it absolutely stupefying to think it might make sense to have a server in NYC call my little clock display every second of every day just to let it know another second has gone by. But the service is there, and it's free, so it's worth a try. It's also really easy to access through Arduino with File/Examples/Adafruit IO Arduino/adafruitio_17_time_subscribe.ino just by following the example code. (You will, of course, need to create an Adafruit IO id to be able to connect.)

The Microcontroller

The Adafruit ESP32 Feather (I think it was this one https://www.adafruit.com/product/3405 but it's soldered in now and impossible to check.) meets the two most important criteria: I had one and it talks wifi. I broke off the battery connector to give it a lower profile so it all fit together nicely.

The Display

The Adafruit 1.2" Four Digit, Seven Segment Display with BackPack https://www.adafruit.com/product/1270 comes in a nice comforting red and the digits are big enough to read from across the room without my glasses. It can be dimmed in 16 steps from 15 down to 0. 0 is still pretty bright in a dark room, but there doesn't seem to be a way to go any dimmer with the Holtek HT16K33 used on these backpacks, unlike the MAX7219 used on some others.

For some reason I can only guess at, the display refused to communicate by I2C until I also connected an Adafruit BME280 breakout board https://www.adafruit.com/product/2652, although the one I had was a much older version with no Stemma connectors. So the clock is also capable of some environmental monitoring, sort of by accident. My guess at the fault is that these old components may have had some mismatch on termination resistors for I2C, or maybe the stars just didn't align, but the fault and solution was repeatable.


Connecting and Packaging

A full size permaproto https://www.adafruit.com/product/590 provided a great support for all the components. The Feather USB supply pin provides 5V to the backpack, with the 3V pin supplying everything else, including the IO pin on the backpack. SDA and SCL are all connected and two pushbuttons are connected so they can pull pins 14 and 32 of the Feather to ground for signalling from the user.

After some extended testing in breadboard form, the whole package now fits in a 3D printed box that mounts to the dresser beside the bed. There's no structural mounting for the separate components, so it's not suitable for the g forces that would go with a mobile environment, or even shipping, but should be fine just sitting in one place.


Software

The basics are just a mash-up of the time subscription example and the BME280 example both provided by Adafruit as part of the libraries. That provides us with Temperature, Pressure, Humidity, and Zulu Time in an ISO format, but doesn't get us the current local time. The code below will need a config.h tab with your own network and Adafruit IO details in it, just like their examples.

Time zone correction is easy with a hard coded location setting int tz = -5. You can get time since local 1970 in seconds by adding tz*3600 to the UTC seconds value. I didn't allow for non-integer time zone offsets and don't know if there would be weird consequences of making tz floating point.

Daylight Savings Time is a little more complicated, since the time always changes at two AM on a Sunday. You need to know what day and date it is, as well as the time, which needs another library. Using the Arduino Library Manager, install "*Time* by *Michael Margolis*", which can be a little hard to find in the list. The include file is TimeLib.h even though the code is in Time.cpp. My own custom dsAdjust() function uses that library to break down the time into elements like day and date, then test them to see if it is North American DST at any particular time. It returns the number of hours (0 or 1) that should be applied to account for DST. ts gets adjusted in handleSecs() every second of every day.

The actual time to be displayed is generated by hhmmCalc(),  pulling out the time as a 1234 integer based on seconds since 1970 local, DST adjusted. 

Initially the plan was just to have it be dim all the time, but the buttons should do something, so the top button will make the display brighter and the bottom button will make it dimmer. Or if you don't touch the buttons, it will get brighter and dimmer according to the time of day. Brightest around two in the afternoon  and dimmest between 9 PM and 7 AM. To prevent it from automatically resetting the level as soon as you take your finger off the button, the automatic reset only happens if you have kept your fingers off the buttons for 12 hours.

The buttons can also surface some of the BME280 data. When the top button is pushed the raw pressure in millibars is displayed and the bottom button will show relative humidity. The pressure is not altitude adjusted, so it will read about 9 or 10 millibars low here on the third floor in Kingston. I'm not so sure about interpreting the humidity, as the temperature of the sensor may be higher than the room and the air in the box may be more or less humid than the room. These are mostly there because I could... not because I should.

Questions

Will it need software updates in the future? It is certainly set up to only run on our home network with a fixed SSID, but what will happen if I get a new router with better security, but still give it the same SSID? It also assumes we remain in the same time zone and the rules for daylight saving time remain the same.

What if I live in Newfoundland? You will need to write some more general (or more specific) software to deal with the half hour time zone effect. This is not a problem I anticipate in my own bedroom, but I think it may be as simple as make tz a double instead of an int and setting it to 3.5.

// Bedroom Clock (RWS) converted from Adafruit IO Time Topic Subscription Example
// and BME280 Example, and display example.
// This code is running on hardware with Adafruit ESP32 Feather processor,
// 7 seg display with I2C backpack, and BME280.
//
// Adafruit invests time and resources providing this open source code.
// Please support Adafruit and open source hardware by purchasing
// products from Adafruit!
//
// Written by Adam Bachman, Brent Rubell for Adafruit Industries
// Copyright (c) 2018 Adafruit Industries
// Licensed under the MIT license.
//
// All text above must be included in any redistribution.

/************************** Configuration ***********************************/

// edit the config.h tab and enter your Adafruit IO credentials
// and any additional configuration needed for WiFi, cellular,
// or ethernet clients.
#include "config.h"

/************************ Example Starts Here *******************************/

// set up the 'time/seconds' topic
AdafruitIO_Time *seconds = io.time(AIO_TIME_SECONDS);

#include <Wire.h> // Enable this line if using Arduino Uno, Mega, etc.
#include <Adafruit_GFX.h>
#include "Adafruit_LEDBackpack.h"
#include "TimeLib.h"
#include <Adafruit_Sensor.h>
#include <Adafruit_BME280.h>

Adafruit_7segment matrix = Adafruit_7segment();
Adafruit_BME280 bme; // I2C

int hhmm = 3333; // hours and minutes as a four digit number hhmm
int tz = -5; // time zone adjustment
int ts = 0; // daylight savings time adjustment
unsigned long lastBrightnessAdjustment = 0; // time in millis()

void setup() {
// start the serial connection
Serial.begin(115200);
// wait for serial monitor to open
while(! Serial);
Serial.print("BedroomClock V1.0\n\nStarting Display, init BME, and Connecting to Adafruit IO");
matrix.begin(0x70);
matrix.setBrightness(0);
unsigned status = bme.begin();
if (!status) {
Serial.println("Could not find a valid BME280 sensor, check wiring, address, sensor ID!");
Serial.print("SensorID was: 0x"); Serial.println(bme.sensorID(),16);
Serial.print(" ID of 0xFF probably means a bad address, a BMP 180 or BMP 085\n");
Serial.print(" ID of 0x56-0x58 represents a BMP 280,\n");
Serial.print(" ID of 0x60 represents a BME 280.\n");
Serial.print(" ID of 0x61 represents a BME 680.\n");
}

// start MQTT connection to io.adafruit.com
io.connect();
// attach message handler for the seconds feed
seconds->onMessage(handleSecs);
// wait for an MQTT connection
while(io.mqttStatus() < AIO_CONNECTED) {
Serial.print(".");
delay(500);
}
// we are connected
Serial.println();
Serial.println(io.statusText());

pinMode(14,INPUT_PULLUP);
pinMode(32,INPUT_PULLUP);
}

double temperature = 0.0, pressure = 0.0, humidity = 0.0;

void loop() {
static int userBrightness = 0;
// increase up to 15 in steps every 1/4 second if top button pressed
if(userBrightness < 15 && millis()-lastBrightnessAdjustment > 250 && !digitalRead(14)){
userBrightness++;
lastBrightnessAdjustment = millis();
Serial.printf("userBrightness raised to: %d\n",userBrightness);
}
// increase down to 0 in steps every 1/4 second if bottom button pressed
if(userBrightness > 0 && millis()-lastBrightnessAdjustment > 250 && !digitalRead(32)){
userBrightness--;
lastBrightnessAdjustment = millis();
Serial.printf("userBrightness lowered to: %d\n",userBrightness);
}
// if no past adjustment, or it's at least 12 hours in the past, make it bright/dim by time
if(lastBrightnessAdjustment == 0 || (millis()-lastBrightnessAdjustment)/3600000 > 12 ){
userBrightness = abs(1400-hhmm)/50; // 28 just past midnight, 20 just before, 14 at 7AM and 9PM
userBrightness = 14 - userBrightness; // 14 in the afternoon, dropping to 0 between 2100 and 0700
if(userBrightness < 0) userBrightness = 0; // don't go below zero
Serial.printf("userBrightness auto set to: %d\n",userBrightness);
}
matrix.setBrightness(userBrightness);

io.run();
temperature = bme.readTemperature();
pressure = bme.readPressure();
humidity = bme.readHumidity();
matrix.writeDigitNum(0, (hhmm / 1000), false); // draw digit by digit to force leading zeros
matrix.writeDigitNum(1, (hhmm / 100) % 10, false);
matrix.drawColon(true);
matrix.writeDigitNum(3, (hhmm / 10) % 10, false);
matrix.writeDigitNum(4, hhmm % 10, false);
if(!digitalRead(14)) matrix.print((int) (pressure/100));
if(!digitalRead(32)) matrix.print((int) (humidity+0.5));
matrix.writeDisplay();
//delay(1);
}


// message handler for the seconds feed
void handleSecs(char *data, uint16_t len) {
Serial.print("Seconds Feed: ");
Serial.println(data);
ts = dsAdjust(atoi(data) + tz *3600); // set daylight savings time status based on seconds since 1970 local standard time
hhmm = hhmmCalc(atoi(data) + (ts+tz)*3600); // set hours and minutes based on DST and zone
Serial.printf("Temp: %6.2f°C Press: %6.0f hPa Hum: %6.2f%%\n",temperature,pressure/100,humidity);
if(!digitalRead(14)) Serial.println("Green Button Pushed -- Pin 14 pulled low\n");
if(!digitalRead(32)) Serial.println("Red Button Pushed -- Pin 32 pulled low\n");
}

int hhmmCalc(time_t secs){ // return an integer like 1234 for 12:34
TimeElements els; // time library breakdown of latest time in elements
breakTime(secs, els); // secs as passed must be adjusted to local time, including DST if applicable
int hhmm = els.Hour*100 + els.Minute;
while(hhmm < 0) hhmm += 2400; // force back into 0-2359 if needed
while(hhmm >= 2400) hhmm -= 2400;
return hhmm;
}

int dsAdjust(time_t secs) { // Calculate the Daylight Savings Time adjustment for North America
TimeElements els; // time library breakdown of latest time in elements
breakTime(secs, els); // secs as passed must be adjusted to local standard time to get the moments of transition right
// Serial.printf("%d %d %d\n",els.Hour,els.Minute,els.Second);
if(els.Month > 3 && els.Month < 11) return 1; // April through October is daylight savings
if(els.Month == 3 && els.Wday == 1 && els.Day > 7 && els.Day < 15) // or if it is the second Sunday in March
if(els.Hour >= 2) return 1; // yes if after 2AM standard
else return 0; // otherwise not if before 2AM standard
if(els.Month == 3 && ((els.Day-els.Wday)-7) >= 0) return 1; // or if it is March on or after the second Sunday
if(els.Month == 11 && (els.Day-els.Wday) < 0) return 1; // or if it is November and before the first Sunday
if(els.Month == 11 && els.Wday == 1 && els.Day < 8 && els.Hour < 3) return 1; // or if it is the first Sunday in November and before 3AM standard (2AM DST)
return 0; // otherwise not
}


Thursday, 6 October 2022

A Case for Adafruit LED Backpacks

 Adafruit just updated https://www.adafruit.com/product/881 the seven segment LED backpacks with STEMMA QT connectors, and that prompted me to pull out some of my old ones and see about building them a case. It should work for the new versions that still seem to use the same mounts. 



Files at https://www.thingiverse.com/thing:5551598 It should be easy to carve off single 30x60mm holders if you have fewer displays, or want to drop them into individual windows in your own case.

The backpacks fit into sockets in the faceplate and are secured by plastic pins you can melt with your soldering iron to hold them in place. The faceplate fits into the stand, which has a pocket to carry a half size breadboard.

Monday, 16 November 2015

ESP8266 and Arduino for Logging Data to the Web

Update 2016-12-30: The process has become much simpler. Use the ThingSpeak or AdafruitIO libraries available for IDE 1.6.x and above to interact and avoid having to assemble your own network communications. ThingSpeak is free for the first 3 million channel updates a year, then paid. AdafruitIO is free, but you are limited to 10 fields total. Both limit the update speeds.

One of my students (Stephane Leahy) insisted I should find a way to include the ESP8266 in my grad measurements course for next term, so I had to give it a try. Now that I've proven I can do it, I'm not sure putting it in a course would be so useful, as this is bound to be a moving target. I think the best advice is to know this is possible, then go looking for a sensible solution when you have an actual application that needs it. The ESP8266 is a really cheap solution to adding WiFi when you need it, and may be enough micro-controller to stand alone if you don't need a lot of analog input.

I bought an Adafruit Huzzah ESP8266 breakout board and combined it with an Arduino Micro and some sensors. I used some sample code Stephane provided, then built on it to overcome some of the difficulties I ran into. Wiring was straightforward following the instructions. I added a TMP36 connected to the A (analog A0) pin on the 8266 and connected the RX/TX pins to listen to output from the Micro.


The 8266 is programmed with an FTDI cable over the serial port, so the micro needs to be unpowered, or at least silent, while the programming the 8266. I sorted this out by using a diode to bridge the two power busses at the top of the board. (It's hiding behind the blue pot.) When the Micro is plugged in, it will power both, with the 8266 getting about 4.1V. When only the 8266 is plugged in the Micro is unpowered and silent. I used the Arduino IDE to program the 8266 because it was familiar. When live, the Micro was just chugging along in data logger mode, spitting out lines of CSV data every second or so to the serial port that looked like this:

477971510,   734,   719,   719, 11001111,  1500,  1466,  1461,  3

The 8266 reads this data, parses it, reads an analog input and converts it to temperature, then sends all of that off to ThingSpeak for logging and reporting in the cloud. The resulting graphs look like this once things work:

This particular sequence shows how measuring temperature requires a little finesse. Sitting in the dark all night, the temperature was stable around 22.5C, but once I started working the temperature around my desk started rising. Unplugging the breadboard let it cool down, only to heat back up when it was plugged in again. Finally around 1500 I moved the board to a windowsill to catch the last few rays of late November sun. The indicated temperature peaked just as the sun went behind the building next door. It's hard to tell what is actual room temperature rise through the day and what is just sensor heating from the electronics or the sun.

ESP8266 Challenges

Analog Reading Scale: It wasn't obvious to me what the 8266 was using as a voltage reference for the analog conversions, so I used my multimeter to measure the conversion constant. The lack of clear documentation for the chip is a significant failing, observed by plenty of others.

The fiddly little buttons need to be actuated in sequence, so it was difficult for me to start the programming sequence with my full size fingers.

Analog disables connection for about 5 seconds for no apparent reason (see above for lack of documentation). This quirk would be enough to make me avoid using the analog input entirely, especially since there's only the one channel.

Retrying connection can crash the whole system with an exception number 9, which the web seems to think has something to do with too many connections open, or something (docs?).

Power hungry is only an issue if you need to run the chip on batteries for an extended period.

The code below should be reasonably readable to show how I got past these hurdles. It took a few days to track down the random exceptions, but I will keep my fingers crossed. Feel free to copy and play around with it. Good Luck!


// a minimal ThingSpeak logger based on Stephane Leahy's code --- Rick Sellens 2015-11-14
#include <ESP8266WiFi.h>

// this file should include the next few lines, but with your ssid, password, and thingspeak channel API key
#include "ssid.h"   // create ssid.h as another file tab in your Arduino IDE project.
// #define SSID "MyNetworkName" 
// #define PASS "MyCleverPassword"
// #define HOST "api.thingspeak.com"   // usually results in 0 or 1 retries to get a connection but takes longer then IP
// String link = "/update?key=TheAPIKeyFromMyThingSpeakChannel";

/*
The first client.connect(HOST,80) after reboot always succeeds with 0 retries.
Defining HOST as numerical and commenting out the analogRead() and delay(5000) connects first time, every time.
Uncommenting the analogRead() and success requires about 500 retries (about 5 seconds at 10 ms each).
Switching to a domain name for HOST succeeds after 1 retry (about 5 seconds).
Uncommenting the delay(5000) succeeds with 0 retries for domain name and 0 or 1 retries for numerical IP.
Making the delay 7000 succeeds with 0 retries most times for the numerical IP.

Behaviour is the same when HOST is 205.189.10.43 for weather.gc.ca, so it seems to be a local issue
Moral is probably to accept failure for a while..... and use the domain name version

The logging sometimes fails, so I added a restart condition if the connection doesn't get made quickly enough.
The latest failure said "ets Jan  8 2013,rst cause:4, bootmode:(1,7) \n\n wdt reset" which the online people
suggest has to do with a watchdog timer reset that may happen if it gets stuck in a loop for more than a second,
so it seems like a good idea to make the serial timeout shorter than a second to see if that makes the problem go away.
It didn't go away
  
 870942170,   4998,    477,    343, 0, 1, 1, 1, 1, 1, 1, 0,  10230,    970,    720, 
 871442210,   4998,    476,    346, 0, 1, 1, 1, 1, 1, 1, 0,  10230,    970,    721, 
 871942210,   4998,    476,    348, 0, 1, 1, 1, 1, 1, 1, 0,  10230,    970,    721, 
 872442250,   4998,    477,    344, 0, 1, 1, 1, 1, 1, 1, 0,  10230,    972,    673, 
 872942280,   4998,    477,    341, 0, 1, 1, 1, 1, 1, 1, 0,  10230,    972,    673, 
 873442310,   4998,    477,    338, 0, 1, 1, 1, 1, 1, 1, 0,  10230,    972,    671, 
 873942350,   4998,    477,    336, 0, 1, 1, 1, 1, 1, 1, 0,  10230,    973,    668, 
 874442390,   4998,    477,    338, 0, 1, 1, 1, 1, 1, 1, 0,  10230,    970,    701, 
 874942420,   4998,    477,    340, 0, 1, 1, 1, 1, 1, 1, 0,  10230,    970,    703, 
 875442450, 875442432, 875442432, 875442432, 875442432, 875442432, 875442432, 875442432, 8, 7, 5, 4, 4, 2, 4, 5, 0, 875442450,   4998,    476,    342, 1111110,  10230,    970, 
Exception (9):
epc1=0x401018af epc2=0x00000000 epc3=0x00000000 excvaddr=0x202c3029 depc=0x00000000

ctx: cont 
sp: 3ffea5f0 end: 3ffea870 offset: 01a0

>>>stack>>>
3ffea790:  ffffffff 00000007 3ffea8c8 40202541  
3ffea7a0:  3738200a 32343435 2c303534 35373820  
3ffea7b0:  34323434 202c3233 34353738 33343234  
3ffea7c0:  38202c32 34343537 32333432 3738202c  
3ffea7d0:  32343435 2c323334 35373820 34323434  
3ffea7e0:  202c3233 34353738 33343234 38202c32  
3ffea7f0:  34343537 32333432 2c38202c 202c3720  
3ffea800:  34202c35 2c34202c 202c3220 35202c34  
3ffea810:  2c30202c 35373820 34323434 202c3035  
3ffea820:  39342020 202c3839 34202020 202c3637  
3ffea830:  33202020 202c3234 31313131 2c303131  
3ffea840:  30312020 2c303332 20202020 2c303739  
3ffea850:  3fff0020 00000000 3ffea894 4020186e  
3ffea860:  00000000 00000000 3ffe9850 40100378  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(1,7)


 ets Jan  8 2013,rst cause:4, boot mode:(1,7)

wdt reset

Apparently exception 9 has something to do with too many floating clients, or something, so I tried lengthening the time
between connection attempts from 10 ms to 100 ms, so it doesn't have to try as many times before the 5 s runs out, or whatever.
2015-11-16: Then it ran all night. 
 */


float smoothTemp = 20.0;
unsigned long t = 0;

void setup() {
  Serial.begin(115200);
  Serial.setTimeout(500L);  // timeout in half a second to keep the wdt reset from triggering on the ESP8266

  // connect to WiFi
  WiFi.begin(SSID, PASS);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("WiFi connected with ");
  Serial.print("IP address ");
  Serial.println(WiFi.localIP());

}

static unsigned raw[] = {0,0,0,0,0,0,0,0,0,0};
static float cal[] = {0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0};
static byte digio[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};

void loop() {
  static unsigned connFail = 0;       // number of times the connection has failed
  static unsigned smoothies = 0;      // number of samples that have fed into the smoothing, or 20000 max
  static unsigned reported = 0;       // set to one when new data comes in, then back to zero when logged
  unsigned data = 750;                // a not-stupid, not-zero value that could have come from the analog input

  // serial incoming lines from the logger like:  477971510,   734,   719,   719, 11001111,  1500,  1466,  1461,  3
  if(Serial.available()){                            // only if there's something waiting
    String line = Serial.readStringUntil('\n');
    //parse the string for data to report
    String s = line.substring(line.length()-1);     // last character is the number of channels        
    unsigned n = s.toInt();
    line = line.substring(0,line.lastIndexOf(","));

    // take n channels of raw values off the end of the string
    for(int i = n-1;i>=0;i--){
      s = line.substring(line.lastIndexOf(",")+1);
      raw[i] = s.toInt();
      line = line.substring(0,line.lastIndexOf(","));
    }
    
    // take the digital data off the end of the string
    s = line.substring(line.lastIndexOf(",")+1);
    s.trim();
    unsigned nd = s.length();
    for(int i = 0;i < nd;i++) digio[i] = s.charAt(i) - '0';
    line = line.substring(0,line.lastIndexOf(","));

    // take n channels of calibrated data off the end of the string
    for(int i = n-1;i>=0;i--){
      s = line.substring(line.lastIndexOf(",")+1);
      cal[i] = s.toFloat();
      line = line.substring(0,line.lastIndexOf(","));
    }

    // take the time since reboot in us off the end of the string
    line = line.substring(0,line.length() - 1);       // drop a digit to fit inside the long int limits
    unsigned ts = (unsigned long) line.toInt() * 10;  // put the digit back on to read rounded down to 10
    
    // report as a check.
    char scratchy[100];
    sprintf(scratchy,"\n%10lu, ",ts);
    for(int i = 0; i < n; i++) sprintf(scratchy,"%s%6u, ",scratchy,(unsigned) cal[i]);
    for(int i = 0; i < nd; i++) sprintf(scratchy,"%s%1u, ",scratchy,(unsigned) digio[i]);
    for(int i = 0; i < n; i++) sprintf(scratchy,"%s%6u, ",scratchy,raw[i]);
    if(connFail == 0) Serial.print(scratchy);
    
    // ditch any leftovers
    while (Serial.available()){ 
      char c = Serial.read();
    }
    reported++;     // some data got reported on the Serial port
  }
  
  data = analogRead(A0);
  if(data != 0){
    float mvData = data /1.056;   // constant is empirically measured
    smoothTemp = smoothTemp *.99 + ((mvData - 500.) / 10.)*.01;
  }
  if (smoothies < 20000) smoothies++;

  // This part should be last in the loop, since it will generate an early return on failure.
  // if( long enough to have valid data && (it has been long enough || this is the first time) ) then log data
  if ( smoothies > 10000 && ((millis() - t > 1000 * 20L) || (t == 0)) ) {
    String url = link + "&field1=" + smoothTemp;
    // report the Serial data if it has been updated a few times since last logging -- ignore plug/unplug
    if(reported > 5) url = url + "&field2=" + (cal[0]/1000) + "&field3=" + (cal[1]/1000)   + "&field4=" + (cal[2]/1000) 
                      + "&field5=" + (1.6 + 0.8 * digio[2])  
                      + "&field6=" + (4.6 + 0.8 * digio[5]) 
                      + "&field7=" + (6.6 + 0.8 * digio[7]);
    WiFiClient client;            // create TCP connection
    if(connFail == 0){ Serial.print("\nOpening connection to: ");  Serial.print(HOST); }
    //delay(5000);                // this delay may let things settle long enough to get a connection first time
    if (!client.connect(HOST, 80)) {    // this connection seems to fail often, except the first try after reboot
      if (connFail == 0) Serial.print(" failed ... ");
      else if(connFail%100 == 0) Serial.print(".");
      connFail++;
      delay(200);             // don't retry immediately -- avoid flooding net connection and ESP exception 9
      if(connFail > 100){     // redo from start if it has taken too long
        Serial.print("Making a clean start of it\n\n");
        WiFi.disconnect();
        delay(1000);
        setup();
        connFail = 0;
      }
      return;       // don't even bother with the GET
    }
    // send GET request to the server, read reply and print it
    Serial.print(" success after "); Serial.print(connFail); Serial.print(" retries!\nRequesting URL: ");
    connFail = 0;
    Serial.print(url); Serial.print(" ... ");
    client.print(String("GET ") + url + " HTTP/1.1\r\n" + "Host: " + HOST + "\r\n" + "Connection: close\r\n\r\n");
    reported = 0;       // the data just got logged
    // wait up to 1000 ms to hear back from the server, then move on
    for(unsigned long waiting = millis(); !client.available() && millis()-waiting < 1000;);
    //while (client.available()) {      // uncomment the while if you want more than just one line
      String line = client.readStringUntil('\r');
      Serial.print(line);
    //}
    t = millis();
  }

}

Wednesday, 28 October 2015

Serial Buffer Sizes on Arduino

Often you want to leave the serial port alone to send or receive data under interrupts while the main loop of the code wanders off to do other things. If you don't get back often enough, the buffer may overflow. The standard buffer size in the Arduino IDE is 64 bytes (or only 16 for really small RAM boards). The code is in arduino/avr/cores/arduino/HardwareSerial.h and looks something like this:

#if !defined(SERIAL_TX_BUFFER_SIZE)
#if (RAMEND < 1000)
#define SERIAL_TX_BUFFER_SIZE 16
#else
#define SERIAL_TX_BUFFER_SIZE 64
#endif
#endif

Altering that code to something like this can get you some more buffer space on larger processors.

#if !defined(SERIAL_RX_BUFFER_SIZE)
#if (RAMEND < 1000)
#define SERIAL_RX_BUFFER_SIZE 16
#else
// this test will create a large buffer on anything big
#if (RAMEND > 2200)
#define SERIAL_RX_BUFFER_SIZE 512
#else
#define SERIAL_RX_BUFFER_SIZE 64
#endif
#endif
#endif

The downside of this is you will probably lose your mods when you next upgrade the IDE, and they won't move with your sketch code if you take it to another platform. I would like to be able to include these defines in my sketch, but I'm not sure how to get them snuck in at the right point in the build process.

Also, this file change won't translate to other processors that install their own code under the IDE, like the Teensy family. Paul Stoffregen has kept all his defines in the individual c code files like teensy/avr/cores/teensy3/serial1.c, where changing these should let you increase the size as needed:

////////////////////////////////////////////////////////////////
// Tunable parameters (relatively safe to edit these numbers)
////////////////////////////////////////////////////////////////

#define TX_BUFFER_SIZE 64 // number of outgoing bytes to buffer
#define RX_BUFFER_SIZE 64 // number of incoming bytes to buffer

I saw a note somewhere recommending power of two buffer sizes for efficiency in index arithmetic, and I can't really imagine going anything other than 64, 128, 256, 512, etc. anyway. In my current application I'm expecting to get packets of around 250 bytes back on request, so I will set the RX buffer size to 512 to be sure it's big enough to catch the whole packet. The request code is short, so I won't increase the size of the TX buffer.


Wednesday, 10 December 2014

Python Plots from Serial Input

The Arduino IDE doesn't do much with serial data returned from the board, other than display it on the serial console. Python provides a platform independent way to listen to your Arduino and draw graphs of analog data, or whatever you want. The code below provides a stripchart display based on the assumption that lines of

millis(), A0, A1, A2, A3, A4, A5

format are coming back from the Arduino on the serial port. It's hardcoded with port and speed, but easy to edit. I think the indents are OK, but this blog platform strips out the tabs on pasting...

Update: I did all of this with an arduino micro. Like the Leonardo, it doesn't reboot when you re-initialize the USB. When I switched to the UNO with the same code, I had to adjust it so it waited and read a few more lines before acting on the data. If you don't want to disrupt the UNO, you probably need to make a direct RS232 connection through an FTDI cable, rather than the onboard USB.



import matplotlib.pyplot as plt
import time
import serial

plt.ion()
ser = serial.Serial('/dev/tty.usbmodemfd1471',57600,timeout=1)
line = ser.readline() # throw away any part lines
while(ser.inWaiting() < 100): # make sure something is coming
  now = 0.0
t=[] # initialize the data lists
d1=[]
d2=[]
d3=[]
d4=[]
d5=[]
d6=[]
while (ser.isOpen()):
  line = ser.readline() # read a line of text
  mylist = line.split(",") # parse it into CSV tokens
  #print mylist
  now = float(mylist[0])/1000 # time now in seconds
  t.append(float(mylist[0])/1000) # from first element as milliseconds
  d1.append(float(mylist[1])) # six data elements added to lists
  d2.append(float(mylist[2]))
  d3.append(float(mylist[3]))
  d4.append(float(mylist[4]))
  d5.append(float(mylist[5]))
  d6.append(float(mylist[6]))
  if(ser.inWaiting() < 100): # redraw only if you are caught up
    plt.clf() # clear the figure
    plt.plot(t,d1) # plot a line for each set of data
    plt.plot(t,d2)
    plt.plot(t,d3)
    plt.plot(t,d4)
    plt.plot(t,d5)
    plt.plot(t,d6)
    plt.axis([now-60,now,min(d1)-50,max(d1)+50])
    plt.xlabel("Time Since Boot [s]")
    plt.draw()

By request, here's the arduino code I was running, which does a bunch more than just print analog values

//// AvgDAQ
// Makes the potentially dangerous assumption that the analog pins are numbered in sequence, starting at A0

//// Constants
int d = 1;
int navg = 10;
int nan = 6;
int nrpt = 500;
int ndisp = 20000;

void setup() {
  for(int i = 0; i < nan; i++) pinMode(A0 + i, INPUT);
  Serial.begin(57600);   
  analogReference(INTERNAL);  // 2.56 volts on the YUN, 1.1 on UNO
  //analogReference(EXTERNAL);  // based on the input to AREF
  Serial.print("\n\nAvgDAQ\n");
}

void loop() {
  static int linesShown = 0;
  unsigned long sums[10];    // doesn't want to work with NAN as a dimension
  getAvgDAQ(sums);
  if(linesShown < ndisp){ 
    showSums(sums);
    linesShown++;
  }
  while(millis()%nrpt);
}

void getAvgDAQ(unsigned long *sums) {
  sums[nan] = (unsigned long) millis();
  for(int i = 0;i < nan;i++) sums[i] = 0;
  for(int i = 0;i < navg;i++) for(int j = 0;j < nan;j++){ 
    sums[j] += analogRead(A0+j); 
    delayMicroseconds(d);
  }
  for(int i=0;i < nan;i++){ 
    //sums[i] *= 1404;  //convert to mv for 1.404 V AREF
    //sums[i] *= 2560;  //convert to mv for 2.56 V internal ref
    sums[i] *= 1100;    //convert to mv for 1.10 V internal ref
    sums[i] /= 1024;
    sums[i] /= navg;    // average over NAVG samples
  }
}

void showSums(unsigned long *sums) {
  char scratch[80];
  
  Serial.print(sums[nan]);
  scratch[0]=0;    // when in doubt code dangerously, writing a string over itself ;-)
  for(int i = 0;i < nan;i++) sprintf(scratch,"%s, %5d",scratch,(int) sums[i]);
  sprintf(scratch,"%s,   *",scratch);
  Serial.print(scratch);
  for(int i=0;i<80;i++) scratch[i] = '.';
  scratch[79] = 0;
  for(int i=0;i<nan;i++){
    long j = sums[i] / 20;
    if(j > 78) j = 78;
    scratch[j] = '0'+i;
  }
  Serial.println(scratch);
}

and of course to get all of this to run, you need to have pyserial and matplotlib installed. On a mac you can try

sudo easy_install pip
sudo pip install pyserial
sudo pip install matplotlib