Syncing RTC To Network Time

thefebruaryman

I have been looking at the same issue tonight and came up with this script

# imports
from machine import Pin
import pycom
import time
import ustruct
from machine import RTC
import socket
from socket import AF_INET, SOCK_DGRAM

pycom.heartbeat(False)

rtc = RTC()
rtc.init((1970, 0, 0, 0, 0, 0, 0, 0))

def getNTPTime(host = "pool.ntp.org"):
    port = 123
    buf = 1024
    address = socket.getaddrinfo(host,  port)[0][-1]
    msg = '\x1b' + 47 * '\0'
    msg = msg.encode()
    TIME1970 = 2208988800 # 1970-01-01 00:00:00

    # connect to server
    client = socket.socket(AF_INET, SOCK_DGRAM)
    client.sendto(msg, address)
    msg, address = client.recvfrom(buf)
    t = ustruct.unpack("!12I", msg)[10]
    t -= TIME1970
    rtc.calibration(t)
    return t

def pin_handler(arg):
    print("Got an interrupt in pin %s" % (arg.id()))
    pycom.rgbled(0x0000ff)
    time.sleep(1)
    pycom.rgbled(0x000000)
    print(getNTPTime())
    print(rtc.now())

# use the button on the expansion board  P10
p_in = Pin('G17',  mode=Pin.IN,  pull=Pin.PULL_UP)
# on button push trigger pin_handler
p_in.callback(Pin.IRQ_FALLING,  pin_handler)

however the calibration method doesn't like it

>>> Got an interrupt in pin P10
Unhandled exception in interrupt handler
Traceback (most recent call last):
File "<stdin>", line 36, in pin_handler
File "<stdin>", line 28, in getNTPTime
OverflowError: overflow converting long int to machine word

livius

@thefebruaryman

what you got when you do?:
print(str(ustruct.unpack("!12I", msg)[10]))

thefebruaryman

t = ustruct.unpack("!12I", msg)[10]

returns

1485764691

which in python I can normally

>>> time.ctime(1485764691)
'Mon Jan 30 08:24:51 2017'

livius

@thefebruaryman
ok - then you succesfully got value
but issue is in calibration procedure
but i suppose you can fix problem by setting time by

ct = time.ctime(t)
rtc.init((ct[0], ct[1], ct[2], ct[3], ct[4], ct[5], 0, 0))

dchappel

Here is what I came up with (builds on Knoahlr contribution below):

import socket
import machine

s=socket.socket()
s.connect(socket.getaddrinfo("66.199.22.67",13)[0][-1])
r=s.recv(1024).decode()
s.close()

year = 2000 + int(r[6:9])
month = int(r[10:12])
day = int(r[13:15])
hour = int(r[16:18])
min = int(r[19:21])
sec = int(r[22:24])
rtc = machine.RTC()
rtc.init((year, month, day, hour, min, sec, 0, 0))
print(rtc.now())

Current time (in seconds since 1970 / UNIX time) is now available using utime.time().

I found that I needed to use the numeric IP address for NIST. Alpha URL gave me a timeout. Since I got it to work with numeric I didn't go back and try to understand why...

Knoahlr

@dchappel You might want to use RegEX,
Assuming Current time is your response from the NIST server.

>>import re
>>CurrentTime='\n57783 17-01-30 16:14:50 00 0 0  37.8 UTC(NIST) * \n'
>>DateList=re.search("[0-9]+-[0-9]+-[0-9]+",CurrentTime).group(0).split("-")
>>DateList
['17', '01', '30']
>>TimeList=re.search("[0-9]+:[0-9]+:([0-9]+.[0-9]+)",CurrentTime).group(0).split(":")
>>TimeList
['16', '14', '50 00']

Cheers.

thefebruaryman

Ok I got it. I have to use utime for localtime method to work with the large int else you get an error.

from machine import Pin
import pycom
import utime
import ustruct
from machine import RTC
import socket
from socket import AF_INET, SOCK_DGRAM

pycom.heartbeat(False)

rtc = RTC()
rtc.init((1970, 0, 0, 0, 0, 0, 0, 0))

def getNTPTime(host = "pool.ntp.org"):
    port = 123
    buf = 1024
    address = socket.getaddrinfo(host,  port)[0][-1]
    msg = '\x1b' + 47 * '\0'
    msg = msg.encode()
    TIME1970 = 2208988800 # 1970-01-01 00:00:00

    # connect to server
    client = socket.socket(AF_INET, SOCK_DGRAM)
    client.sendto(msg, address)
    msg, address = client.recvfrom(buf)
    t = ustruct.unpack("!12I", msg)[10]
    t -= TIME1970
    tuple_time = utime.localtime(t)
    rtc.init((tuple_time))
    client.close()


def pin_handler(arg):
    print("Got an interrupt in pin %s" % (arg.id()))
    pycom.rgbled(0x0000ff)
    utime.sleep(1)
    pycom.rgbled(0x000000)
    getNTPTime()
    print(rtc.now())

# use the button on the expansion board  P10
p_in = Pin('G17',  mode=Pin.IN,  pull=Pin.PULL_UP)
# on button push trigger pin_handler
p_in.callback(Pin.IRQ_FALLING,  pin_handler)

abilio

We've just finished adding support for a NTP (SNTP) client. Syncing time will be as easy as:

import machine
rtc = machine.RTC()
rtc.ntp_sync("pool.ntp.org") # select an appropriate server

By default it will resync every hour, but you can change the period by passing it as a parameter: rtc.ntp_sync("pool.ntp.org", 1200) # do it every 20 minutes

This feature will probably be included in the next release. Hope you'll find it useful.

thefebruaryman

@abilio that's great news and much easier to use. Cheers for the good work!

Knoahlr

@abilio Will the method sync the RTC to local time or UTC ?

abilio

@Knoahlr, to UTC, but then you'll be able to set the time offset calling:

time.timezone(3600) # GMT +1 for the Netherlands

A call to time.localtime() will return the time using the provided offset. If you want the number of seconds since Epoch adjusted to the timezone, you can do something like:

time.time() + time.timezone()

Also, time.gmtime() will be included, working like time.localtime(), but without adjusting for the offset.

So far, this is what would be available in the release.

Knoahlr

@abilio Great, amazing feature.

rdixey

@abilio Apparently this is not yet implemented in 1.6.1.b1 ?

'1.6.1.b1'
>>> os.uname()
(sysname='LoPy', nodename='LoPy', release='1.6.1.b1', version='v1.8.6-450-g31ef5170 on 2017-02-15', machine='LoPy with ESP32', lorawan='1.0.0')
>>> import machine
>>> rtc = machine.RTC()
>>> rtc.ntp_sync("pool.ntp.org")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'RTC' object has no attribute 'ntp_sync'
>>> 
>>> import time
>>> time.timezone(3600)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'timezone'

abilio

@rdixey, you are totally right and I must personally apologize for it. We forgot to merge it with master just to realize it after making the release (yeah, it happened twice :( ) . I will work so is included in the next firmware release.

rdixey

@abilio These new rtc.ntp_sync() & time.timezone() functions work nicely with the 1.6.4.b1 update. They are very convenient!

I think there is a typo on the Utime doc page though. The first function listed there is utime.mktime([secs]), is this intended to be utime.gmtime()?

Thanks for the good work on this :)

livius

@rdixey said in Syncing RTC To Network Time:

mktime

It is standard in many languages - google for it ;-)

santorcuato

@rdixey said in Syncing RTC To Network Time:

@abilio These new rtc.ntp_sync() & time.timezone() functions work nicely with the 1.6.4.b1 update. They are very convenient!

I think there is a typo on the Utime doc page though. The first function listed there is utime.mktime([secs]), is this intended to be utime.gmtime()?

Thanks for the good work on this :)

Its not working for me :-(

Can you post a working example?

Thanks!

rdixey

@santorcuato I used it as a function in the boot.py of my LoPy. I had to set a delay to allow for the round trip to pool.ntp.org or else the function completed before ntp_sync and RTC remained = Unix Epoch . When I ping pool.ntp.org from my LAN the longest round trip was 630ms so I set sleep at 750ms. Less than that was too short.

def setRTCLocalTime():
    rtc = machine.RTC()
    rtc.ntp_sync("pool.ntp.org")
    utime.sleep_ms(750)
    print('\nRTC Set from NTP to UTC:', rtc.now())
    utime.timezone(-18000)
    print('Adjusted from UTC to EST timezone', utime.localtime(), '\n')

Connecting to a LoPy...
Connected
>>> 
PYB: soft reboot

RTC Set from NTP to UTC: (2017, 2, 26, 13, 18, 54, 225113, None)
Adjusted from UTC to EST timezone (2017, 2, 26, 8, 18, 54, 6, 57)

rdixey

@livius Excuse Me :) I already know what mktime() is. And I don't need anyone to give me instructions on using google to find info. I was pointing out a typo in the doc's.

https://docs.pycom.io/pycom_esp32/library/utime.html

mktime is the first function in the list and is described as 'Convert a time expressed in seconds since the Epoch.' I believe this is an error and should be gmtime().

mktime is listed again as the third functon and described correctly.

gmtime() is not in the list at all.

livius

@rdixey said in Syncing RTC To Network Time:

instructions

Without explanation that function is twice in doc - this was my first hint ;-)

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

Syncing RTC To Network Time

Pycom on Twitter