cat m1 overheads



  • I decided to have a look at lte cat m1 data overheads. First thing I did was to change from https to http & the mean packet size dropped from 7.3 to 6.1 kb, so dump all that encryption & you use less data, makes sense.

    Then I had a peek at some individual server uploads;
    to send {'12': [0, 'TAC8313 twr528221 CID135224587 earfcn9410 ss-72 cinr10.7', 1600473753], '4': [0, 'c1 New Version: 131, release 1.20.1.r1', 1600473746], '6': [102, '2.28', 1600473747], '10': [100, 10.7, 1600473752], '8': [103, '0.0', 1600473747], '0': [100, 11.3, 1600472619], '2': [0, ' ta da! FZtank site is up, ver 131', 1600473746]} used 6.08kb
    to send {'0': [102, '2.27', 1600472609], '4': [100, 9.8, 1600472620], '2': [103, '0.0', 1600472609]} used 6.31kb
    Such poor correlation between characters sent & data used suggests most of it must be overheads?



  • Hi,

    Thanks for the reminder
    Here, we use this to test the LTE connection main.py: (The LTE callback does not work in 100% of the cases)

    from network import LTE
    import time
    from http_get import http_get
    import sys
    import machine
    def cb_handler(arg):
        print("CB: LTE Coverage lost")
        s = 120 # wait a while just to confirm that this wasn't a false positive and we truely lost connection
        print("CB: sleep", s)
        time.sleep(s)
        print("CB: deinit")
        lte.deinit()
        print("CB: reset")
        machine.reset()
        # print("arg", type(arg))
        # print("arg", arg)
        # sys.exit()
    
    print("init")
    lte = LTE()
    lte.lte_callback(LTE.EVENT_COVERAGE_LOSS, cb_handler)
    print("attaching..",end='')
    lte.attach(band=20, apn='simpoint.m2m')
    for t in range(30):
        if lte.isattached():
            break
        time.sleep(1)
        print(".",end='')
    if lte.isattached():
        print("attached")
    else:
        print("Could not attach ... resetting")
        machine.reset()
    lte.connect()
    print("connecting [##", end='')
    while not lte.isconnected():
        time.sleep(1)
        print("#",end='')
    print("] connected")
    
    
    for i in range(20):
        print("[ {} ]".format(time.ticks_ms()),end=' ')
        print(http_get())
        s = 2 ** i
        print(i, s, s/60, s/3600)
        time.sleep(s)
    
    print("[ {} ]".format(time.ticks_ms()), " done")
    

    With http_get() (and note that this is http only):

    import socket
    import time
    def_url = 'http://detectportal.firefox.com/'
    def http_get(url = def_url):
        time_start = time.time()
        dur_send = 0
        dur_recv = 0
        dur_total = 0
        bps_send = 0
        bps_recv = 0
        len_recv = 0
        cnt_recv = 0
        success = False
        try:
            print("http_get(", url, ")")
            _, _, host, path = url.split('/', 3)
            print('getaddrinfo')
            ip_port = socket.getaddrinfo(host, 80)[0][-1]
            print(ip_port)
            print('socket')
            s = socket.socket()
            print('connect')
            s.connect(ip_port)
            request = bytes('GET /%s HTTP/1.0\r\nHost: %s\r\n\r\n' % (path, host), 'utf8')
            time_send = time.time()
            print('send request')
            s.send(request)
            dur_send = time.time() - time_send
            if dur_send:
                bps_send = len(request) / dur_send
            print('recv response')
            time_recv = time.time()
            s.settimeout(60)
            while True:
                data = s.recv(100)
                if data:
                    cnt_recv += 1
                    len_recv += len(data)
                    dur_recv = time.time() - time_recv
                    if dur_recv:
                        bps_recv = len_recv / dur_recv
                    if cnt_recv % 1000 == 0:
                        print("received[", time.time(), "]:", len_recv, bps_recv)
                    success = True
                    # print(str(data, 'utf8'), end='')
                    pass
                else:
                    break
            print("recv: done")
            print("close")
            s.close()
        except Exception as e:
            print("http_get failed:", e)
        dur_total = time.time() - time_start
        return (success, dur_send, dur_recv, dur_total, len_recv, bps_send, bps_recv)
    # def http_gets(url = def_url, count=1):
    #     for c in range(0, count):
    #         try:
    #             http_get(url)
    #             print("success[", c, "]")
    #             # break
    #         except Exception as e:
    #             print("failure[", c, "]:", e)
    def http_gets(url = def_url, count=1):
        for c in range(0, count):
            print(c, http_get(url))
    if __name__ == "__main__":
        # http_get()
        # http_get('http://micropython.org/ks/test.html')
        http_gets('http://detectportal.firefox.com/')
        # https_gets('http://pycom.io')
        # http_get('http://ftp.snt.utwente.nl/pub/docs/rfc/rfc1498.json') # 1000 B
        # http_get.http_gets('http://ftp.snt.utwente.nl/pub/docs/rfc/rfc1535.html') #   10 KiB
        # http_get('http://ftp.snt.utwente.nl/pub/docs/rfc/rfc753.html')  #  100 KiB
        # http_get('http://ftp.snt.utwente.nl/pub/test/1M')               #  977 KiB
        # http_get('http://ftp.snt.utwente.nl/pub/test/100M')
        # http_get.http_gets('http://ftp.snt.utwente.nl/pub/test/10M')
    

    Atm I have no simcard on hand to properly test it, but I made no changes since last time I used it

    Gijs



  • @kjm Thank you for the information. I really want to know, what the overhead is related too..... I would love to use the example of Gijs to see, if this huge overhead isn't the standard.



  • @SciWax I'm using Telstra in Australia. I've experimented with hologram sims but the http/https overheads burn through a 1Mb/month hologram sim in a single day (10 min uploads).



  • @kjm Now I'm pretty sure, that I have the same problem. I'm sending around 100 bytes every minute, but around 6.1 KB get spent during a single transmission, which doesn't make alot of sense to me. I'm using an 1nce SIM card and the cat m1 network of the telekom in Germany. Which provider for cat m1 do you use?

    Hello @Gijs ,
    could you find the example already?

    kind regards,

    Sci



  • @kjm I think I can see that too actually. At the moment, I'm sending around 135 bytes of data per minute and I didnt even do it this whole month (Maybe I've experimented for 10 - 15 days in September with data transmissions every minute).... I already used 117 Megabytes of data according to 1nce for this month. It doesn't make sense to me, because I only send such a small amount of data. Granted, I've kept the Cat m1 connection between transmission, because I'm sending every minute anyways. I will investigate this issue more, sending 3 minute averages of measurements every 3 minutes and closing the cat m1 connection between my transmit intervals. I'm also using urequests for POST-requests and check, if I get a 200 back to see if the data transmission were successfull.



  • Hi,

    For my personal tests using LTE (If I remember correctly, it was Cat M1), we were able to make http requests to the mozilla firefox uptime server for only 600 bytes a piece. Im not sure what happens on your downlink, maybe that is where the data is spent (the firefox uptime server only prints "OK" or the like back). Now I remember the sim-provider we used had a smallest delimiter of 1kb per connection.
    I wanted to comment this on your other thread, but thought the difference would be found in https.
    It is quite a lot of over head (6kB) tha cannot be attributed to the message directly.. I dont have the example we used here right now, but I could probably post it coming Monday!
    Gijs


Log in to reply
 

Pycom on Twitter