Gpy AWS IOT Mqtt

[PranithMichael]

Re: UMQTT & AWS IoT

Hi All,

I'm new to Pycom.

I've been trying to understand and set up communication between Gpy and AWS. I followed the guide presented here but could not establish connection between the two.

I get the error 'Could not establish MQTT connection' in my watch window.

For clarity,

1. I am using a Gpy on the expansion board ver 3.0 with the wifi antenna. I've been able to connect to my wifi. I have the correct configurations for the board set up and have pymakr installed on Atom ide.
2. I have created AWS thing, downloaded all the certificates, created and linked policies and certificates to the AWS thing and to each other as well.
3. I would like to mention that the certificates I downloaded from AWS have the following extensions : d82XXXX860-certificate.pem.crt and d82XXXX860-private.pem.key. I tried modifying the extensions to d82XXXX860-certificate.crt and d82XXXX860-private.pem as per the example code but ended up with the same error.
4. During the creation of AWS certificates, all other certificates provide a downloadable file, where as the rootCA certificate download link takes me to this page with 4 options for CA files. I've tried with all the certificates individually but it hasn't helped.

Any pointers will be really appreciated.

[randomrnti]

@randomrnti Figured out my issue was due to the AWS policy only allowing certain clients for the iot:Connect policy. Changing it to the wildcard * fixed my issue.

Another issue though is I can only get it working for Wifi. The moment I attempt this on NB-IoT I'm getting the following errors:

AWSIoTMQTTClient

Socket create error: [Errno -78] MBEDTLS_ERR_NET_SEND_FAILED

simple.py

Traceback (most recent call last):
  File "main.py", line 90, in <module>
  File "main.py", line 79, in run
  File "/flash/lib/simple.py", line 56, in connect
OSError: [Errno 113] ECONNABORTED

This again is on the connect action and I've tried with connecting using both the IP Address and it's domain name.

[randomrnti]

I'm experiencing the same issues. I've checked using both the AWSIoTMQTTClient and the simple.py on the 1.18.2 firmware. For the simple.py solution I found that it was throwing this error:

Traceback (most recent call last):
  File "main.py", line 52, in <module>
  File "main.py", line 32, in run
  File "/flash/lib/simple.py", line 84, in connect
IndexError: bytes index out of range

It looks like it's not getting a response from aws in the simple.py and attempting to read nothing on the socket.

resp = self.sock.read(4)
assert resp[0] == 0x20 and resp[1] == 0x02

I've verified using AWS's python SDK on my computer with the same certificates and mqtt topic. It's working as intended.

[Paul Thornton]

@reidfo said in Gpy AWS IOT Mqtt:

modules:composer.user_said_in, @martinnn, Gpy AWS IOT Mqtt

'file not found' pretty much sounds like... the file is not there?

@PranithMichael Check your Pymakr settings as well. By default it only uploads certain file types to the board.

pymakr.conf

  "sync_file_types": "py,txt,log,json,xml,html,js,css,mpy,pem,cet,crt,key",

This would be my guess to. Give this a try. If your still having issues drop a message to support@pycom.io

[reidfo]

modules:composer.user_said_in, @martinnn, Gpy AWS IOT Mqtt

'file not found' pretty much sounds like... the file is not there?

@PranithMichael Check your Pymakr settings as well. By default it only uploads certain file types to the board.

pymakr.conf

  "sync_file_types": "py,txt,log,json,xml,html,js,css,mpy,pem,cet,crt,key",

[Martinnn]

@pranithmichael I am not a certificate guru, there are different certificate formats. I don't think the name or ending matters. Mine looks like this
-----BEGIN CERTIFICATE-----
MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCB
...hnacRHr2lVz2XTIIM6RUthg/aFzyQkqFOFSDX9HoLPKsEdao7WNq
-----END CERTIFICATE-----
For starters, I didn't use a boot.py but renamed the main_publish.py to main.py.

Are you sure you set the config.py correctly?
AWS_ROOT_CA = '/flash/cert/aws_root.ca'
AWS_CLIENT_CERT = '/flash/cert/xxx-certificate.pem.crt'
AWS_PRIVATE_KEY = '/flash/cert/xxx-private.pem.key'

'file not found' pretty much sounds like... the file is not there?

[PranithMichael]

@martinnn Thank you for your reply.

I did try with the AWS example.
Can you please confirm the extensions of the certificate files that you have used?

AWS_ROOT_CA = '/flash/cert/root-ca.pem'
AWS_CLIENT_CERT = '/flash/cert/ab4XXXXb22-certificate.pem.crt'
AWS_PRIVATE_KEY = '/flash/cert/ab4XXXXb22-private.pem.key'

I included different file upload types in the global settings on the IDE, in order to try and execute the code with .ca, .crt extensions for the rootCA file.

The rootca file that I have used is the RSA 2048 file found here
This file also matches the root CA file that AWS IOT python SDK package generates during the creation of a AWS thing.

In all cases (different extensions for the rootCA file) I get the following error on the window:

Socket create error: CA file not found
Socket send error 128

I cross checked on AWS, the AWS thing is linked with its respective policy and certificates and vice versa.

I invoke main_publish directly from boot.py:

from machine import UART
import machine
import os
uart = UART(0, baudrate=115200)
os.dupterm(uart)
machine.main('main_publish.py')

Im not sure whether this is the right thing to do. Must I compulsorily have a main.py and then invoke methods in main_publish.py?

[reidfo]

@pranithmichael said in Gpy AWS IOT Mqtt:

Just to be sure. The CLIENT_ID is the AWS thing name, AWS_HOST is my host location at xxsomethingxx.iot.us-east-2.amazonaws.com and AWS_PORT is 8883. right?

That is correct.

[Martinnn]

@pranithmichael Did you try the AWS example from Pycom?
https://docs.pycom.io/tutorials/all/aws
After some guesswork which certificate is what and enabling the certificate upload in pymakr, it worked right away (Wipy).

[PranithMichael]

Thank you for having taken the time to reply @reidfo

I downloaded the Python SDK for Linux and OSx that AWS provides while creating a 'aws thing'. There is a start.sh script file that generates a root CA file. It matches the Root CA 1 certificate that you have pointed to.

I set the following of extensions: .crt for certificate file, .key for the private file and .pem for the RootCA file, but didnt have any luck connecting to AWS.

Global settings for the project on the ATOM IDE allows us to mention the type of files that can be uploaded on to the pycom module.
I tried including 'ca' files in order to upload rootCA.ca file but that didn't seem to work either.

Just to be sure. The CLIENT_ID is the AWS thing name, AWS_HOST is my host location at xxsomethingxx.iot.us-east-2.amazonaws.com and AWS_PORT is 8883. right?

All the more confusing to me is the AWS example found in the documentation config.py , where CLIENT_ID and THING_NAME are mentioned as two separate entities.

[PranithMichael]

This post is deleted!

[reidfo]

@pranithmichael I had no luck with the Pycom AWS library. I was finally able to get connected to MQTT using simple.py.

The certificates were a little tricky to figure out (i.e. which root CA cert to use). The certificates should be in PEM format (typically extensions of .crt, .key, and .pem are PEM formatted). The root CA certificate I used was RSA 2048 bit Root CA 1.

Here's my connect function. I hope this helps.

from simple import MQTTClient

def mqttConnect():
    global connection
    global state
    global mqtt
    wdt.feed()
    log('mqttConnect')
    while state != config.CONNECTED:
        try:
            gc.collect()
            state = config.CONNECTING
            log("connecting to mqtt")
            mqtt = MQTTClient(config.CLIENT_ID, server=config.AWS_HOST,
                              port=config.AWS_PORT, keepalive=600, ssl=True,
                              ssl_params={"certfile": config.AWS_CLIENT_CERT,
                                          "keyfile": config.AWS_PRIVATE_KEY,
                                          "ca_certs": config.AWS_ROOT_CA})
            # if mqtt.connect(600):
            mqtt.connect()
            state = config.CONNECTED
            log('MQTT connection established!')

        except Exception as e:
            log(e)
            log('could not establish MQTT connection')
            time.sleep(10)
            continue
    log('end mqttConnect')