Hey @vstadmin if you want to share where you’re getting hung up on the AWS side, I can provide some additional guidance, but the AWS pieces here were just meant as an example to provide an end-to-end picture of what a solution would look like from the cloud to the Notecard and back. You absolutely could place the file in any location of your choosing and fire an inbound Note to the Notecard. Much of the process on the Pi and Python side with the Notecard would be the same, though you may need to adapt things slightly for your use.
Ah, good question! Those represent environment variables that I set within lambda. More info here: Using Lambda environment variables - AWS Lambda
Were those environment variables in your notes anywhere?
i’m also getting an error in the lambda:
[ERROR] KeyError: 'SESSION_TOKEN'
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 18, in lambda_handler
headers = {"X-SESSION-TOKEN": os.environ['SESSION_TOKEN']}
File "/var/lang/lib/python3.7/os.py", line 681, in __getitem__
raise KeyError(key) from None
I have no idea what the SESSION_TOKEN should be.
That session token is also an environment variable, and you need it for accessing the Notehub API. Check out this doc for more info.
I have sketched out the plan. This is my main project for today and tomorrow (if needed). I’ll update you on progress at C.O.B.
Made good progress. Got the project set up and extracted/tested the core code for encoding the file transfer.
Next step is to set up Apache example.
I’ll follow up tomorrow. Given time frame, I may push an example to you before merging to main in the repo.
Update:
Figured out hosting python scripts via Apache2. Have basic server in place. Working on gluing the 2 elements together.
Update: I have an implementation that replies with file chunks served from apache2 on my local machine using Python scripts
I will work on packaging that up and getting a basic doc together for getting started over to you
Looking forward to it! I have a project meeting on Friday am, where I would love to show off the feature.
Thanks!
I have a first pass that I think is ready for you to try. This is just the File Server. It doesn’t do notification to the Notecard as yet. Also, the Notecard existing edge application example is compatible with this implementation of the file server. We are just swapping AWS Lambda for an Apache server.
Attached as zipped TAR file
fileServer.zip (8.9 KB)
which I believe should be extractable with:
tar xvzf file.tar.gz
This does contain a README.md which should help with the setup and configuration.
I’ll work with our teams to update the git repo once we have the kinks ironed out.
Thank you.
I and a couple of teammates are poring over it closely.
I’m not even sure that the apache web server is required for testing. The php that I currently talk to does get called via the webserver, but this fileServerApp.py is opening its own port when executed. Flask should allow it to respond to requests without the need of apache or am I barking up the wrong tree?
Thanks,
Todd
For testing purposes, Flask has it’s own server.
You can just do
$ python fileServerApp.py
to start the test/debug server
For deployment, Flask doesn’t recommend using the built-in server.
You can see deployment options here:
https://flask.palletsprojects.com/en/2.0.x/deploying/
Does that help?
Good morning…
Messing with some preliminary code. I am trying the most simple test I can think of. I put the fileServerApp.py on a local linux server, placed a test.py file in the assets folder and started up flask manually on port 5000.
What you suggested is exactly what Im attempting to do. having firewall issues at the moment, trying to create a NAT to the outside world. I’ll circle back when I have a chance to test this setup. I “think” it will work, but I can’t use the Blues routes with a local IP, so I have to get the server exposed first.
I set up the server @ Digital Ocean and flask is setting there, responding to commands.
I have this function from the test application:
def get_sync_status():
req = {"req": "hub.sync.status"}
rsp = card.Transaction(req)
print(rsp)
if "err" in rsp:
return "No status available"
else:
return rsp["status"]
Im getting the following error from the test application:
*************
Notecard File Downloader Sample version v2021.04-07
*************
Connecting to Notecard...
Configuring Product: com.vsthose.admin:vstcp2...
Configuring ATTN...
Source file 'test.py' has been updated remotely. Downloading...
{'time': 1623421374, 'completed': 1978}
Traceback (most recent call last):
File "./application.py", line 199, in <module>
status = get_sync_status()
File "./application.py", line 98, in get_sync_status
return rsp["status"]
KeyError: 'status'
I am printing rsp and it only the keys: {‘time’: 1623421374, ‘completed’: 1978}
there is no key “status”
If I comment out get_sync_status() and push past it, I get the following error:
*************
Notecard File Downloader Sample version v2021.04-07
*************
Connecting to Notecard...
Configuring Product: com.vsthose.admin:vstcp2...
Configuring ATTN...
Source file 'test.py' has been updated remotely. Downloading...
Requesting first chunk...
Traceback (most recent call last):
File "./application.py", line 205, in <module>
file_contents = get_file_from_remote(file_to_update)
File "./application.py", line 144, in get_file_from_remote
if "err" in file_chunk["body"]:
TypeError: 'NoneType' object is not subscriptable
When I connect to the fileServerApp.py via curl, I can see a response, but when I use the application from the github (modified to remove the ATTN), I don’t any evidence on the server that its getting a connection from the application. I have the route “remoteFileDownloader” created and enabled
and I just printed the response from:
def web_get_chunk(file_to_update, chunk=1):
req = {"req": "web.get"}
req["route"] = "remoteFileDownloader"
req["name"] = f"?file={file_to_update}&chunk={chunk}"
try:
rsp = card.Transaction(req)
print(rsp)
and I see the error:
Requesting first chunk...
{'body': {'err': "notehub route with alias 'remoteFileDownloader' cannot be found"}}
Im not sure why it says that route cannot be found when its clearly there.
I should have pointed that out before
This guide should help: Web Transactions - Blues Developers
Set the Route Alias to remoteFileDownloader
ok, that part solved…
Now I get:
{'err': "hub request error: invalid character '<' looking for beginning of value"}
{'err': "hub request error: invalid character '<' looking for beginning of value"}
{'err': "hub request error: invalid character '<' looking for beginning of value"}
{'err': "hub request error: invalid character '<' looking for beginning of value"}
{'err': "hub request error: invalid character '<' looking for beginning of value"}
{'err': "hub request error: invalid character '<' looking for beginning of value"}
{'err': "hub request error: invalid character '<' looking for beginning of value"}
a few dozen times, followed by:
{'err': "hub request error: invalid character 'h' after object key:value pair"}
{'err': "hub request error: invalid character 'h' after object key:value pair"}
{'err': "hub request error: invalid character 'h' after object key:value pair"}
{'err': "hub request error: invalid character 'h' after object key:value pair"}
{'err': "hub request error: invalid character 'h' after object key:value pair"}
{'err': "hub request error: invalid character 'h' after object key:value pair"}
a few more dozen times until:
{'err': 'cannot allocate 2668 bytes {memory}'}
{'err': 'cannot allocate 2668 bytes {memory}'}
{'err': 'cannot allocate 2668 bytes {memory}'}
{'err': 'cannot allocate 2668 bytes {memory}'}
{'err': 'cannot allocate 2668 bytes {memory}'}
{'err': 'cannot allocate 2668 bytes {memory}'}
at which point the process froze and I have to Ctrl-Z it and kill it.
Any ideas on the above? Those errors are a show stopper.