nmelone/POCloud-Email-Reports
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Completes #1.

Browse Source 
Updates README-local.md to include instructions and configuration for running locally
This commit is contained in:
Patrick McDonagh
2018-04-05 18:07:13 -05:00
parent a462070cf9
commit 74019e4889
2 changed files with 80 additions and 79 deletions
Download Patch File Download Diff File Expand all files Collapse all files

150
README-local.md
View File

@@ -5,91 +5,95 @@ Developed by Patrick McDonagh @patrickjmcd, Henry pump
## Setup
System variables must be set up for the script to run. Add the following lines to /etc/environment
```
SMTP_EMAIL="<yourSMTPemailAddress>"
SMTP_PASSWORD="<yourSMTPpassword>"
```Shell
MESHIFY_USERNAME="<yourMeshifyUsername>"
MESHIFY_PASSWORD="<yourMeshifyPassword>"
```
Create a "files" in the script's directory.
### Setup a Virtual Environment and Install Python Modules
### Install Python Modules
```
pip install xlsxwriter requests tzlocal
```Shell
python3 -m venv env # Creates the virtual environment in a folder ./env
source env/bin/activate # activates the virtual environment
pip install xlsxwriter requests tzlocal meshify # installs the python modules
```
## Configuration Files
### Preparing an S3 Bucket
The script relies heavily on configuration files based on the Meshify devicetype. To configure a device, create a file
named <devicetype>_channels.json file. The file should hold a JSON list.
This section will show you how to configure the S3 Bucket within AWS. It assumes a strong knowledge of AWS platforms.
### Example Configuration File
1. Sign in to your AWS Console and open the S3 dashboard.
2. Create a bucket named "pocloud-email-reports". You may choose to name your bucket differently, but you must update the variable BUCKET_NAME within reports_s3_xlsx.py
3. Open the newly-created bucket and create 3 folders. These names cannot be changed without doing some serious hacking of the reports_s3_xlsx.py file.
- channel_config
- created_reports
- to_files
```
# testdevice_channels.json
### Populating Channel Configs
[
Populating channel config files tells the Lambda function which devices to prepare reports for and which channels to include data from. **Devices will not be recognized unless they have both a Channel Config file and a To file.**
1. Prepare a file named devicetype_channels.json where "devicetype" is the Meshify name for the devicetype.
```touch <devicetype>_channels.json```
2. In the text editor of your choice, develop a JSON **list of objects** that contains properties "meshify_name" and "vanity_name".
```JSON
[
{
"meshify_name": "<channel name in meshify>",
"vanity_name": "<vanity name for report header>"
},
{
"meshify_name": "<another channel name in meshify>",
"vanity_name": "<another vanity name for report header>"
},
]
```
3. Upload this file to the "channel_config" folder in the S3 Bucket.
### Populating To Files
Populating To files tells the Lambda function which devices to prepare reports for and whom to send the reports for each company. **Devices will not be recognized unless they have both a Channel Config file and a To file.**
1. Prepare a file named devicetype_to.json where "devicetype" is the Meshify name for the devicetype.
```touch <devicetype>_to.json```
2. In the text editor of your choice, develop a JSON **object** that contains properties of the format below. CompanyA and CompanyB should be replaced by the full name of the company as recorded in Meshify.
```JSON
{
"meshify_name": "yesterday_volume",
"vanity_name": "Yesteday Volume"
},
{
"meshify_name": "volume_flow",
"vanity_name": "Flow Rate"
},
...
]
"CompanyA": [
"person@email.com",
"place@email.com"
],
"CompanyB": [
"person@email.com",
"thing@email.com"
]
}
```
3. Upload this file to the "to_files" folder in the S3 Bucket.
## Running the Script
As long as everything has been setup correctly, all it takes is running:
```Shell
python local_test.py
```
## Recipients File
This performs a "dry-run" where it will generate the reports and print to the console the list of recipients, but it will not actually send the reports. The script can trigger the sending of reports by running:
In order to send emails containing the reports, configure a recipients json file named <devicetype>_to.json. The
file should hold a JSON object.
### Example Recipients File
```
# testdevice_to.json
{
"Company 1 Name": [
"email1@company.com",
"email2@company.com"
],
"Company 2 Name": [
"email3@company2.com",
"email4@company2.com"
],
...
}
```
## Running the script
```
usage: reports_xlsx.py [-h] [-s] [-p CONFIG_PATH] [-o OUTPUT_PATH] deviceType
positional arguments:
deviceType Meshify device type
optional arguments:
-h, --help show this help message and exit
-s, --send Send emails to everyone in the _to.json file
-c CONFIG_PATH, --config-path CONFIG_PATH
The folder path that holds the configuration files
-o OUTPUT_PATH, --output-path OUTPUT_PATH
The folder path that holds the output files
```
## Configuring the script to be run via crontab
Open the crontab file with `crontab -e`.
Add the following contents:
```
00 07 * * * /usr/bin/python3 /home/ubuntu/POCloud-Scraper/reports_xlsx.py advvfdipp --send --config-path /home/ubuntu/POCloud-Scraper --output-path /home/ubuntu/POCloud-Scraper/files
01 07 * * * /usr/bin/python3 /home/ubuntu/POCloud-Scraper/reports_xlsx.py ipp --send --config-path /home/ubuntu/POCloud-Scraper --output-path /home/ubuntu/POCloud-Scraper/files
02 07 * * * /usr/bin/python3 /home/ubuntu/POCloud-Scraper/reports_xlsx.py abbflow --send --config-path /home/ubuntu/POCloud-Scraper --output-path /home/ubuntu/POCloud-Scraper/files
```
```Shell
python local_test.py --send
```

9
README.md
View File

@@ -8,7 +8,7 @@ Reports will be generated on a schedule by AWS Lambda, a serverless, event-drive
If you would like to run the reports locally without the AWS Lambda Function, refer to [README-local.md](README-local.md)
## Setting it up yourself!
## Setting it up yourself
### Prerequisites
@@ -31,7 +31,7 @@ This section will show you how to configure the S3 Bucket within AWS. It assumes
Populating channel config files tells the Lambda function which devices to prepare reports for and which channels to include data from. **Devices will not be recognized unless they have both a Channel Config file and a To file.**
1. Prepare a file named <devicetype>_channels.json where <devicetype> is the Meshify name for the devicetype.
1. Prepare a file named devicetype_channels.json where "devicetype" is the Meshify name for the devicetype.
```touch <devicetype>_channels.json```
@@ -60,7 +60,7 @@ Populating channel config files tells the Lambda function which devices to prepa
Populating To files tells the Lambda function which devices to prepare reports for and whom to send the reports for each company. **Devices will not be recognized unless they have both a Channel Config file and a To file.**
1. Prepare a file named <devicetype>_to.json where <devicetype> is the Meshify name for the devicetype.
1. Prepare a file named devicetype_to.json where "devicetype" is the Meshify name for the devicetype.
```touch <devicetype>_to.json```
@@ -81,7 +81,6 @@ Populating To files tells the Lambda function which devices to prepare reports f
3. Upload this file to the "to_files" folder in the S3 Bucket.
### Preparing the Lambda function
1. Clone this repository and open it
@@ -118,7 +117,6 @@ Populating To files tells the Lambda function which devices to prepare reports f
You should now have a file named lambda.zip in the main directory of the repo. This is the file to upload into your Lambda function.
### Creating the Lambda Function in AWS
This section will show you how to configure the Lambda function within AWS. It assumes a strong knowledge of AWS platforms.
@@ -147,7 +145,6 @@ This section will show you how to configure the Lambda function within AWS. It a
This will schedule the event to be triggered at 12:00 PM GMT (7:00 AM CST) every day of the week.
9. Save and test your function.
## Contributors
- [Patrick McDonagh](@patrickjmcd) - Owner