........ 
You get this:
Notice "https://e689-2a02-2f01-7302-2b00-5de2-cd4f-147-d37f.eu.ngrok.io", the unique URL ngrok has created for me on their servers. All I have to do now is to configure this URL as the webhook endpoint in the Discord bot configuration page:
mando.py
However, given we are using a single function, we cannot return until we have generated the image and uploaded it to the storage bucket. If we do that, Discord timeouts and the user receives an error. So, I'm using a POST on the /interactions endpoint to create the first reply, and the do a PATCH later to send the image. Discord docs say you can only do that if you received a message via the API gateway, but seems to also work when you get the message via the webhook. This is not something you should do for a production application ! If it works now, it doesn't mean it will always work. As long the docs say to not do that, just don't do that. That's another reason in favor of the deployment with more than one serverless function.
I guess that's pretty much self explanatory. The first argument ("discord-bot-single") is the name you will find the function in the Google cloud. "--trigger-http" instructs the cloud to deploy this function as a web app and create an HTTP endpoint. "--allow-unauthenticated" makes this endpoint available to everyone (required as the Discord server cannot authenticate).
LE: sometimes the result is this:
This post details the implementation of a highly scalable Discord bot using a single serverless function in GCP. It's part of a series of posts which describes the same project, a highly scalable Discord bot which reacts to the "/mando" command by generating a unique mandala and serving the image back, implemented using multiple approaches. I did this to play and experiment with GCP. Might want to read the introductive post first.
TLDR
- It's quite complicated to deploy a Discord bot using a single serverless function; Discord requires a response to their POST no later than a few seconds, or else they consider the bot is not responsive and displays an error message. However, generating the mandala and uploading it to the bucket takes more than a few seconds. Technically, there should be possible to get the underlying data stream that Flask is using to send data back to the client and manually write the response to Discord, but that's a hack I would not use for production code.
- use gcloud CLI (or even better, infrastructure as code), don't click around in the cloud web interface; it's way more easy to reproduce things later, for you or for other people.
- use Functions Framework + ngrok to develop your code locally so that you don't have to wait for the cloud deploy for every minor code update.
- create a new project, don't reuse existing projects; this way, when you're done experimenting, just delete the project and you're good.
- always create a budget so that you're safe.
This approach uses a single Cloud function which does everything. Receives the HTTP invocation from the Discord server, generates the mandala, saves the images to a bucket and then posts a message via HTTP back to the discord server with the image URL.
First step is to install the gcloud CLI. I always prefer to use CLI commands (or even better, infrastructure as code tools, such as Terraform) as opposed to clicking around in the web interface. Should someone else (that someone can be myself a few weeks later) wants to follow the same steps you did, the easiest is to share the Terraform file with him. Second best is to copy paste CLI commands. Clicks in the web interface are the hardest to share.
Always create a budget
Two important things to consider before deploying stuff to cloud:
1. do not reuse existing projects, create a new project when you experiment with something. This way, when your experiment is complete, you simply delete the project and that's it. All cloud resources are deleted.
2. always create a budget ! Understanding how the billing works for a cloud project is hard. Setting a budget protects you from having to deal with a high unexpected cost.
This is how my budget looks like:
Install gcloud CLI & local testing using ngrok
Next step, follow the steps here to install the gcloud CLI. Then follow the steps here to install the local Python Function Framework. Deploy a function to GCP might take a few minutes. Waiting several minutes just to be able to test your function is no fun. Fortunately, you can run your function inside a local emulator. It can even access cloud resources, like file storage buckets.
Just run below commands and your function will execute locally, so you can test it without having to deploy it to the cloud each time.
pip install functions-frameworkfunctions-framework --target=handle_requestFor more information on how to use the Function Framework, read here.
OK, but we are developing an HTTP endpoint which is going to be invoked by the Discord server, which is running in the internet. How can the Discord server invoke an endpoint deployed locally on my laptop, you might ask. Meet ngrok ! It's a tool you install on your machine which create a tunnel between your laptop and their public server. Once started, ngrok provides you with a public address on their server which is tunneled to locally deployed HTTP endpoint port.
Simply download the ngrok tool on your machine and run the command as below (provided the port for your locally deployed HTTP endpoint is 8080).
./ngrok http 8080The source code
Now, the source code:
main.py
import uuid
import functions_framework
import requests
from flask import abort
from flask import jsonify
from google.cloud import storage
from nacl.exceptions import BadSignatureError
from nacl.signing import VerifyKey
from config import public_discord_key, discord_app_id, bucket_name, local_tmp_folder
from mando import create_mando
def upload_blob(source_file_name, destination_blob_name):
storage_client = storage.Client()
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(destination_blob_name)
blob.upload_from_filename(source_file_name)
def reply_bot_is_thinking(interaction_id, interaction_token):
reply_response = requests.post(
url=f"https://discord.com/api/v8/interactions/{interaction_id}/{interaction_token}/callback",
json={
"type": 5,
"data": {
"tts": False,
"content": "generating, please wait...",
"embeds": [],
"allowed_mentions": {"parse": []},
},
},
)
reply_response.raise_for_status()
def reply_mando_result(interaction_token, mando_filename):
mando_url = f"https://storage.googleapis.com/{bucket_name}/{mando_filename}"
reply_response = requests.patch(
url=f"https://discord.com/api/v8/webhooks/{discord_app_id}/{interaction_token}/messages/@original",
json={
"tts": False,
"content": "image",
"embeds": [
{
"thumbnail": {"url": mando_url},
"image": {"url": mando_url},
}
],
"allowed_mentions": {"parse": []},
},
)
reply_response.raise_for_status()
def validate_request(request):
verify_key = VerifyKey(bytes.fromhex(public_discord_key))
signature = request.headers["X-Signature-Ed25519"]
timestamp = request.headers["X-Signature-Timestamp"]
body = request.data.decode("utf-8")
try:
verify_key.verify(f"{timestamp}{body}".encode(), bytes.fromhex(signature))
except BadSignatureError:
return False
return True
@functions_framework.http
def handle_request(request):
is_valid = validate_request(request)
if not is_valid:
abort(401, "invalid request signature")
request_type = request.json["type"]
if request_type == 1:
return jsonify({"type": 1})
else:
interaction_token = request.json["token"]
interaction_id = request.json["id"]
reply_bot_is_thinking(
interaction_id=interaction_id, interaction_token=interaction_token
)
local_file_name = f"{str(uuid.uuid4())}.png"
create_mando(local_tmp_folder=local_tmp_folder, local_file_name=local_file_name)
upload_blob(f"{local_tmp_folder}/{local_file_name}", local_file_name)
reply_mando_result(
interaction_token=interaction_token,
mando_filename=local_file_name,
)
return {}, 200
import random
import matplotlib.pyplot as plt
from RandomMandala import random_mandala
def random_rgb_color():
r = random.randint(0, 255)
g = random.randint(0, 255)
b = random.randint(0, 255)
return "#{:02x}{:02x}{:02x}".format(r, g, b)
def create_mando(local_tmp_folder, local_file_name):
radius_len = random.randint(5, 20)
radius = []
facecolor = []
radius.append(3)
facecolor.append(random_rgb_color())
for i in range(1, radius_len - 1):
radius.append(radius[i - 1] + random.randint(1, 5))
facecolor.append(random_rgb_color())
fig3 = random_mandala(
radius=radius,
rotational_symmetry_order=random.randint(3, 20),
face_color=facecolor,
connecting_function="fill" if random.random() < 0.5 else "bezier_fill",
symmetric_seed="random",
)
fig3.tight_layout()
plt.savefig(f"{local_tmp_folder}/{local_file_name}", dpi=200)
plt.close(fig3)
The entry point is "handle_request" in main.py. As specified in the Discord docs you need to validate any request by using the nacl Python lib. That's an important step, or Discord will refuse to accept your webhook endpoint URL. Right after validation we reply to the Discord server to let it know the bot received the request and is processing.
The hack
LE: this hack does not work all the time. Which is to be expected.
That's a bit of a hack what I did here. The Discord docs instructs to respond to the original POST call received from the Discord server to generate the first response for the user.
However, given we are using a single function, we cannot return until we have generated the image and uploaded it to the storage bucket. If we do that, Discord timeouts and the user receives an error. So, I'm using a POST on the /interactions endpoint to create the first reply, and the do a PATCH later to send the image. Discord docs say you can only do that if you received a message via the API gateway, but seems to also work when you get the message via the webhook. This is not something you should do for a production application ! If it works now, it doesn't mean it will always work. As long the docs say to not do that, just don't do that. That's another reason in favor of the deployment with more than one serverless function.
LE: as expected, this hackish approach does not work all the time. At the moment of writing this article, it worked. One day later, it does not work anymore.
Creating the resources and deploying the code
As I've mentioned below, I'm a fan of CLIs or infrastructure as code as it's easily replicable. For this project, I've used the gcloud CLI.
Creating the bucket to store the mandala images:
gcloud storage buckets create gs://mando_bucket_1234
gsutil defacl ch -u AllUsers:R gs://mando_bucket_1234
gsutil acl ch -u AllUsers:R gs://mando_bucket_1234/**
The first command creates the bucket. Second one allows everybody to access the files stored in that bucket, even if the user is not authenticated, This is necessary for the files to be accessible to the Discord client. The third command is not really necessary. It changes the permission of all files already stored in the bucket to be accessible by everyone. As you have just created the bucket, you don't need this. However, should you have created the bucket and not added the "AllUsers" permissions right after creation, you might need this, so I've added it here anyway.
Deploying the function:
gcloud functions deploy discord-bot-single \
--runtime=python39 \
--region=europe-west6 \
--source=. --entry-point=handle_request \
--trigger-http \
--allow-unauthenticatedThe result
This is to be expected as we are not following what the Discord docs say. Although technically this can probably be fixed by getting access to the underlying data stream Flask is using to send data to the client and send the response manually (from my Python code) before starting on the image processing, that's still a hack. We'll fix this the proper way by using two functions and a PUBSUB topic in the next post.
Comments
Post a Comment