Create array based on filter

[Marcel1]

Hi,

I have a scenario where the input from a database is a code based on A B C D. It can be different combinations of this code, with a minimum of 2 letters but the maximum can be more than 4 because some codes include different letters but bare no relevance to the scenario. Therefore I use a “contains” filter to make sure that only codes that have a minimum of 2 letters (of which 2 should be from A B C D) are passed through.

However, in the next step I need to form an array of values tied to each different letter (Discord API call to bulk delete messages). My database contains the message ID tied to each letter of my code, however if said letter doesn’t exist it doesn’t have a message ID and thus shouldn’t be included in the array.

So what I need is the following:
A way to create an JSON array “messages”: [“123”,“123”,“etc”] based on the output of the filter. If the code contains B C it should only include those values. I tried making this happen with an IF() statement but it’s impossible to include the “,” because the last value shouldn’t have it.

So at this point it should only take the second and fourth value and create an array with that, that I can use in a Discord API call.

But when I select the array from the array aggregrator it gives me an error like this:

It should instead look like "messages":["10293929302","10238293829"] (couldn’t copy, but the numbers starting with 10 from the array above)

I’d be happy to elaborate further on the issue.

[Marcel1]

Updated with images.

[JugaadiTech]

@Marcel1

Updated with images.

^^

Thank you! Very helpful to have

I will start with assuming you have done a test run, with a simple tester dataset and none of this logic, to delete an array of messages? using the same API call.

If not, do that, makes the rest of this easier to tackle.

---

@Marcel1

A way to create an JSON array “messages”: [“123”,“123”,“etc”] based on the output of the filter. If the code contains B C it should only include those values.

@Marcel1

I tried making this happen with an IF() statement but it’s impossible to include the “,” because the last value shouldn’t have it.

First Question:

Have you successfully created a “list” of all the messages you want deleted, ignoring the required data structure that is expected in the discord API Call Module.

E.G.: Could you output the list to a google sheet, and it would be exactly the messages you want deleted?

If you have, This is purely a json formatting problem, and removing the last “,” should be fairly easy.

If Not: Lets break this down one problem at a time.

Lets get a better overview of the logic to build the “list of messages” and some more screenshots of that part of the scenario flow.

Skipping the Painful Route for now, will revisit if needed after more details

Without more information, this gets a bit more painful.

You would need to Download the json output of the last module before the discord api call,
(including any) IML modifications
IF IML is included in the discord module, please put a “set string variable” inbetween the two, and download that output for me.

This is a pain because you would need to scrub the JSON for personally identifiable information, while keeping it valid, and in an identical structure, and some indication to me of what the data type was of the data that was scrubbed.

If you have the dataset structure payload from the successful test run, that would save me some time as well.

But, lets not start with this

[Marcel1]

Hi, thank you for your elaborate reply.

I have done test runs with the API call and it works as intended. At first I created the API call with an IF() statement basically looking like the following:

Body:

{
   "messages": ["IF({{A-msg-id}} = "";0;{{A-msg-id}})","IF({{B-msg-id}} = "";0;{{B-msg-id}})","IF({{C-msg-id}} = "";0;{{C-msg-id}})","IF({{D-msg-id}} = "";0;{{D-msg-id}})"]
}

This deleted all the messages as intended if A B C D all existed with a valid discord message ID. But as soon as one of the letters did not exist with a valid discord message ID (thus returning an empty cell from the database) it showed an error from the API call.

Discord Developer Portal — API Docs for Bots and Developers

Judging by the API documentation above, I figured returning a 0 or any other invalid number should not matter considering the following: “Any message IDs given that do not exist or are invalid will count towards the minimum and maximum message count (currently 2 and 100 respectively).”

Basically I have a maximum of 4 msg-ids that can exist, and in each trigger of the scenario it can be a combination of those 4 (so sometimes A B, sometimes A C, sometimes A B C D, etc). If it is possible to create an array of the value of each msg-id of that particular trigger, and leaving the empty (non existing) values out of the array instead of including them as a “0” my problem is pretty much solved. I tried playing around with the different JSON modules, but I’m not an expert with these kind of things.

If I could elaborate on some things in particular with pictures, please let me know and I’ll take some screenshots.

[JugaadiTech]

That Assumption is inconsistent with my experience using the delete and react to message discord modules

@Marcel1

Judging by the API documentation above, I figured returning a 0 or any other invalid number should not matter considering the following:

It might be a quirk in the make implementation, or the discord API itself.
I might test later to report back to make if this proves to be only a problem on make.

Lets remove Null/0/invalid array items from the API call next.

with the discord module “delete a message”

It does error out without an existing message ID, error handlers have to be setup to compensate or it will crash the scenario.

I found this out when a user beat my bot, to message deletion.

---

Note though:

last time I Tested This specific quirk was before the last time they had to rebuild to match discords new Bot Compliance Standards, i believe mid/late 2021.
Will Re-test and confirm sometime tomorrow.

[Marcel1]

I can’t use the “delete a message” module, because if all letters are present with valid msg-ids it goes past the discord API rate limit. That’s why I need to use the API call to bulk delete if it’s more than 2 messages, and use the “delete a message” module if it’s one message.

[JugaadiTech]

100%, I updated to my message to make that more clear. my first pass was a little unclear apologies.

What i was talking in relation to is I would be surprised if Bulk-delete and delete were not consistent with each other, in terms of how the api handles bad data.
I was not suggesting to use the basic delete.

Heres what i suggest as the next step:

• running a filter to remove all bad values from the array.
• all strings shorter than the DISCORD message ID Length,
• All strings that contain characters or patterns that are not used in discord message ID’s
• etc.

@Marcel1

it goes past the discord API rate limit

I barely slide under the Discord API rate limits with my scenarios, im about to have to upgrade big sections over to “Make API Call”
finished some of the upgrade earlier this year. but its getting urgent very fast

Not a recomended as a longterm solution, but more a temporary fix till the real solution is built out (clickme if interested)

but an alternative option if you need an urgent short term fix, while doing a longterm one.

---

Placing a “sleep module” between each discord action at 4 seconds, deals with the the majority of discord rate limit errors. This accounts for some other scenarios running discord modules as well in parellel.

The Downside of this

1. Increased Make Operations usage (by a brutal amount)
2. you are limited in actions per hour, scenario would need to auto loop till complete because the make scenario max run duration.
3. if messages come in faster than they can be deleted, or even 70% of the speed at which they can be deleted this solution does not work. you are constantly playing catch up.

[Marcel1]

Fair enough, thanks for the clarification. Filtering out the string shouldn’t be too much of a problem for me, but getting the string in the first place seems to be where the issue is at for me. What module would you recommend getting this string if the cells come from Airtable?

Basically, how do I go from here:


(msg-ids exist for all 4 or for a combination of them depending on the trigger but can’t get them in one screenshot)

to here:

[“100289328932”,“100238792387”]

[Marcel1]

Also, I already utilized the sleep module method as a way to get around discord’s rate limit. But I’m here on the forums now to find my solution for the long term haha. I think API calls is it.

I found out that deleting 3 messages is ok for discord, whereas 4 or more is not. So you don’t have to put sleep modules after every module, just after 3. But still it doesn’t seem very efficient, and indeed eats away at operations. If there is a way to array the message ids and filter out the invalid ones it should solve all my problems (and potentially yours if you are going to keep upgrading your workflow with API calls )

[JugaadiTech]

My case was overlap safety,
if you have other live scenarios running.
4-6 depending on a couple of factors, was my sweetspot to not have all my scenarios crash out with errors.

At 3 seconds, between each “Update Member Role, delete message, or post message” it took about 10 minutes into a run for me to get timed out by discord. and thats been pretty consistent 2019,2020,2021 that 3 causes major errors,
even with business scaling.

having attempted to drop the number down in 2022

[JugaadiTech]

in your current final array right before it gets inputted into the discord module.

Is the “bad data” only 0s,
or is it also messages that have been previously deleted that no longer exist.

@Marcel1

and leaving the empty (non existing) values out of the array instead of including them as a “0” my problem is pretty much solved. I

[Marcel1]

Only 0s, no previously deleted messages.

[JugaadiTech]

Okay, that should be easy then gimme a sec:

[JugaadiTech]

Aight.
I setup a quick Demo with a datastore, and set variable
No need to touch the airtable module.

Theres my sample data, with 0s mixed in.

Scenario Running and clearing out the "0"s

you can do this action inside the discord module itself i expect.
No need to use a tool module inbetween.

[Marcel1]

And how would the output of this cleaned up array translate into looking like ["number","number","number"]

[JugaadiTech]

Ignore the extra json around it, i didnt do passes to remove it, cause my datasource is different than yours,

and you are inserting the array into the api request.

Data Processing Steps

1. With the 0’s Direct Output from Make Datastore

{
   "key": "xxxx",
   "data": {
   	"List of Messages": [
   		"1008853307495817307",
   		"1008854129948491928",
   		"1008857677453070337",
   		"0",
   		"643533111766024232",
   		"644627494367920138",
   		"0",
   		"0",
   		"935014684699013130"
   	]
   }
}

2. 0s Removed with this

{{remove{data.`list`;0)}}

{
	"Array Cleaned": [
		"1008853307495817307",
		"1008854129948491928",
		"1008857677453070337",
		"643533111766024232",
		"644627494367920138",
		"935014684699013130"
	]
}

3. Json minified (same as 2) (you should not have to do this in your scenario)

{"Array Cleaned":["1008853307495817307","1008854129948491928","1008857677453070337","643533111766024232","644627494367920138","935014684699013130"]}

4. Extra Stuff Removed

["1008853307495817307","1008854129948491928","1008857677453070337","643533111766024232","644627494367920138","935014684699013130"]

and now you have an array to add into the batch-delete Api call

Let me know where that lands us

[Marcel1]

Turns out there’s an issue with the discord API when you map the body, but is fixed when you assemble the body.