A Better Solution for "How to group bundles in X amount of bundles" (Chunking Arrays)

[samliew]

This is a tutorial for slicing an array into multiple operations.

This is usually needed for external API calls that has a maximum array limit, so we need to split into a smaller batch size (also known as “chunk”, or “chunking”) per operation.

Original Solution

The original solution suggested using three modules with a complex filter:

This uses 2n + 1 operations, n being the number of chunks needed. For example, if we have 1000 items, and we want to output arrays sizes of 100 per operation (10 operations), we require 2*10 + 1 = 21 operations.

The Slice Function

Did you know that you can use the built-in slice function to get selected sub-items from an array?

The slice tooltip states:

Proposed Solution

The new solution only requires 1 operation (a single operation) no matter how many chunks we need:

This is because instead of using an Iterator-Aggregator pair to group the chunks, we can directly use the slice function in your “Make an API Call” module itself!

For example, if we have 1000 items, and we want to output arrays sizes of 100 per operation (10 operations), we require 1 operation (just the repeater module) to generate the 10 bundles (operations).

samliew – request private consultation

Join the Make Fans Discord server to chat with other makers!

[samliew]

Set Up

How to set up the Repeater module to do this?

You can directly calculate the number of chunks needed in the Repeater module. The trick is the “Step” field, to increment the i variable based on the output array (single chunk) size.

In this example, we are using a chunk size of 4.

1. Since the slice function begins from index 0 (first item of the array), the Initial value must be set to 0.

2. To calculate how many Repeats (output operations) we need to chunk them into the array size limit, we use the length function to get the total amount of items in the original array, and divide by the chunk size.

3. Then, the Step field will simply be just the chunk size.

samliew – request private consultation

Join the Make Fans Discord server to chat with other makers!

[Hey_Rebels]

Amazing thank you @samliew

[A_Kettaneh]

Thanks for this showcase that is really clear. I’ve used it to chunk an array that is created by an aggregator.
I realized also that in the case of an array aggregator, the grouping capability is also a great way top manage trunks using :

• {{round(1.__ROW_NUMBER__ / 67.chunkSize + 0.5)}}
  where 1.__ROW_NUMBER__ is the row number returned for each array aggregatore collection
• 67.chunkSize is a variable to set the number of items in each chunk

[samliew]

I have created a module in my private custom app that does these three modules in a single operation.

For more information and installation instructions, see How to send multiple inline keyboard buttons in a single Telegram Bot message with dynamic data?!

— @samliew