A Better Solution for "How to group bundles in X amount of bundles"

samliew · July 29, 2024, 4:21am

This is a tutorial for slicing an array into multiple operations.

This is usually needed for external API calls that has a maximum array limit, so we need to split into a smaller batch size (also known as “chunk”, or “chunking”) per operation.

Original Solution

The original solution suggested using three modules with a complex filter:

This uses 2n + 1 operations, n being the number of chunks needed. For example, if we have 1000 items, and we want to output arrays sizes of 100 per operation (10 operations), we require 2*10 + 1 = 21 operations.

The Slice Function

Did you know that you can use the built-in slice function to get selected sub-items from an array?

The slice tooltip states:

Screenshot_2024-07-29_120710

Proposed Solution

The new solution only requires 1 operation (a single operation) no matter how many chunks we need:

Screenshot_2024-07-29_120741

This is because instead of using an Iterator-Aggregator pair to group the chunks, we can directly use the slice function in your “Make an API Call” module itself!

For example, if we have 1000 items, and we want to output arrays sizes of 100 per operation (10 operations), we require 1 operation (just the repeater module) to generate the 10 bundles (operations).

samliew – request private consultation

Join the Make Fans Discord server to chat with other makers!

samliew · July 29, 2024, 4:33am

Set Up

How to set up the Repeater module to do this?

You can directly calculate the number of chunks needed in the Repeater module. The trick is the “Step” field, to increment the i variable based on the output array (single chunk) size.

In this example, we are using a chunk size of 4.

Since the slice function begins from index 0 (first item of the array), the Initial value must be set to 0.
To calculate how many Repeats (output operations) we need to chunk them into the array size limit, we use the length function to get the total amount of items in the original array, and divide by the chunk size.
Then, the Step field will simply be just the chunk size.