Upwork RSS with chtgpt-4 integrations with Airtable

[Afzal]

Goal:

I aim to streamline the process of retrieving recent job feeds from Upwork RSS, analyzing them with OpenAI’s ChatGPT for icebreaker sentences, and then storing the results in Google Sheets efficiently. I want to optimize iterations and ensure proper formatting for Sheets.

Steps taken:

RSS Module Configuration:

1. Configured an RSS module to fetch recent job feeds from Upwork.

Aggregator Configuration:

2. Utilized an Aggregator tool to combine all bundles into a single array.

Iterator Configuration:

3. Employed an Iterator to spread out all array elements for processing by OpenAI.

OpenAI Configuration:

4. Integrated OpenAI to generate icebreaker sentences for each job description.

Setting multiple variables

5. Set multiple variables to be used in the Text Aggregator without functional operations.

Text aggregator

6. Used a Text Aggregator to consolidate all separate variables into one text format.

Sheets Module Configuration:

7. Utilized the Sheets module to make API calls and store the processed values into Sheets.

Screenshots:

Issues:

Encountering problems with reducing the number of iterations of operations, invalid JSON format when attempting to put all values into Sheets at once. Assistance needed in resolving this issue.

[Afzal]

Goal

Now further, I’m facing difficulties implementing pagination for fetching RSS feeds from Upwork. I need to fetch job feeds with pagination to ensure I get a complete dataset for analysis using ChatGPT and storing in Google Sheets efficiently.

Further Issue:

The primary issue lies in the inability to obtain the total number of records returned from the RSS feed endpoint. Additionally, setting the total number of records manually doesn’t work as expected without proper pagination. This poses a significant challenge in fetching a comprehensive dataset for analysis and storage in Google Sheets. Any guidance or suggestions on resolving this issue would be highly appreciated.

Screenshots:

[NolaDigital]

To bulk insert data into Sheets via API, you need to configure your text aggregator like this:

["iceBreaker","category","minbudget","maxbudget","URL"]

Make sure everything is properly quoted, as it looks from your screenshot that your icebreaker isn’t quoted.

And then wrap the output inside JSON in your sheets module:

[Afzal]

Hi @NolaDigital
Icebreaker is not qouted because it’s already in the quoted format from response filter.