Hi,
If somebody comes across the same challenge as me, where a CSV file contains the same headers several times (e.g. per page, or if you combined several CSV files into one, or in my case with Elvaco energy meters, per device), there’s an easy way to remove those duplicate headers. Other examples on the forums require you to create a temporary data store and individually compare all records, which will be very operations-consuming.
This solution is fairly easy:
Steps:
- Retrieve the CSV file (e.g. from a web or mail hook)
- Parse the CSV file. Set that you do not have any headers.
- Aggregate into an array (see screenshot below)
- Deduplicate the array, which will remove all lines (except the first) that are completely identical (= the header rows)
- The result will be an array with the headers on the first line, and then all value rows without any duplicate headers
Module 3 configuration:
Module 4 configuration:
Good luck! 