Csv encoding problem from google file

Hello

I am parsing a csv that i downloaded from Google (i tried from gmail and from Drive), and i have encoding issues. I have done as @samliew mentioned and use the encoding text module, but i still have problems with special characters.

Here the input of my module


and the output

i am not sure whether it has to do with the input data codepage or not.

Hi @T_Messi ,

I don’t know about the encoding module, but an ugly workaround with array manipulation :

This converts item1,item2,item3,item4,item5,item6,item7 into rows of :
item1 item2 item3 item4
item5 item6 item7

And here’s the json :
split cvs to sheet or array.json (57.2 KB)

Although this consumes (2 + number of rows) operations, it works. You’ll need to adjust column numbers here :

-1 and 3 for 4 columns. (same in filters before ‘get each row’ and ‘add a row sheet’ modules, and in add a row module itself)
image

and here :

Hi!

Thanks for taking the time here :slight_smile:
I’m failing to see how that helps with the encoding problem?

@T_Messi , What is the expected output ?

My scenario gets csv output data and converts it to rows of data that can be exploited (in arrays or copied in google sheet or else).

Now that I look at it better, you want to generate an .csv instead… what’s csv module output when taking in that ‘encoded’ variable?

For now I am using after the encoding module a csv parser, which then does the job for whatever I have to do. My problem is in the encoding of data (I want it in UTF 8 so that my special characters are correctly encoded - as you can see from the ? on the screenshot)

@T_Messi ,

What about just replace that (?) by emptystring?

Here’s my example :

transformation of variables in order:
image
image
image

That would not work as those special characters are actually letters but in German (ü ä or ö) for example
From what I understand from the „convert“ module that should exactly do that but for some reason it is failing to convert correctly…

for those who might have the same issue as me :

I first had to verify the original encoding of the file :
i downloaded it on my MAC and check with terminal command the encoding of the file :
file -I (thats an upper case i ) nameofthefile.csv

and i modified in my encoding module the input data codepage (in my case ISO -8859-1)

and i know have all my german characters correctly displayed

3 Likes