I have a scenario when I’m getting a XML file from certain URL with real estate properties. I then want to parse this XML so that I can iterate over properties and add them to Webflow CMS.
I’ve already setup a scenario with a test XML file (9 properties) and everything was working fine. However, when I tried to use it with a production XML file that has over 800 properties, I’ve run into the following error in the Parse XML module:
Data too long. The log content was cut. The size of the original buffer was 15519678
The file has around 15 MB, but I’m on the core plan so this shouldn’t be an issue as it is within the plan limit of 100 MB. How to overcome this issue?
This isn’t actually an issue, but rather a UI constraint to prevent the loading of large files in the editor, which could potentially freeze your browser.
Based on the screenshots, it appears the XML was parsed successfully. Are there any other challenges you are encountering?
you are right, when I added an error handler the scenario processed further. However, the problem really lies in something different. For large file like mine (15 MB), the XML parser module doesn’t parse the XML correctly as it gives a long string in the ouput. Whereas for a smaller file with the same structure (under 1MB) the XML parser correctly parses the input, so I can turn in into an array in the later modules.
It’s best to show it on screenshots:
Large file (15MB) - XML parser gives a Long String in the output
I just wanna quickly step in to say thanks a lot for sharing what you learned from the support team with us! This is incredibly valuable and has the potential to help many others down the road.
As long as you’re ok with sharing it here (meaning that there’s no personal data/info displayed), we’re ok with it as well
Finally, just FIY: I marked your last comment as a solution in order to make it easier for users in the future who might be looking for similar information.
Just a short overview for all users – the only problem with long XML is that you cannot map elements after loading such a long file. So prepare short version of the same XML, do all the mapping in other modules and when everything is ready, switch the short example of XML to the final long file. After that everything works without any problems.
Once again, thank you very much for the video a pointing us to the right solution.
