How to interact with website/platform if api is unaccessible?

How To
1

Hi,
I’m writing because I want to renew my ads on an classified ads platform (eBay Kleinanzeigen) automatically. Right now, I log in manually and use a Tampermonkey script to repost them, but I’d like to automate this process with Make. However, I think can’t use the API (https://api.ebay-kleinanzeigen.de/api/) for this. From what I’ve read, the API is only available to certain Pro accounts, and I don’t have one.

So, my question is: can I automate this with a headless web scraper that logs in and navigates the site, instead of using the API? If so, what’s the best tool for web scraping? Does Make support web scraping, or is there an alternative? I’ve dabbled a bit in scraping before, but sometimes it doesn’t work consistently.

I’m just not sure what’s the best approach here.

2

Welcome to the Make community!

Yes it might be possible. See How to get information from a platform that doesn't send APIs and has no webhooks - #2 by samliew

However, as each website implements login differently, there is no single way of doing this, and the process (modules) needs to be specifically tailored to the site.

So you basically need to “visit” the site yourself to get the content. This is called Web Scraping.

Web Scraping

For web scraping, a service you can use is ScrapeNinja to get content from the page.

ScrapeNinja allows you to use jQuery-like selectors to extract content from elements by using an extractor function. ScrapeNinja also can run the page in a real web-browser, loading all the content and running the page load scripts so it closely simulates what you see, as opposed to just the raw page HTML fetched from the HTTP module.

If you want an example, take a look at Grab data from page and url - #5 by samliew

AI-powered “easier” method

You can also use AI-powered web scraping tools like Dumpling AI.

This is probably the easiest and quickest way to set-up, because all you need to do is to describe the content that you want, instead of inspecting the element to create selectors, or having to come up with regular expression patterns.

The plus-side of this is that such services combine BOTH fetching and extracting of the data in a single module (saving operations), and doing away with the lengthy setup from the other methods.

For more information on the different methods of web scraping, see Overview of Different Web Scraping Techniques in Make 🌐

Hope this helps! Let me know if there are any further questions or issues.

— @samliew

P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.

3

Hi samliew,
I cant write you a pm so I will ask here. I wanted to ask how much you charge for setting up some api integrations (not the complicated stuff from this thread) for social media (reddit etc.)? My goal is that AI agents running on my computer are able to access new messages that I received and then are able to also make posts themselves. I just dont have the time to figure it out myself. And if I have a basic structure already I think it will be much easier. With my first integration I am fiddling now already for weeks because the formatting of the date value is not working.

4

Did you know, this forum has a Hire a Pro category, where you can post your request for off-site specialised help on other platforms (video call/screenshare/private messaging/etc.)? This may help you get your issue resolved faster especially if it is urgent. It is important to post your request in the Hire a Pro category, as forum members are not allowed to advertise their services elsewhere (like here).

Hope this helps! Let me know if there are any further questions or issues.

— @samliew

P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.

5

Ah great! Thank you very much!