Extract URLs from text

hb · September 9, 2024, 3:58pm

Hey everyone, I would like to extract URLs from some text. There might be one URL, multiple URLs or none. Also there might be https:// or just the domain.

I think I need to use the text parser module and regex, and maybe a repeater module to handle multiple links, but there are so many settings and i don’t know which regex match to use but here is my desired output:

Example input: ChatGPT by OpenAI, Desired output:
Example input: ChatGPT by OpenAI, Learn more at openai.com, Desired output: openai.com
Example input: ChatGPT by OpenAI, Learn more at https://openai.com and MidJourney: Learn more at https://www.midjourney.com Desired output: https://openai.com and https://www.midjourney.com

Any help would be greatly appreciated thank you.

samliew · September 10, 2024, 1:06am

Welcome to the Make community!

Yes, that is possible. You’ll need a minimum of one module:

Screenshot_2024-09-10_090913

Example Output

Screenshot_2024-09-10_090923

This is just an example. Your final solution may or may not look like this depending on your requirements.

Hope this helps! Let me know if there are any further questions or issues.

— @samliew

P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.

hb · September 26, 2024, 8:19am

Thank you Sam!

Please is there a way to remove the query from the url e.g. https://www.midjourney.com?user-id=1234 to https://www.midjourney.com

{{replace("https://www.midjourney.com?user-id=1234"; "\?.*"; emptystring)}}

is not working

hb · September 26, 2024, 9:14am

(Answering for myself)
Its not:

{{replace("https://www.midjourney.com?user-id=1234"; "\?.*"; emptystring)}}

Its:

{{replace("https://www.midjourney.com?user-id=1234"; "/\?.*/"; emptystring)}}

hb · October 3, 2024, 7:07pm

Hi Sam, please how do you turn the collection output into a single array with each link as an item? I’m not sure how to work with collections.

samliew · October 4, 2024, 1:07am

Welcome to the Make community!

You can use the built-in function map

e.g.:

map(1.array; url)

For more information, see the function documentation in the Help Center.

Hope this helps! Let me know if there are any further questions or issues.

— @samliew

P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.

hb · October 4, 2024, 6:50am

Thanks - I’m getting an error message though. Here is my text parser input:

And output:

    {
        "match": "https://openai.com"
    },
    {
        "match": "https://www.midjourney.com"
    }
]

And the input/error Im getting in the set variable map function:

Any help would be appreciated thank you

samliew · October 4, 2024, 7:32am

Change url to match

hb · October 4, 2024, 9:26pm

I’m still getting an error unfortunately

(tried with lower case and uppercase incase it made a difference)

samliew · October 5, 2024, 2:54am

You’ll have to aggregate the multiple bundles from the Text Parser first.

Aggregators

Every result (item/record) from iterator/list/search/match modules will output a bundle. This can result in multiple bundles, which then trigger multiple operations in future modules (one operation per bundle). To “combine” multiple bundles into a single variable, you’ll need to use an aggregator of some sort.

Aggregators are modules that accumulate multiple bundles into one single bundle. An example of a commonly-used aggregator module is the Array aggregator module. The next popular aggregator is the Text Aggregator which is very flexible and can apply to many use-cases like building of JSON, CSV, HTML.

There are other types of aggregator modules, click the below links to find out more:

Hope this helps! Let me know if there are any further questions or issues.

— @samliew

P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.