Map array searching using RegEx

[TT92]

Hi all,

I’m wondering if it’s possible to map an array and filter the answer field using a RegEx, so I can extract all URLs directly from the given answers.

Right now, I’m using five modules to detect phone numbers and URLs within the Q+A array. I’m trying to reduce this and ideally handle everything inside the set variables module.

Phone numbers were easy, since they have a dedicated key (type > phone_number), which already helped me reduce the flow from 5 to 3 modules.

It would be great if I could also extract URLs the same way, directly through a variable mapping using RegEx, so I can simplify the scenario even further. Only the key for mapping url is “string“ but there are multiple type’s answers as string… and the question for url’s are on different places within multiple events in Calendly.

Has anyone managed to do this?

Thanks!

Succes Phonenumber mapping:

Old situation want to improve from 5 to 0 modules:

[Stoyan_Vatov]

Mate I dont know what AI you are using to generate your answers but please stop. There is no match() function in Make.

[Stoyan_Vatov]

Hey there,

can you show what the incoming data looks like?

You are trying to only get the answers that contain a URL inside?

[TT92]

Hi @Stoyan_Vatov

It is an array containing the Q+A.

I another module i can get some more information about the event.

With these 2 modules i succeed to get the phonenumber as this have a type= phone_number to get the position and then map module 1.

But now i want to create a solution to map urls in all the answers. but the type is string, and for Calendly is that the short free answer, and they are many more in the Q+A, so can’t map that position.

[Stoyan_Vatov]

Do you have specific question that are asking for URLs? Say questions on positions 2,3 and 7 for example?

Cause then you can do:

map(1.questions_and_answers[];answer;position;2,3,7)

[TT92]

Unfortunately not.. the position of the url / website or link to profile is on every event different.

That’s the reason why i need to map all to find it.

I’m upgrading now my complete scenario and in V1 i had the below run, but i want to reduce this preferably to 0.

This interate the Q+A array and aggregate to all to text, after that match pattern to find phone number and within the same text match element to find all websites or urls (this can sometimes be more than 1) so the last step is to aggregate the founded bundles to text with a newline.

The reason i want to reduce this is: this Scenario sometimes runs 50/100 times a day. multiply this by 5 modules… and you know the reason and answer

On this moment i can map the phone number so i can delete 1 module.., but preferably i would bring the remaining 4 to 0

[Stoyan_Vatov]

You can definitely do it with an iterator and an aggregator with a filter in between. Or with the code app. But either way its 2 operations. I don’t think you can get this with a formula in a single operation, since the incoming array is too dynamic.

If you want to turn an array in a text string, you can use join(), no need to iterate then aggregate. Also you can search for multiple things inside the same match pattern module.

Any chance you can post the incoming JSON so we can try and play with it to find the most optimal way to get the phone numbers and the emails?

[TT92]

Hi @Stoyan_Vatov

It isn’t quite nice but can get al the answer as a string without join() because the answers already are text.

Would be great if we can find something to search this text string with a formula and RegEx to find the website’s or urls.

Unfortunately the match elements text parser don’t see the given url “test.nl” only www. containing addresses.

Unfortunately i can’t share a payload as it contains many personal client data, but i think we must find a way to get a function/formula running on this below text string (3 hidden websites)

“text”: “+31 6 12345678 hhhhh ccccc aaaaaaa. bbbbbbbb (999,-) , weekschappen - en interm opdrachten - youtube duurzaam hoe dan www.ditiseenwebsite.nl 5 - 10K schaalbaarheid - ik kan mezelf niet schalen. ditisookeentest.nl hoe kan ik meer impact maken, minder tijd. 2 , ik gebruik het, maar zoekende , wat is er nog meer mogelijk? 7-10K Dit begrijp ik! Yes, gaan we regelen! test.nl”

Thank you for your help and thoughts till now!

[TT92]

This is the best pattern i can build to also find websites without www.

Only with this i still need to use match pattern and a text aggregator module..

We need to figure something out to get this in one formula/function/oneline in a set variables module

@samliew any ideas?

[samliew]

Could you go to regex101.com, paste in your Pattern at the top and paste a complete/full example text you are trying to match from below it?

Then, save the regex example and share the link with us here.

This will allow others to assist you here with your pattern. Thanks!

— @samliew

[TT92]

@samliew
Here you go! >>>> ** regex101: build, test, and debug regex** <<<<

And it would be great if the output below could be generated in a single line within a variable module:

[
    {
        "Website(s)": "www.ditiseenwebsite.com\nthisisalsoatest.nl\nhttps://test.eu\ntest.nl"
    }
]

[samliew]

@TT92

And it would be great if the output below could be generated in a single line within a variable module:

No problem, the regular expressions expert is here to save the day!

You can use these built-in functions:

• split
  split(text; separator)

• replace
  replace(text; search string; replacement string)

These functions may have also been covered in the Make Academy tutorials — refer to the Make Academy Course Overview to learn more.

Here’s an example of how you can use these functions together:

{{split(replace(1.text; "/[\w\W]+?\b((?:https?:\/\/|www\.)?[a-z0-9.-]+\.[a-z]{2,}|$)\b(?=[\w\W]+?|$)/g"; "$1|"); "|")}}

^{(copy-paste the above into the field, or type it exactly as shown)}

e.g.:

Module Export - quick import into your scenario

You can copy and paste this module export into your scenario. This will import the modules (with fields/settings/filters) shown in my screenshots above.

1. Move your mouse over the line of code below. Copy the JSON by clicking the copy button on the right of the code, which looks like this:
   

2. Enter your scenario editor. Press ESC to close any dialogs. Press CTRLV (paste keyboard shortcut for Windows) to paste directly in the editor.

3. Click on each imported module and re-save it for validation. There may be some errors prompting you to remap some variables and connections.

JSON module export — paste this directly in your scenario

{"subflows":[{"flow":[{"id":250,"module":"util:ComposeTransformer","version":1,"parameters":{},"mapper":{"value":"+31 6 12345678 hhhhh ccccc aaaaaaa. bbbbbbbb (999,-) , weekschappen - en interm opdrachten - youtube duurzaam hoe dan www.ditiseenwebsite.com 5 - 10K schaalbaarheid - ik kan mezelf niet schalen. thisisalsoatest.nl hoe kan ik meer impact maken, minder tijd. 2 , ik gebruik het, maar zoekende , wat is er nog meer mogelijk? 7-10K https://test.eu Dit begrijp ik! Yes, gaan we regelen! test.nl  test"},"metadata":{"designer":{"x":3808,"y":-56}}},{"id":251,"module":"util:SetVariable2","version":1,"parameters":{},"mapper":{"name":"links","scope":"roundtrip","value":"{{split(replace(250.value; \"/[\\w\\W]+?\\b((?:https?:\\/\\/|www\\.)?[a-z0-9.-]+\\.[a-z]{2,}|$)\\b(?=[\\w\\W]+?|$)/g\"; \"$1|\"); \"|\")}}"},"metadata":{"designer":{"x":4049,"y":-53}}}]}],"metadata":{"version":1}}

Note: Did you know you can reduce the size of blueprints and module export code like the above, using the Make Blueprint Scrubber?

Hope this helps! If you are still having trouble, please provide more details.

— @samliew

[amjadiqbal]

Just adding to samliew’s solution - if you want the final output as a single text string with newlines between each URL (like you showed in your example), you can wrap the whole thing with join() like this:

join(split(replace(1.text; “/[\w\W]+?\b((?:https?://|www.)?[a-z0-9.-]+.[a-z]{2,}|$)\b(?=[\w\W]+?|$)/g”; “$1|”); “|”); newline)

This will give you output like:

www.ditiseenwebsite.com

thisisalsoatest.nl

https://test.eu

test.nl

Alternatively, if you want to keep it as an array for further processing, samliew’s split formula already does that perfectly. Just use join() at the end when you need the text format.

[samliew]

@amjadiqbal

you can wrap the whole thing with join() like this:

No, you don’t even need to use another join function.

{{ replace(1.text; "/[\w\W]+?\b((?:https?:\/\/|www\.)?[a-z0-9.-]+\.[a-z]{2,}|$)\b(?=[\w\W]+?|$)/g"; "$1" + newline) }}

Hope this helps! If you are still having trouble, please provide more details.

— @samliew

[Stoyan_Vatov]

But you can get that without the Match Pattern module. After the iterator place a filter that only allows answers that are urls and follow it with a text aggregator.

[TT92]

Thanks @samliew, good start.

But unfortunately with a bit other given answers (text values) i receive more output bundles than only the url’s.

I like this pattern (?:https?://|www.)?[a-z0-9.-]+.[a-z]{2,} more as in Regex101 the matches are perfectly matched, also direct in the Match Pattern module this one works the best. Only within a set variabel module using replace() it doesn’t succeed…

[samliew]

Please provide all the data samples that are failing, as regex101 sandbox links.

Same instructions above.

[TT92]

@samliew

https://regex101.com/r/8g2bH3/1

Text:
+31 6 12987236 the test company is een onderneming die per 1 maart van start gaat. de producten zijn items en gepersonaliseerde tijdschriften voor de reiziger die even wat hulp nodig heeft bij het samenstellen van hun journey. www.test.eu voor nog meer begeleiding kan ook een consult van een uur ingepland worden voor advies over het samenstellen. test.be de doelgroep is de reiziger (met kinderen) die het overweldigende aanbod van informatie en inspiratie online te overweldigend vind en daar wat hulp bij nodig heeft om zelf die eigen rondreis samen te stellen en daadwerkelijk te gaan maken. www.thetestlive98.com 0 - 1k een eerste indruk maak je maar een keer. dus hoe start je zo optimaal mogelijk. ik gebruik ai voor veel informatie en analyses. 0-1k dit begrijp ik! yes, gaan we regelen!

Unfortunately i can’t share to much data samples that fails as it contains many client/lead data, so when sharing this here, I need to adjust some text

You are the RegEx expert, and i think you can arrange that the groups will be the match or something haha

[samliew]

@TT92

You are the RegEx expert, and i think you can arrange that the groups will be the match or something haha

Try this:

/[\w\W]+?\b((?:https?:\/\/|www\.)?[a-z0-9.-]+\.[a-z]{2,})\b.+?(?=\s(?:\w+\.)+\w{2}|$)/g

[TT92]

Till now al the data samples output looks good! Will keep an close eye the upcoming days/weeks