AI Agents as off-platform chatbot

Questions
1

We’ve recently been implementing our AI agents as Slack apps and are coming across some UX challenges, I’m wondering if anyone else has solved them in Slack or elsewhere? Basically the setup is as follows:

  1. Slack App with webhook interaction to connect to a Make scenario
  2. Make scenario as a ‘listener’ to manage input and response
  3. Make Agent invoked as part of the listener scenario
  4. Make Agent loaded with 10-12 tools to assist with completion of the task.

Now, when we’re testing the agent in isolation all is fine because the ‘testing and training’ panel alongside agent configuration is basically a live stream of messages and tool use.

The issue however is when invoking the agent as a “Make Agents” module in a given scenario (our listener in item 2 above) the agent executes all its ‘execution steps’ before sending any output.

The result is a LOOONG wait time for the Slack user, then a big long set of messages dumped to the user at once, when all planned execution steps are completed. Sometimes this takes up to 10 minutes (which is the maximum allowed) given the complexity of tools and task.

Has anyone else come across this and solved it? I can’t see a way to get a more granular real time output from the Make Agent module, so my only workaround option is to prompt more of a chunked approach in the agent instructions.

3

Hi @Lloyd_Vaughan and welcome to the Make Community!

Are your execution steps scenarios? If so, can you have each step send somthing to the slack channel? If it does take a long time to complete the entire flow, sending intermediate notifications could help.

L

4

One of the chatbot players(Botpress), uses a emoticon on your message to indicate it’s processing. When it’s done, it removes the emoticon reaction on your message, and pushes the output

5

Thanks for the replies!

  1. Yes that’s a good suggestion regarding the execution steps. Some of the steps are scenarios (tool calls) so perhaps passing the channel id to each so at least some kind of updated holding message can be sent would be helpful. Perhaps each tool can output all messages up to the point (or since the previous tool call).
  2. Yes exactly - the way l have it configured is to send a “working” message to the user right before the Make Agents module. However this experience when using multiple tool calls still results in a long delay thereafter - and I’m comparing it with the real time output of the “testing and training” panel on the agent configuration screen and trying to get something more like that.

I think the way agents have been implemented I may just have to live with this until perhaps if/when Make create a native front end experience for agents.

Another workaround I thought of was to explicitly limit the number of tool calls per round, in the instructions.. But this also creates more needless back and forth with the user.

Thanks again!

6

To follow up if anyone is attempting the same - I switched from webhooks to the AI Agent API “Run Agent with Server Sent Events (SSE) Streaming“ AI Agents | Make API | Make Developer Hub

Note it’s not full token streaming, chunks come through as entire 'executionSteps’ but it’s still much better than the Webhook experience.

2 Likes