# Building a Receipt scanning agent

In this tutorial, you will learn how to build a Receipt Scanning agent. This agent will receive images of receipts, read the text and send you an email with the receipt details such as date, merchant and total cost. 

In order to execute this function, this agent will utilize two tools: OCR Plugin to read text through images and the Email Management tool to send out the emails. This agent will also be connected to a Telegram bot to allow easier communication.

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2F1LJWIau1ylCjC21TclBi%2Fimage.png?alt=media&#x26;token=802c8c1d-cbca-4af8-9e54-03dd0fde514c" alt=""><figcaption><p>Engine - Build Section</p></figcaption></figure>

## Step 1: Create your first bot and set up configurations

{% embed url="<https://youtu.be/NqdR8XDLmOI>" %}

## Step-by-step instructions

{% tabs %}
{% tab title="Create a node" %}

### Create a new workflow by clicking "+ New" on the left panel. Your first node will appear on the canvas on the right.&#x20;

### Click "Set Bot" to configure bot settings.&#x20;

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FDmXWU4JRe6WIAwTzkzf0%2Fimage.png?alt=media&#x26;token=053d02c6-eef2-46b0-a557-999a4bbfde87" alt=""><figcaption><p>Engine - Build Section</p></figcaption></figure>
{% endtab %}

{% tab title="Configurations" %}

### Set Basic and Advanced configurations

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FkxgsPbxA1PGT6TmlwjUr%2Fimage.png?alt=media&#x26;token=89b1c750-7634-4ed2-84ef-5db78c72aef1" alt="" width="375"><figcaption><p>Bot Settings - Configurations Section</p></figcaption></figure>

Prompt:

> Act as a personal receipt scanning and expense tracking assistant. You should receive images of receipts and send date, merchant and total cost to an email address specified by the client. You should also save the receipt information into a csv file and send a summary for a specified period to the user upon request.
>
> Follow these steps:&#x20;
>
> 1. Greet the user and introduce yourself as the receipt scanning and expense tracking assistant, explain that the agent can process photos of receipts, extract the date and cost, and automatically send it to the user's specified email address.&#x20;
> 2. Explain that you can also send a summary for a specified date range.&#x20;
> 3. Ask the user for an email address.&#x20;
> 4. Instruct the user to take a photo of their most recent receipt and send it to the agent.
> 5. Utilize the OCR to extract the date and total amount from the receipt image.
> 6. Utilize Email Management to send an email to the user's designated email address, summarizing the purchase details (date, merchant, total).
> 7. Send a confirmation message after you've sent the email.&#x20;

{% hint style="info" %}
This prompt gives a general description of the purpose of this bot as well as specific step by step instructions. It also mentions neccessary tools (OCR and Email Management)&#x20;
{% endhint %}

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FuMvc7TOvOuFsLMfumCxz%2Fimage.png?alt=media&#x26;token=a84ba181-3513-49e1-99b5-a7b1b01e088e" alt=""><figcaption><p>Bot Settings - Configurations Section</p></figcaption></figure>

{% hint style="info" %}
Fault tolerance is set to 5 and Auto Retry to 1, which means that if an errors occurs the bot will try 5 x 1 times before terminating.&#x20;

This number is enough to tolerate minor errors and support bot's usage without [using up too many tokens](https://doc.firstbot.tech/faq#context-window-and-token-usage).&#x20;
{% endhint %}

[Learn more about bot configurations. ](https://doc.firstbot.tech/engine/bots/bot-configurations#id-1.-configurations)
{% endtab %}

{% tab title="AI Model & Memory" %}

### Choose a suitable AI Model

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2F4gqc9wg7SmZCCNiIqMwg%2Fimage.png?alt=media&#x26;token=d08bd46b-811c-4656-8088-b05db2be9d74" alt="" width="375"><figcaption><p>Bot Settings - AI Model Section</p></figcaption></figure>

{% hint style="info" %}
For this bot, the default option - GPT-4O is suitable.&#x20;

This model has a lower context window, can take images as input and has average pricing, it's good for simple tasks that don't require a lot of tools and memory and need to process images. <br>

[Learn more about AI Model configurations. ](https://doc.firstbot.tech/engine/bots/bot-configurations#id-2.-ai-model)
{% endhint %}

### Choose memory settings

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FoOy5PLkA2PqZt0pIkGb7%2Fimage.png?alt=media&#x26;token=24d5148c-0841-424f-94ef-594131190f10" alt=""><figcaption><p>Bot Settings - Memory Section</p></figcaption></figure>

{% hint style="info" %}
Enabling memory for this bot would allow it to be more flexible when processing different requests.&#x20;

Custom RAG is a suitable and more affordable option. For more complicated bots, Open AI Assistant would be more reccommended.&#x20;

Text Embeding Ada is a well performing model with a medium price suitable for this agent.&#x20;

[Learn more about bot configurations. ](https://doc.firstbot.tech/engine/bots/bot-configurations#id-3.-memory)
{% endhint %}
{% endtab %}

{% tab title="Other" %}

### Choose a Tool usage mode&#x20;

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2F8ASwd0sHusceMDShwvkj%2Fimage.png?alt=media&#x26;token=847b3d4c-b4e0-4ae8-808d-f81d061b3a2a" alt=""><figcaption><p>Bot Settings - Tool Section</p></figcaption></figure>

{% hint style="info" %}
Auto mode is a default option for bot implementation, allowing the bot to use all connected, active tools.

[Learn more about tool configurations. ](https://doc.firstbot.tech/engine/bots/bot-configurations#id-4.-tool)
{% endhint %}

### Connect a Telegram bot&#x20;

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FBQg9XvZtTV9Uvi0SBrX9%2Fimage.png?alt=media&#x26;token=3485d977-9c7f-41e9-8337-ccb43f06b86e" alt=""><figcaption><p>Bot Settings - Channel Section</p></figcaption></figure>

{% hint style="info" %}
Paste your Telegram bot token to connect a Telegram bot.&#x20;

[Read detailed instructions on Telegram bot connection.](#connect-a-telegram-bot)

[Learn more about channel configurations. ](https://doc.firstbot.tech/engine/bots/bot-configurations#id-5.-channel)
{% endhint %}

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FupzuRc42oRcqPmTT094T%2Fimage.png?alt=media&#x26;token=c2bb67da-1339-4a72-b2da-fc436b879cff" alt=""><figcaption><p>Bot Settings - Resrouces Section</p></figcaption></figure>

{% hint style="info" %}
This bot will not be using any additional resources.&#x20;

[Learn more about resources configurations. ](https://doc.firstbot.tech/engine/bots/bot-configurations#id-6.-resources)
{% endhint %}
{% endtab %}
{% endtabs %}

## Step 2: Add Email Management Tool

{% embed url="<https://youtu.be/7nUEQsjhvq0>" %}

## Step-by-step instructions

{% tabs %}
{% tab title="Connect a new node" %}

### Drag the point on the first bot to connect a new node. Press "Set Tool" to configure tool settings.

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FqSTDA70ZaXY1Lp82Cj4U%2Fimage.png?alt=media&#x26;token=a6c54f4f-c19e-4af4-a2fa-77e187179e6c" alt="" width="375"><figcaption><p>Engine canvas</p></figcaption></figure>
{% endtab %}

{% tab title="Choose a Template" %}

### Search for "Email Management" and install the tool

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2F74NvhuGcGTfRlmVRvxTG%2Fimage.png?alt=media&#x26;token=8caf32be-b7d9-4802-8f23-614b9c1b6e84" alt=""><figcaption><p>Tool Settings - Script Section</p></figcaption></figure>

{% hint style="info" %}
You can find descriptions of available templates to connect more suitable tools. \
Change permission to access public or private tools.&#x20;

[Learn more about script settings. ](https://doc.firstbot.tech/engine/tools/tool-configurations#id-1.-script)
{% endhint %}
{% endtab %}

{% tab title="Configurations" %}

### Set configuration details

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FTSWv121aXqXk1O0BQj4A%2Fimage.png?alt=media&#x26;token=0950b61a-0e57-404c-8eb6-df657517015b" alt=""><figcaption><p>Tool Settings - Configurations Section</p></figcaption></figure>

{% hint style="info" %}
Describe how your bot should use this tool in the description.&#x20;

[Learn more about configurations settings. ](https://doc.firstbot.tech/engine/tools/tool-configurations#id-2.-configurations)
{% endhint %}
{% endtab %}
{% endtabs %}

## Step 3: Add OCR Tool&#x20;

{% embed url="<https://youtu.be/Deesaiilg8w>" %}

## Step-by-step instructions

{% tabs %}
{% tab title="First Tab" %}

### Drag the point on the first bot to connect a new node. Press "Set Tool" to configure tool settings.

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2F5IkG4zG6fKNJVwPwbDNn%2Fimage.png?alt=media&#x26;token=55397aa0-1ce7-4313-9268-d5da55168b16" alt="" width="375"><figcaption><p>Engine canvas</p></figcaption></figure>
{% endtab %}

{% tab title="Choose a Template" %}

### Search for "OCR" and install the tool

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FYuv7H38uNaOlW9zEINBM%2Fimage.png?alt=media&#x26;token=03160e35-3d7e-4f25-88ff-c3b9d2cddcc0" alt=""><figcaption><p>Tool Settings - Script Section</p></figcaption></figure>

{% hint style="info" %}
You can find descriptions of available templates to connect more suitable tools. \
Change permission to access public or private tools.&#x20;

[Learn more about script settings. ](https://doc.firstbot.tech/engine/tools/tool-configurations#id-1.-script)
{% endhint %}
{% endtab %}

{% tab title="Configurations" %}

### Set configuration details

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FKheieqeihCl30vUkg4Uy%2Fimage.png?alt=media&#x26;token=d846b514-d572-41f9-8481-d853318cc1c2" alt=""><figcaption><p>Tool Settings - Configurations Section</p></figcaption></figure>

{% hint style="info" %}
Describe how your bot should use this tool in the description.&#x20;

[Learn more about configurations settings. ](https://doc.firstbot.tech/engine/tools/tool-configurations#id-2.-configurations)
{% endhint %}
{% endtab %}
{% endtabs %}

## Step 4: Test your bot&#x20;

### Sending a receipt to the agent through Telegram

{% embed url="<https://youtu.be/ea-aKF7tLlg>" %}

### Receipt Details sent by the agent:

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FBkqcpmq9eyql3cvoyk3C%2Fimage.png?alt=media&#x26;token=4aec51a0-925c-4aeb-8e08-338f27e83b4f" alt="" width="375"><figcaption><p>Email sent by the bot</p></figcaption></figure>

## Step-by-step Instructions

{% tabs %}
{% tab title="Telegram Chat" %}

### Open the connected Telegram Bot to start testing!

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FAVlYngSLAw7GqYeKbO8f%2Fimage.png?alt=media&#x26;token=d2e170a9-3659-4f70-95a2-671bc6684d8e" alt="" width="375"><figcaption><p>Bot Configurations - Channel section </p></figcaption></figure>

{% hint style="info" %}
To try out your agent through the Telegram channel, you can access the bot by searching it directly, or by following the link and the QR code inside the bot vonfigurations, Channel section.&#x20;

[Learn more about Telegram bot connection. ](https://doc.firstbot.tech/tutorials/engine/connecting-a-telegram-bot)
{% endhint %}
{% endtab %}

{% tab title="Testing" %}

### Send messages to test your bot's functionality

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FDdjhv8ybUIkpupN0zAhd%2Fimage.png?alt=media&#x26;token=2d5d9d15-64e3-49e4-8961-69b26715f150" alt="" width="162"><figcaption><p>Telegram bot convesation</p></figcaption></figure>

<figure><img src="https://912666480-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FulIQiFZkpFd5li7rHZ8c%2Fuploads%2FERdpeVHUIoQBn0f3SpFt%2Fimage.png?alt=media&#x26;token=f974670e-5d9d-4e15-9991-b61f626fb929" alt="" width="375"><figcaption><p>Log Records of the Email Management Tool</p></figcaption></figure>

{% hint style="info" %}
As you send message to your bot either thorugh the build in console or the connected Telegram bot, the Log Records of each node will show live updates of internal processing. &#x20;

[Learn more about testing with the use of the console. ](https://doc.firstbot.tech/engine/debugging-console)
{% endhint %}
{% endtab %}
{% endtabs %}

## Step 5: Publish&#x20;

Follow our next tutorial to learn how to publish this agent privately, within all members of your organization.&#x20;

{% content-ref url="publish-an-agent" %}
[publish-an-agent](https://doc.firstbot.tech/tutorials/engine/publish-an-agent)
{% endcontent-ref %}