Error encountered while storing reference data snapshot into the storage account

Question

Error encountered while storing reference data snapshot into the storage account

Kelele G Berhe 0

Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-09T17:05:11.16+00:00

Hello Kelele G Berhe,

Welcome to the Microsoft Q&A forum

Could you please provide us with more details about the issue and the full error message you encountered? This will help us understand the issue better and can provide you better assistance
Kelele G Berhe 0 Reputation points

2024-05-09T17:17:30.26+00:00

@Bhargava-MSFT thanks for responding. The issue i am having is that i was working on streaming data from Azure SQL Database, & to do that i created a reference data input from azure sql db, blob input stream & output that pushes to power bi workspace, unfortunatley the streaming job is failing to insert the reference data from azure sql db to the blob input stream & returning "Error encountered while storing reference data snapshot into the storage account" error.
Kelele G Berhe 0 Reputation points

2024-05-09T21:17:39.66+00:00

more into the above issue is, i created azure blob container manually but when streaming job starts running it creates a new container with a name like "guid" but i was expecting the snapshot to be created inside "fucturecontractsmarkettransactionsblob" container.
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-10T21:11:48.7166667+00:00

Hello Kelele G Berhe,

This seems to be an issue with permissions or firewall configuration.

Can you please check if the Stream Analytics job has access to the SQL Database?

For Microsoft's Microsoft Entra ID to verify if the Stream Analytics job has access to the SQL Database, we need to give Microsoft Entra permission to communicate with the database. To do this, go to the "Firewalls and virtual network"/”Firewalls” page in Azure portal again, and enable "Allow Azure services and resources to access this server/workspace."

Reference document:
https://learn.microsoft.com/en-us/azure/stream-analytics/sql-database-output-managed-identity?tabs=azure-sql
I hope this helps. Please let me know if you have any further questions.
Kelele G Berhe 0 Reputation points

2024-05-10T21:38:07.18+00:00

Hello @Bhargava-MSFT I believe it has access, see below, i am able to see the sample data from the reference data input, which is accessing the azure sql database.
Kelele G Berhe 0 Reputation points

2024-05-10T21:56:10.5566667+00:00
or if i am doing it incorrectly, may be help me how can I startover the streaming data from azure sql database to power bi.

This is how i am doing it:

i created a streaming job.

i created a reference input for sql database & assigned a storage account.

i created stream input azure blob & created a container in the same storage account.

wrote a query that joins the reference input & the blob stream input but when i run it, it says the blob container has no data.

i started the streaming job, which creates a new container & dumps data to it instead of to the container i created manually, so the output is no result to show.

am i doing it wrong? is there any better approach to do it?
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-10T22:15:01.47+00:00

Hi Kelele G Berhe,

Thanks for the addtional details. I will try to reproduce this from my end and get back to you with more details. Thanks for your patience.
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-14T01:47:56.2733333+00:00

Hi Kelele G Berhe,

I setup the streaming job and was able to stream the data to powerBI dashboard. when you start the streaming job, it should create an output container. If it’s not the one you manually created, please check the output configuration to ensure it points to the correct container.

I followed this tutorial to setup my environment. Please review it and identify any mistakes.
Kelele G Berhe 0 Reputation points

2024-05-16T20:45:20.7233333+00:00

it is not working for me. Can we please have a call & guid me how should i continue?
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-17T16:57:10.95+00:00

Hi Kelele G Berhe,

As the next step, I suggest opening a support case so that a support engineer can further troubleshoot the issue. If you don't have a support plan, please let me know, and I can provide a one-time free support request for this case. I look forward to hearing from you.
Kelele G Berhe 0 Reputation points

2024-05-17T18:06:10.04+00:00

@Bhargava-MSFT i would like you to give me the one free support.
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-17T18:32:23.4466667+00:00

Kelele G Berhe,

I have sent you a private message. Please check and provide me the requested details.

1 answer

Your answer

Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-09T17:05:11.16+00:00

Hello Kelele G Berhe,

Welcome to the Microsoft Q&A forum

Could you please provide us with more details about the issue and the full error message you encountered? This will help us understand the issue better and can provide you better assistance
Kelele G Berhe 0 Reputation points

2024-05-09T17:17:30.26+00:00

@Bhargava-MSFT thanks for responding. The issue i am having is that i was working on streaming data from Azure SQL Database, & to do that i created a reference data input from azure sql db, blob input stream & output that pushes to power bi workspace, unfortunatley the streaming job is failing to insert the reference data from azure sql db to the blob input stream & returning "Error encountered while storing reference data snapshot into the storage account" error.
Kelele G Berhe 0 Reputation points

2024-05-09T21:17:39.66+00:00

more into the above issue is, i created azure blob container manually but when streaming job starts running it creates a new container with a name like "guid" but i was expecting the snapshot to be created inside "fucturecontractsmarkettransactionsblob" container.
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-10T21:11:48.7166667+00:00

Hello Kelele G Berhe,

This seems to be an issue with permissions or firewall configuration.

Can you please check if the Stream Analytics job has access to the SQL Database?

For Microsoft's Microsoft Entra ID to verify if the Stream Analytics job has access to the SQL Database, we need to give Microsoft Entra permission to communicate with the database. To do this, go to the "Firewalls and virtual network"/”Firewalls” page in Azure portal again, and enable "Allow Azure services and resources to access this server/workspace."

Reference document:
https://learn.microsoft.com/en-us/azure/stream-analytics/sql-database-output-managed-identity?tabs=azure-sql
I hope this helps. Please let me know if you have any further questions.
Kelele G Berhe 0 Reputation points

2024-05-10T21:38:07.18+00:00

Hello @Bhargava-MSFT I believe it has access, see below, i am able to see the sample data from the reference data input, which is accessing the azure sql database.
Kelele G Berhe 0 Reputation points

2024-05-10T21:56:10.5566667+00:00

or if i am doing it incorrectly, may be help me how can I startover the streaming data from azure sql database to power bi.

This is how i am doing it:

i created a streaming job.

i created a reference input for sql database & assigned a storage account.

i created stream input azure blob & created a container in the same storage account.

wrote a query that joins the reference input & the blob stream input but when i run it, it says the blob container has no data.

i started the streaming job, which creates a new container & dumps data to it instead of to the container i created manually, so the output is no result to show.

am i doing it wrong? is there any better approach to do it?
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-10T22:15:01.47+00:00

Hi Kelele G Berhe,

Thanks for the addtional details. I will try to reproduce this from my end and get back to you with more details. Thanks for your patience.
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-14T01:47:56.2733333+00:00

Hi Kelele G Berhe,

I setup the streaming job and was able to stream the data to powerBI dashboard. when you start the streaming job, it should create an output container. If it’s not the one you manually created, please check the output configuration to ensure it points to the correct container.

I followed this tutorial to setup my environment. Please review it and identify any mistakes.
Kelele G Berhe 0 Reputation points

2024-05-16T20:45:20.7233333+00:00

it is not working for me. Can we please have a call & guid me how should i continue?
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-17T16:57:10.95+00:00

Hi Kelele G Berhe,

As the next step, I suggest opening a support case so that a support engineer can further troubleshoot the issue. If you don't have a support plan, please let me know, and I can provide a one-time free support request for this case. I look forward to hearing from you.
Kelele G Berhe 0 Reputation points

2024-05-17T18:06:10.04+00:00

@Bhargava-MSFT i would like you to give me the one free support.
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-05-17T18:32:23.4466667+00:00

Kelele G Berhe,

I have sent you a private message. Please check and provide me the requested details.

Answer 1

Bansari Oza 20

Hi @Kelele G Berhe You able to solve it? I am also facing issue, where after making Storage Account private ASA Jobs been failing.

Share via

Error encountered while storing reference data snapshot into the storage account

1 answer

Your answer