Troubleshoot private connectivity for the staging storage account
This article describes how to diagnose connectivity problems between Cayosoft Guardian, the Forest Recovery agents, and a staging storage account that is reached over an Azure private endpoint. It applies to Cayosoft Guardian 7.4 and later.
Before you begin
Confirm the configuration described in Configure Azure private endpoints for the staging storage account:
- A private endpoint exists for the recovery site virtual network and for the Cayosoft Guardian-side virtual network.
- Each private endpoint is in the same region and subscription as the virtual network that it serves.
- The
privatelink.blob.core.windows.netprivate DNS zone is linked to both virtual networks. - The service principal has the Storage Blob Data Contributor role on the storage account.
Step 1: Verify name resolution
Run the following command on the affected machine, either a recovery site virtual machine or the Cayosoft Guardian Server:
nslookup <storage-account>.blob.core.windows.netThe answer must be a private IP address from the virtual network's address space:
Non-authoritative answer:
Name: <storage-account>.privatelink.blob.core.windows.net
Address: 10.77.88.4
Aliases: <storage-account>.blob.core.windows.net| Result | Cause | Fix |
|---|---|---|
| A public IP address is returned | The private DNS zone does not resolve the query | Link privatelink.blob.core.windows.net to the virtual network, or configure conditional forwarding from your on-premises DNS to an Azure DNS resolver. |
| The name does not resolve at all | The private DNS zone is missing, or the record was not created with the private endpoint | Recreate the private endpoint with private DNS integration enabled, or add the A record manually. |
| Resolution works from Azure but not on-premises | On-premises DNS does not forward privatelink.blob.core.windows.net
|
Add a conditional forwarder to an Azure DNS Private Resolver. Learn more in What is Azure DNS Private Resolver. |
Step 2: Verify network reachability
If the name resolves to the private IP address, confirm that the machine can open a connection to it:
Test-NetConnection <storage-account>.blob.core.windows.net -Port 443TcpTestSucceeded : True together with the private RemoteAddress means the route and any network security group rules allow the traffic.
| Result | Likely cause | Fix |
|---|---|---|
TcpTestSucceeded : True, private RemoteAddress
|
Connectivity is working | Continue with step 3. |
TcpTestSucceeded : False |
A network security group, route table, or firewall blocks TCP 443 to the private IP address | Allow outbound TCP 443 from the subnet to the private endpoint address. |
RemoteAddress is a public IP address |
Name resolution is still using the public endpoint | Return to step 1. |
Step 3: Verify blob access as the service principal
Cayosoft Guardian connects with the service principal and Azure role-based access control, so test with the same identity rather than a storage account key.
az login --service-principal --username <app-id> --tenant <tenant-id> --password <certificate-or-secret>
az storage blob list `
--account-name <storage-account> `
--container-name <container> `
--auth-mode login `
--output table| Result | Likely cause | Fix |
|---|---|---|
| The blob list is returned | Connectivity and permissions are working. | Investigate the recovery job and agent logs instead. |
AuthorizationPermissionMismatch |
The service principal does not have the Storage Blob Data Contributor role on the account | Assign the role. See Permissions for Forest Recovery in Cayosoft Guardian. |
AuthorizationFailure or a 403 response |
Public network access is disabled and the request arrived over the public endpoint | Return to step 1 and confirm that this machine resolves the account to the private IP address. |
| Request timeout | The private endpoint is not reachable from this network | Return to step 2. |
ContainerNotFound |
Cayosoft Guardian has not created the container yet | Rerun the deployment so that Cayosoft Guardian uploads the agent installers. |
NOTE Do not use a storage account access key for this test. Shared key authentication bypasses Azure role-based access control, so a successful key-based test does not prove that Cayosoft Guardian can reach the account.
Step 4: Check common deployment failures
| Symptom | Cause | Fix |
|---|---|---|
| Deployment reports that the storage account is not accessible | Public network access is disabled and no private endpoint serves the Cayosoft Guardian-side network | Create the second private endpoint, or re-enable public network access until it is in place. |
| Deployment reports that the virtual network is not accessible | The virtual network is not in the same region as the recovery site, or the service principal cannot read it | Use a virtual network in the same region, and verify the role assignment on the network. |
| Agents install but never start the restore | Recovery site virtual machines cannot resolve or reach the staging storage account | Repeat steps 1 through 3 from a recovery site virtual machine. |
| Backup data remains in the staging storage account after recovery | Cleanup could not reach the storage account, or the service principal lost write access | Verify the Storage Blob Data Contributor role, and then delete the leftover blobs manually. |
| The hybrid connection is not created | The service principal lacks Azure Relay Owner or the equivalent granular actions | See Permissions for Forest Recovery in Cayosoft Guardian. |
Step 5: Fall back to the public endpoint
Private connectivity is an opt-in mode. If a recovery is blocked and you need to restore service immediately, re-enable Public network access on the staging storage account in Networking > Firewalls and virtual networks, and then retry the deployment. Cayosoft Guardian reaches the storage account over HTTPS on TCP 443, which is the default behavior for all other scenarios.
Comments
0 comments
Please sign in to leave a comment.