I recently needed to copy files that were created in a GitHub action to Azure Blob Storage
The script should run seamlessly locally and on GitHub actions, without having to provide any secrets, I ran into a few issues so hopefully this will be useful to someone.
To do this you first need to create:
a resource group
a managed identity (or service principal but we'll be using managed identity in this blog)
a federated credential
a storage account
permissions for the managed identity to access the storage account
Terraform example:
Azure CLI example:
Then create three secrets in the GitHub repository settings, see GitHub's instructions.
The secrets you need are:
AZURE_CLIENT_ID
AZURE_TENANT_ID
AZURE_SUBSCRIPTION_ID
Now you're ready to create the GitHub action:
There are two critical environment variables in the Copy file to Azure Blob Storage step (which are currently not well documented):
AZCOPY_AUTO_LOGIN_TYPE=AZCLI
This instructs azcopy to use your existing Azure CLI login without having to run azcopy login
AZCOPY_TENANT_ID=${{ secrets.AZURE_TENANT_ID }}
There is a bug in azcopy's federated identity functionality where it tries to authenticate with the Microsoft tenant if no tenant is provided, even though the information should be provided by the Azure CLI login, this is the recommended workaround.
Push the GitHub action to the main branch and run it, the workflow should pass and the output should look roughly like:
Run azcopy sync --compare-hash=md5 \
INFO: Login with AzCliCreds succeeded
INFO: Authenticating to destination using Azure AD
INFO: XAttr hash storage mode is selected. This assumes all files indexed on the source are on filesystem(s) that support user_xattr.
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
Job 6ece7230-2b26-9249-5738-c15551e9c5e3 has started
Log file is located at: /home/runner/.azcopy/6ece7230-2b26-9249-5738-c15551e9c5e3.log
INFO: One or more hash storage operations (read/write) have failed. Check the scanning log for details.
100.0 %, 1 Done, 0 Failed, 0 Pending, 1 Total, 2-sec Throughput (Mb/s): 0.0001
Job 6ece7230-2b26-9249-5738-c15551e9c5e3 Summary
Files Scanned at Source: 1
Files Scanned at Destination: 0
Elapsed Time (Minutes): 0.0334
Number of Copy Transfers for Files: 1
Number of Copy Transfers for Folder Properties: 0
Total Number of Copy Transfers: 1
Number of Copy Transfers Completed: 1
Number of Copy Transfers Failed: 0
Number of Deletions at Destination: 0
Total Number of Bytes Transferred: 14
Total Number of Bytes Enumerated: 14
Final Job Status: Completed
If you check the storage account you'll find the file has been uploaded:
This blog post has covered quite a bit, at the end of this you should have:
Created a managed identity with a federated credential, that has a trust relationship with a GitHub repository on its main branch allowing that branch to login with that managed identity without needing a password or certificate.
That identity has RBAC permissions on a storage account and GitHub actions using that identity is able to upload a file with azcopy