Skip to content

Image Sync

1 post with the tag “Image Sync”

How to Use GitHub Actions + image-syncer for Automated Image Sync from Docker Hub to Azure ACR

Automating Image Sync from Docker Hub to Azure ACR

Section titled “Automating Image Sync from Docker Hub to Azure ACR”

This article explains how to use GitHub Actions and the image-syncer tool to automate image synchronization from Docker Hub to Azure Container Registry, solving the problem of slow Docker Hub access in mainland China and some Azure regions, while improving image availability and deployment efficiency in Azure environments.

The HagiCode project uses Docker images as its core runtime components, with the main images hosted on Docker Hub. As the project has evolved and Azure deployment needs have grown, we encountered the following pain points:

  • Slow image pulls, because access to Docker Hub is limited in mainland China and some Azure regions
  • Relying on a single image source creates a single point of failure risk
  • Using Azure Container Registry in Azure environments provides better network performance and integration experience

To solve these problems, we need to establish an automated image synchronization mechanism that regularly syncs images from Docker Hub to Azure ACR, ensuring users get faster image pull speeds and higher availability in Azure environments.

We are building HagiCode, an AI-driven coding assistant that makes development smarter, more convenient, and more enjoyable.

Smart: AI assistance throughout the entire process, from idea to code, boosting coding efficiency several times over. Convenient: Multi-threaded concurrent operations make full use of resources and keep the development workflow smooth. Fun: Gamification and an achievement system make coding less dull and more rewarding.

The project is evolving rapidly. If you are interested in technical writing, knowledge management, or AI-assisted development, welcome to check it out on GitHub.

When defining the solution, we compared multiple technical approaches:

  • Incremental sync: only synchronizes changed image layers, significantly reducing network transfer
  • Resume support: synchronization can resume after network interruptions
  • Concurrency control: supports configurable concurrency to improve large image sync efficiency
  • Robust error handling: built-in retry mechanism for failures (3 times by default)
  • Lightweight deployment: single binary with no dependencies
  • Multi-registry support: compatible with Docker Hub, Azure ACR, Harbor, and more
  • No incremental sync support: each run requires pulling the full image content
  • Lower efficiency: large network transfer volume and longer execution time
  • Simple and easy to use: relies on familiar docker pull / docker push commands
  • Higher complexity: requires Azure CLI authentication setup
  • Functional limitations: az acr import is relatively limited
  • Native integration: integrates well with Azure services

Decision 1: Set the sync frequency to daily at 00:00 UTC

Section titled “Decision 1: Set the sync frequency to daily at 00:00 UTC”
  • Balances image freshness with resource consumption
  • Avoids peak business hours and reduces impact on other operations
  • Docker Hub images are usually updated after daily builds
  • Maintains full consistency with Docker Hub
  • Provides flexible version choices for users
  • Simplifies sync logic by avoiding complex tag filtering rules

Decision 3: Store credentials in GitHub Secrets

Section titled “Decision 3: Store credentials in GitHub Secrets”
  • Natively supported by GitHub Actions with strong security
  • Simple to configure and easy to manage and maintain
  • Supports repository-level access control
  • Use GitHub Secrets for encrypted storage
  • Rotate ACR passwords regularly
  • Limit ACR account permissions to push-only
  • Monitor ACR access logs

Risk 2: Sync failures causing image inconsistency

Section titled “Risk 2: Sync failures causing image inconsistency”
  • image-syncer includes a built-in incremental sync mechanism
  • Automatic retry on failure (3 times by default)
  • Detailed error logs and failure notifications
  • Resume support
  • Incremental sync reduces network transfer
  • Configurable concurrency (10 in the current setup)
  • Monitor the number and size of synchronized images
  • Run synchronization during off-peak hours

We use an automated GitHub Actions + image-syncer solution to synchronize images from Docker Hub to Azure ACR.

  • Create or confirm an Azure Container Registry in Azure Portal
  • Create ACR access credentials (username and password)
  • Confirm access permissions for the Docker Hub image repository

Add the following secrets in the GitHub repository settings:

  • AZURE_ACR_USERNAME: Azure ACR username
  • AZURE_ACR_PASSWORD: Azure ACR password

Configure the workflow in .github/workflows/sync-docker-acr.yml:

  • Scheduled trigger: every day at 00:00 UTC
  • Manual trigger: supports workflow_dispatch
  • Extra trigger: run when the publish branch receives a push (for fast synchronization)
SequenceParticipantActionDescription
1GitHub ActionsTrigger workflowTriggered by schedule, manual run, or a push to the publish branch
2GitHub Actions → image-syncerDownload and run the sync toolEnter the actual sync phase
3image-syncer → Docker HubFetch image manifests and tag listRead source repository metadata
4image-syncer → Azure ACRFetch existing image information from the target repositoryDetermine the current target-side state
5image-syncerCompare source and target differencesIdentify image layers that need to be synchronized
6image-syncer → Docker HubPull changed image layersTransfer only the content that needs updating
7image-syncer → Azure ACRPush changed image layersComplete incremental synchronization
8image-syncer → GitHub ActionsReturn synchronization statisticsIncludes results, differences, and error information
9GitHub ActionsRecord logs and upload artifactsUseful for follow-up auditing and troubleshooting

Here is the actual workflow configuration in use (.github/workflows/sync-docker-acr.yml):

name: Sync Docker Image to Azure ACR
on:
schedule:
- cron: "0 0 * * *" # Every day at 00:00 UTC
workflow_dispatch: # Manual trigger
push:
branches: [publish]
permissions:
contents: read
jobs:
sync:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Download image-syncer
run: |
# Download the image-syncer binary
wget https://github.com/AliyunContainerService/image-syncer/releases/download/v1.5.5/image-syncer-v1.5.5-linux-amd64.tar.gz
tar -zxvf image-syncer-v1.5.5-linux-amd64.tar.gz
chmod +x image-syncer
- name: Create auth config
run: |
# Generate the authentication configuration file (YAML format)
cat > auth.yaml <<EOF
hagicode.azurecr.io:
username: "${{ secrets.AZURE_ACR_USERNAME }}"
password: "${{ secrets.AZURE_ACR_PASSWORD }}"
EOF
- name: Create images config
run: |
# Generate the image synchronization configuration file (YAML format)
cat > images.yaml <<EOF
docker.io/newbe36524/hagicode: hagicode.azurecr.io/hagicode
EOF
- name: Run image-syncer
run: |
# Run synchronization (using the newer --auth and --images parameters)
./image-syncer --auth=./auth.yaml --images=./images.yaml --proc=10 --retries=3
- name: Upload logs
if: always()
uses: actions/upload-artifact@v4
with:
name: sync-logs
path: image-syncer-*.log
retention-days: 7
  • Scheduled trigger: cron: "0 0 * * *" - runs every day at 00:00 UTC
  • Manual trigger: workflow_dispatch - allows users to run it manually in the GitHub UI
  • Push trigger: push: branches: [publish] - triggered when the publish branch receives a push (for fast synchronization)

2. Authentication configuration (auth.yaml)

Section titled “2. Authentication configuration (auth.yaml)”
hagicode.azurecr.io:
username: "${{ secrets.AZURE_ACR_USERNAME }}"
password: "${{ secrets.AZURE_ACR_PASSWORD }}"
docker.io/newbe36524/hagicode: hagicode.azurecr.io/hagicode

This configuration means synchronizing all tags from docker.io/newbe36524/hagicode to hagicode.azurecr.io/hagicode

  • --auth=./auth.yaml: path to the authentication configuration file
  • --images=./images.yaml: path to the image synchronization configuration file
  • --proc=10: set concurrency to 10
  • --retries=3: retry failures 3 times

Configure the following in Settings → Secrets and variables → Actions in the GitHub repository:

Secret NameDescriptionExample ValueHow to Get It
AZURE_ACR_USERNAMEAzure ACR usernamehagicodeAzure Portal → ACR → Access keys
AZURE_ACR_PASSWORDAzure ACR passwordxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxAzure Portal → ACR → Access keys → Password
  1. Open the Actions tab of the GitHub repository
  2. Select the Sync Docker Image to Azure ACR workflow
  3. Click the Run workflow button
  4. Choose the branch and click Run workflow to confirm
  1. Click a specific workflow run record on the Actions page
  2. View the execution logs for each step
  3. Download the sync-logs file from the Artifacts section at the bottom of the page
Terminal window
# Log in to Azure ACR
az acr login --name hagicode
# List images and their tags
az acr repository show-tags --name hagicode --repository hagicode --output table
  • Rotate Azure ACR passwords regularly (recommended every 90 days)
  • Use a dedicated ACR service account with push-only permissions
  • Monitor ACR access logs to detect abnormal access in time
  • Do not output credentials in logs
  • Do not commit credentials to the code repository
  • Adjust the --proc parameter: tune concurrency based on network bandwidth (recommended 5-20)
  • Monitor synchronization time: if it takes too long, consider reducing concurrency
  • Clean up logs regularly: set a reasonable retention-days value (7 days in the current setup)
Error: failed to authenticate to hagicode.azurecr.io

Solution:

  1. Check whether GitHub Secrets are configured correctly
  2. Verify whether the Azure ACR password has expired
  3. Confirm whether the ACR service account permissions are correct
Error: timeout waiting for response

Solution:

  1. Check network connectivity
  2. Reduce concurrency (--proc parameter)
  3. Wait for the network to recover and trigger the workflow again
Warning: some tags failed to sync

Solution:

  1. Check the synchronization logs to identify failed tags
  2. Manually trigger the workflow to synchronize again
  3. Verify that the source image on Docker Hub is working properly
  • Regularly check the Actions page to confirm workflow run status
  • Configure GitHub notifications to receive workflow failure alerts promptly
  • Monitor Azure ACR storage usage
  • Regularly verify tag consistency

Q1: How do I sync specific tags instead of all tags?

Section titled “Q1: How do I sync specific tags instead of all tags?”

Modify the images.yaml configuration file:

# Sync only the latest and v1.0 tags
docker.io/newbe36524/hagicode:latest: hagicode.azurecr.io/hagicode:latest
docker.io/newbe36524/hagicode:v1.0: hagicode.azurecr.io/hagicode:v1.0

Q2: How do I sync multiple image repositories?

Section titled “Q2: How do I sync multiple image repositories?”

Add multiple lines in images.yaml:

docker.io/newbe36524/hagicode: hagicode.azurecr.io/hagicode
docker.io/newbe36524/another-image: hagicode.azurecr.io/another-image

Q3: How do I retry after a synchronization failure?

Section titled “Q3: How do I retry after a synchronization failure?”
  • Automatic retry: image-syncer includes a built-in retry mechanism (3 times by default)
  • Manual retry: click Re-run all jobs on the GitHub Actions page

Q4: How do I view detailed synchronization progress?

Section titled “Q4: How do I view detailed synchronization progress?”
  • View real-time logs on the Actions page
  • Download the sync-logs artifact to see the full log file
  • The log file includes the synchronization status and transfer speed for each tag
  • Initial full synchronization: typically takes 10-30 minutes depending on image size
  • Incremental synchronization: usually 2-5 minutes if image changes are small
  • Time depends on network bandwidth, image size, and concurrency settings

Add a notification step to the workflow:

- name: Notify on success
if: success()
run: |
echo "Docker images synced successfully to Azure ACR"

Add tag filtering logic to the workflow:

- name: Filter tags
run: |
# Sync only tags that start with v
echo "docker.io/newbe36524/hagicode:v* : hagicode.azurecr.io/hagicode:v*" > images.yaml

3. Add a synchronization statistics report

Section titled “3. Add a synchronization statistics report”
- name: Generate report
if: always()
run: |
echo "## Sync Report" >> $GITHUB_STEP_SUMMARY
echo "- Total tags: $(grep -c 'synced' image-syncer-*.log)" >> $GITHUB_STEP_SUMMARY
echo "- Sync time: ${{ steps.sync.outputs.duration }}" >> $GITHUB_STEP_SUMMARY

With the method introduced in this article, we successfully implemented automated image synchronization from Docker Hub to Azure ACR. This solution uses the scheduled and manual trigger capabilities of GitHub Actions together with the incremental synchronization and error-handling features of image-syncer to ensure timely and consistent image synchronization.

We also discussed security best practices, performance optimization, troubleshooting, and other related topics to help users better manage and maintain this synchronization mechanism. We hope this article provides valuable reference material for developers who need to deploy Docker images in Azure environments.


Thank you for reading. If you found this article useful, please click the like button below 👍 so more people can discover it.

This content was created with AI-assisted collaboration, reviewed by me, and reflects my own views and position.