Databases and service containers

Overview

Bitbucket Pipelines allows you to run multiple Docker containers from your build pipeline. These additional containers support services needed during testing and application operations, including data stores, code analytics tools, and stub web services.

How Service Containers Work

Services are defined in the definitions section of bitbucket-pipelines.yml. Once defined, they can be referenced by any pipeline step requiring them.

Services share a network adapter with the build container and all open their ports on localhost. No port mapping or hostnames are required. For example, if you were using Postgres, your tests just connect to port 5432 on localhost.

Defining Services

Basic Service Definition

definitions:
  services:
    redis:
      image: redis:3.2
    mysql:
      image: mysql:5.7
      variables:
        MYSQL_DATABASE: my-db
        MYSQL_ROOT_PASSWORD: $password
pipelines:
  default:
    - step:
        services:
          - redis
          - mysql
        script:
          - npm test

Key Limitations

Important Constraints

Maximum: 5 service containers per build step
Port restriction: Port 29418 cannot be used
No REST API: Cannot access services and logs via API
No startup wait: No built-in mechanism to wait for service startup
Memory limits: See allocation details below

Memory Allocation

Available Memory

Step Size	Total Memory
1x	4,096 MB
2x	8,192 MB
4x	16,384 MB
8x	32,768 MB

Default Allocation

Build container: minimum 1024 MB
Service containers: 1024 MB each (configurable between 128-3072 MB)
Remaining memory goes to the build container

Custom Memory Allocation

definitions:
  services:
    redis:
      image: redis:3.2
      memory: 512
    postgres:
      image: postgres:15
      memory: 2048
      variables:
        POSTGRES_PASSWORD: testpass
pipelines:
  default:
    - step:
        size: 2x  # 8GB total
        services:
          - redis
          - postgres
        script:
          - npm test

Database Examples

MongoDB

definitions:
  services:
    mongodb:
      image: mongo:7
pipelines:
  default:
    - step:
        services:
          - mongodb
        script:
          - npm install
          - npm test

Connection details:

Host: 127.0.0.1
Port: 27017
No authentication required
Databases are created automatically

Use 127.0.0.1 instead of localhost to avoid IPv6 issues

MySQL with Test User

definitions:
  services:
    mysql:
      image: mysql:5.7
      variables:
        MYSQL_DATABASE: 'pipelines'
        MYSQL_USER: 'test_user'
        MYSQL_PASSWORD: 'test_user_password'
        MYSQL_ROOT_PASSWORD: 'root_password'
pipelines:
  default:
    - step:
        services:
          - mysql
        script:
          - mysql -h 127.0.0.1 -u test_user -ptest_user_password pipelines < schema.sql
          - npm test

Connection details:

Host: 127.0.0.1
Port: 3306
Database: pipelines
User: test_user
Password: test_user_password

MySQL with Root Access

definitions:
  services:
    mysql:
      image: mysql:8.0
      variables:
        MYSQL_DATABASE: 'myapp'
        MYSQL_ROOT_PASSWORD: $MYSQL_ROOT_PASSWORD
pipelines:
  default:
    - step:
        services:
          - mysql
        script:
          - mysql -h 127.0.0.1 -u root -p$MYSQL_ROOT_PASSWORD myapp < setup.sql
          - npm test

PostgreSQL

definitions:
  services:
    postgres:
      image: postgres:15
      variables:
        POSTGRES_DB: 'testdb'
        POSTGRES_USER: 'testuser'
        POSTGRES_PASSWORD: 'testpass'
pipelines:
  default:
    - step:
        services:
          - postgres
        script:
          - psql -h 127.0.0.1 -U testuser -d testdb -f schema.sql
          - npm test

Connection details:

Host: 127.0.0.1
Port: 5432
Database: testdb
User: testuser
Password: testpass

Redis

definitions:
  services:
    redis:
      image: redis:7
pipelines:
  default:
    - step:
        services:
          - redis
        script:
          - redis-cli -h 127.0.0.1 ping
          - npm test

The official redis:7 image does not recognize REDIS_PASSWORD as an environment variable. Redis runs without authentication by default, which is typically fine for CI. If you need password authentication, pass --requirepass as a command argument to the container.

Connection details:

Host: 127.0.0.1
Port: 6379

Elasticsearch

definitions:
  services:
    elasticsearch:
      image: elasticsearch:8.11.0
      memory: 2048
      variables:
        discovery.type: single-node
        xpack.security.enabled: 'false'
        ES_JAVA_OPTS: '-Xms512m -Xmx512m'
pipelines:
  default:
    - step:
        size: 2x
        services:
          - elasticsearch
        script:
          - sleep 30  # Wait for Elasticsearch to start
          - curl http://127.0.0.1:9200/_cluster/health
          - npm test

Multi-Service Examples

Full Stack Testing

definitions:
  services:
    postgres:
      image: postgres:15
      variables:
        POSTGRES_DB: myapp
        POSTGRES_PASSWORD: testpass
    redis:
      image: redis:7
      memory: 512
    rabbitmq:
      image: rabbitmq:3.12
      memory: 1024
pipelines:
  default:
    - step:
        name: Integration Tests
        size: 2x
        services:
          - postgres
          - redis
          - rabbitmq
        script:
          - export DATABASE_URL=postgresql://postgres:testpass@127.0.0.1:5432/myapp
          - export REDIS_URL=redis://127.0.0.1:6379
          - export RABBITMQ_URL=amqp://127.0.0.1:5672
          - npm install
          - npm run test:integration

Microservices Testing

definitions:
  services:
    api-mock:
      image: mockserver/mockserver:5.15.0
      memory: 512
    database:
      image: postgres:15
      variables:
        POSTGRES_PASSWORD: testpass
pipelines:
  default:
    - step:
        services:
          - api-mock
          - database
        script:
          - ./setup-mocks.sh
          - npm test

Waiting for Services

Services don't have a built-in readiness check. Use wait scripts:

Wait for MySQL

- step:
    services:
      - mysql
    script:
      - |
        until mysql -h 127.0.0.1 -u root -p$MYSQL_ROOT_PASSWORD -e "SELECT 1"; do
          echo "Waiting for MySQL..."
          sleep 2
        done
      - npm test

Wait for PostgreSQL

- step:
    services:
      - postgres
    script:
      - |
        until psql -h 127.0.0.1 -U postgres -c "SELECT 1" > /dev/null 2>&1; do
          echo "Waiting for PostgreSQL..."
          sleep 2
        done
      - npm test

Wait for Redis

- step:
    services:
      - redis
    script:
      - |
        until redis-cli -h 127.0.0.1 ping | grep -q PONG; do
          echo "Waiting for Redis..."
          sleep 2
        done
      - npm test

Generic HTTP Wait

- step:
    services:
      - myservice
    script:
      - |
        until curl -f http://127.0.0.1:8080/health; do
          echo "Waiting for service..."
          sleep 2
        done
      - npm test

Common Patterns

Database Migrations

pipelines:
  default:
    - step:
        services:
          - postgres
        script:
          - npm install
          - npm run db:migrate
          - npm test

Test Data Seeding

pipelines:
  default:
    - step:
        services:
          - mysql
        script:
          - mysql -h 127.0.0.1 -u root -p$MYSQL_ROOT_PASSWORD mydb < seeds/test-data.sql
          - npm test

Service Health Checks

pipelines:
  default:
    - step:
        services:
          - postgres
          - redis
        script:
          - ./scripts/wait-for-services.sh
          - npm run test:integration

Best Practices

Allocate appropriate memory - Databases typically need 512MB-2GB
Use specific versions - Pin service images to exact versions
Wait for readiness - Add wait loops before running tests
Use environment variables - Store connection details in variables
Minimize services - Only run what you need (5 service maximum)
Test locally - Verify service configurations work with Docker Compose
Use 127.0.0.1 - Avoid localhost to prevent IPv6 issues

Troubleshooting

Service Not Starting

Check memory allocation - services need sufficient memory to start.

definitions:
  services:
    postgres:
      image: postgres:15
      memory: 1024  # Increase if needed

Connection Refused

Add a wait loop before connecting:

until pg_isready -h 127.0.0.1; do
  sleep 2
done

Out of Memory

Reduce service memory or increase step size:

pipelines:
  default:
    - step:
        size: 2x  # Double available memory
        services:
          - mysql
        script:
          - npm test
definitions:
  services:
    mysql:
      image: mysql:8.0
      memory: 512  # Reduce service memory
      variables:
        MYSQL_DATABASE: test_db
        MYSQL_ROOT_PASSWORD: test

Testing with databases in Bitbucket Pipelines

When conducting tests that require database services, use service containers to run database services in a linked container. Docker provides numerous official database images available on Docker Hub for this purpose.

Key Recommendations

Validate your bitbucket-pipelines.yml configuration using the online validator
Connect to services using 127.0.0.1 rather than localhost to avoid socket connection issues
Service containers operate on a shared network with the build container, with all ports exposed on localhost

Database Testing Examples

MongoDB Testing

image: node:18
definitions:
  services:
    mongodb:
      image: mongo:7
pipelines:
  default:
    - step:
        name: Test with MongoDB
        services:
          - mongodb
        script:
          - npm install
          - export MONGODB_URI=mongodb://127.0.0.1:27017/testdb
          - npm test

Connection details:

Host: 127.0.0.1
Port: 27017
No authentication required
Databases are created automatically upon connection

MySQL Testing

With Test User

image: node:18
definitions:
  services:
    mysql:
      image: mysql:5.7
      variables:
        MYSQL_DATABASE: 'pipelines'
        MYSQL_USER: 'test_user'
        MYSQL_PASSWORD: 'test_user_password'
        MYSQL_ROOT_PASSWORD: 'root_password'
pipelines:
  default:
    - step:
        name: Test with MySQL
        services:
          - mysql
        script:
          - npm install
          # Wait for MySQL to be ready
          - |
            until mysql -h 127.0.0.1 -u test_user -ptest_user_password pipelines -e "SELECT 1"; do
              echo "Waiting for MySQL..."
              sleep 2
            done
          - mysql -h 127.0.0.1 -u test_user -ptest_user_password pipelines < schema.sql
          - export DATABASE_URL=mysql://test_user:test_user_password@127.0.0.1:3306/pipelines
          - npm test

Connection details:

Host: 127.0.0.1
Port: 3306
Database: pipelines
User: test_user
Password: test_user_password

With Root User

definitions:
  services:
    mysql:
      image: mysql:8.0
      variables:
        MYSQL_DATABASE: 'testdb'
        MYSQL_ROOT_PASSWORD: $MYSQL_ROOT_PASSWORD
pipelines:
  default:
    - step:
        services:
          - mysql
        script:
          - |
            until mysql -h 127.0.0.1 -u root -p$MYSQL_ROOT_PASSWORD testdb -e "SELECT 1"; do
              sleep 2
            done
          - mysql -h 127.0.0.1 -u root -p$MYSQL_ROOT_PASSWORD testdb < setup.sql
          - npm test

PostgreSQL Testing

image: python:3.11
definitions:
  services:
    postgres:
      image: postgres:15
      variables:
        POSTGRES_DB: 'testdb'
        POSTGRES_USER: 'testuser'
        POSTGRES_PASSWORD: 'testpass'
pipelines:
  default:
    - step:
        name: Test with PostgreSQL
        services:
          - postgres
        script:
          - pip install -r requirements.txt
          - pip install psycopg2-binary
          # Wait for PostgreSQL
          - |
            until PGPASSWORD=testpass psql -h 127.0.0.1 -U testuser -d testdb -c "SELECT 1" > /dev/null 2>&1; do
              echo "Waiting for PostgreSQL..."
              sleep 2
            done
          - PGPASSWORD=testpass psql -h 127.0.0.1 -U testuser -d testdb -f schema.sql
          - export DATABASE_URL=postgresql://testuser:testpass@127.0.0.1:5432/testdb
          - pytest

Connection details:

Host: 127.0.0.1
Port: 5432
Database: testdb
User: testuser
Password: testpass

Redis Testing

image: node:18
definitions:
  services:
    redis:
      image: redis:7
pipelines:
  default:
    - step:
        name: Test with Redis
        services:
          - redis
        script:
          - npm install
          # Wait for Redis
          - |
            until redis-cli -h 127.0.0.1 ping | grep -q PONG; do
              echo "Waiting for Redis..."
              sleep 2
            done
          - export REDIS_URL=redis://127.0.0.1:6379
          - npm test

Multi-Database Testing

Testing with Multiple Databases

image: node:18
definitions:
  services:
    postgres:
      image: postgres:15
      variables:
        POSTGRES_DB: appdb
        POSTGRES_PASSWORD: testpass
    redis:
      image: redis:7
      memory: 512
pipelines:
  default:
    - step:
        name: Integration Tests
        size: 2x
        services:
          - postgres
          - redis
        script:
          - npm install
          # Wait for services
          - |
            until PGPASSWORD=testpass psql -h 127.0.0.1 -U postgres -d appdb -c "SELECT 1" > /dev/null 2>&1; do
              echo "Waiting for PostgreSQL..."
              sleep 2
            done
          - |
            until redis-cli -h 127.0.0.1 ping | grep -q PONG; do
              echo "Waiting for Redis..."
              sleep 2
            done
          # Set environment variables
          - export DATABASE_URL=postgresql://postgres:testpass@127.0.0.1:5432/appdb
          - export REDIS_URL=redis://127.0.0.1:6379
          # Run migrations and tests
          - npm run db:migrate
          - npm run test:integration

Framework-Specific Examples

Django with PostgreSQL

image: python:3.11
definitions:
  services:
    postgres:
      image: postgres:15
      variables:
        POSTGRES_DB: django_test
        POSTGRES_USER: django
        POSTGRES_PASSWORD: django_password
pipelines:
  default:
    - step:
        name: Django Tests
        services:
          - postgres
        script:
          - pip install -r requirements.txt
          # Wait for database
          - |
            until PGPASSWORD=django_password psql -h 127.0.0.1 -U django -d django_test -c "SELECT 1" > /dev/null 2>&1; do
              sleep 2
            done
          # Set database connection
          - export DATABASE_URL=postgresql://django:django_password@127.0.0.1:5432/django_test
          - python manage.py migrate
          - python manage.py test

Rails with PostgreSQL

image: ruby:3.2
definitions:
  services:
    postgres:
      image: postgres:15
      variables:
        POSTGRES_PASSWORD: postgres
pipelines:
  default:
    - step:
        name: Rails Tests
        services:
          - postgres
        script:
          - bundle install
          - |
            until PGPASSWORD=postgres psql -h 127.0.0.1 -U postgres -c "SELECT 1" > /dev/null 2>&1; do
              sleep 2
            done
          - RAILS_ENV=test bundle exec rake db:create db:migrate
          - RAILS_ENV=test bundle exec rspec

Laravel with MySQL

image: php:8.2
definitions:
  services:
    mysql:
      image: mysql:8.0
      variables:
        MYSQL_DATABASE: laravel_test
        MYSQL_ROOT_PASSWORD: secret
pipelines:
  default:
    - step:
        name: Laravel Tests
        services:
          - mysql
        script:
          - apt-get update && apt-get install -y git mysql-client
          - curl -sS https://getcomposer.org/installer | php
          - php composer.phar install
          - cp .env.testing .env
          - |
            until mysql -h 127.0.0.1 -u root -psecret laravel_test -e "SELECT 1"; do
              sleep 2
            done
          - php artisan migrate --env=testing
          - php artisan test

Spring Boot with PostgreSQL

image: maven:3.9-eclipse-temurin-17
definitions:
  services:
    postgres:
      image: postgres:15
      variables:
        POSTGRES_DB: springboot_test
        POSTGRES_USER: spring
        POSTGRES_PASSWORD: spring_password
pipelines:
  default:
    - step:
        name: Spring Boot Tests
        services:
          - postgres
        script:
          - |
            until PGPASSWORD=spring_password psql -h 127.0.0.1 -U spring -d springboot_test -c "SELECT 1" > /dev/null 2>&1; do
              sleep 2
            done
          - export SPRING_DATASOURCE_URL=jdbc:postgresql://127.0.0.1:5432/springboot_test
          - export SPRING_DATASOURCE_USERNAME=spring
          - export SPRING_DATASOURCE_PASSWORD=spring_password
          - mvn clean test

Test Data Management

Using SQL Files

pipelines:
  default:
    - step:
        services:
          - postgres
        script:
          - npm install
          # Load schema
          - PGPASSWORD=testpass psql -h 127.0.0.1 -U postgres -d testdb -f db/schema.sql
          # Load test data
          - PGPASSWORD=testpass psql -h 127.0.0.1 -U postgres -d testdb -f db/test-data.sql
          - npm test

Database Migrations

pipelines:
  default:
    - step:
        services:
          - mysql
        script:
          - npm install
          - npm run db:migrate
          - npm run db:seed:test
          - npm test

Fixtures and Factories

pipelines:
  default:
    - step:
        services:
          - postgres
        script:
          - pip install -r requirements.txt
          - python manage.py migrate
          - python manage.py loaddata fixtures/test-data.json
          - pytest

Memory Allocation

Service containers receive 1024 MB by default but can be configured between 128-3072 MB:

definitions:
  services:
    postgres:
      image: postgres:15
      memory: 2048
    redis:
      image: redis:7
      memory: 512
pipelines:
  default:
    - step:
        size: 2x  # Increase step size for more total memory
        services:
          - postgres
          - redis
        script:
          - npm test

Best Practices

Wait for services - Always add wait loops before running tests
Use 127.0.0.1 - Use 127.0.0.1 instead of localhost for all services to avoid IPv6 issues
Allocate sufficient memory - Databases typically need 512MB-2GB
Use specific versions - Pin database images to exact versions
Separate test data - Keep test data separate from application data
Clean between tests - Reset database state between test runs
Use transactions - Wrap tests in transactions for faster cleanup
Cache dependencies - Cache npm/pip/composer packages
Parallel tests - Run test suites in parallel when possible
Monitor test duration - Keep test runs under the time limit

Troubleshooting

Connection Refused

Add a proper wait loop:

until mysql -h 127.0.0.1 -u root -p$MYSQL_ROOT_PASSWORD -e "SELECT 1"; do
  sleep 2
done

Service Not Starting

Increase memory allocation:

definitions:
  services:
    postgres:
      memory: 2048

Tests Timeout

Increase step max-time:

options:
  max-time: 30
pipelines:
  default:
    - step:
        services:
          - postgres
        script:
          - npm test

Out of Memory

Increase step size:

pipelines:
  default:
    - step:
        size: 2x
        services:
          - mysql
        script:
          - npm test

Complete Testing Example

image: node:18
definitions:
  services:
    postgres:
      image: postgres:15
      memory: 1024
      variables:
        POSTGRES_DB: myapp_test
        POSTGRES_USER: testuser
        POSTGRES_PASSWORD: testpass
    redis:
      image: redis:7
      memory: 512
options:
  max-time: 30
pipelines:
  default:
    - step:
        name: Lint and Unit Tests
        caches:
          - node
        script:
          - npm install
          - npm run lint
          - npm run test:unit
  branches:
    main:
      - parallel:
          - step:
              name: Unit Tests
              caches:
                - node
              script:
                - npm install
                - npm run test:unit
          - step:
              name: Integration Tests
              size: 2x
              caches:
                - node
              services:
                - postgres
                - redis
              script:
                - npm install
                # Wait for services
                - |
                  until PGPASSWORD=testpass psql -h 127.0.0.1 -U testuser -d myapp_test -c "SELECT 1" > /dev/null 2>&1; do
                    echo "Waiting for PostgreSQL..."
                    sleep 2
                  done
                - |
                  until redis-cli -h 127.0.0.1 ping | grep -q PONG; do
                    echo "Waiting for Redis..."
                    sleep 2
                  done
                # Set environment
                - export DATABASE_URL=postgresql://testuser:testpass@127.0.0.1:5432/myapp_test
                - export REDIS_URL=redis://127.0.0.1:6379
                # Run migrations and tests
                - npm run db:migrate
                - npm run test:integration
              artifacts:
                - test-results/**
  pull-requests:
    '**':
      - step:
          name: PR Tests
          size: 2x
          caches:
            - node
          services:
            - postgres
            - redis
          script:
            - npm install
            - npm run lint
            - npm test

Next Steps

Learn about Service Containers in detail
Configure Variables for database credentials
Set up Caching to speed up test runs
Use Artifacts to save test reports

Was this helpful?

It wasn't accurateIt wasn't clearIt wasn't relevant

Still need help?

The Atlassian Community is here for you.

Ask the Community