Blog

Posted June 1, 2023

CircleCI vs. GitHub Actions vs. GitLab: Key Differences

Choosing a solution to create or improve your CI/CD pipeline may seem like a daunting task. Read on to learn more about the benefits and limitations of choosing CircleCI, Github Actions, and GitLab.

When organizations plan to introduce a continuous integration (CI) pipeline (or modernize the one they have), they’ll come across quite a few options. This article covers three of the most widely used solutions: CircleCI, GitHub Actions, and GitLab CI/CD. These solutions provide the capability to extend beyond basic building and deployments, adding support for test automation and other components that will improve the integrity of the software you deliver.

CircleCI

CircleCI aims to provide a best-in-class continuous integration and continuous delivery (CI/CD) platform for use by development teams. It has already proven itself, and you can find case studies about how it has performed for customers ranging in size from startups to multinational companies (and household names like PayPal).

Features and benefits of CircleCI

Easy setup
Configuration contained within a single config.yml file in the repository
Can run jobs in parallel
SSH into any job to debug your build issues
Uses orbs, which are reusable packages that integrate with third parties
Has pre-built Docker images available for multiple languages
Has support for ARM as a machine type in its SaaS offering
You can use self-hosted runners
Can be entirely self-hosted
Has the lowest price point, including a free tier on their SaaS to get you started

Limitations of CircleCI

Has the lowest price point because it has less functionality included in the price than the other two options discussed in this article
Has a smaller selection of orbs than other equivalent providers
Less flexible when it comes to advanced customizations (like the number of container images used per job)

GitHub Actions

GitHub Actions is the CI/CD functionality within the GitHub platform. GitHub’s platform is the most widely used Git platform on the planet, and it has extensive functionality beyond just Actions – including defect and project tracking, image repositories, and analytics. It was purchased by Microsoft in 2018.

Features and benefits of GitHub Actions

Natively integrated with code repositories
Has pre-built Actions (container images) available for multiple languages as well as third-party integrations
You can use self-hosted runners
Can be entirely self-hosted
Has a free tier on its SaaS offering (which will work for small teams)

Limitations of GitHub Actions

Least mature of the three options discussed here
Only supports GitHub repositories
Advanced security scanning and premium support are add-ons in the top-tier plan, which can get expensive with its per-user pricing (especially if you’re only after the CI/CD functionality)

GitLab CI/CD

GitLab CI/CD extends the GitLab platform to allow developers to run continuous integration and delivery pipelines against their code repositories. Much like GitHub, GitLab offers a full suite of functionality that developers can leverage throughout their development cycles.

GitLab uses an open core model, so its base functionality is open source. As a result, it’s probably the most widely used Git repository within corporate data centers, even though the CI/CD integration is not included in the open core.

Features and benefits of GitLab CI/CD

AutoDevOps makes getting started easy for basic functionality
Embedded into code repositories
Configuration contained within a single gitlab-ci.yml file in the repository
Can use existing container images
You can use self-hosted runners
It’s the only one of the solutions discussed in this article that has a free self-hosted option
Also has a free tier on its SaaS, which will work fine for small teams
Multiple types of security scanning are available within its CI/CD pipeline

Limitations of GitLab CI/CD

Only supports GitLab repositories
Offers a smaller selection of orbs than other equivalent providers
Per-user pricing makes it the most expensive option if you want to use it with full enterprise functionality

Adding Continuous Testing to Pipelines

One of the biggest benefits of having CI/CD pipelines is the ability to enhance them with additional checks and validation points to ensure that reliable software is being delivered to the customer in a fast and consistent way. Leveraging continuous testing is a key part of this initiative.

CircleCI

In CircleCI, you can add continuous testing into a pipeline as a series of steps within a job. Alternatively, as with Sauce Labs, you can use an orb from the CircleCI library that will handle most of the heavy lifting, and the test suite can be run in a single step. The sample YAML below imports the orb and then runs it within the job as a single step by passing it in the directory in which the application is stored.

1
version: 2.1
2
orbs:
3
  saucectl: saucelabs/saucectl-run@1.0.0
4
jobs:
5
  my-app-job:
6
    docker:
7
     - image: cimg/node:lts
8
        auth:
9
          username: mydockerhub-user
10
          password: $DOCKERHUB_PASSWORD
11
    steps:
12
      - checkout
13
      - saucectl/saucectl-run:
14
          working-directory: v1
15
workflows:
16
  my-workflow:
17
    jobs:
18
      - my-app-job

GitHub Actions

Enabling continuous testing in GitHub Actions works much the same way as in CircleCI, except that a prebuilt container is often used for testing purposes instead of an orb. In this case, you define the secrets used to access the container in question, then run the container with the appropriate flags and the location of the working directory where the application you want to test resides.

1
name: saucectl cypress tests v1
2
on:
3
  pull_request: null
4
  push:
5
    paths:
6
      - v1
7
    branches:
8
      - main
9
env:
10
  SAUCE_USERNAME: ${{secrets.SAUCE_USERNAME}}
11
  SAUCE_ACCESS_KEY: ${{secrets.SAUCE_ACCESS_KEY}}
12
jobs:
13
  main:
14
    runs-on: ubuntu-latest
15
    steps:
16
      - name: Checkout
17
        uses: actions/checkout@v3
18
      - name: Saucectl RUN Docker and Cloud
19
        uses: saucelabs/saucectl-run-action@v2
20
        with:
21
          skip-run: true
22
          testing-environment: ""
23
          concurrency: 10
24
      - run: saucectl run
25
        working-directory: v1

GitLab CI/CD

The CI/CD options from GitLab, in my opinion, do not seem to have the same level of support in the ecosystem for prebuilt items to help with continuous testing. Ultimately, this really isn’t a big deal, but it means that you’ll have to add an extra item or two to the YAML so that the latest tools are downloaded instead of being able to pull an image with a single line. Again, as with the other CI tools, a single command is run in the workflow with some parameters included in a configuration file (primarily the location of the application to test).

1
image: alpine:3.9
2
variables:
3
  DOCKER_VERSION: 20.10.5
4
  DOCKER_HOST: tcp://docker:2375
5
  SAUCECTL_VERSION: 0.82.1
6
  SAUCE_USERNAME: ${SAUCE_USERNAME}
7
  SAUCE_ACCESS_KEY: ${SAUCE_ACCESS_KEY}
8
stages:
9
  - test
10
.saucectl:
11
  services:
12
    - docker:${DOCKER_VERSION}-dind
13
  before_script:
14
    - apk add curl
15
    - curl -L -o saucectl.tar.gz
16
      https://github.com/saucelabs/saucectl/releases/download/v${SAUCECTL_VERSION}/saucectl_${SAUCECTL_VERSION}_linux_64-bit.tar.gz
17
    - tar -xvzf saucectl.tar.gz
18
    - mv saucectl /usr/bin/saucectl
19
test hybrid:
20
  extends: .saucectl
21
  image: docker:${DOCKER_VERSION}
22
  stage: test
23
  script:
24
    - saucectl run -c .sauce/config.yml --ccy 10

CircleCI vs. GitHub Actions vs. GitLab CI/CD: Features Compared

	GitHub Actions	CircleCI	GitLab CI/CD
Managed (cloud service)	Yes	Yes	Yes
Self-hosted option	Yes	Yes	Yes
Internal VCS	No	Yes	Yes
Uses external VCS	Yes	No	No
Available on-premises	Yes	Yes	Yes
Real-time debugging of builds	Yes	No	No
Guided new project setup	Yes	No	No
Multi-step pipelines	Yes	No	No
External notifications (e.g., Slack)	Yes	Yes	Yes
Polyglot (multilingual)	Yes	Yes	Yes
Docker/Kubernetes support	Yes	Yes	Yes

Circle CI, GitHub Actions, or GitLab: Which CI Platform Is Best?

Selecting the best CI/CD solution can feel like a daunting task because there are lots of things to consider. All three options reviewed in this article will meet the needs of most developers who desire to improve (or create) the CI/CD pipelines that handle their builds, tests, and even deployments. These platforms all work with technology partners to add additional capabilities that make their solutions more robust than they could have achieved on their own. For example, see the documentation about integrations with Sauce Labs.

Choosing the CI platform that’s best for your organization really comes down to your goals. CircleCI is arguably the best option overall, as it is purpose-built with CI/CD pipelines as its core focus. On the other hand, many organizations find that GitLab and GitHub meet all their needs. In addition, these solutions often work better for organizations that aim to simplify their support structures by utilizing fewer vendors.

Vince Power

Published:

Jun 1, 2023

Topics

CI/CD

Continuous Testing

Share this post

Get started testing with Sauce Labs or request a demo today

Learn how to transform your CI/CD pipeline

EEA

CCPA

Cookie Settings