How Hotjar deploys 50% faster with GitLab

Hotjar is a behavior analytics software that makes it easy to go beyond traditional web analytics and to understand what users are really doing on a website. With both quantitative and qualitative behavior analytics tools, Hotjar provides an overview of how to improve customer experience, performance, and conversion rates.

The company was formed in 2014 and is fully remote, with over 100 people across 20 countries. Over 500,000 sites use Hotjar worldwide, with most customers within the United States and Europe. Word of mouth has been the backbone of the marketing strategy, driving the majority of the customer base.

Because Hotjar is fully remote, communication is one of the biggest challenges. As the company grows, communication becomes increasingly important. “The communication between 100 versus 18 people is incredibly different. There’s a lot more information now. So I think that there’s an element of trying to make sure that we have the information available to the team as they need it, but without drowning everybody in information which is, at points, irrelevant,” said Sara Bent, People Ops specialist at Hotjar.

Hotjar had challenges with how the growing number of developers structured their work and the cumbersome legacy systems in place. All the developers had to work on the same code base using legacy tools, which was slowing productivity. “The problem that we were trying to solve was how to go from this to a more diverse set up, where we have a few infrastructure legacy pieces, but also some new microservice deployments and how to integrate all the new things in a nice way so that we could continue to grow the number of developers internally without productivity going down,” according to Vasco Pinho, Team Lead, SRE at Hotjar.

Developers were using BitBucket for hosting source code and Jenkins for CI/CD. Due to the constraints of some of the legacy applications, they had to develop and maintain large amounts of Jenkins-specific code to support pipelines. This used up a lot of resources and was not an effective use of their time. They were using Kubernetes as a platform for all their microservices and some of the build pipelines.

Hotjar was looking for a tool that offers a Kubernetes integration and a replacement for Jenkins CI/CD. They tried using BitBucket for CI/CD and Concourse, but neither provided the solution they needed. “In terms of Kubernetes native product that supplies the whole life cycle, we actually didn’t find that many competitors. There were some that we looked at, some that we had to self host. Jenkins X is Kubernetes native but we actually found it to be super immature and it still had a lot of bugs,” Pinho said.

Hotjar set up GitLab, ran it locally, and found through the trial that they landed on the right tool. “While our pain points started a search for a replacement to Jenkins, once we tried GitLab’s pipelines, we saw the way they fully integrated the development experience into a single tool, as well as fulfilling all our CI/CD requirements, we started considering moving our code into GitLab as well,” Pinho said. “Our developers considered GitLab’s interface and features to be superior to Bitbucket, namely the code review flow and as such we embraced the whole platform.”

With Jenkins, the teams created a lot of custom codes to do a lot of the work that they are now getting natively with GitLab. On the code management side, they used the cloud version of Bitbucket. Now, they use GitLab.com for all of the development work and to host the CI/CD runs. “It is great and it’s not something that a lot of other providers offer. Being able to not have to worry about hosting the actual website, but still allowing us to have the flexibility to run our own build infrastructure,” Pinho said.

Part of the flexibility that Hotjar wants to maintain is the ability to work remotely, and do it well. “[The founders] worked remotely from the outset and therefore had to establish all of these processes around working remotely. And that’s done a huge job towards making sure we have this very firm foundation on making sure that our remote communication and collaboration works well. We pay conscious effort to this,” Bent says. GitLab’s end-to-end visibility of the workflow process allows developers to see what everyone is working on at all times. The access of information helps to maintain Hotjar’s remote communication.

Every engineering team and some of the customer support team members are using GitLab. Users are happy with the MR process. The overall user experience is a big improvement over BitBucket and the review environments are a game changer for any work that requires visual review. With the review environment, they can catch errors earlier in the pipeline and not hit the stage environment as they previously did.

GitLab integrates natively with Kubernetes, which gives the development team peace of mind because they can trust that the tool will work automatically without constant maintenance. The team previously had issues using Jenkins because a lot of time was spent “gluing things together,” according to Pinho. Developers are now able to focus on production, rather than bugs and fixes. Build CI time has decreased by 30% over previous implementation in Jenkins. Build CI variability has decreased significantly, due to higher density of builds per node and less scale up operations needed.

Hotjar wanted to stay away from solutions where they had to host the source code management and their own workflows. Now, their teams are able to focus on their priority tasks and the main line of business, instead of trying to figure out why Git repositories are down. With GitLab, Hotjar is able to see which updates are in the planning process. “It was definitely something that was impressive. During the trial I think there were two releases. So we got issues that we were finding during the trial fixed in the releases that came out during the trial,” Pinho said.

GitLab projects connect to their AWS EKS cluster, the tests run within the cluster using Kubernetes Operator, it reports back with coverage results, then artifacts are uploaded to AWS ECR/S3. Review environments spin up inside the EKS cluster during review. After the merge, artifacts are deployed again to the EKS to the production environment. “We went from the deployment itself to production that in our old system was taking something like eight minutes and is now gone down to something like four, so half the time to deploy. Within the whole incident resolution scheme, this sounds like very little, but it still impacts things across the board,” Pinho said. Developers are saving time and effort by making use of standalone review-environments instead of in-the-loop shared staging environments, which create bottlenecks.

Hotjar’s remote culture continues to thrive. “Most people are online at the same time, you can expect an MR to be reviewed within hours or minutes, and be able to jump on a video call together to debug issues, discuss projects, etc. We definitely don’t suffer from the usual pains of fully remote offices where any query will only be answered by the next morning,” Pinho said.

GitLab improves Hotjar’s development collaboration capabilities, but also influences how transparent they are. “I think often we see the similarities and it’s a reassuring thing of like, ‘Okay, they’re doing that too and that’s working really well for both of us, so well done,’” Bent said.

Hotjar recently published their handbook after seeing that GitLab’s handbook is public. “It’s nice to be able to see what other companies are doing and even just how people word things and how that reflects their company. But it’s true we didn’t use to have a public team manual and now we do,” Bent added. GitLab’s integrated platform helps to keep Hotjar up to date with cutting-edge software and inspires all-remote culture modernity.