As software continues to eat the world, the need for robust, enterprise-grade applications has become increasingly pronounced. Global spend on enterprise software surpassed $780 billion in 2022. Slow-to-digitize industries like transportation and healthcare have begun to experience the drawbacks of mismanaged, legacy software, while tech-forward counterparts have cultivated a software-first mindset. Engineers and operators at such companies continue to look for ways to streamline their work. The surface area that newer classes of software address has expanded as a result, revealing gaps in traditional architectures and methodologies. The last decade has seen the advent and dissemination of microservices and containers, a growing contingent of cloud-native technologies, and an enduring move toward test-driven development.

There is a broad spectrum of available business software, including external, customer-facing applications, internal tools for business operations, and a plethora of underlying software systems and plumbing required to scalably run user-facing apps. Many companies have had to allocate additional resources towards the “platform” layer — that is, the foundational software to run other applications. This has materialized through a focus on infrastructure management, containerization and orchestration, and scalability across services. The purpose is to shift the developers’ energy back into what they do best: writing business-specific logic, rather than haggling with the intrinsic complexities of distributed systems and microservices. While the most critical workloads of any software company can often be the most complex, the need for systems to handle such intricacies has created an opportunity for a more robust orchestration solution.

Temporal Technologies is a microservices orchestration platform meant to simplify the process of building and maintaining complex or long-running application workflows. Temporal provides the engine to build and reliably run such workflows, allowing developers to focus on building the logic in steps (e.g. how to handle the case of insufficient funds in one's balance), rather than haggling with custom code to manage retries when services fail, failover cases for server outages, or handling of aberrant exception cases. Most steps in a user workflow of any software system depend on a variety of discrete functional steps, and ensuring resiliency against many potential exceptions would traditionally require deep knowledge of the intricacies of timeouts, failure cases, retries, and fallbacks. Temporal handles this background workflow logic for the developer through a suite of APIs and features to alleviate development.

Source: Temporal

Temporal Technologies was founded in 2019 by Maxim Fateev (CEO) and Samar Abbas (CTO). Fateev and Abbas met while working at Amazon Web Services (AWS) in 2009, where they were both involved in the development of Simple Workflow Service (SWF), a platform for coordinating long-running, asynchronous tasks. They saw firsthand the challenges that developers faced in building and managing complex distributed applications and were determined to create a scalable and more robust solution.

The two parted ways in 2014, as Abbas went on to join the Microsoft Service Bus team owning the messaging stack for Azure. At a time when enterprises were just starting to formally adopt microservice architectures, tools like Service Bus became a backbone for orchestrating calls across these services. Through work on both the Simple Workflow Services and Service Bus, Abbas used an internal hackathon to build a more durable orchestrator, now known as the Azure Durable Task Framework (DTF). During Abbas’s stint at Microsoft, Fateev moved on to Google to help build out the company’s big data frameworks.

In 2015, Uber opened a development center in Seattle, where the two reunited on the same team. With many internal teams continuing to spend the majority of time stitching together low-level messaging queues, the pair began their work on Cadence, an open-source workflow engine that built on the lessons learned from SWF and DTF. Cadence quickly gained popularity among Uber engineers and was adopted by dozens of adjacent engineering teams.

As adoption for Microservices continued to scale and the need for the event-driven workflow frameworks grew, the two realized an opportunity to further expand and commercialize their internal project. Built as a successor to Cadence, Fateev and Abbas co-founded Temporal Technologies in 2019 as an open-source project for the general developer community.

Fateev and Abbas have built out their team intentionally, attracting a group of leaders including Charles Zedlewski (COO) in 2021 to oversee product and operations, and Dave Cole (Head of Strategic Growth) to lead growth in September 2023.

Traditionally, software applications were built in a monolithic architecture that relied on a self-contained, singular code base that coupled the business logic together. As such applications grew, the complexity scaled, meaning that even minor regressions in one part of the codebase could impact the availability of the entire application. Microservices were introduced as a way of segmenting services and isolating dependencies. While each service could now operate as a separate application, this introduced microservice hell, a bloat of services somewhat reliant on dependencies yet operating independently.

The shift has influenced the broader software landscape, introducing new classes of concerns. As described by Temporal Technology’s Head of Product, Ryland Goldstein, in 2020:

For example, a simple payment service application might look something similar to the following:

Source: Sheepcode

While logic became more cleanly isolated across independent services, simple operations now depended on a suite of disparate services and dependencies, oftentimes owned and managed by separate teams. The ubiquity of such tasks has exacerbated the risk. As Temporal CEO Fatee expressed in 2020:

There is no one standard definition of a workflow, though it can broadly be defined as a “structured set of objectives or tasks to achieve a desired goal…. [that] provide the most natural way to describe how our processes should work for both humans and computers.” With even the most simple of applications representing a more complex system, no longer could teams rely on distributed system specialists and PhDs to oversee implementation.

With the standard implementation reliant on a mishmash of queues, databases, and ad-hoc services, Temporal abstracts the orchestration layer to a service, centralizing and standardizing the state and error handling.

Source: Youtube

Temporal Workflow Engine

To contextualize Temporal’s engine, one can imagine the workflow of sending a monthly reminder email to a trial user to upgrade their plan. While sending a monthly email is simple in theory, robust implementation involves a series of non-trivial steps, including the creation of a cron job for scheduling, a message queue for emailing, and database insertions for maintaining statefulness.

Temporal provides an abstraction layer over each of these steps, making the application code more resemblant to the pseudocode blueprint. A mirrored implementation with Temporal would look similar to the following:

Source: Temporal

Temporal’s platform is intentionally “code-first” and dependent on user-defined functions to enact an action (e.g. send the email), yet underpins the execution with a more durable runtime. Temporal Applications are comprised of a series of single-function executions, which can be run concurrently. They can communicate with their runtime environment through Activities and other workflow executions through Signals.

Temporal Architecture

The Temporal Architecture consists of a Temporal Server to manage and orchestrate work, worker nodes to perform tasks, and a persistent datastore for state management.

The platform is comprised of a few fundamental components:

1. Temporal Service: The core component of the Temporal platform. This manages the orchestration of distributed workflows, providing a reliable foundation for building complex, long-running applications. It can be thought of as the brains of workflows, keeping track of workflows, and activities, and coordinating worker execution.

2. Temporal Workers: Workers are the compute nodes that run application code. All workflows and activities are compiled into executables, which can be run on worker nodes as background processes. As workflows expand in scope and load, workers can also be scaled up to enable resiliency and scalability.

3. Data Storage: All workflow data, like activity logs and execution state, is stored in a persistent, external data store.

4. Temporal Web UI: A user-friendly web-based interface allows developers and operators to monitor, manage, and gain insights into their distributed workflows, making it easier to troubleshoot and optimize applications.

5. Temporal SDK: The Temporal SDK provides a comprehensive set of tools and APIs for connecting to and interacting with the Temporal Service.

Source: Mikhail Shilkov

Temporal Web UI. Source: Temporal

Temporal Workflows

By default, Temporal doesn’t execute application code at all.. Rather, it orchestrates the execution of code to drive workflows to completion. Temporal runs the execution of these workflows through a series of tasks, or the business logic handled by workflow applications. The temporal server records task inputs and outputs, ensuring that the workflows are resilient against failures, and errors are handled gracefully.

With all state management and queuing handled by the orchestration service, it allows for scalability, reliability, observability, and testing capabilities.

Source: Youtube

Most of the features provided by Temporal are abstractions over foundational resilience logic, allowing developers to more easily devise and implement the workflow blueprint as code. This includes native functionality for communicating with a Temporal Cluster, APIs to develop Workflows, and functionality to create and manage Worker Processes and Activities.

They also expose granular logic within the workflows themselves, including scheduling function calls, handling timeouts, collecting worker heartbeats, and logging activities through workflow execution.

Workflows are logged and surfaced in the Temporal Web UI, which provides developers mechanisms to clearly observe and debug executions.

Source: Temporal

Temporal also breaks down workflows through timeline views to clearly represent the flow of events in time as a workflow is executing.

Source: Temporal

Deployment Offerings

While Temporal does allow for on-premise deployment, it has shifted focus and encouraged customers to leverage its cloud environment, simplifying the tedious and error-prone task of deploying, upgrading, and maintaining the tech stack.

Source: Temporal

Community

Due to Temporal’s Open Source heritage, the team has been long reliant on users and developers to contribute and drive product improvements. Ryland Goldstein, Temporal’s Head of Product, joined as employee number five and immediately set out to bring more structure to serve the community. He stated that:

In addition to having messaged over 600 users on Slack to understand how they were using Temporal, he also introduced stronger community engagement through clearer release notes and a well-maintained blog. As end-users and increasingly purchasing decision-makers, its community continues to play a pivotal role in Temporal’s growth and strategy.

Customer

Despite the company only being five years old as of January 2024, Temporal has established a foothold in notable engineering-driven organizations looking to better manage their complex workflows. Notable customers of Temporal include Snapchat, Box, Stripe, and Netflix.

Source: Temporal

Temporal is a developer tool, primarily sold to platform engineers responsible for managing and building distributed systems. Developers are a notoriously fickle customer persona since they can often build tools for themselves.

Since the product is language agnostic and highly flexible, the use cases are horizontal by nature. Hashicorp uses it for infrastructure provisioning and automation, Stripe uses it to power Stripe Capital and Billing, and Coinbase uses it for managing millions of crypto transactions. Growth has been organic, with developers sharing successes and driving adoption. As former Datadog engineer Kevin Devroede mentioned:

Market Size

The market for global software development tools was estimated to be $4.4 billion in 2021 and is projected to reach $17.2 billion by 2031, representing a CAGR of 14.5%. Another estimate projects that the market could reach as much as $234.7 billion by 2028. The market's growth is being driven by various factors, including the establishment of microservices as a defacto architecture, the pace of cloud adoption, and the evolution of distributed computing requirements.

The workflow engine market is crowded, with a plethora of open-source alternatives, commercial counterparts, and an ever-present option to build in-house. Due to the far-reaching nature of workflows (service orchestration, batch jobs, business processes, ETL tasks), many competitors have tailored their solutions to address a more specific class of use cases. Still, the two most common buckets of competitors include larger-scale commercial offerings and more mature open-source products.

Commercial Workflow Engines

Amazon Step Functions: First launched in 2016, AWS Step Functions also aims to solve the coordination of distributed applications and microservices using visual workflows. The tool provides a graphical console to arrange components of the workflow as a series of steps and enable automatic retries and auto-triggers of subsequent actions. Charges are based on the number of state transitions or steps, required to execute the application.

Step Functions does not allow for implementing business logic with pure code, instead having the developer create blocks that call on separate Lambda functions and AWS services. Due to its tight coupling to the AWS ecosystem, teams already embedded in the AWS service stack may find Step Functions to be the natural workflow executor of choice. That said, Temporal’s offering does provide additional feature sets like communication with running workflows and full control over the individual workflow activities.

Source: Amazon

Google Cloud Workflows: Google entered the workflow engine space with its commercial Workflows product in 2020. The product provides similar services as that of AWS Step Functions, including a fully managed, state-based orchestration platform and visual workflow builder. Workflows can be described in either YAML (”Yet Another Markup Language”) or JSON (JavaScript Object Notation) and are meant to provide simplicity and ease of use. They also provide a visual interface to define steps and allow for simple calls to Google services for running functions and orchestrating external Google services. Since Cloud Workflows are relatively new, there is limited extensibility compared to the customization options available with Temporal.

Source: Google

Open-Source Workflow Engines

Apache Airflow: First released in 2014, Airflow is an open-source workflow orchestration to programmatically author, schedule, and monitor workflows. It is built off the concept of a directed acyclic graph (DAG) to represent the dependencies across tasks, with tasks executed as a workflow process, separating the process from the scheduler. While Temporal has indexed on flexibility and speed of iteration, Airflow has expressed that it focuses on workflows that are mostly static and slowly changing.

Such workflows are most commonly used for data processing, though Airlfow doesn't recommend running data-intensive operations through the defined workflow (and instead delegating those tasks to external services). Due to its native pre-built integrations, Airflow may be preferred when working with a number of external services and APIs.

Source: GitHub

Prefect: Released as a free, open-source solution in 2017, Prefect is a Python-based orchestrator for running data-intensive workflows. Prefect is meant to be dynamic through workflows as code and an API to interact with the Prefect server. Prefect is self-described as the orchestration tool for data and ML engineers, with a focus on data-intensive workflows. Through the concept of decorators, Prefect allows for features like retries, distributed execution, scheduling, and caching.

Source: Prefect

As of June 2021, Temporal’s only revenue stream was through its cloud service and support packages. As of January 2024, it charges via a consumption model, meaning customers only pay for what they need when they need it. Pricing is derived from the use of actions, storage, and support.

Actions are the fundamental consumption unit in Temporal. Each time a workflow is executed, associated actions are collected and represent the state and progress of the Temporal application. The company charges $25 per 1 million actions and prorates additional overage multiples. Storage costs include running charges for active Workflows and retained storage for closed workflows’ event histories. Finally, Temporal provides success and support plans based on needs. Premium plans allow for expedited response times and escalation paths. Temporal also provides a monthly cost estimator to understand expected spend.

Most of Temporal’s costs come from running compute and storage for customers on its cloud offering. While $25 for a million actions may seem inexpensive, workflows can scale quickly, with individual, lightweight workflows dependent on thousands of actions.

While it has not disclosed revenue publicly, Temporal has seen substantial organic adoption. As of its round of funding in early 2023, the company reported having grown revenue and customers by 20x since the previous round in 2022 and experienced 25-50% month-over-month growth in adoption. Over 50K new developers joined Temporal’s open-source community in the year leading up to February 2022, and over 300 new customers signed up over the same period.

The company has also continued to find traction in enterprise customers, with customers like Datadog running over 3 million workflows per month and Netflix amassing 500K daily workflows. With Temporal having invested most of its resources in product and engineering, the majority of its growth has been organic.

In February 2023, Temporal Technologies raised a $75 million Series B extension, following just a year after a $100 million Series B in February 2022 that valued the company at over $1.5 billion. Notable investors included Sequoia Capital, Amplify Partners, and Index Ventures. Temporal reportedly maintained a similar valuation in its Series B extension in the face of a downturn in the tech market. The round brought its total funding raised to $203.5 million.

Increasing Prevalence of Microservice Architectures

As microservices continue to grow in demand and market value, companies will repeatedly run into the same issues that come with technically complex distributed systems. Temporal can continue to capitalize on this market through strong developer evangelism and a product suite tailored toward the developer. Temporal can capitalize on this trend through a keen awareness of the changing microservice landscape and requirements, and establish the product as a mainstay of company infrastructure operations.

Expand Product Suite

Temporal has established itself as more than just a simple workflow orchestration platform and has cultivated an ardent developer community. As it continues to develop its footprint in the enterprise market, it can further build out the abstraction layer, improving granular features that are not natively part of most workflow engines (e.g. rate limiting, heartbeat activities, queries, etc.) and increasing accessibility for less technically proficient teams. Ultimately, Temporal could look to expand outward and develop a strong multi-product offering, while continuing to double down on its core workflow offering.

Evangelism and Education

Temporal began to serve a specific need in a niche in a highly technical space. While engineers who work on the platform side of distributed systems grasp the criticality of a tool like Temporal, it must evangelize its offering to the broader developer community and main buyers. It’s already begun to invest in this through thought leadership and best practices. In order to move upmarket and continue to drive growth, it must articulately tell the story of how and why Temporal fits into a team’s engineering stack.

Enterprise Readiness and Accessibility

Temporal began as an open-source developer platform and has more recently moved to a commercial model based on Temporal Cloud. With its platform serving as the underpinning for mission-critical workflows, it must be certain of the performance and execution reliability for even the most complex of workflows. Additionally, it must continue to meet developers where they are. The product is technical by nature, which can create friction for teams that don’t have the technical implementation resources. Trust is also essential for any software vendor, and with Temporal existing as the foundation for operational workflows, it must ensure it meets the growing needs of enterprise customers.

Unclear Value Proposition

While Temporal has seen success across most of its primary customer base, the product itself and core value proposition can be murky for non-developer teams. Temporal solves specific problems well, which requires proper customer education and value selling. The flexibility of Temporal slots it into multiple categories, including a workflow engine, application state manager, and orchestration platform. Unless it can clearly identify and articulate the core value proposition and financial implications of running workflows on Temporal, there is a risk of not fully capturing open market opportunities.

Temporal is still in the early stages of growth and development, but has established itself as a major player in the orchestration and tooling space. It has found success with highly adept engineering organizations, and its extensive experience in the industry has allowed for unique perspectives on the problems to be solved, right at the time when microservices and complex workflows have become more mainstream. As startups and larger enterprises continue to expand their application suites, they will have broader needs for the software they run, which should help Temporal continue to drive adoption.