This is a guest post from Hubert Nimitanakit, infrastructure engineer at Marqeta.
Marqeta (NASDAQ: $MQ), a modern card issuing platform founded in 2010, enables businesses to create and manage payment cards through open APIs. With over 750 employees, and industry-leading customers that depend on its services, Marqeta grew quickly to process $222 billion in 2023. To support this growth while enabling compliance and security, Marqeta needed to implement secure processes for internal operations that minimize risks associated with manual changes.
One focus of ours in the Infrastructure org at Marqeta is providing secure technology to enhance productivity across our company. We realized that Marqeta could benefit from a development platform that aided developers in quickly building secure applications to speed up manual processes.
Here, I will share how we adapted our internal development patterns to Retool, our chosen platform for flexibility, speed, and the ability to meet our stringent requirements.
Using a high-level development platform like Retool could streamline frontend development and free up engineering resources for core product work. However, its feasibility for Marqeta hinged on its ability to meet specific compliance standards, such as PCI and SOX.
The challenge for Marqeta was to align Retool with its S-SDLC, incorporating CI/CD processes, Git workflows, and AWS infrastructure, while also implementing change management controls for apps affecting production. We sought to combine the speed, consistency, and ease of use of a higher-order platform like Retool with the flexibility to shape the SDLC to fit our exact needs.
Some of the key requirements of Marqeta's S-SDLC were as follows:
- Predictable and safe deployments: The deployment pipeline focuses on safe and predictable software releases, adhering to compliance requirements utilizing multiple quality control gates.
- Human and non-human interactions: Identical results are expected for both human-initiated and automated interactions, maintaining consistency.
- Rapid deployment: The setup enables rapid, gated deployment to live environments, facilitating quick and secure software releases.
- Environment segmentation: Hard segmentation of environment tiers (dev, qa, prod) at both data and network levels are maintained, enabling security and compliance.
To help these challenges, we deployed self-hosted Retool with a multi-instance architecture and leveraged our Git-based SDLC to extend Retool's integration capabilities.
Our team deployed one instance per environment (AWS account corresponding to dev, qa, and prod). This setup enabled a clear separation of environments and compliance adherence, as the accounts have hard data and network segmentation between them.
Our goal was to enable isolated, staged development processes and repository management. Retool only supports one repository associated with an instance or Space, so we tailored our existing deployment processes and internally built CI/CD tooling to work within Retool’s Source Control capabilities.
We leveraged Retool’s Spaces feature to isolate teams and environments. Our automation creates three Spaces and three repositories per development team, for dev, qa, and production, respectively. For each environment, the team-specific, environment-specific Space is integrated with a corresponding repository (i.e., a team’s dev Space is linked with the dev repository in that team’s Github Organization). To get around the one-Space-to-one-repo limitation, we leveraged our CD tooling to copy a zip artifact of the dev repo to the matching qa and production repos progressively (with a manual approval stage for promotion to production). Our CI/CD tooling also manages change approvals and audit documentation as a part of the deployment process.
Here's our Retool SDLC workflow:
- Teams develop their Retool apps in the dev Space and submit PRs to the main branch in the dev repository.
- Upon merging, an OCI artifact (the zipped version of their dev environment) is automatically created.
- The artifact is then copied to the qa repository, which upon merge, syncs the changes to the team’s qa Space. Testing is completed before deploying to production.
- Deployment to production is gated by a manual approval. Once approved, the artifact is merged into the production repository, syncing with the production Space. We implemented controls specifically around auditability, and trace of approvals of changes made to production consists of PR approvals within GitHub and automated logging to our source of truth which is JIRA Change Management project.
Marqeta’s internal tooling for its traditional software development ensures that every code change to production adheres to certain regulatory and technical compliance requirements for approvals and auditability. We achieved the same adherence for Retool app executions by creating a Retool Change Management module. This module integrates with Jira to manage Change approval and logging. To deploy a Retool app to production, that app must be integrated with this Change module.
The Change module provides a UI to orchestrate the change management process, handling approval, execution, as well as automated rollback (in a manner similar to reverting a git commit). If the changes are approved, the changes are automatically executed (or manually triggered if configured). The Change module will log the relevant audit information to the Change ticket along with whether the Change was successful by updating the ticket status to Done or Failed.
The Retool apps we've built act as a bridge between business users and our APIs, providing secure custom interfaces that align with the specific requirements of each use case. Non-technical users are now empowered to quickly and safely perform tasks that once required the involvement of development teams or posed incident risks from manual change.
Our production support team at Marqeta manages incidents, feature requests, and changes for our customers’ configurations. Previously, we did this in a manual fashion that was inefficient and prone to errors.
To solve this, we built a Retool app that automates manual steps with an easy-to-use interface. It provides a secure way for the Production Support team to work faster, with access controls, MFA, and robust error handling built in.
This ensures all changes to customer production environments are approved, logged, tested, and secure by default.
Our GTM Integration Engineering team is using Retool to significantly decrease the time it takes to onboard a new customer and thereby accelerate time-to-value.
One part of the onboarding process involves provisioning customer infrastructure in AWS and then configuring it to complex requirements.
Previously, the process took up to three weeks and depended on two engineering teams. Today, provisioning and configuration take just one to three days.
At Marqeta, we’ve successfully integrated our complex S-SDLC with Retool app development. This setup was designed to meet industry standards, while streamlining development processes, maintaining stringent security controls, and critically, reducing the number of production incidents due to manual change.
Retool allowed us to build internal tools without straining development resources or headcount allocations—or pulling engineers off critical work.
By empowering business users to make customer program changes directly in Retool, we’re achieving increased platform reliability, faster turnaround, improved efficiency, and greater agility in responding to customer needs.
Reader