When I first joined the platform engineering team at a large enterprise insurance company, one of the most common complaints I heard from developers was deceptively simple: “I don’t know where anything is.” They didn’t mean they’d lost their code. They meant the sprawl — dozens of tools, wikis, dashboards, and ticketing systems — had become so unwieldy that even experienced engineers spent significant chunks of their day just navigating the organisational maze to get things done.
That frustration became the catalyst for what would eventually become our Internal Developer Portal (IDP), built on Spotify’s open-source Backstage framework. This post captures the journey — the decisions we made, the problems we solved, and the lessons we learned along the way.
The Problem: Death by a Thousand Tabs
Before the portal, a typical developer’s workflow for spinning up a new microservice looked something like this:
- Raise a Jira ticket requesting a new Git repository
- Wait for a platform team member to manually create it (often 1–2 business days)
- Raise another ticket for a CI/CD pipeline
- Raise yet another ticket for secrets to be provisioned in the vault
- Manually configure environment variables, often by copying from another service and hoping the values were still correct
- Ask a colleague on Microsoft Teams where the Kubernetes dashboard was, because the bookmark had changed again
This wasn’t just slow — it was demoralising. Developers are creative problem-solvers by nature, and we were forcing them to spend their energy on bureaucratic overhead instead of building features. The cognitive load was enormous, and onboarding new team members was particularly painful. A new joiner could easily spend their first two weeks just figuring out how to get a development environment running.
We needed a single pane of glass — a place where developers could discover services, access documentation, provision infrastructure, and get on with their actual work.
Why Backstage?
We evaluated several options, including building a custom portal from scratch, using commercial platforms, and adopting Backstage. Each had trade-offs, and we spent considerable time evaluating the leading contenders.
Building from scratch would have given us maximum flexibility, but we were a small platform team with ambitious goals. We didn’t have the bandwidth to build and maintain a full portal framework while also delivering on our platform roadmap.
Among commercial platforms, we evaluated Port and Cortex, but the pricing and licensing costs to onboard either at enterprise scale made them prohibitive for our organisation.
Backstage hit the sweet spot. As a CNCF open-source project, it came with no licensing costs and a highly extensible plugin architecture backed by a vibrant and growing community. Being self-hosted gave us full control over our data — an important consideration in a regulated industry. Critically, it came with a solid foundation — the software catalog, TechDocs integration, and a template system — that we could build on rather than reinvent. The ability to build custom plugins tailored to our specific platform needs, combined with the fact that Spotify had built it to solve a similar problem at scale, gave us confidence that the architecture would hold up.
One thing I want to be honest about: Backstage is not a turnkey solution. Out of the box, it’s a framework, not a finished product. You need to invest engineering effort to make it useful for your specific organisation. But that investment pays dividends because the result is a portal that fits your workflows rather than forcing you to adapt to someone else’s.
Architecture Decisions
We deployed Backstage on Azure Kubernetes Service (AKS), which was already our primary compute platform. This was a deliberate choice — running the portal on the same infrastructure we managed meant we could apply the same operational practices (monitoring, alerting, scaling) without introducing a separate operational domain.
For the backend, we used PostgreSQL as the catalog database, hosted on Azure Database for PostgreSQL. The software catalog is the heart of Backstage, and we wanted it backed by something robust and familiar.
Authentication was integrated with Azure Active Directory via the Backstage Azure Auth provider. This was non-negotiable in an enterprise context — single sign-on and role-based access control aren’t nice-to-haves, they’re table stakes.
We also made an early decision to use Backstage’s scaffolder (software templates) heavily. The scaffolder became our primary mechanism for self-service — rather than building custom UIs for every provisioning workflow, we defined templates that collected inputs from developers and orchestrated the underlying automation.
Self-Service Capabilities We Built
Repository Creation
The first template we built was for creating new Git repositories. It sounds simple, but the details matter. Our template didn’t just create a repository — it:
- Created the repo in Azure DevOps with the correct naming convention
- Applied branch policies (requiring pull request reviews, build validation)
- Seeded the repo with a standardised project structure based on the selected tech stack (Java Spring Boot, Node.js, Python)
- Created an initial
README.mdwith team ownership information - Registered the new service in the Backstage software catalog automatically
What previously took 1–2 days and multiple tickets now took about 90 seconds.
CI/CD Pipeline Provisioning
Creating pipelines was another major pain point. Our legacy setup involved Jenkins, and pipeline configuration was a mix of Groovy scripts, XML, and tribal knowledge. When we migrated to Azure DevOps, we took the opportunity to standardise.
Each repository template included a pre-configured azure-pipelines.yml that followed our golden path — build, test, scan, containerise, and deploy. The pipeline template was parameterised so that teams could choose their deployment targets (dev, staging, production) and configure environment-specific variables through the Backstage UI.
We also built a separate template for teams that needed to create pipelines for existing repositories. This template would inspect the repo, detect the tech stack, and generate an appropriate pipeline configuration.
Secrets Management
Secrets were one of the trickiest areas. We used Azure Key Vault for secrets storage, but the process for provisioning and rotating secrets was entirely manual. Developers would raise a ticket, a platform engineer would create the secret in the vault, and then somehow communicate the secret name back to the developer.
We built a Backstage template that allowed developers to provision secrets directly into the appropriate Key Vault, scoped to their service and environment. The template enforced naming conventions and access policies, so secrets were automatically accessible to the correct Kubernetes pods through our CSI driver integration. No tickets, no waiting, no miscommunication.
Microservice Catalog
The software catalog became the single source of truth for “what do we have and who owns it?” Every microservice, library, and infrastructure component was registered with metadata including:
- Team ownership and contact information
- API documentation (auto-generated from OpenAPI specs)
- Dependencies (upstream and downstream)
- Deployment status across environments
- Links to monitoring dashboards, logs, and runbooks
This was transformative for incident response. When something broke at 2 AM, the on-call engineer could look up the affected service in the catalog, find the owning team, see the recent deployments, and access the relevant dashboards — all from one place.
Developer Onboarding
We created an onboarding template that provisioned everything a new developer needed on their first day:
- Access to the relevant Azure DevOps projects
- Membership in the appropriate team channels
- A curated list of documentation and getting-started guides
- A pre-configured development environment specification
New joiners went from spending two weeks getting set up to being productive within a day or two. That’s not an exaggeration — we measured it.
Adoption Challenges
Building the portal was the easy part. Getting people to use it was harder.
The “Just Let Me SSH In” Crowd
Some senior engineers had been doing things their way for years and didn’t see the need for a portal. They had their scripts, their bookmarks, and their muscle memory. Forcing them onto the portal would have been counterproductive.
Instead, we focused on making the portal so useful that it became the path of least resistance. When provisioning through the portal was genuinely faster than the manual alternative, adoption happened organically. We also made sure that the portal didn’t remove capabilities — it added a faster path while leaving the manual option available for edge cases.
Data Quality in the Catalog
The software catalog is only as useful as the data in it. Early on, we had services registered with missing ownership information, outdated descriptions, and broken links. This eroded trust — if developers looked up a service and found stale data, they’d stop trusting the catalog entirely.
We addressed this by implementing catalog health checks. A weekly automated report flagged services with missing or outdated metadata, and we worked with team leads to make catalog accuracy part of their team’s definition of done. It took a few months, but data quality improved steadily.
Plugin Maintenance
The Backstage ecosystem moves fast. Plugins get updated, APIs change, and keeping everything in sync requires ongoing effort. We dedicated a portion of each sprint to Backstage maintenance — upgrading core packages, testing plugin compatibility, and addressing deprecations.
This is something I’d encourage anyone adopting Backstage to budget for explicitly. It’s not a “build it and forget it” platform. Treat it like a product with its own roadmap and backlog.
Measuring Success
We tracked several metrics to assess the portal’s impact:
- Time to first commit for new services: Dropped from an average of 3 days to under 2 hours
- Onboarding time for new developers: Reduced from ~10 days to 1–2 days
- Ticket volume for platform provisioning requests: Decreased by approximately 70% within six months
- Catalog coverage: Grew from 0% to 85% of all services registered within the first year
- Developer satisfaction (quarterly survey): Scores improved significantly after portal launch, with qualitative feedback highlighting the portal as one of the most impactful platform investments
The metric I’m most proud of, though, is harder to quantify: the number of developers who started contributing templates back to the portal. When developers go from being consumers of the platform to contributors, you know you’ve built something that resonates.
Lessons Learned
Start with one killer use case. Don’t try to boil the ocean. We started with repository creation because it was a universal pain point and the quickest win. That initial success built credibility and momentum for everything that followed.
Treat the portal as a product, not a project. A portal isn’t done when you launch it. It needs a product owner, a backlog, and regular investment. The moment you stop iterating, it starts decaying.
Invest in documentation — especially for template authors. We wanted other teams to build their own templates, but the learning curve was steep without good documentation. Creating comprehensive guides and example templates accelerated adoption significantly.
Don’t underestimate the cultural shift. An IDP changes how developers interact with infrastructure. That’s a cultural change as much as a technical one. Communicate the “why” clearly and consistently, and be patient with adoption curves.
Integrate with what you have, don’t replace everything. Backstage isn’t a replacement for your existing tools — it’s a layer on top. We integrated with Azure DevOps, Key Vault, Kubernetes, and Confluence rather than trying to replicate their functionality.
What’s Next
We’re continuing to expand the portal’s capabilities. On the roadmap are cost visibility dashboards (showing per-service cloud spend), automated security posture reporting, and deeper integration with our incident management workflow. We’re also exploring Backstage’s newer features like the permissions framework for more granular access control.
Building an Internal Developer Portal with Backstage has been one of the most impactful projects I’ve worked on in my career. It’s not just about the technology — it’s about removing friction and giving developers back their time to do what they do best: build great software.
If you’re considering building an IDP, my advice is simple: start small, iterate fast, and listen to your developers. They’ll tell you exactly what they need.