Platform Engineering represents the evolution of DevOps, transforming how organizations deliver infrastructure and services to development teams. Internal Developer Portals (IDPs) are the cornerstone of this transformation, providing self-service capabilities that eliminate bottlenecks, reduce cognitive load, and accelerate delivery. This comprehensive guide provides everything you need to build, deploy, and scale enterprise-grade IDPs using Backstage, Port, Humanitec, and custom solutions.
The shift from DevOps to Platform Engineering is not optionalโit's an evolutionary necessity. As organizations scale, the traditional DevOps model creates unsustainable cognitive load on developers. Platform Engineering with IDPs abstracts complexity while maintaining flexibility, enabling developers to focus on delivering value rather than managing infrastructure. This guide covers everything from basic concepts to advanced multi-cloud implementations.
๐ Platform Engineering Landscape 2025
๐ Table of Contents
- Why Platform Engineering is Critical
- Internal Developer Portal Fundamentals
- Complete Backstage Implementation
- Port.io Enterprise Setup
- Humanitec Platform Orchestration
- Building Custom IDPs
- Golden Path Templates
- Service Catalog Management
- Scorecards and Governance
- Multi-Cloud Integration Patterns
- Platform Metrics and Observability
- Enterprise Architecture Patterns
๐ค Why Platform Engineering is Critical
Platform Engineering addresses the scalability crisis in modern software delivery. As organizations grow, developers face increasing cognitive load from infrastructure complexity, tool sprawl, and operational responsibilities. This leads to decreased productivity, increased errors, and slower time-to-market.
Platform Engineering solves these challenges by:
- Abstracting Complexity: Hide infrastructure details behind intuitive interfaces
- Standardizing Workflows: Create golden paths that encode best practices
- Enabling Self-Service: Empower developers to provision resources without tickets
- Reducing Cognitive Load: Let developers focus on code, not infrastructure
- Improving Governance: Enforce policies and compliance automatically
Platform Engineering Maturity Model
# Platform Engineering Maturity Assessment
Level 1 - Ad Hoc:
- Manual infrastructure provisioning
- No standardized workflows
- High cognitive load on developers
- Frequent production incidents
Level 2 - Managed:
- Some automation with scripts
- Basic CI/CD pipelines
- Shared documentation
- Reactive incident response
Level 3 - Defined:
- Infrastructure as Code adoption
- Standardized deployment processes
- Service catalogs emerging
- Proactive monitoring
Level 4 - Quantified:
- Full self-service capabilities
- Golden paths implemented
- Metrics-driven decisions
- Automated compliance
Level 5 - Optimized:
- AI-powered recommendations
- Predictive scaling
- Zero-touch operations
- Continuous optimization ๐๏ธ Internal Developer Portal Fundamentals
An Internal Developer Portal is the user interface to your platform, providing a unified experience for discovering, creating, and managing services and infrastructure.
Core Components of an IDP
IDP Architecture Components
# idp-architecture.yaml
components:
service_catalog:
description: "Central registry of all services, APIs, and resources"
features:
- Service discovery
- Ownership tracking
- Documentation hub
- API specifications
- Dependency mapping
software_templates:
description: "Scaffolding for new services and resources"
features:
- Golden path templates
- Best practices enforcement
- Automated provisioning
- Compliance checks
tech_docs:
description: "Centralized technical documentation"
features:
- Markdown support
- API documentation
- Runbooks
- Architecture diagrams
scorecards:
description: "Service health and compliance tracking"
features:
- Production readiness
- Security compliance
- Performance metrics
- Cost optimization
workflows:
description: "Automated processes and pipelines"
features:
- CI/CD integration
- Approval workflows
- Incident management
- Change management ๐ญ Complete Backstage Implementation
Backstage, created by Spotify and now a CNCF project, is the leading open-source platform for building developer portals. Here's a production-ready implementation.
1. Backstage Deployment on Kubernetes
Production Backstage Kubernetes Deployment
# backstage-deployment.yaml
apiVersion: v1
kind: Namespace
metadata:
name: backstage
---
apiVersion: v1
kind: Secret
metadata:
name: backstage-secrets
namespace: backstage
type: Opaque
stringData:
GITHUB_TOKEN: ${GITHUB_TOKEN}
AUTH_GITHUB_CLIENT_ID: ${GITHUB_CLIENT_ID}
AUTH_GITHUB_CLIENT_SECRET: ${GITHUB_CLIENT_SECRET}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: backstage
namespace: backstage
spec:
replicas: 3
selector:
matchLabels:
app: backstage
template:
metadata:
labels:
app: backstage
spec:
containers:
- name: backstage
image: backstage:latest
imagePullPolicy: Always
ports:
- name: http
containerPort: 7007
envFrom:
- secretRef:
name: backstage-secrets
env:
- name: POSTGRES_HOST
value: postgres-service.backstage.svc.cluster.local
- name: POSTGRES_PORT
value: "5432"
resources:
requests:
memory: "512Mi"
cpu: "500m"
limits:
memory: "1Gi"
cpu: "1000m"
livenessProbe:
httpGet:
path: /healthcheck
port: 7007
initialDelaySeconds: 60
periodSeconds: 10
readinessProbe:
httpGet:
path: /healthcheck
port: 7007
initialDelaySeconds: 30
periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: backstage
namespace: backstage
spec:
selector:
app: backstage
type: LoadBalancer
ports:
- name: http
port: 80
targetPort: http
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: backstage
namespace: backstage
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
tls:
- hosts:
- portal.company.com
secretName: backstage-tls
rules:
- host: portal.company.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: backstage
port:
number: 80 2. Backstage Configuration
app-config.production.yaml
# app-config.production.yaml
app:
title: Company Developer Portal
baseUrl: https://portal.company.com
support:
url: https://github.com/company/backstage/issues
items:
- title: Issues
icon: github
links:
- url: https://github.com/company/backstage/issues
title: GitHub Issues
organization:
name: Company
backend:
baseUrl: https://portal.company.com
listen:
port: 7007
csp:
connect-src: ["'self'", 'http:', 'https:']
cors:
origin: https://portal.company.com
methods: [GET, POST, PUT, DELETE]
credentials: true
database:
client: pg
connection:
host: ${POSTGRES_HOST}
port: ${POSTGRES_PORT}
user: ${POSTGRES_USER}
password: ${POSTGRES_PASSWORD}
database: backstage
ssl:
require: true
rejectUnauthorized: false
cache:
store: memory
reading:
allow:
- host: github.com
- host: '*.company.com'
integrations:
github:
- host: github.com
token: ${GITHUB_TOKEN}
gitlab:
- host: gitlab.company.com
token: ${GITLAB_TOKEN}
azure:
- host: dev.azure.com
token: ${AZURE_TOKEN}
proxy:
'/jenkins':
target: 'https://jenkins.company.com'
changeOrigin: true
headers:
Authorization: ${JENKINS_TOKEN}
'/argocd':
target: 'https://argocd.company.com'
changeOrigin: true
secure: false
'/prometheus':
target: 'https://prometheus.company.com'
'/grafana':
target: 'https://grafana.company.com'
techdocs:
builder: 'external'
generator:
runIn: 'docker'
publisher:
type: 'awsS3'
awsS3:
bucketName: 'company-techdocs'
region: ${AWS_DEFAULT_REGION}
credentials:
roleArn: ${AWS_ROLE_ARN}
auth:
environment: production
providers:
github:
production:
clientId: ${AUTH_GITHUB_CLIENT_ID}
clientSecret: ${AUTH_GITHUB_CLIENT_SECRET}
google:
production:
clientId: ${AUTH_GOOGLE_CLIENT_ID}
clientSecret: ${AUTH_GOOGLE_CLIENT_SECRET}
okta:
production:
clientId: ${AUTH_OKTA_CLIENT_ID}
clientSecret: ${AUTH_OKTA_CLIENT_SECRET}
audience: ${AUTH_OKTA_AUDIENCE}
authServerId: ${AUTH_OKTA_AUTH_SERVER_ID}
scaffolder:
defaultAuthor:
name: Backstage Scaffolder
email: scaffolder@company.com
defaultCommitMessage: 'Initial commit from Backstage Scaffolder'
catalog:
import:
entityFilename: catalog-info.yaml
pullRequestBranchName: backstage-integration
rules:
- allow: [Component, System, API, Resource, Location, Template]
locations:
- type: url
target: https://github.com/company/backstage-catalog/blob/main/catalog-all.yaml
- type: github-discovery
target: https://github.com/company/*/blob/main/catalog-info.yaml
- type: github-org
target: https://github.com/company
providers:
github:
company:
organization: 'company'
catalogPath: '/catalog-info.yaml'
filters:
branch: 'main'
repository: '.*'
schedule:
frequency: { minutes: 30 }
timeout: { minutes: 3 }
kubernetes:
serviceLocatorMethod:
type: 'multiTenant'
clusterLocatorMethods:
- type: 'config'
clusters:
- url: ${K8S_PROD_URL}
name: production
authProvider: 'serviceAccount'
skipTLSVerify: false
serviceAccountToken: ${K8S_PROD_TOKEN}
- url: ${K8S_STAGING_URL}
name: staging
authProvider: 'serviceAccount'
serviceAccountToken: ${K8S_STAGING_TOKEN} 3. Custom Backstage Plugins
Cost Management Plugin
// plugins/cost-insights/src/plugin.ts
import {
createPlugin,
createRoutableExtension,
} from '@backstage/core-plugin-api';
import { rootRouteRef } from './routes';
export const costInsightsPlugin = createPlugin({
id: 'cost-insights',
routes: {
root: rootRouteRef,
},
});
export const CostInsightsPage = costInsightsPlugin.provide(
createRoutableExtension({
name: 'CostInsightsPage',
component: () =>
import('./components/CostInsightsPage').then(m => m.CostInsightsPage),
mountPoint: rootRouteRef,
}),
);
// plugins/cost-insights/src/components/CostInsightsPage.tsx
import React, { useState, useEffect } from 'react';
import {
Content,
Header,
Page,
Progress,
InfoCard,
} from '@backstage/core-components';
import { useApi } from '@backstage/core-plugin-api';
import { costInsightsApiRef } from '../api';
import {
LineChart,
Line,
XAxis,
YAxis,
CartesianGrid,
Tooltip,
Legend,
ResponsiveContainer,
} from 'recharts';
export const CostInsightsPage = () => {
const costApi = useApi(costInsightsApiRef);
const [loading, setLoading] = useState(true);
const [costData, setCostData] = useState([]);
const [recommendations, setRecommendations] = useState([]);
useEffect(() => {
const fetchData = async () => {
try {
const [costs, recs] = await Promise.all([
costApi.getCostData(),
costApi.getRecommendations(),
]);
setCostData(costs);
setRecommendations(recs);
} finally {
setLoading(false);
}
};
fetchData();
}, [costApi]);
if (loading) {
return ;
}
return (
{recommendations.map((rec, index) => (
{rec.title}
{rec.description}
Potential Savings: ${rec.savings}/month
))}
);
};
// plugins/cost-insights/src/api/CostInsightsApi.ts
import { createApiRef } from '@backstage/core-plugin-api';
export interface CostData {
month: string;
aws: number;
gcp: number;
azure: number;
}
export interface Recommendation {
title: string;
description: string;
savings: number;
}
export interface CostInsightsApi {
getCostData(): Promise;
getRecommendations(): Promise;
}
export const costInsightsApiRef = createApiRef({
id: 'plugin.cost-insights.service',
});
// plugins/cost-insights/src/api/CostInsightsClient.ts
import { DiscoveryApi, FetchApi } from '@backstage/core-plugin-api';
import { CostInsightsApi, CostData, Recommendation } from './CostInsightsApi';
export class CostInsightsClient implements CostInsightsApi {
private readonly discoveryApi: DiscoveryApi;
private readonly fetchApi: FetchApi;
constructor(options: {
discoveryApi: DiscoveryApi;
fetchApi: FetchApi;
}) {
this.discoveryApi = options.discoveryApi;
this.fetchApi = options.fetchApi;
}
async getCostData(): Promise {
const baseUrl = await this.discoveryApi.getBaseUrl('cost-insights');
const response = await this.fetchApi.fetch(`${baseUrl}/costs`);
return await response.json();
}
async getRecommendations(): Promise {
const baseUrl = await this.discoveryApi.getBaseUrl('cost-insights');
const response = await this.fetchApi.fetch(`${baseUrl}/recommendations`);
return await response.json();
}
} ๐ค๏ธ Golden Path Templates
Golden Paths are pre-approved, production-ready templates that encode best practices and organizational standards. They reduce decision fatigue and ensure consistency across teams.
Microservice Golden Path Template
# template.yaml - Golden Path Microservice Template
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
name: microservice-golden-path
title: Microservice Golden Path
description: Create a production-ready microservice with all best practices
tags:
- recommended
- microservice
- kubernetes
spec:
owner: platform-team
type: service
parameters:
- title: Service Details
required:
- name
- description
- owner
properties:
name:
title: Name
type: string
description: Unique name for the service
pattern: '^[a-z0-9-]+$'
ui:autofocus: true
description:
title: Description
type: string
description: What does this service do?
owner:
title: Owner
type: string
description: Owner team
ui:field: OwnerPicker
ui:options:
catalogFilter:
kind: Group
- title: Technology Stack
required:
- language
- database
properties:
language:
title: Programming Language
type: string
enum:
- java-springboot
- nodejs-express
- python-fastapi
- golang-gin
enumNames:
- 'Java (Spring Boot)'
- 'Node.js (Express)'
- 'Python (FastAPI)'
- 'Go (Gin)'
database:
title: Database
type: string
enum:
- postgresql
- mongodb
- redis
- none
enumNames:
- 'PostgreSQL'
- 'MongoDB'
- 'Redis'
- 'No database'
- title: Infrastructure Options
properties:
kubernetes:
title: Kubernetes Deployment
type: object
properties:
replicas:
title: Initial Replicas
type: integer
default: 2
minimum: 1
maximum: 10
cpu:
title: CPU Request (millicores)
type: integer
default: 100
minimum: 50
maximum: 2000
memory:
title: Memory Request (Mi)
type: integer
default: 128
minimum: 64
maximum: 4096
autoscaling:
title: Enable Autoscaling
type: boolean
default: true
steps:
- id: fetch
name: Fetch Base Template
action: fetch:template
input:
url: ./skeleton/${{ parameters.language }}
values:
name: ${{ parameters.name }}
description: ${{ parameters.description }}
owner: ${{ parameters.owner }}
database: ${{ parameters.database }}
replicas: ${{ parameters.kubernetes.replicas }}
cpu: ${{ parameters.kubernetes.cpu }}
memory: ${{ parameters.kubernetes.memory }}
autoscaling: ${{ parameters.kubernetes.autoscaling }}
- id: publish
name: Publish to GitHub
action: publish:github
input:
allowedHosts: ['github.com']
description: This is ${{ parameters.name }}
repoUrl: github.com?owner=company&repo=${{ parameters.name }}
defaultBranch: main
gitCommitMessage: 'Initial commit'
gitAuthorName: 'Backstage Scaffolder'
gitAuthorEmail: 'scaffolder@company.com'
topics:
- microservice
- ${{ parameters.language }}
- platform-engineering
- id: create-argocd-app
name: Create ArgoCD Application
action: argocd:create-application
input:
appName: ${{ parameters.name }}
appNamespace: argocd
sourceRepoURL: https://github.com/company/${{ parameters.name }}
sourcePath: k8s
destinationServer: https://kubernetes.default.svc
destinationNamespace: ${{ parameters.name }}
syncPolicy: automated
prunePolicy: true
- id: register
name: Register in Catalog
action: catalog:register
input:
repoContentsUrl: ${{ steps.publish.output.repoContentsUrl }}
catalogInfoPath: '/catalog-info.yaml'
- id: create-monitoring
name: Setup Monitoring
action: monitoring:create-dashboard
input:
serviceName: ${{ parameters.name }}
dashboardType: microservice
metrics:
- request_rate
- error_rate
- p95_latency
- cpu_usage
- memory_usage
output:
links:
- title: Repository
url: ${{ steps.publish.output.remoteUrl }}
- title: ArgoCD Application
url: https://argocd.company.com/applications/${{ parameters.name }}
- title: Open in catalog
icon: catalog
entityRef: ${{ steps.register.output.entityRef }} ๐ Service Catalog Management
The service catalog is the heart of your IDP, providing a single source of truth for all services, APIs, and resources in your organization.
Service Catalog Definition
# catalog-info.yaml - Service catalog entry
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: payment-service
title: Payment Service
description: Handles payment processing and billing
labels:
tier: '1'
lifecycle: production
annotations:
github.com/project-slug: company/payment-service
backstage.io/techdocs-ref: dir:.
jenkins.io/job-full-name: payment-service-pipeline
grafana/dashboard-url: https://grafana.company.com/d/payment-service
pagerduty.com/service-id: P123456
datadog.com/service-name: payment-service
sonarqube.org/project-key: payment-service
tags:
- java
- spring-boot
- payments
- critical
links:
- url: https://payment-service.company.com
title: Production URL
icon: web
- url: https://wiki.company.com/payment-service
title: Wiki
icon: docs
spec:
type: service
owner: payments-team
lifecycle: production
system: billing-system
dependsOn:
- component:database-cluster
- component:message-queue
- component:auth-service
providesApis:
- payment-api-v1
- payment-api-v2
consumesApis:
- fraud-detection-api
- notification-api
---
apiVersion: backstage.io/v1alpha1
kind: API
metadata:
name: payment-api-v2
title: Payment API v2
description: RESTful API for payment processing
annotations:
backstage.io/api-definition-url: https://api.company.com/payment/v2/openapi.yaml
spec:
type: openapi
lifecycle: production
owner: payments-team
definition: |
openapi: 3.0.1
info:
title: Payment API
version: 2.0.0
paths:
/payments:
post:
summary: Process payment
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/PaymentRequest'
responses:
'200':
description: Payment processed successfully
---
apiVersion: backstage.io/v1alpha1
kind: System
metadata:
name: billing-system
title: Billing System
description: Complete billing and payment processing system
spec:
owner: payments-team
domain: finance โ Port.io Enterprise Setup
Port is a developer platform that provides a no-code/low-code approach to building internal developer portals with powerful data modeling capabilities.
Port Blueprint Configuration
Port Service Blueprint
# port-blueprints.json
{
"identifier": "microservice",
"title": "Microservice",
"icon": "Service",
"schema": {
"properties": {
"language": {
"type": "string",
"title": "Language",
"enum": ["Java", "Python", "Go", "Node.js"],
"enumColors": {
"Java": "orange",
"Python": "blue",
"Go": "turquoise",
"Node.js": "green"
}
},
"version": {
"type": "string",
"title": "Version"
},
"healthStatus": {
"type": "string",
"title": "Health Status",
"enum": ["Healthy", "Degraded", "Down"],
"enumColors": {
"Healthy": "green",
"Degraded": "yellow",
"Down": "red"
}
},
"tier": {
"type": "string",
"title": "Criticality Tier",
"enum": ["Tier 1", "Tier 2", "Tier 3"],
"description": "Business criticality classification"
},
"cost": {
"type": "number",
"title": "Monthly Cost ($)"
},
"repository": {
"type": "string",
"format": "url",
"title": "Repository URL"
},
"documentation": {
"type": "string",
"format": "url",
"title": "Documentation"
},
"runbook": {
"type": "string",
"format": "url",
"title": "Runbook URL"
},
"slackChannel": {
"type": "string",
"title": "Slack Channel"
},
"oncallSchedule": {
"type": "string",
"title": "On-Call Schedule"
}
},
"required": ["language", "tier", "healthStatus"]
},
"mirrorProperties": {
"teamName": {
"path": "team.name"
},
"environmentCount": {
"path": "environments.length"
}
},
"calculationProperties": {
"isProduction": {
"title": "Is Production",
"calculation": "properties.tier == 'Tier 1'",
"type": "boolean"
},
"monthlyCostAlert": {
"title": "Cost Alert",
"calculation": "properties.cost > 5000",
"type": "boolean"
}
},
"relations": {
"team": {
"title": "Owning Team",
"target": "team",
"required": true,
"many": false
},
"environments": {
"title": "Environments",
"target": "environment",
"required": false,
"many": true
},
"dependencies": {
"title": "Dependencies",
"target": "microservice",
"required": false,
"many": true
},
"pagerdutyService": {
"title": "PagerDuty Service",
"target": "pagerdutyService",
"required": false,
"many": false
}
}
} Port Self-Service Actions
Self-Service Action Configuration
# port-self-service-action.json
{
"identifier": "create_microservice",
"title": "Create Microservice",
"icon": "Git",
"description": "Scaffold a new microservice with all required infrastructure",
"trigger": {
"type": "self-service",
"operation": "CREATE",
"userInputs": {
"properties": {
"name": {
"type": "string",
"title": "Service Name",
"pattern": "^[a-z][a-z0-9-]*$"
},
"language": {
"type": "string",
"title": "Language",
"enum": ["java", "python", "go", "nodejs"],
"enumNames": ["Java", "Python", "Go", "Node.js"]
},
"team": {
"type": "string",
"blueprint": "team",
"title": "Owning Team"
},
"database": {
"type": "string",
"title": "Database Type",
"enum": ["postgresql", "mongodb", "none"],
"default": "postgresql"
},
"includeCache": {
"type": "boolean",
"title": "Include Redis Cache",
"default": false
}
},
"required": ["name", "language", "team"]
}
},
"invocationMethod": {
"type": "GITHUB",
"org": "company",
"repo": "platform-automation",
"workflow": "create-microservice.yml",
"workflowInputs": {
"name": "{{ .inputs.name }}",
"language": "{{ .inputs.language }}",
"team": "{{ .inputs.team }}",
"database": "{{ .inputs.database }}",
"includeCache": "{{ .inputs.includeCache }}"
}
}
}
# GitHub Actions Workflow - create-microservice.yml
name: Create Microservice
on:
workflow_dispatch:
inputs:
name:
required: true
type: string
language:
required: true
type: choice
options: [java, python, go, nodejs]
team:
required: true
type: string
database:
required: true
type: string
includeCache:
required: false
type: boolean
default: false
port_run_id:
required: true
type: string
jobs:
create-microservice:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ secrets.AWS_ROLE }}
aws-region: us-east-1
- name: Generate from template
run: |
cookiecutter templates/${{ inputs.language }} \
--no-input \
service_name=${{ inputs.name }} \
team=${{ inputs.team }} \
database=${{ inputs.database }} \
include_cache=${{ inputs.includeCache }}
- name: Create GitHub repository
run: |
gh repo create company/${{ inputs.name }} \
--private \
--team ${{ inputs.team }} \
--description "Microservice created via Port.io"
- name: Push code
run: |
cd ${{ inputs.name }}
git init
git add .
git commit -m "Initial commit from Port.io"
git remote add origin https://github.com/company/${{ inputs.name }}.git
git push -u origin main
- name: Deploy infrastructure
run: |
cd ${{ inputs.name }}/terraform
terraform init
terraform apply -auto-approve \
-var="service_name=${{ inputs.name }}" \
-var="team=${{ inputs.team }}"
- name: Update Port
uses: port-labs/port-github-action@v1
with:
clientId: ${{ secrets.PORT_CLIENT_ID }}
clientSecret: ${{ secrets.PORT_CLIENT_SECRET }}
baseUrl: https://api.getport.io
operation: PATCH_RUN
runId: ${{ inputs.port_run_id }}
logMessage: "Service created successfully" ๐ค Humanitec Platform Orchestration
Humanitec provides a Platform Orchestration layer that dynamically generates application configurations based on the context.
Humanitec Score Workload Specification
# score.yaml - Humanitec Score specification
apiVersion: score.dev/v1b1
metadata:
name: payment-service
annotations:
humanitec.io/app-id: payment-app
humanitec.io/env-type: development
containers:
api:
image: company/payment-service:${VERSION}
command: ["java", "-jar", "app.jar"]
variables:
DATABASE_URL: ${resources.database.url}
CACHE_URL: ${resources.cache.url}
LOG_LEVEL: ${values.log_level}
resources:
limits:
cpu: "1000m"
memory: "1Gi"
requests:
cpu: "100m"
memory: "256Mi"
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
service:
ports:
http:
port: 8080
targetPort: 8080
protocol: TCP
resources:
database:
type: postgres
class: ${values.database_class}
metadata:
annotations:
humanitec.io/res-id: shared-postgres
cache:
type: redis
class: ${values.cache_class}
dns:
type: dns
class: default
ingress:
type: ingress
class: default
params:
rules:
- host: ${resources.dns.host}
http:
paths:
- path: /
backend:
service:
name: api
port: 8080
# Resource Definitions
---
apiVersion: humanitec.io/v1alpha1
kind: Resource
metadata:
name: postgres-rds
spec:
type: postgres
driver_type: terraform
driver_inputs:
values:
source:
path: terraform/modules/rds
rev: main
url: https://github.com/company/terraform-modules.git
variables:
instance_class: ${context.app.id}-${context.env.id}
database_name: ${context.app.id}_${context.env.id}
engine_version: "14.7"
criteria:
- env_type: production
class: high-performance
---
apiVersion: humanitec.io/v1alpha1
kind: Resource
metadata:
name: redis-elasticache
spec:
type: redis
driver_type: terraform
driver_inputs:
values:
source:
path: terraform/modules/elasticache
rev: main
url: https://github.com/company/terraform-modules.git
variables:
node_type: cache.t3.micro
num_cache_nodes: ${context.env.type == "production" ? 3 : 1}
criteria:
- env_type: production ๐๏ธ Building Custom IDPs
Sometimes organizations need custom IDPs tailored to their specific requirements. Here's a complete architecture for building one.
Custom IDP Backend (Node.js/TypeScript)
// src/server.ts - Custom IDP Backend
import express from 'express';
import { PrismaClient } from '@prisma/client';
import { GitHubAPI } from './integrations/github';
import { KubernetesAPI } from './integrations/kubernetes';
import { TerraformCloud } from './integrations/terraform';
import { MetricsCollector } from './metrics/collector';
const app = express();
const prisma = new PrismaClient();
// Service Catalog API
app.get('/api/services', async (req, res) => {
const services = await prisma.service.findMany({
include: {
team: true,
dependencies: true,
deployments: true,
metrics: {
where: {
timestamp: {
gte: new Date(Date.now() - 24 * 60 * 60 * 1000)
}
}
}
}
});
res.json(services);
});
app.post('/api/services', async (req, res) => {
const { name, description, teamId, template } = req.body;
// Create service in database
const service = await prisma.service.create({
data: {
name,
description,
teamId,
template,
status: 'provisioning'
}
});
// Trigger provisioning workflow
await provisionService(service);
res.json(service);
});
async function provisionService(service: any) {
try {
// Create GitHub repository
const repo = await GitHubAPI.createRepository({
name: service.name,
description: service.description,
team: service.teamId,
template: service.template
});
// Create Terraform workspace
const workspace = await TerraformCloud.createWorkspace({
name: service.name,
repository: repo.url,
autoApply: false
});
// Deploy to Kubernetes
const deployment = await KubernetesAPI.createDeployment({
name: service.name,
namespace: service.teamId,
image: `company/${service.name}:latest`,
replicas: 2
});
// Update service status
await prisma.service.update({
where: { id: service.id },
data: {
status: 'active',
repository: repo.url,
terraformWorkspace: workspace.id,
kubernetesDeployment: deployment.metadata.name
}
});
// Start collecting metrics
MetricsCollector.startCollection(service.id);
} catch (error) {
await prisma.service.update({
where: { id: service.id },
data: {
status: 'failed',
error: error.message
}
});
}
}
// Self-Service Actions API
app.post('/api/actions/:actionId/execute', async (req, res) => {
const { actionId } = req.params;
const { inputs } = req.body;
const action = await prisma.action.findUnique({
where: { id: actionId }
});
if (!action) {
return res.status(404).json({ error: 'Action not found' });
}
// Execute action based on type
const execution = await executeAction(action, inputs);
res.json(execution);
});
// Scorecards API
app.get('/api/scorecards/:serviceId', async (req, res) => {
const { serviceId } = req.params;
const scorecard = await calculateScorecard(serviceId);
res.json(scorecard);
});
async function calculateScorecard(serviceId: string) {
const service = await prisma.service.findUnique({
where: { id: serviceId },
include: {
deployments: true,
incidents: true,
securityScans: true,
costReports: true
}
});
return {
serviceId,
scores: {
reliability: calculateReliabilityScore(service),
security: calculateSecurityScore(service),
performance: calculatePerformanceScore(service),
cost: calculateCostScore(service),
compliance: calculateComplianceScore(service)
},
recommendations: generateRecommendations(service)
};
}
// Database Schema (Prisma)
// schema.prisma
model Service {
id String @id @default(cuid())
name String @unique
description String
template String
status String
repository String?
terraformWorkspace String?
kubernetesDeployment String?
teamId String
team Team @relation(fields: [teamId], references: [id])
dependencies Service[] @relation("ServiceDependencies")
deployments Deployment[]
metrics Metric[]
incidents Incident[]
securityScans SecurityScan[]
costReports CostReport[]
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
}
model Team {
id String @id @default(cuid())
name String @unique
slackChannel String
email String
services Service[]
members User[]
}
model Deployment {
id String @id @default(cuid())
serviceId String
service Service @relation(fields: [serviceId], references: [id])
version String
environment String
status String
deployedBy String
deployedAt DateTime @default(now())
rollbackTo String?
} Custom IDP Frontend (React)
Service Catalog UI Component
// src/components/ServiceCatalog.tsx
import React, { useState, useEffect } from 'react';
import {
Card,
Grid,
TextField,
Select,
MenuItem,
Button,
Chip,
Dialog,
CircularProgress
} from '@mui/material';
import { useApi } from '../hooks/useApi';
import { ServiceScorecard } from './ServiceScorecard';
import { ServiceActions } from './ServiceActions';
interface Service {
id: string;
name: string;
description: string;
team: Team;
status: string;
template: string;
dependencies: Service[];
metrics: Metric[];
}
export const ServiceCatalog: React.FC = () => {
const [services, setServices] = useState<Service[]>([]);
const [filteredServices, setFilteredServices] = useState<Service[]>([]);
const [searchTerm, setSearchTerm] = useState('');
const [teamFilter, setTeamFilter] = useState('all');
const [statusFilter, setStatusFilter] = useState('all');
const [selectedService, setSelectedService] = useState<Service | null>(null);
const [showCreateDialog, setShowCreateDialog] = useState(false);
const api = useApi();
useEffect(() => {
loadServices();
}, []);
useEffect(() => {
filterServices();
}, [services, searchTerm, teamFilter, statusFilter]);
const loadServices = async () => {
const data = await api.get('/api/services');
setServices(data);
};
const filterServices = () => {
let filtered = services;
if (searchTerm) {
filtered = filtered.filter(s =>
s.name.toLowerCase().includes(searchTerm.toLowerCase()) ||
s.description.toLowerCase().includes(searchTerm.toLowerCase())
);
}
if (teamFilter !== 'all') {
filtered = filtered.filter(s => s.team.id === teamFilter);
}
if (statusFilter !== 'all') {
filtered = filtered.filter(s => s.status === statusFilter);
}
setFilteredServices(filtered);
};
const getStatusColor = (status: string) => {
switch (status) {
case 'active': return 'success';
case 'degraded': return 'warning';
case 'failed': return 'error';
default: return 'default';
}
};
return (
<div className="service-catalog">
<div className="catalog-header">
<h1>Service Catalog</h1>
<Button
variant="contained"
color="primary"
onClick={() => setShowCreateDialog(true)}
>
Create Service
</Button>
</div>
<div className="catalog-filters">
<TextField
label="Search"
variant="outlined"
value={searchTerm}
onChange={(e) => setSearchTerm(e.target.value)}
style={{ marginRight: 16 }}
/>
<Select
value={teamFilter}
onChange={(e) => setTeamFilter(e.target.value)}
style={{ marginRight: 16, minWidth: 150 }}
>
<MenuItem value="all">All Teams</MenuItem>
{/* Dynamic team options */}
</Select>
<Select
value={statusFilter}
onChange={(e) => setStatusFilter(e.target.value)}
style={{ minWidth: 150 }}
>
<MenuItem value="all">All Status</MenuItem>
<MenuItem value="active">Active</MenuItem>
<MenuItem value="degraded">Degraded</MenuItem>
<MenuItem value="failed">Failed</MenuItem>
</Select>
</div>
<Grid container spacing={3} className="catalog-grid">
{filteredServices.map(service => (
<Grid item xs={12} md={6} lg={4} key={service.id}>
<Card
className="service-card"
onClick={() => setSelectedService(service)}
>
<div className="service-card-header">
<h3>{service.name}</h3>
<Chip
label={service.status}
color={getStatusColor(service.status)}
size="small"
/>
</div>
<p>{service.description}</p>
<div className="service-card-meta">
<span>Team: {service.team.name}</span>
<span>Template: {service.template}</span>
</div>
<div className="service-card-metrics">
<div className="metric">
<span className="metric-label">Uptime</span>
<span className="metric-value">99.9%</span>
</div>
<div className="metric">
<span className="metric-label">Latency</span>
<span className="metric-value">45ms</span>
</div>
<div className="metric">
<span className="metric-label">Cost</span>
<span className="metric-value">$1,234</span>
</div>
</div>
</Card>
</Grid>
))}
</Grid>
{selectedService && (
<Dialog
open={true}
onClose={() => setSelectedService(null)}
maxWidth="lg"
fullWidth
>
<div className="service-details">
<h2>{selectedService.name}</h2>
<ServiceScorecard service={selectedService} />
<ServiceActions service={selectedService} />
</div>
</Dialog>
)}
<CreateServiceDialog
open={showCreateDialog}
onClose={() => setShowCreateDialog(false)}
onCreated={loadServices}
/>
</div>
);
};