Platform Engineering with Internal Developer Portals: The Complete Guide to Building Self-Service Infrastructure

Platform Engineering represents the evolution of DevOps, transforming how organizations deliver infrastructure and services to development teams. Internal Developer Portals (IDPs) are the cornerstone of this transformation, providing self-service capabilities that eliminate bottlenecks, reduce cognitive load, and accelerate delivery. This comprehensive guide provides everything you need to build, deploy, and scale enterprise-grade IDPs using Backstage, Port, Humanitec, and custom solutions.

The shift from DevOps to Platform Engineering is not optional—it's an evolutionary necessity. As organizations scale, the traditional DevOps model creates unsustainable cognitive load on developers. Platform Engineering with IDPs abstracts complexity while maintaining flexibility, enabling developers to focus on delivering value rather than managing infrastructure. This guide covers everything from basic concepts to advanced multi-cloud implementations.

📋 Table of Contents

🤔 Why Platform Engineering is Critical

Platform Engineering addresses the scalability crisis in modern software delivery. As organizations grow, developers face increasing cognitive load from infrastructure complexity, tool sprawl, and operational responsibilities. This leads to decreased productivity, increased errors, and slower time-to-market.

Platform Engineering solves these challenges by:

• Abstracting Complexity: Hide infrastructure details behind intuitive interfaces
• Standardizing Workflows: Create golden paths that encode best practices
• Enabling Self-Service: Empower developers to provision resources without tickets
• Reducing Cognitive Load: Let developers focus on code, not infrastructure
• Improving Governance: Enforce policies and compliance automatically

# Platform Engineering Maturity Assessment
Level 1 - Ad Hoc:
  - Manual infrastructure provisioning
  - No standardized workflows
  - High cognitive load on developers
  - Frequent production incidents
Level 2 - Managed:
  - Some automation with scripts
  - Basic CI/CD pipelines
  - Shared documentation
  - Reactive incident response
Level 3 - Defined:
  - Infrastructure as Code adoption
  - Standardized deployment processes
  - Service catalogs emerging
  - Proactive monitoring
Level 4 - Quantified:
  - Full self-service capabilities
  - Golden paths implemented
  - Metrics-driven decisions
  - Automated compliance
Level 5 - Optimized:
  - AI-powered recommendations
  - Predictive scaling
  - Zero-touch operations
  - Continuous optimization

🏗️ Internal Developer Portal Fundamentals

An Internal Developer Portal is the user interface to your platform, providing a unified experience for discovering, creating, and managing services and infrastructure.

Core Components of an IDP

# idp-architecture.yaml
components:
  service_catalog:
    description: "Central registry of all services, APIs, and resources"
    features:
      - Service discovery
      - Ownership tracking
      - Documentation hub
      - API specifications
      - Dependency mapping
 
  software_templates:
    description: "Scaffolding for new services and resources"
    features:
      - Golden path templates
      - Best practices enforcement
      - Automated provisioning
      - Compliance checks
 
  tech_docs:
    description: "Centralized technical documentation"
    features:
      - Markdown support
      - API documentation
      - Runbooks
      - Architecture diagrams
 
  scorecards:
    description: "Service health and compliance tracking"
    features:
      - Production readiness
      - Security compliance
      - Performance metrics
      - Cost optimization
 
  workflows:
    description: "Automated processes and pipelines"
    features:
      - CI/CD integration
      - Approval workflows
      - Incident management
      - Change management

🎭 Complete Backstage Implementation

Backstage, created by Spotify and now a CNCF project, is the leading open-source platform for building developer portals. Here's a production-ready implementation.

1. Backstage Deployment on Kubernetes

# backstage-deployment.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: backstage
---
apiVersion: v1
kind: Secret
metadata:
  name: backstage-secrets
  namespace: backstage
type: Opaque
stringData:
  GITHUB_TOKEN: ${GITHUB_TOKEN}
  AUTH_GITHUB_CLIENT_ID: ${GITHUB_CLIENT_ID}
  AUTH_GITHUB_CLIENT_SECRET: ${GITHUB_CLIENT_SECRET}
  POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backstage
  namespace: backstage
spec:
  replicas: 3
  selector:
    matchLabels:
      app: backstage
  template:
    metadata:
      labels:
        app: backstage
    spec:
      containers:
      - name: backstage
        image: backstage:latest
        imagePullPolicy: Always
        ports:
        - name: http
          containerPort: 7007
        envFrom:
        - secretRef:
            name: backstage-secrets
        env:
        - name: POSTGRES_HOST
          value: postgres-service.backstage.svc.cluster.local
        - name: POSTGRES_PORT
          value: "5432"
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /healthcheck
            port: 7007
          initialDelaySeconds: 60
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /healthcheck
            port: 7007
          initialDelaySeconds: 30
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: backstage
  namespace: backstage
spec:
  selector:
    app: backstage
  type: LoadBalancer
  ports:
  - name: http
    port: 80
    targetPort: http
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: backstage
  namespace: backstage
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
  tls:
  - hosts:
    - portal.company.com
    secretName: backstage-tls
  rules:
  - host: portal.company.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: backstage
            port:
              number: 80

2. Backstage Configuration

# app-config.production.yaml
app:
  title: Company Developer Portal
  baseUrl: https://portal.company.com
  support:
    url: https://github.com/company/backstage/issues
    items:
      - title: Issues
        icon: github
        links:
          - url: https://github.com/company/backstage/issues
            title: GitHub Issues
organization:
  name: Company
backend:
  baseUrl: https://portal.company.com
  listen:
    port: 7007
  csp:
    connect-src: ["'self'", 'http:', 'https:']
  cors:
    origin: https://portal.company.com
    methods: [GET, POST, PUT, DELETE]
    credentials: true
  database:
    client: pg
    connection:
      host: ${POSTGRES_HOST}
      port: ${POSTGRES_PORT}
      user: ${POSTGRES_USER}
      password: ${POSTGRES_PASSWORD}
      database: backstage
      ssl:
        require: true
        rejectUnauthorized: false
  cache:
    store: memory
  reading:
    allow:
      - host: github.com
      - host: '*.company.com'
integrations:
  github:
    - host: github.com
      token: ${GITHUB_TOKEN}
  gitlab:
    - host: gitlab.company.com
      token: ${GITLAB_TOKEN}
  azure:
    - host: dev.azure.com
      token: ${AZURE_TOKEN}
proxy:
  '/jenkins':
    target: 'https://jenkins.company.com'
    changeOrigin: true
    headers:
      Authorization: ${JENKINS_TOKEN}
  '/argocd':
    target: 'https://argocd.company.com'
    changeOrigin: true
    secure: false
  '/prometheus':
    target: 'https://prometheus.company.com'
  '/grafana':
    target: 'https://grafana.company.com'
techdocs:
  builder: 'external'
  generator:
    runIn: 'docker'
  publisher:
    type: 'awsS3'
    awsS3:
      bucketName: 'company-techdocs'
      region: ${AWS_DEFAULT_REGION}
      credentials:
        roleArn: ${AWS_ROLE_ARN}
auth:
  environment: production
  providers:
    github:
      production:
        clientId: ${AUTH_GITHUB_CLIENT_ID}
        clientSecret: ${AUTH_GITHUB_CLIENT_SECRET}
    google:
      production:
        clientId: ${AUTH_GOOGLE_CLIENT_ID}
        clientSecret: ${AUTH_GOOGLE_CLIENT_SECRET}
    okta:
      production:
        clientId: ${AUTH_OKTA_CLIENT_ID}
        clientSecret: ${AUTH_OKTA_CLIENT_SECRET}
        audience: ${AUTH_OKTA_AUDIENCE}
        authServerId: ${AUTH_OKTA_AUTH_SERVER_ID}
scaffolder:
  defaultAuthor:
    name: Backstage Scaffolder
    email: scaffolder@company.com
  defaultCommitMessage: 'Initial commit from Backstage Scaffolder'
catalog:
  import:
    entityFilename: catalog-info.yaml
    pullRequestBranchName: backstage-integration
  rules:
    - allow: [Component, System, API, Resource, Location, Template]
  locations:
    - type: url
      target: https://github.com/company/backstage-catalog/blob/main/catalog-all.yaml
    - type: github-discovery
      target: https://github.com/company/*/blob/main/catalog-info.yaml
    - type: github-org
      target: https://github.com/company
  providers:
    github:
      company:
        organization: 'company'
        catalogPath: '/catalog-info.yaml'
        filters:
          branch: 'main'
          repository: '.*'
        schedule:
          frequency: { minutes: 30 }
          timeout: { minutes: 3 }
kubernetes:
  serviceLocatorMethod:
    type: 'multiTenant'
  clusterLocatorMethods:
    - type: 'config'
      clusters:
        - url: ${K8S_PROD_URL}
          name: production
          authProvider: 'serviceAccount'
          skipTLSVerify: false
          serviceAccountToken: ${K8S_PROD_TOKEN}
        - url: ${K8S_STAGING_URL}
          name: staging
          authProvider: 'serviceAccount'
          serviceAccountToken: ${K8S_STAGING_TOKEN}

3. Custom Backstage Plugins

// plugins/cost-insights/src/plugin.ts
import {
  createPlugin,
  createRoutableExtension,
} from '@backstage/core-plugin-api';
import { rootRouteRef } from './routes';
export const costInsightsPlugin = createPlugin({
  id: 'cost-insights',
  routes: {
    root: rootRouteRef,
  },
});
export const CostInsightsPage = costInsightsPlugin.provide(
  createRoutableExtension({
    name: 'CostInsightsPage',
    component: () =>
      import('./components/CostInsightsPage').then(m => m.CostInsightsPage),
    mountPoint: rootRouteRef,
  }),
);
// plugins/cost-insights/src/components/CostInsightsPage.tsx
import React, { useState, useEffect } from 'react';
import {
  Content,
  Header,
  Page,
  Progress,
  InfoCard,
} from '@backstage/core-components';
import { useApi } from '@backstage/core-plugin-api';
import { costInsightsApiRef } from '../api';
import {
  LineChart,
  Line,
  XAxis,
  YAxis,
  CartesianGrid,
  Tooltip,
  Legend,
  ResponsiveContainer,
} from 'recharts';
export const CostInsightsPage = () => {
  const costApi = useApi(costInsightsApiRef);
  const [loading, setLoading] = useState(true);
  const [costData, setCostData] = useState([]);
  const [recommendations, setRecommendations] = useState([]);
  useEffect(() => {
    const fetchData = async () => {
      try {
        const [costs, recs] = await Promise.all([
          costApi.getCostData(),
          costApi.getRecommendations(),
        ]);
        setCostData(costs);
        setRecommendations(recs);
      } finally {
        setLoading(false);
      }
    };
    fetchData();
  }, [costApi]);
  if (loading) {
    return ;
  }
  return (
    
      

      
        
          
            
              
              
              
              
              

              
              
              
            
          
        
        
          

            {recommendations.map((rec, index) => (
              

                
{rec.title}
                
{rec.description}
                

                  Potential Savings: ${rec.savings}/month
                
              
            ))}
          
        
      
    
  );
};
// plugins/cost-insights/src/api/CostInsightsApi.ts
import { createApiRef } from '@backstage/core-plugin-api';
export interface CostData {
  month: string;
  aws: number;
  gcp: number;
  azure: number;
}
export interface Recommendation {
  title: string;
  description: string;
  savings: number;
}
export interface CostInsightsApi {
  getCostData(): Promise;
  getRecommendations(): Promise;
}
export const costInsightsApiRef = createApiRef({
  id: 'plugin.cost-insights.service',
});
// plugins/cost-insights/src/api/CostInsightsClient.ts
import { DiscoveryApi, FetchApi } from '@backstage/core-plugin-api';
import { CostInsightsApi, CostData, Recommendation } from './CostInsightsApi';
export class CostInsightsClient implements CostInsightsApi {
  private readonly discoveryApi: DiscoveryApi;
  private readonly fetchApi: FetchApi;
  constructor(options: {
    discoveryApi: DiscoveryApi;
    fetchApi: FetchApi;
  }) {
    this.discoveryApi = options.discoveryApi;
    this.fetchApi = options.fetchApi;
  }
  async getCostData(): Promise {
    const baseUrl = await this.discoveryApi.getBaseUrl('cost-insights');
    const response = await this.fetchApi.fetch(`${baseUrl}/costs`);
    return await response.json();
  }
  async getRecommendations(): Promise {
    const baseUrl = await this.discoveryApi.getBaseUrl('cost-insights');
    const response = await this.fetchApi.fetch(`${baseUrl}/recommendations`);
    return await response.json();
  }
}

🛤️ Golden Path Templates

Golden Paths are pre-approved, production-ready templates that encode best practices and organizational standards. They reduce decision fatigue and ensure consistency across teams.

# template.yaml - Golden Path Microservice Template
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: microservice-golden-path
  title: Microservice Golden Path
  description: Create a production-ready microservice with all best practices
  tags:
    - recommended
    - microservice
    - kubernetes
spec:
  owner: platform-team
  type: service
  parameters:
    - title: Service Details
      required:
        - name
        - description
        - owner
      properties:
        name:
          title: Name
          type: string
          description: Unique name for the service
          pattern: '^[a-z0-9-]+$'
          ui:autofocus: true
        description:
          title: Description
          type: string
          description: What does this service do?
        owner:
          title: Owner
          type: string
          description: Owner team
          ui:field: OwnerPicker
          ui:options:
            catalogFilter:
              kind: Group
    - title: Technology Stack
      required:
        - language
        - database
      properties:
        language:
          title: Programming Language
          type: string
          enum:
            - java-springboot
            - nodejs-express
            - python-fastapi
            - golang-gin
          enumNames:
            - 'Java (Spring Boot)'
            - 'Node.js (Express)'
            - 'Python (FastAPI)'
            - 'Go (Gin)'
        database:
          title: Database
          type: string
          enum:
            - postgresql
            - mongodb
            - redis
            - none
          enumNames:
            - 'PostgreSQL'
            - 'MongoDB'
            - 'Redis'
            - 'No database'
    - title: Infrastructure Options
      properties:
        kubernetes:
          title: Kubernetes Deployment
          type: object
          properties:
            replicas:
              title: Initial Replicas
              type: integer
              default: 2
              minimum: 1
              maximum: 10
            cpu:
              title: CPU Request (millicores)
              type: integer
              default: 100
              minimum: 50
              maximum: 2000
            memory:
              title: Memory Request (Mi)
              type: integer
              default: 128
              minimum: 64
              maximum: 4096
            autoscaling:
              title: Enable Autoscaling
              type: boolean
              default: true
  steps:
    - id: fetch
      name: Fetch Base Template
      action: fetch:template
      input:
        url: ./skeleton/${{ parameters.language }}
        values:
          name: ${{ parameters.name }}
          description: ${{ parameters.description }}
          owner: ${{ parameters.owner }}
          database: ${{ parameters.database }}
          replicas: ${{ parameters.kubernetes.replicas }}
          cpu: ${{ parameters.kubernetes.cpu }}
          memory: ${{ parameters.kubernetes.memory }}
          autoscaling: ${{ parameters.kubernetes.autoscaling }}
    - id: publish
      name: Publish to GitHub
      action: publish:github
      input:
        allowedHosts: ['github.com']
        description: This is ${{ parameters.name }}
        repoUrl: github.com?owner=company&repo=${{ parameters.name }}
        defaultBranch: main
        gitCommitMessage: 'Initial commit'
        gitAuthorName: 'Backstage Scaffolder'
        gitAuthorEmail: 'scaffolder@company.com'
        topics:
          - microservice
          - ${{ parameters.language }}
          - platform-engineering
    - id: create-argocd-app
      name: Create ArgoCD Application
      action: argocd:create-application
      input:
        appName: ${{ parameters.name }}
        appNamespace: argocd
        sourceRepoURL: https://github.com/company/${{ parameters.name }}
        sourcePath: k8s
        destinationServer: https://kubernetes.default.svc
        destinationNamespace: ${{ parameters.name }}
        syncPolicy: automated
        prunePolicy: true
    - id: register
      name: Register in Catalog
      action: catalog:register
      input:
        repoContentsUrl: ${{ steps.publish.output.repoContentsUrl }}
        catalogInfoPath: '/catalog-info.yaml'
    - id: create-monitoring
      name: Setup Monitoring
      action: monitoring:create-dashboard
      input:
        serviceName: ${{ parameters.name }}
        dashboardType: microservice
        metrics:
          - request_rate
          - error_rate
          - p95_latency
          - cpu_usage
          - memory_usage
  output:
    links:
      - title: Repository
        url: ${{ steps.publish.output.remoteUrl }}
      - title: ArgoCD Application
        url: https://argocd.company.com/applications/${{ parameters.name }}
      - title: Open in catalog
        icon: catalog
        entityRef: ${{ steps.register.output.entityRef }}

📚 Service Catalog Management

The service catalog is the heart of your IDP, providing a single source of truth for all services, APIs, and resources in your organization.

# catalog-info.yaml - Service catalog entry
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: payment-service
  title: Payment Service
  description: Handles payment processing and billing
  labels:
    tier: '1'
    lifecycle: production
  annotations:
    github.com/project-slug: company/payment-service
    backstage.io/techdocs-ref: dir:.
    jenkins.io/job-full-name: payment-service-pipeline
    grafana/dashboard-url: https://grafana.company.com/d/payment-service
    pagerduty.com/service-id: P123456
    datadog.com/service-name: payment-service
    sonarqube.org/project-key: payment-service
  tags:
    - java
    - spring-boot
    - payments
    - critical
  links:
    - url: https://payment-service.company.com
      title: Production URL
      icon: web
    - url: https://wiki.company.com/payment-service
      title: Wiki
      icon: docs
spec:
  type: service
  owner: payments-team
  lifecycle: production
  system: billing-system
  dependsOn:
    - component:database-cluster
    - component:message-queue
    - component:auth-service
  providesApis:
    - payment-api-v1
    - payment-api-v2
  consumesApis:
    - fraud-detection-api
    - notification-api
---
apiVersion: backstage.io/v1alpha1
kind: API
metadata:
  name: payment-api-v2
  title: Payment API v2
  description: RESTful API for payment processing
  annotations:
    backstage.io/api-definition-url: https://api.company.com/payment/v2/openapi.yaml
spec:
  type: openapi
  lifecycle: production
  owner: payments-team
  definition: |
    openapi: 3.0.1
    info:
      title: Payment API
      version: 2.0.0
    paths:
      /payments:
        post:
          summary: Process payment
          requestBody:
            content:
              application/json:
                schema:
                  $ref: '#/components/schemas/PaymentRequest'
          responses:
            '200':
              description: Payment processed successfully
---
apiVersion: backstage.io/v1alpha1
kind: System
metadata:
  name: billing-system
  title: Billing System
  description: Complete billing and payment processing system
spec:
  owner: payments-team
  domain: finance

⚓ Port.io Enterprise Setup

Port is a developer platform that provides a no-code/low-code approach to building internal developer portals with powerful data modeling capabilities.

Port Blueprint Configuration

# port-blueprints.json
{
  "identifier": "microservice",
  "title": "Microservice",
  "icon": "Service",
  "schema": {
    "properties": {
      "language": {
        "type": "string",
        "title": "Language",
        "enum": ["Java", "Python", "Go", "Node.js"],
        "enumColors": {
          "Java": "orange",
          "Python": "blue",
          "Go": "turquoise",
          "Node.js": "green"
        }
      },
      "version": {
        "type": "string",
        "title": "Version"
      },
      "healthStatus": {
        "type": "string",
        "title": "Health Status",
        "enum": ["Healthy", "Degraded", "Down"],
        "enumColors": {
          "Healthy": "green",
          "Degraded": "yellow",
          "Down": "red"
        }
      },
      "tier": {
        "type": "string",
        "title": "Criticality Tier",
        "enum": ["Tier 1", "Tier 2", "Tier 3"],
        "description": "Business criticality classification"
      },
      "cost": {
        "type": "number",
        "title": "Monthly Cost ($)"
      },
      "repository": {
        "type": "string",
        "format": "url",
        "title": "Repository URL"
      },
      "documentation": {
        "type": "string",
        "format": "url",
        "title": "Documentation"
      },
      "runbook": {
        "type": "string",
        "format": "url",
        "title": "Runbook URL"
      },
      "slackChannel": {
        "type": "string",
        "title": "Slack Channel"
      },
      "oncallSchedule": {
        "type": "string",
        "title": "On-Call Schedule"
      }
    },
    "required": ["language", "tier", "healthStatus"]
  },
  "mirrorProperties": {
    "teamName": {
      "path": "team.name"
    },
    "environmentCount": {
      "path": "environments.length"
    }
  },
  "calculationProperties": {
    "isProduction": {
      "title": "Is Production",
      "calculation": "properties.tier == 'Tier 1'",
      "type": "boolean"
    },
    "monthlyCostAlert": {
      "title": "Cost Alert",
      "calculation": "properties.cost > 5000",
      "type": "boolean"
    }
  },
  "relations": {
    "team": {
      "title": "Owning Team",
      "target": "team",
      "required": true,
      "many": false
    },
    "environments": {
      "title": "Environments",
      "target": "environment",
      "required": false,
      "many": true
    },
    "dependencies": {
      "title": "Dependencies",
      "target": "microservice",
      "required": false,
      "many": true
    },
    "pagerdutyService": {
      "title": "PagerDuty Service",
      "target": "pagerdutyService",
      "required": false,
      "many": false
    }
  }
}

Port Self-Service Actions

# port-self-service-action.json
{
  "identifier": "create_microservice",
  "title": "Create Microservice",
  "icon": "Git",
  "description": "Scaffold a new microservice with all required infrastructure",
  "trigger": {
    "type": "self-service",
    "operation": "CREATE",
    "userInputs": {
      "properties": {
        "name": {
          "type": "string",
          "title": "Service Name",
          "pattern": "^[a-z][a-z0-9-]*$"
        },
        "language": {
          "type": "string",
          "title": "Language",
          "enum": ["java", "python", "go", "nodejs"],
          "enumNames": ["Java", "Python", "Go", "Node.js"]
        },
        "team": {
          "type": "string",
          "blueprint": "team",
          "title": "Owning Team"
        },
        "database": {
          "type": "string",
          "title": "Database Type",
          "enum": ["postgresql", "mongodb", "none"],
          "default": "postgresql"
        },
        "includeCache": {
          "type": "boolean",
          "title": "Include Redis Cache",
          "default": false
        }
      },
      "required": ["name", "language", "team"]
    }
  },
  "invocationMethod": {
    "type": "GITHUB",
    "org": "company",
    "repo": "platform-automation",
    "workflow": "create-microservice.yml",
    "workflowInputs": {
      "name": "{{ .inputs.name }}",
      "language": "{{ .inputs.language }}",
      "team": "{{ .inputs.team }}",
      "database": "{{ .inputs.database }}",
      "includeCache": "{{ .inputs.includeCache }}"
    }
  }
}
# GitHub Actions Workflow - create-microservice.yml
name: Create Microservice
on:
  workflow_dispatch:
    inputs:
      name:
        required: true
        type: string
      language:
        required: true
        type: choice
        options: [java, python, go, nodejs]
      team:
        required: true
        type: string
      database:
        required: true
        type: string
      includeCache:
        required: false
        type: boolean
        default: false
      port_run_id:
        required: true
        type: string
jobs:
  create-microservice:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
     
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          role-to-assume: ${{ secrets.AWS_ROLE }}
          aws-region: us-east-1
     
      - name: Generate from template
        run: |
          cookiecutter templates/${{ inputs.language }} \
            --no-input \
            service_name=${{ inputs.name }} \
            team=${{ inputs.team }} \
            database=${{ inputs.database }} \
            include_cache=${{ inputs.includeCache }}
     
      - name: Create GitHub repository
        run: |
          gh repo create company/${{ inputs.name }} \
            --private \
            --team ${{ inputs.team }} \
            --description "Microservice created via Port.io"
     
      - name: Push code
        run: |
          cd ${{ inputs.name }}
          git init
          git add .
          git commit -m "Initial commit from Port.io"
          git remote add origin https://github.com/company/${{ inputs.name }}.git
          git push -u origin main
     
      - name: Deploy infrastructure
        run: |
          cd ${{ inputs.name }}/terraform
          terraform init
          terraform apply -auto-approve \
            -var="service_name=${{ inputs.name }}" \
            -var="team=${{ inputs.team }}"
     
      - name: Update Port
        uses: port-labs/port-github-action@v1
        with:
          clientId: ${{ secrets.PORT_CLIENT_ID }}
          clientSecret: ${{ secrets.PORT_CLIENT_SECRET }}
          baseUrl: https://api.getport.io
          operation: PATCH_RUN
          runId: ${{ inputs.port_run_id }}
          logMessage: "Service created successfully"

🤖 Humanitec Platform Orchestration

Humanitec provides a Platform Orchestration layer that dynamically generates application configurations based on the context.

# score.yaml - Humanitec Score specification
apiVersion: score.dev/v1b1
metadata:
  name: payment-service
  annotations:
    humanitec.io/app-id: payment-app
    humanitec.io/env-type: development
containers:
  api:
    image: company/payment-service:${VERSION}
    command: ["java", "-jar", "app.jar"]
    variables:
      DATABASE_URL: ${resources.database.url}
      CACHE_URL: ${resources.cache.url}
      LOG_LEVEL: ${values.log_level}
    resources:
      limits:
        cpu: "1000m"
        memory: "1Gi"
      requests:
        cpu: "100m"
        memory: "256Mi"
    livenessProbe:
      httpGet:
        path: /health
        port: 8080
      initialDelaySeconds: 30
      periodSeconds: 10
    readinessProbe:
      httpGet:
        path: /ready
        port: 8080
      initialDelaySeconds: 5
      periodSeconds: 5
service:
  ports:
    http:
      port: 8080
      targetPort: 8080
      protocol: TCP
resources:
  database:
    type: postgres
    class: ${values.database_class}
    metadata:
      annotations:
        humanitec.io/res-id: shared-postgres
 
  cache:
    type: redis
    class: ${values.cache_class}
   
  dns:
    type: dns
    class: default
   
  ingress:
    type: ingress
    class: default
    params:
      rules:
        - host: ${resources.dns.host}
          http:
            paths:
              - path: /
                backend:
                  service:
                    name: api
                    port: 8080
# Resource Definitions
---
apiVersion: humanitec.io/v1alpha1
kind: Resource
metadata:
  name: postgres-rds
spec:
  type: postgres
  driver_type: terraform
  driver_inputs:
    values:
      source:
        path: terraform/modules/rds
        rev: main
        url: https://github.com/company/terraform-modules.git
      variables:
        instance_class: ${context.app.id}-${context.env.id}
        database_name: ${context.app.id}_${context.env.id}
        engine_version: "14.7"
  criteria:
    - env_type: production
      class: high-performance
---
apiVersion: humanitec.io/v1alpha1
kind: Resource
metadata:
  name: redis-elasticache
spec:
  type: redis
  driver_type: terraform
  driver_inputs:
    values:
      source:
        path: terraform/modules/elasticache
        rev: main
        url: https://github.com/company/terraform-modules.git
      variables:
        node_type: cache.t3.micro
        num_cache_nodes: ${context.env.type == "production" ? 3 : 1}
  criteria:
    - env_type: production

🏗️ Building Custom IDPs

Sometimes organizations need custom IDPs tailored to their specific requirements. Here's a complete architecture for building one.

// src/server.ts - Custom IDP Backend
import express from 'express';
import { PrismaClient } from '@prisma/client';
import { GitHubAPI } from './integrations/github';
import { KubernetesAPI } from './integrations/kubernetes';
import { TerraformCloud } from './integrations/terraform';
import { MetricsCollector } from './metrics/collector';
const app = express();
const prisma = new PrismaClient();
// Service Catalog API
app.get('/api/services', async (req, res) => {
  const services = await prisma.service.findMany({
    include: {
      team: true,
      dependencies: true,
      deployments: true,
      metrics: {
        where: {
          timestamp: {
            gte: new Date(Date.now() - 24 * 60 * 60 * 1000)
          }
        }
      }
    }
  });
 
  res.json(services);
});
app.post('/api/services', async (req, res) => {
  const { name, description, teamId, template } = req.body;
 
  // Create service in database
  const service = await prisma.service.create({
    data: {
      name,
      description,
      teamId,
      template,
      status: 'provisioning'
    }
  });
 
  // Trigger provisioning workflow
  await provisionService(service);
 
  res.json(service);
});
async function provisionService(service: any) {
  try {
    // Create GitHub repository
    const repo = await GitHubAPI.createRepository({
      name: service.name,
      description: service.description,
      team: service.teamId,
      template: service.template
    });
   
    // Create Terraform workspace
    const workspace = await TerraformCloud.createWorkspace({
      name: service.name,
      repository: repo.url,
      autoApply: false
    });
   
    // Deploy to Kubernetes
    const deployment = await KubernetesAPI.createDeployment({
      name: service.name,
      namespace: service.teamId,
      image: `company/${service.name}:latest`,
      replicas: 2
    });
   
    // Update service status
    await prisma.service.update({
      where: { id: service.id },
      data: {
        status: 'active',
        repository: repo.url,
        terraformWorkspace: workspace.id,
        kubernetesDeployment: deployment.metadata.name
      }
    });
   
    // Start collecting metrics
    MetricsCollector.startCollection(service.id);
   
  } catch (error) {
    await prisma.service.update({
      where: { id: service.id },
      data: {
        status: 'failed',
        error: error.message
      }
    });
  }
}
// Self-Service Actions API
app.post('/api/actions/:actionId/execute', async (req, res) => {
  const { actionId } = req.params;
  const { inputs } = req.body;
 
  const action = await prisma.action.findUnique({
    where: { id: actionId }
  });
 
  if (!action) {
    return res.status(404).json({ error: 'Action not found' });
  }
 
  // Execute action based on type
  const execution = await executeAction(action, inputs);
 
  res.json(execution);
});
// Scorecards API
app.get('/api/scorecards/:serviceId', async (req, res) => {
  const { serviceId } = req.params;
 
  const scorecard = await calculateScorecard(serviceId);
 
  res.json(scorecard);
});
async function calculateScorecard(serviceId: string) {
  const service = await prisma.service.findUnique({
    where: { id: serviceId },
    include: {
      deployments: true,
      incidents: true,
      securityScans: true,
      costReports: true
    }
  });
 
  return {
    serviceId,
    scores: {
      reliability: calculateReliabilityScore(service),
      security: calculateSecurityScore(service),
      performance: calculatePerformanceScore(service),
      cost: calculateCostScore(service),
      compliance: calculateComplianceScore(service)
    },
    recommendations: generateRecommendations(service)
  };
}
// Database Schema (Prisma)
// schema.prisma
model Service {
  id String @id @default(cuid())
  name String @unique
  description String
  template String
  status String
  repository String?
  terraformWorkspace String?
  kubernetesDeployment String?
  teamId String
  team Team @relation(fields: [teamId], references: [id])
  dependencies Service[] @relation("ServiceDependencies")
  deployments Deployment[]
  metrics Metric[]
  incidents Incident[]
  securityScans SecurityScan[]
  costReports CostReport[]
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
}
model Team {
  id String @id @default(cuid())
  name String @unique
  slackChannel String
  email String
  services Service[]
  members User[]
}
model Deployment {
  id String @id @default(cuid())
  serviceId String
  service Service @relation(fields: [serviceId], references: [id])
  version String
  environment String
  status String
  deployedBy String
  deployedAt DateTime @default(now())
  rollbackTo String?
}

Custom IDP Frontend (React)

// src/components/ServiceCatalog.tsx
import React, { useState, useEffect } from 'react';
import {
  Card,
  Grid,
  TextField,
  Select,
  MenuItem,
  Button,
  Chip,
  Dialog,
  CircularProgress
} from '@mui/material';
import { useApi } from '../hooks/useApi';
import { ServiceScorecard } from './ServiceScorecard';
import { ServiceActions } from './ServiceActions';

interface Service {
  id: string;
  name: string;
  description: string;
  team: Team;
  status: string;
  template: string;
  dependencies: Service[];
  metrics: Metric[];
}

export const ServiceCatalog: React.FC = () => {
  const [services, setServices] = useState<Service[]>([]);
  const [filteredServices, setFilteredServices] = useState<Service[]>([]);
  const [searchTerm, setSearchTerm] = useState('');
  const [teamFilter, setTeamFilter] = useState('all');
  const [statusFilter, setStatusFilter] = useState('all');
  const [selectedService, setSelectedService] = useState<Service | null>(null);
  const [showCreateDialog, setShowCreateDialog] = useState(false);
  const api = useApi();

  useEffect(() => {
    loadServices();
  }, []);

  useEffect(() => {
    filterServices();
  }, [services, searchTerm, teamFilter, statusFilter]);

  const loadServices = async () => {
    const data = await api.get('/api/services');
    setServices(data);
  };

  const filterServices = () => {
    let filtered = services;
    if (searchTerm) {
      filtered = filtered.filter(s =>
        s.name.toLowerCase().includes(searchTerm.toLowerCase()) ||
        s.description.toLowerCase().includes(searchTerm.toLowerCase())
      );
    }
    if (teamFilter !== 'all') {
      filtered = filtered.filter(s => s.team.id === teamFilter);
    }
    if (statusFilter !== 'all') {
      filtered = filtered.filter(s => s.status === statusFilter);
    }
    setFilteredServices(filtered);
  };

  const getStatusColor = (status: string) => {
    switch (status) {
      case 'active': return 'success';
      case 'degraded': return 'warning';
      case 'failed': return 'error';
      default: return 'default';
    }
  };

  return (
    <div className="service-catalog">
      <div className="catalog-header">
        <h1>Service Catalog</h1>
        <Button
          variant="contained"
          color="primary"
          onClick={() => setShowCreateDialog(true)}
        >
          Create Service
        </Button>
      </div>
      
      <div className="catalog-filters">
        <TextField
          label="Search"
          variant="outlined"
          value={searchTerm}
          onChange={(e) => setSearchTerm(e.target.value)}
          style={{ marginRight: 16 }}
        />
        <Select
          value={teamFilter}
          onChange={(e) => setTeamFilter(e.target.value)}
          style={{ marginRight: 16, minWidth: 150 }}
        >
          <MenuItem value="all">All Teams</MenuItem>
          {/* Dynamic team options */}
        </Select>
        <Select
          value={statusFilter}
          onChange={(e) => setStatusFilter(e.target.value)}
          style={{ minWidth: 150 }}
        >
          <MenuItem value="all">All Status</MenuItem>
          <MenuItem value="active">Active</MenuItem>
          <MenuItem value="degraded">Degraded</MenuItem>
          <MenuItem value="failed">Failed</MenuItem>
        </Select>
      </div>
      
      <Grid container spacing={3} className="catalog-grid">
        {filteredServices.map(service => (
          <Grid item xs={12} md={6} lg={4} key={service.id}>
            <Card
              className="service-card"
              onClick={() => setSelectedService(service)}
            >
              <div className="service-card-header">
                <h3>{service.name}</h3>
                <Chip
                  label={service.status}
                  color={getStatusColor(service.status)}
                  size="small"
                />
              </div>
              <p>{service.description}</p>
              <div className="service-card-meta">
                <span>Team: {service.team.name}</span>
                <span>Template: {service.template}</span>
              </div>
              <div className="service-card-metrics">
                <div className="metric">
                  <span className="metric-label">Uptime</span>
                  <span className="metric-value">99.9%</span>
                </div>
                <div className="metric">
                  <span className="metric-label">Latency</span>
                  <span className="metric-value">45ms</span>
                </div>
                <div className="metric">
                  <span className="metric-label">Cost</span>
                  <span className="metric-value">$1,234</span>
                </div>
              </div>
            </Card>
          </Grid>
        ))}
      </Grid>
      
      {selectedService && (
        <Dialog
          open={true}
          onClose={() => setSelectedService(null)}
          maxWidth="lg"
          fullWidth
        >
          <div className="service-details">
            <h2>{selectedService.name}</h2>
            <ServiceScorecard service={selectedService} />
            <ServiceActions service={selectedService} />
          </div>
        </Dialog>
      )}
      
      <CreateServiceDialog
        open={showCreateDialog}
        onClose={() => setShowCreateDialog(false)}
        onCreated={loadServices}
      />
    </div>
  );
};

📊 Scorecards and Governance

Scorecards provide visibility into service health, compliance, and best practices adoption across your organization.

# scorecard-definition.yaml
apiVersion: scorecards.platform.io/v1
kind: Scorecard
metadata:
  name: production-readiness
  title: Production Readiness Scorecard
spec:
  targets:
    - kind: Service
      filter: 'tier == "1"'
 
  categories:
    - name: reliability
      weight: 30
      checks:
        - id: has-slo
          title: Has defined SLOs
          description: Service has defined Service Level Objectives
          query: |
            SELECT COUNT(*) > 0
            FROM slos
            WHERE service_id = :service_id
          weight: 10
         
        - id: has-runbook
          title: Has runbook
          description: Service has an up-to-date runbook
          query: |
            SELECT runbook_url IS NOT NULL
            AND runbook_updated_at > NOW() - INTERVAL '90 days'
            FROM services
            WHERE id = :service_id
          weight: 5
         
        - id: has-alerts
          title: Has monitoring alerts
          description: Service has appropriate monitoring alerts configured
          query: |
            SELECT COUNT(*) >= 5
            FROM alerts
            WHERE service_id = :service_id
            AND enabled = true
          weight: 10
         
        - id: has-autoscaling
          title: Has autoscaling enabled
          description: Service can scale automatically based on load
          query: |
            SELECT autoscaling_enabled = true
            FROM deployments
            WHERE service_id = :service_id
            AND environment = 'production'
          weight: 5
   
    - name: security
      weight: 30
      checks:
        - id: vulnerability-scanning
          title: Regular vulnerability scanning
          description: Service is scanned for vulnerabilities weekly
          query: |
            SELECT COUNT(*) > 0
            FROM security_scans
            WHERE service_id = :service_id
            AND scan_date > NOW() - INTERVAL '7 days'
            AND critical_vulnerabilities = 0
          weight: 10
         
        - id: secrets-management
          title: Uses secrets management
          description: Service uses proper secrets management
          query: |
            SELECT uses_vault = true
            OR uses_sealed_secrets = true
            FROM services
            WHERE id = :service_id
          weight: 10
         
        - id: tls-enabled
          title: TLS enabled
          description: All endpoints use TLS
          query: |
            SELECT COUNT(*) = 0
            FROM endpoints
            WHERE service_id = :service_id
            AND protocol = 'http'
          weight: 10
   
    - name: documentation
      weight: 20
      checks:
        - id: has-readme
          title: Has README
          description: Repository has a comprehensive README
          weight: 5
         
        - id: has-api-docs
          title: Has API documentation
          description: Service has OpenAPI/Swagger documentation
          weight: 10
         
        - id: has-architecture-diagram
          title: Has architecture diagram
          description: Service has an up-to-date architecture diagram
          weight: 5
   
    - name: cost-optimization
      weight: 20
      checks:
        - id: right-sized
          title: Resources are right-sized
          description: CPU and memory utilization is optimal
          query: |
            SELECT
              AVG(cpu_utilization) BETWEEN 40 AND 70
              AND AVG(memory_utilization) BETWEEN 40 AND 70
            FROM metrics
            WHERE service_id = :service_id
            AND timestamp > NOW() - INTERVAL '7 days'
          weight: 10
         
        - id: has-cost-tags
          title: Has cost allocation tags
          description: All resources have proper cost allocation tags
          weight: 5
         
        - id: uses-spot-instances
          title: Uses spot instances where appropriate
          description: Non-critical workloads use spot instances
          weight: 5
  scoring:
    levels:
      - name: Gold
        minScore: 90
        badge: 🏆
      - name: Silver
        minScore: 70
        badge: 🥈
      - name: Bronze
        minScore: 50
        badge: 🥉
      - name: Needs Improvement
        minScore: 0
        badge: ⚠️

☁️ Multi-Cloud Integration Patterns

Modern platforms need to support multiple cloud providers seamlessly. Here's how to implement multi-cloud support in your IDP.

# multi-cloud-provisioner.ts
import { CloudProvider, ResourceType, ProvisioningRequest } from './types';
export class MultiCloudProvisioner {
  private providers: Map;
  constructor() {
    this.providers = new Map([
      [CloudProvider.AWS, new AWSProvider()],
      [CloudProvider.GCP, new GCPProvider()],
      [CloudProvider.AZURE, new AzureProvider()],
      [CloudProvider.KUBERNETES, new KubernetesProvider()]
    ]);
  }
  async provision(request: ProvisioningRequest) {
    const provider = this.providers.get(request.provider);
    if (!provider) {
      throw new Error(`Unsupported provider: ${request.provider}`);
    }
    // Validate request
    await this.validateRequest(request);
    // Check quotas
    await this.checkQuotas(request);
    // Provision resources
    const resources = await provider.provision(request);
    // Register in service catalog
    await this.registerResources(resources);
    // Setup monitoring
    await this.setupMonitoring(resources);
    // Apply governance policies
    await this.applyPolicies(resources);
    return resources;
  }
  private async validateRequest(request: ProvisioningRequest) {
    // Validate team permissions
    const team = await this.getTeam(request.teamId);
    if (!team.canProvision(request.resourceType)) {
      throw new Error('Team lacks permission to provision this resource');
    }
    // Validate against organization policies
    const policies = await this.getPolicies();
    for (const policy of policies) {
      if (!policy.evaluate(request)) {
        throw new Error(`Policy violation: ${policy.name}`);
      }
    }
  }
  private async checkQuotas(request: ProvisioningRequest) {
    const quota = await this.getQuota(request.teamId, request.provider);
    const usage = await this.getCurrentUsage(request.teamId, request.provider);
   
    if (usage.cost + request.estimatedCost > quota.costLimit) {
      throw new Error('Cost quota exceeded');
    }
   
    if (usage.resources + request.resourceCount > quota.resourceLimit) {
      throw new Error('Resource quota exceeded');
    }
  }
  private async registerResources(resources: any[]) {
    // Register in catalog
    for (const resource of resources) {
      await this.catalogService.register(resource);
    }
  }
  private async setupMonitoring(resources: any[]) {
    // Configure Prometheus rules
    await this.monitoringService.addResources(resources);
  }
  private async applyPolicies(resources: any[]) {
    // Apply security policies
    await this.policyEngine.enforce(resources);
  }
}

📈 Platform Metrics and Observability

Implement comprehensive metrics and observability to measure platform effectiveness and identify improvement areas.

# grafana-dashboard.json
{
  "title": "Platform Engineering Dashboard",
  "panels": [
    {
      "type": "stat",
      "title": "Adoption Rate",
      "targets": [
        {
          "query": "sum(rate(service_creations_total[5m])) / sum(rate(total_requests[5m]))"
        }
      ]
    },
    {
      "type": "graph",
      "title": "Self-Service Actions",
      "targets": [
        {
          "query": "sum(rate(self_service_actions_total[5m])) by (action_type)"
        }
      ]
    },
    {
      "type": "table",
      "title": "Top Services by Usage",
      "targets": [
        {
          "query": "topk(10, sum(rate(service_usage[5m])) by (service_name))"
        }
      ]
    },
    {
      "type": "stat",
      "title": "Average Provisioning Time",
      "targets": [
        {
          "query": "histogram_quantile(0.95, sum(rate(provisioning_duration_bucket[5m])) by (le))"
        }
      ]
    }
  ]
}

🏢 Enterprise Architecture Patterns

Large organizations require advanced patterns for scaling platform engineering across multiple teams and business units.

Multi-Tenant Platform Architecture

# multi-tenant-k8s.tf
resource "kubernetes_namespace" "tenants" {
  for_each = var.tenants

  metadata {
    name = each.key
    labels = {
      "tenant" = each.key
    }
  }
}

resource "kubernetes_role_binding" "tenant_admin" {
  for_each = var.tenants

  metadata {
    name      = "tenant-admin"
    namespace = kubernetes_namespace.tenants[each.key].metadata.0.name
  }

  role_ref {
    api_group = "rbac.authorization.k8s.io"
    kind      = "ClusterRole"
    name      = "admin"
  }

  subject {
    kind      = "Group"
    name      = each.value.admin_group
    api_group = "rbac.authorization.k8s.io"
  }
}

resource "kubernetes_resource_quota" "tenant_quota" {
  for_each = var.tenants

  metadata {
    name      = "tenant-quota"
    namespace = kubernetes_namespace.tenants[each.key].metadata.0.name
  }

  spec {
    hard = {
      "requests.cpu"    = each.value.cpu_limit
      "requests.memory" = each.value.memory_limit
      "limits.cpu"      = each.value.cpu_limit * 1.2
      "limits.memory"   = each.value.memory_limit * 1.2
      "pods"            = each.value.pod_limit
    }
  }
}

resource "kubernetes_network_policy" "tenant_isolation" {
  for_each = var.tenants

  metadata {
    name      = "default-deny"
    namespace = kubernetes_namespace.tenants[each.key].metadata.0.name
  }

  spec {
    pod_selector {}
    policy_types = ["Ingress", "Egress"]
  }
}

resource "kubernetes_limit_range" "tenant_limits" {
  for_each = var.tenants

  metadata {
    name      = "container-limits"
    namespace = kubernetes_namespace.tenants[each.key].metadata.0.name
  }

  spec {
    limit {
      type = "Container"
      default = {
        cpu    = "500m"
        memory = "512Mi"
      }
      default_request = {
        cpu    = "100m"
        memory = "256Mi"
      }
      max = {
        cpu    = "2"
        memory = "2Gi"
      }
      min = {
        cpu    = "50m"
        memory = "128Mi"
      }
    }
  }
}