Peter Birkholm-Buch

Stuff about Software Engineering

DevEx Analysis: Software Engineering in Growth Products, Carlsberg vs Gartner Benchmarks

2024-10-15 /

Introduction

This report was created by having ChatGPT (Auto) and Claude individually compare and evaluate the initiatives on DevEx that I’ve blogged about in the past against the official Gartner DevEx Best Practices. I then had ChatGPT combine the two results into the report below.

Report

This report evaluates the Developer Experience (DevEx) initiatives within the Software Engineering department in Growth Products at Carlsberg, comparing them against Gartner’s The State of Developer Experience Initiatives report. The initiatives reviewed apply specifically to the Software Engineering team within Growth Products and do not reflect Carlsberg’s global operations.

1. Focus on Tooling and Integration

The Software Engineering team in Growth Products demonstrates strong alignment with Gartner’s recommendations in the area of tooling and integration:

Key Highlights:

  • Emphasis on integrated platforms and tooling, such as the Gaia platform
  • Focus on automation to streamline developer workflows
  • Adoption of AI-driven tools like GitHub Copilot to enhance developer efficiency

The initiatives outlined in “From DevOps to Platform Engineering: How Gaia Transformed Our Approach to Infrastructure Alignment and Developer Experience” highlight how integrated tooling supports smoother workflows, a priority emphasized in Gartner’s best practices.

Final Rating: 4.5/5. The focus on improving developer workflows through integrated platforms and automation closely aligns with Gartner’s recommendations.

2. Embedding Security into the Developer Workflow

The approach taken by the Software Engineering team aligns well with Gartner’s guidance on embedding security into daily developer processes:

Key Highlights:

  • Security integration as part of the daily development workflow
  • Use of tools such as GitHub Advanced Security
  • Inclusion of security roles within the broader DevEx initiative

The best practices detailed in “GitHub Advanced Security Enables Shifting Security Left” demonstrate how security has been embedded into the development pipeline, a practice highly recommended by Gartner for mature DevEx programs.

Final Rating: 4.5/5. The integration of security into the workflow, combined with tooling that supports shifting security left, positions the Software Engineering team’s DevEx initiatives at an advanced level in this area.

3. Emphasis on Professional Growth and Autonomy

The Software Engineering team’s emphasis on professional growth, particularly in relation to senior roles, aligns strongly with Gartner’s recommendations:

Key Highlights:

  • Empowering senior developers to act as facilitators and influencers
  • Promoting autonomy while ensuring organizational alignment
  • Balancing team independence with a controlled framework

The initiatives described in “Seniority, Organizational Influence, and Participation” provide a clear view of how senior developers are encouraged to take on leadership roles that foster cross-team collaboration and influence, aligning with Gartner’s focus on supporting professional growth and autonomy.

Final Rating: 5/5. The focus on autonomy and professional growth, especially for senior developers, represents an exemplary alignment with Gartner’s DevEx principles.

4. Cultural and Organizational Alignment

The initiatives to foster a supportive and collaborative developer culture align closely with Gartner’s recommendations for cultural and organizational alignment:

Key Highlights:

  • Clear definitions of expected behaviors across the Software Engineering department
  • Promotion of psychological safety within teams
  • Fostering a collaborative and inclusive work environment

The best practices outlined in “A Manifesto on Expected Behaviors” establish clear behavioral expectations that create a psychologically safe environment for developers. Gartner stresses the importance of a supportive culture in any comprehensive DevEx program, which these initiatives directly address.

Final Rating: 4.5/5. The Software Engineering department has established a strong cultural foundation that promotes collaboration and safety. Further emphasis on psychological safety could further enhance this area.

5. Measuring DevEx

While the Software Engineering team demonstrates strength in many areas, there is room for improvement in how DevEx outcomes are measured:

Key Highlights:

  • Focus on productivity and automation metrics
  • Some discussion of tooling-related metrics, such as in “GitHub Copilot Metrics”
  • Limited use of broader DevEx metrics, such as DORA or job satisfaction surveys

The current focus on productivity and efficiency aligns with Gartner’s recommendations. However, Gartner also recommends a broader range of metrics, including velocity, developer satisfaction, and retention. Expanding the measurement approach, as suggested in these practices, would provide a more comprehensive understanding of DevEx success.

Final Rating: 3/5. Increasing the use of specific DevEx outcome metrics, such as DORA and job satisfaction, would enhance the ability to measure and track the effectiveness of DevEx initiatives within the Software Engineering department.

Overall Rating: 4.3/5 (Strong Alignment)

The Software Engineering department’s DevEx efforts demonstrate strong alignment with Gartner’s best practices, particularly in the areas of tooling, security integration, autonomy, and culture. The main area for improvement is in the explicit measurement and tracking of DevEx outcomes through broader metrics.

Recommendations

Incorporating Gartner’s benchmarks, the following recommendations are suggested for the Software Engineering department in Growth Products:

  1. Implement More Specific DevEx Metrics: Expanding the use of DORA metrics, job satisfaction surveys, and other developer feedback mechanisms would provide deeper insights into the effectiveness of the DevEx initiatives.
  2. Continue Refining Integrated Tooling: Maintaining the momentum around tools like Gaia and GitHub Copilot will ensure the DevEx program continues to drive productivity and reduce friction in the development workflow.
  3. Sustain Focus on Security Integration and Culture: The strengths in security integration and developer culture should remain a focus, ensuring that these foundational pillars of the DevEx program continue to evolve.
  4. Consider Formalizing the DevEx Program: If not already formalized, creating a structured DevEx program with clear goals, metrics, and accountability would allow for better tracking of progress and outcomes.
  5. Regularly Assess and Iterate on DevEx Initiatives: Using feedback loops and metrics to continuously iterate on the DevEx approach will ensure that it evolves in line with both developer needs and organizational goals.

Conclusion:

The DevEx initiatives within the Software Engineering department in Growth Products at Carlsberg are well-aligned with Gartner’s best practices, particularly in terms of integrated tooling, security, professional growth, and cultural alignment. By expanding the metrics framework to include broader DevEx outcome measurements, the program could reach even higher levels of maturity and effectiveness.

Seniority, Organizational Influence and Participation

2024-10-15 /

The table below is meant to better explain how seniority, organizational influence and behaviors are connected. Anyone wanting to move up the career ladder must master our expected behaviors and participate accordingly.

StrengthEntryIntermediateExperiencedAdvancedExpert
ScopeEntry level professional with limited or no prior experience; learns to use professional concepts to resolve problems of limited scope and complexity; works on assignments that require limited judgment and decision making. Developing position where an employee is able to apply job skills, policies and procedures to complete tasks of moderate scope and complexity to determine appropriate action.Journey-level, experienced professional who knows how to apply theory and put it into practice with full understanding of the professional field; has broad job knowledge and works on problems of diverse scope.Professional with a high degree of knowledge in the overall field and recognized expertise in specific areas.Leader in the field who regularly leads projects of criticality to company and beyond, with high consequences of success or failure.  This employee has impact and influence on company policy and program development. Barriers of entry exist at this level.
AnalogyLearning about rope and knotsCan tie basic knots, learning complex knotsCalculates rope strength, knows a lot about knotsUnderstands rope making, can tie any knotKnows more about rope than you ever will, invented new knot
Influence & ImpactSelfPeersTeamDepartmentCompany
Coding100%100%80%-100%<25%<10%
ParticipationActively participate in discussions and team activities. Seek guidance from more experienced team members. Be open to receiving feedback and learning from mistakes.Contribute to team discussions and share knowledge gained from learning basic concepts. Offer assistance to junior team members. Seek feedback on performance and actively work on improving skills.Actively contribute expertise to team projects and discussions. Mentor junior team members and facilitate knowledge sharing sessions. Regularly seek out new learning opportunities and share insights with the team.Lead collaborative learning initiatives within the team. Actively contribute to the development of best practices and processes. Mentor and coach less experienced team members, fostering a culture of continuous learning.Serve as a subject matter expert, providing guidance and direction on complex projects. Spearhead innovative learning initiatives and contribute to industry knowledge sharing. Act as a mentor and coach for both technical and professional development.

A Manifesto on Expected Behaviors

2024-10-15 /

Introduction

In Software Engineering at Carlsberg our collective success is built on the foundation of individual behaviors that foster a positive, innovative, and collaborative environment. This document outlines the expected behaviors that every team member, irrespective of their role or seniority, is encouraged to embody and develop. These behaviors are not just guidelines but the essence of our culture, aiming to inspire continuous growth, effective communication, and a proactive approach to challenges. As we navigate the complexities of software engineering, these behaviors will guide us in making decisions, interacting with one another, and achieving our departmental and organizational goals. Together, let’s build a culture that celebrates learning, teamwork, and excellence in everything we do.

Learn together, grow together

“We embrace collaboration, share knowledge openly, and celebrate both individual and team success.”

  • Contribute and support: Actively participate in discussions, offer help, and celebrate each other’s successes.
  • Give and receive feedback: Regularly seek and provide constructive feedback for improvement.
  • Share expertise openly: Willingly share knowledge and expertise to benefit the team.

Communicate clearly, connect openly

“We foster understanding through respectful, transparent, and active communication using the right tools for the job.”

  • Listen actively, engage thoughtfully: Pay close attention, ask questions, and respond thoughtfully to diverse perspectives.
  • Clarity over jargon, respect in tone: Communicate with clarity, avoid technical jargon, and use respectful language.
  • Prompt and appropriate: Respond efficiently and tailor your communication to fit the situation and audience.
  • Choose the right channel: Utilize appropriate communication methods based on the message and context.

Continuous Learning and Improvement is the Way!

“We value continuous learning, actively seek opportunities to improve, and celebrate progress together.”

  • Quality First, Every Step of the Way: Never pass a known defect down the stream. If you see something which will cause problems for others then you should stop the work.
  • Challenge yourself and learn: Regularly seek new experiences and reflect on your experiences to improve.
  • Experiment and share: Be open to trying new things and share your learnings with the team.
  • Track your progress: Regularly measure your progress towards goals and adjust your approach as needed.

Own your work, drive results

“We take responsibility, proactively solve problems, and seize opportunities to excel.”

  • Embrace challenges, deliver excellence: Aim for impactful work and go the extra mile for outstanding results.
  • Be proactive problem-solvers: Actively seek, address, and prevent the escalation of challenges by ensuring solutions not only fit within established boundaries but also uphold the highest quality standards.
  • Learn and bounce back: Embrace mistakes as learning opportunities and quickly recover from setbacks.

The Intersection of DevEx and DevSecOps: We need a New Way Forward

2024-10-14 /

Developer Experience (DevEx) is critical for productivity, impact and retaining talent. In a world where software engineers are constantly asked to deliver more, faster, and more securely, companies can’t afford to treat DevEx and DevSecOps as separate priorities.

When these areas are siloed, we end up with fragmented workflows, frustrated developers, and disjointed experiences—counteracting the benefits of initiatives like unified development platforms. To move forward, we need an integrated approach to DevEx and DevSecOps, making security a seamless part of the development process while avoiding the fragmentation that current approaches have caused.

The current fragmented approach to DevSecOps is undermining Developer Experience. DevEx and DevSecOps serve different purposes, but poorly implemented DevSecOps practices can harm DevEx, reducing efficiency and developer satisfaction. It’s about ensuring security practices support developer productivity rather than interfere with it.

The Fragmentation Problem: A Warning for Growing Complexity

As organizations scale, it’s easy to fall into the trap of adding more tools to address new challenges—especially in security. Each new vulnerability or compliance requirement often results in adopting yet another tool. On the surface, this might seem like progress, but in reality, it adds complexity.

Each new platform comes with its own requirements, logins, and signals. Developers must toggle between different tools, piecing together information from multiple sources. This disrupts their workflow and increases the risk of errors. The very tools intended to improve security end up creating friction.

This fragmented approach seems common in many organizations. As more platforms are introduced, workflows become disjointed, and maintaining a unified process becomes harder. The result? Security becomes reactive, and developers spend less time building and improving software.

We need to rethink how we integrate security into the development process. A consolidated approach can help avoid these pitfalls while enhancing both security and productivity.

Our Success with Platform Consolidation: Improving Security and Developer Experience

At Carlsberg, we took a deliberate approach to consolidating our software development tools onto a single platform—GitHub—and used GitHub Advanced Security (GHAS) to shift security left into the developer workflow. This allowed us to address security vulnerabilities at their source, directly within the tools developers are already familiar with.

By integrating security into the developer workflow, our developers could use AI-powered tools like GitHub Copilot to write more secure code as they worked. This approach streamlined the process, reducing the need for developers to toggle between multiple platforms and ensuring that the code we deployed was free from known security vulnerabilities at the time of writing. The impact on Developer Experience (DevEx) has been significant—security is now a natural part of the development process, not an afterthought.

This consolidation not only raised our security posture but also improved developer productivity. By reducing context-switching and embedding security into the natural flow of work, we created a more cohesive, efficient development environment where developers felt empowered to take ownership of both the code and its security.

The Opposite Trend in DevSecOps: Tool Fragmentation and Complexity

While we’ve seen success in consolidating our platform and raising both security and Developer Experience, it’s the norm for many organizations to face the opposite challenge. When implementing DevSecOps, the introduction of more security tools often leads to a fragmented workflow. Developers are required to interact with multiple platforms, each with its own set of logins, signals, and processes, which disrupts their focus and lowers productivity.

Research has shown that this tool-centric approach to DevSecOps can lead to operational gaps, inefficiencies, and a disjointed developer experience. The very tools designed to improve security end up creating friction, making it harder for developers to get their work done. In addition, the immaturity of some automated DevSecOps tools further complicates integration into continuous delivery pipelines, undermining both security and efficiency.

This fragmentation isn’t specific to any one organization; it’s a widespread challenge as security teams strive to keep up with growing threats and compliance demands. The proliferation of tools, however, often leads to more silos and increased complexity—exactly the opposite of what we’ve achieved through platform consolidation.

A Call for Streamlining DevSecOps: Learning from Consolidation

The lesson here is clear: adding more tools to the mix isn’t the answer. To fully realize the potential of DevSecOps, we need to move away from tool fragmentation and focus on embedding security into the developer workflow, as we did with our consolidated platform on GitHub. By simplifying the development process and integrating security from the start, we can achieve better outcomes for both security and Developer Experience.

Security needs to be central, not an afterthought. Rather than bolting on security measures after the fact or adding layers of complexity with new tools, security should be a seamless part of how developers work. By making security a core aspect of the development process, we ensure that it is baked in from the very beginning. This approach not only improves security itself but also enhances the overall Developer Experience by reducing the friction and overhead often associated with traditional security processes.

References

1. DevSecOps People: “Identifying the Primary Dimensions of DevSecOps: A Multi-vocal Literature Review,” discusses the fragmentation of DevSecOps and the challenge of integrating multiple tools into a seamless workflow. https://www.sciencedirect.com/science/article/pii/S0164121224001080

2. AI for DevSecOps: A Landscape and Future Opportunities: This paper outlines the potential of AI in automating and enhancing security tasks within DevSecOps pipelines, but also highlights challenges around tool complexity and immaturity. https://arxiv.org/abs/2404.04839

From DevOps to Platform Engineering: How Gaia Transformed Our Approach to Infrastructure, Alignment, and Developer Experience

2024-10-05 /

Introduction

In the world of cloud development, managing infrastructure effectively while maintaining alignment across teams is a constant challenge. Historically, our DevOps team played a pivotal role in provisioning and managing cloud resources, ensuring developers had what they needed to build and deploy solutions. However, this model wasn’t sustainable as the number of projects grew and cloud environments became more complex. We needed a way to streamline infrastructure management without losing sight of alignment across teams and solutions, while also improving the overall Developer Experience (DevEx).

This realization led us to shift our DevOps team from a traditional support role into a platform engineering team, focused on building and maintaining tools that provide a golden path for developers. The result? Gaia—a platform that has radically transformed how we manage cloud infrastructure, maintain alignment throughout the organization, and drastically improve Developer Experience by embedding infrastructure creation into developers’ existing workflows.

The Evolution from DevOps to Platform Engineering

When we started, our DevOps team handled infrastructure provisioning manually and on a request basis. While this ensured quality control, it created bottlenecks as the number of requests grew, leading to slower project deliveries. Developers were often left waiting for infrastructure to be set up, while the DevOps team struggled to keep up with the workload.

This wasn’t a scalable model, so we pivoted. Rather than manually provisioning infrastructure, we built Gaia—a platform that automates infrastructure creation while maintaining alignment with company policies. Gaia represents our “golden path”—a set of pre-built modules that allow developers to provision cloud resources without needing to worry about governance, security, or configuration inconsistencies.

Not only did Gaia eliminate bottlenecks, but it also integrated directly into the GitHub workflow developers were already using, significantly improving Developer Experience. Developers now interact with the same tools they use for coding, making infrastructure requests feel like a natural extension of their development work.

The Remarkable Impact of Gaia on Developer Experience

Gaia’s impact has been nothing short of remarkable. By automating the infrastructure creation process, we’ve effectively removed the need for the DevOps team to manually create infrastructure for developers. Developers now have a self-service capability to quickly and easily provision what they need on their own, directly from within their existing GitHub workflows, without waiting for approval or intervention from the DevOps team.

This seamless integration has significantly improved Developer Experience in several key ways:

  • Familiarity: Developers don’t have to learn new tools or processes to request infrastructure. They use GitHub, the platform they are already familiar with, ensuring minimal friction when interacting with infrastructure.
  • Speed and Efficiency: With Gaia, infrastructure requests are submitted via GitHub pull requests (PRs), allowing developers to spin up resources quickly. This eliminates the lag time that often occurs when requests are handled through manual ticketing systems.
  • Embedded Governance: Developers no longer have to worry about compliance or governance rules. Every infrastructure resource created via Gaia is automatically aligned with company policies, freeing developers to focus entirely on building solutions without getting bogged down in regulatory details.

By embedding infrastructure creation into the developer workflow through GitHub, Gaia significantly boosts DevEx. Developers are empowered to take control of infrastructure setup, while still benefiting from built-in quality and governance checks that ensure alignment with the company’s standards.

The New Focus of Our Platform Engineering Team

With manual infrastructure creation largely eliminated, the role of the DevOps team has shifted to that of a platform engineering team. Their primary focus is now on maintaining Gaia and the shared modules that are used to provision infrastructure. Whenever new infrastructure resources or cloud services are introduced, the team ensures they are incorporated into Gaia in a way that adheres to company policies, ensuring alignment as our cloud architecture evolves.

This centralized approach allows the platform engineering team to ensure that the development process is as smooth as possible, enhancing the overall Developer Experience by constantly improving the tools developers rely on. Developers no longer need to spend time learning about the intricacies of cloud infrastructure or worry about whether their configurations meet governance requirements.

Integrating Infrastructure Creation into the Developer Workflow

One of the most significant achievements of Gaia is how seamlessly it integrates into the developer workflow. As mentioned, we built Gaia to work within a central repository in GitHub, where developers create pull requests to request infrastructure. These PRs are then reviewed and approved by the platform engineering team, ensuring that every infrastructure change aligns with company policies and best practices.

By embedding infrastructure creation into the PR process, we’ve achieved several goals:

  • Speed: Developers can request infrastructure as part of their normal workflow, without delays or waiting for separate approvals.
  • Quality Control: The PR process provides a natural checkpoint for the platform engineering team to ensure consistency and alignment across all teams and solutions.
  • Alignment: Centralizing infrastructure requests in a single repository ensures that all teams are working from the same set of standards, preventing silos and ensuring that every team follows best practices.
  • Enhanced Developer Experience: Since developers no longer need to switch between tools or wait for external teams, the process feels fluid and integrated. This reduces the cognitive load on developers and enables them to focus more on writing code and building features rather than managing infrastructure logistics.

Gaia’s GitHub-based process has streamlined how developers interact with infrastructure, further aligning infrastructure creation with developer workflows and enhancing their experience by reducing friction and improving productivity.

Conclusion

The transition from a traditional DevOps model to a platform engineering team centered around Gaia has been a game changer for us. By providing developers with a golden path for creating infrastructure, we’ve freed up their time to focus on what they do best: building innovative software solutions. At the same time, we’ve ensured that every infrastructure deployment is aligned with our policies and governance frameworks, without the need for constant oversight.

Gaia has made our infrastructure provisioning faster, more reliable, and more scalable, while allowing our platform engineering team to focus on higher-level work—maintaining the tools that enable this. By embedding infrastructure creation into GitHub workflows, we’ve also enhanced Developer Experience, making infrastructure provisioning a natural extension of the development process.

The future of DevOps, for us, lies in platform engineering, where teams enable developers rather than managing infrastructure requests. Alignment and Developer Experience are no longer afterthoughts—they’re built into the process, ensuring that as we scale, we do so efficiently, consistently, and with a developer-centric approach.

Gaia was built by:

Balancing Autonomy and Alignment in Engineering Teams

2024-10-01 /

The Spotify model has often been referenced as a way to structure engineering teams for agility and independence. It promotes business-owned product teams, where engineers report into product owners, and uses guilds to ensure that teams stay aligned on best practices. However, guilds often become more like “book clubs,” where participation is optional and relies on personal time. This happens because business line managers prioritize deliverables over cross-organizational collaboration, making it difficult to maintain alignment at scale.

Meanwhile, Team Topologies offers a different focus, looking at how different types of teams interact and organize. It doesn’t rely on guilds but instead emphasizes reducing dependencies and clarifying team responsibilities.

One of the main reasons I organize engineers into a single reporting line, rather than under product ownership, is to avoid these pitfalls. By centralizing the reporting structure, I can prioritize time for engineers to focus on cross-organizational standards and collaboration, ensuring alignment across teams without relying on optional participation.

The Importance of Alignment and Shared Processes

While models like Spotify emphasize team independence, they sometimes miss the mark on alignment. It’s critical that teams don’t end up siloed, each solving the same problems in different ways, or worse, working against established company practices. This is where alignment on best practices, methods, and tools becomes crucial.

Take the US Navy SEAL teams as an example. They are known for their ability to operate independently, much like Scrum teams. However, what people tend to overlook is that all SEAL teams undergo the same training, use the same equipment, and follow standardized methods and processes. This shared foundation is what allows them to operate seamlessly when they come together, even though they work independently most of the time.

In the same way, my approach ensures that engineering teams can solve problems on their own, but they’re aligned on best practices, tools, and processes across the company. This alignment prevents the issues often seen in the Spotify model, where teams risk becoming too focused on their own product work, losing sight of the bigger organizational picture.

Scrum Teams Need Independence from Authority

In Scrum teams, the issue goes beyond just estimation—it’s about the entire collaboration model. Scrum is designed to foster equal collaboration, where team members work together, estimate tasks, and solve problems without a hierarchy influencing decisions. When someone on the team, such as a Product Owner, is also the boss, this balance is broken. The premise of Scrum, which relies on collective responsibility and open communication, collapses.

If the Product Owner or any other leader on the team has direct authority over the others, it can lead to a situation where estimates are overridden, team members feel pressured to work longer hours, and decisions are driven by power dynamics rather than collaboration. This undermines the core principles of Scrum, where the goal is for teams to self-organize and be empowered to make their own decisions.

By keeping authority structures out of the Scrum team, we ensure that collaboration is truly equal, and that decisions are made based on the team’s expertise and collective input—not on the directives of a boss.

How We Balance Autonomy and Alignment

Instead of organizing engineers strictly around product owners and budgets—like in the Spotify model—we’ve created a framework where engineers report through a central engineering line. This keeps everyone on the same page when it comes to methods and processes. Engineers still work closely with product teams, but they don’t lose sight of the bigger picture: adhering to company-wide standards.

This approach solves a problem common in both the Spotify and Team Topologies models. In Spotify, squads may go off and build things their way, leading to inconsistencies across the organization. In Team Topologies, stream-aligned teams can become too focused on optimizing their flow, which sometimes means diverging from company-wide practices. By maintaining a central engineering line, we keep our teams aligned while still giving them the autonomy they need to innovate and move quickly.

The Result

Our approach strikes a balance. Teams are free to innovate and adapt to the challenges of their product work, but they aren’t reinventing the wheel or deviating from best practices. We’ve managed to avoid the pitfalls of silos and fragmented processes by ensuring that every team operates within a shared framework—just like how SEAL teams can work independently, but they all share the same training, tools, and methods.

At the end of the day, it’s not about limiting autonomy; it’s about creating the right kind of autonomy. Teams should be able to act independently, but they should do so in a way that keeps the organization moving in the same direction. That’s the key to scaling effectively without losing sight of what makes us successful in the first place.

MACH Architecture: The Promise of Speed, The Reality of Integration Complexity

2024-09-20 /

Introduction

In recent years, the MACH principles—Microservices, API-first, Cloud-native, and Headless—has been touted as the future of composable, agile digital platforms. Advocates of MACH argue that it allows organizations to innovate quickly by decoupling core systems and exposing functionality through APIs. However, as appealing as this modular approach is, it doesn’t introduce fundamentally new principles. In fact, it builds on ideas that have been around for decades, particularly from Service-Oriented Architecture (SOA). See also The Four Tenets of SOA.

Yet, the biggest challenge lies not in the principles or technologies themselves but in the complexity of integration. While MACH tends to gloss over integration issues by positioning APIs as the ultimate solution, the reality is much more nuanced—especially when viewed through the lens of Pace Layered Architecture.

MACH vs. SOA: New Technology, Old Principles

MACH principles are often positioned as cutting-edge compared to older architectures like SOA. However, if we break it down:

  • Microservices in MACH are essentially an extension of SOA’s focus on service decomposition.
  • API-first continues SOA’s emphasis on loose coupling through well-defined interfaces.
  • Cloud-native leverages modern cloud infrastructure, but the idea of distributed systems was central to SOA.
  • Headless separates the front-end from the back-end, much like SOA’s separation of presentation and business layers.

In essence, MACH doesn’t introduce new architectural principles but rather modernizes existing ones with updated technology stacks and operational patterns.

The Forgotten Challenge: Integration

While MACH focuses on modularity, flexibility, and speed, it tends to oversimplify the complexities of integration. Simply saying “APIs solve everything” is reductive. APIs are indeed critical, but the actual process of integrating systems—especially in complex, distributed environments—requires addressing far more nuanced challenges. Integration is not a one-size-fits-all solution; it involves handling synchronous, asynchronous, and event-driven communication depending on the use case.

For example:

  • Synchronous vs. Asynchronous: Synchronous APIs may introduce latency or timeout issues, while asynchronous patterns require careful coordination to ensure data consistency across systems.
  • Event-Driven Architectures: While event-driven patterns can reduce complexity at certain levels, they also introduce new challenges, like handling event sequencing and guaranteeing delivery in distributed systems.

For even more complexity watch “I Made Everything Loosely Coupled. Does My App Fall Apart?” by Gregor Hohpe.

Pace Layered Architecture: A Critical Lens

One of the most valuable ways to understand the complexity of integration is through the Pace Layered Architecture model. This model divides systems based on their rate of change:

  • Systems of Record: These foundational systems (like ERP and finance) change slowly and are stable. Integration at this layer is generally easier because the interfaces are well-defined and don’t change often.
  • Systems of Differentiation: These systems allow for more customization and differentiation, such as customer loyalty programs or product catalogs, and change moderately.
  • Systems of Innovation: Fast-moving, innovation-driven systems, such as user interfaces or headless CMS, evolve rapidly.

If we look at the MACH principles through the lens of a Pace Layered Architecture, then the further down the stack, the slower the systems change and the easier the integration becomes. Conversely, at the top of the stack—where MACH promises speed and flexibility—the challenge of integrating diverse systems and managing rapid change becomes significantly harder.

The Reality of Integration Complexity

As the Pace Layered Architecture model shows, systems lower in the stack (such as ERP or finance) are easier to integrate because they change less often. However, the MACH promise of speed, especially in the higher layers (front-end, APIs), requires solving difficult integration problems. 

In the real world, APIs alone don’t automatically resolve the complexities of integrating systems across different domains and layers. You need to account for:

  • Latency and data consistency: Ensuring real-time data syncs across systems that update at different speeds.
  • Governance and ownership: Defining who controls the data and how changes are propagated across services.
  • Error handling and recovery: Building resilience into systems so that failures in one service don’t cascade into others.

Conclusion

MACH architecture brings modern technology to the table, but it doesn’t fundamentally change the underlying principles of service orientation or integration. While APIs are a key enabler of composable systems, they don’t eliminate the complexities of integration—especially in a world where synchronous, asynchronous, and event-driven patterns coexist. 

Understanding these challenges through the lens of Pace Layered Architecture helps us see that the deeper we go into the stack, the easier integration becomes. But achieving the speed and flexibility that MACH promises at the higher layers requires solving some of the hardest integration problems in modern software architecture.

Do you have GitHub Copilot?

2024-08-30 /

Is a question I’ve been getting more and more at job interviews over the past year and when I say yes we’ve been using it for almost two years I see happy faces.

So having access to GitHub Copilot not only is a key decision making factor for software engineers looking to join your organization but also GitHub Copilot Probably Saves 50% of Time for Developers and GitHub Copilot drives better Developer Experience.

GitHub was also named a Leader in the Gartner first-ever Magic Quadrant for AI Code Assistants: https://github.blog/news-insights/company-news/github-named-a-leader-in-the-gartner-first-ever-magic-quadrant-for-ai-code-assistants

So if you’re a Software Engineering Leader there’s really no (business) reason not to get GitHub Copilot (or any other AI Coding Assistant) for your developers – it will (soon) be a requirement by new hires.

Standard: Event Driven Architectures

2024-08-29 /

This page is an example standard which explains how all systems in Software Engineering in Digital Products in Carlsberg must use event driven architectures.

For context please see: Software Architecture Patterns

This means that each system or service that masters data (Order Service, Customer Service etc.) must raise (create, produce, publish, etc) an appropriate event to the central event hub using a specific topic every time a pre-defined action (business or just plain CRUD) occurs on data in that system or service. Systems or services which listens to these events (consumers, subscribers etc) can then based on meta data in the event decide if it’s required to go to the system or service API and get the full data which is associated with the event.

This means that every system or service which we build must support the following architecture pattern:

This is the order of events:

  • The client performs an action through the API on Service A
  • The change of state on the data in the Service causes the Service to raise an event on the central event hub on a specific topic B
  • The event hub notifies all the consumers that an event on topic B has occurred
  • The consumer examines the event and decides if it’s required to call the producers API to get the entire data of the event
  • The consumer call the producers API with a direct link to get the entire data of the event (ideally using a unique identifier so that the data can be found in the in storage layer without having to search for it)

Building a Better Software Practice: A Guide to Policies, Rules, Standards, Processes and Guidelines

2024-08-26 /

Introduction

When organizations add more people (scale up) it quickly becomes impossible to have everyone sitting in the same room to discuss and agree on matters and share the same approach to doing business.

This is my version of a Software Engineering Quality Handbook where it’s easy to get an overview of how an organization can use a well-structured hierarchy for describing how work is done which is crucial for ensuring clarity, consistency, and compliance within a team. The handbook is built up with 5 layers:

  • Policies: Broad statements that define the organization’s principles and compliance expectations.
  • Rules: Strict directives that must be followed to ensure specific outcomes in certain situations.
  • Standards: Mandatory technical and operational requirements that ensure consistency and quality.
  • Processes: Detailed, step-by-step instructions that specify how to perform specific tasks.
  • Guidelines: Recommended best practices that guide decision-making but are not mandatory.

Each layer of this structure supports the one above it, providing more detail and specificity. Policies and rules sets the foundation and create alignment within the goals of Software Engineering, while standards, procedures and guidelines provide the specifics on how to achieve those goals effectively.

This structure also facilitates easier updates and management. Policies and standards often require more thorough review and approval processes due to their impact and scope, whereas guidelines and procedures can be more dynamic, allowing for quicker adaptations to new technologies or methodologies..

Policies :scroll:

  • Definition: Broad, high-level statements of principles, goals, and overall expectations of the organization.
  • Purpose: To establish core values, company vision, and overarching compliance standards.
  • Lifecycle: Reviewed annually to ensure alignment with evolving legal, technological, and business conditions.
  • Compliance: Mandatory; non-compliance can result in significant legal and business risks.
  • Icon: :scroll: (scroll). It symbolizes official documents, which aligns well with the formal, foundational nature of policies. It’s often used to represent ancient laws and decrees, making it fitting for the foundational rules and standards within an organization.
  • Example: A policy might state that all software developed must comply with GDPR and other relevant data protection regulations.

Rules :scales:

  • Definition: Explicit, often granular directives that are compulsory and usually narrow in scope.
  • Purpose: To ensure specific outcomes or behaviors in particular scenarios.
  • Lifecycle: Reviewed frequently (e.g., annually) to refine and ensure they address current challenges effectively and are adhered to.
  • Compliance: Strictly mandatory; non-negotiable and must be followed exactly as prescribed.
  • Icon: :scales: (scales). It represents justice, balance, and fairness, aligning with the concept of rules ensuring specific outcomes and behaviors are maintained in a structured and equitable manner within an organization.
  • Example: A rule might state that commit messages must include a ticket number from the issue tracker.

Standards :straight_ruler:

  • Definition: Specific mandatory requirements for how certain policies are to be implemented.
  • Purpose: To ensure consistency and quality across all projects by defining technical and operational criteria that must be met.
  • Lifecycle: Updated biennially or as needed to reflect new industry practices and technological advancements.
  • Compliance: Mandatory; essential for maintaining quality and uniformity in outputs.
  • Icon: :straight_ruler: (ruler). It symbolizes measurement, precision, and consistency, which align closely with the idea of standards setting specific requirements and guidelines to ensure quality and uniformity across projects.
  • Example: A standard might specify that all code must undergo peer review or adhere to a particular coding standard like ISO/IEC 27001 for security.

Processes :tools:

  • Definition: Detailed, step-by-step instructions that must be followed in specific situations.
  • Purpose: To ensure activities are performed consistently and effectively, especially for complex or critical tasks.
  • Lifecycle: Regularly tested and updated, ideally after major project milestones or annually, to adapt to process improvements and feedback.
  • Compliance: Mandatory where specified; critical for ensuring consistency and reliability of specific operations.
  • Icon: :tools: (hammer and wrench). It symbolizes tools and construction, fitting for the concept of processes as they provide detailed, step-by-step instructions necessary to construct or execute specific tasks systematically and efficiently.
  • Example: A process might outline the steps for a release process, including code freezes, testing protocols, and deployment checks.

Guidelines :compass:

  • Definition: Recommended approaches that are not mandatory but are suggested as best practices.
  • Purpose: To guide developers in their decision-making processes by providing options that align with best practices.
  • Lifecycle: Evaluated and possibly revised every two to three years, or more frequently to incorporate innovative techniques and tools.
  • Compliance: Optional; best practice recommendations that are advisable but not required.
  • Icon: :compass: (compass). It symbolizes guidance, direction, and navigation, which aligns well with the purpose of guidelines to provide recommended approaches and best practices that help steer decisions in software development.
  • Example: Guidelines might suggest using certain frameworks or libraries that enhance productivity and maintainability but are not strictly required.
« Older posts
Peter Birkholm-Buch
Søvej 6
2665 Vallensbæk Strand
Mobil 26560203
Email peterbb@gmail.com
CVR: 34513287
Bank: Nordea
Konto: 5510-0747692815
SWIFT: NDEADKKK
IBAN: DK3620000747692815
Bitcoin: 1NiBdnmFGhZN2hpM5BRNHqCSD8wpyYNvZf

Follow me on Twitter

See my profile on LinkedIn

© 2024 Peter Birkholm-Buch

Theme by Anders Noren — Up ↑