Stuff about Software Engineering
I was honoured to be invited to speak at the 31st Expert Session at the https://www.evolve-community.com.
We touched on a lot of subjects around Software Engineering and mostly AI.
Thank you Carlos Monteiro and Gustavo Valle for inviting me.
Here are some links to posts which covers some of the topics we spoke about in more detail:
Artificial intelligence (AI) technologies have caused a dramatic change in software engineering. At the forefront of this revolution are AI-Engineers – the professionals who implement solutions within the ‘Builders’ category of AI adoption, as I outlined in my previous post on Four Categories of AI Solutions. These engineers not only harness the power of AI but also redefine the landscapes of industries.
As I recently discussed in “AI-Engineers: Why People Skills Are Central to AI Success” organizations face a critical talent shortage in AI implementation. McKinsey’s research shows that 60% of companies cite talent shortages as a key risk in their AI adoption plans. This shortage makes understanding the AI Engineer role and its distinct skillset more crucial than ever.
But what is an AI-Engineer?
AI-Engineers are skilled software developers who can code in modern languages. They create the frameworks and software solutions that enable AI functionalities and make them work well with existing enterprise applications. While they need basic knowledge of machine learning and AI concepts, their primary focus differs from data engineers. Where data engineers mainly focus on writing and managing data models, AI-Engineers concentrate on building reliable, efficient, and scalable software solutions that integrate AI.
The role of AI-Engineers is crucial in translating technological advancements into strategic advantages. Their ability to navigate the complex landscape of AI tools and tailor solutions to specific business challenges underlines their unique role within the enterprise and software engineering specifically. By embedding AI into core processes, they help streamline operations and foster innovative product development.
As I described in “Keeping Up with GenAI: A Full-Time Job?” the AI landscape shifts at a dizzying pace. Just like Loki’s time-slipping adventures, AI-Engineers find themselves constantly jumping between new releases, frameworks, and capabilities – each innovation demanding immediate attention and evaluation.
For AI-Engineers, this isn’t just about staying informed – it’s about rapidly evaluating which technologies can deliver real value. The platforms and communities that facilitate this learning, such as Hugging Face, become essential resources. However, merely keeping up isn’t enough. AI-Engineers must develop a strategic approach to:
The real value of AI-Engineers becomes clear when we look at concrete implementation data. In my recent analysis of GitHub Copilot usage at Carlsberg (“GitHub Copilot Probably Saves 50% of Time for Developers“), we found fascinating patterns in AI tool adoption. While developers and GitHub claim the tool saves 50% of development time, the actual metrics tell a more nuanced story:
This real-world example highlights several key aspects of the AI-Engineer role:
These findings demonstrate why AI-Engineers need both technical expertise and practical judgment. They must balance the promise of AI automation with the reality of implementation, ensuring that AI tools enhance rather than complicate development processes.
AI Engineering is undeniably a distinct skill set, one that is becoming increasingly indispensable in AI transformation. As industries increasingly rely on AI to innovate and optimize, the demand for skilled AI Engineers who can both understand and shape this technology continues to grow. Their ability to navigate the rapid pace of change while delivering practical business value makes them essential to successful AI adoption. Most importantly, their role in critically evaluating and effectively implementing AI tools – as demonstrated by our Copilot metrics – shows why this specialized role is crucial for turning AI’s potential into real business value.
In my blog post “Four Categories of AI Solutions” (https://birkholm-buch.dk/2024/04/22/four-categories-of-ai-solutions), I outlined different approaches to building AI solutions, but it’s becoming increasingly clear that the decision on how to approach AI hinges on the talent and capabilities within the organization. As AI continues to evolve at lightning speed, companies everywhere are racing to adopt the latest innovations. Whether it’s generative AI, machine learning, or predictive analytics, organizations see AI as a strategic advantage. But as exciting as these technologies are, they come with a less glamorous reality—people skills are the make-or-break factor in achieving long-term AI success.
Reports from both McKinsey and Gartner consistently highlight a serious shortage of skilled AI talent. McKinsey’s latest research suggests that AI adoption has plateaued for many companies not due to a lack of use cases, but because they don’t have the talent required to execute their AI strategies effectively. 60% of companies cite talent shortages as a key risk in their AI adoption plans. (McKinsey State of AI 2023: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2023-generative-ais-breakout-year).
This talent crunch means that organizations are finding it difficult to retain and attract professionals with the skills needed to develop, manage, and scale AI initiatives. 58% of organizations report significant gaps in AI talent, with 72% lacking machine learning engineering skills and 68% short on data science skills (McKinsey AI Tech Talent Survey: https://www.mckinsey.com/capabilities/quantumblack/our-insights/new-mckinsey-survey-reveals-the-ai-tech-talent-landscape).
Given the rapid pace of change in AI tools and techniques, developing an internal team of AI Engineers is one of the most sustainable strategies for companies looking to stay competitive. AI Engineers are those with the expertise to design, build, and maintain custom AI solutions tailored to the specific needs of the organization.
By 2026, Gartner predicts that 30% of new AI systems will require specialized AI talent, making the need for AI Builders even more urgent (Gartner 2023 Hype Cycle for AI: https://www.gartner.com/en/newsroom/press-releases/2023-08-17-gartner-identifies-key-technology-trends-in-2023-hype-cycle-for-artificial-intelligence).
The retention challenge in AI isn’t just about compensation—it’s about providing meaningful work, opportunities for continuous learning, and a sense of ownership over projects. Diverse AI teams are essential for preventing biases in models and creating more robust, well-rounded AI systems. Offering inclusive, supportive environments where AI engineers can grow professionally and personally is essential to keeping top talent engaged (McKinsey AI Tech Talent Survey: https://www.mckinsey.com/capabilities/quantumblack/our-insights/new-mckinsey-survey-reveals-the-ai-tech-talent-landscape).
AI is only as good as the team behind it. As Gartner highlights, the true value of AI will be realized when companies can seamlessly integrate AI tools into their existing workflows. This integration requires not just the right tools but the right people to design, deploy, and optimize these solutions (Gartner 2023 Hype Cycle for AI: https://www.gartner.com/en/newsroom/press-releases/2023-08-17-gartner-identifies-key-technology-trends-in-2023-hype-cycle-for-artificial-intelligence).
Companies that prioritize skills development will have a competitive edge. AI is advancing so quickly that organizations can no longer rely solely on external vendors to provide off-the-shelf solutions. Building an internal team of AI Builders—engineers who are continually learning and improving their craft—is essential to staying ahead of the curve. Offering employees opportunities to reskill and upskill is no longer optional; it’s a necessity for retaining talent and remaining competitive in the AI-driven economy.
As organizations continue to explore AI adoption, the need for specialized AI models is becoming more apparent. Gartner predicts that by 2027, over 50% of enterprises will deploy domain-specific Large Language Models (LLMs) tailored to either their industry or specific business functions (Gartner 2023 Hype Cycle for AI: https://www.gartner.com/en/newsroom/press-releases/2023-08-17-gartner-identifies-key-technology-trends-in-2023-hype-cycle-for-artificial-intelligence).
I believe the role of AI Engineers will continue to grow in importance and complexity. We might see new specializations emerge, such as AI ethics officers ensuring responsible AI use, or AI-human interaction designers creating seamless experiences.
In my view, the most valuable AI Engineers of the future could be those who can not only master the technology but also understand its broader implications for business and society. They might need to navigate complex ethical considerations, adapt to rapidly changing regulatory landscapes, and bridge the gap between technical capabilities and business needs.
The success of any AI initiative rests on the quality and adaptability of the people behind it. For organizations aiming to lead in the AI space, the focus should be on creating a workforce of AI Engineers—skilled professionals who can navigate the ever-changing landscape of AI technologies.
The lesson is clear: to succeed with AI, invest in your people first. The future of AI is not just about algorithms and data—it’s about the humans who will shape and guide its development and application.
As a technologist, it feels like I’m constantly time slipping — like Loki in Season 2 of his show. Every time I’ve finally wrapped my head around one groundbreaking AI technology, another one comes crashing in, pulling me out of my flow. Just like Loki’s painful and disorienting jumps between timelines, I’m yanked from one new release or framework to the next, barely catching my breath before being dropped into the middle of another innovation.
In the last week alone, we’ve had a flood of announcements that make it impossible to stand still for even a second. Here’s just a glimpse:
These developments are not isolated. The latest Stanford AI Index Report 2024 provides some staggering numbers that put this onslaught of innovation into perspective:
The rapid pace of AI advancements presents an overwhelming challenge for technologists. Each new tool or framework isn’t just a minor update—it’s potentially transformative, requiring deep understanding and immediate adaptation. For those managing teams and implementing technological advancements, this fast-moving landscape demands constant learning and vigilance.
For more details and insights on the pace of AI developments, you can dive into the Stanford AI Index Report 2024 at this link:
https://aiindex.stanford.edu/wp-content/uploads/2024/05/HAI_AI-Index-Report-2024.pdf
Now I can go to sleep 😴
Is a question I’ve been getting more and more at job interviews over the past year and when I say yes we’ve been using it for almost two years I see happy faces.
So having access to GitHub Copilot not only is a key decision making factor for software engineers looking to join your organization but also GitHub Copilot Probably Saves 50% of Time for Developers and GitHub Copilot drives better Developer Experience.
GitHub was also named a Leader in the Gartner first-ever Magic Quadrant for AI Code Assistants: https://github.blog/news-insights/company-news/github-named-a-leader-in-the-gartner-first-ever-magic-quadrant-for-ai-code-assistants
So if you’re a Software Engineering Leader there’s really no (business) reason not to get GitHub Copilot (or any other AI Coding Assistant) for your developers – it will (soon) be a requirement by new hires.
Recently GitHub released the GitHub Copilot Metrics API which provides customers the ability to view how Copilot is used and as usual someone created an Open Source tool to view the data: github-copilot-resources/copilot-metrics-viewer.
So let’s take a look at the usage of Copilot in Software Engineering in Carlsberg from end of May to end of June 2024.
I’m focusing on the following three metrics:
As I think they are useful for understanding how effective Copilot is and I would like to get closer to an actual understanding of the usefulnes of Copilot rather than the broad statement offered by both GitHub and our own developers that it saves 50% of their time.
The missing data in the charts is due to an error in the GitHub data pipeline at the time of writing and data will be made available at a later stage.
The low usage in the middle of June is due to some public holidays with lots of people taking time off.
Total Lines Suggested: Showcases the total number of lines of code suggested by GitHub Copilot. This gives an idea of the volume of code generation and assistance provided.
Total Lines Accepted: The total lines of code accepted by users (full acceptances) offering insights into how much of the suggested code is actually being utilized incorporated to the codebase.
Acceptance Rate: This metric represents the ratio of accepted lines to the total lines suggested by GitHub Copilot. This rate is an indicator of the relevance and usefulness of Copilot’s suggestions.
The overall acceptance rate is about 20% which resonates with my experience as Copilot tends to either slightly miss the objective and/or be verbose so that you have to trim/change a lot of code. So if Copilot suggests 100 lines of code you end up accepting 20.
Does this then align with the statements from developers in Software Engineering and GitHub which claim that you save 50% of time using Copilot?
Clearly reviewing and changing code is faster than writing, so even if you end up only using 20% of the suggested code, you will save time.
Unfortunately we don’t track actual time to complete tasks in Jira, so we don’t have hard data to prove the claim.
But is the claim true? Probably – however, I’m 100% convinced that GitHub Copilot drives better Developer Experience.
This is an attempt at trying to create some highlevel patterns for Artificial Intelligence (AI) solutions in order to be able to more easily choose a pattern based on type and problem area.
The goal is to have something like Software Architecture Patterns which again is based on Useful resources on Software & Systems Architecture so that we quickly can choose how to solve problems with AI. This page is also a companion to Four and not 3 Categories of AI Solutions as the patterns on this page is meant for AI-Brewers.
Would be a neglect not to mention RAG here although more of a feature than a solution pattern:
Graphs and orchestration is also commonly referred to as “Agentic” architectures.
Bots and chat-based interfaces powered by Large Language Models (LLMs) address a wide array of problem areas by automating interactions and processing natural language inputs to provide instant, contextually relevant responses.
These AI-driven solutions revolutionize customer service, information retrieval, and interactive experiences by enabling scalable, 24/7 availability without the need for human intervention in every instance.
They excel in understanding and generating human-like text, making them ideal for answering queries, offering recommendations, facilitating transactions, and supporting users in navigating complex information landscapes.
Furthermore, they significantly enhance user engagement by providing personalized interactions, thereby improving satisfaction and efficiency in areas such as e-commerce, education, healthcare, and beyond. By harnessing the capabilities of LLMs, bots and chat interfaces can decode intricate user intents, engage in meaningful dialogues, and automate tasks that traditionally required human intelligence, thus solving key challenges in accessibility, scalability, and automation in digital services.
Chaining LLMs involves linking multiple LLMs in sequence to process information or solve problems in a stepwise manner, where the output of one model becomes the input for the next. This technique utilizes the specialized capabilities of different LLMs to achieve more complex, nuanced, and accurate solutions than could be provided by any single LLM.
Through this approach, developers can create advanced workflows in which each model is tasked with a specific function it excels at, ranging from understanding context to generating content or refining answers. This method significantly enhances the effectiveness and efficiency of AI systems, allowing them to address a wider variety of tasks with greater precision and contextual relevance. Chaining LLMs thus represents a strategic approach to leveraging the complementary strengths of various models, paving the way for more intelligent, adaptable, and capable AI-driven solutions.
Chaining LLMs is particularly effective for solving problems that benefit from a multi-step approach, where each step might require a different kind of processing or expertise. Here are some examples of problems typically solved using chaining:
These examples highlight the versatility of chaining LLMs, enabling solutions that are not only more sophisticated and tailored but also capable of handling tasks that require depth, precision, and a layered understanding of context.
State machines (a directed graph) are abstract machines that can be in exactly one of a finite number of states at any given time. In the context of LLMs and LangChain, a state machine would manage the flow of interactions with the LLM, keeping track of the context and state of conversations or processes.
A framework for orchestrating LLMs is aimed at tackling the intricate challenges of integrating and managing multiple LLMs to work in harmony. Such a framework simplifies the process of combining the capabilities of diverse LLMs, enabling developers to construct more complex and efficient AI-driven solutions. It offers tools and methodologies for seamless integration, enhancing the development process, and allowing for the creation of applications that leverage the strengths of various LLMs. This not only streamlines the development of sophisticated applications but also boosts their performance and scalability, facilitating the customization of AI solutions to meet specific needs and contexts.
Orchestrating LLMs involves coordinating multiple models to work together efficiently, often in parallel or in a dynamic sequence, to tackle complex tasks. This approach is particularly useful for problems that benefit from the combined capabilities of different LLMs, each bringing its unique strength to the solution. Here are some examples of problems typically solved using orchestration:
Orchestration enables the leveraging of multiple LLMs’ strengths in a coordinated manner, offering solutions that are more versatile, scalable, and capable of addressing the multifaceted nature of real-world problems.
This is part 3 of:
Explaining how Carlsberg unifies development on GitHub and accelerates innovation with Copilot in more detail.
By integrating GitHub Copilot into our development workflow, Carlsberg has significantly enhanced the developer experience. Copilot acts as an intelligent coding assistant, offering real-time suggestions and code completions. This seamless integration enables our developers to write more efficient and error-free code. From a business perspective, this translates to accelerated development cycles and a boost in productivity, allowing us to bring innovations to market faster and maintain a competitive edge.
GitHub Copilot transcends simple code suggestions by providing developers with the ability to quickly understand existing codebases and even entire projects. This feature is invaluable for onboarding new team members and tackling complex legacy systems. By asking Copilot to explain intricate code, developers can rapidly grasp functionality without deep-diving into documentation or consulting peers. For Carlsberg, this means reduced ramp-up times for new projects and more efficient utilization of developer time, leading to cost savings and faster project deliveries.
Scaffolding, while necessary, often consumes valuable time that could be better spent on developing business-critical features. GitHub Copilot streamlines this process by generating the foundational code structures automatically. This allows our developers at Carlsberg to concentrate on crafting the unique aspects of our solutions that drive real business value. The direct result is a more agile development process, with resources optimally allocated towards innovation and creating competitive advantages.
Adopting new frameworks and technologies is a constant challenge in the fast-paced tech environment. GitHub Copilot lowers the learning curve for our developers by suggesting how to effectively use new frameworks. This guidance reduces the time spent on trial and error, enabling our team to leverage the latest technologies confidently. For Carlsberg, this capability ensures that we are always at the forefront of technology adoption, enhancing our agility and ability to respond to market changes swiftly.
Monotonous Work, like writing unit tests, though critical for ensuring code quality, can be tedious and time-consuming. GitHub Copilot addresses this by generating unit tests, which developers can then review and refine. This automation not only speeds up the development process but also ensures a high standard of code quality. At Carlsberg, leveraging Copilot for unit testing means our developers can focus more on developing features that add value to the business, while still maintaining a robust and reliable codebase.
Well-crafted documentation is crucial for maintainability and scalability but is often overlooked due to the time it requires. GitHub Copilot aids in this aspect by automatically generating meaningful comments and documentation during code commits or pull request reviews. This not only saves time but also enhances the quality of our documentation, making it easier for developers to understand and work with our code. At Carlsberg, improved documentation directly translates to reduced maintenance costs and smoother collaboration among teams, further driving operational efficiency.
At Carlsberg, our integration of GitHub Copilot into our development workflow has not just been about improving individual elements of the coding process—it’s about a holistic enhancement of the overall developer experience.
GitHub frames Developer Experience as the sum of Productivity, Impact, and Satisfaction. Here’s how Copilot aligns with these components:
By investing in tools that elevate these aspects of the developer experience, Carlsberg is not just improving our software; we are fostering a culture of efficiency, innovation, and satisfaction. This commitment not only enhances our current team’s morale and output but also positions us as a forward-thinking leader in leveraging technology to drive business success.
GitHub Copilot has revolutionized the way we approach software development at Carlsberg, significantly enhancing the overall developer experience. By automating repetitive tasks, simplifying complex codebases, and expediting the learning process for new technologies, Copilot has allowed our developers to focus on what they do best: creating innovative solutions that drive real business value. This not only leads to a more satisfied and engaged development team but also accelerates our time-to-market and improves our competitive stance. The integration of GitHub Copilot into our workflow is a testament to Carlsberg’s commitment to leveraging cutting-edge technology to foster a culture of efficiency, innovation, and continuous improvement. It’s clear that by investing in tools that enhance the developer experience, we’re not just improving our software; we’re building a stronger foundation for our business’s future success.
When driving value from generative AI (GenAI) it’s important to choose the right approach in order to be able to get a return on investment. This page attempts at explaining possible approaches and required resources.
There seems to be 3 major categories of GenAI adopters according to McKinsey and Gartner:
| McKinsey | Gartner | Description |
|---|---|---|
| Takers | Quick Wins | Focus on utilizing existing GenAI tools and models for productivity improvements with minimal customization. These initiatives typically have short time to value and are task-specific, aiming for immediate efficiency gains in routine tasks. |
| Shapers | Differentiating Use Cases | Engage in integrating GenAI tools with proprietary data or adapting them for specific applications. These initiatives aim to achieve competitive advantages, involving medium time to value with higher costs and risks than quick wins. They leverage GenAI to extend current processes and create unique value propositions. |
| Makers | Transformative Initiatives | Concentrate on developing new GenAI models or tools for specialized applications, with the potential to transform business models and markets. These are the most ambitious initiatives, characterized by high cost, complexity, and risk, and a long time to value. They aim for strategic benefits that may be difficult to quantify initially. |
The Total Cost of Ownership (TCO) and Return on Investment (ROI) for GenAI adoption across takers, shapers, and makers categories involve several considerations, including hidden costs, strategic implications, and potential benefits.
Gartner offers insights on measuring GenAI ROI, advocating for a business case approach that simulates potential cost and value realization across GenAI activities. This approach categorizes investments into quick wins, differentiating use cases, and transformational initiatives. Quick wins focus on immediate productivity improvements with short time to value, differentiating use cases aim at competitive advantage with medium time to value, and transformative initiatives have the potential to upend business models with longer time to value but higher costs and complexity. The guide emphasizes the importance of balancing financial returns with strategic benefits, which might be difficult to quantify initially.
I’m introducing an extra “Builders” category into the GenAI adoption landscape beyond merely adopting or adapting, Builders take a step further by crafting bespoke extensions and plugins for GenAI platforms. This initiative is driven by the ambition to tackle intricate, multi-step workflows that typically demand considerable human intervention. The essence of being a Builder lies in their ability to not just work with GenAI but to enhance its core capabilities, enabling solutions that seamlessly bridge various systems and processes. This approach demands a blend of creativity, technical prowess, and a deep understanding of both the technology and the problem domain.
| Category | Description | Required People Resources/Skills | Tools |
|---|---|---|---|
| Takers | Utilize existing GenAI tools for productivity improvements with minimal customization. Aimed at immediate efficiency gains in routine tasks with short time to value. | Basic understanding of AI/ML conceptsSkills in integrating and configuring APIs Ability to adapt third-party GenAI tools to existing workflows | Microsoft Copilot Microsoft Copilot Plugins Enterprise “Chat”-GPTs |
| Shapers | Integrate GenAI tools with proprietary data or adapt them for specific applications to achieve competitive advantages, involving medium time to value with higher costs and risks. | Low/No-code developers Domain experts for data interpretation Project managers with a technical background | Retrieval Augmented Generation (RAG) Microsoft Copilot Studio Microsoft Azure AI Studio |
| Builders | Develop custom solutions or extensions to GenAI platforms to solve complex, multi-step processes that usually require significant human effort. | Advanced programming skills in relevant languages Data scientists for model tuning Experience with GenAI frameworks Systems integration expertise Creative problem-solving abilities | Microsoft Copilot Extensions Microsoft PromptFlow LangChain LangGraph LlamaIndex AutoGen CrewAI (OpenAI Swarm) LLM Function Calling LLM Routing LLM Threat Modelling LLM Security |
| Makers | Develop new GenAI models or tools for specialized applications with the potential to transform business models and markets. Characterized by high cost, complexity, and risk, with a long time to value. | Expertise in deep learning and neural networks Experience in building and training large-scale AI modelsStrong research and development background Ability to work with high-performance computing resources | LLM Models LLM Frameworks LLM Fine-Tuning (LLM Creation and Training) |
The “Builders” category fills the gap between “Shapers,” who mainly adapt existing models for their unique needs, and “Makers,” who create new GenAI models from scratch. Builders leverage powerful frameworks and platforms to create bespoke solutions that automate complex workflows, potentially revolutionizing how businesses approach process automation and efficiency. This distinction underscores the evolving landscape of GenAI adoption, highlighting the increasing sophistication and customization capabilities available to organizations.
The red box on the image above indicates that solutions made in the Takers and lower Shapers category are likely to be overtaken by standard solutions from vendors and the plethora of SaaS AI offerings appearing on a daily basis. Caution should be used when choosing to invest in solutions in this area unless quick wins are important.
Clearly it’s important to have a strategic, well-planned approach to integrating GenAI with emphasis on organizational readiness, skill development, and a focus on applications that offer a competitive advantage – otherwise GenAI just becomes a technology looking for a problem like Blockchain.
I finally got access to ChatGPT Vision and I uploaded a picture of some beers I had at home:
This is the response from ChatGPT:
Nice selection of beers you got there! I see Perlenbacher, which is often associated with Lidl supermarkets, and of course, the iconic Carlsberg. Carlsberg, founded in 1847, is one of the most well-known beer brands globally and is based out of Copenhagen, Denmark. “Birra Messina” seems to be a Sicilian beer. Each of these beers brings a different taste profile to the table. Enjoy responsibly! Would you like to know more about any specific beer?
I then prompted this using TypeChat:
Please attempt to recognize the cans in the picture and fill out the following response:interface Response {
data: Array<{ name: string, brand: string, count: number}>;
}
And I got this from ChatGPT:
Sure, here’s the response based on the image:
interface Response {
data: Array<{ name: string, brand: string, count: number}>;
}
const beerData: Response = {
data: [
{ name: "Premium Pilsner", brand: "Perlenbacher", count: 2 },
{ name: "Pilsner", brand: "Carlsberg", count: 2 },
{ name: "Classic", brand: "Birra Messina", count: 1 }
]
};
This is based on the visible labels in the image.
No training – no nothing. I know the internet is full of examples of people having ChatGPT turning Figma drawings into web sites and apps – but this is totally out of the comfort zone of ChatGPT and I have previous experience trying to teach image recognition of packaged goods to models and it’s not that easy.
Solutions based on this could destroy the incumbent vendors in image recognition in the CPG market.
© 2024 Peter Birkholm-Buch
Theme by Anders Noren — Up ↑