Most of the readers likely have heard about Clayton Christensen’s seminal book called “The Innovator’s Dilemma”. I would really like to paraphrase a piece from his book here as an instance how AI-SWE goes to realize its foothold.

Clayton Christensen differentiates between two varieties of innovation: sustaining and disruptive. Sustaining innovation involves enhancing existing technologies, where recent market entrants often struggle to compete against established firms who can easily integrate such improvements into their existing products. Alternatively, disruptive innovation emerges when products turn into overly sophisticated for his or her market, resulting in an “over-served” market. This creates opportunities for brand spanking new innovations that, although they is perhaps inferior to current state-of-the-art products, offer functionality at a lower cost or in a way unattainable by the leading products.

Okay, now that you simply got a refresher of this idea, hold this thought! We are going to revisit it shortly, but first let’s introduce an emergent organization, the AI-Native firm.

Identical to we saw the arrival of cloud-native firms with the growing ubiquity and economy of cloud computing, which began to are available and alter the landscape, we’ll now see AI-native firms get formed.

As software engineering becomes an increasing number of accessible through generative AI, recent firms will give you the chance to create software without hiring a variety of expensive software developers. Indeed, there may be now a brand new career emerging, called the AI engineer [16]. It should work at a level higher than the actual development of the code, and can deal with training the models and creating the glue to bring the software created by these AI functions into the hands of the end-users.

AI-native firms wouldn’t should cope with the bureaucracy and friction that inevitably settles in with legacy software processes or older architecture, infrastructure, platforms, framework or language decisions made prior to now. They might give you the chance to construct and deploy software using one of the best, more ubiquitous and economical patterns available. What’s more, in the event that they made the incorrect technical decision, no worries, just scrap all of it and re-build it back afresh quickly!

AI-native firms wouldn’t take 9 to 18 months in stealth-mode, as has been common in Silicon Valley. All of the stages of recent technology development, from prototype to Minimum Viable Product (MVP) can be measured in weeks. It could get easier and cheaper for VCs to fund these firms, as they’d give you the chance to simply pivot when the promising product seems to be a dud. These firms will start infiltrating all industries, from taxes to medical care management, and can either begin to threaten the incumbents or would get acquired (or aqui-hired given the variety of employees can be so small). More importantly, they’d rapidly reduce the perception of time-to-market for products within the eyes of the shopper, which might all but force the prevailing successful firms to disrupt themselves.

So, back to Clayton’s concept around disruptive technologies… AI-SWE goes to be inferior to human software engineering to start with, but nothing will give you the chance to compete with its dramatic speed, and low price. Customers will do a cost-benefit assessment and decide to bypass bells and whistles and choose simpler but less expensive alternatives which have the flexibility to evolve rapidly. All of the AI-native firms should ensure is to not compromise on the basics like reliability and security and supply the core value that the purchasers are on the lookout for.

Currently in existing successful firms, IT (centralized or distributed) builds or buys much of the technology that’s used to “go to market”. This ranges from Sales technologies like CRM systems and e-commerce, to marketing technologies like digital online presence and events tech, to customer support technologies like ticketing systems and even data engineering, analytics and science like data pipelines, dashboards, or propensity to purchase models.

Note: Here, after I reference IT, I’m encompassing all of the technology organizations in the corporate. They might be centralized in a proper IT function, or distributed inside independent lines of business. This AI wave is not going to discriminate. It’ll hit all of them.

The corporate can evolve only as fast as these technologies can evolve. The pressure of reduced time to market created by AI-native firms will eventually be felt by the IT teams inside existing firms. Most IT firms are actually encouraging their engineers to securely experiment with AI, and that is sweet. Nonetheless, I consider those that may start proactively preparing for this wave will position their firms to not only break away from their competition but additionally stay resilient on this recent AI-powered economy.

I’d such as you to assume the waterfall in the midst of the forest above as the worth that might be created by AI. All of the AI-Native firms will camp closer to this waterfall, their application architectures, development processes and teams streamlined exclusively for AI-SWE

All the prevailing firms nevertheless are positioned further away. Each of them is ranging from a special place. Some are using languages and platforms which won’t give you the chance to learn from AI. Others have brittle and sophisticated architectures, perhaps built from years of inorganic growth (acquisitions) which won’t lend themselves for AI agents to simply plug in autonomously and contribute, and so forth.

When it comes to the Software Development Life Cycle (SDLC) , All of those existing firms can have their very own unique customization of the work management systems, unique processes of how low-level feature breakdown occurs, their very own source code branching strategies, their very own software gates (code smell %, unit test coverage) right down to their very own standards of how each contributing developer even structures the comment for every github pull request

Each of those above elements would want to alter within the AI-driven world. The AI agents would must be taught the expectations of what have to be within the pull request, of what documentation have to be updated within the work management systems, or easy methods to construct and interact with the testing environments, easy methods to create the change approval documentation and so forth. And this is able to be different for every IT firm. Stated simply,

The impetus to turn into AI-driven must come from inside IT, as no outside entity knows IT higher than itself.

We are going to see a variety of snake-oil salesmen: firms that may pitch this promised land to the IT executives. Even in the event that they have a compelling platform, 60–70% of the work can be adoption and integrating their platform into the prevailing processes, tools and culture.

This will feel daunting for the IT executive to tackle. Nonetheless, let’s all the time do not forget that we’ve been here before. Recall the times after we were all moving out of our data centers into the general public cloud, and even years back, after we were establishing continuous integration for our software teams, or after we were moving from our ad hoc way of doing work to standardized PaaS platforms. As executives now we have managed these major changes in our firms before.

All now we have to do is define a high level structure and plan to get there. Drawing from the various change initiatives in my profession, I actually have thought loads about this AI wave and can attempt to elucidate one possible option to do that. Note that that is just my perspective, and I do know there might be other approaches.

My goal in sharing this approach is to begin your idea machine churning and offer you a running start on this journey.

One thing I’ve learnt in my profession as a pacesetter is it is simple to speak in regards to the ideal goal state, since it has been published heavily in books and talks, and it’s also easy to admire our current state problems. The labor is to construct a bridge from the present state to the long run. Each step on this bridge needs to be planned and thought through. Step one must be close enough to the present state in order that the organization is capable of make a transition naturally. The following steps can’t be too far apart, or your people can be hesitant to step across. These steps should logically construct on top of one another.

I’ve attempted to articulate these steps when it comes to phases, starting with the present state that the majority of us find ourselves in IT:

Currently, several IT organizations are experimenting with AI and inspiring their software engineers to make use of and embrace AI-augmented code development. The standard approach taken by the engineers is to put in an extension on their Interactive Development Environment (IDE) and as they write, they use special keyboard shortcuts to kick within the code helper Agent which reviews their code, or perhaps a comment instructing what’s required in that section of the file, after which the helper auto-suggests just a few lines of code which the engineer can accept or decline [17].

There are multiple options available for these roles to get assistance below are only just a few examples:

• Github co-pilot : by far probably the most pervasive IDE extension for developers, powered by openAI’s GPT4 LLM.
• Many, many IDE extensions (e.g. Code GPT) several of which will let you use OSS LLMs as a substitute of GPT4.
• Free IDE extensions by tech firms extending their capabilities for greater adoption. Examples:

a) Proprietary programming languages (e.g Einstein for Developers extension by Salesforce for the Apex language real-time auto-completion)

b) Core technical competencies (SonarLint extension by SonarSource which focuses on catching quality issues as you code)

c) Extending Platform capabilities (CAST Highlight extension to review software structural safety, pre-acquisition due-diligence of software IP etc)

A typical product delivery team inside IT may look something like this today:

Some engineers have began fiddling with these IDEs, perhaps some lead engineers for scrum teams as well. Across the IT department it could AI-assisted development even be catching on organically.

AI-assisted development is a wonderful first step and all software engineers have to be encouraged and championed internally to take advantage of these as much as possible throughout the bounds of company security policies.

Nonetheless, to borrow a phrase from statistics, that is vital but not sufficient.

While there may be a general amorphous uplift for all developers by utilizing these capabilities, in an effort to truly introduce the “disruptive” changes we mentioned earlier, there may be some level of dedicated energy, thought and structure that should be put in place. This takes us to the primary phase of the journey:

If you happen to fail to plan, then you definitely plan to fail.

Before any concrete actions like team formations etc are done, an assessment exercise must be conducted to know and define the general scope and goals for bringing AI-driven engineering to the IT organization. A solid plan would drive executive confidence, and right-size any investment asks for the long run phases.

To create a practical and actionable plan, it can be crucial to take a look at the processes, tools and technologies from the lens of the AI software agent we’ll introduce. Key areas to document through this exercise can be:

1. Evaluate key software development processes

Stripped right down to the core, what are the present processes every human software engineer must partake in today to give you the chance to contribute code to deliver features? Examples here include planning, sizing, developing, adherence to the peculiar branching strategy, running specific test suites, specific coding, unit-test and documentation requirements etc.

In other words what are key low-level software development processes that the AI software development agents would also have to integrate into to be practically useful.

That is a vital exercise because, in all IT organizations, there are a variety of higher-order processes which are useful for coordination, communication purposes but do circuitously contribute to shipping features. Examples include weekly leadership reports, status updates to senior leadership, coordination meetings across multiple domains. Typically any work that’s driven by Technical Program Managers will be included on this bucket. Integrating AI-driven engineering will not be going to influence those directly (Yes, AI will influence these higher-order processes not directly in a profound way, but that’s beyond the scope of this writeup).

This evaluation and documentation will help scope out the additional services which will must be written for the AI-SWE teams to plug them into the prevailing software delivery workflow.

2. Evaluate current IT infrastructure and environments

When using AI-agent engineering, there are non-trivial issues related to dependency management inside the applying (i.e. the version of run-time, the libraries getting used etc) and API-based dependencies on neighboring applications. My initial experimentation with autonomous AI surfaced various such challenges with deploying software in environments (I’ll document them later in a more technical writeup). There’s the matter of authentication management of the applying beyond a “local” setup. There could also be some manual steps a developer must take to be fully provisioned in deploying to that environment. Such a setup would must be replicated for every AI agent, and clearly automated as much as possible to simply give you the chance so as to add AI agent capability with minimal manual effort. In brief, detailed “low-level” documentation of what it takes a software engineer ranging from scratch in developing some software locally, then deploying to a shared QA environment , after which promoting that code to higher environments all the way in which into production is important to discover the roadblocks an AI-agent will face.

In lots of cases we’ll must be able to not have a 100% path of automated continuous deployment (CD) to production to start with if CD will not be in place for human-driven software engineering (H-SWE) today. For example, if there are humans involved within the change and release management process to ship features to production today, that may still be there when semi-autonomous AI-SWE teams are creating software features by themselves. That doesn’t stop the AI-SWE teams from creating and surfacing all required documentation and testing output to the change board and release managers. (Over time, change and release processes can definitely be powered by AI (not coding, but reasoning) agents as well but that’s beyond the scope of this document)

Just like the software process evaluation, environment evaluation can even show what other capabilities must be streamlined for AI-SWE to give you the chance to remain as autonomous as code gets promoted through higher environments all of the option to production.

3. Discover all applications where AI will be integrated (complexity and impact)

There might be tons of of candidate applications where AI-SWE might be inserted. Nonetheless, a careful evaluation of the complexity of processes surrounding each application’s SDLC, the maturity of the AI-coder LLMs for every application’s language, and the extent of impact of AI-SWE on that application might be required. Identical to in story pointing, Fibonacci scoring will be used against the price and profit dimensions and an overall ROI metric will be created which will help executives resolve the order of the AI-SWE agent deployment (more in Phase 2 below).

Vital note: On the bottom of the priority list of applications must be all those applications where there may be already significant context loss [18]. The issue statement have to be kept true to constructing AI-SWE teams and never to dilute to incorporate legacy applications that mysteriously work and nobody really understands how.

4. Develop a timeline and roadmap for the transition

After all, no planning phase is complete with no published roadmap, and one can be required here as well. As an executive I really like defining timelines, since it forces my team and I to think when it comes to real dates, and deadlines, and enables us to take into consideration all elements and stages of the project holistically. After all, “no plan survives first contact with the enemy” as Sun Tzu stated. These timelines do, and can change, especially since this kind of work has never been done before. We truly don’t know what surprises are in store for us on this journey, and there is no such thing as a user manual on the market for this yet. (In reality, to one of the best of my knowledge, this text will be the first real attempt at a framework from an experienced product leader who’s been within the trenches).

All those aside, the plan will still give a transparent baseline and discover all of the assumptions. As surprises emerge, the plan will be reviewed and adjusted. Nonetheless, those deadlines will still be on the market, publicly communicated and so they all the time drive a healthy pressure on the team to maintain sharp focus and deliver. For all recent projects or features, I all the time say:

“dates can’t change but scope can”

5. AI technology selection

A while must be put aside to do a POC-driven technology selection. It could be good to have some high level clarity on which horse to back (aka which base models to wonderful tune). These models should give you the chance to serve the utmost variety of the applications in the primary quartile of the priority list coming out of step 3 above. Nonetheless, we must give you the chance to remain flexible here. This space is rapidly evolving and innovation may come out later this yr that requires us to alter our decisions. What is significant here is to deal with the non-regrettables, by asking questions comparable to:

• Does the AI technology (the multi-agent frameworks etc) allow us to translate architecture, handling coding, debugging, QA and operations tasks?
• Do these frameworks allow us to simply swap LLMs?
• Are there common benchmarks we will define which enable us to judge the framework’s performance?

This selection have to be based on multiple proof of concepts which show how AI-SWE can integrate with environments and software tooling. There are many cute AI-SWE applications on the market like snack-game, count to 100, and printing hello world in multiple languages. The POCs here should deal with easy but real use cases which are applicable inside IT. It’s these POCs which can give us some level of confidence and show a practical delta from here to the specified goal state which we will put an affordable level of effort estimation against.

To work on this evaluation and planning assessment, I like to recommend a small team comprising of:

• a reliable software engineering leader who knows how the sausage factory works when it comes to software creation within the firm, and is knowledgeable and enthusiastic about the intersection of generative AI. They might be a core change agent for this transformation and can drive this variation for the organization.
• an architect experienced with taking their architectures into production working closely with engineering teams. This must be someone who understands the event process inside IT, in addition to has a deep grounding on the core architectural principles defined by the IT Enterprise Architecture group to make sure they’re reflected in the answer.
• A lead engineer inside one among the initial goal domains who has in-depth and hands-on experience with the total software lifecycle from requirements, to sizing, to environments, to code reviews. I believe the lead will intuitively know where the low hanging fruits are where AI agents have one of the best likelihood of productive contribution to the team.

While the bottom-up organic usage of AI-assisted human software development must proceed, a dedicated team would sharpen deal with bringing AI-SWE into reality.

Specifically , this group would deal with 4 distinct areas. I broke these apart based on the unique skill set and the unique cognitive complexity that applies to every of them.

No organizational design on the earth is ideal. They have a tendency to optimize for some gain while sacrificing one other. The organizational design that I recommend optimizes parallelism and scale with as much autonomy as will be afforded to the person teams, with the important thing operative deal with “moving fast”. These teams are:

Primary Role: Develop, customize, and maintain AI models, especially LLMs, for various software development tasks. This development will start from pre-trained base models and wonderful tune them on the applying teams’ internal software and documentation.

Responsibilities:

• AI Model Customization: Give attention to fine-tuning pre-trained LLMs to know the domain team’s specific codebase, software architecture documentation, and domain knowledge as much as possible. This might be counting on top quality curated data by the information management team (see #2 below). In several cases the applying code that the Model can be fine-tuned for can be written in languages already understood by the bottom model (comparable to Microsoft’s Phi-2). Nonetheless, fresh research could also be required in languages which have not been trained to those models comparable to Terraform which is primarily utilized by infrastructure automation teams. AI Model Quality Assurance (QA): Test AI models to make sure they meet functional requirements and performance benchmarks. These AI models must meet a sturdy evaluation criteria and objective performance metric. A great outside-in test of the output can be the pull requests the AI-SWE agents would send to the lead engineers. Nonetheless, based on the leads’ feedback the performance metrics would must be codified and baked into the model creation pipeline. This team must also explore newly developed methods in academia like BotChat [9] to measure the standard produced by multi-agent AI-SWE teams.
• Research and Development: Stay abreast of the most recent AI advancements and incorporate them into your systems.

Skills Required: knowledge of the software teams specific tech stack, software development and systems integration.

I do not consider a practitioner level of knowledge science knowledge is de facto required for this team since a variety of OSS capabilities like AutoGPT [19] have already been created to help with fine-tuning of base models. This team will not be going to be doing a variety of data-science or ML work like pre-training models on TPU chips in GCP, but it can be a real engineering team, taking the lego blocks and constructing out an AI-software engineering capability, constructing on top of the OSS base models. Essentially, the abilities we’re going to need listed here are solid software engineers. This team nevertheless might want to have a general idea of how pre-trained coding models [20] were created, and might want to stay on top of the most recent developments in the sector of multi-agent software development which is why the R&D responsibility has been noted above.

In a while in a more advanced stage, this team may require to pre-train its own models, which can require data scientists and investment in cloud compute spend, but investing so heavily on this space before proving out real value doesn’t make business sense when a variety of uplift will be gained by constructing on top of pre-trained models.

Primary Role: Manage the information required for training and refining AI models. This team would want to know what type, quality and format of the information is required for the training of the models by Team 1, after which give you the chance to gather and annotate this data. In early days, this and the AI development team could also be one and the identical, but eventually for scaling purposes, this is able to have to stand as a separate team. In reality, to start with creating sample data sets to wonderful tune the models would likely be done by the AI Development team (1) anyway, and would only hand this work off to this team when scaling out across multiple applications after which multiple domains. I discovered a prototype [21] put out by Autogen very applicable where, by pointing an AI agent towards decent code documentation (aka Retrieval Augmented Generation, or RAG) they demonstrated how the AI agent was capable of learn syntax of some recent methods within the library and supply code based on that. We may or may not select to make use of the RAG method here but the instance clearly shows this will be done.

Responsibilities:

• Data Collection and Annotation: Gather and label data, comparable to code repositories, project documentation, and user stories, for AI training. It should be vital to see what the formatting, and annotation needs are other fine-tuned coder models. Within the fine-tuning phase of natural language models, a variety of query and answer (key:value) pair varieties of curated data must be created [2]. This team would want to translate what which means within the domain of AI-SWE.
• Data Quality Assurance: Make sure the accuracy, consistency, and relevance of knowledge used for training AI models. It is a very critical area for the success and quality of the models. Recall from earlier that the effectiveness of the newer small language models is directly coming from the undeniable fact that top quality textbook level data is getting used to coach these models. What this implies is that a variety of energy on this team will go in ensuring that this data is relevant, in-context of the applications where the AI agents might be inserted. This will require reviewing and red-lining the prevailing application documentation (and maybe code) and collaboration with the prevailing teams to enhance their documentation prior to training. While code and general internal documentation for the applications will must be taught to those AI agents, it will be good to research other generic engineering patterns taught to CS undergraduates in class. That will require formatting, chunking and preparing actual top quality academic text for the wonderful tuning of the LLMs as well. Finally, some objective metric on easy methods to measure the standard of the information would must be designed as well.
• Data Privacy and Security: Handle data in compliance with privacy laws and security standards. This team can be tasked to make sure that all the information that’s being curated is abiding by the firm’s privacy and security standards.

Skills Required: Data management, data annotation, understanding of knowledge privacy laws.

Note: A useful side-benefit of the output of this team might be improving general quality of the code and documentation for goal applications. This might likely help the prevailing team as well with things like recent engineer onboarding, in addition to helping existing engineers gain deeper understanding of the applications they could haven’t built, but inherited.

Primary Role: Integrate AI solutions into existing processes and systems and ensure their functionality and reliability. This team provides the “last mile” connectivity for the AI-SWE agents and teams. Without the combination tools and libraries that might be needed, the software and documentation that AI will produce wouldn’t give you the chance to get checked-in, reviewed, deployed to testing environments.

Responsibilities:

• AI Tooling: Develop tools to integrate AI models seamlessly into the prevailing software development pipeline. In point of fact these would likely be APIs and helper libraries that will be used for authenticating with the inner Identity management system, interacting with git , interfacing with environments and the prevailing CI/CD pipeline, and maybe interacting with the work management system (e.g. Jira) for the team.
  You’ll be able to see the pattern here. These are all activities that human coders typically do outside of just writing code. Utility helper software might be invoked by the AI agents to take similar actions. Depending on the extent of maturity and automation within the firm, this work may very well take more effort than the actual AI Development.
• AI Performance Monitoring: Repeatedly monitor AI systems for any issues or deviations from expected performance. Note: This work could also land into an existing MLOps team if one exists.

Skills Required: Experience in software integration, QA testing, performance monitoring, and troubleshooting.

Primary Role: Make sure the ethical development and deployment of AI systems. That is preferably a team that will not be directly reporting throughout the shared services organization. Individuals with this experience typically are already present within the Chief Data Office of firms as a part of the governance teams. They might be plugged in as consulting and informed parties to make sure their inputs are accounted for throughout the creation of those AI-SWE systems.

Responsibilities:

• Ethical Guidelines Development: Provide guidelines for ethical AI use, specializing in fairness, transparency, and accountability for the shared services team to include of their AI implementation.
• Compliance Framework: Provide a minimally viable set of tangible and measurable compliance rules, regulations and ethical standards that the shared services team can incorporate within the implementation checklist. That is an evolving field and due to this fact there must be an ongoing interface established with the information governance team to capture any recent vital regulations or critical AI standards that get created within the industry
  While not the responsibility for this team, it could help direct the integrations team (3) in creating extension packs for auditability comparable to SOX etc, that are capable of robotically generate reports based on the documentation created by the AI-SWE teams that are acceptable to internal and maybe external auditors.

Skills Required: Knowledge of ethical AI principles, legal and regulatory compliance, stakeholder management. It’s important to decide on the partners with an “enabling” attitude, vs. an “obstructing” attitude. The previous will work with the team to seek out workable solutions, while the latter just likes saying “No, you’ll be able to’t do this!” but aren’t motivated to seek out a path forward.

These shared services teams play a critical role in successfully transitioning to an AI-driven software development process. Their effectiveness lies of their ability to collaborate, innovate, and adapt to the evolving landscape of AI technologies.

Once the critical elements of this group have been created and are functional, the exit criteria from Phase 1 into Phase 2 is to create a functional and productive AI-SWE agent built on a customized fine-tuned LLM trained on top quality data for the primary goal application/scrum team. At this point the AI-SWE is functional in a controlled environment.

Nonetheless, As DC Palter [22] put it so eloquently:

90% finished means the product is halfway to release

“there’s an enormous gap between a prototype that works under controlled circumstances and a industrial product that may’t break under any usage. When the product is 90% finished, it’s only about halfway done.”

Phase 2 starts whenever you take this 90% finished prototype (AI-SWE agent) into the true environment.

On this phase we fastidiously bring the primary AI-SWE Agent (note the usage of Agent vs agent, more on this later) right into a real life scrum team and regularly scale up the agent’s responsibilities because it proves its effectiveness. We’ve to watch out during this introduction because we don’t want this to disrupt the goal scrum team’s velocity which has committed story points for the epics under construction for that release.

I propose we bring the AI-SWE Agent into the team led by the lead engineer who had been on this journey from Phase 0. The lead would direct the AI agent with initial tasks might be just to begin providing pull requests for easy technical debt issues (comparable to open defect fixes which we all know AI Agents are getting good at [23] or removing easy technical debt issues identified by code-smells in sonarQube runs [24] etc) and even missing documentation for code and value.

On this phase, all of the integrations and helper libraries are fine-tuned, with the goal of the AI Agent’s contribution to the team becoming frictionless, consistent, and equal (or higher) than the human developers within the team.

Behind the scenes the AI Agent we’re introducing is de facto made up of a mix of multiple smaller agents interacting with one another. This is significant because multi-agent output is usually far superior than a single-turn interaction with a human being, as documented earlier. This cute DALL-E image captures the essence of the team:

A more apt, but far less cooler term to make use of here is an Agent Teamlet. Because of this I adopted the term AI Agent with a capital “A” to distinguish it with the actual atomic agents. Nonetheless I take advantage of the Teamlet and Agent interchangeably throughout the document

Zooming in into the AI Agent (aka Teamlet) we will see below quite a few different agents, each backed by the identical or different LLMs (depending on their specialization)

The agents shown throughout the teamlet could have the next characteristics and responsibilities

The above is just an example of the interactions and the agents that may be required to meet the coding a part of the job of a human engineer. It will not be complete. For example, it’s missing how the environments are going to be handled etc. That will require a totally separate teamlet just to organize prior to this one with the ability to step through its instructions. The goal here was just to elucidate the final concept of the Agent with a capital “A”.

This phase allows the Result in fine-tune the prompts and roles for the assorted multiple agents.

Below is an example conversation amongst these agents taken from one among the Autogen reference examples [21].

Phase 2a will be considered successful when the AI-SWE Agent (teamlet) is capable of tackle real coding user stories from the board, work them and most significantly, their pull requests are at the extent of quality that they’re consistently being approved by the lead engineer and merged into the mainline.

We are actually able to bring these Agents into other domains.

This Phase is more of a rinse and repeat of Phase 2a. The differences here can be introducing other lead engineers and designers to this model of working, and the training of custom LLMs suitable for that domain. For instance, the custom LLM coder created for the Customer Support tech team will must be fine-tuned on a special set of software repositories, application documentation , and maybe may even require a modified set of helper functions (if their software workflows are different) than the custom LLM created for, say, the Finance tech team.

By this time we must always start introducing multiple AI Agent right into a scrum team comparable to Team 1 within the Platform Infrastructure domain within the figure above. This might not only help with parallelism (i.e. Agents taking multiple user stories from the board concurrently in the identical sprint) but additionally provide a segregation of very different duties, comparable to one Agent fully taking over environment creation, and test harness setup, while the opposite one taking over writing and deploying code to that environment.

Funny aside: My brother, a serious fanboy of Star Trek drew comparisons of this idea with the Bynar species [26] which worked in pairs and whose thoughts were interconnected to one another enabling them to speak rapidly and efficiently. Gene Roddenberry can be so elated how AI Agents are materializing his vision!

Phase 2b will be considered complete when each applicable domain has at the very least one AI Agent online and successfully takes user stories from the board, working on them, and updating the work management systems. Most significantly, their pull requests are at the extent of quality that they’re consistently being approved by the lead engineer and merged into the mainline.

We are actually able to enter phase 3.

That is the phase where a team is fully composed of AI-SWE Agents.

A fundamental shift from Phase 2 to Phase 3 is that on this phase, the team is working full and complete Epics, with each of its Agents picking up user stories and coordinating across one another to finish the work.

Individually, this paradigm shift with AI fully developing software might also affect the way in which we chunk work in software (Projects -> Epics -> User Stories -> Tasks). Have in mind that the present option to break down work was done to accommodate human-driven software development. The work breakdown modality for AI-driven development is beyond the scope of this text and can naturally evolve over the upcoming years. For the sake of familiarity (and ease!) we’ll proceed to make use of traditional agile terms like Epics and User Stories here.

On this phase, a better order plane of communication must be established between the assorted Agents (Teamlets), which can mean interaction across one atomic agent inside one composite Agent (TeamLet) is connected and communicating across with one other agent inside one other composite Agent. It could also mean these agents are collaborating via the work management system and issue boards comparable to Github issues. These specifics aren’t relevant. More vital to notice is that every Agent would have complete autonomy for certain tasks. These roles might be:

• Software development Agents (detailed earlier in Phase 2a)
• Product Management Agents which could writer and flesh out Product Requirements documents and chart out roadmaps under the supervision of human product managers
• Technical Author Agents chargeable for customer-facing documentation, internal software documentation, slide creation for All-hands wins, updates to the IT portfolio
• Research Assistants supporting the architects and Product Managers helping with external research around business capabilities, SWOT evaluation and so forth.
• Operations Support Agents, which activate directly through system alerts, or manually when triggered and pointed towards issues to troubleshoot

Essentially, these teams turn into a self-contained unit for an entire product accountable for end to finish responsibility. Vital to notice is that design for AI Agent roles is different than those for humans [27] and we’ll leave some flexibility for what probably the most optimum roles would must be after we get empirical feedback through initial attempts.

Recent studies [28] have shown that coding, communication and planning are areas AI-SWE is beginning to get well at, within the realm of writing pure software. Nonetheless, for AI to perform a few of the activities expected on this phase autonomously requires Generative AI to do a more advanced level of considering called system 2, or slow considering. [29] As of this writing we aren’t there yet, which implies much of the rational, complex considering required during operations troubleshooting or product management activities will still require hand holding and human intervention.

Nonetheless, breakthroughs are expected on this space during 2024, potentially by 2H [2] . Still much of the total epic development should still be achievable with current state capabilities and LLM customization.

An area to notice is that AI does an incredible job with functional requirements but it surely will not be as capable (yet) with non-functional requirements (NFRs) which include performance, trust and scale [30]. There’s human intervention needed to make sure the benchmarks of the organization for these NFRs are indeed developed to spec. This might mean as little as having human end to finish testing capabilities (with the tests after all also developed and run through AI Agents, but with human signoff) or as much as human-driven software enhancements so as to add the expected NFR capabilities.

This phase expects a level of sophistication which is a bit further out to be prescriptive in regards to the exact exit criteria, but generally speaking an excellent barometer of success is to expect full Epics being developed from requirements to functional solutions running in QA. There should still be a have to require human verification on whether Non-functional requirements (NFR) are developed to spec.

Leveraging the teachings learnt from AI teams in a single domain, this phase introduces the autonomous AI teams into other domains. This also starts to boost a few of the senior engineers to the extent of lead in an effort to scale out the teams.

A facet that will not be depicted here but is value mentioning is the necessity for an end to finish QA team (which must be a hybrid team of AI + human engineers). End to finish QA is significant since these AI Teams could also be developing against separate epics’ user acceptance criteria, but the top to finish experience would still must be verified and integration-tested when all of those come together prior to a release. This also extends across performance testing needs. Eventually, when the muscle is developed well enough, the QA function may also be trained into an LLM, but I foresee it happening later than the Software development function, simply due to complexity of interdependencies. This still doesn’t preclude the QA function to heavily utilize AI Agents to judge, write and deploy automated tests and automatic reporting in earlier Phases.

Exit criteria for this phase is surprisingly straightforward: A critical mass (80–100% as decided by executive leadership) of functional AI Teams have been deployed and are contributing to every domain and a clear difference in lead time to alter is beginning to be observed.

Scaling out AI teams

By this time the domains are getting comfortable with the standard of the work of the AI teams, and an increasing number of hands-on software engineers start getting upleveled and elevated to turn into leads, or AI Engineers [16], of their very own AI Teams. While I’m showing a mapping of just one AI Team per human engineer, there is no such thing as a reason why multiple teams can’t be operated by the identical human engineer. The bottleneck of velocity will naturally move to those human AI engineers, but by this time the firm must be having fun with dramatically reduced lead time to changes already. The abilities that these AI engineers would want to deal with would still have a modicum of software since they’re still on point to review and approve the Pull Requests being sent by the assorted AI Agents, and review the documentation being created by them and so forth. Nonetheless all human engineers would want to deal with inimitably human skills referenced in section “Good engineers don’t have anything to fear”

Note that there’ll all the time be a necessity for a handful of human driven software teams. These can be higher order functions nevertheless, and their job can be a security net for the domain. As mentioned earlier, an excellent example of that is to make sure NFRs are being delivered to specification, but there might be other AI shortcomings that may surface. It’s important that these are documented and shared with the domain’s AI Development teams so future versions of the domains’ custom LLMs are superior.

Decentralizing AI Shared Services:

When technology and products begin to mature, it’s my experience that the shared services teams began becoming a bottleneck to velocity. Due to this fact in regular state all organizations must plan to decentralize any AI Shared services team into each domain. Which means that each domain builds native muscle to customize their very own AI LLM models, to curate the training data, and to keep up custom integrations which are peculiar to them.

The shared services team starts to change right into a Center of Excellence (CoE) organization which cares for the services that may all the time be horizontally required and are higher if left standardized. This includes a few of the common integrations. For instance, all domains will proceed to want their AI Agents to give you the chance to integrate with the IT CI/CD pipeline team, or to interact with Github, sonarQube and work management systems like Jira, Asana, BaseCamp etc.

Since IT systems can all the time get audited, this CoE can create and maintain extension packs which teams can plug in and create audit reports (ie. an iteration of the ethics and compliance team from the prior shared services group).

There was research that shows developers getting assistance from AI for writing code tend to jot down less secure code, and what’s more troubling, they’re over optimistic in regards to the safety of this code [31]. This problem may worsen with fully AI powered teams. Because of this it can be crucial that a central check on code quality and library selection is maintained. This will require the CoE to create its own sentinel software which is capable of keep track of secure code and infrastructure across the domains, in addition to to robotically track and add items within the backlog of those teams and report on the general security posture centrally. If an organization already has a robust, well staffed central security organization which has software engineers of its own and are deeply plugged into IT, there is no such thing as a reason why this function can’t be run out of there as well.

Application Architecture could also turn into a bottleneck for velocity if it was not composable, and event driven to begin with. That is something to pay attention to and if the Phase 0 assessment was accomplished fairly, must have also been caught throughout the scoring and prioritization step for every application.

Here’s a bit “behind the scenes” story. After I was in the midst of writing this text, I asked chatGPT to take an opposing stance and explain what the issues can be with having less reliance on human-driven software engineering. The feedback, which I abbreviated below, actually helped me refine my proposed phases comparable to raising the extent of human engineers to a better but much more crucial level, and having an autonomous team being run by each existing engineer within the team.

What follows is chatGPT’s counter arguments, with my responses each of its arguments inline:

Listed here are some counterarguments to contemplate:

1. Complex Problem-Solving and Creativity:

– Human developers excel in creative problem-solving and coping with novel situations. AI, while excellent at handling routine tasks, may struggle with unique or complex problems that require out-of-the-box considering.

– Creativity in design and innovation is a definite human trait that AI cannot fully replicate, especially in areas requiring emotional intelligence and understanding of human experiences.

My response: Valid point. This made me take into consideration how I can raise the extent of the prevailing engineers and place them within the organization where more of their creativity and out-of-the-box considering can happen. That typically is required whenever you’re closer to the shopper, which is what happens where you might be playing a lead engineer role (See Regular State: Phase n)

2. Quality of Code and Maintenance

– AI-generated code may lack the nuanced understanding that experienced human developers bring, potentially resulting in issues in maintainability, scalability, and performance.

– Human oversight is crucial for ensuring code quality, particularly in complex systems where understanding the broader context is significant.

My response: I used to be also considering of the identical thing as the assorted papers I actually have quoted express AI-SWE’s weakness in non-functional requirements. Because of this even within the steady-state, I opted to maintain just a few actual human-based engineering teams which ensure trust, operational resilience and performance elements of the software have been taken under consideration.

3. Ethical and Social Implications:

– Replacing human staff with AI raises significant ethical concerns, including the impact on employment and the broader social implications of reducing the human workforce.

– There’s a risk of losing invaluable human skills and knowledge collected over years, which aren’t easily transferable to AI.

My response: I accounted for this. While I foresee IT organizations will stop growing at the identical rate, the undeniable fact that existing engineers might be repurposed to do higher-order work will ensure meaningful employment while giving them higher visibility to senior leaders and to businesses. After all this does mean that the prevailing engineers would should grow their skills that traditionally belonged to Product Management, and the product managers would want to get comfortable steering AI teams and frameworks.

4. Lack of Adaptability and Intuition:

– AI systems, even advanced ones, may not adapt well to rapidly changing environments or requirements, a standard scenario in software development.

– Humans can intuitively navigate ambiguous or poorly defined problems, a capability that AI currently lacks.

My response: Correct, and that’s the reason each AI-SWE team does have a human engineer steering it. AI-SWE will still need explicit and low level instructions and tweaking because the interface on the opposite side of the applying continues to be a human which only one other human can empathize with.

5. Security, Dependability, Regulatory and Compliance Issues:

– AI systems will be vulnerable to specific varieties of failures, biases, and security issues which are different from those encountered with human teams.

– Dependence on AI for critical development tasks might pose a risk in situations where AI behavior is unpredictable or erroneous.

– AI systems will not be fully equipped to handle the legal and compliance elements of software development, which regularly require human judgment and understanding of the law.

– There is perhaps regulatory challenges or restrictions in certain industries or regions regarding the usage of AI in critical development roles.

My response: Good point. This made me add a responsibility under the Center of Engineering team for security and compliance, even in steady-state. After reading the purpose about regulatory challenges in certain industries (or geographic locations), I did add a caveat within the article that the AI-SWE model will not be applicable in such industries, and the usage of AI for development could have to be in a far more controlled fashion.

6. Communication and Collaboration:

– Effective software development often requires nuanced communication and collaboration with stakeholders, something that human teams are inherently higher at.

– Understanding client needs, empathizing with end-user problems, and negotiating requirements are areas where human interaction is crucial.

My response: Agreed, and that’s the reason the AI engineers and Product Managers each might be in place to handle the communication elements with human stakeholders.

7. Costs of Transition and Maintenance:

– The initial investment in developing, training, and integrating AI systems for software development will be substantial.

– Continuous maintenance, updates, and training of AI systems to maintain them effective also can incur significant costs.

My response: I can see how GPT4 could have been considering that I can be doing the pre-training part in house, which is a incorrect assumption. I’m banking on using available pre-trained base models to wonderful tune to the inner IT software code repositories.

9. Limited Scope of Understanding:

– AI, even with advanced capabilities, may not fully grasp the business context or the subtleties of the market and user needs in the way in which humans do.

– There’s a risk of AI solutions being too narrowly focused, missing the larger picture that human developers often consider.

My response: Agreed, and that’s the reason the AI engineers are still steering the AI-SWE teams in the ultimate state.

10. Risk of Overreliance:

– Over-reliance on AI can result in a decline in human expertise throughout the organization, making it vulnerable to AI failures or limitations.

My response: That is correct and I agree with this sentiment. Nonetheless, this can be a more general risk that your entire software industry goes to face. I’m undecided how we will avoid this at this point. In brief, I don’t think anyone has a solution to this, but I suppose we will discover together over the subsequent few years 🙂

First, let’s address the subject of fear head-on. It is evident from social media studies done on software engineers’ sentiment about AI [30] that fear is by far the dominant emotion amongst developers. It’s important to acknowledge this.

At the identical time, let’s take a step back and have a look at the entire set of skills that good software engineers have [32]

In a summary of many studies done around great software engineers’ traits, you’ll be able to see that the coding part is only a subset of what makes these engineer’s great. If you happen to have a look at the chart above, I’ve marked with a gray arrow all of the areas which an AI agent, despite how advanced it gets, won’t ever give you the chance to mimic. In my view these are the inimitable skills that every one software engineers should intentionally hone and ideal as they might be what distinguishes them and keeps them invaluable in the brand new AI-SWE paradigm-shift. Indeed, these are exactly the abilities that may be accentuated and in demand within the Regular state phase of the AI-powered IT firm that I depicted above.

I’ve been within the technology industry for over 20 years, and have seen my share of tech bubbles and busts, disruptions in infrastructure (on-premise to cloud) and processes (waterfall to agile). I can say that what we’re witnessing now needs to be the largest tsunami ever to hit us within the software world. But don’t just take my word for it, try what Bill Gates has to say [34]. The barriers of entry for moving into constructing software-based technology are going to be worn out, the lead times from idea to market will reduce, firms previously not even working in the identical stratosphere will start competing, and recent nimble firms will emerge which can threaten great legacies. A lot of us work in these existing large firms, and a few of us are in senior positions to influence the course of the ship. Perhaps a few of these successful firms is not going to even acknowledge that that is even an existential crisis approaching until it’s too late. I imagine several will sense the change approaching but would favor to remain in the nice and cozy comfort of the prevailing established order.

And that’s precisely where the chance lies. The few firms that may actively do something about it, will position themselves to interrupt away from the pack, and might be in a dominating position when the change will begin to speed up. These firms might be the good ones. They’ll outlast all others and when the dust settles and the smoke clears they might be standing stronger than ever.

This content incorporates my opinion and will not be intended to be understood because the official position of my employer.

1. Hailin Chen Caiming Xiong et al. ChatGPT’s One-year Anniversary: Are Open-Source LLMs catching up?
2. Andrej Karpathy [1hr Talk] Intro to Large Language Models (timestamped at “stages of making an LLM”)
3. Microsoft — Phi2: The surprising power of small language models (highly performant coder base model)
4. Eric Hartford Meet Samantha — A baby step towards sentient AI
5. Meditron](https://ollama.ai/library/meditron) — Open-source medical large language model adapted from Llama 2 to the medical domain.
6. Erik Nijkamp, Donald Rose — Conversational AI Programming with CodeGen: Let AI Write Code For You
7. Eldan et al — TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
8. Gunasekar et al — Textbook is all you wish
9. Duan et al — BotChat: Evaluating LLMs’ Capabilities of Having Multi-Turn Dialogues
10. ChatDev: easy-to-use LLM-based framework for collective intelligence.
11. MetaGPT: Given one line Requirement, return PRD, Design, Tasks, Repo
12. Autogen (by Microsoft): Enable Next-Gen Large Language Model Applications.
13. Crew AI: Framework for orchestrating role-playing, autonomous AI agents.
14. Oliver Morris: Can AI Team Up with Itself for Our Profit?
15. Waseem et al: Autonomous Agents in Software Development: A Vision Paper
16. Swyx: The Rise of the AI engineer
17. Douvantziz: Coding with AI auto-completion — How does Copilot weigh up?
18. Chelsea Troy — specific reference to Context Loss in her article: Reducing Technical Debt
19. Autogpt: Positive-tune your agent to perfection
20. Feng et a, Codebert: A pre-trained model for programming and natural languages
21. Autogen: RAG example on teaching an agent a brand new software library to make use of for its coding project
22. DC Palter: 7 Things Experienced Founders Know that 1st Time Founders Miss
23. Sobania et al. An Evaluation of the Automatic Bug Fixing Performance of ChatGPT
24. Tian et al Is ChatGPT the Ultimate Programming Assistant — How far is it?
25. Autogen : Multi-agent examples
26. Fandom : Star trek Bynars
27. Oliver Morris , How skilled can AI teams get
28. H Hörnemalm, A. (2023). ChatGPT as a Software Development Tool : The Way forward for Development
29. Kahneman, Considering fast and slow
30. Feng et al , Investigating Code Generation Performance of Chat- GPT with Crowdsourcing Social Data
31. Perry et al, Do Users Write More Insecure Code with AI Assistants?
32. Li et al, What distinguishes great software engineers? Empirical Software Engineering
33. Ben Evans: AI and The whole lot Else, Slush Conference, Helsinki Dec ‘23
34. Bill Gates: The Age of AI has begun

• AI Agent : A multi-agent superset of AI agents working together
• AI agent : An agent backed by a generic or specialized large-language model
• AI Teamlet : same as AI Agent
• AI-SWE : AI-driven Software Engineering: Software written, debugged, built and deployed by AI Agents & AI Teams.
• CI/CD : Continuous Integration & Continuous Delivery
• CoE: Center of Excellence
• CRM: Customer Relationship Management system
• H-SWE: Human-driven Software Engineering: Software , debugged, built and deployed by human beings
• MVP: Minimum Viable Product
• OSS: Open Source Software
• RAG: Retrieval Augmented Generation
• SDLC: Software Development Life Cycle