NextGen IDPs – the future of software development is here

A tale of two development approaches

Two companies have simultaneously discovered a material for constructing exceptionally strong and lightweight space rocket hulls. The first company creates a plan to design and manufacture every component of the space rocket in-house, from navigation equipment, life support systems, to the fuel tanks and communication devices. Lacking expertise in these areas, they need to hire specialists, establish production facilities, and start from scratch in developing these complex components. This approach is obviously time-consuming and complex, delaying the go-to-market timeline. The second company focuses on their core strength, rocket hull fabrication, and for the other critical components they collaborate with established industry leaders, integrating tried-and-tested technologies into their rocket design. As a result, their space rocket is ready for launch considerably sooner than their competitor’s.

Why share this small story when it’s evident to everyone what the right approach is from the start? No one would proceed like the first company; they would always follow the second company’s approach. However, this is often not the case in the software industry. Companies repeatedly build software, tooling, and infrastructure that has already been developed many many times before, and that is auxiliary to their core business. A company constructing an e-commerce platform for groceries will quickly end up with a custom-built user management service, a task list service, an audit log, a build pipeline, monitoring, a testing framework, etc., which have little to do with the e-commerce business.

Why does this occur? There can be many reasons, but it typically starts with the paradox that companies need to release something quickly, and it seems like the easy solution to build simplified versions of the various components needed to launch the solution. However, as the company grows, they soon invest in maturing these components, which takes a lot of time and often requires dedicated teams. Building the components from the start is obviously a distraction and will initially and subsequently slow down time-to-market. It is very common for companies to allocate at least 40-60 percent of their developer capacity to building things unrelated to their core business.

Let’s change that!

PaaS and traditional IDPs

Fortunately, there are solutions available to help companies with some of these challenges. There are two prevailing paradigms: PaaS (Platform as a Service) and deployment-focused IDPs (Internal Developer Platforms). Much has happened in the field of IDPs, which is why we will refer to the latter as traditional IDPs. In essence, PaaS makes it extremely easy to get started, as most automation and infrastructure components come out of the box, leaving the company to focus solely on writing the software. However, as the company grows, it may find itself constrained by the black-box nature of PaaS solutions, which is where traditional IDPs excel. Such IDPs offer enough flexibility to customize a developer environment that provides a streamlined experience for developers. The only issue is that it still requires frequent platform team support, even though they are effective solutions. Additionally, traditional IDPs tend to differentiate between an Internal Developer Platform and an Internal Developer Portal, with the former exposing various tools and frameworks through APIs, and the latter serving as a user-friendly UI that consolidates these tools. Separating the platform from the portal seems archaic, as having the platform without the portal is only a 60% solution.

Unfortunately, neither PaaS nor traditional IDPs address the problem of reinventing software that has been developed countless times before. They are hands-off when it comes to the actual development phase.

Traditional IDPs and PaaS solutions, in many ways, address problems that arose with the Cloud Native movement, including orchestration, provisioning, and container management. In the 2000s, the concept of DevOps emerged, urging teams to take ownership of both building and running software, including maintenance and troubleshooting. This represented a step backward from frameworks that simplified development, where code-to-production was a matter of minutes, not a source of tension between operations and software teams. There were lightweight application servers and frameworks that encouraged rapid iterations, such as Ruby on Rails, followed by the JavaScript revolution. However with the emergence of Cloud based systems, teams were suddenly tasked with complex infrastructure development, diverting focus from the primary objective of delivering business-critical software.

Traditional IDPs and PaaS offer a world where deploying code to different environments is easy and seamless, speeding up development considerably. However, engaging in the actual development phase is crucial for teams to concentrate on writing logic that truly impacts the company.

Next-gen IDPs

While PaaS solutions prioritize simplicity and abstraction, they often lead to vendor lock-in, offering limited configuration and troubleshooting options. In contrast, traditional IDPs demand frequent involvement from the platform team and substantial initial implementation efforts. NextGen IDPs, however, bridge this gap by focusing on the entire software development lifecycle. This includes the actual development of the business-critical software and leverages AI to significantly accelerate turnaround times.

Many traditional IDP providers argue that creating a one-stop solution is impossible, and companies instead must customize workflows to align with their specific practices. While they offer the ability to use different tools and frameworks, it requires platform teams to assemble them. NextGen IDPs on the other hand, are opinionated and provide an out-of-the-box solution based on best practices. In 9 out of 10 cases, these practices are sufficient, while still allowing platform teams to make eventual customizations if needed. Why worry about building things like Continuous Integration/Continuous Deployment (CICD), monitoring, scaling, provisioning, and secret management when these capabilities can come out of the box, offering streamlined solutions to teams within the organization? In the 1 out of 10 cases where best practices fall short, NextGen IDPs offer a modular approach, allowing all the different parts of the platform to be extended and customized to meet the company’s specific needs.

NextGen IDPs are game changers with features as the following:

Architecture Tool

Architecture planning should not occur in tools separate from actual development. Architecture diagrams become outdated as soon as they are created. While they are excellent for detailing the ideal system, it requires discipline to keep them up to date. System architecture is a living organism and is continually evolving. NextGen IDPs incorporate visual tools that mirror the actual implementation 1:1. What you see is what you get, as services and code are generated from these diagrams. Suddenly, the diagrams become the go-to resource for understanding the system, including its services, connections, databases, and providers. Knowledge transfer is that way built into the development process.

Having visual, always up-to-date system diagrams is not only useful for the teams working on the systems but also provides IT leaders with a unique insight and overview of the organization’s systems. This level of understanding was not possible before the arrival of NextGen IDPs.

Service and Code Generation

Based on the above-mentioned architecture diagrams, NextGen IDPs generate services within seconds. The architecture diagram builder allows specifying APIs and their clients, ensuring that services are generated dynamically to reflect the actual APIs. This means fully functional API endpoints and API clients are generated. Additionally, NextGen IDPs possess insights into the system, including environment variables, database choices, authentication, and authorization, allowing them to generate all needed scaffolding code, from REST to the database layer, and create SDKs that significantly accelerate development. Essentially, all scaffolding code is readily available, allowing teams to focus on writing business logic in their preferred IDE. NextGen IDPs avoid the low/no code trap, offering code generation for trivial scaffolding while allowing developers the freedom to construct business logic as they see fit.

AI Assistance

In the era of AI, everything claims to be powered by AI, and while this may be true, it should also provide significant value to users. NextGen IDPs uses AI to create system architectures for users and, more importantly, are moving towards AI enabled creation of models, database schemas, and APIs based on architecture and user input. By having a comprehensive view of the entire system, not just isolated code blocks or classes like other code AI assistance tools, NextGen IDPs can enhance the architect’s capabilities and assist developers in writing code that aligns with the surrounding components.

Building Blocks

By encapsulating logic and exposing interfaces, NextGen IDPs makes it possible to build systems using building blocks that can be downloaded from marketplaces or created by teams within the company for reuse. Why reinvent the wheel when many services, such as payment modules, user management, notifications, and task management, have been developed numerous times before? These services are rarely essential to a company’s core business and should be treated as commodities. Now, we finally have systems that allow for this.

Security, Monitoring and Insights

With complete end-to-end knowledge of the system, NextGen IDPs can offer capabilities previously unattainable. This includes the ability to monitor live traffic between components, providing unique insights into system understanding, troubleshooting, and error detection. While the teams focus on this detailed information, NextGen IDPs additionally provide IT leaders with an overview of metrics and insights from all the organization’s systems. This comprehensive perspective offers first-hand knowledge to make informed decisions based on live and always up-to-date information.

Moreover, NexGen IDPs enforce strict uniform security and compliance regimes across all systems built. This approach includes making zero trust network architectures the default, thereby lowering the cognitive load, relieving individual teams from the burden of implementing complex security features that have little to do with writing business critical software.

To sum it up

So what is a NextGen IDP – is it a PaaS, an Internal Developer Platform or an Internal Developer Portal? It’s all of the three combined and then a lot of features not covered by any of those.

As mentioned, traditional IDPs and PaaS solutions primarily focus on getting software deployed, which are undeniably critical aspects of software development. However, NextGen IDPs encompass the entire software development lifecycle (SDLC), not just integration and deployment. They propose industry best practice workflows with the option for customization when necessary. They are e2e platforms which don’t limit or box the users in. They are extendable by plugins and by building own reusable components. They generate code with the help of AI and make it possible for developers to focus on bringing value to the company and not bother about writing unnecessary code and learning infrastructure frameworks.

NextGen IDPs offers one platform for the whole engineering department to streamline their software processes and facilitate effortless knowledge sharing. It gives the engineers the tools to focus and the managers to create an overview of the organization’s software and practices.

These are exciting times and this is only the beginning. NextGen IDPs is undoubtedly the future of software development.