Software development can be a complex business, even if you only want to complete a feature or bug fix. A lot of time, code and manpower goes into general technology problems. By setting up microservices that work independently of each other, you can develop much faster. Read how Portbase unravelled the monolith.
Portbase is at the centre of Dutch logistics. The non-profit organisation set up in 2009 connects all the parties in the logistics chains of the Dutch ports. Portbase facilitates data sharing between companies and information exchange with authorities in order to work faster, more efficiently and at lower cost.
As in so many organisations, Portbase's IT architecture was, until recently, a real monolith: one jumble of services that depended on each other. This jungle meant that development took a long time and many people had to work simultaneously, with joint releases only occurring once every fortnight at the most. Portbase wanted to change this with autonomous teams that were responsible for certain functionalities without being dependent on other teams, other infrastructure and the monolith.
Launching services with ease
As Lead Software Engineer, René de Waele and his team (and later other teams) set to work in 2018 to realise this transition. ‘The starting point was that we not only made the software scalable but also that each department could focus on domain aspects, or business rules instead of being purely concerned with the technology in the code. In short, only write things down in the code as you want them to be.'
The base for the solution was a tool that De Waele had previously developed: Flux Capacitor. ‘Trading a monolith for microservices is attractive, but the communication between the services is quite a challenge,’ he explains. ‘Thanks to Flux Capacitor, exchanging messages between applications and launching new services is a lot easier. Once you've launched a service, it can communicate with any other connected service.’
A jungle of end points
How did René and his team achieve this? ‘If you go back to the basics and look at what an application really does, you can distinguish three things. Commands are the requests to the application, for example: I want to register a new ship visit. Queries are questions, in this case for example: give me all the visits of this one ship. Then there are Side effects, for example, informing the port or customs that the ship has been reported.
In a dynamic environment where these kinds of messages are exchanged in large numbers and where reliability is essential, the traditional way is cumbersome. For example, services are often called up several times, requiring a load balancer to keep things running smoothly. Because end points keep increasing, the infrastructure becomes unnecessarily complicated, and all those end points also have to be secured. A specialised team of cloud engineers has to deal with these issues, and then you still have to launch your product.’
Microservices
This can be prevented by having the queries and commands handled by a central message broker. The various microservices are not tied together and none of them are directly connected to the Internet. ‘They are not applications that you can access in your browser. That also means they are less vulnerable to hacker attacks. Moreover, the message broker does not understand the content of the messages, but only stores which queries and commands are handled.
The message broker can also take care of load balancing. Moreover, we can follow exactly what happens. Not just the events, but also all incoming API requests, all messages that come in, which browser was used, and so on. At Portbase, we have now created dozens of microservices that read out the logs completely independently of each other.’
Testing applications with ease
Testing is now also much easier and produces much better results. ‘If you want to test different services, it is often a difficult process. You have to create a test environment with a database, different services and a network connection. Because this is cumbersome and error-prone, in practice the functional behaviour of the application as a whole is rarely tested. When testing, we are interested in the behaviour of the entire application. We want to test this constantly, without having to know how the application does it. We have written all our tests in that way.’
Backlog almost cleared
By now, the most important applications are completely separate from the monolith. ‘The transition took no more than six months per application and could be done with a tenth of the number of lines of code compared to the old situation. In other words, 90 per cent of the application consisted purely of technology. Instead of going live with a new version twice a month, we can now do it five to ten times a day. In fact, we now go live with several teams about a hundred times a day, every time with real features.’ Despite so many new things being made live, there are far fewer problems. ‘Before we started, the team had a sizeable backlog and it was growing faster than it was shrinking. After the six-month transition, the backlog was almost completely cleared.’
