Concepts like the client-server model and three-tier architecture require no special introduction. In the traditional setup, the responsibility for the presentation layer usually falls on the client, while data models and business logic live on servers. This arrangement is well-established and easily understandable. Even though there have been trends like the popularity of thick clients and phases where thin clients are preferred, a resource-based API has remained the standard choice for defining the communication between the server and client.
This concept has proven effective for several decades, but the beginning of the mobile age has introduced substantial changes. The landscape no longer relies on a contract between a single server and a client. One server must serve multiple clients built on diverse technologies, each with specific API expectations.
Naturally, the impact of this challenge varies depending on the company’s team layout. Companies that rely on cross-functional teams may experience less disruption. However, delivering a feature across all platforms generally becomes a complex effort. In our case, involving 3-4 engineering teams to deliver one new feature on all platforms introduces a significant communication overhead. When implementing the presentation layer logic on clients, it is necessary to communicate and code the logic separately for each client using different programming languages. The necessity to re-implement a substantial portion of the logic on each client implies extensive testing and significantly prolongs the time required for releasing the feature. Product Managers are genuinely concerned about the slow iterations and extended delivery times.
Additionally, mobile devices differ significantly from the desktop web experience in several ways. Their smaller screens require a more straightforward user interface with fewer visual elements. Also, having too many network connections open can significantly drain the battery life of mobile devices. These reasons demonstrate why relying on a low-level, general-purpose backend API for various clients is not optimal.
We looked into different solutions for our problems and found an architectural approach called Backend for Frontend (BFF). This approach is designed to handle the challenges that arise when dealing with multiple clients. The concept is straightforward: instead of using a single general-purpose backend, we create and maintain separate BFF APIs for each frontend application or interface. These APIs are closely coupled to the specific user experience of each application. Using the BFF approach offers several key benefits:
- Better performance: By using specialized backend services for each frontend application, the number of requests and data sent between the frontend and backend is reduced, resulting in faster load times and better performance.
- Simple maintenance and release process: Separating the backend services into smaller, more manageable pieces allows developers to easily maintain and update their applications, resulting in less downtime and faster recovery times.
- Greater flexibility: By breaking down the backend services into smaller pieces, developers can create more flexible and customizable applications to adapt to different use cases and scenarios.
As we delved deeper into our research, we also came across the server-driven UI pattern. This approach involves the server conveying UI components instead of traditional data resources to the clients. In other words, the server dictates how the UI should look and behave by sending the necessary UI components, and the client renders these components accordingly. This separation of responsibilities allows for greater flexibility in making changes to the UI without requiring extensive updates to the client code.
At Kiwi.com, we avoid overcomplicating things and lengthy waterfall projects that drag on for years without an initial release. Instead, we embrace an agile and iterative approach, prioritizing the prompt release of a prototype. We believe in collecting and analyzing real-world data early on and adjusting our approach accordingly if we don’t see promising results.
We adopted a similar approach when implementing the new architectural pattern for communication between the backend and its clients. Rather than starting on the development of 2-3 entirely new APIs from scratch, we decided to begin by adding a shared aggregation layer on top of our existing low-level backend resources. This configuration requires minimizing discrepancies in the UI designs between the desktop and mobile versions. The responses from the new intermediate aggregation layer endpoints are closely integrated with the design to ensure consistency and minimize redundant logic on the clients.
By introducing this new intermediate layer between the server and the client, we’ve reorganized the division of responsibilities as follows:
- The backend encapsulates clean, low-level data models, implements the business logic on top of them, and exposes a well-defined set of granular resource-based API endpoints.
- The intermediate layer is responsible for aggregating multiple resources from the backend. It transforms and merges the data into distinct, sometimes repetitive, data views that precisely align with concrete visual components.
- Each client (frontend) typically makes requests to a single intermediate layer API endpoint per page and renders visual components without the burden of complex logic.
In our initial implementation of the intermediate aggregation layer, we found it unnecessary to introduce additional network calls. Instead, we extended the Python backend to include the intermediate layer seamlessly. We utilized a dedicated client library atop the low-level backend handlers to achieve a certain isolation level. This approach ensures a clear separation between the intermediate layer and the backend code, promoting modularity and reducing interdependencies.
The intermediate layer retrieves the required resources from the backend. It performs necessary transformations or data manipulations, aggregating them into higher-level structures that correspond to the visual components. This setup also enables smooth collaboration between backend-oriented engineers and frontend developers with a basic understanding of Python for working on the intermediate layer code. Ultimately, it bridges the gap between these roles in the development process.
Here is an example illustrating the response body of an intermediate layer endpoint and the resulting UI for a simple page, demonstrating a direct mapping between the data and the visual components:
{
"header": {
"title": "Here are your options",
"description": "You have the Kiwi.com Guarantee, so we've got you covered. You can get an alternative trip to your destination or a refund on us."
"action": {
"title": "Learn more here.",
"smart_faq_id": 0,
"url": "https://www.kiwi.com/en/help/search/article/0/"
}
},
"resolution_options": {
"alternatives": {
"title": "Choose an alternative trip",
"icon": "airplane",
"description": "We can book one of the alternative trips for you for free."
},
"refund": {
"title": "Request a refund",
"icon": "refund",
"description": "We can offer you one of our refund options."
},
"acknowledgement": {
"title": "Keep the rescheduled trip",
"icon": "clock",
"description": "If the new schedule suits you, we can keep it as it is and update it in your trip details."
}
},
"banners": [
{
"type": "warning",
"title": "Please pick an option as soon as possible.",
"description": "Some options might not be available if you wait."
},
],
"offer_expires_after_seconds": 1757}
Our intermediate layer includes several optimization techniques to enhance performance. One of the optimizations in our intermediate layer is a caching mechanism for responses from backend handlers. This caching mechanism optimizes subsequent requests by retrieving the cached response instead of executing redundant backend logic. Also, to minimize the execution of repetitive SQL queries, the intermediate layer utilizes a database preloading technique. By preloading data before running backend handlers, the intermediate layer reduces the need for duplicate database queries and significantly improves efficiency.
In addition to these optimizations, the intermediate layer also takes on the responsibility of handling localization and currency conversions. We maintain a consistent and user-friendly experience across different platforms and locales by centralizing these localization and conversion tasks.
The initial implementation has brought significant advantages. However, we recognize its limitations and actively discuss future improvements for the intermediate layer. We have multiple scenarios on the table for consideration. One potential outcome is eventually splitting the shared layer and maintaining separate endpoints for web desktop and mobile clients. This evolution would bring us closer to the authentic Backend for Frontend pattern architecture.
Our intermediate layer is currently built on a single platform written in Python, which poses challenges when integrating backend APIs from different teams and APIs deployed to other clusters. To address this issue, we are exploring the possibility of making it more generic and adaptable, serving as an umbrella layer that can communicate with various backend platforms over the network.
During our journey, we have also encountered a few cases where the complexity of the aggregated data structures has made it difficult to seamlessly compose them by combining multiple JSON responses. In such instances, building API responses directly from the database models might be more efficient, eliminating the need for an intermediate abstraction layer. This alternative approach would align us more closely with the server-driven UI pattern architecture.
In conclusion, we have found significant value in transitioning our architecture to a more server-driven approach at Kiwi.com. By adopting a shared aggregation layer on top of existing backend resources, we have minimized discrepancies in UI designs and reduced duplicated work on the clients. However, it is essential to acknowledge the limitations of our current implementation and explore future improvements. Faster iterations remain crucial in our ever-changing industry, allowing us to respond effectively to travel emergencies and similar challenges.
Sources:
- https://samnewman.io/patterns/architectural/bff/
- https://medium.com/mobilepeople/backend-for-frontend-pattern-why-you-need-to-know-it-46f94ce420b0
- https://www.uber.com/en-US/blog/developing-the-actioncard-design-pattern/
- https://www.techtarget.com/searchsoftwarequality/definition/3-tier-application
- https://medium.com/@addyosmani/the-cost-of-javascript-in-2018-7d8950fbb5d4
