Размещено на http://www.allbest.ru

Размещено на http://www.allbest.ru

Introduction

Rapid application development (RAD) is a software development approach of construction of end-user products in conditions of changing user requirements and limited time resources, often with fixed deadlines. Specifications are often replaced by making prototypes. Nevertheless, the quality of the software remains an important requirement.

There are a lot of articles about different frameworks implementing this methodology (for example a comparison of them [1]) explaining how this process can be organized from the project manager's (or similar leading person of the team) point of view. Although there is a whole segment of software companies - studios of development, that make their applications by RAD or similar methodologies, there are not so many researches about technical ways and specificities of the applications development organization and optimization.

The principal difference of such studios from other software businesses is the short lifecycle of products they support. Often customers come with only the idea of the product or the problem they want to resolve by the development and integration of such software. Then the development team should elicit user stories and technical requirements of the future software product, taking into account key development principles of development studios to minimize net costs.

These principles are:

- Usage of common architecture;

- Usage of common libraries and tools;

- Extract code that can be reused into components.

Adherence to them allows easily reuse components between projects and thus reduce development costs. Principles are relevant not only for developer studios and startups but also for large software companies that want to reduce their time-to-market metrics.

The topic of reuse code is very popular among mobile developers these days. In my previous coursework [2] I have described a possible reduction of costs in the development of native mobile applications with usage of Kotlin Multiplatform SDK or Flutter framework. It is possible because of sharing a common code of common components not only between different projects targeted on one operating system but also between different platforms.

The effectiveness of using Kotlin Multiplatform can be illustrated by the case of the Ice Rock mobile development studio [3].That is why the opportunity of an easy transformation of the developing framework into multiplatform is taken into account of the Framework development and pure Kotlin is used for its implementation. Pure means that there are no third-party libraries written in Java or other languages not supporting Kotlin Multiplatform used.

The framework made during current research is targeted on Android clients because of the maturity of Kotlin Multiplatform on the JVM target. This allows avoiding implementation difficulties caused by the beta status of Kotlin Native used for iOS and other client targets and to focus attention on the other problems that are important in the context of this work.

The reuse of components is possible mostly because of the extraction of logical layers inside the application. This can be used with MVP as described in "Mobile Application Development with reusable code” [4]. Neither it can be organized by principles of Clean Architecture as described in my bachelor graduation work[5]. The common scheme for Android application can be described as following: View-> Business logic ->(Networking + Database). From the Business logic layer, the Repository layer is also often extracted. As described in “The Evolution of the Repository Pattern”[6]it is responsible for managing data received from Networking or Database by encapsulating the source of this data for the left Business logic code (Figure 1).

One of the possible ways of Repository organization is flowing all the data from Networking through the Database layer. Which is observable by Business Logic and View layers through some reactive tool (Figure 2). A popular example in Android development of such a tool is Room Flowable API [7].In the Repository a developer should organize the storage of responses got from the Networking layer (for example some API) directly into Database. A consumer layer can then subscribe to changes in the database and reflect them in the UI as soon as they happened.

Figure 1. Common application scheme with Repository

Figure 2. Data flow between layers under Repository

In the domain of Rapid-application development there is a need for coding Database, Networking and Repository layers in each application. However, it is possible if the Repository layer encapsulates this logic and subscribe to changes of data on the server by itself. And developers will use this wrapped into component layers same way as they used Repository before (Figure 3).



Figure 3. Use of Database with self-syncing Framework

The topic of this research is to develop protocols and algorithms needed for such communication between the local Database and Server and implement them in the Framework. It could reuse the number of code and speedup applications development.

The value of research can be evaluated with the help of the article [8] showing the possible growth of global RAD-market up to $95.2B by 2025.When applying this number to 39% market share of Android devices among others[9] the possible market of $37B can be received. A solution that can help reduce the costs of Android RAD-applications even on 1% can pretend to save $370M.

However, there already are some solutions for making requests to remote databases on backend communication easier in client applications prototyping and development. Most popular of them are Firebase Firestore SDK [10] and Backend less SDK [11]. They support corresponding Backend-as-a-Service (BaaS) providers and allow to push and fetch some data from them. Data is keeping in the document-oriented scheme (a subtype of NoSQL). In Firestore the SDK includes caching of requests.

Firestore SDK is an open-source solution that contains separate code for most platforms. There is no code reuse between them because they are written in usual for platform languages (Java for Android, Objective-C for iOS). The code of Backendless SDK is available for reading on GitHub but is proprietary and covered by patents. This SDK is also available for other platforms andis not cross-platform by the organization.

The code of backends of these services is fully unavailable. Therefore, these SDK can be only used with corresponding services with their costs, legal and functional limitations. For example, there is no opportunity to setup notifications when suitable data changed to organize real-time reflection of these changes in UI.Or these servers cannot be located in Russia in order to comply with the legislation on the processing of personal data.

Another similar by functional area analog of the developing Framework is Store library [12]. It is an open-source implementation of a Repository pattern allowing management of caching requests. As for it is written fully in Kotlin, it is aimed to support multiplatform. In addition, it provides an implementation of an in-memory cache. However, it does not provide the server, Database and Networking layer at all.

In contrast to the mentioned analogs, the created Framework consists of fully customizable and open for extension client and server parts. It is achieved with decoupled interfaces segregation for Database, HTTP and Pushes components. The base implementation is contained inside the framework. Because of interfaces, it is possible to implement them by any existing solution. For example, for reducing the size of third-party components developers can implement these interfaces with libraries they already use in the project. It can be also useful for implementation of the Database as an in-memory cache.

Kotlin Multiplatform principles [13] are laid down in the structure of the Framework. All the code is written in Kotlin with the usage of pure Kotlin (or multiplatform) components. It will allow to easily adopt the client code for usage in other than Android platforms and to share more logic between them. It increases the potential of the solution to save more money in the RAD-process.

All the interfaces between inner components are built around the Kotlin Coroutines[14]and Reactive concept implemented by Coroutines Flow [15]. This allows to use it as a Store in reactive apps build with MVI [16] or similar architectural patterns.

To investigate the topic of research the analogs analysis was committed. Together with previous researches about scalable architecture [5], cross-platform mobile development [2] and prototyping [17] it had given a background for the research. Another method used is the adaptation of tools from neighboring areas of knowledge. For example, at the core of first diffing algorithm used approaches from Git-diffing and Recycler View Diff Util [18]. They lead to foundational work of Myers [19] about difference algorithm and its variations.

The aim of the research is to speed up rapid applications and prototypes development by simplifying the work with remote domain data. The object of research is rapid-application development. The subject of study is implementation of domain data share between client and server together with caching it in client.

For achieving this goal, the following objectives in order of priority and importance for creation of the Framework were set up:

1) Designing the architecture of the Framework.

2) Previous includes determination of interaction protocol between server and client sides of the Framework.

3) Identification of API of the Framework for its users.

4) Implementation of core components of Framework:

a) HTTP engine of Server;

b) Push engine of Server;

c) HTTP engine of Client;

d) Push engine of Client;

e) Database on Client;

f) The core of the Framework on the client, which links HTTP, Push and Database components by business logic.

5) Adopt the code of the Framework to Multiplatform readiness from the beginning.

6) Resolving of versioning, caching and synchronization problem.

7) Including access control and authentication logic inside the Framework.

8) Optimization of network traffic consumption by introducing a diffing algorithm.

9) Other optimizations of using resources.

Each of these objectives will be described in further parts of the paper together with solutions utilized for their achievement.

During the research the following questions and hypothesis should be resolved:

1) How to organize the server part of the Framework to observe client changes of data made by other clients? Which server Framework to choose?

2) How to deliver such data updates from server to client? Will it be a continuous request from the Client to the Server? Should some kind of push-notifications appear when the need data change? How to implement such pushes?

3) How to manage the method of caching data? Should the cache work on demand, or should users manage the freshness of data during their requests? How to organize the flexibility of setup caching in the Framework?

4) How to store data on the Client and on the Server? Which database structure to choose? How it will support caching?

5) How to modularize the libraries effectively to maximize its code reuse between client and server parts?

6) Which serialization format for data transfer choose? How this data format will correlate with the chosen database and caching methods?

7) How to reduce the size of data transferred and stores both on client and servers? How data difference algorithm can be integrated into the Framework?

8) How to split and secure data which is directed to different people? How to organize authentication?

9) How to prepare the Framework for possible code reuse in other than Android clients' implementation? How to share logic with Backend?

In the following parts of the thesis answers on these questions and findings during hypothesis research will be described. It starts with the high-level organization of the Framework in common. Then the description of the protocol of client to server communication follows. In the next parts server's and client's specificities are described. They are followed by describing algorithms and components whose implementation answers on research questions and reflects insides from hypothesis checks.

1. High-level description of Framework for domain data organization

Definition of problem and existing analogues trying to solve it

According to the work's subject area - Rapid Applications Development - one of the main goals to be achieved by applying the Framework described in this work is to increase the speed of development of a minimum viable product (MVP) intended to check the hypothesis of the suitability of the developed product to solve customer's problem. Behavioral pattern “IKIWISI” (aka. “I know it when I see it”) mentioned in “Rapid realization of executable domain models via automatic code generation” article [20] assumes that in case of mobile applications development there should be an application with user interface, base business logic and often network communication for at least base flow of principal use case of the application. From this point of view, the prototype can be made in some prototyping tools. For example, AppSheet [21] can make prototypes based on the data from Google Sheets and thus imitates network communication and backend with the underlying database. Or if there should be an accent on outstanding user interface/experience some UI/UX prototyping tools like Figma [22] or Zeplin [23] can be used. All of the mentioned allow launching the prototype on real devices to show it to the customer and imitate real user experience. In addition to limitations of such prototyping, the most important drawback of such an approach is that nothing of made can be reused while scaling the prototype into a real (ready for publishing) application. That is why one of the possible ways of prototypes creation is building the prototype from the collection of easy to use components which can be also used for end-user software. This approach is usually used in software development studios producing lots of prototypes and converting them into real applications.

Lack of libraries for Repository layer organization

To describe in more details the specificities of making prototypes as software from scratch lets enumerate the core function and structure such mobile applications have. A good practice is to utilize at least MV* (MVP [4], MVI [16], MVVM[24], …) pattern or more complex like Clean Architecture [25] with more layers extracted from the Model. Common in them is UI representation in the View layer, business logic together with domain (core entities) named Model and some glue layer between them. It can be Presenter as in MVP with View devoid of any logic, ViewModel as in MVVM with data binding between View and Model, or Reducer as in MVI with the unidirectional data flow.

On View layer the reuse of code (inside one platform) can be achieved by the usage of some library of widgets or UI-component. There are some standards as Material components for Android [26], UIKit for iOS [27], or Flutter Widget Library [28] (which is cross-platform and allows to reuse UI between platforms also). Development companies also often implement their own design systems in libraries of UI components to unify the design and speed up development.

The connection of View and Model layers can be made by some libraries that implement core functionality for the patterns. Some examples are Moxy for MVP[29],MVICore[30] for MVI or ViewModel[31] from Android Jetpack for MVVM. For example, Moxy was developed in mobile development studio which allowed them to reduce boilerplate code to solve Android lifecycle specificities in most of the projects they made.

The business logic and domain are the reason why a certain application exists. Therefore, the only opportunity to minimize code there is to reuse it between platforms.

From the structure of the common mobile application the part connecting business logic with database and networking was intentionally left. Clean Architecture as many other approaches utilizes a Repository pattern [6] to encapsulate the logic of retrieving data neither from some API by the Network layer or from the Database.

One of the closest analogs covering this layer is the Store library [12]. It provides three most common patterns of Repository organization. Method Get returns most quickly came data either from Database or from Network. If the data came from Database it can be notas actual as could be from Network but users receive a more responsive application. Another method is Fresh, which retrieves data only from Network to make it most actual. Another approach is provided by the Stream method. It simply makes the Database observable. Thus, the fresh data from Network firstly goes into Database and only then delivered to the consumer by stream event.

Such a stream of data changes can be consumed in reactive apps build for example with MVI pattern. The only missed part there is who and when should make requests by the Network layer to the server to update the data.

The idea of common Store from Redux architecture for client and server

To explain the core idea of the Framework the Redux [33] architecture as shown in Figure 4can be reviewed with a possible place of the Framework inside. Redux implements the concept of unidirectional data flow: starting from the View layer which converts any user interaction like taps or symbols inserted into actions. Additional possible sources of actions are some Pushes or Callbacks from platform API or external libraries. All of these actions go through the Store component (not the library mentioned above). It keeps also the last state of the application. Reducers corresponding to the features or use cases take the last state and apply action on them receiving a new state and putting it back into the Store. While this transformation Reducers can make also requests to some API or Databases. View is subscribed to changes of state in the Store and reflects the freshest state. To make it possible there is a need for reactive approach utilized.



Figure 4. Place of Store in Redux pattern for Client applications

If think of client mobile application together with the server part (backend) its state is particularly synchronized with the data kept on the server by API and can be considered also as its state. Although Redux is very popular in web-development and starts to gain popularity in mobile clients' development, facts about its use in the backend were not found. However, the existing structure of Spring-based [34] backend applications (as a rather common approach) can be presented as a typical Redux architecture (Figure 5). Instead of the View layer in the client, there is a REST API in the backend, while the traditional Network (API) layer of the client can be absent or replaced by communication with connected services in micro-service architecture [35]. All the other principles stay the same. Store keeps the last state of the application. API calls create corresponding actions that are consumed by Reducers and converted into new states. The changed state can also be consumed through backend external API.



Figure 5. Presentation of Server application in Redux

The full scheme of client-server intercommunication if both of them are represented with Redux-architecture can be seen as in Figure 6. Here the API (Network) layer of the client is replaced by some tool (marked as a two-side arrow) that synchronizes data in Client's and Server's Stores. The Server's API was not reduced as it can be still available in its common form for other Clients (for example Web). Database is considered to be used only for keeping the intermediate states. So, it can be also encapsulated inside the Store component.

Figure 6. Client-Server application with common Store and synchronization tool

The Framework could be the implementation of the common Store for both the Client and the Server. While the business logic of each particular application made with Redux to react on actions can be contained in Reducers the Store is responsible for storing the shared state of the application and synchronization of it between Server and Clients.

Therefore, the domain of this work is laying as in the client applications development as on the server-side. The Framework includes a library that can be used as a Repository layer in Android applications and a backend module that can be run as a wrapper over common for clients remote database or included in more sophisticated backends as a transport layer (or microservice).

Although this idea was inspired by Redux's Store and sharing its state, the Framework (and its parts) can be used not only with Redux or MVI architectures. The client application can be not reactive at all but subscribe on a pipe with data from Server and put its own data in the outgoing pipe instead of making HTTP-requests directly. In fact, the test application uses MVP pattern inside.

Choice of Kotlin Multiplatform for implementation

As the Framework is made for rapid application development one of the most important things is to reduce the number of lines of code written by users (in fact developers). This can be achieved by maximization of code reuse inside the Framework and unification of interfaces for different platforms.

The possibility of business logic code reuse is a reason why Kotlin language is important in the name and content of this work. In the previous research [2] Kotlin Multiplatform was compared with Flutter by the share of code that can be shared and subsequent restrictions caused by them. While the Flutter is considered to be more appropriate for UI reuse, Kotlin Multiplatform is more appropriate for business logic reused.

It is also closer to the target platform which minimizes the number of limitations cause than any of cross-platform framework use. As for both technologies are rather new there are not so many cross-platform libraries. However, Kotlin allows using already existing platform libraries by wrapping them with common interfaces.

It is possible because of the Expect-Actual mechanism [32]. This is a kind of gate to separate common code from platform specificities.

With keyword Expect values, classes or functions that expected to do the same but implemented in platforms in their own way can be marked. Then for each of the targets (Android, iOS, …) implementations for Expect declarations should be marked with Actual keyword. Thus, for use of this structure in multiplatform code, there is a single multiplatform API while they are implemented differently inside. For the first look, this mechanism is very similar to common interfaces and their implementations. But there are two main differences:

1) If using interfaces than on some platform their implementations can be missed. Kotlin compiler checks that all the targets implemented all the Expect-s and throws compilation error if not. Therefore, there is no chance to forget to implement something.

2) If using interfaces all the implementations were tried to compile into every target. There should be some manual checks for the platform and appropriate dependency injection. Kotlin compiler instead takes the only needed implementation Actual and substitutes Expect by it. Actually, this makes possible platform libraries use.

As for now, the client library is targeted only on Android, there already are reserve parts for additional platform implementation. So, the idea can be tested on one platform and then be easily scaled later.

Principles of the Framework organization and tasks for it implementation

This approach is also following the unidirectional data flow principle. In contrast to usual client-server communication over HTTP REST requests where client sends a request and waits for a response and has no real-time data updates from server (similar to the way of View with Model communication in MVC or MVP), the developing communication is similar to MVI (or Redux) approach where clients send data changes (actions) and subscribe on any state updates from server. Not client goes to server to ask for updates, but server sends them through the Push mechanism.

Another important idea for the solution of the designated task is an openness for any improvement or extension by the replacement of used technologies (underlying libraries) with their analogs if it gives any profit. For example, for reducing the number of overall 3rd party libraries used the Framework's developer (user of the Framework) may want to replace HTTP or Database implementation while core interfaces important for the logic of the Framework will be left unchanged. This approach was inspired by the architecture of cross-platform networking framework Ktor[36]. It contains both HTTP-client and HTTP-server plus additional tools around it. All these tools are linked with each other by interfaces or expect-actual mechanisms. However, some implementations can be easily replaced by others. For example, there is a ktor-client-core module with Expect interfaces that are implemented by ktor-client-okhttp or ktor-client-android on Android or ktor-client-ios for iOS. Depending on connected implementation HTTP requests on Android can be made by OkHttp library or tools from Android SDK.

The implementation of the Framework was carried out according to the incremental model by reaching a working version of the backend or the client part and testing them together. This methodology was chosen because of lots of uncertaintiesas in libraries and approaches used as in the end structure of the Framework.

Based on the high-level organization of the Framework the list of tasks that should be completed to make this work outstanding among the analogs was created:

1) Setup of base modules of the Framework.

2) Creation of base HTTP connection between client and server over REST API. This includes the choice of underlying libraries and serialization protocol.

3) Choice and setup of the local client database. Retrieving data in an observable form. Resolving the specificity of working SQLite in the Coroutines thread model.

4) Choice of the data structure.

5) Creation of mechanism of retrieving callbacks in the client when the server's data changed. Choice of the underlying implementation.

6) Combination of previously made HTTP, Database and Push mechanisms into a working prototype of the Framework.

7) Extraction of the client library from the client code.

8) Change of data structure because of the scalability limitations of the previous one.

9) Determination of changes in client's and backend's data while absent connection and merging of common state after reconnection.

10) Utilization of data diffing algorithms to reduce the size of data transferred.

11) Splitting and securitization of data directed to different clients. Organization of authentication process.

Conclusions and results for second chapter

In this chapter the problem together with limitations that were taken as the basis for the organizationof the Framework were described. The conclusions made following paragraphs are:

1) There no analogues that allow to reuse the code of prototype in implementation of ready to use application.

2) The common structure of client application was described. Although there lots of libraries that allow to reuse code written for View, Presentation and Model layers, there almost no solution connecting Repository, Network and Database layers together. Existing Store library implements only link between Network and Database and does not cover the problem of automatic synchronization of data with server.

3) The Redux architecture used for client (mostly web) applications development was applied for backend application structure. It allowed to notice the possibility of extraction common Store component for State sharing between client and server over some synchronization mechanism. This idea became a basis for the Framework organization.

4) Kotlin Multiplatform allows to reuse code between different client platforms and also with a server. As for the need of code reuse is one of the most important for rapid applications development the framework was chosen as a base for the Framework implementation.

5) The Framework should be applicable for use with any client architecture. The inner structure should wrap core components with interfaces to be open for replace by other implementations.

In next chapter details of the Framework implementation arranged in chronological order of component creation will be described.

2. Implementation of the Framework for domain data synchronization

Description of code base's modularization

To start the description of tasks completed during the Framework creation the structure of codebase should be described. As shown in Figure 7 it consists of 4 Gradle modules. The App module is an example of the Android client application that can be made with the use of this Framework. Common is a multiplatform module containing models and algorithms common for server and client parts. Backend is a JVM-based module containing the backend application that can be deployed and used as a simple wrapper over a remote database. Neither it can be included inside a more complex backend system. ClientLib module is a multiplatform module containing common code for client use of the Framework. There is also an Android-specific source set which includes code and libraries specific for JVM and Android. As for now, they cannot be used in other platforms such as iOS. However, this part can be easily complemented by other platform implementation analogs to use the whole Framework on them. This module can be included as a library in other client applications. The server-client communication, in fact, is carried out between Backend and App modules while others are needed for code reuse.



Figure 7. Structure of Gradle modules of the Framework's codebase

Firstly, the task of base module setup was taken. Its solution should answer research questions about effective modularization and maximization of code and logic reuse between backend and different clients including Android. Mostly these questionswere solved by the use of Kotlin Multiplatform approach [2], [13]. From the beginning,there was no separate ClientLib module in the scheme above to speedup initial code writing and reduce usability issues connected with separate module compilation and inclusion as a library. Multiplatform Common and ClientLib modules can be compiled into all the targets supported by Kotlin Multiplatform. This includes JVM, Native (for example, iOS), JavaScript, and much more. Module compiled into JVM-bytecode can be used by Backend targeted to JVM and also by Android application. Later the same Kotlin code together with the common code from ClientLib can be compiled by Kotlin Native compiler backend into an iOS-compatible framework (binary package) and used in iOS client application. iOS is mentioned most as the second popular mobile platform, the specificities of Android applications are most common with. But this code can be also compiled for use in most of the other targets (such as Web, Windows, macOS and so on).

Creation of standard HTTP connection between client and server over REST API

After the base structure of the Framework was set up, there was a task of creation of base HTTP connection between client and server over REST API. Among the JVM-based solutions for backend, the Spring Boot Framework [34] is one of the most popular. It was compared[37] with the Ktor[36]. The last was chosen because of three main reasons:

1) It has a multiplatform structure and corresponding HTTP clients. This allows increasing code reuse between modules by using the same libraries for serialization and other data preprocessing.

2) It can be easily decoupled to microservices by the functional nature of the routing model [38].

3) It is developed fully in Kotlin and in tight contact with Kotlin's team.

For the client's network library Ktor was also chosen because it is still the only multiplatform client lib providing such functionality.

Yet another choice was made is a serialization format and library. JSON was chosen as the most common and convenient for testing REST API-requests and only serialization format supported by the Ktor server. From libraries there are three converters compatible with KtorContentNegotiation[39]: kotlinx.serialization, Gson and Jackson. Kotlinx.serialization does not support easily collections serialization, that is why preference came to Gson. In future there may be a migration to more compressed data format such as Protobuf (if it also will be supported as a Content Negotiation format by Ktor).

The drawback of choice of Gson as a librarycan be got at the stage of iOS-support addition. As Gson and Jackson use Java reflection mechanism for serialization and most important deserialization and native code does not support reflection there may be need to rethink of use kotlinx. serialization library as a multiplatform implementation of serialization not using reflection at all.

Another profit of using JSON as a serialization format is the opportunity to use it also for keeping the data in the relational database. Such method of caching (instead of keeping each entity in a separate table) is also used by Firebase Firestore library [10].

An important setup for JSON serialization made is serializing nulls. Later the algorithm of diffing will be described. Important for the format there is the representation of removed nodes as null. So, if JSON serialization by default will omit keys with null values the diffing will be working incorrectly.

With chosen libraries the base connection was setup with an opportunity to made CRUD (Create = Post, Read = Get, Update = Put, Delete = Delete) HTTP-requests from the client to the server. This configuration was named standard in title of a paragraph as CRUD methods were represented with corresponding methods of HTTP. However, this representation will be changed in future part of work.

The backend was deployed to Heroku [40] service for client testing relief.

Choice and setup of the local client database for retrieving data in an observable form

The next step was a choice and setup of a database for local client data caching. The choice was made in favor of SQLDelight[41]. It is an open-source and multiplatform wrapper over the SQLite database. SQLite by itself is available at least for both Android and iOS and provides the opportunity to receive callbacks from the table when it's data changes.

As an alternative, there was an idea to use a NoSQL database instead of relational SQLDelight. The possible variant is the use of cross-platform Couchbase libraries set [42], [43]. As for now, there are no clear reasons to use it besides seeming more natural way of keeping tree-structured data. Also, the opportunity of wrapping these several separate platform-dependent libraries with a common multiplatform interface (with Expect-Actual keywords) for use in multiplatform projects needs additional research. However, there is an interface for the Database dependency in the ClientLib module, therefore later the SQLDelight can be easily replaced by wrapped Couchbase or another similar NoSQL database and compared by functional orefficient advantages.

Android thread model warns of long-term retrieving the data from the network or database in the main thread because it is responsible for UI reaction and refresh. Such calls cause the freezing of UI and therefore are undesirable for users. Use of Coroutines is the way to avoid such a situation that recommended by Google for Android applications [44] and supports multiplatform [14].

Another important feature should be possibility of reactive fetch of data from Database. It will allow to get changes in state as soon as was changed in Database. Figure 8 illustrates the difference with imperative approach, where consumer of data from database should manually make the request to receive new data. With reactive mechanism the data from Networking layer can be simply put into Database and consumer will receive it immediately. Coroutines library includes a Flow mechanism for this purpose[15].

Flow by its nature is similar to an implementation of ReactiveX[45] specification of cold streams. While additional concept of Coroutines Channels implements hot streams and provides functioning of Flow and a two-side pipe of data that are used for organization of events broadcast (Broadcast Channels) in the Framework.

Figure 8. Difference between imperative and reactive approaches of working with Database

Therefore, it was decided to wrap SQLDelight callbacks of table changes into Flow emitting the most actual data. This opportunity is supported in the library but is not described properly. There was found a misuse scenario caused by the specificity of SQLite to run all the SQL-requests in a single separate thread. Coroutines by themselves do not fix the thread, where execution is performed. That was described in the issue to SQLDelight library [46] with a suggestion to describe in the documentation the need of configuration separate Coroutine with a single thread for database work. And the same was made in the project to reach need functionality.

On the server-side the database is wrapped with Repository and implemented as in-memory cache. Therefore, all the data can be lost when the server application restarts. This can be easily changed by implementing a database interface with any server-oriented database. But this outstands from the domain of this work.

Choice of the data format and creation of mechanism of callbacks in the client when the server's data changed

For the first tests of the Framework in Common module was created two test entities: User and Post. They had separate endpoints on the backend and separate tables in the local Database. However, the sample code for making HTTP-requests and Database-work were duplicated as a corresponding logic on Backend. This disadvantage of the Framework organization was unnoticed at the initial stage but lead to a change of data format when trying to decouple ClientLib from the code. This problem together with possible and chosen solution will be described later.

To reverse the way of data changes spread between Client and Server and thus make the unidirectional data flow the mechanism of delivering these changes from Server to Client needed.

At this point there were no callbacks from the backend about data changes yet. To implement this several approaches could be utilized. An obvious and most easy way is to repeat HTTP requests from the client to the server in a cycle with some delay period. There are two drawbacks in this approach. First is a significant delay between when data changed on the server and the client made the request. The second one is that client makes requests much more frequently than it needed and thus makes an extra load on the server. Because of such limitations the search was directed to an inverted way of notification by the server when it needed with some analog of push-notifications.

The first one of rather common mechanism of push-notifications is implemented by Firebase Cloud Messaging [47]. It provides interfaces for sending notifications from the backend and receiving them in the Android or iOS client. The advantage of this approach is lower computation time and energy consumption by the client application because the listener is encapsulated in Google Play Services in Android. The obvious disadvantages are binding to the Firebase service account and Google Play Services. That is why push-notifications cannot be received on devices without them installed (for example a huge number of Huawei devices).

The next candidate to be an implementation of push-notifications was usage of raw sockets. Thus, the client application would establish a connection with the server and hold it for some time. During this the server is able to send data to the client when it changes. The disadvantages of such an approach are caused by a low level of abstraction of sockets in the network communication model. A need to implement some data transfer causes duplication of some existing application-level protocols such as HTTP or WebSocket.

That is why WebSocket was chosen to implement push notifications. It is an abstraction over sockets with similar to the HTTP interface and the opportunity to add additional interceptors like serialization with almost no cost. WebSocket is also implemented in Ktor set of libraries [36] It is also multiplatform and available both for the client and the server. Push mechanismwas also covered by an interface to be able to change the implementation (for example on Firebase Cloud Messaging pushes) in the future. However, this interface segregation helped already during the Framework base flow implementation. Ktor WebSocket client (Android) library failed to be setup for receiving data and thus was temporarily changed by more obvious OkHttp WebSocket implementation [48]wrapped with Expect-Actual and put into Android-specific sourceset of ClientLib module.

The principal scheme of new data notifications can be described as following. When the network connection appears, the Framework on the client establishes the WebSocket connection to the server. When the server realizes that data consumed by some client was changes and it there is a connection to it, it sends the metadata about data changed to this WebSocket. While principal parts of the Framework (HTTP, Database and WebSocket) were not connected the client just logged these notifications in testing purposes. The possible improvement here is to reduce the number of push-notifications made by the placement of additional logic if data need to be updated in the server.

Implementation ofprototype of the Framework with previously made Network, Storage and Push mechanisms

After three main components of the Framework were implemented and separately tested the next task was to connect them into one working prototype that can make CRUD HTTP requeststo the server, receive notifications about data changes and keep a local copy of this data. Now when the client receives the notification over the WebSocket about data changed it make a decision of data update need. If need it makes an HTTP request to the server and puts the response into Database. The Database by observable Flow is connected with the UI of a sample application which reflects the actual state of the database. As there only two test entities at this stage (User and Post), they are displayed as the list in RecyclerView.

From the CRUD requests to the server Create (Post), Update (Put) and Delete remained in the implementation as usual REST API requests and were reflected in the interface of the Framework. While the Read (Get) request becomes redundant for users because of the replacement of it by the fresh state of Database and a flow emitting new data as soon as they are changed. Create, Update and Delete operations, in fact, were later replaced with single applying diff operation implemented as a Post request to the server.

In the previous paragraph the need to observe network connection status change to establish a WebSocket connection was mentioned. The most suitable interface for that could be a Flow with network connection status that can be subscribed on and linked with other reactive organization of the Framework. However, there was no suitable and ready to use solution for this purpose found. The closest is the ReactiveNetwork library for Android that provides a similar approach implemented for RxJava. The specificity because of which such a library exists is the difference in an appropriate mechanism of determination of network state on Android depending on the version of the operating system. There is an issue in this library for Kotlin and Flow support [49] however it was not implemented yet. That is why was decided to port this library to Kotlin and Flow to be used inside the Framework. The multiplatform interface was also extracted to have an opportunity to implement it in iOS. This library was be returned back to the open-source community separate from the Framework [50].

Extraction of the client library from the client code with change of data format because of the scalability limitations of the previous one

After the first prototype implementing the base use case of the Framework on test data was made the need to separate a module with ClientLib from the App module appeared. As was written before, initially this module was not separated to make the development easier. The code independent from concrete entities was placed into this new module. It includes helper code for three components of the client Framework: HTTP client, Database and WebSocket. And the code which glues them under the Repository interface. However, during the process of separation, the problem with the initial choice of data format revealed. The code of these components turned out to be tightly coupled with the entities which laid down in the Common module. For the addition of new entities into the domain of the application there was a need to repeat the code for creation and work with tables in the client database, and HTTP-requests serialization in both client and server.

One of the possible solutions of this limitation could be leaving the situation as is and instruction of the Framework users on how to implement corresponding code for new entities. However, as for the Framework is developing for rapid application development and aimed to reduce the lines of code written by users, this variant is inappropriate.

Another solution is to create a code generation engine based on annotation processing [51]. This approach is common to use as in the rapid application development domain [20] as in other popular in Android-development libraries such as Moxy, Dagger 2, Room. The drawback of this solution besides the complexity of implementation is the significant negative influence on the speed of compilation caused by the annotation processing mechanism. Therefore, this variant was left for the case if no other good solution was found.

However, there was a more appropriate third solution found. It is the use of a single high-level entity that can include inside any tree-structured scheme of user entities. They can be represented as a map, where the key is the name of entity and the value is a list of such entities. Each of them is also a map, where keys are names of properties (or fields) and values are their actual values. This scheme is similar to the way NoSQL (or document-oriented) databases work.

In the Figure 9the representation of Entitystructure is shown. Asimpleuser-defined class with oneproperty (as User in Figure 11) can be kept directly in leaf property of corresponding Entity instance. A more complex structure as a Post in Figure 10 can be represented in nesting of Entities. The number of nesting levels could be more if Post's properties were represented by more complex than String data types.

In Figure 11 structure of Entities tree representing two user-defined entities User and Post is shown. All of the user-defined entities by themselves are represented as keys in the root Entity data map. And instances of these entities are kept in key-value pairs of entities' data map. A more detailed example of how this data mapping occurs will be shown in 4th chapter.



Figure 9. Representation of Entity class in Kotlin

Figure 10. User-defined Post class and its representation in Entity

Figure 11. Tree representation of Entity with User and Post user's entities

Such a data scheme allows using predefined HTTP and Database helpers for these only entities, while the user-defined will be able to be added without new code need. The existing structure of the Framework with extracted entity-independent code allowed to easily migrate to a new data scheme. The only concern is about the effectiveness of using a relational database with a JSON representation of a new scheme written inside instead of using special NoSQL Databases. However, this left as a further way of possible Framework optimization.

The task of migration to a new data scheme ends with the new Framework prototype tested with the same functionality that was there before the migration.

Determination of changes in Client's and Server's data while absent connection and merging of common state after the reconnection

The next step is to think about the offline mode of the client application. As there is a cached version of data in the local database - there no problems to implement displaying its content in the user interface. The problem comes up in case of simultaneous (from both the client and the server-side) change of the same data. After the client will reestablish the connection to the server it needs to synchronize data changes such that both would have the same data state and no data changes would be lost. There also several approaches can be used.

The first of them is to make the local copy of the data changed after a lost connection on both client and the server. After reestablishing the connection need just to make a merge of these differences. The details of such difference determination algorithms will be described in the next paragraph. If for the client there will be a single copy of data with such difference inside, there is a problem in the server because of need to keep as many as clients copies of the same data. If server changes for each client will not be saved and clients changes will be applied to the last server state after reconnection there is a chance of data loss because client changes can be based on outdated data state.

...