TP2.3: Service Mashups and Developer Support

Subproject manager
Prof. Dr. Dr. h.c. Manfred Broy
Subproject manager
Dr. habil. Christian Prehofer


In a world of connected devices (IoT), there will be a huge amount of data that will be constantly recorded and used for real-time and/or historical analysis. Such analysis can lead to important insights regarding individual and group preferences and patterns of end-users (e.g. mobility models), the state of engineering structures (e.g. as in structural health monitoring), the future state of the physical environment (e.g. flood prediction in rivers). These insights can in turn allow the creation of sophisticated, high-impact applications. Traffic congestion can be avoided by using learned traffic patterns. Damages in buildings and bridges can be better detected and repairs can be better planned by using structural health monitoring techniques. More accurate prediction of floods can enhance the ways authorities and individuals react to them.
Despite the great potential of combining IoT sensing and actuating with data analysis, developing applications that control the operation of IoT sensors and actuators is alone a challenge. Developers have to write complex boilerplate code to communicate with heterogeneous devices. They have to perform custom data transformations on raw data to make them useful for later stages. Device identification and coordination are other distinct challenges in IoT application development.
To deal with some of these challenges, dedicated IoT development tools called IoT mashup tools can be used. Such tools expedite the process of creating and deploying simple IoT applications that consume data generated from IoT sensors, publish data to external services or other devices. They typically offer graphical interfaces for specifying the data flow between sensors, actuators, and services, which lowers the barrier of creating IoT applications for end-users.
Deriving insights from data collected from IoT devices is a separate challenge that falls primarily into the topic of data analysis and machine learning. In the past years, a number of mature tools have emerged that focus on manipulation and analysis of data of high volume, velocity, and variety commonly referred to as Big Data. Big Data analytics tools allow massively parallelized data analysis and machine learning algorithms to operate on data sets that reside in large clusters of commodity machines in a cost-effective way.
Although the integration of existing IoT mashup tools with Big Data analytics tools can allow the seamless development of sophisticated, high-impact applications, it is far from straightforward. Such integration should go beyond merely specifying IoT mashups that only act as data providers for Big Data clusters. Mashup developers should also be able to specify Big Data analytics jobs and consume their results within a single application model.
Existing IoT
Figure 1
It is important to interlink both the worlds in an integrated way. There are certain scenarios in which business logic of mashups may need input from Big Data jobs i.e. mashups may require to trigger data analysis. Similarly, after the execution of Big Data jobs there may arise need to perform some additional task i.e. may need to trigger a flow in mashup. Here we try to illustrate with a few example scenarios where the integration of Big Data analytics and business logic in application development is really useful to generate value for end-users.

Mashup involving Big Data analytics for traffic management
Consider a road traffic monitoring system which is used for traffic management and data collection. The system records live traffic data including flow monitoring i.e. number of vehicles using a certain motorway during different times of the day. The system automatically detects incidents like accidents, congestion, fire breakage etc. The recorded data is stored in clusters of Big Data systems. On detection of a new incident, the system generates an alarm. It is crucial to know the actual reason behind the alarm either to initiate appropriate counter measures for the smooth handling of city traffic or to treat it as a false alarm. Mashups are a good fit in this scenario as developing applications using tools is both quick and cost effective. A mashup designed to find the cause for such a scenario takes the incident as input and does a preliminary analysis to determine if such an incident ever occurred in near past. Based on this analysis, it decides if it has the prerequisite information to initiate counter measures or if it needs to derive more knowledge about the incident by running a set of analytic jobs on the data sets stored in the clusters.
Road incidents trigger a mashup to initiate appropriate counter measures to handle traffic efficiently.
Figure 2 (a) Road incidents trigger a mashup to initiate appropriate counter measures to handle traffic efficiently.


Deriving knowledge from Big Data analytic jobs to facilitate decision making.
Figure 2 (b) Deriving knowledge from Big Data analytic jobs to facilitate decision making.
In the event of availability of past information about similar incidents it can initiate appropriate counter measures directly. Otherwise, it invokes a set of jobs in parallel to perform further analysis to determine the cause of the incident like fire breakage, accidents or normal peak periods of the day etc. The application analyses the outcome of these jobs to facilitate an appropriate decision making. Figure a depicts the flow of the mashup from incident detection till its decision to either perform some Big Data analytics or take decisions based on historical information. Figure b depicts the parallel execution of Big Data jobs, analysis of their results to foster initiation of counter measures.

Challenges
– Support development of services & IoT applications using mashups.
– Support integration of third party APIs.
– Simplifying specification of complicated business logic and increasing transparency for the developer.
– Simplify data mediation in mashups.
– Support for easy access of IoT devices and consuming services offered by them.
– Support for integration of Big Data Analytics in application development.

Objectives
– RESTful representation of devices and services for easy mashup.
– Simplified visual notations for specifying complex logic like Big Data queries.
– Abstraction to hide code generation and automated data transformation between system components.
– Bridging the technological gaps of IoT & Big Data systems for easy integration.

Approach
– Develop a middleware which provides a REST interface to the Big Data systems and various big data analytic tools.
– Using an existing mashup tool like IBM Node-RED to demonstrate early integration using the principles of REST communication.
– Development of a set of visual semantics to facilitate specification of big data jobs within a mashup tool.
– Development of a mapper to map the visual flows into target big data query languages.
– Development of a control block in-between the mashup tool and Big data system to allow full integration and handle the issues of concurrency, deadlocks and multiple invocation of the same business logic.