Real-Time Wireless Control Networks

Team

Faculty: Chenyang Lu, Yixin Chen

PhD Students: Rahav Dor, Dolvara Gunatilaka, Yehan Ma

Alumni: Octav Chipara, Abusayeed Saifullah, Chengjie Wu, You Xu, Bo Li

---

WirelessHART Network Architecture (Credit: HART Communication Foundation)

Wireless sensor-actuator networks represent a new generation of communication technology for industrial process monitoring and control. With the adoption of WirelessHART, an open wireless sensor-actuator network standard, recent years have seen successful real-world deployment of wireless control in process industries. Industrial control systems impose stringent real-time and reliability requirements on wireless control networks. We are developing a new real-time scheduling theory and network protocols for real-time wireless control networks. Our research addresses both practical problems in current WirelessHART networks and fundamental challenges faced by future wireless control networks.

This work is sponsored by NSF through grant CNS-1320921 (NeTS).

Real-time Scheduling Theory for WirelessHART

We established a novel real-time scheduling framework for wireless control networks based on WirelessHART by bridging real-time scheduling theory and wireless networking.

Dynamic transmission scheduling: We devised both optimal and near optimal policies for dynamic priority scheduling of transmissions for real-time flows between sensors and actuators. We observed that transmission conflict (due to half-duplex radio of the nodes) plays a major role in communication delays, making the traditional real-time scheduling policies less effective for transmission scheduling in WirelessHART networks. Using this key observation, we designed an optimal algorithm based on branch and bound, and a heuristic called Conflict-aware Least Laxity First (C-LLF) for dynamic priority scheduling. C-LLF integrates the degree of conflicts associated with a transmission into LLF, and outperforms traditional real-time scheduling policies. This work was presented at [RTSS'10]. We also provided a schedulability analysis for earliest deadline first (EDF), a common dynamic priority scheduling in WirelessHART networks. This work was presented at [IWQoS'14].

Fixed-priority transmission scheduling: For wireless control with firm requirements on network latency, a delay analysis is required to quickly assess the schedulability of the real-tme flows, specially for online admission control and workload adjustment in response to network dynamics. We provided an efficient analysis of the worst-case communication delays of periodic real-time flows that are scheduled based on fixed priorities in a WirelessHART network. The key insight in the analysis is to map real-time multi-channel transmission scheduling to real-time multiprocessor scheduling. This approach allows us to build on existing real-time scheduling theory while focusing on incorporating unqiue features of wireless communication in the analysis. Our analysis establishes safe upper bounds on end-to-end delays, thereby enabling effective schedulability tests for WirelessHART networks. We also proposed optimal and near-optimal priority assignment algorithms based on local search and heuristic search, respectively. Our search approach leverages the lower and upper delay bounds provided by our delay analysis to reduce the search space. The delay analyses for single independent routes and for graph routes were presented at [RTAS'11] and [RTSS'15], respectively, and priority assignments was presented at [ECRTS'11].

Cyber-Physical Co-Design for Wireless Control Systems

Rate Selection for Wireless Control Systems: In a wireless control system, the choice of sampling rates of the feedback control loops must balance control performance and communication delays. A low sampling rate usually degrades the control performance while a high one may cause excessive communication delays also leading to degraded control performance. We addressed the scheduling-control co-design problem of sampling rate selection to optimize the overall control cost in a WirelessHART network. The resulting constrained optimization is challenging since it is non-differentiable, non-linear, and not in closed-form. We proposed and evaluated five optimization methods including greedy heuristic, subgradient method, simulated annealing based penalty method, and gradient descent method and interior point method upon a differentiable convex relaxation. Our result in this work has drawn some interesting conclusions towards co-design. In particular, we have shown the interior point method and the simulated annealing based adaptive penalty method as the two most effective approaches for rate selection. They represent the opposite ends of the tradeoff between control cost and execution time, while the interior method is likely the most effective approach in practice due to its run time efficiency. This work shows the promise of cyber-physical co-design, where tailoring real-time scheduling analysis allows for an elegant and efficient optimization approach for wireless control systems. This work was nominated for Best Paper Award at [RTAS'12].

Incorporating Emergency Alarms in Reliable Wireless Process Control: Many real world process control systems must handle various emergency alarms under stringent timing constraints in addition to regular control loops. However, despite considerable theoretical results on wireless control, the problem of incorporating emergency alarms in wireless control has received little attention. We develop the first systematic approach to incorporate emergency alarms into wireless process control. The challenge in emergency communication lies in the fact that emergencies occur occasionally, but must be delivered within their deadlines when they occur. We propose efficient real-time emergency communication protocols based on slot stealing and event-based communication. We build an open-source WirelessHART protocol stack in the Wireless Cyber-Physical Simulator (WCPS) for holistic simulations of wireless control systems, and conduct systematic studies on a coupled water tank system controlled over a 6-hop 21-node WSAN. Our results demonstrate our real-time emergency communication approach enables timely emergency handling, while allowing regular feedback control loops to effectively share resources in WSANs during normal operations. [ICCPS'15]

Figure 1. Wireless Networked Control System

Wireless Routing and Control: Wireless sensor-actuator networks (WSANs) are being adopted in process industries because of their advantages in lowering deployment and maintenance costs. While there has been significant theoretical advancement in networked control design, only limited empirical results that combine control design with realistic WSAN standards exist. We conduct a cyber-physical case study on a wireless process control system that integrates state-of-the-art network control design and a WSAN based on the WirelessHART standard. The case study systematically explores the interactions between wireless routing and control design in the process control plant. The network supports alternative routing strategies, including single-path source routing and multi-path graph routing. To mitigate the effect of data loss in the WSAN, the control design integrates an observer based on an Extended Kalman Filter with a model predictive controller and an actuator buffer of recent control inputs. We observe that sensing and actuation can have different levels of resilience to packet loss under this network control design. We then propose a flexible routing approach where the routing strategy for sensing and actuation can be configured separately. The proposed asymmetric routing configuration with different routing strategies for sensing and actuation can effectively improve control performance under significant packet loss. Our results highlight the importance of co-joining the design of wireless network protocols and control in wireless control systems. This work was presented at [ICCPS'16].

Implementation and Empirical Studies of Industrial WSAN Protocols

Wireless sensor-actuator networks (WSANs) offer an appealing communication technology for process automation applications. While industrial WSANs have received attention in the research community, most published results to date focused on the theoretical aspects and were evaluated based on simulations. There is a critical need for experimental research on this important class of WSANs. We developed an experimental testbed by implementing several key network protocols of WirelessHART, an open standard for WSANs widely adopted in the process industries, including Time Slotted Channel Hopping (TSCH) at the MAC layer and reliable graph routing supporting path redundancy. We performed a comparative study of the two alternative routing approaches adopted by WirelessHART, namely source routing and graph routing. Our study shows that graph routing leads to significant improvement over source routing in term of worst-case reliability, at the cost of longer latency and higher energy consumption. It is therefore important to employ graph routing algorithms specifically designed to optimize latency and energy efficiency. This work was presented at [EWSN'15]

To meet stringent reliability requirements of industrial applications, industrial standards such as WirelessHART adopt Time Slotted Channel Hopping (TSCH) as its MAC protocol. Since every link hops through all the channels used in TSCH, a straightforward policy to ensure reliability is to retain a link in the network topology only if it is reliable in all channels used. However, this policy has surprising side effects. While using more channels may enhance reliability due to channel diversity, more channels may also reduce the number of links and route diversity in the network topology. We empirically analyze the impact of channel selection on network topology, routing, and scheduling on a 52-node WSAN testbed. We observe inherent tradeoff between channel diversity and route diversity in channel selection, where using an excessive number of channels may negatively impact routing and scheduling. We propose novel channel and link selection strategies to improve route diversity and network schedulability. Experimental results on two different testbeds show that our algorithms can drastically improve routing and scheduling of industrial WSANs. This work will be presented at [INFOCOM'17]

CapNet: Real-time Wireless Management Network for Data Center Power Capping

Data center management (DCM) is increasingly a significant challenge for enterprises hosting large scale online and cloud services. Machines need to be monitored, and the scale of operations mandates an automated management with high reliability and real-time performance. Existing wired networking solutions for DCM come with high cost. Wireless sensor networks provide a cost-effective networking solution for DCM while satisfying the reliability and latency performance requirements of DCM. We have developed CapNet, a real-time wireless sensor network for power capping, a time-critical DCM function for power management in a cluster of servers. CapNet employs an efficient event-driven protocol that triggers data collection only upon the detection of a potential power capping event. We deploy and evaluate CapNet in a data center. Using server power traces, our experimental results on a cluster of 480 servers inside the data center show that CapNet can meet the real-time requirements of power capping. CapNet demonstrates the feasibility and efficacy of wireless sensor networks for time-critical DCM applications. This work was reported at [RTSS'14].

Publications

• D. Gunatilaka, M. Sha and C. Lu, Impacts of Channel Selection on Industrial Wireless Sensor-Actuator Networks, IEEE International Conference on Computer Communications (INFOCOM'17), May 2017 PDF

• M. Sha, D. Gunatilaka, C. Wu and C. Lu, Empirical Study and Enhancements of Industrial Wireless Sensor-Actuator Network Protocols, IEEE Internet of Things Journal, accepted. PDF

• C. Lu, A. Saifullah, B. Li, M. Sha, H. Gonzalez, D. Gunatilaka, C. Wu, L. Nie and Y. Chen, Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems, Special Issue on Industrial Cyber-Physical Systems, Proceedings of the IEEE, 104(5): 1013-1024, May 2016. PDF

• B. Li, Y. Ma, T. Westenbroek, C. Wu, H. Gonzalez and C. Lu, Wireless Routing and Control: a Cyber-Physical Case Study, ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS'16), April 2016 PDF

• C. Wu, D. Gunatilaka, A. Saifullah, M. Sha, P.B. Tiwari, C. Lu and Y. Chen, Maximizing Network Lifetime of WirelessHART Networks under Graph Routing, IEEE International Conference on Internet-of-Things Design and Implementation (IoTDI'16), April 2016. PDF

• C. Lu, A. Saifullah, B. Li, M. Sha, H. Gonzalez, D. Gunatilaka, C. Wu, L. Nie and Y. Chen, Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems, to appear in Proceedings of the IEEE, 2016. PDF

• A. Saifullah, D. Gunatilaka, P. Tiwari, M. Sha, C. Lu, B. Li , C. Wu, and Y. Chen, “Schedulability analysis under graph routing for WirelessHART networks”, IEEE Real-Time Systems Symposium (RTSS'15), December 2015. PDF

• B. Li, L. Nie, C. Wu, H. Gonzalez and C. Lu, Incorporating Emergency Alarms in Reliable Wireless Process Control, ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS'15), April 2015. PDF

• M. Sha, D. Gunatilaka, C. Wu and C. Lu, Implementation and Experimentation of Industrial Wireless Sensor-Actuator Network Protocols, European Conference on Wireless Sensor Networks (EWSN'15), February 2015. PDF

• A. Saifullah, Y. Xu, C. Lu and Y. Chen, End-to-End Communication Delay Analysis in Industrial Wireless Networks, IEEE Transactions on Computers, accepted. PDF

• A. Saifullah, S. Sankar, J. Liu, C. Lu, R. Chandra and B. Priyantha, CapNet: a Real-Time Wireless Management Network for Data Center Power Capping, IEEE Real-Time Systems Symposium (RTSS'14), December 2014. PDF

• C. Wu, M. Sha, D. Gunatilaka, A. Saifullah, C. Lu and Y. Chen; Analysis of EDF Scheduling for Wireless Sensor-Actuator Networks, ACM/IEEE International Symposium on Quality of Service (IWQoS'14), May 2014. PDF

• A. Saifullah, C. Wu, P. Tiwari, Y. Xu, Y. Fu, C. Lu and Y. Chen, Near Optimal Rate Selection for Wireless Control Systems, ACM Transactions on Embedded Computing Systems, Special Issue on Real-Time and Embedded Technology and Applications, 13(4s), Article 128, April 2014. PDF

• O. Chipara, C. Lu and G.-C. Roman, Real-time Query Scheduling for Wireless Sensor Networks, IEEE Transactions on Computers, 62(9): 1850-1865, September 2013. PDF

• A. Saifullah, C. Wu, P. Tiwari, Y. Xu, Y. Fu, C. Lu and Y. Chen; Near Optimal Rate Selection for Wireless Control Systems, IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS'12), April 2012. (Best Paper Nominee) PDF

• A. Saifullah, Y. Xu, C. Lu and Y. Chen; Priority Assignment for Real-time Flows in WirelessHART Networks, Euromicro Conference on Real-Time Systems (ECRTS'11), July 2011. PDF

• O. Chipara, C. Wu, C. Lu and W.G. Griswold, Interference-Aware Real-Time Flow Scheduling for Wireless Sensor Networks, Euromicro Conference on Real-Time Systems (ECRTS'11), July 2011. PDF

• A. Saifullah, Y. Xu, C. Lu and Y. Chen; End-to-End Delay Analysis for Fixed Priority Scheduling in WirelessHART Networks, IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS '11), April 2011. PDF

• A. Saifullah, Y. Xu, C. Lu, and Y. Chen; Real-time Scheduling for WirelessHART Networks; IEEE Real-Time Systems Symposium (RTSS '10), December 2010. PDF

Selected Talks

• Dependable Wireless Control through Cyber-Physical Co-Design, Keynote, International Conference on Embedded Wireless Systems and Networks (EWSN), February 2016. PDF

• Real-Time Wireless Control Networks for Cyber-Physical Systems, University College Cork, Ireland, July 2014. PDF

• Challenges in Wireless Control Networks for Cyber-Physical Systems, Panel on Networking Challenges for Cyber-Physical Systems, INFOCOM, May 2014. PDF

• Real-Time Wireless Control Networks for Cyber-Physical Systems, Keynote, International Conference on Pervasive and Embedded Computing and Communication Systems (PECCS'14), January 2014. PDF

• Real-Time Wireless Control Networks for Cyber-Physical Systems, Keynote, IEEE International Symposium on Industrial Embedded Systems (SIES'13), June 2013. PDF

• Real-Time Wireless Control Networks: Challenges and Directions, NITRD National Workshop on the New Clockwork for Time-Critical Systems, October 2012. PDF

• Real-Time Wireless Sensor Networks, Royal Institute of Technology (KTH), Stockholm, Sweden, April 2010. PDF