Centre for Tactile Internet with Human-in-the-Loop

As the global population grows, and urbanization becomes more prevalent, cities often struggle to provide convenient, secure, and sustainable lifestyles due to the lack of necessary smart technologies. Fortunately, the Internet of Things (IoT) has emerged as a solution to this challenge by connecting physical objects using electronics, sensors, software, and communication networks. This has transformed smart city infrastructures, introducing various technologies that enhance sustainability, productivity, and comfort for urban dwellers. By leveraging Artificial Intelligence (AI) to analyze the vast amount of IoT data available, new opportunities are emerging to design and manage futuristic smart cities. In this review article, we provide an overview of smart cities, defining their characteristics and exploring the architecture of IoT. A detailed analysis of various wireless communication technologies employed in smart city applications is presented, with extensive research conducted to determine the most appropriate communication technologies for specific use cases. The article also sheds light on different AI algorithms and their suitability for smart city applications. Furthermore, the integration of IoT and AI in smart city scenarios is discussed, emphasizing the potential contributions of 5G networks coupled with AI in advancing modern urban environments. This article contributes to the existing literature by highlighting the tremendous opportunities presented by integrating IoT and AI, paving the way for the development of smart cities that significantly enhance the quality of life for urban dwellers while promoting sustainability and productivity. By exploring the potential of IoT, AI, and their integration, this review article provides valuable insights into the future of smart cities, demonstrating how these technologies can positively impact urban environments and the well-being of their inhabitants.

OBJECTIVE: To provide an overview of ML models and data streams utilized for automated surgical phase recognition. BACKGROUND: Phase recognition identifies different steps and phases of an operation. ML is an evolving technology that allows analysis and interpretation of huge data sets. Automation of phase recognition based on data inputs is essential for optimization of workflow, surgical training, intraoperative assistance, patient safety, and efficiency. METHODS: A systematic review was performed according to the Cochrane recommendations and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. PubMed, Web of Science, IEEExplore, GoogleScholar, and CiteSeerX were searched. Literature describing phase recognition based on ML models and the capture of intraoperative signals during general surgery procedures was included. RESULTS: A total of 2254 titles/abstracts were screened, and 35 full-texts were included. Most commonly used ML models were Hidden Markov Models and Artificial Neural Networks with a trend towards higher complexity over time. Most frequently used data types were feature learning from surgical videos and manual annotation of instrument use. Laparoscopic cholecystectomy was used most commonly, often achieving accuracy rates over 90%, though there was no consistent standardization of defined phases. CONCLUSIONS: ML for surgical phase recognition can be performed with high accuracy, depending on the model, data type, and complexity of surgery. Different intraoperative data inputs such as video and instrument type can successfully be used. Most ML models still require significant amounts of manual expert annotations for training. The ML models may drive surgical workflow towards standardization, efficiency, and objectiveness to improve patient outcome in the future. REGISTRATION PROSPERO: CRD42018108907.

Multi-access edge computing (MEC) has recently been proposed to aid mobile end devices in providing compute- and data-intensive services with low latency. Growing service demands by the end devices may overwhelm MEC installations, while cost constraints limit the increases of the installed MEC computing and data storage capacities. At the same time, the ever increasing computation capabilities and storage capacities of mobile end devices are valuable resources that can be utilized to enhance the MEC. This article comprehensively surveys the topic area of device-enhanced MEC, i.e., mechanisms that jointly utilize the resources of the community of end devices and the installed MEC to provide services to end devices. We classify the device-enhanced MEC mechanisms into mechanisms for computation offloading and mechanisms for caching. We further subclassify the offloading and caching mechanisms according to the targeted performance goals, which include throughput maximization, latency minimization, energy conservation, utility maximization, and enhanced security. We identify the main limitations of the existing device-enhanced MEC mechanisms and outline future research directions.

Telesurgical robotics, as a technical solution for robot-assisted minimally invasive surgery (RAMIS), has become the first domain within medicosurgical robotics that achieved a true global clinical adoption. Its relative success (still at a low single-digit percentile total market penetration) roots in the particular human-in-the-loop control, in which the trained surgeon is always kept responsible for the clinical outcome achieved by the robot-actuated invasive tools. Nowadays, this paradigm is challenged by the need for improved surgical performance, traceability, and safety reaching beyond the human capabilities. Partially due to the technical complexity and the financial burden, the adoption of telesurgical robotics has not reached its full potential, by far. Apart from the absolutely market-dominating da Vinci surgical system, there are already 60+ emerging RAMIS robot types, out of which 15 have already achieved some form of regulatory clearance. This article aims to connect the technological advancement with the principles of commercialization, particularly looking at engineering components that are under development and have the potential to bring significant advantages to the clinical practice. Current RAMIS robots often do not exceed the functionalities deriving from their mechatronics, due to the lack of data-driven assistance and smart human–machine collaboration. Computer assistance is gradually gaining more significance within emerging RAMIS systems. Enhanced manipulation capabilities, refined sensors, advanced vision, task-level automation, smart safety features, and data integration mark together the inception of a new era in telesurgical robotics, infiltrated by machine learning (ML) and artificial intelligence (AI) solutions. Observing other domains, it is definite that a key requirement of a robust AI is the good quality data, derived from proper data acquisition and sharing to allow building solutions in real time based on ML. Emerging RAMIS technologies are reviewed both in a historical and a future perspective.

A 5G campus network is a 5G network for the users affiliated with the campus organization, e.g., an industrial campus, covering a prescribed geographical area. A 5G campus network can operate as a so-called 5G non-stand-alone (NSA) network (which requires 4G Long-Term Evolution (LTE) spectrum access) or as a 5G standalone (SA) network (without 4G LTE spectrum access). 5G campus networks are envisioned to enable new use cases, which require cyclic delay-sensitive industrial communication, such as robot control. We design a rigorous testbed for measuring the one-way packet delays between a 5G end device via a radio access network (RAN) to a packet core with sub-microsecond precision as well as for measuring the packet core delay with nanosecond precision. With our testbed design, we conduct detailed measurements of the one-way download (downstream, i.e., core to end device) as well as one-way upload (upstream, i.e., end device to core) packet delays and losses for both 5G standalone (SA) and 5G non-standalone (NSA) hardware and network operation.We also measure the corresponding 5G SA and 5G NSA packet core processing delays for download and upload. We find that typically 95% of the SA download packet delays are in the range from 4–10 ms, indicating a fairly wide spread of the packet delays. Also, existing packet core implementations regularly incur packet processing latencies up to 0.4 ms, with outliers above one millisecond. Our measurement results inform the further development and refinement of 5G SA and 5G NSA campus networks for industrial use cases. We make the measurement data traces publicly available as the IEEE DataPort 5G Campus Networks: Measurement Traces dataset (DOI 10.21227/xe3c-e968).

The initial vision of cellular communications was to deliver ubiquitous voice communications to anyone anywhere. In a simplified view, 1G delivered voice services for business customers, and only 2G for consumers. Next, this also initiated the appetite for cellular data, for which 3G was designed. However, Blackberry delivered business smartphones, and 4G made smartphones a consumer device. The promise of 5G is to start the Tactile Internet, to control real and virtual objects in real-time via cellular. However, the hype around 5G is, again, focusing on business customers, in particular in the context of campus networks. Consequently, 6G must provide an infrastructure to enable remote-controlled mobile robotic solutions for everyone—the Personal Tactile Internet. Which role can information and communication theory play in this context, and what are the big challenges ahead?

In this work we propose the combination of large interactive displays with personal head-mounted Augmented Reality (AR) for information visualization to facilitate data exploration and analysis. Even though large displays provide more display space, they are challenging with regard to perception, effective multi-user support, and managing data density and complexity. To address these issues and illustrate our proposed setup, we contribute an extensive design space comprising first, the spatial alignment of display, visualizations, and objects in AR space. Next, we discuss which parts of a visualization can be augmented. Finally, we analyze how AR can be used to display personal views in order to show additional information and to minimize the mutual disturbance of data analysts. Based on this conceptual foundation, we present a number of exemplary techniques for extending visualizations with AR and discuss their relation to our design space. We further describe how these techniques address typical visualization problems that we have identified during our literature research. To examine our concepts, we introduce a generic AR visualization framework as well as a prototype implementing several example techniques. In order to demonstrate their potential, we further present a use case walkthrough in which we analyze a movie data set. From these experiences, we conclude that the contributed techniques can be useful in exploring and understanding multivariate data. We are convinced that the extension of large displays with AR for information visualization has a great potential for data analysis and sense-making.

A variety of psychological and physical phenomena elicit variations in the diameter of pupil of the eye. Changes in pupil size are mediated by the relative activation of the sphincter pupillae muscle (decrease pupil diameter) and the dilator pupillae muscle (increase pupil diameter), innervated by the parasympathetic and sympathetic branches, respectively, of the autonomic nervous system. The current guidelines are intended to inform and guide psychophysiological research involving pupil measurement by (1) summarizing important aspects concerning the physiology of the pupil, (2) providing methodological and data-analytic guidelines and recommendations, and (3) briefly reviewing psychological phenomena that modulate pupillary reactivity. Because of the increased ease and tractability of pupil measurement, the goal of these guidelines is to promote accurate recording, analysis, and reporting of pupillary data in psychophysiological research.

Adaptive body representation requires the continuous integration of multisensory inputs within a flexible 'body model' in the brain. The present review evaluates the idea that this flexibility is augmented by the contextual modulation of sensory processing 'top-down'; which can be described as precision control within predictive coding formulations of Bayesian inference. Specifically, I focus on the proposal that an attenuation of proprioception may facilitate the integration of conflicting visual and proprioceptive bodily cues. Firstly, I review empirical work suggesting that the processing of visual vs proprioceptive body position information can be contextualised 'top-down'; for instance, by adopting specific attentional task sets. Building up on this, I review research showing a similar contextualisation of visual vs proprioceptive information processing in the rubber hand illusion and in visuomotor adaptation. Together, the reviewed literature suggests that proprioception, despite its indisputable importance for body perception and action control, can be attenuated top-down (through precision control) to facilitate the contextual adaptation of the brain's body model to novel visual feedback.

In this paper, we present MIRIA, a Mixed Reality Interaction Analysis toolkit designed to support the in-situ visual analysis of user interaction in mixed reality and multi-display environments. So far, there are few options to effectively explore and analyze interaction patterns in such novel computing systems. With MIRIA, we address this gap by supporting the analysis of user movement, spatial interaction, and event data by multiple, co-located users directly in the original environment. Based on our own experiences and an analysis of the typical data, tasks, and visualizations used in existing approaches, we identify requirements for our system. We report on the design and prototypical implementation of MIRIA, which is informed by these requirements and offers various visualizations such as 3D movement trajectories, position heatmaps, and scatterplots. To demonstrate the value of MIRIA for real-world analysis tasks, we conducted expert feedback sessions using several use cases with authentic study data.

We present Marvis, a conceptual framework that combines mobile devices and head-mounted Augmented Reality (AR) for visual data analysis. We propose novel concepts and techniques addressing visualization-specific challenges. By showing additional 2D and 3D information around and above displays, we extend their limited screen space. AR views between displays as well as linking and brushing are also supported, making relationships between separated visualizations plausible. We introduce the design process and rationale for our techniques. To validate Marvis’ concepts and show their versatility and widespread applicability, we describe six implemented example use cases. Finally, we discuss insights from expert hands-on reviews. As a result, we contribute to a better understanding of how the combination of one or more mobile devices with AR can benefit visual data analysis. By exploring this new type of visualization environment, we hope to provide a foundation and inspiration for future mobile data visualizations.

The release of chemicals and microorganisms from various sources, such as industry, agriculture, animal farming, wastewater treatment plants, and flooding, into water systems have caused water pollution in several parts of our world, endangering aquatic ecosystems and individual health. World Health Organization (WHO) has introduced strict standards for the maximum concentration limits for nutrients and chemicals in drinking water, surface water, and groundwater. It is crucial to have rapid, sensitive, and reliable analytical detection systems to monitor the pollution level regularly and meet the standard limit. Electrochemical biosensors are advantageous analytical devices or tools that convert a bio-signal by biorecognition elements into a significant electrical response. Thanks to the micro/nano fabrication techniques, electrochemical biosensors for sensitive, continuous, and real-time detection have attracted increasing attention among researchers and users worldwide. These devices take advantage of easy operation, portability, and rapid response. They can also be miniaturized, have a long-life span and a quick response time, and possess high sensitivity and selectivity and can be considered as portable biosensing assays. They are of special importance due to their great advantages such as affordability, simplicity, portability, and ability to detect at on-site. This review paper is concerned with the basic concepts of electrochemical biosensors and their applications in various water quality monitoring, such as inorganic chemicals, nutrients, microorganisms' pollution, and organic pollutants, especially for developing real-time/online detection systems. The basic concepts of electrochemical biosensors, different surface modification techniques, bio-recognition elements (BRE), detection methods, and specific real-time water quality monitoring applications are reviewed thoroughly in this article.

PURPOSE: Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill. METHODS: To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment. RESULTS: F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team). CONCLUSION: Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.

This paper presents a substantial review of the deployment of wearable sensors for healthcare applications. Wearable sensors hold a pivotal position in the microelectronics industry due to their role in monitoring physiological movements and signals. Sensors designed and developed using a wide range of fabrication techniques have been integrated with communication modules for transceiving signals. This paper highlights the entire chronology of wearable sensors in the biomedical sector, starting from their fabrication in a controlled environment to their integration with signal-conditioning circuits for application purposes. It also highlights sensing products that are currently available on the market for a comparative study of their performances. The conjugation of the sensing prototypes with the Internet of Things (IoT) for forming fully functioning sensorized systems is also shown here. Finally, some of the challenges existing within the current wearable systems are shown, along with possible remedies.

Flow routing can achieve fine-grained network performance optimizations by routing distinct packet traffic flows over different network paths. While the centralized control of Software-Defined Networking (SDN) provides a control framework for implementing centralized network optimizations, e.g., optimized flow routing, the implementation of flow routing that is adaptive to varying traffic loads requires complex models. The goal of this study is to pursue a model-free approach that is based on reinforcement learning. We design and evaluate QR-SDN, a classical tabular reinforcement learning approach that directly represents the routing paths of individual flows in its state-action space. Due to the direct representation of flow routes in the QR-SDN state-action space, QR-SDN is the first reinforcement learning SDN routing approach to enable multiple routing paths between a given source (ingress) switch-destination (egress) switch pair while preserving the flow integrity. That is, in QR-SDN, packets of a given flow take the same routing path, while different flows with the same source-destination switch pair may take different routes (in contrast, the recent DRL-TE approach splits a given flow on a per-packet basis incurring high complexity and out-of-order packets). We implemented QR-SDN in a Software-Defined Network (SDN) emulation testbed. Our evaluations demonstrate that the flow-preserving multi-path routing of QR-SDN achieves substantially lower flow latencies than prior routing approaches that determine only a single source-destination route. A limitation of QR-SDN is that the state-action space grows exponentially with the number of network nodes. Addressing the scalability of direct flow routing, e.g., through routing only high-rate flows, is an important direction for future research. The QR-SDN code is made publicly available to support this future research.

The use of digital technology is increasing rapidly across surgical specialities, yet there is no consensus for the term 'digital surgery'. This is critical as digital health technologies present technical, governance, and legal challenges which are unique to the surgeon and surgical patient. We aim to define the term digital surgery and the ethical issues surrounding its clinical application, and to identify barriers and research goals for future practice. 38 international experts, across the fields of surgery, AI, industry, law, ethics and policy, participated in a four-round Delphi exercise. Issues were generated by an expert panel and public panel through a scoping questionnaire around key themes identified from the literature and voted upon in two subsequent questionnaire rounds. Consensus was defined if >70% of the panel deemed the statement important and <30% unimportant. A final online meeting was held to discuss consensus statements. The definition of digital surgery as the use of technology for the enhancement of preoperative planning, surgical performance, therapeutic support, or training, to improve outcomes and reduce harm achieved 100% consensus agreement. We highlight key ethical issues concerning data, privacy, confidentiality and public trust, consent, law, litigation and liability, and commercial partnerships within digital surgery and identify barriers and research goals for future practice. Developers and users of digital surgery must not only have an awareness of the ethical issues surrounding digital applications in healthcare, but also the ethical considerations unique to digital surgery. Future research into these issues must involve all digital surgery stakeholders including patients.

BACKGROUND: Artificial intelligence (AI) has recently achieved considerable success in different domains including medical applications. Although current advances are expected to impact surgery, up until now AI has not been able to leverage its full potential due to several challenges that are specific to that field. SUMMARY: This review summarizes data-driven methods and technologies needed as a prerequisite for different AI-based assistance functions in the operating room. Potential effects of AI usage in surgery will be highlighted, concluding with ongoing challenges to enabling AI for surgery. KEY MESSAGES: AI-assisted surgery will enable data-driven decision-making via decision support systems and cognitive robotic assistance. The use of AI for workflow analysis will help provide appropriate assistance in the right context. The requirements for such assistance must be defined by surgeons in close cooperation with computer scientists and engineers. Once the existing challenges will have been solved, AI assistance has the potential to improve patient care by supporting the surgeon without replacing him or her.

Future ofces are likely reshaped by Augmented Reality (AR) extending the display space while maintaining awareness of surroundings, and thus promise to support collaborative tasks such as brainstorming or sensemaking. However, it is unclear how physical surroundings and co-located collaboration infuence the spatial organization of virtual content for sensemaking. Therefore, we conducted a study (N=28) to investigate the efect of ofce environments and work styles during a document classifcation task using AR with regard to content placement, layout strategies, and sensemaking workfows. Results show that participants require furniture, especially tables and whiteboards, to assist sensemaking and collaboration regardless of room settings, while generous CHI '22, April 29-May 5, 2022, New Orleans, LA, USA free spaces (e.g., walls) are likely used when available. Moreover, collaborating participants tend to use furniture despite personal layout preferences. We identifed diferent placement and layout strategies, as well as the transitions in-between. Finally, we propose design implications for future immersive sensemaking applications and beyond.

The first release of 5G technology is being rolled out worldwide. In parallel, 3GPP is constantly adding new features to upcoming releases covering well-known use cases. This raises the questions i.) when will 6G be introduced?, ii.) how can 6G be motivated for the stakeholders, and iii.) what are the 6G use cases? In this work, we present the perspective of these stakeholders, namely the network operators, manufacturers, and verticals, identifying potential 5G shortcomings and the remaining 6G solution space. We will highlight the Metaverse as the enabler for 6G addressing omnipresent daily challenges and the upcoming energy problem.

This paper considers the phenomenology of depersonalisation disorder, in relation to predictive processing and its associated pathophysiology. To do this, we first establish a few mechanistic tenets of predictive processing that are necessary to talk about phenomenal transparency, mental action, and self as subject. We briefly review the important role of 'predicting precision' and how this affords mental action and the loss of phenomenal transparency. We then turn to sensory attenuation and the phenomenal consequences of (pathophysiological) failures to attenuate or modulate sensory precision. We then consider this failure in the context of depersonalisation disorder. The key idea here is that depersonalisation disorder reflects the remarkable capacity to explain perceptual engagement with the world via the hypothesis that "I am an embodied perceiver, but I am not in control of my perception". We suggest that individuals with depersonalisation may believe that 'another agent' is controlling their thoughts, perceptions or actions, while maintaining full insight that the 'other agent' is 'me' (the self). Finally, we rehearse the predictions of this formal analysis, with a special focus on the psychophysical and physiological abnormalities that may underwrite the phenomenology of depersonalisation.