SYNAPSE
An interactive medium for spatiotemporal behavioral Data collection and Analysis
Author’s Note:
Architecture is morphing, as its becoming more and more data-informed. Sunlight, Radiation, wind, energy efficiency, water, and material efficiency – almost every aspect of building fitness is quantifiable.
In fact, not just architecture, but every aspect of our lives today is quantifiable… calories, miles, health, online behavior. All this data, combined with increasing power of computing in the age is algorithms and machine learning is enabling a seamless user experience.
Our built environment is no longer isolated to this experience.
On one hand, solutions to make building more intuitive and responsive to the users are being developed, and on the other, ideas like open Data – where multiple agents in a city can be analyzed together, are enabling us to understand our cities better. With this paradigm shift, we are also faced with unprecedented problems.such as personal privacy for both physical and virtual identities.
However, development of extraordinary materials, technologies and evidence-based –data-driven processes – allows us a unique opportunity to overcome these issues and develop systems which are powerful, ethical and intuitive.
As we developed our project in this context, it became clear to us that there are two main focuses that we are dealing with – Active surfaces and Adaptive spatiotemporal strategies. And that they are not only intrinsically connected to each other but together, provide a novel and powerful opportunity for architecture and urban design.
Research Topic
” How can we develop an interactive medium for spatiotemporal control and analysis, which allows an enhanced architectural performance and user experience? “
Abstract
The absence of accurate, real-time information about individual’s spatial behavior leaves designers and decision makers without a real space performance metric and may consequently lead to poor judgment and speculations. This work-in-progress research and experimentations aim to provide designers and decision makers with an information gathering tool for users’ spatial behavioral patterns towards a better evidence-based, contextual, data-driven process and behavioral research.
Using surface embedded Graphene Nanoplatelets, capacitive sensing system along with machine learning processes, this research presents an analytical and interactive medium for user’s presence sensing, position tracing, and gesture recognition.
Moreover, we present several cases in which this system acts as an actuator when paired with various active layers for applications such as localized heating and haptic feedback.
Introduction
Architecture has always been in the process of transformation, responding to the technological, social and cultural changes. In recent years, acceleration in digital media technologies has had a very significant impact on multiple facets of architecture ranging from design and modeling, communication, fabrication, and automation in buildings.
This transformation has allowed architects to approach design in a data-centric and algorithmic logic. However, means of reliable data collection are restricted to devices and sensor which are not an intrinsic part of architectural materiality.
This research aims at developing SYNAPSE, a system which can be a constitutional part of a variety of architecture interfaces like floors, walls, facade systems and even furniture. This capacitive sensing based system is adaptable to different scales, shapes as well as materials and is programmable to suit different applications and scenarios. Graphene, a carbon-based nanomaterial, is being used as a key constituent of the system, for its extreme conductive properties.
This paper first provides a detailed description of Synapse system composition followed by a description of each element that constitutes the system and various factors that affect its performance. Also included are the detail on collected data analysis and user interaction methods using the system. The paper closes with an examination of benefit synapse could provide in enhancing architectural performance and user experience.
Current state of Art
The principal application of the SYNAPSE system is a sensory interface for architectural elements. The sensing mechanism has been developed as a construction of emergent graphene technology. As a system, this material together with its application is without precedence in architecture and related industries.
Simple sensing mechanisms take a passive binary form with the sole function of triggering a signal in the presence of an object, such as those implemented in Passive Infrared sensors, light barriers, and pressure mats. In the case of capacitive sensing, recent developments have demonstrated more advanced architectural applications with the capacity for:
| multiple presence sensing modalities
| position tracing, tagging, and gesture recognition
| activation of responsive layers
| temporization data processing and accompanying Machine Learning
Current state of Art
System
The system utilizes the principle of mutual capacitive sensing, that is widely used in touchscreen technologies and can be embedded in various surfaces ranging from walls, facade, furniture, floor and even warped on a curved surface.
The sensing system can be coupled with a variety of active layers that can gain from presence sensing, and that does not interfere with the coupled capacitance of the electrodes. The active layers can vary from heating, shading with thermochromic materials, inductive charging and haptic feedback.
This system allows us to accurately determine the position of proximity of an object on the floor, at any given time. Reading position as a passive input from the user, we can determine the direction of motion of the user or any conductive object, their size, speed, and quantity. Using the same sensor, we also use it in gait analysis and potentially for user identification applications.
Using the same assembly, we are also able to program it to recognize user gestures, as active inputs, which can be used for feedback from the user or to actuate a specific action.
The design of the capacitive grid, including the size and spacing of the electrodes, has an impact on its sensing capabilities. Computational and Machine Learning methods can be employed to derive the grid design, as described later.
System Layering
Capacitive Sensing
Capacitive sensing is a proximity sensing technology. Capacitive sensors work by generating an electric field and detecting nearby objects by sensing whether this field has been disrupted. Capacitive sensors can detect anything that is conductive or that has a significantly different permittivity than air, like a human body or hand.
The superior conductive properties of graphene leverage itself effectively to this cause.
We investigated a range of inks, patterns and how they refer to the distance that can be achieved with capacitive proximity sensors. Later on, we checked the relation between the type of ink with the distance.
Graphene
Since graphene was isolated and characterized in 2004, intensive research efforts have been dedicated to pairing it’s unparalleled material properties with novel architectural applications. Recent success in the production of graphene using existing platforms of advanced manufacturing provides access for architectural-scale testing and development.
However, to fully leverage the advantages of graphene during this process, a threshold must be crossed from handling the material at the level of particle physics to that of building fabric. The development of graphene as a digital matter medium in design and fabrication systems common to architecture could progress the integration of information technologies. These have the potential to shape not only interaction between elements of the built environment, but the interaction between building elements and their inhabitants.
Some of the beneficial properties of graphene are represented in the following figure.
In nanoplatelet form,which is used for this research, graphene presents scope for development of surface-embedded patterning, with low barriers to entry in cost and open-source access.
Graphene properties
Ink
Custom-Made conductive ink is used to deposit electrodes on a given substrate. Graphene is the critical ingredient, and being hydrophobic, it needs a binder to obtain a uniform dispersion in water. As the first stage of the investigation, we got familiar with graphene inks. A research into the variety of binders and mix proportions took place.
The Ratio of the binder to graphene nanoplatelets is the decisive factor in achieving a required conductivity. We can program the mix such that it provides a high conductivity while using for electrodes and slightly lowering conductivity and increasing the resistivity to an optimal level to be used for graphene heating pads.
The final ink mixture and process of fabrication was perfected before we went into the further investigation
Substrates
The sensing matrix is supported by a substrate that forms the base layer of each system construction. Over the evolution of the design process, a series of materials were trialed for compatibility with conductive ink.
Evaluation of results eliminated a range of typical architectural materials from practical application. The remainder provided a matrix of prospective materials for further investigation.
Performance of sensing system is dependent on a substrate. The key indicator of performance was resistance measured for an applied current over a controlled surface area and linear distance. Sampled materials and significant results are provided in Figures below.
Pattern
Ink line widths and geometrical patterns had an impact on ink’s conductivity. We tested different widths using the same link and width of the electrode is directly proportional to its surface resistance. 30mm wide lines yielded resistance values that are optimal for this research.
Another test was conducted using different geometrical patterns of the same ink surface area. Different patterns had different sensing range due to their distinctive projected electrical fields. Until a detailed study of the different electrical fields is conducted, we are implementing a grid of strips as a pattern for the different prototypes. For this test, we used self-capacitance system and we have planned to use mutual capacitance system to test different patterns in the near future.
We also observed that a pattern with non-uniform width is ineffective, as the conductivity between line endpoints is governed (among other parameters) by the minimum width across the section.
Heating system
As building heating accounts for a substantial amount of energy, there lies a great potential to save energy by using dynamically controlled, localized heating (Ratti and Claudel2015).
We tested the use of Graphene Nanoplatelets as heating pads to be used in conjunction with the sensory surface
to provide localized heating. By changing the mix of graphene ink, we can achieve an ink mixture that can be rapidly heated to very high temperatures with relatively
low voltage. We also observed that as the temperature goes higher, the resistance goes lower.
Another test was conducted in which presence sensing was used to actuate different heating pads. This way, only pads with great proximity to the user are activated.
Heating Tests – Temperature
Localized Heating Tests – Programmability
Footstep Tracing
By embedding graphene electrodes in concrete tiles and using mutual capacitive sensing, the system was able to detect a footstep through 75mm thick concrete substrate through shoes. In a comparison between 45 mm tile and 70mm tiles, there was negligible difference in sensing performance.
Further, by using a high-resolution grid and analyzing the non-concurrent incoming signals, we are able to distinguish between different phases of a walk cycle. This feature is used to determine the walking direction as well as the stance phase speed.
One of the methods we use for analyzing this behavior is dynamic time warping. By representing the capacitive effects as a wave we concluded that this method is beneficial for concluding a direction and pace of pedestrians.
Capacitive sensing through concrete
Analyzing non-concurrent signals to determine walking direction.
Scaling up Strategies
- Monolithic Film
The ‘sensing’ assembly can be embedded in the floor or wall surfaces, either under existing finish or in the form of custom made ‘active tiles’.
Both methods can be manufactured on a large scale. This research proposes a system that includes two perpendicular electrodes, on either side of a PETG sheet, followed by a protective layer on both the sides. The sheets can be used in the form of a continuous roll that can be embedded on a variety of buildups.
monolithic sheet system
2. Modular Tiles
Alternatively, graphene electrodes can be miniaturized to be embedded in modular systems such as concrete or wood tiles.
Prototyping
01 Capacitive Sensor matrix :
Two main focuses of our experimentation and prototyping were, firstly to test, understand and work with a large scale capacitive sensor and secondly, to create large-scale capacitive sensor using Graphene conductive ink.
In the early stages of the research, a copper-based capacitive sensor was developed which allowed us to developed computational tools for location sensing.
This prototype was soon replaced with graphene-based sensors which not only allowed location sensing but also pressure sensing.
02 Gesture Recognition and Localised heating
Hand gestures provide natural and intuitive computer-human interface that eliminates the need for switches. Capitalizing on Mutual Capacitance, we examined the use of gestures as an input mechanism. We used an open source, cross-platform, machine learning library for real-time gesture recognition. We utilized the Gesture Recognition Toolkit[Nick Gillian] along with the Example-based Sensor Prediction system [Mellis] to train a set of 8 different hand gestures. (figure below)
We used the Dynamic Time Warping (DTW) algorithm to classify the temporal signals we receive from the sensors.DTW algorithm creates different time series templates for different gestures, then warps the real-time sensor signals to find the best match. After preparing sensor calibration samples, we calibrate the real-time incoming signal and fine tune them as required. We then pre-process incoming signal by using the Moving Average Filter module to remove some amount of high-frequency noise from our 16-dimensional signal. We then compute the first derivative of the signal using the Derivative pre-processing module. For feature extraction, we use the Time Domain Features method to get the signal amplitude with time.
A prototype was developed at the culmination of first out of two semesters devoted to this research. We are testing sensing, gesture activated control and heating properties of our system. The prototype consists of three layers, two sensing layers topped with an active localised heating layer. The substrate used is 3mm Poly(methyl methacrylate).
We trained the system for localised heating activation. When a user circles a specific area on the prototype, graphene heating pads in that area get activated. The system can be further developed to allow for automatic localised heating based on user presence in space.
03 Active Urban Tiles
Active Urban tiles are developed as a replacement for existing, non-sensing Urban tiles. These tiles could provide accurate, anonyms, large-scale spatiotemporal information that puts the citizen in the urban design feedback loop.
These tiles are developed keeping in mind high traffic movement in urban areas and required material strength.
These tiles will not only allow sensing capabilities but will also allow easy and accurate detection of cracks or deformation in urban space, leading to effective maintenance.
04 Continuous film
This large-scale walkable prototype is ideal as an undermat which can be paired with any of the existing flooring or wall materials. The design of the electrodes is such that it allows for maximum sensing area and hence a higher resolution of data. With this sample, we are able to collect real-time data on pedestrian tracking, space occupancy, and walking direction. The same system can be easily trained to recognize gestures as well.
A 0.5 mm thick flexible PETG sheet is used as a substrate and the fabrication involved stenciling, ink deposition and lamination. The connection between graphene electrodes and end cables proved too tricky and a conductive tape is used for the same.
Spatiotemporal Analysis
Data collected from the large sample is used to generate occupancy maps, which allow users and decision maker to read interpret this data with ease.
Occupancy maps in figure 43 shows two information sets :
1) Activation – the number of pedestrians passing through a specific area is denoted by the height of the mesh. The highest peak stands for the most visited space.
2) engagement – the colors denote user engagement of the space i.e. time spent in a specific space. Warm colors represent more engagement.
Activation and engagement, together give a holistic index of occupancy in a space.
Application
01 Architectural :
In indoor spaces, active sensory surfaces help to predict spatial performance through diverse behavioral data of similar spaces. Moreover, real-time data can enhance the performance and user experience. By having a clear and accurate understanding of the spatial user behavior, space can consequently be adapted according to the specific needs of the inhabitant.
Dynamic space configuration can be achieved as a result of a behavioral analysis. By understanding how people use spaces in an unbiased way, evidence-based conclusions can be drawn and spaces can adapt better to the different user’s spatial requirements.
Some of the spaces where such system can be of great bene t to are public and civic buildings as well as working areas such as hospitals, exhibition spaces, theatres, educational institutions as well as retail venues.
02 Dynamic Space Configuration :
We have developed a spatiotemporal localization sensor, with which we can map occupancy and behavioral patterns of inhabitants in a space. The occupancy pattern, when overlapped with other spatial information such as Sunlight, Temperature and visual connectivity, can give a cohesive and holistic spatial performance overview. Based on this information many spatial issues can be revealed – like pedestrian bottlenecks, spatial congestion, ignored/ disdained spaces as well as accident-prone areas.
One of the solutions to some of these problems is ‘dynamic space configuration’; where the spatial arrangement of any item in a space is computationally decided, based on the aforementioned parameters. For this computational exercise, the factors taken into considerations are sunlight, visual connections, and frequent pedestrian paths.
03 Superilla Case study :
The Superblock of Poblenou is a pilot project where pedestrian and vehicular spaces are physically redefined to included Public space, play zones, sports zones and urban renewal locations.
The goal of this onsite testing was to investigate whether these rede ned spaces are being used as expected by the urban designers. This on-site testing of the prototype was performed at one of the streets where the former vehicular area has been reassigned to pedestrians.
The findings showcase that the policy change has not resolved in the desired outcome as pedestrians still prefer to walk on the side pavement rather than in the middle.
04 Urban Vision
Synapse could provide accurate, anonyms, large-scale spatiotemporal information that puts the citizen in the urban design feedback loop. By recording traces of footsteps, the system can provide dynamic occupancy maps, trend prediction, and historical pedestrian information.
Such information provides relevant authorities in urban areas with accurate, real-time metrics of user engagement, public spaces performance, put strategies for site and crowd management, perform risk evaluation, predict maintenance works, and deploy sufficient emergency services.
By analyzing public pedestrian traffic data, Synapse’s density maps could illustrate multi-layer information:
| The occupancy rate at different time periods
| Pedestrian traffic bottlenecks
| Crowd gathering and dispersion routes
| Pedestrian type: group or individual
| Age group analysis: based on pace & step size