Collaboratively Eliciting Gestures for Geospatial Data Exploration on an MSE with Tangibles and Styluses

Karen Penaranda Valdivia (Corresponding author: Karen Penaranda Valdivia.) This work involved human subjects in its research. Approval of all ethical and experimental procedures and protocols was granted by the Toronto Metropolitan University (TMU) Research Ethics Board under Application No. 2025-138 and performed in line with TCPS 2 (2022). Karen Penaranda Valdivia, Nujaimah Ahmed, Aswah Butt, Mashrufa Orchi, Dr. Roozbeh Manshaei, Dr. Emmanuel Kyeremeh, and Dr. Ali Mazalek are with the Synaesthetic Media Lab, Toronto Metropolitan University, Toronto, ON M5G 1P5, CA (e-mail: karen.penaranda@torontomu.ca). Sarah Hoyos-Hoyos and Dr. Robert McLeman are with the Department of Geography and Environmental Studies, Wilfrid Laurier University, Waterloo, ON N2L 3C5, Canada. Dr. Gabby Resch is with the Faculty of Business and Information Technology, Ontario Tech University, Oshawa, ON L1G 0C5, Canada. Dr. Jamy Li is with the School of Computing, Engineering, & The Built Environment, Edinburgh Napier University, Edinburgh EH11 4BN, UK. The authors were affiliated with the above institutions at the time the work was performed. This work has been supported by the Social Sciences and Humanities Research Council, the Canada First Research Excellence Fund, the Canada Research Chairs program, the Canada Foundation for Innovation, and the Ontario Ministry of Research and Innovation.

Abstract

Large tabletop displays and multi-surface environments offer potential for enhancing visual data exploration and collaborative work with geospatial datasets. These systems typically rely on multi-touch interactions, which can pose challenges when the multi-touch sensors misrepresent transitory movements as control inputs, leading to interruptions. Active tangibles and styluses offer an alternative to multi-touch interactions in MSEs, and have shown the potential to facilitate sense-making around large datasets. However, further research is needed to better understand how these modalities can be effectively leveraged for interacting with geospatial data visualizations. To address this, a gesture elicitation study was conducted in which users suggested interactions for 16 geospatial data visualization tasks, presented as a realistic collaborative workflow co-designed with geography and migration researchers. The study produced a taxonomy of user-defined gestures using tangibles and styluses for engaging with geospatial data, along with a thematic analysis of users’ experiences with visualization tasks and interaction techniques.

I Introduction

Multi-surface environments (MSEs) and large displays offer considerable benefits for collaborative work in interdisciplinary domains [3, 16, 24, 37, 57]. MSEs with Tangible User Interfaces (TUIs) have been shown to enhance users’ interactions with data visualizations by facilitating the organization of spatial information [4, 36, 51]. Accordingly, there is a growing interest in the benefits of MSEs for visualizing and interacting with geospatial data and map-based tasks [15, 42, 52, 55, 57]. However, multi-touch has often been used as the primary input modality [37, 42, 52], despite prior studies showing that TUIs can outperform it in key tasks for handling geospatial data, such as manipulation and acquisition [2, 58], sorting [2], grouping [2], and layout manipulation [2].

Multi-touch interfaces also have two-dimensional and technical challenges that can limit users’ interactions and interrupt their workflow. TUIs provide a distinct advantage for spatial interactions by offering physical directness that is not bound to two-dimensional surfaces. While multi-touch interfaces allow users to visualize and interact with a large number of objects on a planar surface [50], TUIs support enhanced collaboration, as users can more easily observe and respond to a wider range of physical gestures enabled by the tangible dimension [50, 7]. Second, multi-touch interfaces may have challenges in supporting users’ learning and discovery with data-intensive areas due to finger-size, occlusion, and real estate occupied by Windows, Icons, Menus, and Pointer (WIMP) style controls [13, 63, 64, 60]. For example, screen real-estate management could be an issue due to finger-size and occlusion creating barriers for touching small targets [13, 63, 64, 60]. Additionally, improperly sized WIMP-controls could contribute to cognitive dissonance [13, 60]. Lastly, technical challenges, such as misrepresenting accidental touches or transient movements as inputs, can lead to workflow interruptions.

In geospatial contexts, TUIs have primarily been applied to urban planning and collaborative data analysis, often incorporating displays (e.g. [15, 5, 34, 35, 36, 27, 69, 1, 68]) , VR or AR systems (e.g., [15, 35]), multi-touch (e.g., [5, 27, 1]), mobile devices (e.g.,[5]), or malleable materials such as clay or sand (e.g., [57, 44, 55]). However, a gap remains in understanding how tangibles and styluses can be used collaboratively in a map-based context [35], especially since the latter have been shown to support engaging and intuitive user interactions [4, 15, 27, 33, 60] that could leverage spatial data exploration. Tangibles have shown promise to support handling of large geospatial datasets [34], while supporting distributed and embodied cognition [23]. The precision of styluses has also been found to mitigate many limitations of not having multi-touch [65], and users can benefit from using styluses for annotations [67]. As a result, the value proposition of integrating tangibles and styluses could be in their potential to support data-intensive visualizations by leveraging digital inputs and visual feedback through precise hand movements facilitated by the stylus. Further work is needed to understand how tangibles and styluses can be used in a geospatial data exploration context.

This study aims to investigate how to design interactions for a tabletop MSE that supports collaborative geospatial data exploration using active tangibles and a stylus. An elicitation study was conducted with participants of varying geospatial expertise, who collaboratively co-produced cartographic-perspective and system-perspective tasks while exploring map-based data. Cartographic-perspective tasks are defined as interactions that involve manipulating data within the map itself, while system-perspective tasks refer to interactions involving controls, commands, or menus [52]. The elicitation study method was chosen as it has proven effective for uncovering user-defined gestures and interaction techniques that might not have been readily apparent to designers [40, 31]. This study and its referents, which are short descriptions of system outcomes that define the expected result of an interaction, were co-designed with migration, geography, and Human Computer-Interaction (HCI) collaborators to reflect realistic workflows and key geospatial visualization interactions for interdisciplinary migration research teams (e.g., [52]).

The contextual need for this work arose from the migration research collaborators’ desire for a tool that could allow researchers with varied levels of data literacy and quantitative expertise to collaboratively explore and find patterns and make sense of complex geospatial data. Many of the migration researcher collaborators expressed that their interdisciplinary teams often explored data individually, sharing only completed findings rather than engaging in shared research methodologies. A map-based TUI could help by enabling collaborative map visualizations, which can promote engagement of diverse stakeholders in transparent decision-making [70].

The research questions addressed by this study are:
RQ1. How should tangibles and styluses be employed to manipulate geospatial data visualization in an MSE?
RQ2. How should tangibles and styluses be used in a collaborative geospatial data visualization workflow?
RQ3. How does participant expertise influence gesture and proposal generation?

To address these questions, 36 participants with varying levels of geospatial data modelling expertise were paired into novice, expert, and mixed pairs and asked to propose interactions for 16 tasks using one or more tangibles, a stylus, or both. It is hoped that the results from this study will inform the future development of a map-based TUI by identifying interaction patterns in how users employ tangibles and styluses to manipulate geospatial data, while enhancing the understanding of their collaborative elicitation process.

Across all expertise levels, participants preferred uni-modal interactions, which aligned with previous findings [31]. The single tangible was the most popular modality and was often used for data exploration and direct manipulation of the mapped layers. The stylus was second and often used for annotations, drawings, or freehand selections. Expanding beyond previous works [31, 35, 60, 34], it was found that multi-modal inputs (tangible and stylus) complemented each other for compound tasks across tabletop and wall displays; users often confirmed or activated free-hand selections made by the stylus with a tangible. Mixed-expertise pairs produced more matched and simpler proposals than novice or expert pairs, suggesting that cross-expertise dialogue may have influenced creativity and feasibility. This work contributes to design knowledge of user preferences for uni-modal, bi-manual, and multi-modal tangible interactions in collaborative geospatial data exploration, presents consensus analysis and a gesture set of 166 unique interactions, and highlights cross-expertise collaborative interaction patterns.

II Related Work

This section reviews (1) geospatial tangible user interfaces, (2) collaborative exploration of large datasets with tangibles and MSEs, (3) the elicitation study methodology, and (4) the potential effect of participants’ varying expertise.

Refer to caption — Figure 1: Map layers representing net migration, precipitation, gold mining, and conflict in Ghana. The front layers (precipitation, gold mining, and conflict) are overlaid on the background net migration layer.

II-A Geospatial Tangible User Interfaces

Advances in GIS have introduced a wide range of features for acquiring, processing, and sharing geospatial information [18]. However, this complexity increases the level of mental effort required for learning and retention, which can detract from knowledge construction [35]. As the field of geographical data visualization evolves from a specialist domain to one involving interdisciplinary actors [22, 9], there is a growing need for geovisualizations that are comprehensible, easy-to-use, and that mitigate cognitive overload [22, 9].

Maquil et al. [35, 34] introduced a bottom-up approach through geospatial tangible user interfaces (GTUI), referring to systems that use physical objects to manipulate large-scale digital maps and geospatial data projected on a tabletop screen.¹¹1More general work, such as [60], exploring the design of space gestural interaction with active tokens through user-defined actions, and [33], which examines at tangibles for genome data, are also relevant; though not reviewed in detail here based on a focus on geospatial data and combined tangible-stylus exploration. Their work showed that GTUIs can support face-to-face collaboration and communication, and that playful, motivating tangibles helped multi-expertise users better organize and understand spatial data. For large datasets, tangibles facilitate filtering data [53] and adjusting abstraction levels [14, 53], both of which are essential for promoting comprehensibility. Styluses can further enhance GTUIs by providing more precise input than fingers, improving the intuitiveness and fluidity of interactions in TUIs [29]. Their annotation capabilities also help compensate for the absence of multi-touch [65] and some advanced GIS features in GTUIs [34].

GTUIs have predominantly been used for urban logistics, crisis management, and traffic simulations [35], leaving a gap in understanding how they could support migration studies, another highly interdisciplinary field. At the time of writing this paper, the authors found no prior work that explicitly studied how GTUIs could be incorporated into the context of exploring map-based migration data with both tangibles and styluses.

TABLE I: Referent and Task Descriptions

#	Referents	Task Types	Category of Task
1	Navigate map and select Ghana.	Maneuver/ [8, 54], Select [8, 12, 38, 43]	Cartography
2	Access and display net migration layer.	Accessing extra information [47], Share [21]	Cartography
3	Change year of data layer.	Change (range) [8, 19]	Cartography
4	Add sub-driver layer.	Add (objects) [8]	System-Perspective
5	Identify patterns by annotating.	Annotate [8, 19, 21]	Cartography
6	Save visualization.	Save [8, 32, 47]	System-Perspective
7	Move map from table to wall.	Move[8, 46]	System-Perspective
8	Duplicate map on the wall.	Copy [8, 46]	System-Perspective
9	Move map from wall to table.	Move [8, 46]	System-Perspective
10	Change driver layer.	Change (range) [8, 19]	Cartography
11	Create highlight on map.	Highlight [8, 21, 46]	Cartography
12	Brush and link highlight to two maps on wall.	Brushing and linking [47], Highlight [8, 21, 46]	Cartography
13	Merge maps together.	Merge [8, 19]	System-Perspective
14	Split.	Split [8, 19]	System-Perspective
15	Remove, brush, and link.	Delete (objects, sets, graphical objects) [8], Brushing and Linking [47]	System-Perspective
16	Delete.	Delete [8, 47]	System-Perspective

Prior research demonstrates the versatility of GTUIs and TUIs across geospatial domains, primarily with non-stylus tangibles. For example, Mitigation in Urban areas: Solutions for Innovative Cities (MUSIC) by Maquil et al. [34], was a GTUI for urban planning using a tabletop screen with wooden tangible blocks. Venice Unfolding by Heidmann, Condotta, and Duval [41] was a GTUI for architectural exploration that used a tabletop screen with polyhedron-shaped tangibles. Smart City Logistics by Guerlain, Cortina, and Renault [20] developed a GTUI for exploration of commercial and energy management needs that used a tabletop screen and tangible tokens. TanGeoMS by Tateosian et al. [57] was a GTUI for terrain analysis and land management that used map projections onto sand and building-block tangibles. Yuan et al. [69] explored industrial and environmental planning with a TUI on a tabletop using IoT blocks, where the former supported dynamic saving of data. Tangible Globes by Satriadi et al. [49] was a TUI for exploring the tangible-virtual interplay of geospatial data exploration that used AR and tangibles in the form of small globes. TangibleNet by Takahira et al. [56] was a TUI for linking map-based nodes on a vertical screen that used projected spatial data and magnet tangibles.

Despite this breadth, most existing work does not use migration data or explore how a tangible dial and stylus can be used, which may be different than other data or interaction tools. In addition, many of these past works focus on a single tabletop display and do not explore how tangibles can be used with multiple screens, such as combined tabletop and wall displays (e.g., [35, 34, 41, 57]).

II-B Collaborative Exploration of Large Datasets with Tangibles and MSEs

Incorporating MSEs, such as wall and tabletop displays, for big data exploration supports both individual and collaborative visual analytics, promotes equity in physical interactions [25, 1], and enables co-located problem-solving and discussion [26, 1]. The conceptual and trans-disciplinary clarity offered by spatial information can also serve as a powerful enabler in addressing societal challenges [30].

Extending GTUIs beyond a single tabletop screen to include vertical wall displays (e.g. [36, 34, 35, 1, 28]), offers additional collaborative space interaction space [31], allows users to analyze and perceive more information at once [45], and mitigates cognitive overload by helping users to create mental maps with lesser need of virtual navigation [48]. León et al. [31] analyzed how participants explored geospatial data using an interactive wall display, including styluses but not tangible pucks, and found that it facilitated exploration. Moreover, a review of 28 papers on multi-surface interactions specifically with geospatial data found that the most used interaction devices were handheld devices (e.g., phones, tablets), computers, and wall displays [52]. These papers suggest the importance of wall displays in geospatial data exploration, despite their exclusion from some past works.

II-C Elicitation Studies

Elicitation studies are an interaction design methodology focused on identifying user-defined interactions [66]. In these studies, participants are presented with system outcomes, known as referents, and then asked to propose gestures, known as symbols, that would trigger those outcomes [66, 31]. Researchers group these symbols into unique sets of gestures, called signs, and derive a consensus set, capturing the most agreed-upon interaction for each referent [66, 31]. The elicitation study methodology was appropriate for this study since GTUIs employing tangibles and stylus-based interactions on MSEs remain largely under-explored, and the methodology enables the discovery of users’ interaction techniques, tendencies, and preferences that might not have been readily apparent in the initial design phase [40].

The elicitation study methodology is also suitable for uncovering consensus in users’ proposed multi-modal interactions [31], such as with this study’s combined use of tangible(s) + stylus. Some notable examples of elicitation studies relevant to the present work include Valdes et al. [60] and León et al. [31]. Valdes et al. [60] had 19 users do a query-building task for big data (i.e., genomic application) on a large display (i.e., tabletop screen) with tangible and gestural language inputs. They identified a user-generated gesture vocabulary of 21 distinct gestures. León et al. [31] had 20 users do an interaction elicitation study in pairs to explore how users used touch, speech, pen, and mid-air gesture manipulations while exploring data collaboratively on a large display (i.e., vertical screen). They identified 1015 proposals, with participants often favouring touch and speech modalities.

Elicitation studies can also probe how expertise shapes interaction proposals, which is important in domains like migration research, where familiarity with geospatial modelling varies. While several elicitation studies have recruited participants based on expertise [40, 31] or prior interface experience [31], the authors do not know of any past elicitation studies that explicitly compared participants’ expertise and their performance. This gap may be surprising considering the importance of literature on expertise in design [10].

III Study Design

This section covers the apparatus and input modalities, the participants, the referents, and the study procedure.

III-A Apparatus and Input Modalities

The study took place in a university lab equipped with 55-inch MultiTaction Cell screens (16:9 aspect ratio). The wall configuration consisted of nine vertical screens, arranged as five extended displays, while the tabletop setup included three screens arranged as two extended displays. Two chairs were placed in front of the tabletop, providing users the flexibility to sit, stand, or walk around the tabletop and vertical screens during the study. To support documentation of the interactions for data analysis, the study included an observer and two recording cameras on opposite ends of the tabletop.

Each pair received pentagon-shaped tangible dials (representing active tangibles) and a pen (representing a stylus). Each tangible featured an M5Stack Dial, ESP32-S3 Smart Rotary Knob with a 1.28-inch round touchscreen, encased in a 3-inch, 3D-printed pentagonal case (see Fig.2). The dial was selected for its rotary encoder for menu selections, flexible USB-C or battery power options, and development flexibility, which facilitates integration with the JavaScript-based Mapbox interface. The tangible’s pentagon case was inspired by five major drivers of migration: environmental, socio-cultural, economic, demographic, and political [59]. This shape enabled the design team to explore how physical affordances might influence user gestures. However, the study’s facilitators did not enforce any expectations, giving participants full autonomy to decide whether or not to incorporate the tangible’s shape into their interactions. The tangible cases were printed in three arbitrary colours: yellow, green, and white.

The stylus was chosen to support natural and expressive inputs, including annotating, drawing, tapping, and circling or lassoing [17]. Its precision could mitigate the limitations introduced by the lack of multi-touch input [65]. Multi-touch on the tabletop and wall displays was intentionally disabled to encourage collaboration and prevent workflow disruptions. During initial design and testing of map-based interactions on the tabletop display, some users’ transitory movements were mistakenly treated as commands. Compared to multi-touch, TUIs can support enhanced collaboration by enabling users to have a greater range of physical gestures that are not bound by the 2D dimension [50, 7]. Multi-touch may disrupt collaborative elicitations due to the influence of finger sizes, occlusion, and the size of WIMP-controls [13, 63, 64, 60].

III-B Participants

Thirty-six participants with self-described novice or expert geospatial modelling knowledge were recruited, and grouped into 18 pairs: 6 expert-expert, 6 mixed, and 6 novice-novice. Most participants (72.2%) were around 20 to 30 years old, and 60.0% were undergraduate or graduate university students. Educational backgrounds spanned social sciences, media studies, environmental and aeronautical engineering, and geographical sciences. Of the 36 participants, 11% reported using MSEs daily, 20% reported using MSEs 2 to 3 times per week, while 66.0% indicated they rarely or never used MSEs. Despite this, 61.0% held a positive perception of MSEs prior to the study, driven by either curiosity or prior experience, while 35.0% were indifferent or uninterested. In terms of stylus use, only 11.0% reported using a stylus at least once per week or daily, while 57.0% responded ”never” or ”rarely.” Among those who did use a stylus, 26.0% reported using it for digital drawing, and 23.0% for note-taking or annotation. Notably, all stylus users used them in conjunction with handheld devices.

TABLE II: Definitions and Illustrations Used to Code Participants’ Proposed Gestures

Gesture	Definition Used for Coding
Place	Changing position through the air without contacting a surface in the middle of the movement action.
Press	Tap with the finger, touch with the finger, applying pressure with finger(s) onto object, except for swipe.
Slide	Tangibles keep surface contact, move the tangible(s) or stylus across the screen(s).
Rotate	Turn or pivot around a fixed point while holding a part (i.e., dial) or entire body of input devices (i.e., pentagon body or stylus).
Lasso	Make a selection by creating a circular or loop-shaped path. The selection must be enclosed. Ensure surface contact with screen(s).
Draw	Make lines or shapes by moving across the screens. The selection does not have to be enclosed.

III-C Referents

The elicitation study was co-designed by a team of migration researchers, geographers, data scientists, and HCI researchers with varying levels of geospatial modelling expertise. The team curated a realistic workflow of 16 task-oriented referents for map-based visualizations or traditional GUI tools (refer to Table I). Referents’ visualizations were created from Mapbox-based screenshots of data layers for net-migration, precipitation, gold-mining, and conflicts in Ghana from 1985 to 2010 (see Fig 1), constrained by available coded data for that region and period. The referents covered both cartographic-perspective tasks, involving direct manipulation of map data [52], and system-perspective tasks, involving interaction with menus and controls [52].

Since a realistic geospatial workflow also involves repetitive interactions, such as loading data for multiple layers, this study aimed to reduce task-redundancy via some referents that required each participant to achieve the same result but with different time periods (i.e., referents 2 or 3) or data-layer selections (i.e., referents 2, 4, 6, 7, and 8). This motivated the use of two screens (1 and 2) (see Fig. 2) to present referent descriptions and visualizations, and for selected referents 2-4, 6-8, to display alternate descriptions or corresponding map views.

III-D Elicitation Study Procedure

This study’s procedures aimed to mitigate the legacy biases of elicitation studies [6, 31, 40] by employing 'priming', 'production' and 'partners' [40] in their activities. Before starting, each pair received three coloured tangible dials (active tangibles) and a pen (stylus). They were asked to complete four activities: (1) view an introductory (priming) presentation to briefly review the elicitation study methodology and geospatial data exploration; (2) complete a pre-task survey to assess their geospatial knowledge; (3) participate in the elicitation study in partners, such that production was encouraged through referents that promoted their collaborative design of open-ended tasks (proposals) [11]; and (4) complete a post-task questionnaire to reflect on their experience.

To mimic a realistic workflow of handling geospatial data, each pair (participants A and B) were asked to role-play as migration researchers. For each of the 16 referents, A and B were shown descriptions and their corresponding map visualization(s) on wall displays 1 and 2 (shown in Fig. 2). Then, A and B were given two minutes to verbally brainstorm proposal interactions. Next, A and B wrote and optionally sketched two proposals each on paper, with A focusing on wall display 2 and B on wall display 1, and with the requirement that at least one proposal per participant incorporates a tangible. After, A and B were asked to physically perform and verbally describe their four gesture proposals to each other.

Three coders then independently reviewed the interaction proposals, corrected or supplemented the proposal details based on video recordings and optional participant drawings, and then standardized following a procedure similar to that outlined by Morris [40] and León [31]. This standardization involved categorizing proposals into unique signs based on the input modalities used, the data attributes that were manipulated, and the end goal or outcome of the interaction.

IV Results

This section presents results for the maximum consensus (MC), consensus-distinct ratio (CDR), and statistical analysis.

IV-A MC and CDR Analysis

Coders identified 1152 proposals, which were grouped into 166 unique interactions or signs (based on distinct proposals per referent). The gesture coding framework, including key definitions and illustrative examples, is summarized in Table II.

Since standard agreement scores for elicitation studies do not account for multiple proposals per referent [31, 62], this study instead uses Morris’ consensus metrics (i.e., MC and CDR) [31, 39, 62, 40] to capture the frequency of the top proposals when each participant contributed two proposals per referent. These metrics, as shown in Table IV were chosen as high MC and CDR values may be indicative of strong agreement on a primary interaction with few close competitors [40]. To illustrate, higher MC values may indicate that the proposed interactions are more intuitive among participants[11], and higher CDR values may indicate that there is a greater proportion of interactions that exhibit some consensus among participants [61].

Across all referents and modalities, the mean MC was 18.6% , and using a consensus threshold of 2, the mean CDR was 0.212. Among participants, interactions with '1 tangible only' tended to exhibit similar commonalities ( $\overline{MC}_{1\ tangible\ only}=14.9\%$ ) as interactions with '1 stylus only' ( $\overline{MC}_{1\ stylus\ only}=14.7\%$ ). Interactions with '1 stylus only' may show more intuitive or natural usage patterns, as the mean CDR of these interactions is higher than that for '1 tangible only' ( $\overline{CDR}_{1\ stylus\ only}=27.5\%$ versus $\overline{CDR}_{1\ tangible\ only}=23.7\%$ ).

Multi-modal interactions had much lower mean MC and CDR values compared to the means from the interactions with '1 tangible only ' or '1 stylus only '. The mean CDR values were: $\overline{MC}_{tangible(s)\ +\ stylus}=1.2\%$ , $\overline{CDR}_{tangible(s)\ +\ stylus}=7.4\%$ , $\overline{MC}_{tangibles\ (\geq 2)}=1.6\%$ , and $\overline{CDR}_{tangiblesonly\ (\geq 2)}=5.7\%$ . In comparison with uni-modal interactions, the low mean MC and CDR values may indicate that multi-modal interactions had more individualization and diverse user behaviours than those with single-input tools. The complexity of the users’ proposals may have also been higher with multi-input tools, but it is important to note that the number of multi-elicitations is much less than uni-modal interactions, as shown in Table III.

TABLE III: Breakdown of Signs by Modality and Expertise

Modality	# Signs	Contribution by Expertise
1 Tangible Only	93	E: 64, M: 68, N: 57
1 Stylus Only	63	E: 41, M: 47, N: 33
Tangible(s) + Stylus	4	E: 4, M: 1, N: 2
Tangibles Only $(\geq 2)$	6	E: 3, M: 5, N: 2

•

^a E = expert pair, M = mixed pair, N = novice pair.
•

^b E, M, N pairs may have generated the same unique proposals.

IV-B Statistical Analysis

Two sets of statistical tests were conducted: one treating each proposal as the unit of analysis and another treating each participant as the unit. For proposal-level analyses, Pearson’s Chi-Squared tests were used for categorical measures (e.g., proposal matching, modality). Analysis of Variance (ANOVA) was used for interval measures (e.g., proposal length) when comparing the three types of pairs (expert, mixed, novice). For participant-level measures, Welch two-sample t-tests were used to compare the two knowledge levels (expert vs. novice). All statistical tests were performed using R (version 4.4.1) and RStudio (version 2024.04.2+764).

TABLE IV: Maximum Consensus (MC) and Consensus Distinct Ratio (CDR) by Modality for User-Elicited Interactions

#	Referent	MCI(s)	All I.M.		I.M. 1		I.M. 2		I.M. 3		I.M. 4
			MC	CDR	MC	CDR	MC	CDR	MC	CDR	MC	CDR
1	Navigate map, select Ghana	[Slide, Rotate, Press]	14%	0.204	14%	0.208	6%	0.235	–	–	6%	0.333
2	Access net migration layer	[Rotate, Press]	25%	0.191	25%	0.227	11%	0.188	–	–	6%	0.167
3	Change data layer year	[Press, Rotate]	22%	0.208	22%	0.240	11%	0.267	–	–	–	–
4	Add sub-driver layer	[Rotate, Press, Press]	14%	0.304	14%	0.435	11%	0.400	–	–	–	–
5	Annotate patterns	[Lasso]	31%	0.118	8%	0.430	31%	0.182	6%	0.200	–	–
6	Save visualization	[Press]	28%	0.248	28%	0.250	22%	0.286	3%	0.330	–	–
7	Move map table to wall	[Slide]	22%	0.229	14%	0.269	22%	0.231	–	–	6%	0.250
8	Duplicate map on wall	[Press, Slide]	11%	0.148	11%	0.143	8%	0.250	–	–	–	–
9	Move map wall to table	[Slide]	11%	0.208	11%	0.250	11%	0.222	–	–	–	–
10	Change driver layer	[Press, Rotate]	22%	0.286	22%	0.267	8%	0.600	–	–	–	–
11	Create map highlight	[Lasso]	11%	0.208	11%	0.250	11%	0.222	–	–	–	–
12	Brush link 2 wall maps	[Press, Slide]	11%	0.131	11%	0.214	6%	0.087	–	–	–	–
13	Merge maps	[Slide]	17%	0.136	6%	0.250	17%	0.111	–	NA	–	–
14	Split maps	[Rotate, Press]	11%	0.269	11%	0.261	8%	0.385	–	–	8%	0.167
15	Remove/brush/link	[Place, Place]	22%	0.261	17%	0.280	22%	0.286	6%	0.333	–	–
16	Delete	[Draw]	19%	0.250	14%	0.304	19%	0.308	–	–	–	–

•

^a MCI = Most common interaction(s); I.M. = Input modality; I.M. 1 = tangible only; I.M. 2 = stylus only; I.M. 3 = tangible + stylus; I.M. 4 = tangibles only $(\geq 2)$ .
•

^b ”–” = # Unique Proposals is 1 (less than threshold of 2) so NULL.
•

^c NA = No Proposals elicited.

IV-B1 Effect of Pair Type

Proposal-level analysis (i.e, whether the given proposal was repeated by another participant for a given referent by pair type) using Pearson’s Chi-squared test was significant $\chi^{2}(2)=15.5,\quad p<0.001$ . Proposals from individuals in mixed pairs were more likely to be matched by another participant than those from expert or novice pairs. The count of non-matched proposals generated by individuals in mixed versus expert versus novice pairs is 200 ¿ 168 ¿ 146 proposals, and comparatively for non-matched proposals is 184 ¡ 216 ¡ 238. This suggests that mixed pairs generated higher-quality (more likely to be common) proposals than those in uniform pairs.

An ANOVA on proposal length (i.e., number of steps in a proposal) with pair type as a between-proposal variable was also significant $F(2,1149)=32,\quad p<0.001,\quad\eta_{p}^{2}=0.05$ . Proposals from individuals in mixed pairs had the shortest number of steps, followed by proposals generated from those in expert pairs and then those generated in novice pairs, $M=2.65$ , $SD=1.13$ vs $M=2.82$ , $SD=1.15$ and $M=3.39$ , $SD=1.66$ .

A Chi-squared test on modality by pair type was not significant, $\chi^{2}(6)=10.4,\quad p=0.011$ .

Treating the unit of analysis as an individual participant, an ANOVA on the total number of matched proposals out of 32 with pair type as a between-participant variable was marginally significant, $F(2,33)=3.1$ , $p=0.57$ , $\eta_{p}^{2}=0.16$ . Participants in mixed pairs produced a higher number of matched proposals, $M=16.7$ , $SD=4.1$ , vs. expert pairs $M=14$ , $SD=3.5$ and vs. novice pairs $M=12.2$ , $SD=5.5$ , though it did not reach significance.

An ANOVA test on the average length of proposals with pair type as a between-participant variable was significant, $F(2,33)=6.0$ , $p=0.006$ , $\eta_{p}^{2}=0.27$ . Participants in mixed pairs produced shorter interactions than those in uniform pairs (especially versus novice pairs), $M=2.65$ , $SD=0.36$ , vs. expert pairs $M=2.82$ , $SD=0.51$ , and novice pairs $M=3.39$ , $SD=0.72$ . Follow-up Chi-squared tests on each motion type (e.g., Slide, Rotate, etc.) found that mixed pairs used fewer Lasso motions and that novice pairs used more Rotate, Press and Place motions $p_{s}\leq 0.013$ . Therefore, participants in mixed pairs performed better than those in uniform pairs.

A Chi-squared test on the average number of proposals designed per modality by user knowledge was not significant, $\chi^{(2)}(6)=0.86$ , $p=0.99$ .

IV-B2 Effect of Expertise

Treating the unit of analysis as a proposal, Pearson’s Chi-squared test on the proposal modality (e.g., ”1 Tangible Only”) by user knowledge (i.e., ”Expert” or ”Novice”) was not significant, $\chi^{(2)}(3)=5.5$ , $p=0.14$ . Pearson’s Chi-squared test on the proposal matching (i.e., whether the given proposal was repeated by another participant for a given referent) by user knowledge was also not significant, $\chi^{(2)}(1)=1.5$ , $p=0.21$ . A Welch two-sample t-test on the proposal length with user knowledge as a between-proposal variable was significant, $t(1087)=-3.8$ , $p<0.001$ . Proposals produced by experts were simpler (shorter) than those made by novices, $M=2.8$ , $SD=1.2$ vs $M=3.1$ , $SD=1.5$ . Follow-up Chi-squared tests on each motion type (e.g., Slide, Rotate, etc.) found that novices used more Place motions than experts, $\chi^{(2)}(1)=10$ , $p<0.01$ .

Treating the unit of analysis as a participant, a Welch two-sample t-test on the total number of matched proposals out of 32 comparing experts with novices was not significant, $t(32)=0.78$ , $p=0.44$ . A Welch two-sample t-test on the mean length of proposal comparing experts with novices was also not significant, $t(28)=-1.5$ , $p=0.14$ . A Chi-squared test on the average number of proposals designed per modality by user knowledge was not significant, $\chi^{(2)}(3)=0.30$ , $p=0.96$ (see Table III).

V Discussion

V-1 Input Modality Preferences for Cartographic or System-Perspective Tasks

The '1 tangible only' had the highest MC values across 9 out of the 16 referents (i.e., referents 1, 2, 3, 4, 8, 9, 10, 12, 14 in Table IV). This was followed by the '1 stylus only ', which had the highest MC values across 7 out of the 16 referents (i.e., referents 5, 6, 7, 9, 11, 13, 15, 16 in Table IV). There was a tie with referent 9, and the full breakdown of the interactions with the highest MC values by task-type is shown in Table V. The '1 tangible only ' and the '1 stylus only ' had higher user-preferences and agreements compared to the 'tangible(s) + stylus ' or 'tangibles only $(\geq 2)$ ', but the tangible was favoured for cartographic tasks, and the stylus was favoured for more system-perspective tasks.

V-2 Proposals by Knowledge Level

Participants in mixed expertise teams performed better (i.e., more matched and simpler proposals) than those in uniform expertise teams, especially versus novice teams. No evidence was found that the proposal modalities used differed across pair type or user knowledge groups, with a single tangible or a single stylus preferred in most cases.

V-3 Participants Monitored Each Other’s Activity Within Personal Spaces and Often Provided Feedback

Participants frequently looked over each other’s work while documenting their elicited proposals. Some participants voiced their feedback to add additional detail or modifications. Differences emerged, however, between expert, mixed, and novice pairs.

Three out of six expert pairs were particularly vocal about how gestures should be documented, often interrupting each other with ”what if” questions, and over-focusing on potential failure modes. This occasionally led to territorial behaviour when one partner altered the other’s proposal. Some experts drifted into suggesting their ideas for new referents or functionalities, which were documented for future work, but required the study’s facilitators to refocus them on the current task to finish the study.

Mixed pairs often exhibited more passive exchanges than expert groups. However, two out of six mixed groups had experts who voiced some concern with how the novices provided feedback of the proposals. In one pair, a novice repeatedly interrupted the expert while seeking reassurance, which the expert later described as irritating and wished the novice had shown more confidence. In another pair, the expert had to repeatedly encourage a timid novice to speak up, highlighting how knowledge imbalances can create hesitancy and skew proposals toward one participant’s ideas.

Novice participants tended to be more talkative with each other and less concerned about how their partners documented their elicitations. Five out of six novice pairs exhibited fluid and supportive communication when one of their partners requested documentation support. During these conversations, however, sometimes novice members added additional gestures to their initial proposals. These seemed less constrained by legacy biases or physical restrictions in comparison to the suggestions made by expert or mixed pairs. For example, one novice pair imagined throwing the tangible in the air from the tabletop to the wall-display to project a map. Another pair discussed the possibility of throwing the tangible in the air and letting it fall onto the tabletop to delete the maps. Expert and mixed pairs mostly focused on planar gestures with few in-air actions (or collisions).

V-4 Supporting an Embodied Metaphor of Migration Drivers through the Tangible’s Pentagon Shape

The tangible used in this study was a low-fidelity prototype with a pentagonal case, embodying a metaphor of the five macro-drivers of migration (i.e, environmental, sociocultural, economic, demographic, and political [59]). Most participants did not immediately recognize this metaphor, but after the priming presentation, some began to prefer using the tangible for performing cartographic interactions and the stylus for system-perspective interactions (refer to Table V). To illustrate, participants tended to grab a tangible to manipulate map layers from the tabletop to the wall displays, and they also favoured using different coloured tangibles to represent distinct layers.

V-5 Potential Benefits of the Tangible(s) and Stylus for the Final System

The tangible(s) and the stylus supported interactive communication and collaboration across users of mixed geospatial expertise and professions. Brainstorming with both input modalities on the tabletop created a safe shared workspace for novices and experts to exchange ideas and see beyond legacy biases. Indeed, this shared workspace was not free from the tensions of participants’ disagreements, mimicking real-life scenarios that collaborative migration or policy researchers face. However, the tangible(s) and the stylus afforded users a visual means to convey gestures that might not have been fully apparent by verbal communication. In turn, this supported conflict-resolution by balancing novices’ creativity with experts’ insights on feasibility and utility. Employing the tangible(s) and the stylus in the final system may support migration and policy researchers in their understanding of how diverse users process and interact with spatial information; an interdisciplinary understanding of the latter can become an enabler for solving social issues [30].

TABLE V: Input Modality of Interactions with Highest MC per Referent

Task Type	1 Tangible	1 Stylus	Tie
Cartography	5	1	0
System Perspective	3	6	1

VI Limitations

Recruitment predominantly drew on a migration institute at an undergraduate-focused university, with over 90% of participants under 40. Participants were paired by self-reported knowledge level and scheduling rather than by prior familiarity. To better capture the diversity of migration researchers, future work will include delocalizing the research with a focus on broadening age groups and pairing familiarity. Further study is also required to understand how diverse users’ metaphorical mappings would change with the addition of other input modalities in a working prototype. Nevertheless, the employment of diverse knowledge groups to collaboratively propose cartographic and system-perspective tasks with tangibles highlights the potential of this study to inform the design of intuitive spatial data-exploration.

VII Conclusion

This elicitation study showed how expert and novice users can propose cartographic and system-based interactions for a collaborative geospatial visualization. The pentagon-shaped tangible, embodying the five macro-drivers of migration [59], worked well for directly manipulating map layers, while the stylus was better suited to perform annotations, drawings, free-hand selections, and dragging or moving tangible-based selections. Together, the tangible-only and the stylus-only modalities complemented one another.

Despite differing geospatial expertise, participants converged on a preference for uni-modal interactions, with tangible-only the most frequently used modality and stylus-only a close second (refer to Table IV). Multi-modal modalities (i.e., $\geq 2$ tangibles or the tangible(s) + the stylus) were used less often for completion of complex tasks, perhaps influenced by this study’s 16 referents that offered a low-level workflow with limited compound tasks. Future work will address this limitation through a working prototype that will present more complex modelling requiring higher-level tasks.

Across all referents and modalities, both experts and novices showed generally low MC and CDR values (refer to Table IV), with differences between groups not reaching statistical significance, suggesting similar perceptions of task demands and complexity. Future work will broaden the participant pool in terms of age and discipline and use a task-based study with a working prototype to ensure accessibility. It is hoped that the findings of this study will extend beyond migration studies, informing the design of collaborative tangible systems that support mixed-methods researchers in discussion and evidence-based decision-making.

References

[1] H. Afkari, V. Maquil, and D. Anastasiou (2020) Exploring Opportunities of Tabletop Interfaces for Promoting and Analysing Collaboration. In ETIS 2020, Sienna, Italy, pp. 1–6 (en). Cited by: §I, §II-B, §II-B.
[2] S. Al-Megren and R. A. Ruddle (2016-02) Comparing Tangible and Multi-touch Interaction for Interactive Data Visualization Tasks. In Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, TEI ’16, New York, NY, USA, pp. 279–286. External Links: ISBN 978-1-4503-3582-9, Link, Document Cited by: §I.
[3] D. Anastasiou, A. Coppens, and V. Maquil (2024-04) Gesture combinations during collaborative decision-making at wall displays. i-com 23 (1), pp. 57–69 (en). Note: Publisher: Oldenbourg Wissenschaftsverlag External Links: ISSN 2196-6826, Link, Document Cited by: §I.
[4] L. Angelini, D. Lalanne, E. V. Hoven, O. A. Khaled, and E. Mugellini (2015) Move, Hold and Touch: A Framework for Tangible Gesture Interactive Systems. Machines 3 (3), pp. 173–207 (English). Note: Num Pages: 173-207 Place: Basel, Switzerland Publisher: MDPI AG External Links: Link, Document Cited by: §I, §I.
[5] L. Besançon, P. Issartel, M. Ammi, and T. Isenberg (2017-01) Hybrid Tactile/Tangible Interaction for 3D Data Exploration. IEEE Transactions on Visualization and Computer Graphics 23 (1), pp. 881–890. Note: Conference Name: IEEE Transactions on Visualization and Computer Graphics External Links: ISSN 1941-0506, Link, Document Cited by: §I.
[6] C. Beşevli, O. T. Buruk, M. Erkaya, and O. Özcan (2018-05) Investigating the Effects of Legacy Bias: User Elicited Gestures from the End Users Perspective. In Proceedings of the 2018 ACM Conference Companion Publication on Designing Interactive Systems, Hong Kong China, pp. 277–281 (en). External Links: ISBN 978-1-4503-5631-2, Link, Document Cited by: §III-D.
[7] S. Brave, H. Ishii, and A. Dahley (1998-11) Tangible interfaces for remote collaboration and communication. In Proceedings of the 1998 ACM conference on Computer supported cooperative work, Seattle Washington USA, pp. 169–178 (en). External Links: ISBN 978-1-58113-009-6, Link, Document Cited by: §I, §III-A.
[8] M. Brehmer and T. Munzner (2013-12) A Multi-Level Typology of Abstract Visualization Tasks. IEEE Transactions on Visualization and Computer Graphics 19 (12), pp. 2376–2385. Note: Conference Name: IEEE Transactions on Visualization and Computer Graphics External Links: ISSN 1941-0506, Link, Document Cited by: TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I.
[9] D. Çay, T. Nagel, and A. E. Yantaç (2019-10) What is happening in the city? A case study for user-centred geovisualisation design. Journal of Location Based Services 13 (4), pp. 270–292 (en). External Links: ISSN 1748-9725, 1748-9733, Link, Document Cited by: §II-A.
[10] N. Cross (2004-09) Expertise in design: an overview. Design Studies 25 (5), pp. 427–441 (en). External Links: ISSN 0142694X, Link, Document Cited by: §II-C.
[11] A. Danielescu and D. Piorkowski (2022-04) Iterative Design of Gestures During Elicitation: Understanding the Role of Increased Production. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, New York, NY, USA, pp. 1–14. External Links: ISBN 978-1-4503-9157-3, Link, Document Cited by: §III-D, §IV-A.
[12] F. de Morais Bezerra, P. Melo, and J. P. Costa (2014-01) Visual and Interactive Comparative Analysis of Individual Opinions: A Group Decision Support Tool. Group Decision and Negotiation 23 (1), pp. 101–125 (English). Note: Num Pages: 101-125 Place: Dordrecht, Netherlands Publisher: Springer Nature B.V. External Links: ISSN 09262644, Link, Document Cited by: TABLE I.
[13] S. M. Drucker, D. Fisher, R. Sadana, J. Herron, and M.C. Schraefel (2013-04) TouchViz: a case study comparing two interfaces for data analytics on tablets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris France, pp. 2301–2310 (en). External Links: ISBN 978-1-4503-1899-0, Link, Document Cited by: §I, §III-A.
[14] N. Elmqvist and J. Fekete (2010) Hierarchical Aggregation for Information Visualization: Overview, Techniques, and Design Guidelines. IEEE Transactions on Visualization and Computer Graphics 16 (3), pp. 439–454. External Links: ISSN 1941-0506, Link, Document Cited by: §II-A.
[15] B. Ens, S. Goodwin, A. Prouzeau, F. Anderson, F. Y. Wang, S. Gratzl, Z. Lucarelli, B. Moyle, J. Smiley, and T. Dwyer (2021-02) Uplift: A Tangible and Immersive Tabletop System for Casual Collaborative Visual Analytics. IEEE Transactions on Visualization and Computer Graphics 27 (2), pp. 1193–1203. Note: Conference Name: IEEE Transactions on Visualization and Computer Graphics External Links: ISSN 1941-0506, Link, Document Cited by: §I, §I.
[16] J. Fuchs, R. Rädle, D. Sacha, F. Fischer, and A. Stoffel (2013-12) Collaborative data analysis with smart tangible devices. In Proc. SPIE 9017, P. C. Wong, D. L. Kao, M. C. Hao, and C. Chen (Eds.), San Francisco, California, USA, pp. 1–15 (en). External Links: Link, Document Cited by: §I.
[17] K. I. Gero, L. Chilton, C. Melancon, and M. Cleron (2022-06) Eliciting Gestures for Novel Note-taking Interactions. In Designing Interactive Systems Conference, Virtual Event Australia, pp. 966–975 (en). External Links: ISBN 978-1-4503-9358-4, Link, Document Cited by: §III-A.
[18] M. F. Goodchild (2015-04) Space, place and health. Annals of GIS 21 (2), pp. 97–100 (English). Note: Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/19475683.2015.1007895 External Links: ISSN 1947-5683, Link, Document Cited by: §II-A.
[19] D. Gotz and M. X. Zhou (2009-01) Characterizing Users’ Visual Analytic Activity for Insight Provenance. Information Visualization 8 (1), pp. 42–55. Note: Publisher: SAGE Publications External Links: ISSN 1473-8716, Link, Document Cited by: TABLE I, TABLE I, TABLE I, TABLE I, TABLE I.
[20] C. Guerlain, S. Cortina, and S. Renault (2016) Towards a Collaborative Geographical Information System to Support Collective Decision Making for Urban Logistics Initiative. Transportation Research Procedia 12, pp. 634–643 (en). External Links: ISSN 23521465, Link, Document Cited by: §II-A.
[21] J. Heer and B. Shneiderman (2012-02) Interactive Dynamics for Visual Analysis: A taxonomy of tools that support the fluent and flexible use of visualizations. Queue 10 (2), pp. 30–55. External Links: ISSN 1542-7730, Link, Document Cited by: TABLE I, TABLE I, TABLE I, TABLE I.
[22] E. W. Hess-Lüttich (2011) Spatial turn: On the concept of space in cultural geography and literary theory. Lexia-Rivista di semiotica/Journal of semiotics (9-10), pp. 23–42. Note: Publisher: CIRCE “Centro Interdipartimentale di Ricerche sulla Comunicazione” Cited by: §II-A.
[23] E. Hornecker, R. J. Jacob, C. C. Hummels, B. Ullmer, A. Schmidt, E. A. van den Hoven, and A. Mazalek (2008) TEI goes on: Tangible and embedded interaction. IEEE Pervasive Computing 7 (2), pp. 91–96. Cited by: §I.
[24] W. Hwang, R. Shadiev, C. Tseng, and Y. Huang (2015) Exploring Effects of Multi-Touch Tabletop on Collaborative Fraction Learning and the Relationship of Learning Behavior and Interaction with Learning Achievement. Educational Technology & Society 18 (4), pp. 459–473. External Links: Link Cited by: §I.
[25] A. Ioannou and C. Antoniou (2016) Tabletops for Peace: Technology Enhanced Peacemaking in School Contexts. Journal of Educational Technology & Society 19 (2), pp. 164–176. Note: Publisher: International Forum of Educational Technology & Society External Links: ISSN 1176-3647, Link Cited by: §II-B.
[26] P. Isenberg, D. Fisher, S. A. Paul, M. R. Morris, K. Inkpen, and M. Czerwinski (2011) Co-Located Collaborative Visual Analytics around a Tabletop Display. IEEE Transactions on Visualization and Computer Graphics 18 (5), pp. 689–702. External Links: Link Cited by: §II-B.
[27] C. E. Jones and V. Maquil (2015) Twist, Shift, or Stack? - Usability Analysis of Geospatial Interactions on a Tangible Tabletop. In Proceedings of the 1st International Conference on Geographical Information Systems Theory, Applications and Management, Barcelona, Spain, pp. 170–177. External Links: ISBN 978-989-758-099-4, Link, Document Cited by: §I.
[28] C. E. Jones and V. Maquil (2016) Towards Geospatial Tangible User Interfaces: An Observational User Study Exploring Geospatial Interactions of the Novice. In Geographical Information Systems Theory, Applications and Management, C. Grueau and J. Gustavo Rocha (Eds.), Vol. 582, pp. 104–123. Note: Series Title: Communications in Computer and Information Science External Links: ISBN 978-3-319-29588-6 978-3-319-29589-3, Link, Document Cited by: §II-B.
[29] J. Koch, J. Pearson, A. Lucero, M. Sturdee, W. E. Mackay, M. Lewis, and S. Robinson (2020-04) Where Art Meets Technology: Integrating Tangible and Intelligent Tools in Creative Processes. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu HI USA, pp. 1–7 (en). External Links: ISBN 978-1-4503-6819-3, Link, Document Cited by: §II-A.
[30] W. Kuhn (2012-12) Core concepts of spatial information for transdisciplinary research. International Journal of Geographical Information Science 26 (12), pp. 2267–2276 (en). External Links: ISSN 1365-8816, 1362-3087, Link, Document Cited by: §II-B, §V-5.
[31] G. M. León, P. Isenberg, and A. Breiter (2024-02) Eliciting Multimodal and Collaborative Interactions for Data Exploration on Large Vertical Displays. IEEE Transactions on Visualization and Computer Graphics 30 (2), pp. 1624–1637. External Links: ISSN 1077-2626, 1941-0506, 2160-9306, Link, Document Cited by: §I, §I, §II-B, §II-C, §II-C, §II-C, §III-D, §III-D, §IV-A.
[32] Z. Liu and J. Stasko (2010-11) Mental Models, Visual Reasoning and Interaction in Information Visualization: A Top-down Perspective. IEEE Transactions on Visualization and Computer Graphics 16 (6), pp. 999–1008. Note: Conference Name: IEEE Transactions on Visualization and Computer Graphics External Links: ISSN 1941-0506, Link, Document Cited by: TABLE I.
[33] R. Manshaei, U. Mayat, S. A. Imtiaz, V. Andric, K. Aliar, N. Abu Hantash, K. Masood, G. Resch, A. Bakogeorge, S. Sabatinos, and A. Mazalek (2022-11) Tangible Chromatin: Tangible and Multi-surface Interactions for Exploring Datasets from High-Content Microscopy Experiments. Proc. ACM Hum.-Comput. Interact. 6 (ISS), pp. 558:51–558:72. External Links: Link, Document Cited by: §I, footnote 1.
[34] V. Maquil, L. De Sousa, U. Leopold, and E. Tobias (2015-04) A geospatial tangible user interface to support stakeholder participation in urban planning. In 2015 1st International Conference on Geographical Information Systems Theory, Applications and Management (GISTAM), Spain, pp. 1–8. External Links: Link Cited by: §I, §I, §II-A, §II-A, §II-A, §II-B.
[35] V. Maquil, U. Leopold, L. M. De Sousa, L. Schwartz, and E. Tobias (2018-04) Towards a framework for geospatial tangible user interfaces in collaborative urban planning. Journal of Geographical Systems 20 (2), pp. 185–206 (en). External Links: ISSN 1435-5949, Link, Document Cited by: §I, §I, §II-A, §II-A, §II-A, §II-A, §II-B.
[36] V. Maquil, T. Psik, and I. Wagner (104) The ColorTable: a design story. In Proceedings of the 2nd international conference on Tangible and embedded interaction, Bonn Germany, pp. 97–104 (en). External Links: ISBN 978-1-60558-004-3, Link, Document Cited by: §I, §I, §II-B.
[37] F. Marinho Rodrigues, T. Seyed, F. Maurer, and S. Carpendale (2014-11) Bancada: Using Mobile Zoomable Lenses for Geospatial Exploration. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, ITS ’14, New York, NY, USA, pp. 409–414. External Links: ISBN 978-1-4503-2587-5, Link, Document Cited by: §I.
[38] M. Meyer, M. Sedlmair, and T. Munzner (2012-10) The four-level nested model revisited: blocks and guidelines. In Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors - Novel Evaluation Methods for Visualization, BELIV ’12, New York, NY, USA, pp. 1–6. External Links: ISBN 978-1-4503-1791-7, Link, Document Cited by: TABLE I.
[39] M. R. Morris, A. Danielescu, S. Drucker, D. Fisher, B. Lee, M. C. Schraefel, and J. O. Wobbrock (2014-05) Reducing legacy bias in gesture elicitation studies. Interactions 21 (3), pp. 40–45 (en). External Links: ISSN 1072-5520, 1558-3449, Link, Document Cited by: §IV-A.
[40] M. R. Morris (2012-11) Web on the wall: insights from a multimodal interaction elicitation study. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces, Cambridge Massachusetts USA, pp. 95–104 (en). External Links: ISBN 978-1-4503-1209-7, Link, Document Cited by: §I, §II-C, §II-C, §III-D, §III-D, §IV-A.
[41] T. Nagel, F. Heidmann, M. Condotta, and E. Duval (2010-10) Venice unfolding: a tangible user interface for exploring faceted data in a geographical context. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, Reykjavik Iceland, pp. 743–746 (en). External Links: ISBN 978-1-60558-934-3, Link, Document Cited by: §II-A, §II-A.
[42] T. Nagel, M. Maitan, E. Duval, A. Vande Moere, J. Klerkx, K. Kloeckl, and C. Ratti (2014-05) Touching transport - a case study on visualizing metropolitan public transit on interactive tabletops. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, AVI ’14, New York, NY, USA, pp. 281–288. External Links: ISBN 978-1-4503-2775-6, Link, Document Cited by: §I.
[43] W. Pike, J. Stasko, C. Remco, and T. O’Connell (2009) The Science of Interaction. Information Visualization 8 (4), pp. 263–274 (English). External Links: Link, Document Cited by: TABLE I.
[44] B. Piper, C. Ratti, and H. Ishii (2002-04) Illuminating clay: a 3-D tangible interface for landscape analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Minneapolis Minnesota USA, pp. 355–362 (en). External Links: ISBN 978-1-58113-453-7, Link, Document Cited by: §I.
[45] P. Pirolli, S. K. Card, and M. M. Van Der Wege (2001-03) Visual information foraging in a focus + context visualization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’01, New York, NY, USA, pp. 506–513. External Links: ISBN 978-1-58113-327-1, Link, Document Cited by: §II-B.
[46] J. Raskin (2000) Humane Interface, The: New Directions for Designing Interactive Systems. 4 edition, Addison-Wesley Professional, United States of America (en). External Links: ISBN 978-0-201-37937-2, Link Cited by: TABLE I, TABLE I, TABLE I, TABLE I, TABLE I.
[47] R. E. Roth (2012-11) Cartographic Interaction Primitives: Framework and Synthesis. The Cartographic Journal 49 (4), pp. 376–395 (en). External Links: ISSN 0008-7041, 1743-2774, Link, Document Cited by: TABLE I, TABLE I, TABLE I, TABLE I, TABLE I.
[48] M. J. Rovine and G. D. Weisman (1989-09) Sketch-map variables as predictors of way-finding performance. Journal of Environmental Psychology 9 (3), pp. 217–232 (en). External Links: ISSN 02724944, Link, Document Cited by: §II-B.
[49] K. A. Satriadi, J. Smiley, B. Ens, M. Cordeil, T. Czauderna, B. Lee, Y. Yang, T. Dwyer, and B. Jenny (2022-04) Tangible Globes for Data Visualisation in Augmented Reality. In CHI Conference on Human Factors in Computing Systems, New Orleans LA USA, pp. 1–16 (en). External Links: ISBN 978-1-4503-9157-3, Link, Document Cited by: §II-A.
[50] B. Schneider, M. Tobiasz, C. Willis, and C. Shen (2012-11) WALDEN: multi-surface multi-touch simulation of climate change and species loss in thoreau’s woods. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces, ITS ’12, New York, NY, USA, pp. 387–390. External Links: ISBN 978-1-4503-1209-7, Link, Document Cited by: §I, §III-A.
[51] T. Seyed, C. Burns, M. Costa Sousa, F. Maurer, and A. Tang (2012-11) Eliciting usable gestures for multi-display environments. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces, Cambridge Massachusetts USA, pp. 41–50 (en). External Links: ISBN 978-1-4503-1209-7, Link, Document Cited by: §I.
[52] Z. Shakeri Hossein Abad, C. Anslow, and F. Maurer (2014-11) Multi Surface Interactions with Geospatial Data: A Systematic Review. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, Dresden Germany, pp. 69–78 (en). External Links: ISBN 978-1-4503-2587-5, Link, Document Cited by: §I, §I, §II-B, §III-C.
[53] M. Spindler, C. Tominski, H. Schumann, and R. Dachselt (2010) Tangible views for information visualization. In ACM International Conference on Interactive Tabletops and Surfaces, Saarbrücken Germany, pp. 157–166 (en). External Links: ISBN 978-1-4503-0399-6, Link, Document Cited by: §II-A.
[54] R.R. Springmeyer, M.M. Blattner, and N.L. Max (1992-10) A characterization of the scientific data analysis process. In Proceedings Visualization ’92, United States of America, pp. 235–242. External Links: Link, Document Cited by: TABLE I.
[55] P. Tabrizian, A. Petrasova, B. Harmon, V. Petras, H. Mitasova, and R. Meentemeyer (2016-10) Immersive tangible geospatial modeling. In Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, SIGSPACIAL ’16, New York, NY, USA, pp. 1–4. External Links: ISBN 978-1-4503-4589-7, Link, Document Cited by: §I, §I.
[56] K. Takahira, W. Kam-Kwai, L. Yang, X. Xu, T. Fujiwara, and H. Qu (2025) TangibleNet: Synchronous Network Data Storytelling through Tangible Interactions in Augmented Reality. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, Japan, pp. 1–18. Note: arXiv:2504.04710 [cs] External Links: Link, Document Cited by: §II-A.
[57] L. Tateosian, H. Mitasova, B. Harmon, B. Fogleman, K. Weaver, and R. Harmon (2010-11) TanGeoMS: Tangible Geospatial Modeling System. IEEE Transactions on Visualization and Computer Graphics 16 (6), pp. 1605–1612. External Links: ISSN 1077-2626, Link, Document Cited by: §I, §I, §II-A, §II-A.
[58] P. Tuddenham, D. Kirk, and S. Izadi (2010-04) Graspables revisited: multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, New York, NY, USA, pp. 2223–2232. External Links: ISBN 978-1-60558-929-9, Link, Document Cited by: §I.
[59] T. M. University (2024) What makes MEMO Unique. Educational (en). External Links: Link Cited by: §III-A, §V-4, §VII.
[60] C. Valdes, D. Eastman, C. Grote, S. Thatte, O. Shaer, A. Mazalek, B. Ullmer, and M. K. Konkel (2014-04) Exploring the design space of gestural interaction with active tokens through user-defined gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto Ontario Canada, pp. 4107–4116 (en). External Links: ISBN 978-1-4503-2473-1, Link, Document Cited by: §I, §I, §I, §II-C, §III-A, footnote 1.
[61] R. Vatavu and J. O. Wobbrock (2016-05) Between-Subjects Elicitation Studies: Formalization and Tool Support. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose California USA, pp. 3390–3402 (en). External Links: ISBN 978-1-4503-3362-7, Link, Document Cited by: §IV-A.
[62] R. Vatavu and J. O. Wobbrock (2022-01) Clarifying Agreement Calculations and Analysis for End-User Elicitation Studies. ACM Trans. Comput.-Hum. Interact. 29 (1), pp. 5:1–5:70. External Links: ISSN 1073-0516, Link, Document Cited by: §IV-A.
[63] D. Vogel and P. Baudisch (2007-04) Shift: a technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’07, New York, NY, USA, pp. 657–666. External Links: ISBN 978-1-59593-593-9, Link, Document Cited by: §I, §III-A.
[64] S. Voida, M. Tobiasz, J. Stromer, P. Isenberg, and S. Carpendale (2009-11) Getting practical with interactive tabletop displays: designing for dense data, ”fat fingers,” diverse interactions, and face-to-face collaboration. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, Banff Alberta Canada, pp. 109–116 (en). External Links: ISBN 978-1-60558-733-2, Link, Document Cited by: §I, §III-A.
[65] W. Willett, Q. Lan, and P. Isenberg (2014) Eliciting Multi-touch Selection Gestures for Interactive Data Graphics. The Eurographics Association (en). Note: Artwork Size: 5 pages ISBN: 9783905674699 Pages: 5 pages Publication Title: EuroVis - Short Papers External Links: Link, Document Cited by: §I, §II-A, §III-A.
[66] J. O. Wobbrock, M. R. Morris, and A. D. Wilson (2009-04) User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston MA USA, pp. 1083–1092 (en). External Links: ISBN 978-1-60558-246-7, Link, Document Cited by: §II-C.
[67] J. O. Wobbrock, B. A. Myers, and J. A. Kembel (2003-11) EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion. In Proceedings of the 16th annual ACM symposium on User interface software and technology, Vancouver Canada, pp. 61–70 (en). External Links: ISBN 978-1-58113-636-4, Link, Document Cited by: §I.
[68] W. Yan, Y. Murakami, A. Yasuda, T. Makihara, R. Fujimoto, and S. Nakayama (2021) Developing the eXtended Tangible User Interface as an Experimental Platform for Geo CPS. In Emerging Technologies for Disaster Resilience, M. Sakurai and R. Shaw (Eds.), pp. 11–32 (en). Note: Series Title: Disaster Risk Reduction External Links: ISBN 978-981-16-0360-0, Link, Document Cited by: §I.
[69] Z. Yuan, W. S. Ng, S. Tat Goh, and Y. Zhou (2018-10) Study of Design Method for Tangible User Interface in IoT Paradigm. In TENCON 2018 - 2018 IEEE Region 10 Conference, Jeju, Korea (South), pp. 2071–2074. External Links: ISBN 978-1-5386-5457-6, Link, Document Cited by: §I, §II-A.
[70] A. Zolnai (2014-12) Map stories can provide dynamic visualizations of the Anthropocene to broaden factually based public understanding. The Anthropocene Review 1 (3), pp. 243–251 (en). Note: Publisher: SAGE Publications External Links: ISSN 2053-0196, Link, Document Cited by: §I.