Gardony Map Drawing Analyzer: Software for quantitative analysis of sketch maps

Abstract

Sketch maps are effective tools for assessing spatial memory. However, despite their widespread use in cognitive science research, sketch map analysis techniques remain unstandardized and carry limitations. In the present article, we present the Gardony Map Drawing Analyzer (GMDA), an open-source software package for sketch map analysis. GMDA combines novel and established analysis techniques into a graphical user interface that permits rapid computational sketch map analysis. GMDA calculates GMDA-unique measures based on pairwise comparisons between landmarks, as well as bidimensional regression parameters (Friedman & Kohler, 2003), which together reflect sketch map quality at two levels: configural and individual landmark. The configural measures assess the overall landmark configuration and provide a whole-map analysis. Individual landmark measures, introduced in GMDA, assess individual landmark placement and indicate how individual landmarks contribute to the configural scores. Together, these measures provide a more complete psychometric picture of sketch map analysis, allowing for comparisons between sketch maps and between landmarks. The calculated measures reflect specific and cognitively relevant aspects of interlandmark spatial relationships, including distance and angular representation. GMDA supports complex environments (up to 48 landmarks) and two software modes that capture aspects of maps not addressed by existing techniques, such as landmark size and shape variation and interlandmark containment relationships. We describe the software and its operation and present a formal specification of calculation procedures for its unique measures. We then validate the software by demonstrating the capabilities and reliability of its measures using simulation and experimental data. The most recent version of GMDA is available at www.aarongardony.com/tools/map-drawing-analyzer.

Introduction

During navigation, people gain knowledge of an environment’s spatial layout, including the locations of points of interest, the distances between them, and their relative placement (Montello, 1998). A critical question in spatial cognition concerns how spatial memory develops and how internal and external factors influence and shape this development. In order to tackle this important question, one must employ sensitive and reliable measures of spatial memory. One measure often used in spatial cognition research is map drawing. Map drawing provides an intuitive assessment of spatial memory, requiring participants to sketch the allocentric configuration of environment features. Sketch maps have been used extensively in psychological experiments, dating back to seminal work by Lynch (1960), and they have proven an effective and reliable measure of spatial memory (Billinghurst & Weghorst, 1995; Blades, 1990; Newcombe, 1985; Tversky, 1981) and predictor of wayfinding performance (Rovine & Weisman, 1989). Despite their utility in assessing spatial memory, quantitative analysis and scoring of sketch maps remains cumbersome and unstandardized (Golledge, 1976). Here we present easy-to-use software that calculates measures of map completeness and organization, along with simulation and experimental data to validate its effectiveness.

Challenges to sketch map analysis

The difficulty inherent in quantitative analysis of sketch maps lies in their information richness. A sketch map’s primary unit is the landmark, itself a nebulous construct whose nature few definitions adequately capture across a range of environments (Sorrows & Hirtle, 1999). Indeed, any salient component of an environment, such as buildings or roads, can be categorized as a landmark, as can emergent properties such as intersections. The first step to evaluating a sketch map is to define the finite set of landmarks that constitutes the target environment (i.e., the environment to which the sketch map is critically compared). As an example, consider a local playground as the target environment. This playground may contain a swing set, a carousel, a sandbox, a slide, and a seesaw in some configuration (see Fig. 1a), and a given sketch map may contain some or all of these landmarks in some other configuration (see Fig. 1b). The goal of sketch map analysis is to measure the fit of the sketch map’s landmark configuration to the actual, target configuration. This is a challenging task. A map is a symbolic representation containing simple and abstract objects (Blaser, 2000). Sketch maps cannot retain the absolute positional information of the target environment; otherwise, a map of a major city, for example, would be miles across! Rather, maps retain the relative placement of landmarks in a compressed space. We say that the map in Fig. 1b is “good” because the landmarks are generally placed accurately relative to each other. But assessing relative landmark placement using pairwise comparisons rapidly becomes computationally expensive as the number of landmarks increases.

Fig. 1

An example target environment—a local playground (a)—and a hand-drawn sketch map representing the target environment (b)

Using the example of the playground, we would first compare the location of the swing set to those of all other landmarks. Next, we would compare the carousel to the other landmarks (except the swing set, which we have already done), repeating this process for a total of ten pairwise comparisons. When making these comparisons, we might consider the following questions: Is each landmark correctly placed relative to the other landmarks? Are the relative distances and angles between each landmark and the others preserved? For our playground example with five landmarks, approaching these comparisons without computer automation is tenable. However, as the number of landmarks (nL) in the environment increases, the number of necessary pairwise comparisons (nPC) increases at a polynomial rate. The combinatorial explosion of comparisons for landmark-rich maps necessitates a computer-based approach, as is shown by the following equation:

$$ nPC=\frac{nL!}{2!\left( nL!-2\right)}\ \mathrm{or}\ \mathrm{simply}\ \left(\begin{array}{c}\hfill nL\hfill \\ {}\hfill 2\hfill \end{array}\right) $$

Previous approaches to sketch map analysis

With such computational difficulties, researchers often eschew computational approaches to sketch map analysis in favor of qualitative approaches. For example, researchers may score maps through a subjective evaluation of map features (Billinghurst & Weghorst, 1995). Independent judges would assign maps subjective ratings of “map goodness,” record a metric count of appropriate landmark labels, and create a subjective relative landmark position score in which the positions of individual landmarks are evaluated relative to surrounding landmarks and roads (Carassa, Geminiani, Morganti, & Varotto, 2002; Coluccia, Bosco, & Brandimonte, 2007; Coluccia, Iosue, & Brandimonte, 2007; Zanbaka, Lok, Babu, Ulinski, & Hodges, 2005). Others could determine important map features and hand-score them using carefully designed rubrics (Brunyé & Taylor, 2008a, b). This approach may involve counting missing route segments and wrong turns (Hegarty, Montello, Richardson, Ishikawa, & Lovelace, 2006) or using Likert scales to rate road and layout orientation (Woollett & Maguire, 2010). On the surface, these approaches appear to incorporate quantitative approaches, but they have severe limitations.

First, in many cases a subset of environment features, such as landmark and road placement, are binary-scored—for example, either a feature is correctly placed/orientated or it is not. Although the resulting averaged data are quantitative, they neither reflect metrical information about the direction/degree of sketch map error nor consider all comparisons of the sketch map. Second, these approaches require carefully designed scoring rubrics to evaluate maps. Not only is this process time-consuming, but, because rubrics are designed before data are collected, they are often specific to the researcher’s hypotheses undermining their objectivity. They may overlook map features not predicted by the researcher’s hypotheses, precluding the discovery of novel or unexpected findings that could have important theoretical implications. Third, these approaches require multiple independent judges to evaluate maps. Training judges is time-consuming, and even the best-designed scoring rubric has some subjectivity. This subjectivity necessitates using two or more independent judges and assessing scoring reliability, thus slowing the analysis considerably. More generally, when it comes to sketch map analysis, “reinventing the wheel” appears to be the standard, and this variation in approach makes it difficult to interpret the literature as a whole.

In this landscape of varied approaches, one method of sketch map analysis has gathered general acceptance, bidimensional regression (BDR). BDR is a statistical technique that avoids the perils and pitfalls of subjective map analysis and provides quantitative measures of the degree of resemblance between sets of points in a 2-D plane (Friedman & Kohler, 2003; Tobler, 1994). BDR inputs the coordinates of a sketch map’s landmarks and the coordinates of the target environment’s landmarks and yields reliable measures of configural accuracy. Presently, BDR is the preferred quantitative sketch map analysis technique in spatial cognition. However, BDR has some disadvantages. BDR requires that both the sketch map and the target environment have the same number of landmarks, and thus is not suitable for incomplete maps. But memory is often incomplete (Fontaine, Edwards, Tversky, & Denis, 2005)! Researchers have avoided this issue by providing participants with a landmark set prior to map drawing, but this shifts the task demands from “map drawing” to “landmark arrangement.” Using this cued-recall approach can mask differences in landmark information retrieval (i.e., knowing that there is a landmark and/or knowing its identity) between experimental conditions. In addition, BDR requires the extraction of landmark coordinates from the sketch map. This is straightforward for computer-based landmark arrangement tasks, but time-consuming and error-prone for traditional paper-and-pencil sketch maps. Finally, BDR provides measures of overall configural accuracy but does not provide measures indicating how individual landmarks contribute to the overall configuration. This more fine-grained information may prove indispensible in understanding the cognitive underpinnings of spatial knowledge.

Gardony Map Drawing Analyzer

The Gardony Map Drawing Analyzer (GMDA) is a standalone map analysis program with an easy-to-use graphical interface, It provides a range of quantitative measures to better compare sketch maps to target environment layouts. The software compares landmark locations on the participant’s sketch map to those in the target environment. It provides novel quantitative measures reflecting the relative canonical (NSEW) and metrical (distances and angles) placements between landmarks, as well as BDR measures. GMDA supports complex environments (up to 48 landmarks) and, importantly, handles maps with missing landmarks, a shortfall of BDR. Users can specify landmark locations on the sketch map by positioning graphical 2-D points or bounding rectangles with the computer mouse, allowing for intuitive and automated coordinate extraction. The software outputs summary data of the sketch map’s configural accuracy, the coordinates of the target environment’s and sketch map’s landmarks, and a log of the pairwise comparisons to comma-separated-value (.CSV) files that can be readily opened by Excel, SPSS, R, or other data management and analysis software.

Additional functionality for advanced users exists, including the ability to save and reload participants’ landmark configurations, save screenshots, and reanalyze maps en masse. GMDA can be installed on computers running Windows and can be updated and upgraded automatically via an embedded software update module. For downloading and other information, visit www.aarongardony.com/tools/map-drawing-analyzer. The software is copyrighted by the first author and is protected by an open-source end-user license agreement that is distributed with the software. This license agreement grants users a perpetual and nonexclusive license to use, modify, and redistribute the software to suit their needs.

Definition of terms and software modes

Before describing the software in detail, a definition of commonly used terms and a description of the software’s modes is needed. The target environment refers to the environment to which the participant’s sketch map is critically compared. For example, if a participant drew a sketch map of a university campus, the target environment would comprise the university campus itself. The term sketch map always refers to the participant’s sketch map of the target environment. A configuration refers to an arrangement of landmarks in 2-D space.

GMDA has two modes: basic and advanced. The main distinction between these modes is how they represent landmarks. In basic mode, landmarks are represented by a single 2-D point (x, y), denoted as a numeric landmark label (see Fig. 2a). In advanced mode, landmarks are represented by a bounding rectangle, a landmark box (see Fig. 2b). In most cases, basic mode is adequate to analyze sketch maps. However, in some cases advanced mode is preferred. First, landmarks in the target environment may differ substantively in size, and the researcher may want to measure landmark size distortion. Second, certain landmarks, such as roads, may be differently shaped and have different extents than others. Third, landmarks may have containment relationships with each other (e.g., a lake contains a boat, an office contains a desk). Representing landmarks by a single point cannot capture these common cases, and a bounding rectangle is more appropriate for the task. We will explore how advanced mode reflects these map features later.

Fig. 2

(a) In basic mode, landmark locations are represented by a single point, a landmark label. The top left of the landmark label represents the landmark location. (b) In advanced mode, landmark locations are represented by a bounding rectangle, a landmark box

Software workflow

In the following sections, we detail the steps required to analyze sketch maps with GMDA. These steps involve a one-time recording of the target environment’s landmark locations in a coordinates file and a series of steps for each sketch map to be analyzed relative to the target environment. Figure 3 depicts a schematic of this process.

Fig. 3

Schematic of the GMDA analysis procedure

Building a coordinates file

The user first creates a coordinates file, which is a .CSV file containing the Cartesian coordinates of the target environment’s landmarks. The user can build a coordinates file in GMDA by clicking File → New Coordinates File. Coordinates files can contain up to 48 landmarks. The user has two options for creating a coordinates file in GMDA. Users can manually enter landmark names and their coordinates (see Fig. 4a) or can use the graphical interface to arrange landmark labels/boxes on a perfect map of the target environment (see Fig. 4b). Entered coordinates must be Cartesian—that is, increasing x values indicate positioning toward the right, and increasing y values indicate upward positioning (see Fig. 5). There are no numerical limits to the range of possible coordinate values. However, if scaling and/or translation of the configuration is a dependent variable of interest, we recommend setting the coordinate range to ±350. Constructing coordinates files graphically automatically extracts the Cartesian coordinates from the user’s arrangement (coordinate range: ±350). Note that advanced mode does not support manual coordinate entry, because of the complexity and potential for user error when assigning multiple coordinates per landmark. Once a user has built the coordinates file, he or she can use it repeatedly to score maps of the same target environment. Coordinates files are mode-specific: A coordinates file created in basic mode cannot be used for advanced mode analysis, and vice versa. Coordinates files are stored in the local Resources folder, which is found in the same directory as the GMDA executable.

Fig. 4

Building a coordinates file in basic mode using manual (a) and graphical (b) coordinate entry

Fig. 5

Cartesian coordinate system. Note that the coordinates files used by GMDA must adhere to a Cartesian coordinate system

Analyzing a sketch map

Once a coordinates file for the target environment is created, sketch map analysis can begin. This is a three-step process. The user selects the appropriate software mode (basic or advanced), loads the sketch map image and coordinates file, and then arranges the landmark labels/boxes in order to calculate the measures. We describe each of these steps in turn. Figure 6 depicts the software’s home screen, from which sketch map analysis is conducted.

Fig. 6

Example screenshot of GMDA’s home screen

Step 1: Selecting the software mode

The user toggles between modes by clicking the “Basic” and “Advanced” buttons in the center right of the window. Yellow highlighting denotes the currently active mode. Switching modes resets the program to its starting state, removing any loaded sketch map images and resetting the positions of landmark labels/boxes.

Step 2: Loading the sketch map image and coordinates file

Collected sketch maps should be scanned and converted to image files (.JPG is preferable). At this stage, we strongly recommend cropping the sketch map image so that it is square (i.e., the length and width, in pixels, are identical). If this step is not taken, GMDA will automatically rescale the map image to fit the square analysis window and will notify the user. This procedure maintains the aspect ratio when reducing image size to fit the window, which ensures that interlandmark distances and angles in the rescaled sketch map are preserved, but makes positioning landmark labels/boxes more difficult by reducing the size of the workspace. Once the sketch map has been scanned and cropped, the user can load it into GMDA by clicking File → Open Map Image and selecting the image. Once the map image is loaded, the user can load the coordinates file. The last-used coordinates file is preloaded on startup, so the user can skip this step if the intent is to use the same coordinates file from the last session.

Step 3: Arranging labels/boxes and calculating measures

With the sketch map loaded, the next step is to select the target environment to which the map will be compared. The user selects the target environment from the drop-down Map ID menu. This menu contains all the user-created coordinates files stored in the Resource folder. Once the target environment is selected, landmark labels or boxes appear, depending on the software mode. The user then arranges the landmark labels/boxes on the sketch map using the mouse. In basic mode, users simply click and drag each landmark label to its location on the sketch map or double click it to mark a landmark as missing. In advanced mode, the user moves each landmark box by clicking and dragging its center box and manipulating the box’s size and shape using the points on its perimeter (see Fig. 2). Double-clicking the center box marks the landmark as missing. Once the landmark label/boxes are positioned, the analysis controls on the right are activated, and the software is ready to calculate measures of configural accuracy (see Fig. 7). At this point, the user has a few options. The user can rotate the map and labels/boxes using the rotation buttons at the top right. Rotation can be used to correct improperly rotated maps and will influence some (but not all) of the calculated measures. When and how to use this feature will be discussed later. Clicking “Preview” will display the calculated configural measures in a pop-up window but will not save data files. Clicking “Calculate” will save a summary data file containing the calculated measures, and a raw file containing a log of the pairwise comparisons as well as the Cartesian coordinates of the target environment and the sketch map’s landmarks. Files are saved in the local Data folder. It will also automatically save a log of the participant’s landmark configuration in a configuration file located in the local Configurations folder. This file can be used to reload a participant’s landmark configuration for reanalysis.

Fig. 7

Software state prior to conducting an analysis in basic (a) and advanced (b) modes

Additional features

GMDA has several additional features that aid the analysis process. As was touched on previously, each time a data file is saved, a configuration file is also saved. Users can also generate configuration files manually by first arranging landmark labels/boxes and then clicking File → Save Configuration. Saved configurations can be loaded back into GMDA for visualization and reanalysis of previously analyzed landmark configurations (File → Load Configuration). This can be especially useful in troubleshooting unexpected outliers that may have been caused by an erroneous landmark label/box arrangement. Along similar lines, multiple configuration files can be reanalyzed en masse using batch reanalysis (File → Batch Reanalysis). To use this feature, the user first sets the target environment using the drop-down Map ID menu. Then the user selects multiple configuration files representing sketch maps of the target environment. These files are then reanalyzed, producing a new summary and raw file for each processed configuration file. This feature is particularly useful if software updates modify the existing measures or introduce new ones. Finally, the user can take a screenshot of the current software state (File → Take Screenshot).

Specification of configural measures

GMDA calculates two groups of measures: measures unique to GMDA and bidimensional regression (BDR) measures. GMDA-unique measures are calculated to examine the canonical and metrical relationships between landmarks on the sketch map using pairwise comparison. BDR measures are calculated by inputting the Cartesian coordinates of the sketch map’s drawn landmarks and the coordinates of the target environment’s landmarks into BDR equations. The measures detailed below describe computations that operate on landmarks and their (x, y) coordinates.

One important point is that basic and advanced mode differ in how they define a landmark. Recall that in basic mode each landmark is represented by a single 2-D point, whereas in advanced mode each landmark is represented by a landmark box composed of eight peripheral 2-D points (see Fig. 2). For basic mode what constitutes a landmark is intuitive, the (x, y) coordinates of the single point that defines the landmark’s location. For example, basic mode represents the campus center in Fig. 2a (landmark label #8) by a single (x, y) pair. Advanced mode defines a landmark as the (x, y) coordinates of the landmark box’s 8 peripheral points. For example, the landmark box that defines the campus center in Fig. 2b is composed of eight peripheral points, each of which are represented by a single (x, y) pair. Thus, for a given target environment containing n landmarks and a complete sketch map, a basic-mode analysis would process n landmarks, yielding $\left(\begin{array}{c}\hfill n\hfill \\ {}\hfill 2\hfill \end{array}\right)$ pairwise comparisons, and an advanced mode analysis would process 8n landmarks, yielding $\left(\begin{array}{c}\hfill 8n\hfill \\ {}\hfill 2\hfill \end{array}\right)-n\left(\begin{array}{c}\hfill 8\hfill \\ {}\hfill 2\hfill \end{array}\right)$ pairwise comparisons (peripheral points from the same landmark box are not compared). The calculations detailed below are identical between modes but advanced mode is more computationally expensive because it processes more landmark coordinates. In the following section, we provide comprehensive descriptions of GMDA-unique measures (see Appendix A for detailed calculations procedures) and a conceptual introduction to the BDR measures. For more in-depth specification of BDR see Friedman and Kohler (2003).

GMDA-unique measures

GMDA-unique measures are calculated by pairwise comparison of landmarks. The configural measures detailed below are divided into two groups: canonical and metrical.

Canonical measures

Canonical measures compare each landmark’s position relative to all other landmarks using canonical directions (NSEW). These are not strict canonical directions, per se, rather they reflect the positioning of landmarks in Cartesian space. Consider the playground’s slide and carousel in Fig. 1a. In this example because the carousel’s x coordinate is greater and y coordinate is less than the slide’s respective (x, y) coordinates we would define the carousel as North and East of the slide. The North/South (N/S) dimension can also be thought of as mapped to Up/Down (U/D) dimension and East/West (E/W) to Right/Left (R/L), using the map as the referent. To calculate the canonical measures, first each landmark in the target environment is compared to all other landmarks in pairwise fashion. The number of necessary comparisons increases with the total number of landmarks in the target environment (nTL) and is equal to $\left(\begin{array}{c}\hfill nTL\hfill \\ {}\hfill 2\hfill \end{array}\right)$. The software determines the N/S and E/W relationship for each comparison first for the target environment and then for the sketch map. The following canonical measures are calculated on the basis of these comparisons.

Canonical organization & SQRT(canonical organization)

To calculate canonical organization (CanOrg), the software iterates through the $\left(\begin{array}{c}\hfill nTL\hfill \\ {}\hfill 2\hfill \end{array}\right)$ pairwise comparisons and calculates canonical scores by determining if the canonical relationships for each comparison in the sketch map match the target environment. A correct canonical comparison receives one point and an incorrect one receives zero points. Going back to the playground example, we see in Fig. 1a that the carousel is N and E of the slide. In the Fig. 1b sketch map, the carousel is also N and E of the slide. This comparison therefore receives 2 points, one for the N/S and one for the E/W relationship. Importantly, comparisons to a landmark missing from the hand-drawn map are automatically scored as zero. This makes CanOrg distinct from the other measures described below, because it considers and penalizes missing landmarks. The sum of canonical scores (numerator) divided by the number of canonical comparisons (denominator) $\left(2\left(\begin{array}{c}\hfill nTL\hfill \\ {}\hfill 2\hfill \end{array}\right)\right)$ is the CanOrg score. This proportional measure ranges from 0 to 1 with higher scores indicating better configural accuracy and landmark recall.

Researchers who wish to use CanOrg should consider two limitations of the measure. First, whereas CanOrg provides a single measure of configural accuracy and completeness, this property can obfuscate the source of the map’s error. For example, a map with a low CanOrg score may omit several landmarks or alternatively may be relatively complete but misconfigured. Because of this ambiguity, we recommend interpreting CanOrg contextualized with the number of correctly drawn landmarks, which GMDA also reports. Second, as missing landmarks in the sketch map increase, CanOrg scores decrease nonlinearly. Consider a perfectly drawn sketch map of a target environment that contains n landmarks total (nTL). This sketch map would have a CanOrg score of 1. Since landmarks are marked missing from the sketch map the number of drawn landmarks in the sketch map (nDL) decreases. CanOrg_perfect decreases exponentially as nDL decreases and is modeled by the following equation:

$$ CanOr{g}_{perfect}=\frac{\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)}{\left(\begin{array}{c}\hfill nTL\hfill \\ {}\hfill 2\hfill \end{array}\right)} $$

This is problematic for two reasons. First, there is greater variance in CanOrg scores for well-drawn maps than poorly drawn. Second, many statistical tests require measures to be normally distributed. The distribution of CanOrg is often nonnormal and positively skewed. To correct this, GMDA provides a square-root-corrected measure, SQRT(CanOrg), that linearizes the exponential relationship between CanOrg and nDL (see Fig. 8). If one is analyzing maps containing missing landmarks, we recommend using this measure.

Fig. 8

Canonical organization (CanOrg) of a perfectly drawn map as a function of the number of drawn landmarks. As landmarks are marked missing, CanOrg decreases exponentially. Note that SQRT(CanOrg) is preferable, because it linearizes the exponential function

Canonical accuracy

Consider a sketch map that omits several landmarks but accurately represents the landmarks depicted. This sketch map would receive a low CanOrg score because of zero-scoring missing landmarks, which would consequently mask the configural accuracy of the depicted landmarks. Canonical accuracy (CanAcc) accounts for this possibility. CanAcc is calculated identically to CanOrg except CanAcc only considers drawn landmarks. The sum of canonical scores for drawn landmarks (numerator) divided by the number of canonical comparisons for drawn landmarks (denominator) $\left(2\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)\right)$ is the CanAcc score. This proportional measure ranges from 0 to 1 with higher scores indicating better configural accuracy for depicted landmarks.

Metrical measures

The canonical measures reflect a sketch map’s overall completeness and configural accuracy, but an obvious downside is that they lack fine-grained resolution. For example, in the Fig. 1b sketch map the carousel is north and east of the slide and thus the canonical comparisons (N/S & E/W) are correct. However, the interlandmark distance is shorter than in the target environment and the interlandmark angle is rotated clockwise, but by an angle smaller than what would alter the canonical comparison. The canonical measures are not sensitive to such errors. As a result, we designed additional metrical measures. These measures, as with CanAcc, only consider drawn landmarks.

Scaling bias

Scaling bias measures the direction of scaling of interlandmark distances on the sketch map. A sketch map’s interlandmark distances may be compressed (i.e., shorter) or expanded (i.e., longer) relative to the target environment. For example, in Fig. 1, the distance between the slide and the carousel is compressed, and the distance between the sandbox and the seesaw is expanded. Scaling bias measures the direction of these biases in distance representation. It compares the interlandmark Euclidean distances for the $\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)$ landmark comparisons on the sketch map to the equivalent comparisons in the target environment. First the software calculates the maximum interlandmark Euclidean distance for both the sketch map (maxD _SM) ) and the target environment (maxD _TE ). Importantly, maxD _TE only considers landmarks drawn in the sketch map. Then all the interlandmark Euclidean distances in the sketch map (d _SM ) and target environment (d _TE ) are scale-equalized into distance ratios by dividing by their maximum distances, maxD _SM and maxD _TE , respectively. This yields $\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)$ distance ratios for the sketch map and target environment. The software then iterates through each landmark comparison subtracting the target environment’s distance ratio (dr _TE ) from the sketch map’s (dr _SM ). These difference scores (dr _Diff ) are then summated (numerator) and divided by the number of landmark comparisons (denominator) $\left(\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)\right)$. Positive values indicate expansion of interlandmark distances on the sketch map and negative values indicate compression.

Distance accuracy

Distance accuracy measures the accuracy of scaling of interlandmark distances on the sketch map. It is calculated in a similar fashion to scaling bias but considers the magnitude of interlandmark distance error rather than direction. As with scaling bias, distance ratio difference scores (dr _Diff ) between the sketch maps and target environment are calculated. Then, the absolute value of each difference score is computed. These scores are then summed (numerator) and divided by the number of landmark comparisons (denominator) $\left(\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)\right)$. Finally, this error score is subtracted from 1. This proportional measure ranges from 0 to 1 with larger scores indicating more accurate representation of interlandmark distances.

Rotational bias

Rotational bias measures the direction of angular error of interlandmark angles on the sketch map. A sketch map’s interlandmark angles may be rotated clockwise or counterclockwise relative to the target environment. For example, in Fig. 1, the angle between the sandbox and the swing set is rotated clockwise, and the angle between the swing set and the carousel is rotated counterclockwise. Rotational bias measures the direction of these biases in angular representation. It compares the interlandmark angles for the $\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)$ landmark comparisons on the sketch map to the equivalent comparisons in the target environment. First the software calculates the $\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)$ interlandmark angles (in radians) for the sketch map and the target environment. Then the software iterates through each landmark comparison subtracting the target environment’s angle (ang _TE) from the sketch map’s (ang _SM). These angular difference scores (ang _Diff) are then averaged. The averaging procedure is necessarily more complex than computing the arithmetic mean because angles are circular quantities (e.g., 0° ≡ 360°). Each difference score is first converted to an (x, y) coordinate on the unit circle. In other words, the polar coordinates of the angle difference are converted to Cartesian coordinates. Then the x and y coordinates are separately averaged into one x–y pair, $\left(\overline{x},\overline{y}\right)$, which lies on the unit circle. Rotational bias is the resulting angle when $\left(\overline{x},\overline{y}\right)$ is converted back to polar coordinates, the circular mean (Berens, 2009). The range of possible values falls between +180 and –180. Positive values indicate clockwise rotation of interlandmark angles on the sketch map and negative values indicate counterclockwise rotation.

Angle accuracy

Angle accuracy measures the accuracy of interlandmark angles on the sketch map and considers the magnitude of interlandmark angular error rather than direction. As with rotational bias, angular difference scores (ang _Diff) are calculated. However, at this point they are converted to degrees (±180°). The absolute value of each difference score is computed and then the scores are summated (numerator) and divided by the number of landmark comparisons (denominator) $\left(\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)\right)$. This angular error score is then scaled to a proportion by dividing by 180. Lastly this proportion is subtracted from 1. This proportional measure ranges from 0 to 1 with larger scores indicating more accurate interlandmark angle representation.

Measures for individual landmarks

The GMDA-unique configural measures described above address the question “How accurately is the overall map drawn?” Another important question when analyzing sketch maps is “How accurately is each landmark drawn?” For example, a sketch map may be drawn accurately for the most part but contain a few poorly placed landmarks. To address this question, GMDA also provides its unique measures for individual landmarks. For each landmark, the same calculation procedures are used except only pairwise comparisons that contain the landmark are considered. As such, for each landmark in the sketch map, nDL – 1 comparisons enter into the calculation of each measure. Since these calculations only consider drawn landmarks canonical organization is not calculated. The interpretation of canonical accuracy, scaling bias, rotational bias, distance accuracy, and angle accuracy remains the same for an individual landmark. Importantly, the calculations of these measures differ between basic and advanced mode. Recall that for the purpose of the measure calculations, the definition of a landmark differs between basic and advanced modes. In advanced mode, a landmark is defined as the (x, y) coordinates of the eight peripheral points around its landmark box (see Fig. 2b). To account for this, GMDA averages the measures of each landmark’s peripheral points to produce one individual measure for each landmark. Note that the circular mean is used for rotational bias (Berens, 2009). For more detailed description of this procedure, see Appendix A. Another important aspect of these measures is that averaging the GMDA measures of the individual landmarks is equal to the configural measures of that sketch map. This property allows the researcher to examine how specific landmarks contribute to the overall configural scores.

Bidimensional regression and novel applications

GMDA also provides bidimensional regression (BDR) parameters by inputting the Cartesian coordinates of the sketch map’s drawn landmarks and the respective target environment’s landmarks into BDR equations. BDR equations are detailed in the original work (Friedman & Kohler, 2003). Missing landmarks are not scored. As in unidimensional regression, specifying which variable (sketch map or target environment) is the independent variable (IV) and which is the dependent variable (DV) impacts the regression equation. For the purposes of sketch map analysis, Friedman and Kohler advocate inputting the sketch map as the DV because then the parameters and transforms reflect how one’s cognitive map (as measured by the sketch map) was derived from the target environment. For this reason, GMDA enters the sketch map’s coordinates as the DV by default, but users can specify the sketch map as IV in the software’s options if they wish. Here we provide a brief conceptual overview of the BDR parameters and transforms with the sketch map set as the DV. The correlation coefficient, r, measures the degree of resemblance between sets of point configurations. This measure, which hypothetically ranges from 0 to 1, is insensitive to scaling, translation, and rotation of the sketch map relative to the target environment. The interpretation of r is similar to the unidimensional case; values closer to 1 represent a better “fit” between the sketch map’s landmark configuration and the target environment. Further, r ² refers to the proportion of variance of the sketch map’s landmark configuration explained by the target environment’s configuration. Another parameter, distortion index (DI), is closely related to r ². This measure, originally proposed by Waterman and Gordon (1984) and expanded by Friedman and Kohler, denotes the percentage of distortion in the sketch map and is reported as a percentage hypothetically ranging from 0 to 100. DI ² is the proportion of variance of the sketch map’s landmark configuration that remains unexplained by BDR. DI relates to r ² by the following equation: ${\left(\frac{DI}{100}\right)}^2=1-{r}^2$.

BDR also provides other parameters that measure the extent to which the map’s landmark configuration is translated, scaled, and rotated relative to the target environment’s axis system. Two parameters, alpha 1 (α ₁) and alpha 2 (α ₂), denote the extent of horizontal and vertical translation of the sketch map’s environment, respectively. Negative values indicate translation left and down and positive values indicate right and up, respectively. Scale (ϕ) denotes the extent to which the sketch map’s landmark configuration is expanded or contracted. Values greater than 1 indicate expansion and values less than 1 indicate contraction. Angle (theta) (θ) denotes the extent to which the sketch map is rotated. Negative values indicate clockwise rotation and positive values indicate counterclockwise rotation.

GMDA calculates and outputs these configural measures as well as the intermediary parameters that are used in their calculation (beta 1, beta 2, DMax, D; for descriptions, see Friedman & Kohler, 2003). GMDA goes a step further in advanced mode, computing BDR parameters for individual landmarks. Advanced mode is uniquely suited for this task, because each landmark is abstracted by the eight (x, y) coordinates that represent the landmark. Thus, for each landmark, these points can be abstracted as an eight-landmark sketch map and then compared to the corresponding landmark in the target environment, itself an eight-landmark map. For each landmark, GMDA inputs the Cartesian coordinates of the landmark’s eight peripheral points and the corresponding coordinates in the target environment into the BDR equations and outputs the BDR parameters. GMDA reports all of the BDR parameters, but not all are interpretable when applied to single landmarks. We recommend that only r, alpha 1, alpha 2, and scale be interpreted. r captures the extent to which a landmark is correctly shaped. For example, a given landmark in the target environment may be longer than it is wide (e.g., a road), but depicted in the sketch map as wider than it is long. Shape distortion like this would result in a lowered individual r. Similar to their configural interpretations, alpha 1 and alpha 2 denote horizontal and vertical translation of the landmark, respectively, and scale denotes the extent to which a landmark’s size is expanded or contracted. These measures may be used alone or in conjunction with the individual GMDA measures to discover anomalous landmarks exerting a strong influence on the configural measures.

Sketch map rotation: Best practices and its influence on measures

In many cases, a researcher will want to correctly orient the sketch map prior to analysis, orienting the sketch map’s landmark configuration with that of the target environment. This is straightforward when sketch maps are drawn accurately, but not as clear when the sketch map’s landmark configuration is disorganized as compared to the target environment. In either case, rather than subjectively orienting the sketch map, we recommend using a minimize configural theta strategy to correctly orient the sketch map. To do so, first arrange the landmark labels/boxes, and then rotate the sketch map. The rotate buttons (highlighted in Fig. 9a) rotate the sketch map and the arranged landmark labels/boxes in 90° increments (Fig. 9b). After each rotation, preview the configural measures. Select the orientation that yields a theta value closest to 0. Using this orientation will maximize the configural measures that are influenced by sketch map rotation.

Fig. 9

Sketch map rotation buttons (a) and example of the software state following a 90° clockwise rotation (b). Note that the software is shown in basic mode, but the rotation mechanics operate identically in advanced mode

We advocate the minimize configural theta strategy because sketch map rotation influences some of the calculated measures in predictable ways. Table 1 presents the configural GMDA-unique measures and BDR parameters for the four possible rotations (0°, 90°, 180°, and 270°) of the sketch map depicted in Fig. 9. From the table, it is clear that for this sketch map the initial rotation (0°) minimizes theta and is thus adequately oriented relative to the target environment. The table also demonstrates a few key points about how sketch map rotation influences the configural measures. Rotation does not influence scaling bias, distance accuracy, r, scale, or DI. This makes sense, because these measures concern interlandmark distances, which are unaffected by rotation. However, SQRT(canonical organization), canonical accuracy, rotational bias, angle accuracy, alpha 1, alpha 2, and theta are affected by rotation. Specifically, bias values increase and accuracy values decrease when the sketch map is rotated further away from the correct orientation. This is important to consider in experiments in which sketch map orientation is a dependent variable of interest. Such experiments may contain manipulations that promote sketch map rotation in a consistent manner. In these cases, researchers should analyze sketch maps in their drawn orientation, noting rotational bias and theta values, and then minimize configural theta and reanalyze to obtain configural measures unaffected by the sketch map rotation.

Table 1 Configural GMDA-unique measures and bidimensional regression (BDR) parameters as a function of sketch map rotation for the example map depicted in Fig. 9

Our last point on sketch map rotation concerns advanced mode. Advanced mode denotes landmarks with a landmark box, but this poses a problem during map rotation. Landmark boxes possess their own intrinsic orientation, which must be matched with the sketch map orientation. If a sketch map is not correctly oriented, GMDA must match the orientation of the landmark boxes to that of the sketch map. Said another way, GMDA must determine the order to enter the (x, y) coordinates of each landmark box’s peripheral points into the BDR equations, such that the sketch map’s order matches that of the target environment. Consider the landmark box for the campus center in Fig. 2b. GMDA can enter the (x, y) coordinates for this box in four different ways by starting at a corner peripheral point and rotating clockwise until all eight points are entered. To determine the correct landmark box orientation, GMDA cycles through the four coordinate entry options and selects the landmark box orientation that maximizes configural r. This process ensures (to the extent possible) that the landmark boxes’ orientations match the sketch map’s orientation. GMDA then calculates its measures using this orientation.

Validating simulations and experiments

We now present experiments and simulations to validate GMDA and its measures. In Simulation 1, we generated thousands of random sketch maps, scoring them with GMDA in basic mode. We show that the both configural and individual measures follow predictable patterns for random maps and can be readily explained by established statistical distributions. In Experiment 1, we collected maps of a university campus from undergraduates, analyzing them with GMDA. We show that GMDA and BDR measures are inter- and intracorrelated and demonstrate how the measures pattern together to measure distance representation and angular configuration. In Simulation 2, we explored how advanced mode handles special cases in which sketch maps violate interlandmark size, shape, and containment relationships and demonstrated how the individual landmark measures can assess individual landmark contributions to overall sketch map accuracy. Finally, we discuss different ways that researchers can use GMDA to generate inferences about spatial mental representations. Together, this work validates GMDA, showing that its measures provide a comprehensive and reliable picture of sketch map accuracy.

Simulation 1

Random map generation

We first created a random 24-landmark target environment, generating random (x, y) coordinates for each landmark. We chose 24 landmarks because this number provides a sufficiently complex target environment for simulation and lies in the middle of the possible numerical range of landmarks GMDA supports (up to 48 landmarks). The landmarks’ (x, y) coordinates randomly varied between –350 and +350. This range permitted the randomly generated target environment to match the dimensions of the sketch map window in the software (700 × 700 pixels). We randomly generated several candidate target environments and selected one in which landmarks were sufficiently spread out. This target environment was used in all subsequent simulations.

Next, we created 1,000 random configurations of the 24 landmarks (i.e., sketch maps) in similar fashion. In each configuration the landmarks’ (x, y) coordinates randomly varied between –350 and +350. We then created three additional randomly generated 1,000 map sets in which we manipulated the number of drawn landmarks (nDL) as a subset of the target landmarks: 18, 12, and 6. In each configuration, we randomly selected nDL landmarks that were then randomly placed as before. The nonselected landmarks were marked missing. In total, we created four sets of 1,000 randomly generated sketch maps that reflect different levels of landmark recall: 100 %, 75 %, 50 %, and 25 % drawn. Finally, we analyzed the map sets in reference to the randomly generated target environment using GMDA’s batch analysis function.

Distribution fitting

We calculated descriptive statistics for the GMDA-unique and BDR configural measures for the four map sets and visually examined the distributions using histograms. For all measures the histograms resembled documented continuous statistical distributions. As a result, we tested the fit of our simulated data sets against these best-fit distributions with the Kolmogorov–Smirnov (KS) test, using the fitdistrplus package, version 1.0-2, of the R environment for statistical computing (Delignette-Muller, Pouillot, Denis, & Dutang, 2010). We selected a stringent alpha level of .01 to prevent Type I errors, given the large sample size of our simulated data sets. Table 2 presents descriptive statistics and results of our distribution fitting procedure. We note that despite the stringent alpha level of the KS tests, some measures yielded a significant KS statistic, suggesting that these measures are not explained by their respective best-fit distributions. In these cases, we tested additional candidate distributions but did not find other distributions that yielded smaller KS statistics. Therefore, we maintain that our selected distributions best explain the data and that the significant KS statistics likely stem from the large sample size of our simulations. We also conducted the same procedure for the GMDA-unique individual measures, calculating descriptive statistics and viewing distributions for one of the 24 landmarks in the four map sets. Descriptive statistics and distributions for the individual landmarks resembled those observed for the configural measures in all map sets. Thus, our discussion of the configural measures applies to the individual measures as well.

Table 2 Descriptive statistics of configural GMDA-unique and BDR measures from random sketch map simulation, best fitting distributions, and observed Kolmogorov–Smirnov (KG) statistics (four total simulations [n = 1,000 randomly generated maps] were run, manipulating the percentage of total maps drawn [100, 75, 50, 25])

Table 2 reveals several important features of the simulations. First, in all cases the configural measure distributions were explained by established statistical distributions. Notably, all GMDA-unique measures were normally distributed except rotational bias, which was uniformly distributed. Furthermore, the distributions of canonical organization, accuracy, and angle accuracy were centered at .5, suggesting that for these measures a score of .5 reflects chance performance when the sketch map is correctly oriented. Rotational bias’s uniform distribution is intuitive, because all rotations should be equally probable with randomly generated maps. Second, the measures maintained their best-fit distributions as the percentage of landmarks drawn decreased. Canonical organization, which is penalized for missing landmarks, decreased as the percentage of landmarks drawn decreased, but still maintained its normal distribution. Other measures, which only score drawn landmarks, maintained both their means and distributions. It should be noted that as the percentage of landmarks drawn decreased, variance increased for measures of the drawn landmarks, but decreased for canonical organization. Interestingly, r appeared to increase as the percentage of landmarks drawn decreased. We explored this further by simulating additional map sets ranging between two and 24 drawn landmarks and calculating configural measures. Figure 10 depicts how the configural accuracy measures varied as a function of the number of drawn landmarks. The GMDA-unique measures maintained stable values as the number of drawn landmarks decreased, but r increased following a pareto distribution. Because of this, we do not recommend using r for maps containing few landmarks (<8). Overall, Table 2 demonstrates that GMDA’s configural measures and their distributions follow predictable patterns, reflecting their reliability as measures of sketch map configural accuracy.

Fig. 10

Mean configural measures for randomly generated sketch maps (n = 1,000) as a function of the number of drawn landmarks (range: 2–24)

Experiment 1

Simulation 1 demonstrated how GMDA-unique and BDR measures vary for randomly generated sketch maps. However, we would be remiss to generalize our Simulation 1 findings to the maps that researchers would collect and analyze using GMDA. Indeed, sketch maps of well-learned environments should be far more accurate than randomly generated maps. Therefore, we collected sketch maps of the Tufts University (TU) campus from TU undergraduates to examine the relationships between the software’s measures in more systematic maps.

Method

Participants

A total of 100 TU undergraduates (M age = 19.7, 52 male) participated for monetary compensation. The sample was evenly split between class years (first-year, sophomore, junior, senior).

Materials

We created an 8.5 × 11 in. map drawing sheet that consisted of a centered 7.3 × 9 in. rectangle in which participants drew their maps. This sheet also provided a single intersection and an orienting arrow to indicate the uphill direction (TU is located on a hill). We created a basic mode coordinates file in GMDA that contained 36 TU landmarks and manually entered their Cartesian-transposed latitude and longitude coordinates obtained from Google Earth (Google, Inc., 2014).

Procedure

Participants drew a sketch map of the TU campus, from memory, on a map-drawing sheet, taking as long as needed to complete the task. The approximate average completion time was 10 min.

Analysis

We scored the maps using GMDA in basic mode. We did not correct rotation for any of the maps, because the provided drawing sheet included orienting information. We divided the resulting configural measures into two conceptual groups: bias measures and accuracy measures. Bias measures reflect aspects of the sketch map that are directional in nature, such as rotation or scaling. The bias measures include scaling bias, rotational bias, scale, and theta. Accuracy measures reflect the sketch map’s overall nondirectional accuracy/error. These measures decrease equivalently with errors in either direction. The accuracy measures include SQRT(canonical organization), canonical accuracy, distance accuracy, angle accuracy, r, and DI. We then computed descriptive statistics and bivariate correlations for both the bias and accuracy measures (see Table 3), using circular statistics when appropriate (Berens, 2009).

Table 3 Descriptive statistics and correlation matrix of configural map measures obtained from university campus maps. Measures are conceptually divided into two groups: bias measures (1–4) and accuracy measures (5–10)

Descriptive statistics

Table 3 reveals several notable aspects of the map measures calculated from a real map set. First, the measures maintained the best-fit distributions found in Simulation 1. Using the same procedure from Simulation 1, we fit the Table 2 distributions to the TU map set. All of the measures, except rotational bias and theta, maintained their respective best-fit distributions (all KS statistics < KS_crit = 0.1358, α = .05). It should be noted that both r and DI switched the direction of their skew, with r being negatively skewed and DI being positively skewed. This reflects that, in general, participants in this sample produced high-quality sketch maps, a clear contrast with the random maps in Simulation 1. Rotational bias and theta were not uniformly distributed, as in Simulation 1, but instead were normally distributed, as confirmed by the KS test (α = .05). This finding makes sense, given that participants received orienting information, greatly reducing the sketch map rotation variance.

Correlations between bias measures

Rotational bias was highly correlated with theta (circular–circular correlation coefficient = ρ _cc = –.963), and the means were nearly reciprocals of one another, –28.6 and 27.5, respectively. This suggests that the measures are functionally the same. We explored this assumption further by calculating the same correlation for the random map set from Simulation 1 (nDL = 24). This analysis also revealed a significant correlation (ρ _cc = –.905). Taken together, these findings strongly suggest that rotational bias and theta index the same rotational metric. This is important to consider because rotational bias, unlike theta, can be calculated for individual landmarks. Another important point is that bias measures indexing distances are dissociable from those indexing rotation. Scaling bias and scale are uncorrelated with rotational bias and theta. Interestingly, scaling bias was not correlated with scale suggesting these two measures reflect different aspects of sketch map distance representation, with the former representing the scaling of interlandmark distances and the later representing the scaling of the entire landmark configuration.

Correlations between accuracy measures

In general, the GMDA-unique and BDR accuracy measures were inter- and intracorrelated with some exceptions. Notably, distance accuracy was correlated with neither canonical accuracy nor angle accuracy. This suggests that distance accuracy uniquely indexes distance representation, whereas canonical and angle accuracy index angular configuration. However, distance accuracy correlated with SQRT(canonical organization), suggesting that more complete maps also possessed better distance representation. Furthermore, distance accuracy was most highly intercorrelated with the BDR accuracy measures, suggesting that they too index distance representation.

Principal components analysis

Those measures that correlate likely reflect similar aspects of the sketch map. To further explore this point, we conducted principal components analyses (PCAs) separately for the bias and accuracy measures. Table 4 presents the extracted components and their factor loadings, and Table 5 presents the factor loadings for the components whose eigenvalues exceeded 1. For both the bias and accuracy measures, two components appeared to index distance representation and angular configuration, on the basis of their loadings. The correlative nature of the map measures can pose problems for analyses that assume noncollinearity, such as multiple regression. Because of this, we recommend either choosing representative measures of each factor or creating composite scores when conducting statistical analyses.

Table 4 Extracted components, eigenvalues, and percentage of variance explained from principal components analyses of the bias and accuracy measures

Table 5 Factor loadings of extracted components from principle components analyses of bias and accuracy measures

Simulation 2

Our final validating simulations explored how GMDA’s basic and advanced modes differ and the utility and interpretation of individual landmark measures. Recall that advanced mode denotes landmarks with a bounding rectangle, a landmark box. Rectangles have both variable size and shape, as compared to a single point (i.e., basic mode), and thus are presumed to reflect unique landmark features. These features are (1) variations in landmark size, (2) variations in landmark shape, and (3) landmark containment relationships. In Simulation 2, we provided example maps and demonstrated that advanced mode is capable of detecting these features. Then we consider the individual landmark measures, demonstrating that they capture differences in individual landmark placement and how individual landmarks contribute to a sketch map’s overall configural scores.

Landmark size distortion

We first examined whether advanced mode could detect landmark size distortion in sketch maps. Because basic mode represents landmarks with a single (x, y) point, it cannot detect landmark size distortions. Consider a sketch map in which the center position of each landmark is identical to the landmark’s position in the target environment, but landmark size varies. In this case, size distortion would not change the (x, y) coordinates in basic mode, but would affect the size of the landmark boxes in advanced mode. To test this, we created a simple target environment consisting of four rectangles and a sketch map in which we manipulated landmark size while holding position constant (see Fig. 11). We then analyzed the map in basic and advanced mode. Even though this sketch map’s landmarks differed in size from the target environment, basic mode awarded it perfect scores because the center position, which is marked by a cross in the figure, was unchanged. In contrast, both advanced mode’s configural and individual landmark measures decreased, reflecting the size distortion (see Table 6). Notably, the individual landmark BDR parameter scale captured the direction and magnitude of the size distortion. Scaled-up landmarks and scaled-down landmarks possessed individual scale values greater and less than 1, respectively.

Fig. 11

Sketch map (black) in which size (but not position) has been manipulated relative to the target environment (gray). Landmarks 1 and 4 have been scaled down, and Landmarks 2 and 3 have been scaled up. Crosses denote the center positions of landmarks

Table 6 Configural and individual landmark measures, calculated in advanced mode, for sketch map with landmark size distortion (Fig. 11)

Landmark shape distortion

We next examined whether advanced mode could detect landmark shape distortion. Again, basic mode is ill-suited for this task, because a landmark’s shape can change without affecting its center position. To test this, we created a simple target environment consisting of four rectangles. We then created a sketch map in which we manipulated landmark shape while holding position constant (see Fig. 12), analyzing the map in both software modes. As with size distortion, basic mode was unable to detect shape distortion in this map, giving it perfect scores. In contrast, both advanced mode’s configural and individual landmark measures decreased, reflecting the shape distortion (see Table 7). Notably, the individual landmark BDR parameter r captured the magnitude of the shape distortion.

Fig. 12

Sketch map (black) in which shape (but not position) has been manipulated relative to the target environment (gray). Landmarks 1 and 4 have been compressed vertically, and Landmarks 2 and 3 have been compressed horizontally. Crosses denote the center positions of landmarks

Table 7 Configural and individual landmark measures, calculated in advanced mode, for sketch maps with landmark shape distortion (Fig. 12)

Landmark containment violation

Finally, we examined whether advanced mode could detect violations of containment relationships. This is again problematic for basic mode, because containment relationships between landmarks can change while the landmarks’ center positions are maintained. To test this, we created a target environment consisting of four rectangles that were nested within each other (see Fig. 13a). We then created a sketch map in which we swapped the containment relationships between landmarks (ex. Landmark 1 CONTAINS Landmark 2 → Landmark 2 CONTAINS Landmark 1) while maintaining each landmark’s center position (see Fig. 13b), analyzing the map in both software modes. As expected, basic mode was unable to detect the containment violations, giving it perfect scores. In contrast, both advanced mode’s configural and individual landmark measures decreased, reflecting the containment violation (see Table 8).

Fig. 13

Sketch map (b) in which interlandmark containment relationships (but not position) have been manipulated relative to the target environment (a). In contrast to the target environment, in the sketch map Landmark 2 contains Landmark 1, and Landmark 4 contains Landmark 3. Crosses denote the center positions of landmarks

Table 8 Configural and individual landmark measures, calculated in advanced mode, for sketch maps with containment violation (Fig. 13)

Applications of advanced mode

Given advanced mode’s ability to detect these map features and the additional calculation of individual landmark BDR parameters, one might wonder why not use advanced mode all the time? We offer two reasons that researchers should prefer basic mode. First, analyzing a sketch map in advanced mode is more time-consuming and computationally intensive. Resizing and shaping landmark boxes necessarily takes more time than does positioning landmark labels. Choosing advanced mode can slow analysis considerably for a large set of sketch maps. Furthermore, advanced mode requires more pairwise comparisons than basic mode ($\left(\begin{array}{c}\hfill 8\cdotp nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)-nTL\left(\begin{array}{c}\hfill 8\hfill \\ {}\hfill 2\hfill \end{array}\right)$ vs. $\left(\begin{array}{c}\hfill nTL\hfill \\ {}\hfill 2\hfill \end{array}\right)$), and is thus more computationally expensive. However, on the basis of our testing, even complex maps containing 48 landmarks only took a few seconds to process. Second, when analyzing maps that do not possess (or possess few of) the previously discussed map features, basic and advanced analyses yield similar results. Because of the added temporal demands and redundancy with basic mode in most cases, we recommend using advanced mode only if sketch maps contain the previously discussed map features and the features are relevant to the experimenter’s research questions.

Individual landmark measures

An important advantage of GMDA over previous sketch map analysis approaches is the inclusion of configural measures for individual landmarks. GMDA calculates its unique measures for individual landmarks that reflect their placement accuracy. To demonstrate this, we first created a target environment: an office building floor that included four offices, each with two objects. We defined each object, office, and the floor as landmarks, thus including all levels of this environment’s hierarchy, for a total of 13 landmarks. We then created a sketch map of this office floor in which some landmarks were correctly placed, whereas others were not. Specifically, we moved two objects out of their office and moved one office to a new location, keeping the objects in place inside it. Figure 14 depicts the sketch map and target environment. We then analyzed the sketch map in basic mode. All GMDA-unique measures were calculated for individual landmarks, but for simplicity we will only discuss canonical accuracy. Figure 15 depicts the canonical accuracy (CA) for each landmark. Incorrectly relocated landmarks had lower individual canonical accuracy scores, reflecting their inaccurate placement (CA_table = .67, CA_computer = .5, CA_{office 3} = .29). Importantly, the mean of the GMDA-unique individual landmark measures equaled the overall configural measures (circular mean for rotational bias). Thus, we can interpret the magnitude of a landmark’s deviation from this mean as a measure of that landmark’s influence on the overall configural measure. There are different ways to calculate this influence. We offer one suggestion here.

Fig. 14

Sketch map of an office target environment. Most landmarks are in identical positions relative to the target environment. However, Office 4’s objects and Office 3 have been relocated (target environment landmark locations are shown in gray)

Fig. 15

Canonical accuracy for individual landmarks in the office sketch map. Relocated landmarks are shown in red

$$ LandmarkInfluence=100\times \frac{\left| individual\kern0.5em landmark\kern0.5em measure- configural\kern0.5em measure\right|}{{\displaystyle \sum_1^n\left| individual\kern0.5em landmark\kern0.5em measure- configural\kern0.5em measure\right|}} $$

Using this formula, we see that Office 3’s position exerts the most influence on configural canonical accuracy (28 %), whereas the table and computer exert less—6 % and 16 %, respectively. So Office 3 was the largest contributor to the canonical accuracy error. As is evident from this example, the GMDA-unique individual landmark measures are useful for detecting erroneously placed landmarks and their influence on the overall sketch map configuration. In addition, they can be used to distinguish well-learned landmarks in a set of sketch maps, perhaps discovering environment “hubs” and/or landmarks that are consistently placed incorrectly. Thus, the individual measures, used in conjunction with the configural measures, can yield a more complete psychometric picture of sketch map accuracy.

Using GMDA for inference

GMDA yields multiple dependent measures that provide several options for data analysis, but this plurality can overwhelm the first-time user, making mapping the measures to inferences difficult. In this section, we discuss how GMDA can be used to explore different research questions (applications of BDR have been discussed previously, see Friedman & Kohler, 2003). GMDA’s measures have many overlapping properties that lend themselves well to inference (see Table 9 for a summary). First, recall that we conceptually divided the measures into bias measures that reflect directional aspects of the sketch map, and accuracy measures that reflect nondirectional accuracy/error. Researchers may investigate bias measures if they are interested in research questions with directional predictions. For example, does an experimental manipulation cause participants to rotate their mental representations clockwise (rotational bias and theta) or scale up interlandmark distances (scaling bias)? Conversely, accuracy measures (canonical accuracy and r) can address nondirectional research questions, such as whether a manipulation affects map quality as a whole. Second, concerning the GMDA-unique measures, we distinguished between the canonical measures (canonical organization/accuracy) and the metrical measures (scaling/rotational bias, distance/angle accuracy). This distinction captures mental representation granularity. The canonical measures consider relative N/S/E/W placement, but not fine-grained distance and angular relationships as the metrical measures do. Researchers can exploit this distinction, exploring how manipulations influence the coarse-grained versus fine-grained development of spatial mental representations, by comparing the canonical and metrical measures. Third, our PCA revealed that the dependent measures group into two components that index distance and angular representation. Given this grouping, it appears that GMDA is well-suited for research questions exploring how these different types of representations build during spatial mental representation formation. Fourth, GMDA provides both unique measures and BDR parameters, and Experiment 1 demonstrated that GMDA and BDR measures are correlated. Researchers may be unsure which measures to use. As a general rule, when considering overall map quality (i.e., configural measures), BDR is appropriate (r is typically reported). However, if the target environment contains few landmarks or the map is missing several landmarks, GMDA-unique measures are more appropriate. These measures maintain stable values as the number of drawn landmarks decreases (see Fig. 10). Finally, GMDA yields its unique measures for individual landmarks, a significant addition to existing techniques. Researchers can use these measures to examine questions concerning representations of individual landmarks. For example, does manipulating the saliency of landmarks influence representational accuracy for those landmarks? Furthermore, using the individual BDR parameters (r, scale) in advanced mode can reveal subtle distortions in individual landmarks. For example, are more salient landmarks distorted in shape (r) or increased in size (scale) in one’s mental representation? We believe that the research questions surveyed above scratch the surface of a broad body of potential research that GMDA can address. Indeed, GMDA has great potential as an inference tool across a wide variety of cognitive domains.

Table 9 Properties of GMDA’s calculated measures for drawn landmarks

Discussion

The simulations and experiments detailed above validate the Gardony Map Drawing Analyzer’s stable properties for both randomly generated and real sketch maps. In Simulation 1, we showed that when scoring random maps, GMDA and bidimensional regression (BDR; Friedman & Kohler, 2003) measures follow established statistical distributions and maintain these distributions when increasing numbers of landmarks are missing from the sketch map. In Experiment 1, we scored a large set of sketch maps from a well-learned environment (a university campus) with GMDA. We showed that the explanatory distributions for the randomly generated maps in Simulation 1 continued to best explain the data. We also found that the measures were highly correlated, and a subsequent PCA revealed that the measures group into two components of map accuracy, distance representation and angular configuration. Finally, in Simulation 2, we demonstrated that GMDA’s advanced mode is able to capture unique features of sketch maps: variations in landmark size and shape, and interlandmark containment violations. Furthermore, we showed that GMDA’s unique individual landmark measures are able to measure how individual landmarks contribute to a sketch map’s overall configural scores. Taken together, the simulation and experimental results show that GMDA provides configural and individual landmark measures that reliably index sketch map quality.

Limitations

Despite its advantages over current techniques, GMDA has limitations. First, in GMDA landmarks necessarily couple identity and location memory. However, previous research has shown that identity and location memory are distinct and can be measured separately (Hasher & Zacks, 1979; Naveh-Benjamin, 1987; Treisman & Zhang, 2006; Vogel, Woodman, & Luck, 2001). Consider a map in which an outline of one landmark is drawn but the identity label is omitted. In this case, the participant remembered the location of the landmark but could not recall its identity. Researchers using GMDA to analyze this map would have no choice but to mark this landmark missing, even though it is clear that the participant had some memory of the landmark. Second, we designed GMDA’s interface to be compact in order to fit on present-day computer monitors, which tend to have at least 900 pixels of vertical and horizontal resolution. However, the 700 × 700 pixel sketch map window is often cramped when scoring sketch maps containing several landmarks. Third, GMDA’s sketch map rotation functionality currently only supports rotation in 90° increments. However, it is likely that researchers will collect sketch maps that are rotated in non-right-angle orientations. In cases in which researchers wish to correct such maps’ orientations to 0°, we recommend rotating the map images with external image manipulation software. Fourth, advanced mode’s landmark boxes provide advantages over basic mode for certain map features, such as differential landmark size, shape, and containment. However, they are represented by rectangles, and thus are not well-suited for certain landmarks, such as oddly shaped or diagonally oriented landmarks and complex routes, such as windy roads. In these cases, the researcher can position landmark boxes to surround the landmarks, but will inevitably include nonlandmark space and/or other landmarks within the bounding box. Finally, GMDA also exhibits a limitation of map drawing in general: Accurate map drawing depends in part on drawing ability. Differential drawing ability in participants thus could confound the map-drawing data (Davies & Pederson, 2001; Golledge, 1976). Because of this, researchers should keep in mind that drawing ability will add variance to sketch map data.

Spatial cognition specific and domain-general applications

GMDA has several immediate applications in spatial cognition research, as well as the potential for use in a wide array of cognitive domains. At its most basic, GMDA can be used to measure differences in spatial memory between participant groups and the influence of experimental manipulations on sketch map accuracy. Researchers can also use GMDA, specifically the individual landmark measures, to measure differences in landmark knowledge and how experimental manipulations influence the development of this knowledge. More generally, intermittent map drawing during learning of a novel environment can reveal how spatial memory development unfolds. Because GMDA’s measures group into distance representation and angular configuration components, researchers can examine how these components of spatial memory develop and whether and how experimental manipulations facilitate or impair these components. Additionally, should a researcher wish to use a landmark arrangement task (commonly adopted in BDR-based experiments), GMDA’s graphical user interface allows participants to “draw” their map directly in the software. Participants can then manipulate landmark labels/boxes on a map template image, effectively removing the need for researchers to analyze maps. In this way, GMDA can be used not only as a data analysis tool, but also a data collection tool.

Turning to more domain-general applications, GMDA can be used to score any configural representation, such as hand drawings. Drawing tasks are used in several cognitive domains. For example, the clock-drawing task (CDT) is one of the most widely used tests of neuropsychological function in clinical domains (Rabin, Barr, & Burton, 2005; Shulman, 2000). However, there is great variation in how the CDT is scored (Freedman et al., 1994). GMDA could be used to quantitatively assess the quality of clock drawings, enriching current CDT qualitative analysis techniques. Another possible application of GMDA and its measures is in eyetracking. Eyetracking experiments often yield “fixation maps” in which (x, y) points corresponding to fixation locations are recorded on a visually presented image. GMDA can be used to measure configural differences between fixation maps or between fixation maps and predefined regions of interest (ROIs). Furthermore, advanced mode could be used to measure the scaling of fixation areas relative to ROIs. GMDA can be similarly applied to multidimensional scaling (MDS) representations. MDS represents measurements of similarity (or dissimilarity) among pairs of items as distances between points in a low-dimensional space (Borg & Groenen, 2005; Cox & Cox, 2000). Two-dimensional MDS representations depict points in 2-D space and are thus well-suited for comparison with GMDA. These are just some of the possible domain-general applications of GMDA, but it is clear that GMDA, like BDR, can be applied in any context in which 2-D data sets are compared.

Conclusions

Sketch maps offer a window into the mental representation of space, but computational approaches to sketch map analysis are presently inconsistent and inadequate. Researchers have approached this problem either by relying on subjective evaluation of maps or bidimensional regression (Friedman & Kohler, 2003). The former is time-intensive and sensitive to bias. The latter, though computational and widely accepted, requires complete landmark knowledge and landmark coordinate extraction, and it cannot accommodate nested landmarks. In the present article, we presented the Gardony Map Drawing Analyzer, a sketch map analysis software package, and described its operation, measures of configural and individual landmark placement, and validating simulations and experiments. We showed that GMDA yields both BDR measures and novel GMDA-unique measures that are reliable and reflect aspects of spatial mental representations, such as canonical relationships, distances, and angles. In addition to its novel measures, GMDA introduces new analysis techniques, such as the calculation of GMDA-unique individual landmark measures and BDR individual landmark measures. In our view, this combination of analyses at the configural and landmark levels positions GMDA as the current most comprehensive approach to sketch map analysis. Our hope is that researchers across cognitive science disciplines will find our software useful in revealing the subtle intricacies of spatial memory development.

Appendix A: Procedures and formulae for calculating GMDA-unique measures

Canonical organization (CanOrg)

nTL :

number of landmarks in target environment

$$ \begin{array}{cc}\hfill \mathrm{basic}\ \mathrm{mode}:n=\left(\begin{array}{c}\hfill nTL\hfill \\ {}\hfill 2\hfill \end{array}\right)\hfill & \hfill \mathrm{advanced}\ \mathrm{mode}:n=\left(\begin{array}{c}\hfill 8\cdotp nTL\hfill \\ {}\hfill 2\hfill \end{array}\right)-nTL\left(\begin{array}{c}\hfill 8\hfill \\ {}\hfill 2\hfill \end{array}\right)\hfill \end{array} $$

Calculation procedure for calculating canonical scores:

For each of the n landmark comparisons:

• 1. If the N/S placement of the sketch map’s landmark pair is correct, award 1 point

• 2. If incorrect, award 0 points

• 3. If one (or both) landmarks are missing from sketch map, award 0 points

• 4. Repeat Steps 1–3 for the E/W dimension

$$ CanOrg=\frac{{\displaystyle \sum_{i=1}^ncanonicalscor{e}_i}}{2n} $$

SQRT( CanOrg )

$$ SQRT(CanOrg)=\sqrt{CanOrg} $$

The following measures can be calculated for the overall map (configural) or for each individual landmark (individual):

nDL :

number of drawn landmarks in the sketch map

$$ \begin{array}{cc}\hfill \mathrm{basic}\ \mathrm{mode}:n=\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)\hfill & \hfill \mathrm{advanced}\ \mathrm{mode}:n=\left(\begin{array}{c}\hfill 8\cdotp nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)-nTL\left(\begin{array}{c}\hfill 8\hfill \\ {}\hfill 2\hfill \end{array}\right)\hfill \end{array} $$

The calculation procedure is identical to that for CanOrg, except that comparisons containing missing landmarks are omitted from the calculation.

Calculation procedure for calculating canonical scores:

For each of the n landmark comparisons containing 2 drawn landmarks:

• 1. If the N/S placement of the sketch map’s landmark pair is correct, award 1 point

• 2. If incorrect, award 0 points

• 3. If one (or both) landmarks are missing from sketch map, award 0 points

• 4. Repeat Steps 1–3 for the E/W dimension

$$ \begin{array}{c}\kern1em CanAc c=\frac{{\displaystyle \sum_{i=1}^ncanonicalscor{e}_i}}{2n}\kern1em \\ {}\kern.5em CanAc{c}_{individual}=\frac{{\displaystyle \sum_{i=1}^ncanonicalscor{e}_i}}{2n}\mathrm{where}{n}_{basic}=nDL-1,{n}_{advanced}=8\cdot nDL-{8}^{\ast}\end{array} $$

* Note: In advanced mode, the software reports the mean of the calculated individual measures of the landmark’s 8 peripheral points.

d _SM :

interlandmark Euclidean distance for a pairwise landmark comparison in sketch map

d _TE :

interlandmark Euclidean distance for a pairwise landmark comparison in target environment

maxD _SM :

maximum interlandmark distance in sketch map

maxD _TE :

maximum interlandmark distance in target environment, only drawn landmarks are considered

dr _SM :

distance ratio for a pairwise landmark comparison in sketch map

dr _TE :

distance ratio for a pairwise landmark comparison in target environment

dr _Diff :

difference score between distance ratios for sketch map and target environment

nDL :

number of drawn landmarks in the sketch map

$$ \begin{array}{cc}\hfill \mathrm{basic}\ \mathrm{mode}:n=\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)\hfill & \hfill \mathrm{advanced}\ \mathrm{mode}:n=\left(\begin{array}{c}\hfill 8\cdotp nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)-nTL\left(\begin{array}{c}\hfill 8\hfill \\ {}\hfill 2\hfill \end{array}\right)\hfill \end{array} $$

Calculation procedure for calculating distance ratios:

For each of the n pairwise landmark comparisons:

• 1. d _SM , maxD _SM , d _TE , and maxD _TE are calculated using the distance formula and the (x, y) coordinates of the sketch map’s and target environment’s landmarks, respectively.

• 2.
  $$ d{r}_{SM}=\frac{d_{SM}}{max{D}_{SM}} $$
• 3.
  $$ d{r}_{TE}=\frac{d_{TE}}{max{D}_{TE}} $$
• 4.
  $$ d{r}_{Diff}=d{r}_{SM}-d{r}_{TE} $$

$$ \begin{array}{c}\kern1em ScalingBia s=\frac{{\displaystyle \sum_{i=1}^nd{r}_{Dif{f}_i}}}{n}\kern1em \\ {}\kern1em ScalingBia{s}_{individual}=\frac{{\displaystyle \sum_{i=1}^nd{r}_{Dif{f}_i}}}{n}\mathrm{where}{n}_{basic}=nDL-1,{n}_{advanced}=8\cdot nDL-{8}^{\ast}\kern1em \\ {}\kern1em DistanceAccurac y=1-\frac{{\displaystyle \sum_{i=1}^n\left|d{r}_{Dif{f}_i}\right|}}{n}\kern1em \\ {}\kern1em DistanceAccurac{y}_{individual}=1-\frac{{\displaystyle \sum_{i=1}^n\left|d{r}_{Dif{f}_i}\right|}}{n}\mathrm{where}{n}_{basic}=nDL-1,{n}_{advanced}=8\cdot nDL-{8}^{\ast}\kern1em \end{array} $$

* Note: In advanced mode, the software reports the arithmetic mean of the calculated individual measures of the landmark’s eight peripheral points.

Rotational bias and angle accuracy

L1 and L2 represent the two landmarks in each pairwise landmark comparison, the SM subscript refers to the sketch map’s landmarks, and TE to the target environment’s.

.x and .y refer to the x and y coordinates representing the location of a landmark

ang _SM :

interlandmark angle (in radians) for a pairwise landmark comparison in sketch map

ang _TE :

interlandmark angle (in radians) for a pairwise landmark comparison in target environment

ang _Diff :

difference score (in radians) between angles for sketch map and target environment

nDL :

number of drawn landmarks in the sketch map

$$ \begin{array}{cc}\hfill \mathrm{basic}\ \mathrm{mode}:n=\left(\begin{array}{c}\hfill nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)\hfill & \hfill \mathrm{advanced}\ \mathrm{mode}:n=\left(\begin{array}{c}\hfill 8\cdotp nDL\hfill \\ {}\hfill 2\hfill \end{array}\right)-nTL\left(\begin{array}{c}\hfill 8\hfill \\ {}\hfill 2\hfill \end{array}\right)\hfill \end{array} $$

Note that in the following equations, atan2 accepts parameters in ( y , x ) order, as is conventional in several programming languages.

For each of the n pairwise landmark comparisons:

• 1.
  $$ an{g}_{SM}= atan2\left(L{2}_{SM}.x-L{1}_{SM}.x,L{2}_{SM}.y-L{1}_{SM}.y\right) $$
• 2.
  $$ an{g}_{TE}= atan2\left(L{2}_{TE}.x-L{1}_{TE}.x,L{2}_{TE}.y-L{1}_{TE}.y\right) $$
• 3.
  $$ an{g}_{Diff}= an{g}_{SM}- an{g}_{TE} $$

$$ \begin{array}{c}\kern1em RotationalBia s=\left(\frac{180}{\pi}\right) atan2\left(\frac{{\displaystyle \sum_{i=1}^n \sin an{g}_{Dif{f}_i}}}{n},\frac{{\displaystyle \sum_{i=1}^n \cos an{g}_{Dif{f}_i}}}{n}\right)\kern1em \\ {}\kern1em RotationalBia{s}_{individual}=\left(\frac{180}{\pi}\right) atan2\left(\frac{{\displaystyle \sum_{i=1}^n \sin an{g}_{Dif{f}_i}}}{n},\frac{{\displaystyle \sum_{i=1}^n \cos an{g}_{Dif{f}_i}}}{n}\right)\mathrm{where}{n}_{basic}=nDL-1,{n}_{advanced}=8\cdot nDL-{8}^{\ast}\kern1em \\ {}\kern1em AngleAccurac y=1-\left(\frac{{\displaystyle \sum_{i=1}^n\left|\left(\frac{180}{\pi}\right) an{g}_{Dif{f}_i}\right|}}{180n}\right)\kern1em \\ {}\kern1em AngleAccurac{y}_{individual}=1-\left(\frac{{\displaystyle \sum_{i=1}^n\left|\left(\frac{180}{\pi}\right) an{g}_{Dif{f}_i}\right|}}{180n}\right)\mathrm{where}{n}_{basic}=nDL-1,{n}_{advanced}=8\cdot nDL-{8}^{\ast}\kern1em \end{array} $$

* Note: In advanced mode, the software reports the mean (circular mean for rotational bias, arithmetic mean for angle accuracy) of the calculated individual measures of the landmark’s eight peripheral points. For rotational bias, 8n is substituted for n in the equation in order to correctly average the angular difference scores.