Journal Search Engine
Search Advanced Search Adode Reader(link)
Download PDF Export Citaion korean bibliography PMC previewer
ISSN : 1598-7248 (Print)
ISSN : 2234-6473 (Online)
Industrial Engineering & Management Systems Vol.16 No.4 pp.604-618
DOI : https://doi.org/10.7232/iems.2017.16.4.604

A Comparative Study on the Detectability between Haptic and Auditory Signals under Driving and Non-driving Task Conditions

Sang Ho Kim*, Jong Gyu Shin, Jose Fernando Sabando
School of Industrial Engineering, Kumoh National Institute of Technology Gumi, Republic of Korea
Corresponding Author, kimsh@kumoh.ac.kr
20170124 20170727 20170816

ABSTRACT

Objective: The objective of this study is to compare the driver’s response rate and time between auditory and haptic signals under different task conditions. It is for checking whether haptic signals can be used as a primary and independent channel of the multimodal in-vehicle warning system. Background: As recent automobiles are evolving into intelligent information systems the amount of information that drivers need to process increase correspondingly. Too much information presented simultaneously on visual displays may distract drivers’ attention and it may cause serious traffic accidents. It is hypothesized that multimodal in-vehicle information systems can be a possible solution for reducing the driver’s cognitive overload. Method: Experiments were conducted to gather response rate and time for given warning signals delivered in auditory and haptic modalities, using a computerized interactive driving simulator. The subjects were tested under two different task conditions: driving and non-driving. Response rate and time for the haptic signals were compared with those for the auditory signals. Conclusion: The study shows that there is no significant difference in response rate between the two modalities. For response time, auditory signals draw faster responses compared to the haptic signals in 70 msec on average. Application: The result from this study supports that haptic signals can be used as substitute warnings under the condition that other channels are busy and signals can be initiated a bit earlier than the auditory signals.


초록


    1.INTRODUCTION

    Driving is a cognitively demanding task that requires high level of driver’s attention (Campbell et al., 2016). The drivers would face serious danger if they fail to detect and respond properly to the potential hazards (Campbell et al., 2007). To enhance the traffic safety, various functions have been developed and installed in recent automobiles (Lu et al., 2010). Those functions, so called advanced driver assistance systems (ADAS), could give warnings for the potentially hazardous conditions detected in advance and intervene autonomously sometimes to handle sluggish responses of the driver. This innovation of the automotive technology became affordable due to new information and communication technologies (ICT) such as microprocessors, electronic displays, sensors, and mobile networks between others. It means that the automobiles nowadays are not just mechanical systems, but evolving into intelligent information systems (Salmon et al., 2007).

    In-vehicle information system (IVIS) change the way drivers interact with their vehicles substantially. However, the focus of developing automotive information technology so far has been that of adding new functions with unfamiliar interfaces. The drivers could feel uneasy to interact with the new systems and even get frustrated when the alarms are coming from the IVIS (Kim et al., 2016). In this context, it becomes critical to manage the ways of presenting information on the IVIS. Despite of the functional purpose of IVIS, rivalry of information displays might cause distractions on the drivers and deteriorate their performances and safe driving (Stevens et al., 2002).

    Drivers need to occupy their sensory channels to access critical information from the IVIS (Green et al., 1993). Vision is the most versatile, but vulnerable sensory channel of the driver. IVIS used to present most of the information on visual displays despite the driver’s need to keep their eyes on the road ahead. To reduce this overload of the visual channel while interacting with the IVIS (Li and Burns, 2013), it has been proposed to use multimodal interfaces that occupy other sensory channels like audition or touch sensation instead of vision (Ferris and Sarter, 2010; Murata et al., 2014). For more effective use of multimodal IVIS, the usability of multimodal IVIS and its effect on the information processing of the drivers need to be investigated (Politis et al., 2013). To ease the burden of human information processing, physical parameters of the stimuli presented on the displays should be carefully designed to secure detectability, discriminability, and compatibility (Sanders and McCormick, 1992). Detectability means any signal randomly given to convey certain information, should be detected easily by the driver under varying atmospheric conditions. Highly detectable signals will allow less missing and false alarming errors as well as faster response time. Discriminability is how easy it is for a subject to distinguish or differentiate between a set of signals; while meaningfulness deals with the ease with which a subject can gather information or assign meaning from a given signal configuration.

    Researchers have investigated the possibility of using haptic signals as a means to substitute other sensory input signals. Myles and Bissell (2007) on their research about the tactile modality argued that “the tactile modality is a viable choice for the deliverance of information” and provided a table with the vibration sensitivity on body site listed from high to low; hands, soles of feet, larynx region, abdomen, head region, gluteal region, were the most sensitive body parts. Meng and. Spence (2015) conducted experiments with haptic signals and made their conclusions that “tactile warning signals should be tested under conditions of high perceptual load, with infrequent presentation rate, before their implementation in realistic driving situations.”

    Chang (2010) on a different research evaluated the use of a haptic seat on a vehicle to support vehicle navigation system. The result of the study showed that the reaction time is faster with haptic signals, middle with visual and slower with auditory. On this experimental setup, different configurations of 12 haptic actuators were used to deliver different instructions while auditory signals were delivered on a speech based format. In a research performed by Straughn (2009), the subjects were asked to avoid a pedestrian simulating an emergency scenario by turning the steering wheel after getting the warning signal provided 4 seconds before the time to collision. Straughn’s results showed that the response performance of haptic signals were 0.2 seconds faster than auditory ones.

    In recent days, haptic signals are being used in some multimodal IVIS; however, as a secondary and redundant channel mostly (Visvikis et al., 2008). It means that they are used in addition to the other primary sensory channels in order to regain the driver’s attention, or to provide more abstracted-lower resolution information.

    The purpose of this study is to validate the usability of haptic signals as an independent channel on the interaction design of multimodal IVIS in order to reduce the cognitive burden of the driver due to the information overload on the visual and auditory channels. The first step for validating the usability of haptic signals in IVIS is to confirm its detectability is equally likely to those of other primary channels. More advanced usability issues, discriminability and meaningfulness of the haptic signals in driving conditions could be discussed after confirming its detectability.

    This study also determines proper range of parameter setting of haptic signals in terms of acceleration and frequency to ensure better detectability on the basis of absolute evaluation. A gap analysis is performed for relative evaluation of the haptic signals’ detectability against auditory signals in the driving environment.

    2.METHODS

    2.1.Experimental Setups

    Experiments were designed and conducted to compare the detectability of haptic and auditory signals under different parametric configurations. The detectability of a given signal was measured by its response rate (RR) and response time (RT), which will be referred to as RR and RT from this point on.

    For measuring the detectability, RR and RT were selected as performance metrics due to the fact that drivers have to perform accurate maneuvers in a time constrained environment as part of their main task, driving. RR was constructed as the initial measure to analyze because activating the signal is meaningless if it is not transmitted properly. This means that the driver could end up with missing valuable and sometimes urgent information that would help avoiding a safety incident. This probability of missing a signal does not only affect haptic signals, but auditory signals as well. In this sense, if haptic signals are to validate as detectable as other primary signals, they should at least have the same level of performance in accuracy as auditory signals. However, safe driving is based not only on accuracy of actions. It also depends on the promptness of drivers’ responses to safety hazards. In this sense, the performance in terms of speed (which is going to be measured by RT) is an essential attribute when detecting a signal and executing a response.

    Experiments were conducted in 4 sessions, each of which is for haptic driving and non-driving conditions, and auditory driving and non-driving conditions, respectively. Each experimental session was managed as a two factor within-subject design to avoid large error terms that might come from individual differences among the subjects. Fourteen graduate and undergraduate students who have valid driver’s licenses participated as the subjects. To avoid carry-over effect during the experiments, the subjects were given a time in order to get accustomed with using the equipment and the procedure of the experiment. The subject’s mission was to press a button attached on the steering wheel when they perceived the signals.

    Haptic signals were modulated by different frequency and acceleration combinations and each signal lasted for 3 seconds irrespective of the responses. These modulated combinations of frequency and accelerations were to determine proper ranges of haptic design parameters that were configured to secure good detectability. The findings from previous researches were considered to set up treatment conditions, including the range of frequencies and accelerations. According to previous research, frequency of haptic signals should be higher (e.g. 80, 140, 250Hz) than those of natural vibrations; in addition, human is most sensitive for a haptic frequency band between 200 and 250Hz (Campbell et al., 2014). The frequency variation was determined as 7 levels between 80 to 350Hz considering the channel capacity of human beings in absolute judgment (Wyse et al., 2012).

    The acceleration variation was determined as 5 levels between 1 to 5 G. According to previous research, the recommended accelerations are between 2.02 to 2.65 G for the seat pan and in the range of 2.65 to 3.38 G for the backrest when the car seat is used as haptic displays (Campbell et al., 2014). Haptic signals were delivered randomly and unbalanced through 4 body parts: upper arm, wrist, thigh and hand. The parts were selected based on previous findings and due to their easiness of deploying haptic signals by making direct contact with signal transport media in the car. Upper arm was considered as the proper position for making contact with seat belt, thigh with car seat pan, wrist with the aid of a wearable device like smart watch, and hand with the steering wheel.

    Auditory signals were used for the purpose of comparing them with haptic signals and hence obtaining a relative judgment about the haptic signals detectability. To make a proper and fair judgment, auditory signals should be coded efficiently. Auditory signals were modulated by different frequency and sound pressure level (SPL) combinations to find out proper range of signal design parameters like it was done for the haptic signals. The auditory signals were provided for 3 seconds (same time as haptic signals). The frequency of auditory signals were varied with 7 levels between 0.3 to 10KHz. The ISO standard recommends to include frequency components between 0.5 to 2.50KHz when delivering auditory signals. In this study, relatively wide band of frequency was used to identify most distinguishable signals within audible frequencies. The sound pressure level was varied with 5 levels from 60 to 80dB. The average ambient noise level around the car was considered as 60dB (Campbell et al., 2014), and it was set for the lowest level. According to a finding of a previous research; driver can detect auditory signals of 75dB easily (Kang, 2014) and this was considered to setup the upper level of auditory SPL.

    Table 1 summarizes the experimental design for haptic and auditory modalities. The number of treatment conditions are the same at driving and non-driving scenarios.

    2.2.Equipment

    A medium-fidelity driving simulator with Logitech™ control interface was used to emulate driving tasks and environment. This simulator provides environment noises and vibrations which are transmitted through speakers and seat to the driver; however, they must be less intense than on a real life scenario. Haptic signals were generated by haptic actuators named Haptuator Mark II. Haptuator Mark II is a rod-type actuator having 9×9×32 mm dimensions made by TactileLabs™, and it makes unidimensional vibrations within optimal range from 50 to 500Hz (TactileLabs, 2012). The surface contact with the subjects’ skin was its longitudinal dimension of 9×32 mm. The actuators were attached to the 4 body parts previously mentioned. In addition, for the haptic signals experiments, subjects were provided earphones with background music in order to increase noise and make them rely more on the haptic sense. Auditory signals were presented as monotone beep sounds using a pair of speakers made by Britz™. To modulate the signal characteristics and control their sequence of presentation in a random order, software program was coded with LabView™. D/A converter module on the NI 9264 made by National Instrument was used to convert the signals designed in digital formats into direct currents and deliver them to the haptuator and the speakers. A/D converter module on the NI 9264 was used to collect analogue signals generated when the button on the steering wheel of the driving simulator is getting pressed. Figure 1 shows the whole setup of equipment utilized in this study.

    2.3.Analysis of Data

    The RT for a given signal was measured by the definition of time lapse between the signal presentation and the button pressing. The RR was defined as the number of responded signals per given signals; if the subject had pressed the button within 3 seconds of the signal being delivered, it was counted as a responded signal (or Hit, according to Signal Detection Theory). Otherwise, it was counted as a not responded signal (or Miss). False alarms and correct rejections were not considered in this study. Average RR across all the subjects was then calculated for each level of signal conditions. Average RT for a given treatment condition was calculated by the responded data only. Chi-square test and Analysis of variance (ANOVA) procedures were conducted to compare if there were statistically significant differences in the average RR and RT respectively among the different levels of treatment conditions. T-test was performed to compare the differences between the two modalities.

    3.RESULTS

    3.1.Response Rate

    3.1.1.Haptic Signals

    To identify the effect of 7 different levels of frequency on haptic signal detection tasks, RR of the subjects across all accelerations within each frequency were analyzed and the result is depicted in Figure 2. This analysis was done for checking out whether if there is a specific frequency of haptic signals which yields significantly lower RR with respect to the others and thus needs to be excluded for getting higher level of detectability. The results of chi-square test showed that there were statistically significant differences among the average RR of the haptic signal frequencies under driving (pvalue = 0.002) and non-driving (p-value = 0.000) condition. The RR under the non-driving condition for the frequency band between 80 to 250Hz were significantly higher (over 93%) than that of 350Hz (82.4%). The relatively lower RR of frequency 350Hz (75.7%) comparing with those of 80 to 250Hz (over 83%) was observed under driving condition too. When this low RR of 350Hz was excluded from the original dataset, the average RR under non-driving increased up to 92.6% whereas that under driving increased up to 84%. The chisquare result showed that the statistically significant effect of the frequency on the RR was disappeared with this selected frequency band between 80 to 250Hz under both of driving (p-value = 0.515) and non-driving (pvalue = 0.900) condition. It can be said that around 84% of unimodal haptic signals can be detected by the drivers if the signal has proper frequencies.

    Identical analysis procedures were applied to investigate the effect of 5 different acceleration levels on the RR of the haptic signals and their results is shown in Figure 3. This was for checking out whether if there is a specific treatment condition of haptic acceleration which yields significantly lower RR that needed to be excluded. The results from the chi-square showed that there were statistically significant differences among the average RR of the haptic signal accelerations under both driving and non-driving condition (p-values <0.001). The RR for 1G under driving and non-driving, 65.0% and 77.2% respectively, was fairly lower than those of the acceleration 2-5G’s. When these data obtained from 1G treatment condition were excluded, the average RR under nondriving increased to 94.5% whereas that under driving increased to 87.1%. The chi-square test conducted on this reduced dataset showed that the statistically significant effect of the acceleration on the RR with this selected acceleration band between 2 to 5G disappeared on nondriving mode (p-value = 0.068), as well as under driving mode (p-value = 0.158).

    3.1.2.Auditory Signals

    In order to perform a fair and reliable comparison with the haptic signals, the effect of design parameters in auditory signals need to be analyzed. If some range of frequency or sound pressure level of auditory signals is significantly lower on performance than the others, it should be excluded to attain an insightful comparison with those performances of haptic signals.

    The effect of different auditory frequencies on the RR was analyzed and the result is depicted in Figure 4. For the non-driving mode, the RR did not show significant differences (p-value = 0.911) among the different frequencies. For the driving mode, the effect of frequency seemed to increase slightly but was not statistically significant either (p-value = 0.206).

    The effect of sound pressure level (SPL) on the RR of the auditory signals was also analyzed and the result is summarized in Figure 5. The chi-square showed that the effect of SPL on the RR was not significant under nondriving mode (p-value = 0.353), but significant under driving mode (p-value = 0.001). The RR for 60dB SPL (76.9%) under driving mode was significantly lower than those for the other SPL over 60dB. If these data obtained from 60dB treatment condition was excluded, the average RR under driving mode increased to 86.8%, and the effect of SPL becomes not significant on driving mode (p-value = 0.753) as well as on non-driving mode (p-value = 0.624).

    3.1.3.Comparison of Haptic and Auditory Response Rates

    To identify the effect of two different modalities of signals (haptic and auditory) on the RR of the drivers, ttests for the mean difference were applied on the RR data obtained from each of the driving and non-driving mode.

    For the initial experimental condition where all levels of signal design parameters are included, auditory signals (95.6%) showed relatively higher RR than that of the haptic signals (91.1%) under the non-driving mode, this difference was statistically significant (p-value = 0.000). Under the driving mode, the RR of the auditory signal (84.6%) was also higher than that of the haptic signals (82.4%), but it was not statistically significant (pvalue = 0.211). The average RR was 86.8% for haptic signals and 90.2% for auditory signals. These results are shown in Figure 6.

    The effect of signal modality on the RR was analyzed once more with the reduced data set where data from haptic signals with 350Hz, 1G and auditory signals with 60dB treatment conditions were excluded from the original data. The relatively higher RR for the auditory signals (95.9%) compared with that of the haptic signals (95.0%) under the non-driving mode was also statistically significant (p-value = 0.034). The difference of the RR between auditory (86.5%) and haptic (85.2%) signals was not statistically significant (p-value = 0.335) under the driving mode. The improved average RR was 90.1% for haptic signals and 91.2% for auditory signals. These results are shown in Figure 7.

    As shown on Figure 6 and Figure 7, a comparison was made between the RR obtained in the original dataset and the reduced data set of auditory and haptic signals on both driving modes. On this figure it is observed that the gap between signal modality in RR on non-driving mode decreases on 3.7% (from 4.6% to 0.9%) when proper parameters for signal coding is considered. A similar improvement effect happens on driving mode where the gap between modalities is reduced to 0.9% (from 2.2% to 1.3%).

    For the original data set, the difference occurred by changing mode of driving was of 11.1% for auditory and of 8.7% for haptic. For the reduced data set, the mean value of both modalities got improved and the gap was also reduced to 9.4% for auditory and to 9.8% for haptic modality.

    The analysis for interaction effect between the modalities and the driving mode, which was shown on Figure 6 and Figure 7 indicated that the gradient is more pronounced for auditory signals than that of haptic signals. This means that the amount of performance decrease in RR for changing mode of driving is different by the signal modality.

    3.2.Response Time

    3.2.1.Haptic Signals

    In the RT analysis, the haptic treatment condition that did not provide proper RR (350Hz, 1G) was excluded and this reduced data set with 6 different frequencies and 4 different acceleration combinations was analyzed.

    To identify the effect of 6 different levels of frequency on haptic signal detection tasks, RT of the subjects across all accelerations within each frequency were analyzed and the result is depicted in Figure 8. The results of ANOVA showed that there were no statistically significant differences among the average RT of the haptic signal frequencies under driving mode (p-value = 0.557) and non-driving mode (p-value = 0.982).

    Identical analysis procedures were conducted to investigate the effect of the selected acceleration levels (2 to 5 G) on the RT of the haptic signals and their results is shown in Figure 9. The results from ANOVA showed that there were no statistically significant differences among the average RT of the haptic signal accelerations under both driving (p-value = 0.141) and non-driving mode (pvalue = 0.557).

    The interaction effect between frequency and acceleration on the RT of the haptic signals was not statistically significant (driving mode: p-value = 0.659 and non-driving mode: p-value = 0.237).

    3.2.2.Auditory Signals

    A proper range of design parameters for the auditory signals was already determined on the basis of RR in the previous sections. The RT analysis for the auditory signals was also conducted with this reduced data set.

    The effect of different auditory frequencies on the RT was analyzed and the result is depicted in Figure 10. For the driving mode (p-value = 0.667), the RT did not show significant differences among the different auditory frequencies. However, on non-driving mode the frequencies are found as a factor for making different RT (p-value = 0.001).

    The effect of SPL on the RT of the auditory signals was also analyzed and the result is summarized in Figure 11. For the non-driving mode (p-value = 0.452) as well as for the driving mode (p-value = 0.108) the RT did not show significant differences among the different SPL.

    Finally, there is no significant interaction between frequency and SPL (driving mode: p-value = 0.837 and non-driving mode: p-value = 0.061).

    3.2.3.Comparison of Haptic and Auditory Response Times

    In order to identify the effect of two different modalities of signals (haptic and auditory) on the RT of the drivers, t-tests for the mean difference were applied on the RT data obtained from each of the driving and nondriving mode as shown on Figure 12.

    The relatively slower performance for the haptic signals (708 msec) compared with that of the auditory signals (615 msec) under the non-driving mode was statistically significant (p-value = 0.00). The difference of the RT between haptic (787 msec) and auditory (739 msec) signals was also statistically significant (p-value = 0.00) under the driving mode. The average RT was 748 msec for haptic signals and 677 msec for auditory signals.

    On Figure 12, a comparison was made between the RT of auditory and haptic signals on both driving modes. On this figure it is shown that the reduction of performance in RT according to changing modality from haptic to auditory is of 15.1% (93 msec) for non-driving mode, while 6.5% (48 msec) for driving mode.

    The difference in RT occurred by changing mode of driving from non-driving to driving is of 20.2% (124 msec) for the auditory signals and of 11.2% (79 msec) for the haptic signals.

    The analysis for interaction effect between the modalities and the driving mode on the RT, which was shown on Figure 13 indicated that the gradient is more pronounced for auditory signals than that of haptic signals. This is a similar finding as the one shown on RR analysis; however, on RT this interaction seems to be stronger because the difference of the gradients is a bit more severe.

    3.3.Body Part Sensitivity

    To investigate whether the sensitivity of different body parts influenced on the detectability of haptic signals, the average RR across all the frequency and acceleration conditions delivered to 4 different body parts were analyzed and summarized in Figure 13. For the non-driving mode, hand showed the highest RR of 95.8% among the four body parts whereas those of the others were around 88%. Hand and wrist showed relatively higher RR (around 86%) than upper arm and thigh (around 79%) under the driving mode. The differences of RR between the two different modes of driving for each body parts were identified as 11.8%, 9.1%, 8.3%, and 2.6% points for thigh, hands, upper arm, and wrist, respectively. Wrist showed the most robust characteristics on the RR irrespective of the driving and non-driving mode.

    4.DISCUSSION

    4.1.Response Rate

    4.1.1.Haptic Signals

    When analyzing the initial 7 conditions of frequency, as shown on Figure 2, there was a significant difference in the RR between the different treatment levels of frequency. However, after the identified cause (350Hz) for this adverse effect was excluded from the original dataset, the RR was improved in average with no statistical difference irrespective of the frequencies. That means haptic frequencies between 80 to 250Hz has fairly steady RR: haptic frequency can be modulated within this range without losing detectability at least in terms of RR.

    As for the acceleration conditions, it also had some deficiencies when its proper range of modulation was not accounted for, as shown on Figure 3. When a subset (1 G) of the initial coding was excluded from the original dataset, the RR was improved in average on both of driving and non-driving modes. In addition of the mean RR getting improved, haptic accelerations between 1 to 5 G showed fairly steady RR: haptic acceleration can be regulated within this range without losing detectability at least in terms of RR.

    Previous experiments for testing the goodness of haptic signals had used a haptic seat as a signal display (Fitch et al., 2011) and therefore receiving the stimuli on the legs or the back of the subject tested (Ji et al., 2010); however, on this experiment the approach was to use 4 different body parts (upper arm, wrist, thigh and hand). Different body parts were selected because some noise can be transmitted through the seat vibrations because of mechanical or road conditions and thus making it harder for the driver to detect the signal.

    It is noted that no significant interaction effect was found among the design parameters of the haptic signals considered in this study (frequency, acceleration). This makes designers free to utilize different combination of the haptic parameters without considering their interactions on the RR under the proper range. This result is a prerequisite for designing high resolution haptic information. Since there is no significant interaction between the factors, the signals could be coded in different combinations (6 configurations of frequencies by 4 levels of accelerations) in order to transmit different message with each configuration if their discriminability can be assured in the next stage of usability validation.

    4.1.2.Auditory Signals

    When the RR was analyzed on auditory signals across all frequencies utilized in this study, the mean values were found to be similar as shown on Figure 4. This means that auditory signals with any frequency among original data set (0.3kHz-10kHz) can be used without a decrease in performance at least in terms of RR. With this result it can be said that the auditory signals considered in the study have a proper range of frequency and ensure a fair comparison with haptic signals.

    When the effect of SPL was analyzed, some deficiencies on RR of the initial range under the driving mode were found. However, after the root cause (60dB) was distinguished and excluded from the original dataset, the mean value of RR was getting improved as shown on Figure 5. When only the proper values of SPL were considered (65dB-80dB), average RR on both modes of driving became statistically similar in addition to the improvement of the mean values. Since the RR is similar across different levels of the reduced SPL range, each one of them can be used without considering the performance decrease in detectability in terms of RR.

    With these results and explanations, it can be said that the parameters used for the auditory signals considered in this study are carefully selected for fair relative comparison with haptic signals in terms of RR.

    4.1.3.Comparison of Haptic and Auditory Response Rates

    A decreased performance of detectability in terms of RR is observed on both modalities when driving mode changed from non-driving to driving. It seems that when the subjects were engaged in detecting signals in nondriving mode, their cognitive resources were spent on this task alone, thus obtaining better performance. However, when people are engaged in signal detection tasks in parallel with driving task, their cognitive resources are scattered among the different activities to be performed by the driver.

    On both modalities, the RR is improved when signal design parameters are coded properly. Therefore, it is confirmed once again, like in previous researches, that a proper coding of information is determinant on the effectiveness of the signal irrespective of the modality to be used (Haas and Edworthy, 1996; Ferris et al., 2009).

    It was shown on Figure 6 and 7 that RR was getting better for both modalities as the original dataset was being reduced by proper setting of design parameters. As for the difference in RR between modalities in nondriving mode, it was reduced from 4.6% to 0.9 but the difference was still statistically significant. On driving mode, the difference in RR between haptic and auditory signals was also decreased (2.2% to 1.3%) when the dataset was reduced to the recommended design values. However, in this result it was shown that the differences in both datasets (original and reduced) were not statistically significant. With this finding, it can be said that haptic signals could be used with the same expected accuracy as those of auditory signals in multimodal alert systems under driving mode.

    Another notable finding from the results is that haptic signals are more robust for changing mode of driving in at least 2.4% (11.1% difference between driving modes in auditory and 8.7% difference in haptic) for the original dataset and in 0.4% (9.4% difference between driving modes in auditory, 9.8% difference in haptic) for the reduced dataset.

    Based on the previously stated discussion and comparisons, it can be said that haptic signals are as detectable as auditory signals in terms of RR for the driving mode. There is no guarantee yet about the complete detectability in terms of the other metric, time, and that is discussed on the next section.

    4.2.Response Time

    4.2.1.Haptic Signals

    As shown on figure 8, within the selected range of frequencies (80 to 250Hz), RT for the haptic signals did not change significantly despite of the changing frequencies under both of the driving and non-driving modes. This no significant effect of the haptic frequencies on RT as well as RR (which was discussed in the previous section) means that any haptic signal within this reduced frequency range is equally detectable in terms of both performance metrics.

    The effect of acceleration was also found as not significant on the RT within the range of 2 to 5 G, and thus different haptic signals modulated by changing acceleration within this proper range can be used without detectability problem. The effect of acceleration on haptic RT was shown on Figure 9.

    As for the interaction effect between frequency and acceleration, it was not significant on both of driving and non-driving modes. Therefore, the different configurations made by acceleration and frequency combination can be utilized without producing a diminished performance in terms of RT as well as RR.

    4.2.2.Auditory Signals

    On Figure 10, the effect of frequency change (0.3 to 10kHz) on RT was analyzed for auditory signals. Irrespective of the frequencies used to modulate auditory signals, the mean RT was statistically similar in the different levels of non-driving mode as well as driving mode. This similar mean value of the auditory frequencies on RT as well as RR means that any auditory signal within this frequency range is equally detectable in both metrics.

    The effect of SPL on auditory RT on the range of 65 to 80dB was also not significant across the different levels on driving mode as shown on Figure 11. However, on non-driving mode the auditory frequency was found as a factor for making difference in RT of the signal. This difference must be caused from the different sensitivity of human to the auditory frequencies, and thus needs to be considered in the signal design.

    As for the interaction effect between frequency and SPL, it was not statistically significant also. Therefore, the different configurations between frequency and SPL can be modified without reducing the performance in terms of RT as well as RR. The reason for evaluating the performance metric of the auditory signals was to provide a proper basis for relative judgment of the detectability performance of haptic signals.

    With the results shown in this experiment, it can be assured that a fair comparison can be made in terms of RR and RT between the modalities since the best parametric codings were arranged for haptic and auditory signals by careful reduction on the original data set.

    4.2.3.Comparison of Haptic and Auditory Response Times

    A direct comparison between the RT of the two modalities under different driving modes are shown on Figure 12. There are statistically significant differences in RT between auditory and haptic signals on both driving modes.

    Change of driving mode is supposed to affect the availability of cognitive resources. This means that the more tasks and distractions the driver has, the slower the RT is to get. In the results, there was reinforcing evidence for the higher cognitive load of driving mode with respect to the non-driving mode. In auditory modality

    In auditory modality the consequent reduction in performance when changing mode from non-driving to driving is of 20.2% (124 msec) while this reduction was 11.2% (79 msec) in haptic modality. This means haptic signals are more robust for the changing mode of driving for about 9% points.

    Even though auditory has a significantly faster average RT than haptic on both driving modes (15.1% non-driving mode and 6.5% on driving mode); it is evident that haptic response is more robust across the different driving modes. This is clearly illustrated by Figure 12 where the auditory modality showed a more pronounced gradient of change in RR with respect to the haptic modality as the driving mode changed from nondriving to driving.

    This could mean that when the driver is focused on driving, it is harder to perceive new alarm signals through traditional modalities (visual & auditory) due to little margin of cognitive resources for heavy divided attention. When the cognitive workload increases, the distinctiveness of auditory signals seems to be reduced more prominently than haptic signals (Mohebbi et al., 2009). However, more attentional resources are available in a relatively lower engaged modality (haptic), and thus it ensures a more stable signal detection.

    Although there is a statistical difference in RT between the haptic and auditory signals at driving mode (6.5% or 48msec), this is not that critical limitation for not engaging haptic signals as independent warning signals. Assuming that someone is driving a car at 100km/h in the highway, the difference of 48 sec will be 1.3m. If cars are moving at 50km/h, the gap of stopping distance will be 0.7 m between the two modalities. Compensatory methods also can be applied to make up for this difference in RT.

    One compensatory method could be to deliver the signal earlier when using haptic modality. This way, the sluggishness in response can be accounted for and the risks can be minimized. Another method could be to provide less urgent information through this modality.

    The results of this study that compare the performance of haptic signals with other modality in terms of time differ from those found on previous research. In Chang and Straughn’s experiment which was mentioned in introduction section, haptic response was faster than that of auditory. It is noted that there was a difference in experimental setups. In their research the performed task was decision making (detection, discrimination, understanding, response) while the goal was to measure the signal detection performance only in terms of RR and RT on this study. In this sense the difference in time found on performance of this research between the different modalities is inherent to the human physical limitations for each sensory channel; however, compensatory measures can be taken into account for this difference such as sooner signal emission, simpler information coding, among others.

    At this point of the study and when using proper values of signal configuration, it can be said that the detectability of haptic signals can be competitive in terms of RT subject to proper compensatory methods.

    4.3.Body Part Sensitivity

    The RR is different depending on the site used to deploy the stimuli, as shown on Figure 13. Among the body parts analyzed, wrist is considered the most appropriate place to deliver the vibrotactile stimuli impulse due to its robustness. Only 2.6% difference was found between the RR on different driving modes; not only that, on the driving mode the wrist was the body part with the highest average RR. One issue that remains is how to use the wrist as a place for dispatching the haptic signals on real driving environment. One option must be using wearable devices (smart watches, fit trackers) where a small haptic actuator can be easily enclosed. Hand is the highest RR position under non-driving mode and should be treated as the second option when delivering haptic signals; the display would be the steering wheel and would ensure a high engagement of this body part; however, it is not really robust (9.1% difference on different driving modes). Upper arm and thigh resulted in the lowest relative detectability for driving condition and therefore are not recommended as haptic signals displays interfaces for driving or emergency situations but could be used for not critical tasks or in combination with other modalities. This result is coherent with that of Myles and Bissell that stated that hands were the most sensitive part when it comes to vibration sensitivity.

    5.CONCLUSIONS

    The objective of this study was to confirm the possibility of using haptic signals independently as part of a multimodal warning display for IVIS. As its first step, the detectability of haptic signals was analyzed in terms of RR and RT and later compared with auditory signals which is a widely accepted sensory component of multimodal alarm systems.

    By the comparison of the RR of auditory and haptic signals, it is concluded that haptic signals are equally detectable with respect to that of the auditory signals subject to careful selection of signal design parameters. Proper signal coding is fundamental for both signals modality. When proper design parameters were set, the mean RR of haptic and auditory signals was improved and became statistically similar, meaning that one modality could be used in place of the other, with no diminishment in RR. Haptic signals were detectable when coded with a frequency between 80Hz-250Hz and an acceleration in the range of 2G-5G.

    By the comparison of the RT of auditory and haptic signals, it is concluded that haptic signals has slightly slower response time than that of the auditory signals, but the difference is practically not significant. The time gap in RT between the two modalities can be managed by some compensatory measures proposed earlier in this study.

    The usability of the haptic signals in terms of detectability is reinforced by its robustness for the changing task environment. There is a significant decrease in performance metrics for both modalities when changing driving mode from non-driving to driving. However, haptic signals were more robust than auditory signals in 0.4% points for RR and in 9% points for RT. Auditory response was more sensible when changing between driving modes: when noise and cognitive load increases. It is concluded that haptic sensory channel appears to be less affected to the environment noise when compared in RR and RT against the auditory channel.

    Based on the measures proposed in this experiment (RR, RT) and according discussion it can be assured that haptic signals are, as a matter of fact, detectable and could be used as part of a multimodal display for IVIS without diminishing performance.

    One possible application of haptic signals as an independent display can be on an alert for drowsiness detection system installed in public transportation vehicles. If the system is detecting certain a slight level of driver’s drowsiness, it can send a warning signal to indicate that there is a risk; if an auditory signal is selected as the modality to deliver this message, passengers could feel uneasy and get frustrated. This will make the situation much worse. However, if the interaction is handled with a haptic signal, the communication will be only between the IVIS and the driver. If the driver after the preventive warning continues in a drowsiness state, a multimodal warning (haptic + auditory) can be delivered to reduce the risk of an accident.

    5.1.Limitations

    The primary task in this study was driving but it was performed in a simulated environment. The driving performance was not controlled deliberately nor measured while doing the secondary task which was signal detection (Petermeijer et al., 2015).

    Like it was mentioned in the introduction, F. Meng and C. Spence suggested that “tactile warning signals should be tested under conditions of high perceptual load, with infrequent presentation rate, before their implementation in realistic driving situations.”

    In the experimental setups of this study, the high perceptual load was occupied in some extent when measuring responses in the driving mode. This can be confirmed with the different performance metrics obtained when analyzing non-driving mode and driving mode.

    Regarding the signal presentation rate, haptic and auditory signals were given randomly but much more frequently than on a real driving environment. Therefore, attention level of the subjects were much higher and thus the absolute values for accuracy (RR) and speed (RT) tends to be overestimated.

    Nonetheless, in a sense of comparing the performance of drivers when dealing with haptic and auditory signals, the validity in its detectability is still viable because the same experimental conditions were applied for both modalities. This means that both modalities were equally affected by the conditions and in this way the relative comparison is valid.

    The experiments were performed in a controlled laboratory environment in spite of the fact that vibrations and noise were provided by the simulator. Those environmental noises might be lower in magnitude than on real environment, and there is a chance for underestimating the effect of natural vibrations on RR and RT.

    5.2.Further Research

    Even though there were some limitations to the conditions of the study, they were managed and their effect was controlled as it was explained in the previous section. In this study it has been shown that haptic signals are detectable. This is a relevant finding because it could open the doors for further experimentation of the application of haptic signals in modern IVIS, not only as a complement of the current signal modalities used; but as an independent and self-reliant modality that could be used to declutter the other senses as well as to reorganize current IVIS information distribution strategy.

    With the confirmed detectability of haptic signal, the challenge lies on how to code these signals to ensure high levels of detectability while maintaining driving comfort, safety and engagement with priority tasks (Baldwin et al., 2012).

    Next research steps will be dealing with the discriminability and meaningfulness of the signals. It should be defined how many different haptic signals can be successfully distinguished by people while maintaining acceptable levels of performance and how to design comprehensible haptic signals that convey meaningful messages to people. The more transparent the message is for the subject or driver, the less cognitive processing the driver will have to perform.

    ACKNOWLEDGMENT

    This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2012R1A1A4A01006316).

    Figure

    IEMS-16-604_F1.gif

    Setups for equipment and task environment.

    IEMS-16-604_F2.gif

    The effect of haptic signal frequencies on the response rate.

    IEMS-16-604_F3.gif

    The effect of haptic signal accelerations on the response rate (RR).

    IEMS-16-604_F4.gif

    The effect of auditory signal frequencies on the response rate (RR).

    IEMS-16-604_F5.gif

    The effect of auditory signal SPL on the response rate (RR).

    IEMS-16-604_F6.gif

    Comparison of RR between auditory vs. haptic with original data set.

    IEMS-16-604_F7.gif

    Comparison of RR between auditory vs. haptic with reduced data set.

    IEMS-16-604_F8.gif

    The effect of haptic signal frequencies on the response time (RT).

    IEMS-16-604_F9.gif

    The effect of haptic signal accelerations on the response time (RT).

    IEMS-16-604_F10.gif

    The effect of auditory signal frequencies on the response time (RT).

    IEMS-16-604_F11.gif

    The effect of auditory signal SPL on the response time (RT).

    IEMS-16-604_F12.gif

    Comparison of RT between auditory vs. haptic.

    IEMS-16-604_F13.gif

    The effect of body parts where haptic signals were applied on the response rate (RR).

    Table

    Summary of independent variables and their levels

    Frequency effects on haptic RR under non-driving

    Frequency effects on haptic RR under driving

    Acceleration effects on haptic RR under non-driving

    Acceleration effects on haptic RR under driving

    Frequency effects on auditory RR under non-driving

    Frequency effects on auditory RR under driving

    SPL effects on auditory RR under non-driving

    SPL effects on auditory RR under driving

    Haptic signals RT–non-driving. two-factor within-subject design

    Auditory signals RT–non-driving. two-factor within-subject design

    Haptic signals RT–driving. two-factor within-subject design

    Auditory signals RT–driving two-factor within-subject design

    REFERENCES

    1. BaldwinC.L. EisertJ.L. GarciaA. LewisB. PrattS.M. GonzalezC. (2012) Multimodal urgency coding: Auditory, visual, and tactile parameters and their impact on perceived urgency. , Work, Vol.41 (1) ; pp.3586-3591
    2. CampbellJ.L. DoerzaphZ.R. RichardC.M. (2014) Human factors design principles for the drivervehicle interface (DVI , Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, ; pp.1-6
    3. CampbellJ.L. RichardC.M. BrownJ.L. McCallumM. (2007) Crash warning system interfaces: Human factors insights and lessons learned. , National Highway Traffic Safety Administration Final Report, Vol.810 ; pp.697
    4. CampbellJ. L. BrownJ. L. GravingJ.S. RichardC. M. LichtyM. G. (2016) Human factors design guidance for driver-vehicle interfaces. , National Highway Traffic Safety Administration Final Report,
    5. ChangW. KimS.H. PyunJ.K. JiY.G. (2010) Evaluation of haptic seat for vehicle navigation system. , Journal of the Ergonomics Society of Korea, Vol.29 (4) ; pp.625-629
    6. FerrisT.K. SarterN. (2010) When content matters: The role of processing code in tactile display design. , IEEE Trans. Haptics, Vol.3 (3) ; pp.199-210
    7. FerrisT. HameedS. SarterN. (2009) Tactile displays for multitask environments: The role of concurrent task processing code. , EuroHaptics Conference, 2009 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics 2009. Third Joint,
    8. FitchG.M. HankeyJ.M. KleinerB.M. DingusT.A. (2011) Driver comprehension of multiple haptic seat alerts intended for use in an integrated collision avoidance system. , Transp. Res., Part F Traffic Psychol. Behav., Vol.14 (4) ; pp.278-290
    9. GreenP. LevisionW. PaelkeG. SerafinC. (1993) Suggested human factors design guidelines for driver information systems, Technical Report UMTRI-93-21, University of Michigan Transportation Research Institute,
    10. HaasE.C. EdworthyJ. (1996) Designing urgency into auditory warnings using pitch, speed and loudness. , Comput. Contr. Eng. J., Vol.7 (4) ; pp.193-198
    11. JiY.G. LeeK.G. HwangW.I. (2010) Haptic perceptions in the vehicle seat. , Hum. Factors Ergon. Manuf. Serv. Ind., Vol.21 (3) ; pp.305-325
    12. KangH.M. (2014) The relative effectiveness of auditory pedestrian collision warning system as functions of stimulus-response compatibility, warning timing and types of driver distraction, Master’s degree. dissertation, Pusan National University,
    13. KimS.H. SabandoJ.F. KimW.H. (2016) An ecological interface design approach for developing integrated and advanced in-vehicle information system. , Indian J. Sci. Technol., Vol.9 (16)
    14. LiY. BurnsC. (2013) Perceived urgency of tactile warnings , 2013 IEEE International Conference on Systems, Man, and Cybernetics,
    15. LuS.N. TsengH.W. LeeY.H. JanY.G. LeeW.C. (2010) Intelligent safety warning and alert system for car driving. , Tamkang Journal of Science and Engineering, Vol.13 (4) ; pp.395-404
    16. MengF. SpenceC. (2015) Tactile warning signals for in-vehicle systems. , Accid. Anal. Prev., Vol.75 ; pp.333-346
    17. MohebbiR. GrayR. TanH.Z. (2009) Driver reaction time to tactile and auditory rear-end collision warnings while talking on a cell phone. , Hum. Factors, Vol.51 (1) ; pp.102-110
    18. MurataA. KanbayashiM. MoriwakaM. YamadaK. (2014) Preventive safety by auditory-vibrotactile automotive warning system-effectiveness of auditorytactile automotive warning that makes the most of advantages of auditory and tactile modalities , Proceedings of the SICE Annual Conference 2014,
    19. MylesK. BinseelM.S. (2007) The tactile modality: A review of tactile sensitivity and human tactile interfaces, http://www.dtic.mil/get-trdoc/pdf?AD=ADA468389
    20. PetermeijerS.M. AbbinkD.A. MulderM. de WinterJ.C. (2015) The effect of haptic support systems on driver performance: A literature survey. , IEEE Trans. Haptics, Vol.8 (4) ; pp.467-479
    21. PolitisL. BrewsterS. PollickF. (2013) Evaluating multimodal driver displays of varying urgency , Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, ; pp.92-99
    22. SandersM.S. McCormickE.J. (1992) Human Factors in Engineering and Design., McGraw-Hill,
    23. SalmonP.M. ReganM. LenneM.G. StantonN.A. YoungK. (2007) Work domain analysis and intelligent transport systems: Implications for vehicle design. , Int. J. Veh. Des., Vol.45 (3) ; pp.426-448
    24. StevensA. QuimbyA. BoardA. KerslootT. (2002) Design guidelines for safety of invehicle information systems,
    25. StraughnS.M. GrayR. TanH.Z. (2009) To go or not to go: Stimulus-response compatibility for tactile and auditory pedestrian collision warnings. , IEEE Trans. Haptics, Vol.2 (2) ; pp.111-117
    26. TactileLabsTactileLabs (2012) http://tactilelabs.com/
    27. VisvikisC. SmithT. L. PitcherM. SmithR. (2008) Study on lane departure warning and lane change assistant systems. , Transport Research Laboratory Project Report, 978-1-908855-69-5,
    28. WyseL. NanayakkaraS. SeekingsP. OngS. H. TaylorE. (2012) Perception of vibrotactile stimuli above 1 kHz by the hearing-impaired,