Wireless Sensory Glove System developed for advanced Human Computer Interface

Human Computer Interfaces (HCIs) support users to interact or simply control any kind of devices founded on machinery basis. Very simple and common interfaces are represented by mouse and keyboard by which a user interact with the personal computer “machine”. It is however evident how these tools can be particularly “limited” since they “act” only in a 2D superficial environment and cannot provide an immersive experience. So in the latter years new kind of interfaces have been investigated in order to expand the user capabilities in a 3D space, then increasing the realism degree too. In this paper we deal with a new kind of these interfaces. In fact we developed a sensory glove capable to measure all human hand Degree of Freedom (DoF), “translating” them into commands for personal computers. In particular we definedappropriate design rules, and treated details of the signal conditioning electronics surrounding the sensory glove to satisfy the requests of an efficient HCI system.


Introduction
Bend sensors have been endowed in a Lycra based glove so to furnish electrical signals when flexed by user hand [1]. The overall system of sensors, glove, electronic interfaces and wireless part, form the so called sensory glove (also known as data glove or instrumented glove), which we referred here with the term o f HITEG-Glove since our group name (Health Involved Technical Engineering Group). In addition a virtual environ ment has been created so to give to the user a useful real-time visual feedback v ia ad-hoc avatar to his/her actions.
The measured movements of an user hand furnish electrical signals that we converted into pc actions, such as movingthe cursor in a v irtual 3D space, opening/closing applications, playing virtual instruments or even creating or assembling virtual objects (see Fig. 1). For the latter case the hand avatar reproduce exactly all the user hand movements, i.e. flex-extension, abdu-adduction of finger movements, wrist angular positions.
In the past different sensory gloves have been already developed by other research groups, but this paper intends to furnish a new methodology to investigate the properties o f the bend senso rs, to d efin e requ iremen ts fo r the electronic circuit ry including its wireless part, to propose a circu itry so lut ion in o rd er to pro fit by th e very final

Materials
In order to realize one prototype of a sensory glove we could choice among different sensors since, over the time, different kind of sensors have been successfully adopted for different sensory gloves, both commercial and non-commercial ones.

55
International Journal of Information Science 2012, 2(5): 54-59 As research products there are the TUB-Sensor Glove (Technical University of Berlin), the Sig ma Glove (Sheffield University), the Washington University Glove and the Accele Glove (Washington University), the Acceleration Sensing Glove (Berkeley Un iversity), and the WU Glove (University of Wuerzburg, Germany).
The HITEG-Glove here presented is mostly based on bend sensors capable of measuring bending angles thanks to their capability to convert a mechanical bending into an electrical resistance variation.
We measured performances of several bend sensors, manufactured by Flexpo int Sensor System Inc. and Image S.I., d ifferent in length and encapsulation materials.

Characterization of the sensors
The first fundamental step is to measure the resistance value vs. bending angle pairs, so to evaluate the bend sensor performances to correctly utilize them in a sensory glove.
The measures on the sensors can be performed both "on-line" and "off-line". The "on-line" is intended as a method of measurements performed directly with the sensors already embedded in the glove and the glove donned by a tester [2] or by a dummy hand [10]. The "off-line" is intended as a method to measure the sensors prior to insert them into the glove, and it is usually performed with a procedure based on a tradit ional gonio meter to measure the corresponding sensor outputs of a few bending angles. The "on-line" method is the most adopted one because treats the final performances of the overall system, but only with the "off-line" method we believe that it is possible to determine the real intrinsic properties of the sensors, since their performance inevitably get worst when used in a glove. We found that the measures obtained by the "off-line" method can be generally worsened because of human factors such as the operator's expertise. So, here we propose a fully-automated method for "off-line" measurements.
We characterized the sensors in their resistance variation vs. bending angle thanks to an indigenously designed and fabricated set-up, based on hinges where the sensors lay on, and a stepper motor which provides the rotation of one wing of the hinge (with respect to the other which is fix constrained), simulating a human finger joint rotation, as shown in Fig. 2. The adopted motor, necessary for the movement imposed to the rotating wing, was a Trinamic PD-109 two phase hybrid stepper, microstepping optimized. It was provided with a Trinamic Motion Control Language (TM CL) which consists of an instruction set of motion control co mmands.
On the basis of a host computer PC based software development environ ment, the TM CL-IDE, mot ion control commands were given to the motor. A rigid frame provides the necessary stability to the system. The motor was fixed on an optical bench by angular Newport EQ80-E shores. The motor motion was transmitted to the h inge's axis thanks to an universal rigid jo int in order to obtain an excellent stability. For testing a sample, an h inge's wing was fixed on the structure, wh ile the other wing was free to be moved by the motor.
With this measurement set-up, each sensor could be characterized in a -90° to 180° (fro m in ward to outward) range for programmab le step value of bending angle, number of measurement repetitions and mechanical actuator speed. At known angles, the resistance values of the sensors are measured by an Agilent 34405A mult imeter [9,11]. The investigated bend sensors have a large measurement range for outward bending fro m 0° to 150°, and correspondingly the resistance normally changes from 15 to 440 kΩ. The sensors man ifested a really negligible hysteresis and a good repeatability of the measures. Among all the performed measurements, some relevant results are showed in Fig. 3.
We performed measurements of resistance mean values and standard deviations, on 6 different 3 inches length polyimmide encapsulated Flexpoint sample sensors: each sensor was characterized repeating measurements 10 t imes, varying bending angle fro m 0° to 150° and return.
After the characterization, the sensors were mounted on a Lycra based glove, one sensor for each finger joint (see Fig.  4).The resistance variation vs. bending angle characteristic of each sensors was utilized to correctly convert the electric resistance value into the corresponding finger joint flexed angle.

Design of the Electronic Circuitry
The measured electric resistance signals, coming fro m the sensors, were then conditioned, recorded and sent to a receiving location for further exp loitation.
For the electronic interface and the signal conditioning circuit ry we found generally convenient to have: • a sufficiently large resistance variation for sensors; • good "robustness" for the circu it, because there is more noise in wearable applications, consequently, direct resistive sensor-to-microcontroller interfaces are not strongly suggested; • a simplified circu it structure for smaller size; an integrated and removable battery; • a very lo w power consumption, to get a longer service time in wh ich a continuous monitoring can be applied without battery replacement or recharge; • comfort to wearers for user activ ities. This is why we report here a solution we adopted as convenient for the previous requirements for a sensory glove system. A novel approach for analog signal conditioning before A/D conversion, wh ich matches the requirements is presented. System configuration, accuracy and resolution have been analyzed in-depth and design rules have been defined.
Experimental results showed that this electronic interface exhibits less than 1% error in a large measurement range for bend sensor rotation angle. It also showed a good stability to power supply interference. The interface has been successfully applied to a glove-based measurement system of hand gesture.
The optimized electronic interface fo r sensors here concerned was based on a differential instru mentation amp lifier. Fig. 5 schemat izes the proposed electronic interface. It consists of a group of voltage dividers for each resistive sensors, so to extract a voltage signal fro m sensor resistance variation using a first stage input buffer. Subsequently a second stage provides to properly shift/amp lify the sensor signals with the possibility of finely adjusting both gain and offset level to make the levels of output voltage dividers fit the input range of a PIC microcontroller 12 b its A/D converter. This was our added value, because, in this way, we could profit by the very sensitivity properties of the bend sensors, recordingreasonable meaningful signal variation corresponding to very little joint bending. Then the microcontroller sent the digital signals in serial form to a general purpose PC fo r post elaboration, reconverting them to the corresponding bending angles of the joints. Vo ltage dividers were used because of their simp le structures and potential high dynamic measurement ranges they can furnish. In order to minimize the size of the electronic interface, a single conditioning circuit o f the signal, wh ich could be used by every sensor implementing a polling routine on a mu ltip lexer, was designed. It is important to notice that the voltage signal variation range varies fro m sensor to sensor; this is because the technological process of factory doesn't produce identical devices (as it results clear by observing the standard deviation reported on the seven calibration points of the characteristic curve reported in Fig. 3). Another reason is that the maximu m bending angle of each sensor depends on the joint it is applied to; for examp le the sensors of the Pro ximal InterPhalangeal jo ints (see Fig. 6 for reference), which perform the maximu m bending angle possible (typically 120° in a wholly able subject), react with the largest resistance variation.
For such reasons it was necessary to choose, in designing the instrumented differential amp lifier, a voltage gain (and a level shift) wh ich have to be the best match in order to make the signals of all the sensors fit the input range of a PIC microcontroller 12 bits A/D converter. Considering a single voltage divider (represented in the box left in Fig. 5), a significant issue in the design is how to set R ref .
The single element has the following voltage divider: so, after a 120° bending: where R sens_min corresponds to 0° bending, whereas R sens_max to 120° bending, which is the maximu m allowab le flection of a finger joint and ∆V i to the consistent voltage variation.
In order to maximize the signal sweep for the maximu m allo wed flection degrees, the voltage divider resistance R ref can be yield nullifying the corresponding partial derivative: to obtain: R ref_opt = � R sens_max R sens_min (4) which corresponds to the geometric mean of the extreme sensor resistance values. If the sensor bending sweep is not always the same, an optimized reference resistor for each sensor has to be chosen.The normalized voltage signal variation co ming fro m each sensor becomes: where: q = � R sens_max / R sens_min (8) The equation 7 provides the maximu m voltage divider signal variation with the optimized value for R ref_opt .
Furthermore it can be seen that a bend sensor exhibit ing the largest sweep in resistance for a given bending angle is required, because ∆ V i /V cc → 1 for q → ∞ , even if this sensitivity is smoothed fro m the root. This is the reason why we preferred in this project the Flexpo int bend sensors (q 2 =20) over those fro m Image (q 2 =6), as it is rep resented in Fig. 7, where the voltage divider sweep is plotted against the choice of the reference resistance for different q values. When used inside a voltage div ider, sensor response linearity does not depend upon linearity of sensor resistance. This can be easily seen in Fig.8, where the measured voltage response resulting from the voltage divider is plotted for different values of the reference resistance, whereas the dashed plot represents that one corresponding to the optimized reference resistance in accordance to (4).
As it can be seen, the voltage response is misleading for low bending angles. On the other hand, the dotted plot is referred to an ideal (linear) voltage response, which could be obtained if the sensor resistance response would be the one also plotted in Fig.3.

ADC Resolution
Further investigation was required to set the appropriate resolution for the A/D converter inside the microcontroller. Naming V ni the noise coming fro m the signal conditioning circuits and V nq the quantization noise, where: It can be seen that the resolution N can be chosen from the following inequality: Since the rms noise measured at the output of the signal conditioning circuits is V n =2mV, the above equation yields N<11.
On the other hand, to guarantee a one degree resolution even for s mall bending angles, corresponding to 5 mV output voltage in the characteristic plotted in Fig.8 without signal conditioning circu its, the required nu mber o f b its is given by: Since the embedded A/D converter has 12 bit, the above mentioned conclusions allow the PIC to fo rce 5 LSBs to zero.

Wireless Connecti vi ty
For the sensory glove wireless connectivity, some requirements are mandatory: • it must guarantee human safety; • it has to assure a reasonable data rate; • it has to cover a reasonable distance (range of transmission); • it has to have low power consumption (for long life battery powered devices) and lo w t ransmission power (to reduce effects of radio signals to the body); Some other requirements are preferable but less stringent: • it is convenient to adopt an international standard • it is preferab le to be self-configurab le for mo re data gloves (or other dresses provided with sensors) at time For all these reasons the connection between the micro controller and the co mputer for elaboration was realized using a MRF24J40MA transceiver (see Fig. 9) which support the ZigBee protocol for Wireless Personal Area Network (WPA N) development [9].
ZigBee is a low-cost, low-power, wireless mesh networking standard based on IEEE 802.15.4 definit ions of Physical and Medium Access Control layers (PHY / MAC).
By using the informat ion provided by the datasheet of the Chipcon CC2420/ZigBee, we co mputed the power P d dissipated by a single Tx/ Rx device. Assuming the utilisation of a battery PP3 (9V), with a capacity Cb of 500 mAh (equivalent to 16200 J) and a constant discharge, assuming a continuous monitoring, the expected battery lifetime is 3 days (see [9] fo r details), wh ich can be considered a suitable value for our purposes. Figure 9. The adopted MRF24J40MA transceiver used for HITEG Glove wireless communication

Avatar Representati on
We realized a very accurate avatar representation via video based framewo rk of recorded hand movements.
Once data were correctly acquired and converted into digital form, all values were sent to PC with a specific protocol useful to disambiguate and recognize the exact sensor under investigation (among all the 15 adopted, one at time) and its value. So the data were tid ily stored in a specific MySQL database, one record for each sensor, one field for each recording time. In such a way data can be useful re-called and utilized in simple nu merical format or, mo re effectively, ut ilized to replicate the real hand movement by a virtual avatar on a PC screen. With this aim, it was realized a Graphic User Interface (GUI), programmed in C++ language, by means of Windows Application Program Interfaces (API) and DirectX 9.0c. The software converted digital values into bending degree values for each finger joints and represented all postures on a graphical body model.
A comp lete 3D body model was realized starting fro m Blender, which is an open source mu ltip latform software. In order to animate the model mesh and make it move, translating real hu man actions to virtual act ions in the simu lated environment, we defined an armature which is made of a series of invisible bones connected to each other via parenting or constraints, that allow us to pose and deform the geometry that surrounds it, in th is case the mesh.
The armaturewas used for building skeletal systems to animate the postures of characters and anything else which needs to be animated (see Fig.10, A and B). The armature modifier allowed objects to be deformed by bones: as a bone moves, it deformed or moved the vertices (single points of a mesh) associated with it. The mesh surface is analogous to the skin of the human body. The armature is also called Skeleton. There we re various great advantages fro m the utilizat ion of 3D v irtual model of the hand.
During the pre-processing data phase, the model was utilized as a support tool to qualitatively verify the measurement repeatability.
During the real-time visualizat ion phase, the model allowed the hand visualizat ion fro m different points of view, a continuous monitoring of the coherence of data streams and a rapid re-calibrat ion if necessary.
During the post-processing data phase, thanks to the model, it was possible to replay all the fingers movements in slow / rapid / frame-by-frame mot ion and to isolate even just one finger at a time, remov ing the others from the view, in order to focus the operator's attention only on some important details (see Fig. 11). Figure 11A. reproduction session: software allows user to see an acquisition session off-line, and by rotating 3D model in any direction, it is possible to analyze reproduction from different viewpoints