A novel semi-automated approach for forensic facial reconstruction in Sri Lanka

The process involved capturing a 3D model of the skull and digitally sculpting facial muscles on the model with the aid of forensic facial markers. Different facial components were added to the completed face model in order to improve the identification process. Separate analyses were also conducted for both facial tissue thickness and facial component variations in Sri Lankans to achieve an improved result. Females of the age category 20-30 of average weight were used in this study.

Introduction of a more efficient, semi-automated 3D Computer graphics based technique of facial reconstruction was the aim of this study.
The process involved capturing a 3D model of the skull and digitally sculpting facial muscles on the model with the aid of forensic facial markers. Different facial components were added to the completed face model in order to improve the identification process. Separate analyses were also conducted for both facial tissue thickness and facial component variations in Sri Lankans to achieve an improved result. Females of the age category 20-30 of average weight were used in this study.
The facial tissue thickness analysis conducted by the research team confirmed that tissue thickness data of other countries cannot be adopted for facial reconstruction in the local context and that Sri Lankans have a different facial soft tissue thickness mainly in the following areas; Gonion, Sub M2, Supra M2 and the area beneath the chin. The facial feature analysis discovered the mean values of the nasal and eye indexes which were

Introduction
The right to be buried in accordance with one's religion and traditions in a grave that bears one's own name is a basic human right. That right is closely similar to the rights of the relatives of a missing person to know the status of their loved ones. Thus with regard to unrecognizable corpses its identification is of high importance. Natural or man-made disasters such as terrorist bombings, mob uprisings, tsunami, etc result in innumerable unidentified dead bodies. Murder victims plunged deep underground and surfaced years later would also be unidentifiable dead bodies. In such situations, when all other identification methods fail, investigations are directed towards facial reconstruction stage. Once a face is reconstructed it is circulated via digital and print media with the hope of someone being able to identify it. Upon such identification, the stated identification is verified using mechanisms such as DNAor dental records analyses.
This process is also known as facial approximation, because the end result of this process is a face that would only hold resemblance to the deceased, and will not be the exact face of the person.
In the global context, although forensic facial reconstruction is used in a very productive manner, it is at its infancy in Sri Lanka. Hence introducing a more efficient multimedia based technique to the local forensic officials in order to improve the efficiency and the accuracy of the reconstruction process is the aim of this study. This study introduces a novel semi-automated method for the reconstruction process.
To develop a solution enabling computer based forensic facial reconstruction in Sri Lanka, several skulls were acquired from Sri Jayewardenepura and Karapitiya Medical Faculties along with the photographs of the deceased for evaluation purposes.
Once the above materials were acquired, the process was carried out with a pre-requisite stage and four main steps as follows. Pre-requisites: Step 01: Capturing the 3D model of the skull Step 02: Placement of Landmarks Step 03: Digitally Sculpting the Face Step 04: Adding Facial Components Step 05: Evaluation of identification A forensic medical expert extracted the following details from the skeletal remains. This study was carried out as a proof of concept for the female, 20-30 year old medium weight category. Hence forensic cases where the deceased was of a medium weight, 20-30 years old, females were chosen and so were the tissue thickness samples.
Thirty one anatomic and physiognomic relevant points (tissue thickness landmarks) were identified on the skull ( Figure 1). The number of landmarks was determined by the research team to suit the nature of the muscle sculpting method. Previous studies in this regard have been performed with differing numbers of landmarks (1)(2)(3)(4).
Therefore, sample of 30 was created from females of the age category 20-30. By using MRI and CT images of the sample group, tissue thickness depths for each of those anatomical landmarks ( Figure 1) were acquired. Head/sinus scans (without the patient identification details) obtained from medical institutions were used for this purpose. After the tissue thickness analysis it was found that there is a considerable difference between the facial soft tissue thicknesses of Sri Lankans and other races. Hence applying tissue thickness values of other countries to a Sri Lankan skull model would result in inaccurate approximations. During the comparisons it was also observed that Sri Lankans have differing facial soft tissue thickness depths at the areas of beneath Chin, Gonion, Sub M2 (Below the second mandibular molar) and Supra M2 (Above the second maxillary molar) ( Figure 2).

Pre-requisite: B. Ascertainment of facial tissue thicknesses
For facial component analysis, the mean values of nasal and eye indices were calculated. For this analysis, noses and eyes of 400 volunteers of the age category 20-30 was photographed. Facial components were divided under three categories; narrow, medium and broad.
Using the photographs of the group, the Nasal Index was calculated using the following "Broca Agreement" formula.
Nasal Index = x 100 Nose height was followed to measure variations in the eyes.

Eye width
Step 01: Capturing the 3D model of the skull To capture the skull, a Vivid 910 3D scanner of Konica Minolta™ (Figure 3) was used. This scanner has already been used to generate models of human components in the Medical Field (5).
Once the 3D models were acquired using the 3D Scanner, it was observed that most of the test skulls had fractures and were damaged. This was not an unexpected dilemma since most of the human remains unearthed after years would be in a dilapidated stage.
However this status of the skulls rendered it hard to conduct an accurate reconstruction. Hence it was required to make certain rectifications to the 3D models of the skulls. A 3D editing software application was utilized in this stage to successfully rectify the acquired 3D models (  Step 02: Placement of Landmarks Step 03: Digitally Sculpting the Face For this, tissue thickness data obtained from a group of people who were similar to the deceased in terms of age and sex were used ( Figure 5) by using 3DX Max software application.
By using those depth values, the height of the tissue thickness landmarks placed on the model were then adjusted. In contrast to the other mechanisms adopted in facial reconstruction in other studies, this study used a novel facial muscle sculpting based approach for reconstruction.
Several techniques were tried out and it was decided to use the ZBrush™ software for the muscle reconstruction since it is digital sculpting software. Each facial muscle was carefully constructed and the thicknesses of each of these muscles were determined by the landmarks (Figure 6). The sculpting of the basic muscles was followed by the construction of more complex facial components such as eyes and nose along with applying skin on top of all the muscles (Figure 7).
A realistic appearance for the reconstructed face was obtained by adding modeled facial components such as eyes and noses appropriately.
As facial feature data from other regions would not be appropriate, a separate facial feature analysis was conducted on Sri Lankans. Using the mean reference values of Nasal and Eye Indices, facial components (different sizes and shapes) (Refer Table II, III) were added to the 3D model to improve the possibility of identification ( Figure 8).
Step 04: Adding Facial Components Step 05: Evaluation of final appearance The method adopted to reconstruct the face resulted in a 3D face model that had a fair resemblance to the deceased person's actual photograph. This was evaluated using both non-technical and technical evaluation techniques to ensure the accuracy.
"Face pool comparison", is the non technical evaluation used for digital photos. The reconstructed face and four other photographs with actual photograph of the deceased were e-mailed among university students (Figure 9).
( ) The non-technical evaluation technique used for CT photos was the "Resemblance Rating". CT photo with the reconstructed face images were e-mailed among university students ( Figure 10).  In our study, a 3D editing software was used to find the orientation and to superimpose. In the superimposition technique which is a widely accepted way of evaluating outputs of this sort (1), the image of the model was placed on top of the actual photo to analyze whether the contours match. This allowed to verify the soft tissue alignment and to see if there were any obvious errors ( Figure 12).
The final superimposition figure was evaluated by Free-morphing Superimposition evaluation technique.
In the "Photogrammetry analysis", the 2 technical evaluation method, the distance between defined facial components were measured and compared between the model and the actual photograph ( Figure 13). The Photogrammetry analysis was done by using facial recognition software.  The results of the process and evaluation techniques of the study are as follows. Table 1 shows the tissue thickness values that were gathered using MRI and CT scans of medium weight females of 20-30 years (Table 1). Nasal index of 20-30 years females are shown below.

Result 1: facial tissue thicknesses
After which the indices were categorized as narrow, medium and broad and the mean N.I values for each of the categories were calculated ( Table 2).
Eye index of 20-30 years females were as follows.
After which the indices were categorized as narrow, medium and broad and the mean E.I values for each of the categories were calculated (Table 3).

Nose Analysis
Eye Analysis

Result 3: Evaluations
In "Face Pool Comparison", the Non-Technical Evaluation method used for Digital photos, 117 responded. More than 50% of the respondents managed to successfully select the correct image ( Figure 14: 2 person). Other evaluation surveys showed similar results.
In "Resemblance Rating", the Non-Technical Evaluation technique used for CT photos, 34 responded. Almost all responded that the reconstructed image was either similar or identifiable to the CT photos. Case 03 evaluation survey showed similar results (Figure 15). nd Photogrammetry Analysis, a Technical Evaluation technique, by using facial recognition software showed significant similarity in all 4 cases. eg. Figure 16 shows the photogrammetry analysis for case 01. Results of the "Superimposition", the 2 Technical Evaluation method also showed significant similarity in all 4 cases, so as in case 01 ( Figure 12). Free-morphing software application was used to evaluate the superimposition technique and it showed perfect match in all 4 cases. eg. In case 01 ( Figure 17).
Both these technical evaluation methods resulted in depicting that the difference between the reconstructed face and the deceased persons photograph to be negligible.

Discussion
There is a standard procedure that is followed by forensic officials when they receive an unrecognizable corpse. First the bodies are allowed for recognition by facial features by the relatives or friends and usually, fresh bodies are identified by f a c i a l r e c o g n i t i o n . Wi t h p u t r e f a c t i o n , facial recognition may not be possible. In failing that, the forensic officials would ascertain general and tentative identification of the dead bodies. In 'general identification' forensic officials will estimate values for age, sex, race and stature. For 'tentative identification', documents, ornaments, clothes, scars, tattoos, stigmata, habits, occupational stigmata, deformities and radiological findings are documented. This information will then be published in media and also compared with missing persons' database. If a relative or a friend comes forward, they are asked to submit antemortem data such as X-rays, medical records, dental records etc of their missing person. Those facts can be used to confirm the identification by using primary identifiers; dental, radiology, DNAand finger prints techniques.
If no relatives or friends come forward those unidentified remains would remain unidentified. e.g.
If an isolated skull is found, relatives or friends may not be able to identify the same. In exhumations too, the skeletons may not be identified. In such instances, unidentified remains are buried after 14 days, following all faith ceremony.
Therefore, the ultimate resolution for unidentified remains is the facial reconstruction. The reconstructed face is published in media and is compared with missing persons' data base. Relatives or friends may be able to identify the reconstructed face and we get the opportunity to obtain antemortem data of the missing person and then the identification can be confirmed by using primary identifiers. Finally, a funeral can be arranged according to the deceased's faith and dispose upholding the dignity.
Facial reconstruction was not implemented in Sri Lanka due to non availability of facial tissue thickness and facial component (eyes, nose) data, and high cost of purchasable automated solutions. Manual method of facial reconstruction is time consuming, less productive and also expertise is lacking.
The goal of our research was to find a feasible, computer based, 3D, semi-automated facial reconstruction solution to Sri Lanka.
This study encompassed reconstructions where the sources were taken from CT scans of living people to increase the number of reconstructions done. In these cases the skull and the actual photograph of the face were acquired using surface rendering DICOM images of CT scans (Figure 18).
Since 3D scanners are not commonly available, it was found that using CT images of the skull can also be used as a viable option to generate 3D models. The advantage of this method is that since the CT scanner possesses the capability to capture details precisely, the resulting model would be of high accuracy and detailed. The DICOM output of the CT scanner was imported in to an image processing application dedicated to DICOM images [Osirix-viewer] and the 3D model was obtained.
The recommended semi-automated methodology of reconstructing based on facial muscles; especially in the field of computer based 3D reconstruction is a novel approach. The literature stated that the manual reconstruction method resulted in a natural looking output but lacked in efficiency and expertise. Further, we found that a fully automated technique would be more efficient compared to the manual technique. But the output due to the process being fully automated resulted in an unrealistic final model. Thus it was decided to use a semi-automated technique by combining the advantages of both manual and fully automated techniques. This methodology automates the steps of the reconstruction process where inefficiencies occurred in the manual process. eg. Creating a plasticine mould of the skull, sculpting muscles manually using modeling clay. Since the manual reconstruction is done mostly by modeling the facial muscles on a Figure 18: ACT scan case model of the skull, the authors were of the opinion that having a direct map between this technique and the computerized 3D reconstruction process would result in a more natural looking output. Thus muscles were sculpted manually in a digital environment which unlike in the fully automated process resulted in a more natural and realistic output.
Since both evaluations gained positive results as well, reconstructing the face using the muscle-based anatomy approach was proven to be reliable and accurate to a sufficient extent. The following problems were addressed in this research (Table 4).
After the cost analysis was done it was quite apparent that the solution presented was feasible especially in monetary terms. The cost of the software used, amounts to $700 and that compared with the systems used in other countries (UK $ 75,000) is a very low cost solution. The time consumed was also considerably less when compared with the manual technique. Furthermore, the expertise in 3D modeling would not be an issue either due to the reason that to a 3D modeling artist the muscle sculpting learning curve would be substantially short. Comprehensive manuals and documentation have been created by the research team ( Figure 19). The quality of the solution can be assessed in following way. It is a user friendly technique. Instead of sculpting, easy digital method is used. Expertise are now available in Sri Lanka and comprehensive manuals have been created for these 3D modelling softwares. It is scalable. If tissue thickness of other age groups also available it is easy to fit into any age category.
We used accepted standards, optimized the accepted tissue thickness landmarks to 31 Furthermore, we accepted face evaluation methods such as and Superimposition were used. We also used accepted Facial Muscle Anatomy Model and Standard Facial Component Comparison techniques such as Facial approximations for the female 20-30 year age group and medium weight cases were produced by this research. The outputs were successful and thoroughly evaluated using accepted evaluation techniques.

Abate. A .
Face-pool Comparison, Resemblance Rating, Photogrammetry Analysis, Broca Agreement for Nasal Index and Halder Method for Eye Index.

Lack of Tissue thickness data of Sri Lankans
Tissue Thickness Analysis

Lack of facial features differences data on Sri Lankans
Facial Feature Analysis

Expert technical knowledge on facial reconstruction process
3D sculpting based simplified process

High Cost of exiting solutions
700 $ software cost

Time Constraint 4 -5 Hours
For the process, optimized tissue thickness landmarks and an accepted facial muscle anatomy based model were used. Although the research was performed on certain age and weight groups this process would be scalable to any other age/weight groups as well (given that the tissue thickness data for all those groups are collected).
Since all the attempted evaluation methods resulted in satisfactory outcomes it can be concluded that the method adopted in this study to conduct the reconstruction is suitable and would ensure an accurate output. Hence the adopted procedure was identified to be recommendable to be adopted for future facial reconstructions in Sri Lanka.
It can also be concluded that implementing a "National unit for facial reconstruction", which would adopt this procedure would be a very timely act for the island to uphold the rights of the deceased and their relatives. Finally, this would be a solution in identification for Judicial Medical Officers, police investigators, Archaeologists and also a relief for the relatives and friends of the missing person.