Blogs
Read more about different aspects and implications of this project
Speaker recognition is a technology that uses computer algorithms to analyze speech patterns and determine the identity of the speaker in a recording. Speaker recognition is an integral part of the ROXANNE platform because the identities of speakers in recordings from criminal investigations are usually not known. However, the speaker recognition scenario in criminal investigations differs from the most other speaker recognition scenarios because in addition to the audio of the recordings, we also have access to information about how the recordings are related. In particular, information about who has talked to whom is important. This information forms a network structure between the recordings. Naturally, it is important for the investigation to accurately uncover the network structure. Interestingly, using prior knowledge about the expected network structure in criminal networks may also improve the accuracy of speaker recognition.
Law enforcement agencies investigate criminal networks to find participants, understand their role in the network and, eventually, collect evidence for prosecution. Opportunities to identify criminals often lie in the traces that their communication leaves behind. A cell phone leaves many different traces, but the availability and usability of these modalities may vary. Therefore, it is worthwhile to invest in multiple modalities, increasing the chance of a useful result.
Ethics oversight in ROXANNE exists across four levels: the European Commission Ethics Checks; the External Ethics Board; the Internal Ethics Board; ethics-focussed work within the project. Overall, the ROXANNE project is in a positive position in terms of meeting the strict ethical requirements from the Commission. So far, the project is also fulfilling the recommendations generated within the project to ensure compliance with ethical, societal, and legal standards. The strict oversight that the ROXANNE project is subject to ensures that none of these standards are violated in the work of the project. The ROXANNE consortium is committed to ensuring that the ROXANNE project is carried out in a responsible way, and that the technologies developed are used in an ethical and lawful manner.
Roxanne project will include link prediction algorithms to enhance criminal network analysis by law enforcement agencies. A recent study shows that link prediction can predict links among criminals, although the effectiveness and accuracy of the prediction may be affected by the amount of unknown information and the specific type of relational data under analysis
In order to implement an approach that incorporates ethical and legal concerns in ROXANNE, there are two research focuses that specifically deal with ethical and legal concerns which are led by Trilateral. The first area of focus ensures that the research carried out during the ROXANNE project abides by the standards of research ethics and that data protection legislation is applied to training algorithmic models used in the platform. A second research focus considers the issues which ROXANNE could raise in terms of ethics, societal values, fundamental rights, and applicable law if it is used in real investigations.
The main problem in the application of ASR systems in forensics is the accuracy and reliability of the results of such system. The accuracy of identification methods depends on a number of factors that cannot always be assessed. Since it is very difficult to assess the impact of all the factors encountered in forensic speaker examinations, the performance of such systems can best be determined using voice databases developed on the basis of audio recordings submitted for examinations. Despite the variety of created voice databases that attempt to record voices under a variety of conditions, forensic investigations still encounter factors, whose impact on an automated speaker recognition system, is often unknown. ROXANNE BLOG briefly presents characteristics of voice databases - BALSAS_LTv1.1 and BALSAS-200LT of the Forensic Centre of Lithuania (LTEC), also describes ASR VOICE training performance using BALSAS_LTv1.1. In the ROXANNE project, LTEC will use the described voice databases (BALSAS_LT200 and BALSAS_LTv1.1) to assess the recognition accuracy of the existing ASR systems VOICE and BATVOX, as well as the ones developed internally.
At present speaker identification by voice in forensics, criminalistics and other applications receive a great deal of attention and huge financial resources. In many cases, this is due to the increasing use of sound recording devices, and in particular, the widespread use of mobile technologies in criminal activities and their recordings, and accordingly, the use of various latest trending technologies in the fight against organised crime and international terrorism. In the ROXANNE project, speaker recognition is one of the main and most important elements of the system. However, direct application of automatic speaker recognition (ASR) systems in forensics raises a number of issues. In general, ASR methods work well only under controlled conditions, sufficiently good signal quality and relatively long duration. The main problem in the application of ASR systems in forensics is the accuracy and reliability of the results of such system. This blog briefly describes problems arising using this systems in forensic speaker identification, and prospects of effective using ASR system with traditional auditory-instrumental methods.
Audio is an important modality for criminal investigations. This is true for different kinds of audio and in particular for spoken content. Be it a part of files, streams or posts which are accessible on the web and social media, a recording collected from mobile devices for investigations and interviews, or a conversation recorded via intercepted telephone calls - speech can be encountered in many environments. Audio data by itself, however, presents unstructured information. Without having access to the spoken content, no further analysis and processing is possible. Enter Automatic Speech Recognition (ASR).
With the increased reliance on new and rapidly evolving technologies to commit and facilitate crime, law enforcement agencies (LEA) must keep pace with the constant technological developments to stay ahead of criminals and maintain a safe environment. The ROXANNE advanced technical platform aims to enhance the quality of investigative work in tracking and identifying criminals by developing an innovative solution that combines speech, text and video data analysis with the analysis of organized crime networks. To be effective and lawfully implemented, such innovative police technologies must be part of an appropriate legal framework based on the principle of law, fundamental human rights and other applicable legislation ( Often innovative fields, such as data analysis for law enforcement purposes and the use of novel analytical methods, are subject to legal uncertainty due to outdated legal texts or insufficient guidance from legislators, be it domestic or regional)
Criminals often employ sophisticated technologies in planning and executing their illegal activities, as well as concealing the revenues stemming from them. Suppressing organized crime remains an important but challenging task for Law Enforcement Agencies (LEAs) for several reasons. In this context, visualisation plays a key role in augmenting the human analysis and providing context to the analyst.
Maximizing the speed of investigations is one of the key factors in the presence of dynamically changing and technologically savvy organized criminals. A number of automated network analysis methods were developed in the past that can be borrowed to help investigators to identify key players, paths and relations within the resulting networks in order to analyse the development of the network, find effective strategies toward its destabilisation or disruption and overall to increase investigation effectiveness.
For law enforcement agencies (LEAs) to keep up in this new environment, they must change their approach to criminal investigations as relying mainly on physical evidence and witness statements is no longer sufficient in many cases. Training in research-based investigative procedures and access to related tools and resources can help law enforcement officers carry out successful investigations. It can help LEAs solve cases more swiftly and even prevent crimes in some cases.
A combination of “synchronous” and “asynchronous” modes can be adopted for WbT instructions. This blog discusses various possibilities around the same.
The ROXANNE project has a principal objective to enhance the LEAs’ efforts to discover criminal networks and identify their members. It has capitalised on certain aspects of AI technologies. Speech and language technologies (SLTs), visual analysis (VA) and network analysis (NA) will become the basis of ROXANNE platform, which will enhance criminal network analysis capabilities by providing a framework for extracting evidence and actionable intelligence.
Airbus has designed its Multi-source and Multi-media analysis solution through the Concept of Generic Architecture which enables the regrouping of multimedia processing software products and presenting them in different possible configurations, based on one common and generic architecture.
This blog discusses various aspects of communication, exploitation and dissemination related to this project.
The project kick-off meeting was held at Idiap Research Institute, in Martigny Switzerland, on 3-4 September 2019.
Probabilistic modelling for speaker recognition in criminal networks
