Print Icon
 

Bi-Annual Newsletter: Fourth Edition

About ROXANNE

ROXANNE advanced data analysis solution, developed following a privacy by design approach, aims to enhance and assist police investigations, while reducing the cost and burden on the society owing to organized criminal activities. In the ROXANNE consortium, Law Enforcement Agencies, industry and academia, are joining forces in the fight against organized crime.


The main goal of this project is to develop an interactive platform with AI intelligence that will combine advanced text, speech, and language technologies, along with network analysis all into one platform. The ROXANNE project fully complies with the European legal framework and ensures that both privacy and individual rights are well respected. ROXANNE aims not only to improve and assist police investigations but also to speed up investigative processes and support LEA decision-making.

Police of Czech Republic & ROXANNE Project


From the beginning of the project ROXANNE in August 2019, Police of the Czech Republic sees a lot of potential in the platform for speaker identification, which is being developed by the consortium of several European countries, associated under the project. We see a lot of innovative ideas on this platform.


We also recognize the improvements in speaker identification technology by adding text analysis technologies. The power of bi-directional network analysis visualization in combination technologies is a very useful idea and we are looking forward to improvements and integration of other technologies like face recognition, object detection and others.


Police of the Czech Republic finds it very useful, that the whole project will be Opensource. It will allow the Law Enforcement Agencies from different countries to integrate most of the project components to its unique environments. In words on PCR, „even though we may not be able to use the ROXANNE platform as a whole in the environment of the Czech Republic, we will still be able to pick its interesting components and integrate them to our applications”.


In October 2021, Police of the Czech Republic sent two of its representatives to the second field test of ROXANNE Platform at the National Forensic Institute of Netherlands at Hague. We had an opportunity there to test the first version of the platform, developed under the ROXANNE project. Even though some further works on the Platform are still expected, the current interface was well arranged and intuitive enough. It looks like despite some early technical issues, development of the ROXANNE Platform is on the right track.

     

Progress in the last 6 months


In the last six months, the Roxanne project’s major focus was:

  • to prepare a simulated dataset for crime investigation,
  • to further improve technologies developed in the project, and combine them in an innovative way with an ultimate goal of improving investigation capabilities of LEAs and
  • to explore innovative technologies to support crime investigation research. 

As the criminal investigations contain sensitive and confidential material and are nonpublic by nature, they are subject to restricted access. Understanding the need for such data for research, the Roxanne Simulated Dataset (ROXSD) was prepared by the Roxanne technical partners along with the Law Enforcement Agency (LEAs) considering various scenarios of the criminal investigation following research ethics. The multilingual, multimodal ROXSD dataset aims to depict a realistic criminal investigation and is openly accessible for research purposes. 

Figure 1: ROXSD dataset call recording process and multilingual support 

To leverage different technologies (e.g., text, speech, and network analysis), an integrated platform has been designed by the Roxanne technical partners to exploit the ROXSD dataset and other datasets. The all-in-one research platform combines speech, text, and video processing algorithms with criminal network analysis for combating organized crime. 

Figure 2: All-in-one Roxanne platform

In the Roxanne project, innovative technologies were explored which can be applied to enhance the Roxanne platform performance and speed up the investigation process. One such technology (“mention network”) was implemented in the platform to detect people who are mentioned in phone calls. e.g., Paul and Marc talk about John. It means they both know John, and the network should reflect this.

Figure 3: Mention Network

     

Research work during the past 6 months

 

For the second field test, various technological contributions and components were organized under our new research platform, Autocrime. We introduced Autocrime as a tool to process audio conversations between wire-tapped phone numbers. The platform is used to interface the interaction between various technical components. To demonstrate its capabilities, we used our in-house developed crime simulation dataset, ROXSD. We extended the dataset significantly with raw files and metadata. The new platform has already been installed by the project partners and several partner LEAs.

The technical partners continue their research in combining individual technologies to assist LEAs. In particular, novel strides were made in the interaction of speech, natural language processing and social network analysis. While individual technologies were already developed by responsible partners for the first field test, the combinatorial technologies demanded the collaboration of multiple partners. These were displayed during the second field test using the Autocrime platform. 

 

We also worked closely with a partner LEA on two real cases using anonymised data. One was a phone tap case and another a burglary network. The focus was on examining social and criminal networks using network analysing algorithms and methods. The results from analysing both networks were also presented at the second field test. We have also started looking into detecting criminal presence in internet spaces in collaboration with the LOCARD project. We are investigating the possibilities of detecting cyber-predators using linguistic cues. 

Figure 4: Preview of Autocrime

     

2nd Field Test of ROXANNE Project


The second field test of ROXANNE’s project aimed to present a comprehensive and advanced platform, incorporating fine-tuned components that combine different capabilities and will be capable of processing and analyzing complex datasets. The main objective was to demonstrate how previously known information can be combined with what is automatically extracted by ROXANNE technologies.

The event took place on October 8, 2021, on a hybrid mode, both at NFI premises and virtually. On a beautiful and misty autumn morning, more than 100 participants from various backgrounds (end users, LEAs, Advisory Board members, Ethics Board members, and DG Home experts) were welcomed by The Head of NFI’s Digital and Biometric Traces Division, Erwin van Eijk, followed by an overview of ROXANNE project by Petr Motlicek.


ROXANNE platform’s latest capabilities, its performance and usefulness for solving criminal cases through streamlined data analysis were demonstrated throughout the day by two real case scenarios, including cases built around anonymised real data provided by an internal law enforcement agency, as well as the presentation of the training platform. Later in the afternoon, a hands-on session allowed participants to delve deeper by actively experimenting with the platform functionalities. Their valuable inputs and feedback crowned a successful and delightful day, and will allow a well-rounded assessment for further analysis and development.

2nd Field Test - Group photo (October 2021)

     

Diverse aspects of ROXANNE

 

Implications of AI Regulation (proposal) for ROXANNE

The areas for application of Artificial Intelligence (AI) are constantly expanding, from security sector drones, to life-saving medical diagnosis and drug discovery in the pharmaceutical industry, and an ever-growing range of smart home appliances or social media filters. Unsurprisingly, countries attempt to keep pace with these technological advances by providing for adequate legal frameworks, through sector-specific or horizontal legislative acts. As AI lies at the core of the ROXANNE project, the consortium is closely monitoring the legal landscape for initiatives that may impact on the project’s sustainability. Precisely, the ROXANNE developed technological solutions need to be legally viable in order to attain the exploitation objectives.


As part of these efforts, the ROXANNE team is closely following the negotiations related to the European Commission proposal for a Regulation on AI (AI proposal). This EU initiative aims to address potential risks associated with AI use and development, such as algorithmic opacity and complexity, tool autonomy, data bias, by mitigating safety and fundamental rights concerns, to enhance public trust. The AI proposal has a broad technological scope as it relies on a neutral AI definition, covering all AI systems, including techniques that are not yet known. The proposal foresees rules applicable during the development of AI, which would be applicable to the research phase, such as in ROXANNE at the moment. Also, it contains rules related to the use of AI systems, differentiating between obligations for operators and providers of such technologies. The AI proposal follows a risk-based approach to the AI systems, categorized as:

  • Prohibited, when posing unacceptable risk;
  • High risk, which are permitted subject to compliance with specific obligations/requirements as well as following an ex-ante assessment in some cases, i.e. real-time remote biometric identification;
  • Minimal risk, which are permitted with no restrictions.

The ROXANNE platform would fall under the high risk category given its intended use by law enforcement authorities, and since it involves biometric identification and categorization of natural persons. The AI proposal foresees for high risk AI systems to establish and implement risk management processes in light of the intended purpose, and to comply with the following requirements:

  • use high-quality training, validation and testing data;
  • establish documentation and design logging features;
  • ensure transparency and inform users how to use the system;
  • ensure human oversight;
  • ensure robustness, accuracy and cybersecurity;

These requirements form an integral part of ROXANNE research and have been guiding the development work of technologies. Training and testing data was subject of deep reflection in the project. Technological partners rely on proprietary datasets to train the computer models of their technological components. In addition, law enforcement authorities participating in the project shared some real data, from closed cases and in anonymized form, to test the latest ROXANNE capabilities. This data was also used to demonstrate the latest platform capabilities during the second field test held in October 2021. To overcome the multiple legal, ethical and security hurdles related to the use of real data, the consortium built a synthetic dataset, inspired from a real drug case, for further platform testing and demonstration. As the ROXANNE platform matures, traceability and accountability are at the core of its privacy by design approach, which foresees user-identification and accountability mechanisms. These features will enable logging of all data processing for auditability, traceability, and potentially evidence admissibility purposes. The ROXANNE team puts much effort in being as transparent as possible about the work that it carries out, as testified by the open nature of project events and its regular communication to the public. To this end, the ROXANNE field tests were accessible to all interested participants, while the regular blog posts explain the technological processes and methods behind the ROXANNE platform. Moreover, within the project, an entire work package is dedicated to designing and providing comprehensive training to prospective end-users for an informed use of the platform and individual tools. User training is essential to secure controlled platform handling, including accurate interpretation of results. Finally, ROXANNE is designed as an assistive tool that aims to facilitate the work of authorized end-users, not to replace them, by streamlining the analysis of large volumes of investigative data. The aim of the technology is to help investigators by guiding the focus of limited resources on most relevant case data. A decision-making mechanism integrated in the platform highlights this human-centric operation by requiring critical engagement from operators at key stages of technology use (i.e., pre/post-analysis).


The AI proposal will become directly applicable 2 years after its adoption, once the European Parliament and Council agree on a compromise text. Although it is certain that this will not occur within the ROXANNE lifespan, the project team is committed to building a future-proof ROXANNE platform.

  

ROXANNE in collaboration with FORMOBILE and LOCARD

Partners in ROXANNE are collaborating with colleagues from FORMOBILE and LOCARD on a joint paper on the impact of the proposed eEvidence Regulation. This proposed Regulation seeks to regulate how electronic evidence is requested, produced, and preserved in criminal investigations that cross international borders. Following a risk series of discussions between experts in ethics, law, and societal issues from the ROXANNE, FORMOBILE, and LOCARD projects, colleagues have agreed to collaborate on a joint paper dealing with the impact that the proposed eEvidence regulation could have for rights to privacy and data protection in terms of the collection, analysis, and sharing of electronic evidence. The aim is to develop practical policy recommendations that can contribute to the fight against crime and terrorism whilst maintaining appropriate levels of privacy. From ROXANNE, partners in the project’s Ethics, Legal, and Societal issues team will be bringing the knowledge and results they have gathered during their work on the ROXANNE project to contribute to this policy paper.

 

Forthcoming FORMOBILE event

ROXANNE Ethics and Legal partners will be sharing their results and experiences at the forthcoming FORMOBILE event. If you would like to attend, you can register here.


     

Blogs, articles, publications

Blogs:

 

View more ROXANNE news, blogs and media on our website. 


Articles:

 

For more related news updates follow us on Twitter and LinkedIn.

 

Publications:


     

This project has received funding from the European Union’s Horizon 2020 Work Programme for research and innovation 2018-2020, under grant agreement n°833635. "Disclaimer: the document reflects only the author's views and the European Commission is not responsible for any use that may be made of the information contained therein."