The challenge of documenting civilian harm during modern warfare increasingly extends beyond verifying evidence to finding it amid an overwhelming stream of digital content. In response to that challenge, Bellingcat has developed and tested a machine learning system designed to help researchers identify potential incidents of civilian harm more quickly, reducing the time spent searching vast quantities of social media posts while leaving final verification in human hands.
The methodology, detailed by Bellingcat after years of testing during Russia’s full-scale invasion of Ukraine, demonstrates how artificial intelligence can be integrated into open-source investigations without replacing human judgment. Instead, the system is designed to prioritize posts most likely to contain evidence of civilian harm, allowing investigators to devote more time to verification rather than discovery.
Between February 2022 and September 2025, Bellingcat staff and volunteers collected, geolocated, and documented more than 2,500 incidents of civilian harm linked to the war in Ukraine. During that effort, researchers also built and refined a machine learning model capable of ranking Telegram posts according to their likelihood of containing information about civilian casualties or other forms of harm affecting civilians.
According to Bellingcat, the technology significantly reduced the search and selection process required to identify relevant posts. The organization said the experience may provide a practical framework for journalists, researchers, and conflict-monitoring organisations working in other conflict zones, including Sudan and parts of the Middle East, where enormous volumes of user-generated content present similar investigative challenges.
For the project, Bellingcat adopted a broad definition of civilian harm that extended beyond deaths and injuries. Researchers also included mental trauma, displacement, destruction of infrastructure, loss of livelihoods, and other indirect consequences of armed conflict. That framework guided both the collection of training data and the development of the artificial intelligence system.
The project began with an extensive manually verified dataset. Researchers assembled nearly 5,850 Telegram posts that had already been confirmed as containing evidence of civilian harm. To provide the model with examples of unrelated content, they also collected more than 48,500 additional Telegram posts that did not contain civilian harm by examining posts surrounding each verified incident. After removing duplicate and deleted material, the resulting dataset contained more than 54,000 examples that formed the foundation for training the system.
Each Telegram post was enriched with additional metadata obtained through the platform’s application programming interface, including publication times, reactions, engagement levels, and textual content. Where original posts had been deleted, Bellingcat supplemented the missing information using preserved copies stored in its own Auto Archiver database.
Transforming that information into a form that computers could analyse required extensive feature engineering, a process that converts raw information into numerical values suitable for machine learning algorithms. Researchers incorporated both technical metadata and practical knowledge gained from years of manually reviewing conflict-related content.
Among the features included were publication details such as whether a post had been forwarded, edited, or was replying to another message. Engagement metrics including view counts, shares, total reactions, and even the number of crying-face emoji reactions were also incorporated after researchers observed recurring patterns during manual investigations.
The textual component of the model relied heavily on multilingual analysis. Researchers compiled lists of Ukrainian and Russian keywords associated with civilian harm, including references to Shahed drones and guided aerial bombs. The model also measured semantic similarity between Telegram posts and broader English-language concepts such as casualties, hospitals, schools, and injuries using sentence transformer technology capable of identifying similarities in meaning even when different words were used.
To capture the overall context of each message, Bellingcat employed a multilingual transformer model from the BERT family of language models, generating numerical representations that allowed the system to compare meaning across multiple languages. Notably, the initial version of the model deliberately excluded analysis of photographs and videos despite acknowledging that visual content could further improve performance in future iterations.
Researchers evaluated four machine learning algorithms using the completed dataset: Logistic Regression, Random Forest, XGBoost, and LightGBM. The data was divided into separate training, validation, and testing groups to ensure that the models were evaluated using previously unseen material rather than data they had already encountered during training.
Performance was measured using established machine learning metrics that balanced the ability to identify genuine incidents of civilian harm while minimizing false positives. According to Bellingcat, XGBoost consistently achieved the strongest results across the evaluation criteria and was selected as the final model.
The organisation also examined which factors most influenced the system’s decisions. Semantic similarity between post content and civilian harm terminology ranked among the strongest indicators, alongside multilingual language embeddings produced by the BERT model. Emotional engagement, particularly posts attracting large numbers of crying-face emoji reactions, also emerged as a significant predictive feature, reflecting patterns previously identified by researchers during manual screening.
To evaluate the rapidly evolving capabilities of large language models, Bellingcat later conducted separate tests using Google’s Gemma and Gemini models. Researchers processed a sample of Telegram posts through multiple language models, both with and without the engineered features developed during the project, and compared their performance against the custom-built machine learning systems.
Although the large language models outperformed the baseline Logistic Regression model, none surpassed the XGBoost system in the project’s primary performance measure. Researchers found that Gemini 2.5 Flash achieved stronger results than Gemini 3.5 Flash in several respects, but concluded that the custom-trained model remained better suited to identifying relatively rare instances of civilian harm while limiting unnecessary false alarms.
Alongside the technical development, Bellingcat devoted significant attention to ethical considerations surrounding automated decision-making in conflict research. The organisation identified risks including automation bias, in which researchers might place excessive trust in machine-generated recommendations, and algorithmic bias, where models reproduce patterns and limitations present within their training data.
Researchers also highlighted the psychological benefits of reducing prolonged exposure to distressing imagery and reports of violence. By allowing artificial intelligence to perform much of the initial sorting process, investigators were able to spend less time reviewing large volumes of traumatic material while concentrating their efforts on detailed verification.
To address ethical concerns, Bellingcat implemented several safeguards. These included randomly selecting additional posts that the model did not flag, monitoring which features influenced predictions, and comparing historical patterns to identify potential bias. The organisation also chose not to rely on large language models within its operational workflow because of concerns relating to privacy, transparency, and the handling of sensitive conflict-related information.
Following the selection of the XGBoost model, Bellingcat integrated the technology into a dedicated user interface that ranked Telegram posts from most to least likely to contain evidence of civilian harm. Once researchers confirmed a relevant post, it was automatically preserved using the organisation’s Auto Archiver system before being transferred into ATLOS, Bellingcat’s collaborative verification platform, where investigators could continue the verification process. Feedback from staff and volunteers indicated that the system reduced the time required to monitor numerous Telegram channels while improving the diversity and quality of incidents entering the verification pipeline.
Bellingcat concludes that while artificial intelligence cannot replace human investigation, carefully designed machine learning systems can help researchers manage the growing volume of digital information emerging from conflict zones. The organisation says the methodology can be adapted beyond Ukraine to support investigations into civilian harm and human rights violations in other conflicts, offering newsrooms and monitoring organisations a practical framework that combines automation with human oversight while remaining attentive to the ethical responsibilities of documenting war.
