UNHRC Italy

UNHCR Italy partnered with Adroit to support its Regular Giving conversion activity, starting with a model built specifically for telemarketing. As the project progressed, UNHCR Italy and Adroit identified an opportunity to go further: a second model to identify a broader pool of supporters likely to convert from one-off donations to committed monthly giving, focused on spontaneous conversions rather than those driven by a specific campaign.

That opened up new audiences to target, but it also raised the bar on the model itself. Rather than simply flagging donors who responded to a given campaign, the model needed to generalise well across a large and varied donor base, while mitigating the biases that past contact history and differing communication channels could otherwise introduce.

Adroit developed a Regular Giving Spontaneous Conversion model using a robust, interpretable binary logistic regression framework. The model was built on a balanced dataset — equal parts converts and non-converts — to combat class imbalance and improve generalisation.

Key predictive features

Giving recency, email engagement (particularly clicks), last gift value (with a clear step-up in conversion above €30), giving frequency and diversity of causes supported, inbound contact history, and length of relationship all fed the model, with gender adding a subtler, combination-dependent signal.

Two scoring models

Adroit built two versions covering the different audiences: Model A for spontaneous RG conversion, and Model B for triggered RG conversion following a one-off donation. Each donor was scored using binary indicators, with ntiled (e.g. decile) outputs created for campaign planning and reporting.

Open, reusable delivery

The models were delivered back as rulesets in Python, with all code and scoring logic provided in a user-friendly script. This let UNHCR independently apply the models to their own data at any future point, without depending on Adroit to rerun it. Adroit’s consultants also visited the team in Rome to present findings, confirm assumptions, and help align the models with UNHCR’s internal processes.

Before going live, the model was validated against newly available 2024 data, with performance evaluated across multiple deciles and centiles and showing a clear, consistent uplift in conversion rates. To fairly test it against UNHCR’s existing selection rules, Adroit used stratified random sampling to build balanced test and control groups across gender, contact history and donor tenure — setting up a statistically sound A/B test between model-selected and business-rules-selected donors. Initial results showed strong discrimination, with conversion rates tapering consistently down the score.

Rollout live test

Adroit and UNHCR then ran a stratified A/B test at scale: a test group of the top 1% (centile) of model scores (n = >2,000), against a control group of donors selected via UNHCR’s refined business rules. Both groups were contacted by the same telemarketing agency , using identical scripts, timings and ask amounts, with random assignment stratified by gender, donor tenure and past campaign contact history.

The rollout test showed the model outperforming UNHCR’s existing business rules:

1  23% uplift in conversion

The test group converted at 1.6% against 1.3% for the control group — a 23% uplift in conversion rate over business rules.

2  39.8% uplift among reached donors

Among successfully reached contacts, the test group converted at 5.01% against 3.58% for the control group.

3  ~80% confidence level

The results show roughly 80% confidence that the model is outperforming the standard business rules — below the typical 95% threshold for statistical significance, which is difficult to reach at these volumes, but strongly suggestive that the model is outperforming BAU selection and very unlikely to be performing worse.

UNHCR Italy can now prioritise donors more likely to convert, enhance business-rule targeting with data-driven selections, and maintain explainability and independence thanks to the open code delivery. Following the success of this model, UNHCR Italy has the tools to iteratively refine its targeting strategy and revisit the model as new data accumulates.