Serbia, algorithmic discrimination rehearsals

When the Social Card law came into force in Serbia in March 2021, many expressed concerns about its possible impact on people in need. Since then, over 22,000 people have stopped receiving social benefits. The concerns were clearly well founded.

The law provides for the introduction of a register for the collection and processing of personal data of those who receive some form of financial social assistance, in order to assess their overall economic situation and possibly redistribute the resources allocated to this item of the state budget. To determine socio-economic status, the law prescribes the collection of an extensive list of personal data, taken from various sources and centralised in a single register which, through an automated evaluation process, establishes whether and at what level one is eligible to continue receiving benefits.

This system has aroused two concerns. The first is related to the already problematic situation of social assistance in Serbia, which needs a reform that includes more people by giving them higher subsidies, while the law seems aimed at redistributing existing resources and excluding some of the current beneficiaries, rather than including new ones.

The second concerns the violation of the rules on the collection and processing of personal data as well as those that should protect citizens from decisions made against them by automated systems, which risk reproducing and accentuating discrimination that already affects vulnerable subjects, such as for example the Roma community.

A network of associations that goes by the name of ESCR-Net, and which includes Amnesty International, A 11 – Initiative for Economic and Social Rights and the European Roma Rights Center, has recently presented to the Serbian Constitutional Court a legal opinion in which it lists the critical issues of the law.

A non-inclusive system

According to data collected by A 11, in February 2022 (before the Social Card went into operation) there were 211,266 people receiving social assistance, while by August the number had dropped to 189,036 – a drop by 10.5 per cent.

The system has only come into effect in some cities, explained A11 coordinator Danilo Curcic, because many social work centres do not have the technology and expertise to connect to the central register. There is therefore also an issue related to the cost of implementing and maintaining this system, which makes it unsustainable, according to Curcic.

In 11 of the 15 cities from which A11 obtained data, there has been a drop in the number of beneficiaries, in some cases with percentages far above the national figure (Ada -28.7%, Irig -28.8%). In other cases, instead, the number has increased, sometimes significantly (Leskovac +17.1%). “Since there is no transparency on the functioning of the system, it is not clear why some social work centres have increased the number of beneficiaries”, Curcic explains. “In any case, the increase cannot be due to the Social Card, since it does not give centres the opportunity to identify those who are still not receiving assistance and possibly include them in the coverage”.

As Radio Slobodna Evropa explains, social assistance for single individuals currently amounts to 10,385 dinars per month (about 88 Euros), a figure which is also lower than the threshold of 12,695 dinars set by the government itself as a minimum for getting out of poverty in 2020. Furthermore, again according to government data, in the same year 6.9% of the Serbian population fell into conditions of absolute poverty: just over 475,000 people, i.e. two and a half times the number of people who receive social assistance today.

One of the elements that can lead to exclusion from social assistance is reselling materials such as plastic, cardboard, and glass for separate collection, practiced by many Roma people, who are also the poorest segment of the population. “When conferring the material, people must show their tax code in order to be paid – explains Francesca Feruglio, coordinator of ESCR-Net – in this way these earnings, however limited, are recorded and the data ends up in the central register of the Social Card, in several cases resulting in the suspension of the contribution. This is totally unfair considering the overall condition of these people”.

Automatic discrimination systems

One of the criticisms levelled at the law lies in the fact that it produces automated decisions that have a strong impact on people’s lives, without foreseeing the intervention of a human being in the process. In essence, once the Social Card algorithm processes the assessment of the individual case, it sends an alert to the local office managing the person’s file. The official does have a window of time to verify the validity of the decision, but such window is deemed insufficient, especially considering that offices are widely and severely understaffed. In fact, an automated process is put in place.

The Ministry of Labour, in its impact assessment of the law, claims that the citizen is protected because there is always the possibility for an official of the competent office or the Ministry to review the assessment and change its outcome, at the citizen’s request. The reality, however, Curcic explains, is that “you only have 15 days to appeal against the decision, and almost nobody does. In many cases because they do not know they can do it, or because being illiterate they do not have the means to act. So they just let the six months pass to reapply for the social allowance. But even if this is successful, that is six months of benefits lost, and there is no guarantee that the application will be accepted or that there will not be a new revocation”.

Biljana Djordjevic, MP of the Ne davimo Beograd movement, asked for transparency from the Ministry about the functioning of the algorithm and the number of people whose benefits were withdrawn due to the introduction of the Social Card. “They answered that there is no algorithm”, Djordjevic explains. “According to them, the system merely collects data, then it is the social worker who makes the assessment. The testimonies collected show that this is not the case”. And indeed, if there is no algorithm, on what basis would alerts be sent from the ministry to the social work centres? It is quite obvious that there is data processing, but not on what basis it takes place. “We also asked to have access to the code that actually performs the data collection”, Djordjevic continues, “but we were denied due to ‘copyright issues’ and ‘national security'”.

As regards the drop in beneficiaries, the ministry did not provide precise data, but explained away the reduction with the fact that the country has been experiencing progressive development over the years and consequently an improvement in people’s living conditions, therefore decreasing the need for assistance. While it is true that figures on absolute poverty in Serbia have been slowly improving in recent years (but still getting worse than in 2008 and 2009), the need remains to broaden the pool of beneficiaries, while “it is quite clear that the law serves to exclude”, says Djordjevic.

“I am struck by the fact that the impact assessment made by the ministry is so poor”, comments Gianclaudio Malgieri, associate professor of Law at the University of Leiden.

“There is now a certain consensus on the fact that when we talk about human intervention in decisions made through automated systems, this must be significant. Nothing like this can be glimpsed here. Furthermore, when faced with systems that make decisions within a second, the procedure for appealing should be simplified. In this case, however, the standard procedures apply, which essentially lead to ordinary jurisdiction, with all that this entails in terms of time and costs. In the text of the law on the Social Card, then, it is said that this system may also have effects on vulnerable social groups and subjects.

“At this point, however, a list of additional protections should follow: there is none of this. Serbia – continues Malgieri – has also adopted a law for personal data protection modeled after the European GDPR Regulation (GDPR) and has also signed and ratified the Council of Europe Convention 108+. The country is not lagging behind on the subject of personal data. The law on the Social Card acts in violation of the rules that Serbia itself established.”

Unlimited data collection

According to the analysis carried out by A 11, the Social Card Act collects as many as 135 categories of data on the person. To do this, it draws from various sources including the Central Register of Citizens, the Ministry of Internal Affairs, the Tax Administration, and the Pension and Disability Insurance Fund. Some of the requested data are particularly problematic, such as ethnicity (Art. 7(1)(6)). Others, like whether a person owns a weapon, are hardly relevant to the assessment of socio-economic status.

“Citizens are asked about ethnicity”, Curcic comments, “but not whether their house is connected to the electricity grid or the water system”. The fear is that the law, beyond the purposes it states, will end up becoming a system of profiling and mass surveillance of beneficiaries. “Also in this case – continues Malgieri – there is the risk of a violation of the Serbian law on the protection of personal data.

The Social Card implies the collection of a quantity of data that is not justified by the purpose of the law itself. Among other things, Article 4 mentions “the prevention of poverty and the elimination of the consequences of social exclusion” among the purposes of collecting personal data. But the purposes of the processing, also in accordance with the Serbian data protection law, must be specific and precise, they cannot be so generic”.

The legal opinion submitted by ESCR-Net to the Constitutional Court will be accompanied in the coming weeks by an initiative that Biljana Djordjevic is working on: “If at least 25 MPs submit a request to the Constitutional Court, the latter is obliged to rule on the case raised”, she explains. “Unfortunately, however, no time limits are defined, so the Court could decide to postpone the issue continuously. We can only make use of the legal instruments at our disposal. The important thing now is to keep the attention on the issue high”. Djordjevic’s concern is that the central register, once created, will serve as a basis for further forms of discrimination and exclusion”.